Compare commits

..
Author SHA1 Message Date
beardthelion 60519f3d73 Merge remote-tracking branch 'origin/main' into pr-384
# Conflicts:
#	Cargo.lock
#	crates/gitlawb-node/src/api/repos.rs
#	crates/gitlawb-node/src/cert.rs
#	crates/gitlawb-node/src/db/mod.rs
#	crates/gitlawb-node/src/git/smart_http.rs
#	crates/gitlawb-node/src/git/store.rs
#	crates/gitlawb-node/src/main.rs
#	crates/gitlawb-node/src/webhooks.rs
2026-11-09 07:52:28 -06:00
c805e35f40 fix(node): bound GraphQL query cost (#552)
* fix(node): bound GraphQL query cost

* fix(node)!: bound repo names so emitted cursors always parse

The reposPage cursor embeds base64(JSON(owner_did, name)) while
parse_repo_cursor rejects cursors over 4096 bytes, but repo names were
unbounded on every write path. A single long-named visible repo made the
server emit an endCursor it then rejected as "invalid repository cursor",
dead-ending pagination for every caller past that row.

Bound names to MAX_REPO_NAME_LEN (100 bytes) at the create_repo and
fork_repo handlers via a shared validate_repo_name helper (bad charset is
still a 400), and reject overlong peer-supplied names in
Db::upsert_mirror_repo (the sync caller already drops the error and skips
the mirror).

Also in this round:
- docs/graphql-pagination.md states the 200 page-size maximum and corrects
  the over-budget remedy: a smaller limit does not reduce the reposPage or
  repos price, which is charged at the full MAX_VISIBLE_REPO_PAGE_SIZE
  scan regardless of limit.
- Tests pin refUpdates(limit: -1) clamping to zero rows, the
  parse_repo_cursor wrong-shape-JSON arm with its exact "invalid
  repository cursor" message, the longest-admissible-name cursor
  round-trip, and the mirror write-path rejection.

BREAKING CHANGE: the GraphQL `repos` field now returns an error instead of
the full list when more than MAX_VISIBLE_REPO_PAGE_SIZE (200) repositories
are visible; larger lists must use `reposPage` with `limit`/`after`.
Repository names are additionally limited to 100 bytes at creation, fork,
and mirror-registration time.

* fix(node): address beardthelion P3 findings on name bound and input messages

- Assert each rejection's own message in the reposPage invalid-inputs
  loop: the loop no longer accepts either message for every query, so a
  misattributed message (limit arm returning the cursor message) goes red.
- Pin the create/fork name-bound wiring through the handler: new test
  drives create_repo with an overlong name via test_state_lazy and asserts
  the 400; deleting the validate_repo_name call site would proceed to the
  lazy DB pool and fail with a connection error instead.
- Document the API-side vs disk-side validate_repo_name divergence on the
  new function, and tie the store's literal 100 to MAX_REPO_NAME_LEN so
  the two validators cannot drift apart on length.

---------

Co-authored-by: Marlowe <[email protected]>
Co-authored-by: cairn-intern <[email protected]>
2026-11-08 23:28:09 +00:00
a4026e84ff fix(node): honor pre-assigned task assignee on claim (#550)
* fix(node): honor pre-assigned task assignee on claim

Stop strangers from overwriting a reserved assignee_did (and receiving
the UCAN). Bind finish_task to the stored assignee; return 403 on
reserved steals; keep exact stored DID forms; normalize blank assignees;
pin production claim/finish SQL race guards with FOR UPDATE tests; trim
broadcast by_did; and tighten blank/SQL re-check regressions.

* test(git-remote-gitlawb): wait for shim before real_git_fetch on Windows

Windows CI can race: git fetch connects before the shim accept loop is
polling, yielding connection aborted (os error 10053) with 0 POSTs.
Probe the listener after spawn so multi-round and withheld-path tests
start only once the shim is ready.

* test(git-remote-gitlawb): retry transient Windows fetch connection errors

Windows CI can still abort localhost connections (os error 10053) after the
shim is ready. Retry fetch_with_helper up to three times on transient
connection failures before asserting.

* test(git-remote-gitlawb): revert Windows fetch workarounds

The retry-on-10053 and wait-for-shim probes papered over the accepted-socket
inheriting the listener's non-blocking state. #312 fixed that at the source
(normalize_accepted_stream) and is merged, so take the workarounds back out
and let the real fix carry the Windows lane.

* test(git-remote-gitlawb): pin the fetch retry's absence with an exact GET count

The POST assertions are floors, so a reintroduced fetch-level retry could
satisfy them with a second full fetch while masking the withheld shape.
A retry cannot avoid re-advertising, so count GET /info/refs in the shim
and assert exactly one per fetch in both the multi-round and withheld
tests. The merge from main also brings #312's normalize_accepted_stream
into this head, closing the round's rebase ask.

* fix(node): mirror assignee normalization across GraphQL task mutations

GraphQL createTask returned the pre-insert input, so a whitespace-only
assignee came back as a reservation the store never kept; complete and
fail compared the caller against the raw stored assignee where REST and
Db::finish_task trim; and task-event broadcasts reported the caller
string rather than the stored assignee form REST publishes. All three
paths now mirror the REST and DB contract: the create response is the
stored row, finish gates compare the trimmed stored assignee, and
by_did on claim/complete/fail broadcasts derives from the returned row.

---------

Co-authored-by: Ayush7614 <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-08 21:38:53 +00:00
38bab4191b fix(node): version ref_certificates for forward compat (#26 split 3/4) (#548)
* fix(node): version ref_certificates for forward compat (#26 split 3/4)

Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split
into four narrow PRs. This is Split PR 3 (certificate / CLI
compatibility). No findings are assigned to it; it is required
because the protocol/client work is independently reviewable and
should not remain coupled to post-receive recovery.

The wire-format problem: the ref-cert payload is a fixed 7-field
JSON blob the node signs. Future changes (additional pusher
attestations, content-binding fields, secondary signatures) would
either break every existing cert or be impossible to ship without
a flag day. Versioning the cert makes the format forward-compat
and explicit.

NEW MIGRATION v28
  ref_certificates gains `version INTEGER NOT NULL DEFAULT 1`.
  DEFAULT 1 means an existing row reads as v1 with no backfill
  needed; NOT NULL means a missing value is a hard error rather
  than a silent v0.

NEW FIELD ON RefCertificate
  `version: u32`. v1 is the pre-versioning 7-field payload
  (no `version` key in the signed JSON, so a v1 cert is
  byte-for-byte identical to today's cert). v2+ certs will
  include a `version` key and any new fields, both
  backwards-compatible: an old client reading a v2 cert sees the
  unknown `version` field and either ignores it (graceful) or
  refuses to verify (explicit).

API CHANGES
  - list_certs and get_cert now include `version` in the JSON
    response.
  - issue_ref_certificate sets `version: 1` on every new cert.

GL CHANGES
  - gl cert show reads the `version` field; missing field
    defaults to 1 (forward-compat with old servers). v1 certs
    go through the unchanged verify path. v2+ certs are
    explicitly rejected with an "upgrade the client" message
    rather than silently guessing the payload shape — a future
    client that supports v2 cannot accidentally treat a v1 cert
    as v2.

GOLDEN-FORMAT FIXTURE
  v1_payload_matches_frozen_canonical_form pins the v1 signing
  payload byte-for-byte. Reverting this to a v2 shape (adding a
  `version` key) is what the versioned format forbids: an old
  client reading a v1 cert must not see a `version` key in the
  signed JSON. The same byte-form is what gl's verify_signature
  reconstructs in crates/gl/src/cert.rs, so any drift between
  node and CLI breaks the test instead of silently rendering
  every real cert INVALID.

TESTS
  - 3 v1_payload_tests in crates/gitlawb-node/src/cert.rs:
    frozen-canonical-form, round-trip structural, default
    version is 1.
  - 2 new tests in crates/gl/src/cert.rs:
    missing_version_defaults_to_1_and_verifies,
    explicit_version_1_takes_v1_path.
  - 12 existing ref_certificate_tests still pass with the
    version column added.

Compiles clean, 1091 tests pass with 0 regressions, gl 4 cert
tests pass, clippy clean under -D warnings, fmt clean.

Cross-PR overlap (declared in the PR description):

  - #134 (anchors auth): independent. The cert API is unchanged
    in shape; auth gates apply as before.
  - #285 (advisory-lock session affinity): independent.
  - #306 (Content-Digest on signed requests): independent.
  - #314 (small-order Ed25519): independent. v1 certs verify
    through the same path, which already enforces the small-
    order check from #314 on the embedded node DID.
  - #324 (libp2p keypair persistence): independent.
  - #325 (gossip ref-update auth): independent. PR 1's signed
    HTTP push is the durable-intent producer; this PR owns the
    cert shape. The two PRs read and write the same cert table
    but do not step on each other (PR 1 calls
    issue_ref_certificate_idempotent; this PR keeps the
    deterministic cert id but does not change who calls what).
  - #382 (replication withheld-subtree trees): independent.

* fix(node,gl): address review feedback on cert version compat

- Drop the duplicate `version` ALTER from the v1 migration bundle; v28
  is now the sole owner of the column.
- Add a v28 upgrade-path test (v25/v26 pattern) that seeds a legacy
  pre-v28 ref_certificates row, drops the column + migration record,
  re-runs migrations, and asserts DEFAULT 1 + legacy row reads as v1.
- Extract a shared `v1_signing_payload` builder so the live signer
  and the frozen-vector test share one literal; a regression in the
  signer now fails the test.
- Tighten gl's `version` parsing: missing key → v1, present value
  must be a JSON integer that fits in u32 and equals 1. Null, strings,
  floats, overflow, and unknown versions all return Err instead of
  collapsing to v1 (Reviewer 2). Add a truth-table test pinning every
  cell of the parser.
- Extend the v1 forward-compat tests to actually round-trip through
  `verify_signature`, and add a regression test that pins the verdict
  branch rejects v2 even when the v1 signature would otherwise verify.

* fix(cert): address CI failures on PR #386

- cargo fmt --all (CI fmt check was red on the prior commit)
- v28 upgrade-path test: switch version_column_default from
  fetch_one to fetch_optional, so a pre-v28 database with the
  column dropped returns Ok(None) rather than RowNotFound. The
  precondition assertion is what detects a missing column;
  RowNotFound would have masked a regression.
- clippy `needless_borrow` on the v2 verdict-branch test:
  drop the leading `&` on `cert_json["signature"].as_str()`.

* fix(gl,node): make the v2 rejection and issuer anchor real, and test them as such

Round-2 findings, all three demonstrated by execution:

- cmd_show's Ok(v != 1) verdict arm was dead code (parse_cert_version
  admits only 1), and the test aimed at it copied the match instead of
  calling the command. The match now mirrors the parser's contract and
  two end-to-end mockito tests drive cmd_show itself: a version-2 cert
  with a valid v1 signature fails --verify with the version named.

- --verify no longer accepts the queried node's self-reported DID as a
  trust anchor: the node that served a forged cert can serve a matching
  self-report, and that fallback passed a wholly forged certificate with
  exit 0. --verify now requires --expect-node before claiming a trusted
  issuer, with a test proving the refusal (and the anchored positive
  control passing).

- The node-side version test asserted version == 1 on its own literal;
  it now cross-checks the issuing construction: a version-1 claim must
  verify against the v1 payload builder's bytes rebuilt from the cert's
  own fields, and the v1 payload must carry no version key.

* fix(node,gl): address round-2 reviewer findings on #26 split 3/4 cert compat

BREAKING CHANGE: gl cert show <id> --verify now requires --expect-node <did>; the node's self-reported DID is no longer trusted as a cert anchor.

* fix(node): bind cert v2 test to live issuer via sqlx::test

* fix(node,gl): address round-3 reviewer findings on #26 split 3 cert compat

- Stamp new certs v1 until v2 verifier ships; v2 issuance refused by shipped gl
- Add v2_signing_payload with version in signed bytes, downgrade test
- Verify signature in well-formed test, add DB read-back to issuer test
- Fix README/db comments and version-refusal message

* fix(gl): route cert list through error_for_status; correct cert show example

Round 4 reviewer findings on #26 split 3 cert compat:

- P2 (cert.rs:113): cmd_list called .json() straight through on the
  cert list endpoint and let a gated 404 (whose body lacks a
  `certificates` key) fall into the empty-list branch, printing "No
  ref certificates" with exit 0. Wire cmd_list through the same
  error_for_status() gate that cmd_show and resolve_cert_id already
  use, and pin the new behavior with a load-bearing test
  (`cmd_list_surfaces_gated_404_as_err`) that drives cmd_list end
  to end against a 404 mock. cmd_show is unchanged.

- P3 (README.md:314): the documented `gl cert show <id> --verify`
  example bound the certificate ID as the REPO positional, because
  the parser is `gl cert show [OPTIONS] <REPO> <ID>`. Insert the
  missing `<repo>` positional so the example matches the CLI
  parser.

The original BREAKING CHANGE footer in bb553b67 carried the same
shortened `gl cert show <id> --verify` example; this commit's
footer supersedes it with the canonical command, which is what
release-note scrapers read. The round-2 commit message itself is
left in history unchanged; the new footer is the one callers
should rely on.

BREAKING CHANGE: gl cert show <repo> <id> --verify requires --expect-node <did>; the queried node's self-reported DID is no longer accepted as a cert anchor.

* style(gl): fix fmt in cert list response parsing

* fix(gl): label version refusals distinctly and clean cert show help

A cert declaring an unsupported version printed INVALID, which reads as
a failed Ed25519 check even though verify_signature never ran. The
display now prints UNSUPPORTED VERSION for version refusals and keeps
INVALID for real signature failures. The Show help no longer cites
review rounds or commit history.

* docs(node): align RefCertificate.version rustdoc with the signed-bytes v2 contract

---------

Co-authored-by: Gravirei <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-08 21:38:02 +00:00
d84c445f94 fix(gl): derive View: URL from node instead of hardcoding gitlawb.com (#370) (#549)
* fix(gl): derive View: URL from node instead of hardcoding gitlawb.com (#370)

gl repo create and gl mirror print a View: link that was unconditionally
pointed at gitlawb.com, which 404s for repos on self-hosted nodes.

Make the node advertise an optional web_url on GET / (sourced from the
new GITLAWB_WEB_URL env var). The CLI fetches GET / after repo creation
and only prints View: when the node supplies web_url. Nodes without the
config skip the line entirely, which is better than printing a broken link.

* fix(gl): validate web_url response and trim trailing slashes (#370)

Address review feedback:
- Check GET / response status before parsing; only omit View: on
  success responses that lack web_url
- Trim trailing '/' from web_url to avoid double-slash in the URL
- Skip the line when web_url is empty after trimming

* fix: address review feedback on View-URL PR (#370)

- Extract fetch_node_web_url() helper for testability; add 6 unit tests
  covering: web_url present, trailing-slash trim, absent, empty string,
  whitespace-only, and server error
- Reject empty/whitespace-only GITLAWB_WEB_URL at boot in Config::validate()
- Trim whitespace from web_url in the CLI helper (not just trailing slashes)
- Document GITLAWB_WEB_URL in .env.example alongside GITLAWB_PUBLIC_URL
- Mirror uses shared fetch_node_web_url() helper

* fix: validate web_url format and surface node info failures (#370)

Node side: GITLAWB_WEB_URL must now parse as an absolute http(s) URL —
scheme-less hosts and other garbage fail at boot instead of serving a
View link no browser can follow.

CLI side: a non-success GET / status or a malformed advertised web_url
now warns on stderr (with the status/value) instead of silently dropping
the View link; successful responses lacking the field still omit it
quietly, as self-hosted nodes without a web front-end are expected to.

* fix: reject query/fragment web_urls and warn on node info errors (#370)

Both web_url validators now reject URLs carrying a query string or
fragment: the CLI builds View links by string-appending /{owner}/{repo},
so "?a=1" would swallow the repo path into the query. The CLI also no
longer swallows transport-level failures on GET / — a refused or failed
connection warns on stderr like the other degraded-link paths.

* fix(gl): cap node info body and reject control-byte web_urls (#370)

Treat GET / like every other caller-chosen node reply: bound the body
read at 8 KiB and parse from the cap (peer.rs precedent). A web_url
containing control or bidi characters — which passes URL parsing but
would reach the terminal verbatim through the View: line — is now
rejected as malformed, with a defanged preview in the warning.

* chore: bump h2 to 0.4.19 for RUSTSEC-2026-0258

h2 0.4.13 is vulnerable to unbounded empty DATA frames (remote DoS);
0.4.16+ carries the fix. Lockfile-only bump, no manifest change.

* fix(gl): close merge regressions on #370 branch and pin View: with integration test

The a6113fd8 merge onto #370 left the workspace red: a missing closing
`}` in `web_url_rejects_non_absolute_or_non_browser_urls` so the next
test nested inside it (config.rs), a duplicate `url = "2"` workspace
dep (Cargo.toml, one from the redirect-rule PR, one from #370), and
`fetch_node_web_url` derefing `CappedBody` as `&str` instead of
`.text` (repo.rs, regression in the same merge). All three are
straightforward to repair; the brace was the only one blocking CI.

The new `cmd_create_view_url_tracks_node_advertisement` integration
test drives the real `gl repo create` binary against a mockito node
that answers both `POST /api/v1/repos` and `GET /`, and asserts the
printed `View:` line tracks the advertised web_url — present with
the advertised origin when set, absent when the node omits it. The
helper-level `fetch_node_web_url_*` tests would still pass if
`cmd_create` reverted to a hardcoded URL; this one would not, so it
is the load-bearing check for #370.

* fix(gl,node): normalize stored web_url and reject userinfo (#370 review)

Two P2s from the #370 review addressed at the configuration boundary.

`Config::validate()` now rewrites `self.web_url` to the canonical
`url::Url::to_string()` after `validate_web_url` succeeds, so the
regular and degraded `GET /` handlers advertise the same string that
was actually validated instead of the operator-typed one. The CLI
trims a second time in `fetch_node_web_url`, which hid the divergence
on the View: line, but any other consumer parsing `web_url` directly
would see a non-canonical base. `validate` is now `&mut self` to
carry the write; the one non-test caller already binds `mut` and the
two test bindings are updated to match.

Both `validate_web_url` (node) and `fetch_node_web_url` (CLI) now
reject URLs with a non-empty username or a password: the configured
value is published on public `GET /` and reprinted on the View: line,
so an accidentally configured `https://user:[email protected]`
would land in public metadata, terminal scrollback, and CI logs.
New regression tests cover user-only and user+password forms on both
sides, plus a normalization test that the stored value matches the
canonical form (with and without operator padding).

* fix(gl,node): classify capped node-info bodies and reject encoded bidi in web_url

An oversized or half-read GET / body previously fell through JSON parse
failure into the same silent-None arm as a node with no web front-end.
The classify step now consults the capped body's truncated and
read_failed flags and warns before omitting the View link.

web_url validation on both sides also scanned only literal bytes for
control and bidi characters, so a percent-encoded reordering character
(%E2%80%AE) passed boot validation and CLI ingest even though a browser
decodes it into a bidi spoof. Both validators now check the decoded host
and path (INV-6). Also fixes the control-byte test, which served invalid
JSON and passed without reaching the check it pinned.

---------

Co-authored-by: Gravirei <[email protected]>
Co-authored-by: Gravirei <[email protected]>
2026-11-08 18:54:05 +00:00
beardthelionandAyush7614 42cc893fb3 fix(gl): make doctor exit non-zero on Fail-class checks (#357) (#547)
* fix(gl): make doctor exit non-zero on Fail-class checks (#357)

gl doctor printed per-row status and 'Some checks failed' prose, but its
exit code was always 0. A diagnostic whose exit status cannot express
failure is a trap waiting for the first 'gl doctor && ...' to land
somewhere it should not (#357).

Three changes:

1. Re-tier the 'GITLAWB_NODE unset' check from Fail to Warn. The CLI's
   --node flag defaults to https://node.gitlawb.com (PUBLIC_NODE) and the
   CLI works fine without the env var, so an unset env is advisory, not
   a failure. Without this re-tier the obvious 'return non-zero on any
   failure' change would have flipped gl doctor to exit 1 on a stock
   working install, which is exactly the regression the issue warns
   against.

2. Route the exit-code decision through a new has_failures(&[Check])
   helper: exit 1 if and only if at least one row is Fail-class.
   Warn-class rows (iCaptcha offline, version drift, shell-alias
   shadowing, GITLAWB_NODE unset) keep the process at exit 0 because
   those conditions are still printed to the user and the obvious
   '&& pipeline' pattern stays valid. Use std::process::exit(1) rather
   than returning Err so anyhow's error frame does not duplicate the
   user-facing summary that already prints.

3. Three unit tests pin the new predicate against the three regimes:
   all-Ok, warn-only, and a single Fail row tripping the exit. The
   warn-only case is the regression guard for the re-tier: if anyone
   flips GITLAWB_NODE back to Fail, exit_predicate_is_false_for_warn_only
   fails too, so the two halves of the fix cannot drift apart.

Verified end-to-end against the public gitlawb node:
  * unset GITLAWB_NODE, missing identity/registration  -> exit 1
  * unset GITLAWB_NODE, full healthy install           -> exit 0 (warn)
  * missing git-remote-gitlawb                         -> exit 1

* fix(gl): pin doctor exit wiring and GITLAWB_NODE Warn tier

- Extract GITLAWB_NODE env classification into gitlawb_node_env_check()
  so unit tests can pin the Warn tier directly; the previous
  has_failures-only tests built their own Check::warn rows and could not
  see a re-tier back to Fail.

- Add three unit tests for the extracted helper (unset/empty is Warn,
  set is Pass, loopback is Pass).

- Add integration probe crates/gl/tests/doctor_exit.rs that drives the
  real binary with a temp --dir and --node http://127.0.0.1:1 and asserts
  exit 1. Deleting the std::process::exit(1) gate keeps the unit suite
  green but breaks this probe, closing the gap reviewer noted.

Co-authored-by: review fix for #391

* fix(gl): align GITLAWB_NODE operator copy with PUBLIC_NODE

* fix(gl): dial the public node when GITLAWB_NODE is empty

Check 3 groups an empty env with unset (Warn), but clap still binds
args.node to "" and the connectivity probe dialed it verbatim, so a
GITLAWB_NODE="" install showed a Fail row and exited 1. effective_node_url
normalizes empty or whitespace-only input to PUBLIC_NODE before the
probe so empty matches unset on check 4.

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-08 04:42:15 +00:00
0189990ac9 fix(node): bound federated repository aggregation (#546)
* fix(node): bound federated repository aggregation

* fix(node): propagate federated peer truncation

* fix(node): Bound federation latency and preserve truncated peer pages.

* fix(node): complete federation review follow-ups

Register the bounded peer-query fixture in the writer ledger, name federation bounds, and share the state-owned limiter across routers and the cleanup sweep. Cover exhausted-bucket rejection before database access.

* fix(node): pin federated response contract end to end (#466 review)

Address beardthelion's P2: the federated envelope was only exercised at
unit level, so hardcoding "truncated": false, deleting the failed-peer
truncated flag, or dropping node_did from enrich_federated_repo each left
all nine federated tests green.

Add four route tests driving build_router and asserting on the real HTTP
200 body:
- envelope fields (count, nodes_queried, truncated) and local annotations
  (node_url, node_did, local) for a seeded public repo
- truncated == true when the peer table overflows MAX_FEDERATED_PEERS
  (unreachable peer URLs, no live peer needed)
- truncated == true on the failed-fetch path with the table under the bound
- peer annotations (node_url, node_did, local: false) via a mockito peer

* fix: strengthen federated response bound tests per review

- db/mod.rs: add writer-ledger entries and disposition docs for the
  three federated route tests touching the peers table
- api/repos.rs: seed the envelope test repo under a non-node owner DID
  so node_did must equal the actual node DID; pin the outbound peer
  query to exact limit=200&offset=0; make the peer-table-overflow test
  use a reachable stub for every peer so the count bound is the only
  possible reason for truncated=true, and assert nodes_queried

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: Marlowe <[email protected]>
2026-11-08 02:24:53 +00:00
beardthelion 61d8a4f8f1 fix(core): compute created-skew in checked arithmetic (#358) (#545)
created= is attacker-supplied i64 with no range clamp. now - created
overflows on i64::MIN (panic in debug, wrap in release), and the wrap
case made abs() return i64::MIN, which is not > 300, so a skew of 2^63
seconds read as fresh. Treat an unrepresentable difference as the
largest possible skew.
2026-11-07 23:24:17 +00:00
b33562f1a9 fix(node): bound Identify-derived Kademlia addresses (#544)
* fix(node): bound Identify-derived Kademlia addresses.

* fix(node): preserve refreshed Identify addresses.

* fix(node): Log rejected known-peer addresses for diagnosis.

* fix(node): bound Identify report processing and refresh eviction

Avoid retaining peers with empty or rejected address reports, bound canonicalization to a fixed input prefix, and move refreshed addresses behind stale addresses in global eviction order. Add regressions for each bound and below-cap growth.

* fix(node): address CodeRabbit majors on Identify address bounds

- Reject Identify address admissions once the global address budget
  is full instead of evicting another peer's entry. Cross-peer
  eviction let a flood of new identities push honest peers'
  addresses out of the book and, when the evicted address was their
  last one, out of the Kademlia routing table. Per-peer eviction of
  the reporting peer's own addresses is unchanged.
- Re-add refreshed Identify addresses to Kademlia, not just newly
  added ones. Kademlia may have dropped a retained address after a
  failed dial or never kept it for a full k-bucket, and a TTL
  refresh alone would never re-admit it.

* fix(node): address #462 review findings from beardthelion

- remove the write-only insertion_order bookkeeping (field,
  remove_insertion_token, and its call sites) and collapse the repeated
  peers.get/get_mut lookups in IdentifyAddressBook::update into a single
  entry binding
- drop a peer row at the end of update() when every candidate was
  rejected, so zero-address rows no longer linger until the expiry tick
- add a regression test proving per-peer eviction still applies at full
  global budget (fails if the global check is hoisted above eviction)
- extract the explicit-address retention predicate into the tested
  is_explicit_address helper used by both Kademlia removal sites

* fix(node): pin row-removal and expiry guards in Identify address book

Address beardthelion's round-two P3 findings on #462:

- Add test that a fully-rejected fresh peer at full global budget
  leaves no peers row (pins the empty-row drop in update()).
- Add test that update() past IDENTIFY_ADDRESS_TTL reports the stale
  address in changes.removed (pins expiry reporting).
- Add test that expire() drops rows emptied by expiry (pins the
  sibling drop in expire_peer).
- Distinguish per-peer from cross-peer eviction in the saturation
  test: insert a sentinel peer before the victim and assert its
  address survives, so a global-oldest eviction policy would fail.

* fix(node): pin refresh predicate negative direction per beardthelion round 3

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: Marlowe <[email protected]>
Co-authored-by: cairn-intern <[email protected]>
2026-11-07 22:49:33 +00:00
beardthelionandGravirei d3845ced69 fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#121) (#543)
* fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#134)

Add auth rejection to list_pins and list_anchors handlers. The pin index
spans the entire node and would expose metadata for every object ever
pushed; anonymous callers must not see it.

- Require AuthenticatedDid extension in both handlers, return 401 when absent
- Add server.rs regression test for anonymous rejection through build_router
- Fix closed-pool tests to pass authenticated requests (test 503 path)

* fix(node): enforce authentication on /api/v1/ipfs/pins and /api/v1/arweave/anchors routes

* fix(node): format arweave routes for consistency in build_router function

* fix(node): gate scoped /api/v1/arweave/anchors reads on repository visibility

Closes the review follow-up on #134: a signed non-reader could pass the
auth-layer check on `?repo=<owner>/<name>` and obtain private-repo anchor
metadata (ref names, old/new SHAs, CIDs, irys tx ids). The auth check
proved only that the signature was valid; it did not bind the request to
the canonical repo-read policy.

* Parse the slug through `validate_repo_slug` (same helper the sync path
  uses) and call `authorize_repo_read` before the SQL query. Missing
  repos, quarantined mirrors, and signed non-readers all collapse to the
  standard 404 (`repo_not_found`).
* The unscoped global listing stays auth-only (#121 narrowing; the wider
  permissionless-identity enumeration is the #136 class, out of scope).
* Clamp the limit to `[0, 200]` so `?limit=-1` no longer reaches Postgres
  as `LIMIT -1`.

Five new `build_router` regressions cover the full production contract
through the real `optional_signature` middleware: scoped unsigned 401,
scoped owner 200, scoped non-reader 404 with no anchor metadata leaked
in the body, scoped missing-repo 404 indistinguishable from non-reader,
and `?limit=-1` clamped to an empty 200.

* fix(node): keep /api/v1/ipfs/pins out of the CID resolver's per-IP flood brake

Follow-up to #134: the previous reordering put `/api/v1/ipfs/pins` and
`/ipfs/{cid}` under the same `rate_limit_by_ip` + `IpRateLimiter` layer
stack, so both routes shared the resolver's flood brake bucket. That
bucket is documented as the once-per-request brake for `GET /ipfs/{cid}`
because the resolver can drive bounded-but-expensive repo walks. The
pin listing is a single `list_pinned_cids()` call and has no reason to
share the bucket. Sharing meant /ipfs/{cid} traffic could exhaust the
bucket and 429 the pins endpoint, and signed pin polling could exhaust
the bucket and 429 legitimate CID reads.

Compose `ipfs_routes` from two sub-routers with independent policies,
then merge:
  * `/ipfs/{cid}` keeps `optional_signature` + `rate_limit_by_ip` + the
    `IpRateLimiter` extension. Same behavior as before #134.
  * `/api/v1/ipfs/pins` gets only `optional_signature`. Auth-required
    (handler denies with 401), but no rate limit — it has nothing to
    flood.

Two new `build_router` regressions exercise the bucket independence
through `TrustedProxy::XForwardedFor`:
  * 5 signed pin requests against a size-2 CID bucket all 200, then a
    `/ipfs/{cid}` from the same IP also 200 (pins did not debit the
    bucket).
  * A `/ipfs/{cid}` exhausts a size-1 CID bucket (200, then 429), then
    a signed pin from the same IP is 200 (the bucket does not gate
    pins).

All 65 tests in `server::tests::` + `test_support::tests::ipfs_*`
still pass, including the existing 429 tests for the CID resolver.

* fix(node): normalize scoped anchor SQL filter to stored repo slug

After authorize_repo_read succeeds, the SQL filter used the raw ?repo=
string (format!("{owner}/{name}")). Anchor rows are stored with
normalize_owner_key(owner_did)/name (short key), so a caller using
?repo=did🔑.../name passed authz but got anchors: [] — a false
empty page that broke callers that legitimately pass the full DID
form. Build the filter from record.owner_did normalized and
record.name instead, matching the shape the seeder uses.

* All three findings addressed on bug_fix_2:

  P2 — signed_scoped_anchors_full_did_form_matches_stored_slug_through_build_router now seeds
  a second repo under a different owner and asserts irys-other-repo-tx is absent. Negative
  control: flipping the production filter to None made the test go RED with count: 2 showing
  the cross-repo leak.

  P3 — let short = owner_did.split(':').next_back().unwrap() replaced with let short =
  crate::db::normalize_owner_key(&owner_did).to_string(). Reader and writer now share one
  convention.

  P3 — PR #134 description updated: pins bullet now describes the actual pre-merge layering
  (and cites the axum-layer-vs-merge-pitfall memory entry), and round 2's stored-slug
  normalization is added as its own bullet with the regression's coverage.

  5/5 anchor-related tests pass, cargo fmt --check clean.

* fix(node): pass SignatureBodyLimit state to optional_signature layers

* test(node): complete listing-denial regressions for the new gates

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 21:46:53 +00:00
beardthelionandGravirei bac49f150a fix(node): owner-gate create_task repo_id against hosted repos (#541)
* fix(node): owner-gate create_task repo_id against hosted repos (#496)

* fix(node): address review on #496 gate — opaque lookup errors, no-persist asserts, GraphQL arm pins, honest oracle comment

* fix(node): address P3 review on #496 gate — shared denial text, honest oracle wording, db-fault pins

- Give the REST and GraphQL copies of the create_task owner gate one denial
  sentence: add REPO_OWNER_ONLY_MESSAGE, used by require_repo_owner and the
  REST forbidden() envelope, and pin it from both surface tests.
- Carry the corrected one-bit-oracle wording into the GraphQL gate comment and
  the REST test docstring (drop the "oracle-free" claim).
- Fault-inject the gate's repo lookups: drop machine_id (get_repo_by_id) for
  REST and GraphQL, and quarantined (is_repo_quarantined) for REST, asserting
  the fixed opaque db_error envelope and no schema text.

* docs(node): stop claiming caller-scoped task reads in #496 gate comments

The create_task gate is a write gate. Unknown/quarantined repo ids are stored
verbatim, and the REST/GraphQL task-list paths are caller-independent on this
base, so the comments must not imply those ids are read-scoped to the
delegator/assignee. Describe only the enforced ownership check; note the
one-bit oracle split without asserting a read contract.

* fix(node): mirror AppError's 503 split and opaque INSERT in create_task

- db_error now sends pool-class failures to 503 db_unavailable like
  AppError::Db (db_unavailable made pub(crate)); query faults stay the opaque
  500. Unit tests cover the split and context-wrapped sqlx errors.
- Route the create_task INSERT through db_error instead of serializing
  e.to_string(); a REST fault-injection test pins the opaque envelope.
- The deny/allow matrix reads back the unknown and quarantined tasks to prove
  the id is stored verbatim, and drops no-leak asserts on the fixed forbidden
  envelope that could never carry the repo id.

* test(node): pin stored-verbatim repo_id on GraphQL gate arms

The REST deny/allow matrix reads each accepted task back and asserts the
stored repo_id is the supplied id; the GraphQL unknown/quarantined arms only
asserted no errors, so a GraphQL regression that dropped, normalized, or
rebound the id stayed green. Read those tasks back by the mutation's returned
id and assert the stored label matches.

* test(node): pin createTask repo-ownership marker in the GraphQL authz fence

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 21:31:50 +00:00
beardthelionandGravirei b4a3122500 fix(core): reject zero threshold, canonicalize did:key, document replay (#509) (#540)
* fix(core): reject zero threshold, canonicalize did:key, document replay (#509)

* test(core): drop error echoes from assert messages flagged by CodeQL

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 15:48:05 +00:00
Gravirei 64e4c68331 docs(node): correct round-4 comment claims on client outcome and gate cost 2026-10-11 19:53:13 +06:00
Gravirei 47f920a3f0 fix(node): cap ref-update parsing in-loop and dedupe identical commands 2026-10-11 19:53:13 +06:00
Gravirei bb7cf433d4 fix(node): cap receive-pack ref-update count before per-ref work and intent 2026-10-01 13:09:15 +06:00
Gravirei 18a38603f5 fix(node): surface git's report-status when a stdin EPIPE follows child exit 2026-10-01 13:09:15 +06:00
Gravirei 632bb3e6d1 chore: update rustls 0.23.37 → 0.23.45 to resolve RUSTSEC-2026-0285 2026-09-29 10:20:46 +06:00
Gravirei da3ead7990 fix(node): gate trust-score bump on a new push event row 2026-09-29 10:20:46 +06:00
Gravirei 5e0af57b27 fix(node): no-report durability, cert landing order, bounded reconcile git, flush checks 2026-09-28 09:49:19 +06:00
Gravirei c1f62c4a74 fix(node): sibling-wait outcome, no-report path, bounded prune and guard retry 2026-09-23 10:40:49 +06:00
Gravirei cccc834b15 fix(node): bound guard attempts, sibling-wait outcome, fail-closed prune enumeration 2026-09-21 22:35:38 +06:00
Gravirei 060add048c fix(node): guard cleanup retry with fates, fail-closed mirror prune, refusal coverage 2026-09-19 03:02:21 +06:00
Gravirei 0269c4a831 fix(node): phase-aware intent guard, mirror import prune, terminal-state merge 2026-09-18 11:26:28 +06:00
Gravirei 3952e9c7df chore: cargo fmt 2026-09-17 17:49:44 +06:00
Gravirei edf067be46 fix(test): correct inv22 namespace gate assertions to match actual implementation
The test was looking for the wrong patterns:
- UTF-8 check: error message string, not pre-loop condition
- Namespace check: `*r == "refs/gitlawb"`, not `ref_name ==`
- Visibility exemption: `&&` logic (both false to exclude), not `||`
2026-09-17 17:41:24 +06:00
Gravirei b98eb2cc54 fix(node): close review round on namespace gate, intent lifecycle, and promotion atomicity
Namespace gate hardening:
- Refuse non-UTF-8 ref-update lines (git accepts bytes ≥0x80; gate never sees them)
- Catch bare 'refs/gitlawb' name (starts_with missed it)
- Add deny test for both holes plus refusal wiring verification

Intent lifecycle guards:
- Add IntentDropGuard: terminalize stranded intents on dropped futures
- Bound refuse_receive_pack_before_git to 10s (no statement_timeout on pool)
- Guard mirror fetch: hideRefs blocks non-exempt refs/gitlawb/* from +refs/*:refs/*

Promotion merge atomicity:
- Take row under SELECT FOR UPDATE so merge is atomic with state read
- Prevents racing due worker from stranding newly applied children outside report
2026-09-17 17:28:34 +06:00
Gravirei 10c185af46 fix(node): close review round on pre-git refusal lifecycle, namespace gate, and promotion merge 2026-09-14 09:09:40 +06:00
Gravirei 41534a99ab fix(node): close review round on outbox ordering, marker visibility, and repair reachability
Address the P1/P2 review findings: await the webhook delivery claim
before accepted-child deletion; exempt the marker namespace from the
fail-closed ref peeling guard so visibility walks tolerate blob-pointing
request markers; keep request marker deletion bounded in the purge path;
promote cross-page siblings via idempotent report superset; run
stuck-parent repair when the prepared queue is empty; bound the durable
intent write while admission permits are held; remove dead proof
verification scaffolding.
2026-09-11 01:09:35 +06:00
Gravirei d888b0397f fix(node): single partial-completion model with disposition gate
Split apply_request_effects into phase-1 bundle (push, cert, anchor,
history, webhook, proof ack before child deletion) and phase-2 gate
that retries while non-cancelled siblings remain; Nothing only when
nothing owed. Gate tail, Tigris, and effects on PostGitDisposition
with documented replication carve-out; add partial-sibling webhook
and disposition tests; fix stale comments.
2026-09-08 11:37:44 +06:00
Gravirei c780571289 fix(node): correct marker prereq comment to warn-and-proceed 2026-09-07 12:51:48 +06:00
Gravirei 15d012bc4d fix(node): document attended-restart commit policy and pin executor retry arms
Track outcome-commit success, skip metrics/touch on failure, correct
the lease-expiry comment to startup reconcile; add restart-repair test
for stuck received parents plus RED tests for landing-history and
proof-ack failure paths.
2026-09-07 08:57:16 +06:00
Gravirei a7666de12a fix(node): centralize live retry, bound outcome commit, close retention orphans
Route live Retry/Err through schedule_request_retry_or_quarantine so
attempt progression matches the drain; retry post-git outcome commits
3x before restart repair; block terminal purge while non-terminal
children exist; add v27 upgrade-path test.
2026-09-06 23:42:48 +06:00
Gravirei 690c47cde2 fix(node): load-bearing coverage for intent, guards, and delivery ledger
Handler intent test plus inv22 gate so disabling the durable insert
turns red; unpack-false, mixed ok/ng, divergent cancelled, partial
sibling, and proof-ack tests pin each executor guard; webhook
deliveries record sent_at with stale-pending reclaim; stale
parsed_report comment corrected.
2026-09-06 22:41:06 +06:00
Gravirei e60405fe13 fix(node): close reviewer round on outcome authority and lifecycle ownership
Gate effects on unpack_ok with cleared ordinal and terminal no-effects,
intersect executor with applied children, treat absent report-status as
indeterminate regardless of exit, require landing history before child
delete, claim due requests with recoverable leases, dedupe webhooks by
occurrence ledger, ack-gate proof retention with terminal disposition,
fair marker tombstones with dead-letter, kill_on_drop marker bounds,
and loud migration version collision guard.
2026-09-06 10:08:23 +06:00
Gravirei 70e5e86bac fix(node): keep recovery prereq upgrade best-effort on push path
Refusing pushes when git config upgrade fails broke fake-git and
non-repo disk-path tests with 503. Reconcile already fails closed
on missing reflogs, so downgrade to warn-and-proceed.
2026-09-05 20:55:33 +06:00
Gravirei 73bc7fde05 fix(node): harden request occurrence lifecycle, proof, retry, and retirement
Reject incomplete report framing and omitted refs as indeterminate,
preserve uncertain children on completion, terminalize all-rejected,
bind reconcile with landing history, durable proof with ack gate,
occurrence-keyed anchors, background due loop, deletion quarantine
with operator resolve, children-before-parent purge with marker
tombstones, recovery prereq upgrade check, and bounded marker runner.
2026-09-05 20:30:11 +06:00
Gravirei 4e45f6a858 fix(node): converge durable post-receive to singular request-level lifecycle
Unify producer, evidence, aggregate outcome, executor, retry, and
retirement on the request row: atomic intent and outcome commits,
synthetic normalized reports for implicit-ok/reconciled paths,
reconcile parent promotion with fail-closed deletions and
competing-claimant guard, retry progression from both executable
states with backoff/quarantine, terminal-parent-gated purge with
marker cleanup and hidden refs, minimal digest-only intent plus
request-level RFC9421 proof (v32).
2026-09-04 21:21:28 +06:00
Gravirei 2d64a008fd fix(node): reference-transaction marker + quarantined state + failure matrix (#26 split 1/4 step 5)
Steps 2-4 gave the request row the unit of work, the shared
executor, and the bounded retirement policy. Step 5 closes the
evidence gap: the reconcile now requires a per-request marker
ref (refs/gitlawb/requests/<id>) whose value matches
request_bytes_hash. A missing or mismatched marker quarantines
the request; an operator reclassifies it.

- v31 migration: adds `quarantined` to the state vocabulary and a
  partial index for operator queries.
- Handler writes the marker ref before git-receive-pack; the
  marker is causally bound by being in the same async task as
  the receive-pack call. The marker's value is content-addressed
  (git hash-object of the request bytes), so the gate compares
  consistent SHAs on both sides.
- git::store::read_ref reads a single ref's value, returning
  Ok(None) for absent refs. Used by the marker gate.
- git::store::marker_value_for computes the content-addressed
  marker value; both the live handler and the reconcile use it
  so the write and the read agree.
- Reconcile gains a marker gate between the age check and the
  reflog proof. Mismatch or absent ⇒ mark_request_quarantined +
  mark_children_rejected_for_quarantined_parent.
- effects_max_attempts bound (config knob, default 8) flips
  retry-stuck requests to `quarantined` after N attempts,
  closing the infinite-retry DoS window.
- New failure_matrix_tests submodule covers the spec's outcome ×
  ref-kind × exit-point × recovery-scenario matrix (6 cells).
- New inv26_step5_marker_quarantine_and_bound_are_wired gate
  asserts the marker gate, the bound check, the handler's
  pre-receive-pack ordering, and every load-bearing helper.
- Existing 7 reconcile tests updated to stage a marker ref via
  the new `stage_marker` test helper.
2026-09-03 14:54:23 +06:00
Gravirei a014d8b669 fix(node): bounded retirement purge for terminal request rows (#26 split 1/4 step 4)
The v30 partial index idx_receive_pack_requests_completed_at
exists but no code reads it. Step 4 wires a periodic purge task
that deletes terminal `complete` and `rejected_at_git` rows
older than the retention window, along with their per-ref
children. `quarantined` (a step-5 state) is never purged by the
timer.

- `purge_completed_receive_pack_requests(older_than, limit)` —
  deletes parent requests using the v30 partial index.
- `purge_completed_pending_ref_transitions(older_than, limit)` —
  deletes children of purged parents, gated on the child's own
  applied_at / cancelled_at.
- `purge_request_queue(db, retention_days, limit)` — orchestrator
  in durable_outbox.rs that calls both and returns the totals.
  The deletion order (parents first, then children) is the
  contract; a crash mid-purge leaves orphaned children that the
  next pass will pick up.
- `spawn_queue_lifecycle_sweep` in main.rs — periodic task on
  the same detached pattern as `spawn_legacy_cid_sweep`, 24-hour
  interval, shutdown-aware. The interval matches the spec's "one
  per cluster per day" target.
- Config knobs `queue_retention_days` (default 7, range 1..=365)
  and `queue_purge_batch` (default 1000, matches
  DRAIN_PER_PASS_LIMIT).
- 3 new `durable_outbox::drain_tests::purge_*` tests pin the
  contract: only old terminal rows are deleted, the second pass
  is a no-op, and a row inside the retention window survives.
- New `inv26_step4_queue_lifecycle_purge_is_wired` gate asserts
  the wiring (main.rs calls the purge, the DB helpers exist, the
  Config knobs exist).
2026-09-03 11:53:00 +06:00
Gravirei 95ac6ae862 fix(node): factor out apply_request_effects; drain walks receive_pack_requests (#26 split 1/4 step 3)
The step-2 commit moved the request row to be the unit of work but
left the per-ref effects fan-out inline in the handler and the
drain walking pending_ref_transitions per-ref. Step 3 factors the
effects fan-out into apply_request_effects(state, request_id) and
rewrites the drain to walk receive_pack_requests.

- apply_request_effects lives in durable_outbox.rs; the live handler
  in api/repos.rs and the drain both call it. Idempotent on
  (request_id, accepted_ordinal). Returns EffectsOutcome::{Done,
  Nothing, Retry}.
- Drain switches from list_pending_ref_transitions_applied +
  derive_one to list_receive_pack_requests_due +
  apply_request_effects. derive_one and the per-ref drain entry
  points are deleted.
- The step-3 stubs in db/mod.rs (mark_request_effects_pending,
  mark_request_complete, list_receive_pack_requests_due) lose
  their #[allow(dead_code)] annotations; every stub has a caller
  in this PR.
- Reconcile (reconcile_prepared_from_disk_all) is unchanged.
  The crash window between intent durable and outcomes commit
  still routes through per-ref reflog proof.
- Drain tests rewritten to stage receive_pack_requests rows; the
  per-ref fixtures (make_row, lookup_accepted_ordinal) are gone.
- New inv26_step3_live_and_drain_share_apply_request_effects
  assertion pins the live/drain sharing and the per-request
  drain walk.
2026-09-03 10:37:08 +06:00
Gravirei 5dfacdab17 fix(node): use raw bytes for request_bytes_hash (#26 split 1/4 step 2)
The v30 migration defined request_bytes_hash as BYTEA but the handler
was binding a hex-encoded String. Postgres rejected the insert with
"column request_bytes_hash is of type bytea but expression is of type
text" and the handler shed the push with a 503.

Switch the handler to bind the raw 32-byte digest and update the
ReceivePackRequest struct field type to Vec<u8> to match. The
drain-side test fixture already used Vec<u8> and needs no change.

Pinned by the bin test suite going from 24 failures (all the
receive-pack-cap tests that send a 4-byte body) to 0 regressions
on the new model.
2026-09-03 08:52:59 +06:00
Gravirei 9438db4075 fix(node): rewrite receive-pack handler against the request-level model (#26 split 1/4 step 2)
The v30 migration added receive_pack_requests and dropped first_ref_name,
but no Rust code used either. The handler still ran a four-branch per-ref
state flip plus a first_ref_name rewrite after git returned, which is the
mixed-outcome bug a push with the first ref rejected exposes: the live
path and the drain computed different push-event ids.

- Insert a receive_pack_requests row in state `received` BEFORE git runs
  (insert_receive_pack_request), carrying the request bytes and the
  SHA-256 of the body. Crash between intent and git return is now
  recoverable via the reconcile step.
- Drop the first_ref_name rewrite at api/repos.rs:2454. The push event
  identity is now (request_id, accepted_ordinal); accepted_ordinal is
  computed once at the per-ref state flip.
- Re-key push_event_id_for and ref_cert_id_for on (request_id, ordinal).
  Anchor job id stays per-transition.
- Transition the request row to outcomes_committed (with parsed_report
  and accepted_ordinal) or rejected_at_git (no report + non-zero exit)
  in the same handler tail as the per-ref state flip. The drain (step 3)
  picks up outcomes_committed rows; today the live path also runs the
  per-ref effects inline.
- Stub the step-3 surface (mark_request_effects_pending, complete,
  list_receive_pack_requests_due, update_request_attempt) with
  #[allow(dead_code)] + contract-pin comments so the rewrite is
  bisectable: reverting this commit restores the pre-rewrite live path.

The U5 gate (replication tail spawn inside push_succeeded, before
guard.release) is preserved; inv22_replication_tail_spawns_at_the_
durability_boundary stays green.
2026-09-01 23:24:08 +06:00
Gravirei 4cb783ae3d fix(db): v30 migration — add receive_pack_requests table and ordinal column
The reviewer's round-5 finding: the push event identity was encoded
into a mutable per-ref column (first_ref_name) whose correct value is
knowable only after git, and the correction was not committed
atomically with the per-ref outcomes. A crash between git updating
a later ref and the rewrite left the durable rows naming the
rejected ref, and derive_one's row.ref_name == row.first_ref_name
guard then meant no push event ever landed for the accepted child.

Per the state-transition model at
.gravirei/plans/state-model-durable-post-receive.md, this migration
introduces the request-level record that owns the push event and the
trust score, and extends the per-ref child with an ordinal column
the drain and the effect executor walk together. first_ref_name is
dropped; the push event id is keyed on
(request_id, accepted_ordinal) and lives on the request row.

The next commit (handler rewrite) wires the live path and the
recovery drain against the new tables.

BREAKING CHANGE: pending_ref_transitions.first_ref_name is dropped.
Callers that read or write that column must move to
receive_pack_requests.accepted_ordinal.
2026-09-01 21:57:00 +06:00
Gravirei 4c95ca5603 fix(node): address round-5 reviewer findings on #26 split 1/4 2026-09-01 11:58:17 +06:00
Gravirei fe7963eeb1 fix(node): drop redundant reflog gate; implicit-ok on exit-zero no-report 2026-09-01 11:42:19 +06:00
Gravirei d1b7c0ba6e fix(node): remove empty line after doc comment 2026-09-01 03:36:57 +06:00
Gravirei 3eaba7ebbc fix(node): rustfmt round-4 reviewer fixes 2026-09-01 03:25:38 +06:00
Gravirei f49ae0fd8b fix(node): address round-4 reviewer findings on #26 split 1/4 2026-09-01 03:20:44 +06:00
Kevin Codex c2ad0e770e fix(node): read the report through both side-band frames
The report-status is framed twice whenever the client negotiates
side-band-64k, which `git push` over smart HTTP does: the outer
side-band envelope carries a stream that is itself pkt-line encoded.
One strip pass left the first line as `000eunpack ok`, so the unpack
check failed and parse_report_status returned None — read by the caller
as "no report", which keeps every declared ref. The per-ref landing
gate was therefore inert for exactly the pushes it exists to filter.

The regression test is a byte-for-byte capture from git 2.50.1
rejecting one ref of a two-ref push, because a hand-written
single-framed fixture parses under the old code and hides the case.
2026-08-31 22:04:14 +08:00
Gravirei 1247f4ed1b fix(node): bound the reflog read to a recent tail (#26 split 1/4)
`ref_reflog_entries` slurped the whole reflog. A reflog grows one line
(~150 bytes) per ref update and how often a ref is updated is
PUSHER-controlled, so that was an attacker-sized allocation, taken once
per stranded row, at startup — the moment the node can least absorb it.
Every other attacker-influenced read in this crate is bounded
(read_body_capped, MAX_SIGNATURE_ENTRIES, the pin/walk budgets); this one
should be too.

The bound costs nothing in proving power, which is why it is a ceiling
rather than a tradeoff: the landing gate only ever accepts an entry
stamped at or after `created_at - REFLOG_CLOCK_SKEW`, and reflogs are
append-ordered oldest-first, so every entry that could possibly qualify
is at the tail. The read now seeks to the last REFLOG_TAIL_BYTES (256KB,
~2000 recent updates to one ref) and `take`s that many bytes, so the
allocation stays bounded even if the file grows between the stat and the
read.

A window into the middle of a file starts mid-record, so the bytes before
its first newline are discarded: without that, a boundary landing inside
a record's old SHA leaves a suffix that still tokenizes and yields a
truncated SHA as a bogus `old -> new` pair — and landing proof is an
exact pair match, so a bogus pair is a fabricated proof. An entry older
than the window reads as absent, which is the safe direction: the caller
treats "no matching entry" as unproven and leaves the row where it is.
`Ok(None)` for a missing file, and the absence-of-evidence contract, are
unchanged.

Tests: ref_reflog_entries_reads_whole_records_from_the_recent_tail (lays
the file out so the boundary lands 20 bytes into a record's old SHA — the
alignment that would otherwise parse), an_entry_older_than_the_reflog_tail_reads_as_unproven.
2026-08-31 19:27:47 +08:00
Gravirei e6f2e154ac fix(node): walk the reconcile backlog on a keyset cursor (#26 split 1/4)
The multi-pass reconcile looped but never advanced. Each pass re-issued
the same cursor-less `LIMIT n` query, and unlike the drain — which
deletes every row it finishes, so its next page is always new work — the
reconcile leaves every row it cannot promote exactly where it was. One
unprovable row at the head of the ordering therefore pinned page one on
every pass, and the backlog behind it was never examined at all, which
is the condition the multi-pass loop was added to remove. Rows keep
ageing toward MAX_RECONCILE_AGE while they wait, so "picked up next
restart" can mean "never recovered".

Pages now resume strictly after the `(created_at, id)` of the last row
EXAMINED, via `list_pending_ref_transitions_prepared_or_uncertain_after`.
A keyset cursor rather than an OFFSET because rows leave the set
underneath the walk when they are promoted: an absent row simply does
not appear on a later page, and it can never shift an unvisited row into
a page that was already read. Pass budget, per-pass limit, short-page
exit and the residual-backlog warning are unchanged.

This matters more with the reflog gate in place: a landing in a repo
that keeps no reflog is PERMANENTLY unprovable, so before this it would
jam page one on every startup rather than once.

Tests: reconcile_all_walks_the_backlog_past_the_first_page,
reconcile_all_advances_past_an_unprovable_row (whose blocker is a
no-reflog row, the class the gate introduces).
2026-08-31 19:24:55 +08:00
Gravirei a7a2df0e0e fix(node): require reflog proof before the reconcile promotes a row (#26 split 1/4)
Closes the second half of the reviewer's round-3 [P1]: the startup
reconcile still promoted on `disk_refs.get(ref) == row.new_sha`, which
also matches a coincidental current tip (old=B, new=A while the ref is
already A). That is not an exotic case — it is the ordinary shape of a
REJECTED push, since git refuses an update whose expected old value does
not match the ref. Both the SHA check and the MAX_RECONCILE_AGE window
pass, the push never happened, and the drain then writes a push event, a
signed certificate and an anchor for it.

The live path proves landings from git's report-status body; that body is
gone by the time the reconcile runs, so the proof is re-derived from the
repository. `git::store::ref_reflog_entries` reads `logs/<ref>` and a row
is promoted only when an entry carries its exact `old -> new` pair,
stamped at or after the row was written (REFLOG_CLOCK_SKEW absorbs git's
whole-second truncation against the row's sub-second `created_at`). The
timestamp floor is what separates a landing from a later push that
re-introduced the same pair: proof must postdate the intent it proves.

Deletions stay exempt from the reflog half and keep the absence-plus-age
rule already in place — git removes a ref's reflog along with the ref, so
no other evidence can exist for one.

`init_bare` now sets `core.logAllRefUpdates`, which bare repos default
off; a repo with no reflog yields no proof and its rows stay put for
human-attended recovery rather than becoming accounting the node cannot
substantiate.

Tests: reconcile_refuses_a_coincidental_tip_with_no_reflog_proof,
reconcile_promotes_a_row_the_reflog_proves,
reconcile_leaves_a_row_prepared_when_the_repo_keeps_no_reflog,
reconcile_refuses_a_reflog_entry_older_than_the_row,
reconcile_still_promotes_a_landed_deletion_which_can_have_no_reflog,
init_bare_keeps_reflogs_so_a_landing_can_be_proven,
ref_reflog_entries_is_none_when_the_repo_kept_no_log.

No migrations added or renumbered.
2026-08-31 19:24:55 +08:00
Gravirei 3bdf7065d5 fix(node): update inv22 gate tests for receive_pack_raw refactor
- F3 gate: scan for smart_http::receive_pack_raw( instead of receive_pack(
- U5 gate: scan for all_refs_ok instead of receive_result.is_ok()
2026-08-31 17:02:32 +06:00
Gravirei 40248b404a fix(node): fix CI failures from round-3 changes
- Add COMMENT ON TABLE to v29 migration so migration_bodies_are_non_empty passes
- Return error on non-zero receive-pack exit (preserving backward compat
  with tests that expect Err(AppError::Git(_))) while still parsing
  report-status for outbox row handling
2026-08-31 16:42:26 +06:00
Gravirei 974d9dc5d2 fix(node): address reviewer round-3 findings on #26 split 1/4
- P1: Delete outbox rows after live durable effects complete so they
  don't replay on every restart
- P1: Parse git report-status for per-ref ok/ng results; mark only
  proven rejections as cancelled, uncertain outcomes as recoverable
- P1: Introduce 'uncertain' state for receive-pack errors where some
  refs may have landed; reconcile checks these against disk at startup
- P2: Promote deletion transitions during reconcile (new_sha == ZERO_SHA
  with absent ref = successful deletion)
- P2: Loop reconcile across multiple passes so backlogs beyond the first
  page are processed in the same startup

Closes review round 3 findings from reviewer-1 and reviewer-2.
2026-08-31 16:22:44 +06:00
Gravirei 2638063160 fix(node): address four reviewer findings on #26 split 1/4 (round 2) 2026-08-30 14:07:21 +06:00
Gravirei 1fa9a1f8dd fix(node): address four reviewer findings on #26 split 1/4
- P1-A: add startup reconcile step that promotes `prepared` rows to
  `applied` when the on-disk ref matches the row's `new_sha`. The
  recovery drain (which only reads `applied` rows) can now pick up
  a ref that landed when the live handler's
  `mark_pending_ref_transitions_applied` call errored or was
  interrupted. Strict SHA equality is the load-bearing check — a
  `prepared` row whose target did NOT actually land stays
  `prepared`.
- P1-B: route the live handler's cert issuance through
  `cert::issue_ref_certificate` (the upsert) instead of
  `issue_ref_certificate_idempotent` (DO NOTHING). A re-push to the
  same ref now updates the cert's `old_sha` / `new_sha` /
  `pusher_did` / `issued_at` / `signature` to the new transition
  while preserving the deterministic `cert_id`. The recovery drain
  keeps the idempotent variant; both paths collapse to one row.
- P2-A: refactor the drain into a `drain_pending_ref_transitions_with`
  testable seam that does per-row log-and-continue, and add
  `drain_pending_ref_transitions_all` that loops
  `DRAIN_PER_PASS_LIMIT=1000` rows for `DRAIN_MAX_PASSES=10` passes.
  A failing row no longer stalls the batch; a backlog above 1000
  rows is fully processed across passes.
- P2-B: add a `first_ref_name` column to `pending_ref_transitions`
  via migration v28. The live handler hoists a `first_ref_name`
  local and persists it on every row of the same `request_id`. The
  drain's `derive_one` keys the push event id on
  `row.first_ref_name` instead of `row.ref_name`, so live and
  recovery produce the same id and `ON CONFLICT (id) DO NOTHING`
  collapses a multi-ref push to one push event row (and one trust-
  score bump). Cert and anchor ids stay per-ref / per-transition.
2026-08-29 22:41:28 +06:00
Gravirei 07109f4ec1 fix(node): wire durable outbox into the receive-pack handler (#26 split 1/4)
This is the handler-level half of Split PR 1. The previous commit
added the migration and the DB methods; this one threads them
through crates/gitlawb-node/src/api/repos.rs:2007 (git_receive_pack),
the cert issuer, and the startup drain.

CHANGES IN THE HANDLER
======================

In git_receive_pack, AT THE LAST POSSIBLE MOMENT before the
smart_http::receive_pack call, the handler now:

  1. Generates a per-handler request_id (UUID).
  2. Captures the raw Signature, Signature-Input, and Content-Digest
     headers from the request.
  3. Calls db.insert_pending_ref_transitions(request_id, ...) which
     writes one row per ref update in state 'prepared'.

The receive_pack call runs as before. After it returns:

  4. On Ok: db.mark_pending_ref_transitions_applied(request_id) —
     the row is the ONLY thing that promotes a 'prepared' row to
     'applied', and the drain reads only 'applied' rows. A process
     crash before this call leaves the row in 'prepared', which the
     drain never promotes.
  5. On Err: db.mark_pending_ref_transitions_cancelled(request_id) —
     a failed receive_pack leaves the row in 'cancelled', which the
     drain never promotes.

This is what closes the reviewer's two proofs:

  Proof 1 (crash window): if the process dies after
  mark_pending_ref_transitions_applied but before the bookkeeping
  writes, the row is in 'applied' and the next startup drain
  re-derives the push event, the per-ref certificate (carrying the
  ORIGINAL pusher DID, not a placeholder), and the anchor handoff.
  The drain uses the persisted authentic pusher DID and signature
  header, not a recovered placeholder.

  Proof 2 (failed receive-pack): the row is only ever flipped to
  'applied' in the explicit Ok branch above. A 'prepared' or
  'cancelled' row is invisible to the drain, so a failed or dropped
  receive_pack cannot turn a prepared intent into completed
  accounting or anchoring.

BOOKKEEPING IS NOW DETERMINISTIC-ID
===================================

The post-Ok bookkeeping at api/repos.rs:2448 now uses:

  - record_push_with_id with push_event_id_for(request_id, first_ref)
    — ON CONFLICT (id) DO NOTHING, so a recovery re-pass is a no-op.
  - issue_ref_certificate_idempotent with
    ref_cert_id_for(request_id, ref_name) — ON CONFLICT (repo_id,
    ref_name) DO NOTHING, returns None if a live-path cert already
    exists.
  - insert_anchor_job_idempotent with
    anchor_job_id_for(repo_id, ref_name, old_sha, new_sha) — the
    per-transition tuple key, so two pushes to the same ref produce
    one anchor upload per landed state.

The legacy entry points (record_push, issue_ref_certificate,
insert_ref_certificate) remain for callers that prefer a fresh UUID
per cert; they are #[allow(dead_code)] for the PR 3 cert/CLI compat
pass to decide whether to keep or remove.

STARTUP DRAIN
=============

crates/gitlawb-node/src/main.rs calls
durable_outbox::drain_pending_ref_transitions(state, 1000) ONCE
before serving, after migrations and after the existing peer /
quarantine prunes. Non-fatal: a transient drain failure logs and
leaves the rows for the next startup.

durable_outbox::drain_pending_ref_transitions reads every 'applied'
row, calls derive_one (which re-derives the three artifacts using the
persisted authentic pusher DID and signature header), then deletes
the row. A second drain pass is a no-op for both the artifacts
(idempotent inserts) and the row (gone after the first pass).

NEW END-TO-END TESTS
====================

crates/gitlawb-node/src/durable_outbox.rs adds three end-to-end
tests in drain_tests, complementing the eight DB-layer tests in
db::pending_ref_transition_tests:

  - drain_re_derives_all_three_artifacts_for_an_applied_row: the
    reviewer's first proof. Inserts a row in 'applied' state (the
    crash window), drains, asserts exactly one push event row,
    exactly one cert row carrying the original pusher DID (not a
    placeholder), and exactly one anchor job row. Asserts the
    deterministic cert id matches. Asserts a second drain pass is a
    no-op.
  - cancelled_row_produces_no_artifacts: the reviewer's second proof
    for the cancelled state. A row in 'cancelled' (receive_pack
    returned Err) is invisible to the drain.
  - prepared_row_produces_no_artifacts: the reviewer's second proof
    for the prepared state. A row in 'prepared' (handler crashed
    between insert_prepared and the post-Ok branch) is invisible to
    the drain.

Each test names the invariant it pins and the production line it
covers. Reverting that line turns the named assertion red.

Compiles clean, 1099 tests pass with 0 regressions, clippy clean
under -D warnings, fmt clean.

Cross-PR overlap (declared in the PR description):

  - #134 (anchors auth): composes. The /arweave/anchors route
    already requires auth; this PR does not change the route.
  - #285 (advisory-lock session affinity): composes. The durable
    intent is written inside the same handler that holds the lock
    from #285; no changes to the lock layer.
  - #306 (Content-Digest on signed requests): composes. PR 1
    persists the Content-Digest header that #306 makes mandatory.
  - #314 (small-order Ed25519): independent. PR 1's tests use strong
    keys.
  - #324 (libp2p keypair persistence): independent. PR 1 does not
    touch p2p identity.
  - #325 (gossip ref-update auth): independent. PR 1's signed
    envelope is the HTTP-side equivalent, not the gossip-side.
  - #382 (replication withheld-subtree trees): independent. PR 1
    does not touch replication or pin selection.
2026-08-28 23:17:28 +06:00
Gravirei 25f8b716e7 fix(node): durable post-receive outbox at the DB layer (#26 split 1/4)
Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split the
work into four narrow PRs. This is Split PR 1 (durable post-receive
lifecycle) at the DB layer; the handler refactor in
crates/gitlawb-node/src/api/repos.rs:2007 (git_receive_pack) lands in
the next slice so the test can drive the failure injection end-to-end.

The pre-outbox crash window the reviewer flagged: receive_pack can
apply a ref to disk and return Ok, and a process exit, a dropped
future, or a DB failure before the bookkeeping at
crates/gitlawb-node/src/api/repos.rs:2361 (push event + cert + webhook)
loses the recovery record. Startup drain enumerates only sources
written from that bookkeeping, so it cannot reconstruct the missing
work. The partial fallback that re-derives from a row present in the
bookkeeping substitutes did:key:recovered and an empty attestation,
which is not equivalent to the original authenticated push.

This commit adds the durable boundary the handler will lean on.

NEW TABLE pending_ref_transitions (migration v27):
  - Written by the handler BEFORE smart_http::receive_pack, in state
    'prepared', carrying the verified pusher DID, the raw RFC 9421
    signature header, signature-input, and content-digest that
    authorized the push, the request id, and the parsed ref update.
  - The handler transitions the row to 'applied' on receive_pack Ok
    or 'cancelled' on Err. The drain reads only 'applied'.
  - A failed or cancelled receive-pack therefore leaves the row in
    'prepared' or 'cancelled', which the drain never promotes. This
    is what closes the reviewer's second proof ("a failed or cancelled
    receive-pack does not turn a prepared intent into completed
    accounting or anchoring").

NEW TABLE anchor_jobs (migration v27, owned by PR 1, consumed by PR 2):
  - One row per (repo_id, ref_name, old_sha, new_sha) transition.
    PR 1 inserts it on 'applied'; PR 2 reads it and updates claimed_at.
  - ON CONFLICT (id) DO NOTHING makes the insert idempotent on the
    deterministic id, so a recovery re-pass cannot create a second
    upload request. This is the handoff boundary; the bundler call
    itself is PR 2.

NEW DB METHODS on Db:
  - insert_pending_ref_transitions: writes one 'prepared' row per
    ref update, returns the persisted rows.
  - mark_pending_ref_transitions_applied / _cancelled: state flip,
    gated on the FROM state, idempotent.
  - list_pending_ref_transitions_applied: drain query, oldest first.
  - delete_pending_ref_transition: called by recovery after the
    artifacts land; a third pass is a no-op.
  - record_push_with_id: ON CONFLICT (id) DO NOTHING on the
    deterministic id.
  - insert_ref_certificate_idempotent: ON CONFLICT (repo_id, ref_name)
    DO NOTHING, returns None if a live-path cert already exists.
  - insert_anchor_job_idempotent: ON CONFLICT (id) DO NOTHING on the
    deterministic per-transition id.

NEW HELPERS in db/mod.rs:
  - deterministic_id: SHA-256 hex with an ASCII Unit Separator
    between fields so two distinct tuples never collide on prefix
    overlap.
  - push_event_id_for, ref_cert_id_for, anchor_job_id_for: the
    derived ids above, one helper per artifact so a caller cannot
    derive a wrong id by mistake.

NEW STRUCTS:
  - PendingRefTransition: the row shape.
  - AnchorJob: the handoff row shape.
  - pending_state: const strings ('prepared' / 'applied' /
    'cancelled') shared by tests, the producer, and the drain so a
    typo on one side cannot silently mismatch the other.

NEW TESTS in db::pending_ref_transition_tests (8 tests, all green):
  - insert_then_mark_applied_flips_state_for_every_ref: producer
    contract.
  - mark_applied_is_idempotent_on_repeat: re-fire is a no-op.
  - cancelled_rows_are_not_returned_by_the_drain: reviewer's second
    proof at the DB layer.
  - prepared_rows_are_not_returned_by_the_drain: same proof for the
    pre-flip state (handler crashed before reaching post-Ok).
  - mark_cancelled_is_idempotent_on_repeat: counterpart.
  - drain_then_re_derive_is_idempotent: reviewer's first proof at
    the DB layer. Inserts a row in 'applied' state directly via
    insert_pending_ref_transition_for_test, drains it, derives the
    artifact ids twice, exercises record_push_with_id and
    insert_anchor_job_idempotent directly, asserts exactly one push
    event row and exactly one anchor job row regardless of how many
    times the drain runs.
  - deterministic_id_avoids_prefix_overlap_collisions: the
    separator regression test.
  - push_event_id_for_is_stable: derived ids match across calls and
    differ on each varied input.

OTHER:
  - Make RefUpdate and its fields pub(crate) so the DB methods can
    iterate the parsed ref updates. No public API change.

NOT IN THIS SLICE (the handler refactor, next commit):
  - The receive-pack handler does not yet call insert_pending_ref_
    transitions before the receive_pack call, nor mark_applied /
    mark_cancelled after. The DB layer is in place for it; the
    handler will call these methods and the startup drain will be
    wired in main.rs.
  - The startup drain in main.rs is not yet called; it will iterate
    list_pending_ref_transitions_applied, re-derive the artifacts,
    and delete the row.
  - The cert/push event issuance in cert.rs and the bookkeeping in
    api/repos.rs:2361 are not yet changed to use the deterministic
    ids. The helper functions exist and are tested; the callers
    follow.

Compiles clean, clippy clean under -D warnings, fmt clean.
2026-08-28 22:49:11 +06:00
52 changed files with 23476 additions and 615 deletions

No files matched your search

+6
View File
@@ -39,6 +39,12 @@ GITLAWB_KEY=/data/keys/identity.pem
# Publicly reachable URL of this node (used in peer announcements)
GITLAWB_PUBLIC_URL=https://your-node.example.com
# Base URL for the web view of repos on this node (used by `gl` to print a
# working View: link after repo creation). Omit for nodes with no web front-end.
# Distinct from GITLAWB_PUBLIC_URL: that is API reachability for peers;
# this is browser reachability for humans.
# GITLAWB_WEB_URL=https://gitlawb.com
# ── Server ────────────────────────────────────────────────────────────────
GITLAWB_HOST=0.0.0.0
GITLAWB_PORT=7545
+1 -1
View File
@@ -81,7 +81,7 @@ time once updated.
**Requirements:**
- Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs)
- PostgreSQL — required for the node. Use the bundled `docker-compose.yml` for local dev.
- PostgreSQL 16+ — required for the node (it uses `pg_input_is_valid`, which exists only on PostgreSQL 16 and later). Use the bundled `docker-compose.yml` for local dev.
- Docker (optional, for full-stack local testing)
**Environment variables:**
Generated
+9 -5
View File
@@ -2208,7 +2208,7 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.48.0",
]
[[package]]
@@ -2701,7 +2701,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de"
dependencies = [
"data-encoding",
"syn 2.0.117",
"syn 1.0.109",
]
[[package]]
@@ -3563,6 +3563,7 @@ dependencies = [
"libp2p-swarm",
"mockito",
"multiaddr",
"percent-encoding",
"prometheus",
"reqwest",
"serde",
@@ -3580,6 +3581,7 @@ dependencies = [
"tracing",
"tracing-subscriber",
"unicode-normalization",
"url",
"uuid",
"zeroize",
"zstd",
@@ -3599,6 +3601,7 @@ dependencies = [
"gitlawb-core",
"icaptcha-client",
"mockito",
"percent-encoding",
"reqwest",
"serde",
"serde_json",
@@ -3607,6 +3610,7 @@ dependencies = [
"tokio",
"tracing",
"tracing-subscriber",
"url",
"urlencoding",
"uuid",
]
@@ -3642,9 +3646,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.18"
version = "0.4.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "839c0e8a181239723652be9062bb56ca5bf5f64011f73b623f6f4fc59086a228"
checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
dependencies = [
"atomic-waker",
"bytes",
@@ -7949,7 +7953,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.61.2",
"windows-sys 0.48.0",
]
[[package]]
+3 -1
View File
@@ -54,8 +54,10 @@ uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core)
# take a parsed URL without pulling reqwest into gitlawb-core; also used for the
# absolute-browser-url validation on GITLAWB_WEB_URL in the node and CLI)
url = "2"
percent-encoding = "2"
# HMAC
hmac = "0.12"
+2
View File
@@ -53,6 +53,7 @@ Good today:
- Local or Docker node startup.
- Postgres-backed repo metadata.
- Bounded GraphQL queries with [repository pagination](docs/graphql-pagination.md).
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
@@ -311,6 +312,7 @@ metadata local disk / optional S3
| DID | A user, agent, or node identity derived from an Ed25519 public key. |
| HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. |
| Ref certificate | Signed record of a ref update. Useful for audit and replication. |
| Ref certificate versioning | The wire format is versioned; new certificates are currently issued as v1. `gl cert show <repo> <id> --verify` requires `--expect-node <did>`, and the queried node's self-reported DID is **not** accepted as a trust anchor. v2 is reserved for a future payload shape that carries the version inside the signed bytes; this client refuses to verify v2+ rather than guessing. |
| UCAN | Delegation token for future capability-based workflows. |
| Peer announce | Node-to-node HTTP announcement of DID + public URL. |
| Gossipsub | libp2p topic for ref-update events. |
@@ -137,6 +137,11 @@ enum ShimMode {
struct Shim {
base_url: String,
posts: Arc<AtomicUsize>,
/// Advertisement (GET /info/refs) count. One `git fetch` performs exactly
/// one advertisement, so tests assert this is EXACTLY one: any fetch-level
/// retry that comes back re-advertises and goes red on the spot, which the
/// lower-bound POST assertions alone cannot see (#275 round 6).
gets: Arc<AtomicUsize>,
stop: Arc<AtomicBool>,
handle: Option<std::thread::JoinHandle<()>>,
}
@@ -166,15 +171,17 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
let addr = listener.local_addr().unwrap();
let base_url = format!("http://{addr}");
let posts = Arc::new(AtomicUsize::new(0));
let gets = Arc::new(AtomicUsize::new(0));
let stop = Arc::new(AtomicBool::new(false));
let posts_t = posts.clone();
let gets_t = gets.clone();
let stop_t = stop.clone();
let handle = std::thread::spawn(move || {
while !stop_t.load(Ordering::SeqCst) {
match listener.accept() {
Ok((stream, _)) => {
handle_conn(stream, &repo, mode, &posts_t);
handle_conn(stream, &repo, mode, &posts_t, &gets_t);
}
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(Duration::from_millis(5));
@@ -187,6 +194,7 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
Shim {
base_url,
posts,
gets,
stop,
handle: Some(handle),
}
@@ -212,7 +220,13 @@ fn normalize_accepted_stream(stream: &TcpStream) {
stream.set_read_timeout(Some(Duration::from_secs(30))).ok();
}
fn handle_conn(stream: TcpStream, repo: &Path, mode: ShimMode, posts: &AtomicUsize) {
fn handle_conn(
stream: TcpStream,
repo: &Path,
mode: ShimMode,
posts: &AtomicUsize,
gets: &AtomicUsize,
) {
normalize_accepted_stream(&stream);
let mut reader = BufReader::new(stream);
@@ -263,6 +277,7 @@ fn handle_conn(stream: TcpStream, repo: &Path, mode: ShimMode, posts: &AtomicUsi
receive_pack(repo, false, &body),
)
} else if method == "GET" && target.contains("/info/refs") {
gets.fetch_add(1, Ordering::SeqCst);
// v0 advertisement, wrapped exactly as the node's info_refs does.
let adv = upload_pack(repo, true, b"");
let mut wrapped = pkt(b"# service=git-upload-pack\n");
@@ -1007,6 +1022,7 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
let server = std::thread::spawn(move || {
let posts = AtomicUsize::new(0);
let gets = AtomicUsize::new(0);
let stream = loop {
match listener.accept() {
Ok((s, _)) => break s,
@@ -1021,7 +1037,7 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
// the assertions below run on every platform.
stream.set_nonblocking(true).unwrap();
accepted_tx.send(()).unwrap();
handle_conn(stream, &repo, ShimMode::Normal, &posts);
handle_conn(stream, &repo, ShimMode::Normal, &posts, &gets);
});
let mut client = TcpStream::connect(addr).unwrap();
@@ -1085,6 +1101,17 @@ fn real_git_multi_round_fetch_completes() {
posts >= 2,
"fixture did not force multi-round negotiation (observed {posts} POST(s)); the bridging path was not exercised"
);
// Exactly one advertisement: the POST bound above is a floor, so a
// reintroduced fetch-level retry could satisfy it by running a second full
// fetch instead of multi-round negotiation. Re-advertising is what a
// second fetch cannot avoid, so this is the committed guard on the
// retry's absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
// The fetched tip is present and the clone's object graph is intact.
let server_head = String::from_utf8(git(&server, &["rev-parse", "HEAD"])).unwrap();
@@ -1177,6 +1204,17 @@ fn real_git_withheld_shaped_first_post() {
"helper hung on a withheld-shaped response (should forward-and-terminate, not deadlock). stderr:\n{stderr}"
);
// Exactly one advertisement, pass or fail: the POST assertions below are
// floors, so a reintroduced fetch-level retry could mask the withheld
// shape behind a second full fetch. A retry cannot avoid re-advertising,
// so this is the committed guard on its absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
if out.status.success() {
// Real git accepted the mid-negotiation pack: the withheld multi-round
// path works end to end with no extra handling.
+157 -4
View File
@@ -5,6 +5,28 @@
//!
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
//!
//! ## Replay
//!
//! A certificate proves *who* asserted a ref update and *what* they asserted.
//! It does not prove the assertion is fresh. [`RefUpdateBody::seq`] and
//! [`RefUpdateBody::nonce`] are inside the signed bytes (see
//! [`RefUpdateBody::to_signing_bytes`]), so a verifier can trust the values,
//! but deciding whether a nonce is being reused or a `seq` is going backwards
//! needs state.
//!
//! This crate is deliberately stateless — it is embedded in both nodes and
//! clients — so replay is **not** enforced here and cannot be. Ownership of
//! the check sits with the verifier that accepts and persists certificates,
//! which must:
//!
//! - reject a `nonce` it has already accepted for the same `(repo, ref_name)`;
//! - reject a `seq` that does not advance past the last accepted `seq` for
//! that `(repo, ref_name)`.
//!
//! Until such a verifier exists, replay of ref-update certificates is not
//! enforced anywhere in this workspace. This contract exists so the first one
//! inherits the requirement instead of re-deriving it.
use std::collections::HashSet;
@@ -41,11 +63,19 @@ pub struct RefUpdateBody {
pub from: String,
/// The new commit hash (SHA-256 hex).
pub to: String,
/// Monotonically increasing sequence number. Prevents replay.
/// Monotonically increasing sequence number, scoped to `(repo, ref_name)`.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a `seq` that does not
/// advance for the same repo and ref. See the module docs on replay.
pub seq: u64,
/// RFC 3339 timestamp of this update.
pub timestamp: DateTime<Utc>,
/// Random nonce for deduplication.
/// Random nonce, unique per certificate.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a nonce it has already
/// accepted for the same repo and ref. See the module docs on replay.
pub nonce: String,
}
@@ -140,7 +170,16 @@ impl RefUpdateCert {
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once.
///
/// `threshold` must be at least 1. Zero is rejected instead of being
/// treated as "no signatures required": a caller that derives the
/// threshold from configuration and gets 0 would otherwise accept any
/// certificate, whether its signatures verify or not.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
if threshold == 0 {
return Err(Error::RefCert("threshold must be at least 1".to_string()));
}
let valid = self.verify_all()?;
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
@@ -378,7 +417,7 @@ mod tests {
}
#[test]
fn threshold_zero_is_always_satisfied() {
fn threshold_zero_is_rejected() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
@@ -389,6 +428,120 @@ mod tests {
&kp,
)
.unwrap();
assert!(cert.satisfies_threshold(&[], 0).unwrap());
// Zero must never read as "no signatures required", even with an empty
// maintainer set.
let err = cert.satisfies_threshold(&[], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"zero must be refused as an invalid threshold"
);
}
/// The threshold is checked before any signature work, so a zero is
/// reported as a threshold error even when the certificate's signatures
/// are also invalid. Otherwise a broken certificate would let a caller
/// mistake the failure for a legitimate verification result.
#[test]
fn threshold_zero_is_rejected_before_signature_verification() {
let kp = Keypair::generate();
let other = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures[0].sig = other.sign_b64(b"unrelated bytes");
let err = cert.satisfies_threshold(&[kp.did()], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"the invalid threshold must win over the invalid signature"
);
}
/// `seq` and `nonce` are the verifier's replay inputs, so they have to be
/// inside the signed body; tapping either one must invalidate the
/// certificate rather than leave it verifying with different values.
#[test]
fn replay_fields_are_covered_by_the_signature() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
7,
&kp,
)
.unwrap();
cert.verify_all().expect("baseline must verify");
let mut bumped = cert.clone();
bumped.body.seq = 8;
assert!(
matches!(bumped.verify_all(), Err(Error::SignatureInvalid)),
"changing seq must invalidate the signature"
);
let mut renamed = cert.clone();
renamed.body.nonce = "different-nonce".to_string();
assert!(
matches!(renamed.verify_all(), Err(Error::SignatureInvalid)),
"changing nonce must invalidate the signature"
);
}
/// A signature minted by a real key must not verify when the signer is
/// spelled in another multibase alphabet. Otherwise the same identity
/// would satisfy a maintainer allowlist under several DID strings.
///
/// Base64url is deliberate: its 47-character method-id fits under the
/// pre-existing length bound, so this test reaches the canonical-prefix
/// rule and fails on the pre-fix code instead of being refused for length.
#[test]
fn verify_all_rejects_a_non_canonical_did_spelling() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let mut prefixed = vec![0xed, 0x01];
prefixed.extend_from_slice(&kp.verifying_key().to_bytes());
cert.signatures[0].signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base64Url, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = cert
.verify_all()
.expect_err("a non-canonical did:key spelling must not resolve");
assert!(
err.to_string().contains("base58btc"),
"rejection must come from the canonical-prefix rule, not the length guard"
);
assert!(
!matches!(
cert.satisfies_threshold(&[cert.signatures[0].signer.clone()], 1),
Ok(true)
),
"the alternate spelling must not satisfy a threshold"
);
// The canonical spelling of the same key is unaffected.
cert.signatures[0].signer = kp.did();
assert!(cert.satisfies_threshold(&[kp.did()], 1).unwrap());
}
}
+158 -3
View File
@@ -20,6 +20,19 @@ use crate::{Error, Result};
/// Value: 0xed01 encoded as a varint.
const ED25519_MULTICODEC: &[u8] = &[0xed, 0x01];
/// The only multibase prefix a `did:key` method-specific id may carry.
///
/// The DID:key spec encodes Ed25519 keys as base58btc, which is multibase
/// `z`. `multibase::decode` happily accepts every other alphabet, and each
/// spelling of the same key bytes would produce a distinct DID string while
/// gates, allowlists, and database rows compare DIDs as strings — one
/// identity would read as several. Rejecting the alternatives here keeps the
/// mapping from `did:key` to key single-valued for every caller of
/// [`Did::to_verifying_key`] (HTTP signatures, UCAN, ref certs, peer
/// announces). `gitlawb-attest` enforces the same rule for attestation
/// signers.
const DID_KEY_MULTIBASE_PREFIX: char = 'z';
/// A Decentralized Identifier.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)]
@@ -71,8 +84,14 @@ impl Did {
/// Resolve the Ed25519 verifying key from a `did:key`.
///
/// Returns `Err` if the DID is not a `did:key`, the key bytes are malformed,
/// or the key is a small-order (weak) point.
/// Only the canonical base58btc (`z`) spelling is accepted. Other
/// multibase encodings decode to the same key but spell a different DID
/// string, so accepting them would let one identity appear under several
/// identifiers to string-comparing gates and allowlists.
///
/// Returns `Err` if the DID is not a `did:key`, uses a non-canonical
/// multibase prefix, the key bytes are invalid, or the key is a
/// small-order (weak) point.
pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() {
return Err(Error::InvalidDid(format!(
@@ -94,9 +113,27 @@ impl Did {
));
}
let (_, bytes) =
// Cheap prefix gate before the decode, then the base check on what
// multibase actually parsed. The first keeps a clearly non-canonical id
// off the decoder. The second is belt-and-braces — multibase maps a
// leading `z` only to `Base58Btc` — but it states the requirement
// directly instead of relying on that mapping, and `gitlawb-attest`
// makes the same pair of checks.
if !self.method_id().starts_with(DID_KEY_MULTIBASE_PREFIX) {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
let (base, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
if base != Base::Base58Btc {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
if !bytes.starts_with(ED25519_MULTICODEC) {
return Err(Error::InvalidDid(
"not an ed25519 multicodec key".to_string(),
@@ -389,6 +426,124 @@ mod tests {
did.to_verifying_key().expect("a real did:key must resolve");
}
/// `multibase::decode` accepts every alphabet, so the same Ed25519 key can
/// be spelled as several `did:key` strings. Only base58btc is canonical;
/// the rest must be refused so one identity cannot appear as several to
/// gates and allowlists that compare DID strings.
#[test]
fn to_verifying_key_rejects_non_base58btc_multibase() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[9u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let canonical = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
canonical.to_verifying_key().unwrap().to_bytes(),
signing.verifying_key().to_bytes(),
"base58btc is the accepted spelling"
);
// These spellings fit inside the method-id bound, so they reach the
// canonical-prefix check and must be refused there.
for base in [Base::Base64Url, Base::Base32Lower] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert_ne!(
did.as_str(),
canonical.as_str(),
"the same key must not compare equal under another spelling"
);
let err = did
.to_verifying_key()
.expect_err("non-canonical multibase must be refused");
assert!(
matches!(err, Error::InvalidDid(_)),
"must be an InvalidDid, got {err:?}"
);
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
// A base16 spelling of the same key is longer than the method-id
// bound, so it is refused by the length guard before the prefix check.
// Either way it must not resolve.
for base in [Base::Base16Lower, Base::Base16Upper] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert!(
matches!(did.to_verifying_key(), Err(Error::InvalidDid(_))),
"a base16 spelling of a real key must not resolve"
);
}
}
/// base58flickr (`Z`) is the non-bitcoin sibling alphabet. Its spelling of
/// a real key fits under the length bound, so it must be refused for the
/// prefix, not merely for its size.
#[test]
fn to_verifying_key_rejects_a_base58flickr_spelling() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[11u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!(
"did:key:{}",
multibase::encode(Base::Base58Flickr, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("base58flickr must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// An empty method-specific id must be refused by the prefix rule without
/// panicking on a missing first character.
#[test]
fn to_verifying_key_rejects_an_empty_method_id() {
let did: Did = "did:key:".parse().expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("an empty method id must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// A canonical base58btc spelling is not enough on its own: the bytes must
/// also carry the ed25519-pub multicodec. This pins the check that stops a
/// key of another type from resolving as an Ed25519 key.
#[test]
fn to_verifying_key_rejects_base58btc_with_a_wrong_multicodec() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[12u8; 32]);
let mut buf = vec![0x12, 0x00]; // sha2-256, not ed25519-pub
buf.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!("did:key:{}", multibase::encode(Base::Base58Btc, &buf))
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("a non-ed25519 multicodec must be refused");
assert!(
err.to_string().contains("ed25519"),
"the error must name the expected multicodec: {err}"
);
}
/// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key
/// satisfies the verification equation for any message and, converted to
/// Montgomery form for X25519, yields the all-zero shared secret, so an
+20
View File
@@ -566,6 +566,26 @@ mod tests {
assert!(sig.check_created().is_ok());
}
#[test]
fn extreme_created_timestamps_reject_without_wrapping() {
let kp = Keypair::generate();
let did = kp.did();
let parse = |created: i64| {
let sig_input =
format!(r#"sig1=("@method");keyid="{did}";alg="ed25519";created={created}"#);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
};
// i64::MIN used to panic in debug (subtraction overflow) and wrap to a
// passing skew in release; now - 2^63 wrapped to i64::MIN, whose abs()
// wrapped back to itself and read as fresh. Both must reject.
for created in [i64::MIN, i64::MAX, Utc::now().timestamp() - i64::MAX] {
assert!(
parse(created).check_created().is_err(),
"created={created} must not pass the freshness gate"
);
}
}
#[test]
fn parse_error_missing_sig1_prefix() {
let err = HttpSignature::parse(
+2
View File
@@ -23,6 +23,8 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
axum = { version = "0.8", features = ["http1", "http2", "json", "ws"] }
async-graphql = { version = "7", features = ["chrono", "uuid", "tracing"] }
async-graphql-axum = "7"
+2
View File
@@ -52,6 +52,7 @@ pub async fn list_certs(
"node_did": c.node_did,
"signature": c.signature,
"issued_at": c.issued_at,
"version": c.version,
})
})
.collect();
@@ -92,5 +93,6 @@ pub async fn get_cert(
"node_did": cert.node_did,
"signature": cert.signature,
"issued_at": cert.issued_at,
"version": cert.version,
})))
}
+3 -1
View File
@@ -11,7 +11,8 @@ use crate::state::AppState;
/// Hard ceiling on rows any ref-update feed returns in one request. Shared by the
/// shared collector's clamp and the per-handler request caps so they can't drift.
const MAX_VISIBLE_REF_UPDATES: i64 = 200;
/// `pub(crate)` so the GraphQL complexity meter can price against the same bound.
pub(crate) const MAX_VISIBLE_REF_UPDATES: i64 = 200;
/// Collect up to `limit` ref-update rows visible to `caller`, newest first,
/// paging past rows the feed gate drops. Filtering after a plain SQL `LIMIT`
@@ -437,6 +438,7 @@ mod ref_updates_feed_tests {
node_did: "did:key:z6MkNode".into(),
signature: "sig".into(),
issued_at: Utc::now().to_rfc3339(),
version: 1,
}
}
+13 -5
View File
@@ -8367,6 +8367,7 @@ mod tests {
#[tokio::test]
async fn get_by_cid_per_source_cap_sheds_same_source_admits_other() {
let mut state = crate::test_support::test_state_lazy();
state.db.pool().close().await;
// Global pool has room; the per-source cap is 1.
state.git_ipfs_walk_semaphore = Arc::new(Semaphore::new(8));
state.git_ipfs_walk_per_caller = crate::rate_limit::PerCallerConcurrency::new(1, 100);
@@ -8393,16 +8394,23 @@ mod tests {
"a source at its per-source /ipfs walk cap must shed 503 with global capacity free"
);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(body["error"], "overloaded");
// A DIFFERENT source is NOT shed by the per-source cap: it clears admission and
// proceeds (then errors on the lazy DB, which is not a 503).
// proceeds to the closed DB, which has a distinct db_unavailable error code.
let resp = ipfs_router(state)
.oneshot(get_cid(&cid, Some(other)))
.await
.unwrap();
assert_ne!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"a different source must not be shed by the per-source cap"
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(
body["error"],
crate::error::DB_UNAVAILABLE_CODE,
"a different source must clear admission and reach the closed database"
);
}
+18 -4
View File
@@ -80,15 +80,20 @@ pub(crate) fn did_matches(a: &str, b: &str) -> bool {
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// Client-facing denial shared by every repo-owner gate copy. `require_repo_owner`
/// renders it as `AppError::Forbidden`; a handler that emits its own
/// `(StatusCode, Json)` envelope instead of `AppError` (e.g.
/// `api::tasks::create_task`) must use this constant too, so two copies of one
/// gate cannot drift in observable output.
pub(crate) const REPO_OWNER_ONLY_MESSAGE: &str = "only the repo owner can perform this action";
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
/// and the author check (close policy) share one normalization.
pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> {
if did_matches(caller, &record.owner_did) {
Ok(())
} else {
Err(AppError::Forbidden(
"only the repo owner can perform this action".into(),
))
Err(AppError::Forbidden(REPO_OWNER_ONLY_MESSAGE.into()))
}
}
@@ -242,6 +247,11 @@ mod authz_guard {
(events, "list_repo_events", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
// #496: create_task is signer-self AND owner-when-repo-scoped — a
// repo_id naming a hosted repo admits tasks only from its owner.
// Both halves are pinned: the did_matches row guards the signer
// binding, this one the repo-ownership gate.
(tasks, "create_task", "require_repo_owner("),
(tasks, "claim_task", "did_matches("),
(tasks, "complete_task", "did_matches("),
(tasks, "fail_task", "did_matches("),
@@ -341,7 +351,11 @@ mod authz_guard {
(
"create_task",
"createTask",
&["require_signer(ctx)?", "did_matches("],
&[
"require_signer(ctx)?",
"did_matches(",
"require_repo_owner(",
],
),
(
"claim_task",
File diff suppressed because it is too large. Load diff
+123 -11
View File
@@ -30,6 +30,33 @@ fn forbidden(msg: &str) -> (StatusCode, Json<Value>) {
)
}
/// This module's error shape with an opaque body: the real error is logged
/// server-side and never serialized (#250), matching the envelope
/// `AppError::Db` renders. Also mirrors its status split: pool-class failures
/// are a retryable 503 `db_unavailable`, everything else the opaque 500
/// `db_error`, so a transient outage is not reported as permanent.
fn db_error(e: anyhow::Error) -> (StatusCode, Json<Value>) {
tracing::error!(error = %format!("{e:#}"), "task database error");
if let Some(sql) = e.chain().find_map(|c| c.downcast_ref::<sqlx::Error>()) {
if crate::error::db_unavailable(sql) {
return (
StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})),
);
}
}
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})),
)
}
// ── Request / response types ──────────────────────────────────────────────────
#[derive(Deserialize)]
@@ -101,6 +128,37 @@ pub async fn create_task(
if !crate::api::did_matches(&auth.0, &body.delegator_did) {
return Err(forbidden("delegator_did must be the authenticated signer"));
}
// #496: a caller-supplied repo_id must name a repo the caller owns. Without
// this, any signed caller can plant a task — payload and UCAN included —
// under a foreign repo id, where the named assignee can claim it. Resolve
// the id against hosted repos: a hosted, non-quarantined repo admits tasks
// only from its owner (403 otherwise). An id naming no hosted repo
// (unknown, or quarantined — which is hidden as if it did not exist, so
// 403ing it would confirm a real id) is stored verbatim as an opaque
// label: this is a write gate and does not bind an unresolved id to a
// repo. Task reads are caller-independent on this base (the list endpoints
// are not repo-scoped), before and after this change. The 403-vs-201 split
// between a hosted repo id and an unknown one is itself a one-bit oracle —
// a stranger holding a private repo's id learns it is hosted here — but
// denying foreign repos requires drawing exactly that line, and repo ids
// already surface via the task list on this base. Repo-less tasks are
// unaffected.
if let Some(repo_id) = body.repo_id.as_deref() {
let record = state.db.get_repo_by_id(repo_id).await.map_err(db_error)?;
if let Some(record) = record {
let quarantined = state
.db
.is_repo_quarantined(&record.id)
.await
.map_err(db_error)?;
if !quarantined && crate::api::require_repo_owner(&record, &auth.0).is_err() {
// Same denial text the GraphQL copy relays from
// `require_repo_owner`, so the two copies of this gate do not
// disagree in observable output.
return Err(forbidden(crate::api::REPO_OWNER_ONLY_MESSAGE));
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -117,12 +175,7 @@ pub async fn create_task(
updated_at: now,
deadline: body.deadline,
};
state.db.create_task(&task).await.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "error": e.to_string() })),
)
})?;
let task = state.db.create_task(&task).await.map_err(db_error)?;
Ok((StatusCode::CREATED, Json(task_to_json(&task))))
}
@@ -175,16 +228,28 @@ pub async fn claim_task(
return Err(forbidden("assignee_did must be the authenticated signer"));
}
let task = state.db.claim_task(&id, &auth.0).await.map_err(|e| {
if e.downcast_ref::<crate::db::TaskReservedForOtherAssignee>()
.is_some()
{
return forbidden("task not claimable: reserved for another assignee");
}
(
StatusCode::CONFLICT,
Json(json!({ "error": e.to_string() })),
)
})?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.filter(|s| !s.is_empty())
.map(str::to_string)
.unwrap_or_else(|| auth.0.clone());
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did: auth.0,
by_did,
at: Utc::now().to_rfc3339(),
});
Ok(Json(task_to_json(&task)))
@@ -219,14 +284,14 @@ pub async fn complete_task(
})?;
if !crate::api::did_matches(
&auth.0,
existing.assignee_did.as_deref().unwrap_or_default(),
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
return Err(forbidden("only the task assignee can complete it"));
}
let by_did = auth.0;
let task = state
.db
.finish_task(&id, "completed", body.result.as_deref())
.finish_task(&id, "completed", body.result.as_deref(), &by_did)
.await
.map_err(|e| {
(
@@ -234,6 +299,13 @@ pub async fn complete_task(
Json(json!({ "error": e.to_string() })),
)
})?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.filter(|s| !s.is_empty())
.map(str::to_string)
.unwrap_or(by_did);
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -272,7 +344,7 @@ pub async fn fail_task(
})?;
if !crate::api::did_matches(
&auth.0,
existing.assignee_did.as_deref().unwrap_or_default(),
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
return Err(forbidden("only the task assignee can fail it"));
}
@@ -280,7 +352,7 @@ pub async fn fail_task(
let reason = body.reason.unwrap_or_default();
let task = state
.db
.finish_task(&id, "failed", Some(&reason))
.finish_task(&id, "failed", Some(&reason), &by_did)
.await
.map_err(|e| {
(
@@ -288,6 +360,13 @@ pub async fn fail_task(
Json(json!({ "error": e.to_string() })),
)
})?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.filter(|s| !s.is_empty())
.map(str::to_string)
.unwrap_or(by_did);
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -297,3 +376,36 @@ pub async fn fail_task(
});
Ok(Json(task_to_json(&task)))
}
#[cfg(test)]
mod tests {
use super::*;
/// The gate's DB helper must mirror `AppError::Db`: a pool-level failure is
/// the retryable 503 `db_unavailable`, a query fault the opaque 500.
#[test]
fn db_error_matches_app_error_status_split() {
let unavailable = db_error(anyhow::Error::from(sqlx::Error::PoolTimedOut));
assert_eq!(unavailable.0, StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(unavailable.1 .0["error"], crate::error::DB_UNAVAILABLE_CODE);
assert_eq!(
unavailable.1 .0["message"],
crate::error::DB_UNAVAILABLE_MESSAGE
);
let query = db_error(anyhow::Error::from(sqlx::Error::Protocol(
"column \"machine_id\" does not exist".into(),
)));
assert_eq!(query.0, StatusCode::INTERNAL_SERVER_ERROR);
assert_eq!(query.1 .0["error"], "db_error");
assert_eq!(query.1 .0["message"], crate::error::DB_ERROR_MESSAGE);
assert!(!query.1 .0.to_string().contains("machine_id"));
}
/// `.context()` layers must not hide the sqlx error from the status split.
#[test]
fn db_error_sees_sqlx_through_context() {
let wrapped = anyhow::Error::from(sqlx::Error::PoolClosed).context("repo lookup failed");
assert_eq!(db_error(wrapped).0, StatusCode::SERVICE_UNAVAILABLE);
}
}
+1
View File
@@ -668,6 +668,7 @@ mod tests {
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new_bounded(12, Duration::from_secs(60), 10_000),
shutdown_tx: tokio::sync::watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
+535 -13
View File
@@ -11,23 +11,34 @@ use uuid::Uuid;
use crate::db::RefCertificate;
use crate::state::AppState;
/// Issue a signed ref-update certificate for a successful push.
/// Issue a signed ref-update certificate for a successful push. The
/// live receive-pack handler calls this on every successful push.
/// Build the v1 signing payload — the EXACT bytes the node signs and
/// gl's `verify_signature` reconstructs. This is the single source
/// of truth shared by `issue_ref_certificate` and the frozen
/// canonical-form tests below; drift here is what would break every
/// existing cert.
///
/// Builds a canonical JSON payload, signs it with the node's Ed25519 key,
/// persists the certificate, and returns it.
pub async fn issue_ref_certificate(
state: &AppState,
/// Field order and key set are fixed: a default `serde_json::Value`
/// serializes `Map<String, Value>` with sorted keys, so any change to
/// the literal is observable as a different byte sequence on the
/// wire.
///
/// v1 is the unversioned legacy shape: no `version` key in the signed
/// JSON. From v2 onward the version belongs INSIDE the signed bytes
/// (see `v2_signing_payload`): leaving it as an unsigned sibling
/// would let a future v2 cert be downgraded to v1 and still verify
/// cleanly, since the bytes would be identical.
pub(crate) fn v1_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
) -> Result<RefCertificate> {
let node_did = state.node_did.to_string();
let issued_at = Utc::now().to_rfc3339();
// Build the canonical signing payload.
let payload = serde_json::json!({
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
@@ -35,13 +46,180 @@ pub async fn issue_ref_certificate(
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
});
})
}
/// Build the v2 signing payload — the future shape for versioned
/// certs. Identical to v1 except the wire `version` is part of the
/// signed bytes, so stripping or flipping the `version` column
/// invalidates the signature instead of verifying cleanly under the
/// other version's path.
///
/// NOT yet issued: `issue_ref_certificate` still stamps v1 because
/// the shipped `gl` verifier only supports v1. This builder exists
/// to pin the downgrade-resistant shape now, before any v2 cert is
/// ever signed.
#[allow(dead_code)]
pub(crate) fn v2_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
"version": 2,
})
}
/// Issue a signed ref-update certificate for a successful push.
///
/// `cert_id` is the deterministic id derived from `(request_id,
/// ref_name)` (see [`crate::db::ref_cert_id_for`]). It is required so
/// the recovery drain and the live handler produce the same primary
/// key: a live push followed by a recovery pass collapses to a
/// single cert row, and a re-push to the same `(repo, ref)` updates
/// the existing row's `old_sha` / `new_sha` / `pusher_did` /
/// `issued_at` / `signature` to the new transition while preserving
/// the original `id` (the `insert_ref_certificate` upsert is
/// keyed on `(repo_id, ref_name)` and only updates fields when the
/// new `issued_at` is strictly greater).
///
/// #26 Split PR 1 P1-B: the live handler routes through this
/// function (the upsert). After the reviewer-1 round-2 fix, the
/// recovery drain also routes through this function (P1: refresh a
/// stale cert), so both paths use the same deterministic `cert_id`
/// and the same upsert. A re-pass is always safe:
///
/// - Live handler → live upsert: re-push updates the row, preserves
/// the original `id`. The contract pinned by
/// `insert_ref_certificate_upserts_on_repo_ref` is restored.
/// - Live handler → recovery: live's `ON CONFLICT (id) DO UPDATE`
/// preserves the original `id`; the recovery's same upsert is
/// a no-op for an equal-`issued_at` re-run and a refresh for a
/// strictly-newer one.
/// - Recovery → live handler: the recovery wrote a row with the
/// deterministic `id`; the live upsert (which preserves `id` and
/// only updates other fields when `issued_at` is strictly newer)
/// is a no-op for an equal-`issued_at` re-run and a refresh for
/// a strictly-newer one.
#[allow(dead_code)] // round-trip test in db/mod.rs pins the upsert contract; the live path and the drain use issue_ref_certificate_with_issued_at
pub async fn issue_ref_certificate(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id: &str,
) -> Result<RefCertificate> {
issue_ref_certificate_with_issued_at(
state, repo_id, ref_name, old_sha, new_sha, pusher_did, cert_id, None,
)
.await
}
/// #26 Split PR 1 round 4 — variant that lets the caller stamp the
/// cert's `issued_at` with a transition-time timestamp instead of
/// `Utc::now()`. The recovery drain passes the persisted
/// `row.created_at` so a replay after a later live cert does not
/// outrank the live cert in the `EXCLUDED.issued_at >
/// ref_certificates.issued_at` upsert guard.
///
/// The live handler uses the default `issue_ref_certificate` (no
/// override), which keeps `Utc::now()` — the reviewer's invariant
/// is that `issued_at` reflects the transition time, and for a
/// live push the transition time and the wall-clock are the same.
///
/// `issued_at_override` is honored verbatim; passing a value not in
/// RFC 3339 form is a logic bug (the upsert will mis-order), so
/// callers must use the row's persisted `created_at`.
///
/// # Clippy allow — too many arguments
/// This is the explicit "stamp a transition-time `issued_at`"
/// variant of `issue_ref_certificate`. The drain
/// (`durable_outbox::derive_one`) is the in-crate caller; the
/// test `replay_of_stale_row_does_not_overwrite_live_cert_b` pins
/// the contract that a recovery replay's `issued_at` does NOT
/// outrank a later live cert. Adding a struct-arg would be a
/// larger refactor for two callers (live + drain) and obscure the
/// parallel to `issue_ref_certificate` (which is `#[allow]`'d for
/// the same reason historically).
#[allow(clippy::too_many_arguments)]
pub async fn issue_ref_certificate_with_issued_at(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id: &str,
issued_at_override: Option<String>,
) -> Result<RefCertificate> {
let cert = build_ref_certificate(
state,
repo_id,
ref_name,
old_sha,
new_sha,
pusher_did,
Some(cert_id.to_string()),
issued_at_override,
)
.await?;
state.db.insert_ref_certificate(&cert).await
}
/// Shared cert construction: build the JSON payload, sign it with the
/// node key, and assemble the `RefCertificate` row. `cert_id_override`
/// lets the recovery path plug in a deterministic id; the live path
/// passes `None` and gets a fresh UUID. `issued_at_override` lets
/// the recovery path stamp the cert with the original transition
/// time so the upsert's `issued_at > issued_at` guard correctly
/// orders transitions regardless of write order.
#[allow(clippy::too_many_arguments)]
async fn build_ref_certificate(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id_override: Option<String>,
issued_at_override: Option<String>,
) -> Result<RefCertificate> {
let node_did = state.node_did.to_string();
// P1 (reviewer-1 round 4): when the caller passes a transition-
// time `issued_at` (the recovery drain passes `row.created_at`),
// use it verbatim so the upsert's per-column guard
// `EXCLUDED.issued_at > ref_certificates.issued_at` correctly
// orders transitions regardless of write order. The live handler
// passes `None` and gets `Utc::now()` — for a live push the
// transition time and the wall-clock are the same.
let issued_at = issued_at_override.unwrap_or_else(|| Utc::now().to_rfc3339());
// Build the canonical signing payload via the shared builder so
// the frozen-vector test below and the live signer cannot drift
// apart — a regression in either side fails both.
let payload = v1_signing_payload(
repo_id, ref_name, old_sha, new_sha, pusher_did, &node_did, &issued_at,
);
let payload_bytes = serde_json::to_vec(&payload)?;
let signature = state.node_keypair.sign_b64(&payload_bytes);
let id = cert_id_override.unwrap_or_else(|| Uuid::new_v4().to_string());
let cert = RefCertificate {
id: Uuid::new_v4().to_string(),
id,
repo_id: repo_id.to_string(),
ref_name: ref_name.to_string(),
old_sha: old_sha.to_string(),
@@ -50,9 +228,353 @@ pub async fn issue_ref_certificate(
node_did,
signature,
issued_at,
// #26 Split PR 3: the wire-format version. New certs stamp
// v1 — the pre-versioning 7-field payload with no `version`
// key in the signed JSON — because the shipped `gl`
// verifier only supports v1. Stamping v2 here would issue
// certificates the shipped client refuses (round-3
// finding: node pinned `version == 2` while gl pinned
// "2 is refused", with nothing exercising the
// composition). v2 is reserved for the future shape
// defined by `v2_signing_payload`, which carries the
// version INSIDE the signed bytes; land the v2 verify
// path before stamping v2.
version: 1,
};
// Persist and return the row as it exists in the database (on a
// conflict the existing row survives when it is newer).
state.db.insert_ref_certificate(&cert).await
}
#[cfg(test)]
mod v1_payload_tests {
//! #26 Split PR 3 — cert payload version compat.
//!
//! The v1 cert payload is the frozen canonical byte form the
//! node signs. gl's `verify_signature` reconstructs the same
//! payload and verifies. Any drift in field order, key set, or
//! whitespace breaks every existing cert. The tests here pin
//! the v1 payload shape, the round-trip, the cert id
//! determinism per the v1 version, and the v2 read forward
//! compat (an unknown version refuses to verify rather than
//! guessing).
//!
//! Reviewer 1 finding: the live signer (`issue_ref_certificate`)
//! and the frozen-vector test (`v1_payload_matches_frozen_canonical_form`)
//! previously held two separate `serde_json::json!` literals,
//! so a regression in the signer could not fail this test.
//! Both now go through the shared `v1_signing_payload` builder.
use super::{v1_signing_payload, v2_signing_payload};
use crate::db::RefCertificate;
use gitlawb_core::identity::Keypair;
/// v1 payload is the same shape the legacy code signed, byte
/// for byte. Reverting this assertion to the v2 shape (which
/// includes a `version` key) is what the versioned format
/// forbids: an old client reading a v1 cert must not see a
/// `version` key in the signed JSON.
///
/// The literal is built via the shared `v1_signing_payload`
/// helper that `issue_ref_certificate` also uses, so the live
/// signer and the frozen vector share one source of truth.
#[test]
fn v1_payload_matches_frozen_canonical_form() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let frozen = format!(
r#"{{"new":"{}","node":"did:key:z6MkNode","old":"{}","pusher":"did:key:z6MkPusher","ref":"refs/heads/main","repo_id":"repo-1","ts":"2026-07-22T00:00:00+00:00"}}"#,
"a".repeat(40),
"0".repeat(40),
);
assert_eq!(serde_json::to_string(&payload).unwrap(), frozen);
}
/// Round-trip: sign the v1 payload with a keypair, build a
/// RefCertificate with version: 1, and verify the structure.
/// This is the construction the live code does on every push.
///
/// Uses the shared builder for both halves so a regression in
/// the live signer (adding/removing/renaming a field) cannot
/// pass this test by accident.
#[test]
fn v1_ref_certificate_structure_is_well_formed() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
);
let payload_bytes = serde_json::to_vec(&payload).unwrap();
let sig = kp.sign_b64(&payload_bytes);
let cert = RefCertificate {
id: "cert-id".into(),
repo_id: "repo-1".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "a".repeat(40),
pusher_did: "did:key:z6MkPusher".into(),
node_did: node_did.clone(),
signature: sig.clone(),
issued_at: "2026-07-22T00:00:00+00:00".into(),
version: 1,
};
assert_eq!(cert.version, 1, "new certs carry version: 1");
// The signed payload reconstructs identically: gl's
// verify_signature would build the same JSON, hash the
// same bytes, and verify the same signature.
let reconstructed = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let reconstructed_bytes = serde_json::to_vec(&reconstructed).unwrap();
assert_eq!(
reconstructed_bytes, payload_bytes,
"the round-trip serialization must be byte-identical; \
this is what makes gl's verify_signature succeed"
);
// Round 3 (P3 reviewer): byte-equality alone never calls
// `identity::verify`, so a bad signature helper would pass.
// Verify the computed signature against the reconstructed
// bytes — the same check gl performs.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &reconstructed_bytes, &sig_bytes)
.expect("well-formed v1 cert signature must verify");
}
/// Reviewer 1 finding: the live signer must share the
/// canonical-form literal so a regression in either side
/// fails both. This test pins the shared builder's output
/// against the literal a future v2 byte stream would have
/// to break — if someone adds a `version` key to
/// `v1_signing_payload`, every existing cert breaks verify,
/// and this test is one of the canaries that fires.
#[test]
fn v1_signing_payload_has_no_version_key() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let obj = payload.as_object().expect("payload is a JSON object");
assert!(
!obj.contains_key("version"),
"v1 signing payload must not carry a `version` key — adding \
one changes the bytes and breaks every existing cert's \
signature. v2 certs build a different payload, not this one"
);
// Pin the exact set so a future field addition is caught.
let expected: std::collections::BTreeSet<&str> =
["new", "node", "old", "pusher", "ref", "repo_id", "ts"]
.into_iter()
.collect();
let actual: std::collections::BTreeSet<&str> = obj.keys().map(String::as_str).collect();
assert_eq!(actual, expected, "v1 payload key set is frozen");
}
/// Round 3 (P2 reviewer): from v2 onward the version belongs
/// INSIDE the signed bytes. v1 stays the unversioned legacy
/// shape; v2 carries `"version": 2` in the payload so a
/// version flip without re-signing invalidates the signature
/// instead of verifying cleanly under the other path.
#[test]
fn v2_signing_payload_binds_version_and_resists_downgrade() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let ts = "2026-07-22T00:00:00+00:00";
let v1 = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2 = v2_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2_obj = v2.as_object().expect("v2 payload is a JSON object");
assert_eq!(
v2_obj.get("version"),
Some(&serde_json::json!(2)),
"v2 payload must carry the version inside the signed bytes"
);
let v1_bytes = serde_json::to_vec(&v1).unwrap();
let v2_bytes = serde_json::to_vec(&v2).unwrap();
assert_ne!(
v1_bytes, v2_bytes,
"v1 and v2 payloads must differ, otherwise a version flip is cryptographically invisible"
);
// A signature over v1 must NOT verify as v2 and vice versa.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_v1 = kp.sign_b64(&v1_bytes);
let sig_v1_bytes: [u8; 64] = URL_SAFE_NO_PAD.decode(&sig_v1).unwrap().try_into().unwrap();
assert!(
gitlawb_core::identity::verify(&vk, &v1_bytes, &sig_v1_bytes).is_ok(),
"v1 signature must verify under v1 bytes"
);
assert!(
gitlawb_core::identity::verify(&vk, &v2_bytes, &sig_v1_bytes).is_err(),
"a v1 signature presented as v2 (downgrade/upgrade flip) must not verify"
);
}
/// The live issuer stamps the version the shipped client can
/// verify. The gl client refuses to verify v2 certs explicitly;
/// this test pins that the version the ISSUING PATH claims
/// agrees with the payload shape it actually signs.
///
/// Round 2: the prior form built a `RefCertificate` literal with
/// `version: 1` and asserted `version == 1` — true no matter what the
/// signer does, so flipping the issuer's version (or its payload shape)
/// left it green. This one drives the actual
/// `issue_ref_certificate` so a regression in the issuer flips the test.
/// The DB row is created against `test_support::test_state`, which
/// gives us a real `AppState` with a real node keypair.
///
/// Round 3: the issuer stamps v1 because the shipped `gl`
/// verifier only supports v1 (stamping v2 shipped certs the
/// client refuses). The binding to the live issuer is kept —
/// flipping the stamp still turns this test red — without
/// changing what production issues.
#[sqlx::test]
#[allow(clippy::async_yields_async)]
async fn issuer_stamps_v1_over_v1_payload(pool: sqlx::PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let repo_id = "repo-issuer-observe";
let ref_name = "refs/heads/main";
let old = "0".repeat(40);
let new = "a".repeat(40);
let pusher = "did:key:z6MkPusher";
let cert_id = "cert-issuer-observe";
// Drive the live issuer and read the persisted row back by
// id, so the test covers the stored `version` column and
// not just the in-process return value.
let cert = crate::cert::issue_ref_certificate(
&state, repo_id, ref_name, &old, &new, pusher, cert_id,
)
.await
.expect("issue_ref_certificate must succeed");
// The cert returned by the issuer is the source of truth
// for both the version claim and the signed bytes.
assert_eq!(cert.repo_id, repo_id);
assert_eq!(cert.ref_name, ref_name);
// Cross-check 1: the issuer's claimed version must match
// what the live function stamps. This assertion is bound
// to the live function, not a hand-built literal:
// flipping the stamp in `issue_ref_certificate` breaks it
// through the actual call path.
assert_eq!(
cert.version, 1,
"the live issuer must stamp v1 until a v2 verifier ships; \
flipping the stamp breaks this assertion through the actual call path"
);
// Cross-check 1b: the persisted row carries the same stamp.
let stored = state
.db
.get_ref_certificate(&cert.id)
.await
.expect("persisted cert must be readable")
.expect("persisted cert must exist");
assert_eq!(
stored.version, cert.version,
"the stored version column must match the issued claim"
);
// Cross-check 2: the signature on the cert must verify
// against the v1 payload builder's bytes, not against some
// other shape the issuer might have introduced. This is
// what gl's `verify_signature` does for a v1 cert: rebuild
// the payload from the cert's own fields and verify the
// signature. If the issuer ever signs a different shape
// while still claiming v1, every shipped client breaks —
// and so does this.
let rebuilt = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let vk = cert
.node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &serde_json::to_vec(&rebuilt).unwrap(), &sig_bytes)
.expect("a v1 cert's signature must verify against the v1 payload shape");
// Cross-check 3: the v1 signed payload must NOT contain a
// version key — v2 is the future shape that adds one inside
// the signed bytes (see `v2_signing_payload`). The v1
// payload (which the cert is signed over) is frozen at 7
// fields. A change that adds `version` to the v1 payload
// breaks every existing cert and this test.
let obj = rebuilt
.as_object()
.expect("rebuilt v1 payload is a JSON object");
assert!(
!obj.contains_key("version"),
"the v1 signing payload must not carry a version key — \
adding one changes the bytes and breaks every existing \
cert's signature"
);
}
}
+314 -7
View File
@@ -66,6 +66,12 @@ pub struct Config {
#[arg(long, env = "GITLAWB_PUBLIC_URL")]
pub public_url: Option<String>,
/// Base URL for the web view of repos on this node (e.g. https://gitlawb.com).
/// When set, `GET /` advertises it as `web_url` so the CLI can print a
/// working `View:` link. Omit for nodes that have no web front-end.
#[arg(long, env = "GITLAWB_WEB_URL")]
pub web_url: Option<String>,
/// Comma-separated list of bootstrap peer URLs to announce to on startup
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>,
@@ -756,6 +762,46 @@ pub struct Config {
value_parser = clap::builder::RangedU64ValueParser::<u64>::new().range(0..=86_400)
)]
pub pin_repair_sweep_delay_secs: u64,
/// #26 Split PR 1 step 4 — receive-pack queue retention window.
/// Terminal `complete` and `rejected_at_git` rows older than
/// this are eligible for the periodic purge. `quarantined`
/// rows are never purged on a timer. The v30 partial index
/// `idx_receive_pack_requests_completed_at` keeps the scan
/// cheap regardless of the value.
#[arg(
long,
env = "GITLAWB_QUEUE_RETENTION_DAYS",
default_value_t = 7,
value_parser = clap::builder::RangedU64ValueParser::<i64>::new().range(1..=365)
)]
pub queue_retention_days: i64,
/// #26 Split PR 1 step 4 — receive-pack queue purge batch size.
/// Each periodic purge pass deletes at most this many terminal
/// rows per batch. The drain and the purge share the same
/// `DRAIN_PER_PASS_LIMIT` budget; see the spawn function in
/// `main.rs` for the wiring.
#[arg(
long,
env = "GITLAWB_QUEUE_PURGE_BATCH",
default_value_t = 1000,
value_parser = clap::builder::RangedU64ValueParser::<i64>::new().range(1..=100_000)
)]
pub queue_purge_batch: i64,
/// #26 Split PR 1 step 5 — effects-executor retry bound. The
/// drain flips a request to `quarantined` after this many
/// `EffectsOutcome::Retry` returns, closing the
/// infinite-retry DoS window. Default 8 matches the spec; set
/// lower in tests.
#[arg(
long,
env = "GITLAWB_EFFECTS_MAX_ATTEMPTS",
default_value_t = 8,
value_parser = clap::builder::RangedU64ValueParser::<i32>::new().range(1..=1000)
)]
pub effects_max_attempts: i32,
}
impl Config {
@@ -803,7 +849,13 @@ impl Config {
/// Cross-field boot validation. Single-field ranges are enforced by clap; this
/// catches combinations that ship a denial-of-service under otherwise-valid
/// values. Call once at startup and fail fast on `Err`.
pub fn validate(&self) -> Result<(), String> {
///
/// `&mut self` is required so the validator can normalize `web_url` to
/// its canonical form after validation: the configured value is
/// advertised on public `GET /`, and any consumer reading
/// `config.web_url` later must see the same string that was actually
/// validated.
pub fn validate(&mut self) -> Result<(), String> {
// Concurrent git writes pin the dedicated lock pool for their whole
// duration (connection-affine advisory lock in acquire_write). The main
// pool no longer carries that occupancy.
@@ -920,10 +972,101 @@ impl Config {
.map_err(|e| e.to_string())?;
}
// GITLAWB_WEB_URL is advertised in GET / so the CLI can print a working
// View: link. Clap maps "" to Some("") which would produce a broken URL;
// reject early instead of serving a malformed link. Non-blank values must
// additionally parse as an absolute http(s) URL — the CLI appends
// `/{owner}/{repo}` to it, so scheme-less hosts ("gitlawb.com") and other
// garbage would render links no browser can follow.
if let Some(raw) = &self.web_url {
match validate_web_url(raw) {
Ok(()) => {}
Err(reason) => {
return Err(format!(
"GITLAWB_WEB_URL {reason} — \
set it to an absolute URL like https://gitlawb.com or leave it unset."
));
}
}
// Normalize to one canonical form so every consumer (the regular
// and degraded `GET /` handlers, anyone who reads
// `config.web_url` directly) sees the same string that was
// actually validated. The CLI's own defensive trim would paper
// over the divergence on the `View:` line, but that hides the
// bug from any other consumer that parses the value verbatim.
let canonical = raw
.trim()
.parse::<url::Url>()
.map(|u| u.to_string())
.map_err(|_| "is not a valid absolute URL".to_string())?;
if canonical != *raw {
self.web_url = Some(canonical);
}
}
Ok(())
}
}
/// Validate a `web_url` value for use as a browser-reachable base URL.
/// Blank values are rejected (clap maps `--web-url ""` to `Some("")`), and
/// non-blank values must parse as absolute `http`/`https` URLs with no
/// query, fragment, or userinfo — the CLI treats this as a string prefix
/// to append paths to, so anything else produces links no browser can
/// follow (`?a=1/owner/repo` puts the repo path inside the query string);
/// and userinfo is rejected outright because the configured value is
/// published on public `GET /` and reprinted on the CLI's `View:` line,
/// so an accidentally configured password would land in public metadata,
/// terminal scrollback, and CI logs.
pub(crate) fn validate_web_url(raw: &str) -> Result<(), String> {
if raw.trim().is_empty() {
return Err("must not be empty or whitespace-only".into());
}
let parsed: url::Url = raw
.trim()
.parse()
.map_err(|_| "is not a valid absolute URL".to_string())?;
match parsed.scheme() {
"http" | "https" => {}
other => return Err(format!("must use http or https, got '{other}:'")),
}
// url::Url cannot represent a URL without a host for http/https schemes, so
// reaching here guarantees an absolute browser-usable base.
if parsed.query().is_some() {
return Err(
"must not contain a query string — it is used as a path prefix for View links".into(),
);
}
if parsed.fragment().is_some() {
return Err(
"must not contain a fragment — it is used as a path prefix for View links".into(),
);
}
if !parsed.username().is_empty() || parsed.password().is_some() {
return Err(
"must not contain a username or password — it is advertised publicly and printed on the View: line"
.into(),
);
}
// Bidi/control bytes can hide behind percent-encoding: a literal scan of
// the wire form misses `.../%E2%80%AE` even though a browser decodes it
// into a reordering character on the rendered View: link. Check the
// decoded host and path before the value is advertised on GET / (INV-6).
let host = parsed.host_str().unwrap_or("");
let decoded_path = percent_encoding::percent_decode_str(parsed.path()).decode_utf8_lossy();
if host
.chars()
.chain(decoded_path.chars())
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c))
{
return Err(
"must not contain control or bidi characters, even percent-encoded — the View: line is terminal output"
.into(),
);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -1156,6 +1299,30 @@ mod tests {
);
}
#[test]
fn queue_lifecycle_knobs_default_conservatively() {
let c = Config::parse_from(["gitlawb-node"]);
// 7-day retention matches the v30 partial index comment
// and the spec at .gravirei/plans/state-model-durable-post-receive.md.
assert_eq!(c.queue_retention_days, 7);
// 1000 rows per pass matches DRAIN_PER_PASS_LIMIT in durable_outbox.
assert_eq!(c.queue_purge_batch, 1000);
assert_eq!(
Config::parse_from(["gitlawb-node", "--queue-retention-days", "30"])
.queue_retention_days,
30
);
assert!(Config::try_parse_from(["gitlawb-node", "--queue-retention-days", "0"]).is_err());
assert!(Config::try_parse_from(["gitlawb-node", "--queue-retention-days", "366"]).is_err());
assert_eq!(
Config::parse_from(["gitlawb-node", "--queue-purge-batch", "500"]).queue_purge_batch,
500
);
assert!(Config::try_parse_from(["gitlawb-node", "--queue-purge-batch", "0"]).is_err());
}
#[test]
fn ipfs_walk_per_source_defaults_and_rejects_out_of_range() {
assert_eq!(Config::parse_from(["gitlawb-node"]).ipfs_walk_per_source, 4);
@@ -1566,7 +1733,7 @@ mod tests {
.expect("default config must validate");
// An under-sized lock pool relative to the push cap is rejected.
let under_lock = Config::parse_from([
let mut under_lock = Config::parse_from([
"gitlawb-node",
"--db-lock-pool-max-connections",
"16",
@@ -1579,7 +1746,7 @@ mod tests {
);
// Exactly at the push cap without non-push headroom is rejected.
let push_only = Config::parse_from([
let mut push_only = Config::parse_from([
"gitlawb-node",
"--db-lock-pool-max-connections",
"32",
@@ -1592,7 +1759,7 @@ mod tests {
);
// Main pool can be smaller than pushes + headroom when the lock pool carries writes.
let split = Config::parse_from([
let mut split = Config::parse_from([
"gitlawb-node",
"--db-max-connections",
"16",
@@ -1607,13 +1774,153 @@ mod tests {
);
// Main pool still needs the app headroom floor.
let under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
let mut under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
assert!(
under_main.validate().is_err(),
"db_max_connections below DB_POOL_APP_HEADROOM must be rejected"
);
}
#[test]
fn web_url_rejects_empty_and_whitespace() {
// Unset is fine.
Config::parse_from(["gitlawb-node"])
.validate()
.expect("no web_url must validate");
// A real URL validates.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://gitlawb.com".into());
assert!(cfg.validate().is_ok());
// Empty string is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("".into());
assert!(cfg.validate().is_err(), "empty web_url must be rejected");
// Whitespace-only is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" ".into());
assert!(
cfg.validate().is_err(),
"whitespace-only web_url must be rejected"
);
}
#[test]
fn web_url_rejects_non_absolute_or_non_browser_urls() {
// Scheme-less host: parses nowhere, renders a broken View: link.
for bad in [
"gitlawb.com",
"not a url",
"ftp://files.example.com",
"javascript:alert(1)",
"https://", // no host
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(cfg.validate().is_err(), "web_url {bad:?} must be rejected");
}
// Query strings and fragments corrupt the appended repo path, since
// web_url is used as a raw string prefix (`?a=1` would swallow
// `/{owner}/{repo}` into the query).
for bad in [
"https://gitlawb.com?a=1",
"https://gitlawb.com/?utm_source=docs",
"https://gitlawb.com#section",
"https://gitlawb.com/#faq",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (query or fragment) must be rejected"
);
}
// Surrounding whitespace around a valid URL is tolerated.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://gitlawb.com ".into());
assert!(
cfg.validate().is_ok(),
"padded-but-valid web_url must validate"
);
// Percent-encoded bidi must not boot: the wire form carries no Cf
// code point, but a browser decodes %E2%80%AE into a reordering
// character on the rendered View: link (INV-6). Literal bidi and
// control bytes are rejected the same way.
for bad in [
"https://example.com/%E2%80%AE",
"https://example.com/\u{202e}spoof",
"https://example.com/%1b[31m",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (control or bidi, even encoded) must be rejected"
);
}
// Non-default ports and paths are fine.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("http://localhost:8080/ui".into());
assert!(cfg.validate().is_ok(), "port+path web_url must validate");
// Userinfo must be rejected: `web_url` is published on public
// `GET /` and reprinted on the CLI's `View:` line, so an
// accidentally configured password would land in public metadata,
// terminal scrollback, and CI logs. Reject username-only and
// user+password forms.
for bad in [
"https://[email protected]",
"https://user:[email protected]",
"https://:[email protected]",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (userinfo) must be rejected"
);
}
}
/// `validate()` accepts whitespace-padded input but the stored value
/// must match the canonical form actually validated, so every consumer
/// (the regular and degraded `GET /` handlers, anyone who reads
/// `config.web_url` directly) sees the same string. Without this, an
/// operator that sets `GITLAWB_WEB_URL=" https://git.example "` boots
/// a node that advertises `" https://git.example "` on public
/// metadata — different from what was validated, different from what
/// any other consumer would parse.
#[test]
fn web_url_is_normalized_to_canonical_form_after_validation() {
// Padded input is rewritten to the canonical form. `url::Url`'s
// own `to_string` is what we use for the canonicalization, and
// it always appends a path-separator slash to a bare-host URL,
// so the stored value is `https://git.example/` (which is also
// what every browser would normalize the operator-typed form to).
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://git.example ".into());
cfg.validate().unwrap();
assert_eq!(
cfg.web_url.as_deref(),
Some("https://git.example/"),
"stored web_url must be the canonical form, not the operator-typed one"
);
// Already-canonical input is left alone after a no-op
// re-normalization — same form comes back out, so any consumer
// reading the stored value sees a stable string across calls.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://git.example/".into());
cfg.validate().unwrap();
assert_eq!(cfg.web_url.as_deref(), Some("https://git.example/"));
}
/// The DECLARED default, read off the parser rather than out of a parse.
///
/// `Config::parse_from` consults the process environment, so on a host that
@@ -1878,7 +2185,7 @@ mod tests {
std::os::unix::fs::symlink(&target, &link).unwrap();
let key_path = link.join("p2p.key");
let config = Config::parse_from([
let mut config = Config::parse_from([
"gitlawb-node",
"--p2p-port",
"0",
@@ -1973,7 +2280,7 @@ mod tests {
let key_path = build(base.path());
let before = snapshot(base.path());
let config = Config::parse_from([
let mut config = Config::parse_from([
"gitlawb-node",
"--p2p-port",
"7546",
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -101,7 +101,7 @@ pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
pub(crate) fn db_unavailable(e: &sqlx::Error) -> bool {
matches!(
e,
sqlx::Error::PoolTimedOut
+7 -4
View File
@@ -118,7 +118,7 @@ impl RepoStore {
/// Test-only: every guard from this store parks in `release` right before the
/// `pg_advisory_unlock` await, until `gate` is notified. Dropping the future
/// while it is parked reproduces a client disconnect inside `release`.
#[cfg(test)]
#[cfg(all(test, unix))]
pub fn with_pre_unlock_gate(mut self, gate: Arc<tokio::sync::Notify>) -> Self {
self.pre_unlock_gate = Some(gate);
self
@@ -126,7 +126,7 @@ impl RepoStore {
/// Test-only: how many write guards from this store have reached the Tigris upload
/// site. See [`RepoWriteGuard::release`].
#[cfg(test)]
#[cfg(all(test, unix))]
pub fn tigris_upload_site_reached(&self) -> usize {
self.upload_site_reached
.load(std::sync::atomic::Ordering::SeqCst)
@@ -1717,8 +1717,11 @@ fn validate_repo_name(repo_name: &str) -> Result<()> {
if repo_name.is_empty() {
anyhow::bail!("repo_name is empty");
}
if repo_name.len() > 100 {
anyhow::bail!("repo_name exceeds 100 chars");
// Byte bound shared with the API-side validator via `MAX_REPO_NAME_LEN`:
// the two validators differ on charset by design, but must not drift on
// length, or an API-admitted name fails here after the proof is spent.
if repo_name.len() > crate::db::MAX_REPO_NAME_LEN {
anyhow::bail!("repo_name exceeds {} chars", crate::db::MAX_REPO_NAME_LEN);
}
// Repo names are `[A-Za-z0-9._-]+` minus path-traversal traps.
if repo_name.contains("..") {
+465
View File
@@ -220,6 +220,7 @@ pub fn response_served_pack(output: &[u8]) -> bool {
///
/// Accepts a push. The caller MUST verify HTTP Signature auth before
/// calling this function.
#[allow(dead_code)] // used by tests; production uses receive_pack_raw
pub async fn receive_pack(
git_bin: &str,
repo_path: &Path,
@@ -244,6 +245,219 @@ pub async fn receive_pack(
.body(Body::from(output))?)
}
/// Run `git-receive-pack` and return the raw stdout bytes together with
/// the process exit status. Unlike [`receive_pack`], this does NOT bail
/// on a non-zero exit: the caller needs the stdout (which contains the
/// report-status with per-ref ok/ng results) even when the process
/// exits non-zero. A timeout still returns `Err`.
///
/// `reflog_action` (typically `gitlawb-request:<request_id>`) is
/// exported as `GIT_REFLOG_ACTION` so the ref transaction writes
/// request-bound landing evidence into the reflog. Recovery requires
/// that evidence before auto-promoting a stranded row; without it a
/// later request recreating the same old/new tuple is
/// indistinguishable from the stranded one.
#[allow(dead_code)]
pub async fn receive_pack_raw(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Vec<u8>, bool)> {
receive_pack_raw_with_reflog(git_bin, repo_path, request_body, timeout, admission, None).await
}
/// [`receive_pack_raw`] with explicit reflog binding.
pub async fn receive_pack_raw_with_reflog(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
reflog_action: Option<&str>,
) -> Result<(Vec<u8>, bool)> {
let mut command = Command::new(git_bin);
command
.arg("receive-pack")
.arg("--stateless-rpc")
.arg(repo_path);
if let Some(action) = reflog_action {
command.env("GIT_REFLOG_ACTION", action);
}
let (out, err, status, _admission) = drive_git_child_raw(
command,
request_body,
timeout,
"git-receive-pack",
admission,
)
.await?;
// On timeout, drive_git_child_raw returns Err — we never reach here.
// On success/non-zero exit, we have the stdout and exit status.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
tracing::warn!(stderr = %stderr, "git-receive-pack exited non-zero");
}
Ok((out, status.success()))
}
/// Parse the git-receive-pack report-status output to determine per-ref
/// success/failure. Returns `(unpack_ok, per_ref_results)` where
/// `per_ref_results` is a list of `(ref_name, is_ok)`.
///
/// The report-status format (after the sideband framing) is:
/// ```text
/// unpack ok\n (or: unpack fail\n)
/// ok <refname>\n (per successful ref)
/// ng <refname> <reason>\n (per rejected ref)
/// \n (empty line terminates)
/// ```
///
/// Returns `None` if the output cannot be parsed (e.g. the client did
/// not request report-status, or the output is truncated). In that
/// case the caller should treat all refs as uncertain.
pub fn parse_report_status(output: &[u8]) -> Option<(bool, Vec<(String, bool)>)> {
let text = std::str::from_utf8(output).ok()?;
// Strip sideband framing: each line starts with a pkt-line length
// prefix and a channel byte (1 = stdout, 2 = stderr). The actual
// data starts after the first `0000` flush packet or after we
// strip sideband bytes.
//
// Framing completeness is part of authority, at whichever level the
// framing lives. A double-framed capture carries the report stream
// (itself pkt-line framed, flush-terminated) inside the sideband
// envelope — real git output has been observed with NO outer flush,
// so the outer level cannot require one; the inner frame's flush is
// authoritative there. A single-framed (or unframed) payload has no
// inner frame, so its only completion marker is the outer flush:
// output ending after a complete `unpack ok` / `ok <ref>` packet but
// before the outer `0000` is a truncated report and yields `None`
// (uncertain), not a successful report.
let (stripped, outer_flush) = strip_sideband(text, false)?;
// The report-status vocabulary (`unpack ...`, `ok ...`, `ng ...`)
// never opens with four hex digits, so a hex quartet here is inner
// pkt-line framing, not report text.
let stripped = if stripped.len() >= 4
&& stripped.as_bytes()[..4]
.iter()
.all(|b| b.is_ascii_hexdigit())
{
strip_sideband(&stripped, true)?.0
} else if !outer_flush {
return None;
} else {
stripped
};
// The report is framed TWICE when the client negotiated side-band-64k,
// which `git push` over smart HTTP does — so this is the common case, not
// an exotic one. The outer frame is the side-band envelope; band 1 carries
// the report-status stream, which is ITSELF pkt-line encoded. Real bytes
// from git 2.50.1 rejecting one ref of a two-ref push:
//
// 0057\x01000eunpack ok\n0028ng refs/heads/main non-fast-forward\n...
//
// After one pass the first line reads `000eunpack ok`, which fails the
// `unpack ok` check below and makes this return None — i.e. "no report",
// which the caller treats as inconclusive and keeps every declared ref.
// The per-ref gate would then be inert for exactly the pushes it exists to
// filter. The second pass above already removed the inner pkt-line
// framing where present (and required its flush); it is a no-op
// on the single-framed shape, because plain report text does not begin
// with four hex digits.
let lines: Vec<&str> = stripped.lines().collect();
if lines.is_empty() {
return None;
}
// First non-empty line is "unpack ok" or "unpack fail".
let unpack_line = lines.iter().find(|l| !l.is_empty())?;
let unpack_ok = if unpack_line.starts_with("unpack ok") {
true
} else if unpack_line.starts_with("unpack fail") {
false
} else {
return None;
};
let mut results = Vec::new();
for line in &lines[1..] {
let line = line.trim();
if line.is_empty() {
break;
}
if let Some(rest) = line.strip_prefix("ok ") {
results.push((rest.to_string(), true));
} else if let Some(rest) = line.strip_prefix("ng ") {
// "ng <refname> <reason>" — skip the reason
let ref_name = rest.split_whitespace().next()?.to_string();
results.push((ref_name, false));
}
}
Some((unpack_ok, results))
}
/// Strip git sideband framing from a pkt-line encoded output.
/// Sideband-encoded lines start with a 4-hex-digit length, then a
/// channel byte (0x01=stdout, 0x02=stderr), then payload. Returns the
/// decoded payload plus whether the stream saw its terminating flush
/// packet, or `None` if the framing is malformed. With `require_flush`,
/// a stream that never sees its flush is also `None` — the caller uses
/// this for the level whose completion marker is authoritative (see
/// [`parse_report_status`]).
fn strip_sideband(text: &str, require_flush: bool) -> Option<(String, bool)> {
let mut output = String::new();
let mut pos = 0;
let bytes = text.as_bytes();
let mut saw_flush = false;
loop {
if pos == bytes.len() {
break;
}
if pos + 4 > bytes.len() {
// Truncated framing mid-header: fail closed.
return None;
}
let len_str = std::str::from_utf8(&bytes[pos..pos + 4]).ok()?;
let len = usize::from_str_radix(len_str, 16).ok()?;
if len == 0 {
// Flush packet — end of sideband stream
saw_flush = true;
break;
}
if len < 4 || pos + len > bytes.len() {
// Truncated pkt-line extends past EOF: incomplete framing.
return None;
}
let pkt_data = std::str::from_utf8(&bytes[pos + 4..pos + len]).ok()?;
pos += len;
// Sideband: first byte is channel (1=stdout, 2=stderr)
if let Some(payload) = pkt_data.strip_prefix('\x01') {
output.push_str(payload);
} else if pkt_data.starts_with('\x02') {
// stderr — skip (git error messages)
} else {
// Not sideband encoded — pass through
output.push_str(pkt_data);
}
}
if require_flush && !saw_flush {
// Truncated at a packet boundary: every packet parsed, but the
// stream never completed. Indistinguishable from a report git was
// killed mid-write, so no authority.
return None;
}
if output.is_empty() {
None
} else {
Some((output, saw_flush))
}
}
/// Sends SIGTERM to a child's whole process group on drop, unless disarmed first.
///
/// A served `git upload-pack`/`receive-pack` forks helpers such as `pack-objects`.
@@ -578,6 +792,121 @@ async fn drive_git_child(
Ok((out, admission))
}
/// Like [`drive_git_child`], but returns stdout and stderr even on a
/// non-zero exit status. Used by [`receive_pack_raw`] so the caller
/// can parse the report-status output from a failed `git-receive-pack`.
async fn drive_git_child_raw(
mut command: Command,
input: Bytes,
timeout: Duration,
what: &str,
admission: Option<AdmissionGuard>,
) -> Result<(
Vec<u8>,
Vec<u8>,
std::process::ExitStatus,
Option<AdmissionGuard>,
)> {
command
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
#[cfg(unix)]
command.process_group(0);
let mut child = command.spawn()?;
let mut stdin = child.stdin.take();
let mut stdout = child.stdout.take().context("git stdout was not piped")?;
let mut stderr = child.stderr.take().context("git stderr was not piped")?;
#[cfg(unix)]
let pgid = child.id().map(|id| id as i32);
#[cfg(unix)]
let mut group_guard = KillGroupOnDrop {
child: Some(child),
pgid,
admission,
};
let mut out = Vec::new();
let mut err = Vec::new();
let interact = async {
let write = async {
match stdin.take() {
Some(mut s) => s.write_all(&input).await,
None => Ok(()),
}
};
#[cfg(unix)]
let child_ref = group_guard.child_mut();
#[cfg(not(unix))]
let child_ref = &mut child;
let (write_result, r_out, r_err, status) = tokio::join!(
write,
stdout.read_to_end(&mut out),
stderr.read_to_end(&mut err),
child_ref.wait(),
);
r_out?;
r_err?;
Ok::<_, anyhow::Error>((write_result, status?))
};
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status, admission) = match timed {
Ok(result) => {
#[cfg(unix)]
let admission = group_guard.disarm();
let (write_result, status) = result?;
(write_result, status, admission)
}
Err(_elapsed) => {
#[cfg(unix)]
{
reap_group_on_timeout(group_guard.child_mut()).await;
drop(group_guard.disarm());
}
#[cfg(not(unix))]
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
drop(admission);
}
return Err(GitServiceTimeout.into());
}
};
// Prefer the child's collected answer over a stdin-write error: an early
// rejection (unpack failure, command-level deny) makes git print a complete
// report-status and exit WITHOUT draining stdin, so a body larger than the
// pipe buffer EPIPEs the write while the answer is already on stdout. The
// child is reaped (the join above waited on it), so `out`/`err`/`status`
// are complete and authoritative — returning Err here would discard them:
// `receive_pack_raw_with_reflog` would propagate the write error and the
// handler would mark every prepared row `uncertain` instead of driving
// `cancelled` per the report. That durable fate is the effect: on a
// non-zero exit the client still gets `Err(AppError::Git)` (the handler
// maps the failed run itself, not this write), while on exit 0 an error
// here would answer a push git actually landed with a server error. The
// non-raw `drive_git_child` orders the same way (exit status and git's own
// failure first, stdin-write error last — pinned by
// `run_git_service_surfaces_git_stderr_over_a_stdin_epipe`), but it must
// fail on non-zero because its callers do not parse output from a failed
// child; the raw driver exists precisely to parse that output.
if let Err(e) = &write_result {
tracing::warn!(
err = %e,
what = %what,
"git stdin write failed after child exit; surfacing collected output"
);
}
Ok((out, err, status, admission))
}
/// Map a validated service name to its git subcommand. Only the two fixed
/// subcommands can come out, so an unrecognized service can never reach git's argv.
fn service_to_command(service: &str) -> Result<&'static str> {
@@ -868,6 +1197,86 @@ mod tests {
use std::process::Command;
use tempfile::TempDir;
/// Byte-for-byte capture from `git receive-pack` 2.50.1 rejecting one ref of
/// a two-ref push, with side-band-64k negotiated — what `git push` over
/// smart HTTP actually sends. The report is framed twice: the outer
/// side-band envelope, then the report-status stream's own pkt-lines.
///
/// This fixture is the point of the test. A hand-written single-framed
/// string parses fine with only one strip pass, so a fixture invented to
/// match the parser hides the exact case the parser is for.
const REAL_SIDEBAND_REPORT: &[u8] =
b"0057\x01000eunpack ok\n0028ng refs/heads/main non-fast-forward\n0018ok refs/heads/third\n0000";
#[test]
fn a_real_double_framed_sideband_report_is_read_not_treated_as_absent() {
let (unpack_ok, refs) =
parse_report_status(REAL_SIDEBAND_REPORT).expect("a real git report must parse");
assert!(unpack_ok, "unpack line must be read through both frames");
assert_eq!(
refs,
vec![
("refs/heads/main".to_string(), false),
("refs/heads/third".to_string(), true),
],
"the rejected ref must be distinguished from the accepted one"
);
}
/// The single-framed shape must keep working: the second pass has to be a
/// no-op there, not a corruption. Plain report text does not begin with
/// four hex digits, which is what makes that safe.
#[test]
fn a_single_framed_report_still_parses_after_the_second_pass() {
// Framed programmatically rather than by a hand-counted hex prefix: a
// wrong length makes the parser return None, which would look exactly
// like the regression this pair of tests exists to catch.
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let (unpack_ok, refs) =
parse_report_status(single.as_bytes()).expect("single-framed must parse");
assert!(unpack_ok);
assert_eq!(refs, vec![("refs/heads/x".to_string(), true)]);
}
#[test]
fn truncated_mid_packet_report_is_indeterminate_not_partial() {
// Truncate a valid single-framed report mid-packet (overrun):
// must return None so the handler persists the whole request as
// indeterminate instead of committing a partial ok set.
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let truncated = &single.as_bytes()[..single.len() - 6];
assert!(
parse_report_status(truncated).is_none(),
"mid-packet truncation must be indeterminate"
);
}
/// Truncation exactly at a packet boundary (every packet complete, but
/// the outer flush never arrived — e.g. git killed after the last `ok`
/// line) must also be indeterminate: the payload is complete but the
/// frame is not, and framing completeness is part of authority. The
/// mid-packet sibling above cannot catch this; only the flush gate can.
/// Removing the `require_flush` enforcement returns `Some` and turns
/// this red.
#[test]
fn truncated_at_packet_boundary_report_is_indeterminate_not_partial() {
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let no_flush = &single.as_bytes()[..single.len() - 4];
assert!(
parse_report_status(no_flush).is_none(),
"a report missing its outer flush must be indeterminate, \
even with every packet intact"
);
// Sanity: the untruncated fixture still parses (flush present).
assert!(
parse_report_status(single.as_bytes()).is_some(),
"the same bytes with the flush must parse"
);
}
#[test]
fn service_to_command_maps_allowlist_and_rejects_passthrough() {
assert_eq!(
@@ -2641,6 +3050,62 @@ mod tests {
);
}
// The raw driver's counterpart: a rejection-shaped early exit where git
// prints a COMPLETE report-status and exits without draining stdin, so a
// body larger than the pipe buffer (~64 KiB) EPIPEs the write while the
// answer is already on stdout. `receive_pack_raw` must surface that report
// (Ok + stdout + exit status), not discard it as a stdin-write error — an
// Err here loses parseable output: `receive_pack_raw_with_reflog`
// propagates it and the handler marks every prepared row `uncertain`
// instead of driving `cancelled` from the report (the durable fate is the
// effect; the client-visible status on a failed run is `AppError::Git`
// either way). Goes RED if the stdin-write check returns Err after the
// join. Both exit codes are pinned: receive-pack rejections and successes
// both carry reports the handler must parse.
#[cfg(unix)]
#[tokio::test]
async fn receive_pack_raw_surfaces_report_status_over_a_stdin_epipe() {
for exit_code in [0, 1] {
let tmp = tempfile::TempDir::new().unwrap();
std::fs::write(tmp.path().join("report.bin"), REAL_SIDEBAND_REPORT).unwrap();
// Never reads stdin: the pending body blocks the pipe, then EPIPEs
// once the fake exits. `cat` delivers the report first.
let git_bin = write_fake_git(
tmp.path(),
&format!("#!/bin/sh\ncat \"$(dirname \"$0\")/report.bin\"\nexit {exit_code}\n"),
);
let body = Bytes::from(vec![0u8; 256 * 1024]);
let (out, ok) = receive_pack_raw(
git_bin.to_str().unwrap(),
tmp.path(),
body,
Duration::from_secs(60),
None,
)
.await
.expect(
"a report-status printed before exit must not be discarded as a \
stdin-write error",
);
assert_eq!(
ok,
exit_code == 0,
"the exit status rides the Ok return for both report shapes"
);
let (unpack_ok, results) = parse_report_status(&out)
.expect("git's report survives the EPIPE and stays parseable");
assert!(unpack_ok, "unpack ok parses");
assert_eq!(
results,
vec![
("refs/heads/main".to_string(), false),
("refs/heads/third".to_string(), true),
],
"the per-ref report reaches the handler instead of an HTTP error"
);
}
}
// ── #192 F1: response_served_pack detector (real git output) ────────────
// A completing request (want+flush+done) yields NAK + a real packfile;
File diff suppressed because it is too large. Load diff
+104 -5
View File
@@ -333,6 +333,17 @@ pub(crate) fn run_bounded_git(
/// Full peeling is why this is not `for-each-ref %(*objecttype)`, which
/// dereferences only one tag level and so misclassifies a tag-of-a-tag-of-a-
/// commit as a non-commit.
///
/// Internal node-managed refs are exempt from this check since they point
/// to blobs by design, not repository content: request markers
/// (`refs/gitlawb/requests/*`, per-push marker blobs) and issue records
/// (`refs/gitlawb/issues/*`, JSON blobs). The exemption lists exactly
/// those two namespaces: any other `refs/gitlawb/*` ref or non-commit
/// content ref still fails closed. Pushes into `refs/gitlawb/*` are
/// refused at the push edge, so no new bypass ref can appear; the
/// visibility logic only cares about content refs, not these two
/// internal metadata namespaces whose binding and advertisement hiding
/// are verified on the push path.
fn assert_all_refs_are_commits(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<()> {
let refs_out = run_bounded_git(
git_bin,
@@ -351,12 +362,29 @@ fn assert_all_refs_are_commits(repo_path: &Path, git_bin: &str, deadline: Instan
return Ok(());
}
// Peel every ref in one `git cat-file --batch-check` pass: one `<refname>^{}`
// Filter out only the two node-managed blob namespaces before checking
// object types. Scoped to `refs/gitlawb/requests/` and
// `refs/gitlawb/issues/` — not all of `refs/gitlawb/*` — so any
// other internal or content ref pointing at a non-commit still
// fails closed.
let content_refs: Vec<&str> = refnames
.iter()
.filter(|r| {
!r.starts_with("refs/gitlawb/requests/") && !r.starts_with("refs/gitlawb/issues/")
})
.copied()
.collect();
if content_refs.is_empty() {
return Ok(());
}
// Peel every content ref in one `git cat-file --batch-check` pass: one `<refname>^{}`
// query per line, one output line per input line, in order. cat-file echoes the
// full query on a `<query> missing` line, so output scales with refname length;
// run_bounded_git drains stdout concurrently with the stdin write, so the pipe
// cannot deadlock, and the whole peel is bounded by the shared walk deadline.
let queries = refnames
let queries = content_refs
.iter()
.map(|r| format!("{r}^{{}}"))
.collect::<Vec<_>>()
@@ -372,15 +400,15 @@ fn assert_all_refs_are_commits(repo_path: &Path, git_bin: &str, deadline: Instan
let peel_stdout = String::from_utf8_lossy(&peel_out);
let types: Vec<&str> = peel_stdout.lines().map(str::trim).collect();
// A short read means at least one ref went unclassified — fail closed.
if types.len() != refnames.len() {
if types.len() != content_refs.len() {
anyhow::bail!(
"git cat-file returned {} lines for {} refs; \
refusing to produce a partial (under-withheld) set",
types.len(),
refnames.len()
content_refs.len()
);
}
for (refname, kind) in refnames.iter().zip(types.iter()) {
for (refname, kind) in content_refs.iter().zip(types.iter()) {
// git emits `<query> missing` (not the objecttype) when the peel target
// is absent; the status word is the last token.
if kind.split_ascii_whitespace().last() == Some("missing") {
@@ -3552,4 +3580,75 @@ esac\n";
"a blob also reachable via an allowed path must not be withheld"
);
}
/// A planted non-exempt `refs/gitlawb/*` blob ref fails the commit walk
/// closed, while the two exempt namespaces (`requests/`, `issues/`) pass.
/// The exemption narrowing removed a serving path, so this test pins the
/// denial: widening the exemption back to all of `refs/gitlawb/*` turns
/// the expect_err red. Real git, scratch bare repo.
#[test]
fn planted_non_exempt_gitlawb_blob_ref_fails_closed() {
use std::time::{Duration, Instant};
let td = TempDir::new().unwrap();
let bare = td.path().join("repo.git");
std::fs::create_dir_all(&bare).unwrap();
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&bare)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
let run_out = |args: &[&str]| {
let out = Command::new("git")
.args(args)
.current_dir(&bare)
.output()
.unwrap();
assert!(out.status.success(), "git {args:?} failed");
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
run(&["init", "-q", "--bare", "."]);
// A blob object to point refs at (node markers and issues are blobs).
std::fs::write(bare.join("marker-body"), b"marker\n").unwrap();
let blob = run_out(&["hash-object", "-w", "marker-body"]);
let check = |want_ok: bool, why: &str| {
let result =
assert_all_refs_are_commits(&bare, "git", Instant::now() + Duration::from_secs(30));
if want_ok {
result.expect(why);
} else {
let err = result.expect_err(why);
assert!(
err.to_string().contains("refs/gitlawb/evil"),
"the denial must name the offending ref, got: {err:#}"
);
}
};
// The bare `refs/gitlawb` name fails closed too (checked first: git
// forbids a bare ref coexisting with the subtree below, D/F conflict).
run(&["update-ref", "refs/gitlawb", &blob]);
let result =
assert_all_refs_are_commits(&bare, "git", Instant::now() + Duration::from_secs(30));
let err = result.expect_err("bare refs/gitlawb blob ref must fail closed");
assert!(
err.to_string().contains("refs/gitlawb"),
"the denial must name the offending ref, got: {err:#}"
);
run(&["update-ref", "-d", "refs/gitlawb", "--no-deref"]);
// Exempt namespaces with blob targets pass (node-managed by design).
run(&["update-ref", "refs/gitlawb/requests/r1", &blob]);
run(&["update-ref", "refs/gitlawb/issues/i1", &blob]);
check(true, "requests//issues/ blob refs are exempt and must pass");
// A non-exempt blob ref fails closed.
run(&["update-ref", "refs/gitlawb/evil", &blob]);
check(false, "a non-exempt refs/gitlawb blob ref must fail closed");
// Cleanup proves the denial is about that ref, not residue.
run(&["update-ref", "-d", "refs/gitlawb/evil", "--no-deref"]);
check(true, "removing the evil ref must restore the walk");
}
}
+684 -2
View File
@@ -3,7 +3,7 @@ pub mod query;
pub mod subscription;
pub mod types;
use async_graphql::Schema;
use async_graphql::{Schema, SchemaBuilder};
use std::sync::Arc;
use crate::db::Db;
@@ -14,6 +14,22 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
// Keep ordinary schema discovery and application queries usable while
// bounding validation work and the number of resolver selections per request.
const GRAPHQL_MAX_COMPLEXITY: usize = 400;
// The current public schema is shallow; this leaves headroom for composed
// clients and the canonical getIntrospectionQuery without allowing
// recursively nested documents to grow unchecked.
const GRAPHQL_MAX_DEPTH: usize = 14;
fn apply_query_limits<Query, Mutation, Subscription>(
builder: SchemaBuilder<Query, Mutation, Subscription>,
) -> SchemaBuilder<Query, Mutation, Subscription> {
builder
.limit_complexity(GRAPHQL_MAX_COMPLEXITY)
.limit_depth(GRAPHQL_MAX_DEPTH)
}
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
@@ -81,7 +97,7 @@ pub fn build_schema(
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>,
) -> GitlawbSchema {
Schema::build(QueryRoot, MutationRoot, SubscriptionRoot)
apply_query_limits(Schema::build(QueryRoot, MutationRoot, SubscriptionRoot))
.data(db)
.data(ref_update_tx)
.data(task_event_tx)
@@ -91,6 +107,10 @@ pub fn build_schema(
#[cfg(test)]
mod tests {
use super::*;
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
use crate::db::{MAX_VISIBLE_REPO_PAGE_SIZE, MAX_VISIBLE_TASK_PAGE_SIZE};
use async_graphql::{EmptyMutation, EmptySubscription, Object, Value};
use std::sync::atomic::{AtomicUsize, Ordering};
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
@@ -160,6 +180,668 @@ mod tests {
assert!(!err.message.contains("failed to open"));
}
fn production_test_schema() -> GitlawbSchema {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
production_test_schema_with_db(Arc::new(Db::for_testing(pool)))
}
fn production_test_schema_with_db(db: Arc<Db>) -> GitlawbSchema {
build_schema(
db,
tokio::sync::broadcast::channel(1).0,
tokio::sync::broadcast::channel(1).0,
)
}
#[tokio::test]
async fn ref_update_subscription_alias_budget() {
assert_subscription_alias_budget("refUpdates { repo }").await;
}
#[tokio::test]
async fn task_event_subscription_alias_budget() {
assert_subscription_alias_budget("taskEvents { taskId }").await;
}
async fn assert_subscription_alias_budget(field: &str) {
use futures::StreamExt;
use std::time::Duration;
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(
Arc::new(Db::for_testing(pool)),
ref_tx.clone(),
task_tx.clone(),
);
for count in [7, 8] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let query = format!("subscription {{ {fields} }}");
let mut responses = Box::pin(schema.execute_stream(query));
if count == 8 {
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.expect("over-budget subscriptions must reject before waiting for events")
.expect("the validation error must be returned");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
continue;
}
// Poll the accepted operation far enough to register its receivers.
assert!(
tokio::time::timeout(Duration::from_millis(30), responses.next())
.await
.is_err()
);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 7);
let _ = ref_tx.send(RefUpdateBroadcast {
repo: "owner/repo".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "1".repeat(40),
pusher_did: "did:key:test".into(),
node_did: "did:key:node".into(),
timestamp: "2026-01-01T00:00:00Z".into(),
owner_did: "did:key:test".into(),
});
let _ = task_tx.send(TaskEventBroadcast {
task_id: "task".into(),
old_status: "pending".into(),
new_status: "claimed".into(),
by_did: "did:key:test".into(),
at: "2026-01-01T00:00:00Z".into(),
});
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.unwrap()
.unwrap();
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_ne!(response.data, Value::Null);
drop(responses);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
}
}
#[sqlx::test]
async fn production_mutation_budget_accepts_seven_and_rejects_eight(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let caller = "did:key:reviewer";
let now = chrono::Utc::now().to_rfc3339();
for index in 0..8 {
db.create_task(&crate::db::AgentTask {
id: format!("task-{index}"),
repo_id: None,
kind: "test".into(),
status: "pending".into(),
delegator_did: caller.into(),
assignee_did: None,
capability: "test".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = production_test_schema_with_db(db.clone());
for count in [8, 7] {
let fields = (0..count)
.map(|n| {
format!("r{n}: claimTask(id: \"task-{n}\", assigneeDid: \"{caller}\") {{ id }}")
})
.collect::<Vec<_>>()
.join(" ");
let response = schema
.execute(
async_graphql::Request::new(format!("mutation {{ {fields} }}"))
.data(crate::auth::AuthenticatedDid(caller.into())),
)
.await;
if count == 8 {
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(
db.list_tasks(Some("pending"), None, 8).await.unwrap().len(),
8
);
continue;
}
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
7
);
assert_eq!(
db.list_tasks(Some("claimed"), Some(caller), 8)
.await
.unwrap()
.len(),
7
);
assert_eq!(
db.get_task("task-7").await.unwrap().unwrap().status,
"pending"
);
}
}
#[sqlx::test]
async fn production_list_cost_scales_with_the_same_alias_count(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
// Bounds are per-root named constants so the cases move with the
// documented maximums instead of going stale as literals.
for (field, selection, max) in [
("refUpdates", "repo", MAX_VISIBLE_REF_UPDATES),
("tasks", "id", MAX_VISIBLE_TASK_PAGE_SIZE),
(
"reposPage",
"nodes { name }",
MAX_VISIBLE_REPO_PAGE_SIZE as i64,
),
] {
// Two aliases at the maximum always exceed the budget.
let fields = (0..2)
.map(|n| format!("r{n}: {field}(limit: {max}) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
// At limit 1 two aliases fit for the row-priced roots; reposPage
// prices the full corpus scan, so a single alias is the most that
// fits under the budget.
let aliases = if field == "reposPage" { 1 } else { 2 };
let fields = (0..aliases)
.map(|n| format!("r{n}: {field}(limit: 1) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{field}: {:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
aliases
);
}
}
#[tokio::test]
async fn production_limits_reject_mutation_aliases_and_large_lists() {
let schema = production_test_schema();
// Limits are derived from the MAX_* constants so the boundaries move
// with the documented maximums instead of going stale as literals.
for (prefix, count, field) in [
("mutation", 8, "claimTask(id: \"missing\", assigneeDid: \"did:key:test\") { id }".to_owned()),
("mutation", 8, "createTask(delegatorDid: \"did:key:test\", input: { kind: \"test\", capability: \"test\" }) { id }".to_owned()),
("mutation", 8, "completeTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("mutation", 8, "failTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("query", 8, "task(id: \"missing\") { id }".to_owned()),
("query", 2, format!("refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }}")),
("query", 2, format!("tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id }}")),
("query", 2, format!("reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ nodes {{ name }} }}")),
("query", 8, "reposPage(limit: 1) { nodes { name } }".to_owned()),
] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{prefix} {{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
}
#[sqlx::test]
async fn production_repos_page_serves_the_documented_maximum(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
for selection in [
"nodes { name }",
"nodes { name ownerDid } hasNextPage endCursor",
] {
let response = schema
.execute(format!(
"{{ reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ {selection} }} }}"
))
.await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response.data.into_json().unwrap()["reposPage"]["nodes"],
serde_json::json!([])
);
}
}
#[tokio::test]
async fn seven_root_aliases_are_accepted() {
struct Root(Arc<AtomicUsize>);
#[Object]
impl Root {
#[graphql(complexity = "50 + child_complexity")]
async fn repos(&self) -> Vec<Nested> {
self.0.fetch_add(1, Ordering::Relaxed);
vec![Nested]
}
}
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
Root(calls.clone()),
EmptyMutation,
EmptySubscription,
))
.finish();
let fields = (0..7)
.map(|n| format!("r{n}: repos {{ value }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(calls.load(Ordering::Relaxed), 7);
}
#[tokio::test]
async fn expensive_root_aliases_are_rejected_before_database_access() {
// Use the production builder with a lazy pool: rejection must precede DB access.
let schema = production_test_schema();
let fields = (0..8)
.map(|n| format!("r{n}: repos {{ name }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.data, Value::Null);
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ordinary_schema_introspection_remains_available() {
let schema = production_test_schema();
let canonical_introspection = r#"
query IntrospectionQuery {
__schema {
queryType { name }
mutationType { name }
subscriptionType { name }
types {
...FullType
}
directives {
name
description
locations
args {
...InputValue
}
}
}
}
fragment FullType on __Type {
kind
name
description
fields(includeDeprecated: true) {
name
description
args {
...InputValue
}
type {
...TypeRef
}
isDeprecated
deprecationReason
}
inputFields {
...InputValue
}
interfaces {
...TypeRef
}
enumValues(includeDeprecated: true) {
name
description
isDeprecated
deprecationReason
}
possibleTypes {
...TypeRef
}
}
fragment InputValue on __InputValue {
name
description
type { ...TypeRef }
defaultValue
}
fragment TypeRef on __Type {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
}
}
}
}
}
}
}
}
"#;
let response = schema.execute(canonical_introspection).await;
assert!(
response.errors.is_empty(),
"canonical introspection query failed against production schema: {:?}",
response.errors
);
assert_ne!(response.data, Value::Null);
}
#[tokio::test]
async fn ref_updates_and_tasks_single_alias_max_limit_complexity_contract() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
pool.close().await;
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(Arc::new(Db::for_testing(pool)), ref_tx, task_tx);
// refUpdates(limit: MAX_VISIBLE_REF_UPDATES):
// Single-field selection has child_complexity 1 -> cost 100 + 200 + 1 = 301 <= 400.
// It passes validation and reaches the resolver (which yields db error on lazy pool).
let single_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }} }}"
))
.await;
assert!(
!single_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field refUpdates at max limit must pass complexity validation: {:?}",
single_ref.errors
);
assert!(
single_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field refUpdates at max limit must reach the resolver: {:?}",
single_ref.errors
);
// Two-field selection has child_complexity 2 -> cost 100 + 200 + 2 = 302 <= 400.
// The documented max-200 request stays reachable with multiple fields
// (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo refName }} }}"
))
.await;
assert!(
!multi_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field refUpdates at max limit must pass complexity validation: {:?}",
multi_ref.errors
);
assert!(
multi_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field refUpdates at max limit must reach the resolver: {:?}",
multi_ref.errors
);
// tasks(limit: MAX_VISIBLE_TASK_PAGE_SIZE):
// Single-field selection has child_complexity 1, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 1 <= 400.
// It passes validation and reaches the resolver.
let single_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id }} }}"
))
.await;
assert!(
!single_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field tasks at the maximum page size must pass complexity validation: {:?}",
single_task.errors
);
assert!(
single_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field tasks at the maximum page size must reach the resolver: {:?}",
single_task.errors
);
// Two-field selection has child_complexity 2, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 2 <= 400.
// A request at the documented maximum stays reachable with multiple
// fields (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id status }} }}"
))
.await;
assert!(
!multi_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field tasks at the maximum page size must pass complexity validation: {:?}",
multi_task.errors
);
assert!(
multi_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field tasks at the maximum page size must reach the resolver: {:?}",
multi_task.errors
);
}
#[tokio::test]
async fn two_repos_aliases_exceed_the_budget() {
// `repos` is priced as the full page it scans
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child), so two `repos { name }`
// aliases cost 2 * 251 = 502 > 400 and the document is rejected before
// any resolver runs. If the price reverts to `50 + child_complexity`
// the aliases cost 2 * 51 = 102, validation passes, and the failure
// mode changes (resolver DB errors on the lazy pool): this test goes
// red on revert.
let schema = production_test_schema();
let response = schema
.execute("{ r0: repos { name } r1: repos { name } }")
.await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn seven_repos_page_aliases_exceed_the_budget() {
// `reposPage` is priced as the full corpus scan
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity), so seven
// `reposPage(limit: 1) { nodes { name } }` aliases cost
// 7 * (50 + 200 + 2) = 1764 > 400 and the document is rejected
// before any resolver runs. Under the previous row-priced formula
// the same aliases cost 7 * (50 + 1 + 2) = 371, validation passes,
// and the failure mode changes (resolver DB errors on the lazy
// pool): this test goes red on revert. Eight aliases are rejected
// under both formulas, so seven is the load-bearing boundary.
let schema = production_test_schema();
let fields = (0..7)
.map(|n| format!("r{n}: reposPage(limit: 1) {{ nodes {{ name }} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ref_updates_fixed_base_is_charged() {
// The 100 base in the refUpdates complexity formula charges for the
// collector's fixed work (loading the deduped repo set, the
// quarantined set, and the visibility rules, then walking up to 16
// keyset pages) even at limit: 1. Four aliases at limit: 1 cost
// 4 * (100 + 1 + 1) = 408 > 400 and must be rejected. If the base
// disappears, the cost drops to 4 * (1 + 1) = 8 and this test fails.
let schema = production_test_schema();
let fields = (0..4)
.map(|n| format!("r{n}: refUpdates(limit: 1) {{ repo }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[derive(Clone, Copy)]
struct Nested;
#[Object]
impl Nested {
async fn child(&self) -> Nested {
Nested
}
async fn value(&self) -> i32 {
1
}
}
struct CountingQuery(Arc<AtomicUsize>);
#[Object]
impl CountingQuery {
async fn nested(&self) -> Nested {
self.0.fetch_add(1, Ordering::Relaxed);
Nested
}
}
#[tokio::test]
async fn query_depth_limit_accepts_fourteen_and_rejects_fifteen() {
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
CountingQuery(Arc::clone(&calls)),
EmptyMutation,
EmptySubscription,
))
.finish();
let query_at_depth = |depth: usize| {
let selection = (2..depth).fold("value".to_string(), |selection, _| {
format!("child {{ {selection} }}")
});
format!("{{ nested {{ {selection} }} }}")
};
let accepted = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH)).await;
assert!(
accepted.errors.is_empty(),
"depth {GRAPHQL_MAX_DEPTH} should be accepted: {:?}",
accepted.errors
);
assert_eq!(calls.swap(0, Ordering::Relaxed), 1);
let rejected = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH + 1)).await;
assert_eq!(rejected.data, Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
assert_eq!(calls.load(Ordering::Relaxed), 0);
}
#[tokio::test]
async fn production_schema_enforces_depth_limit() {
let schema = production_test_schema();
// Depth 14 is accepted through the production build_schema path.
let of_types_14 = (0..9).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query_14 =
format!("{{ __schema {{ types {{ fields {{ type {{ {of_types_14} }} }} }} }} }}");
let accepted = schema.execute(&query_14).await;
assert!(accepted.errors.is_empty(), "{:?}", accepted.errors);
// Construct an introspection query of depth 15 using __schema.
// 1: __schema
// 2: types
// 3: fields
// 4: type
// 5..14: ofType (10 times)
// 15: name
// Total depth = 15 > GRAPHQL_MAX_DEPTH (14).
let of_types = (0..10).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query = format!("{{ __schema {{ types {{ fields {{ type {{ {of_types} }} }} }} }} }}");
let rejected = schema.execute(&query).await;
assert_eq!(rejected.data, async_graphql::Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
+419 -7
View File
@@ -22,6 +22,7 @@ pub struct MutationRoot;
#[Object]
impl MutationRoot {
#[graphql(complexity = "50 + child_complexity")]
async fn create_task(
&self,
ctx: &Context<'_>,
@@ -36,6 +37,33 @@ impl MutationRoot {
}
let delegator_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
// #496: same write gate as the REST handler — a repo_id naming a
// hosted, non-quarantined repo admits tasks only from its owner, so a
// stranger cannot file under a foreign repo id. An id naming no hosted
// repo (unknown, or quarantined — hidden as if it did not exist) is
// stored verbatim as an opaque label; the gate does not bind an
// unresolved id to a repo. Task reads are caller-independent on this
// base (the tasks resolver is not repo-scoped), before and after this
// change. As in tasks.rs, the success-vs-denial split between a hosted
// repo id and an unknown one is itself a one-bit oracle — a stranger
// holding a private repo's id learns it is hosted here — but denying
// foreign repos requires drawing exactly that line.
if let Some(repo_id) = input.repo_id.as_deref() {
let record = db
.get_repo_by_id(repo_id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if let Some(record) = record {
let quarantined = db
.is_repo_quarantined(&record.id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if !quarantined {
crate::api::require_repo_owner(&record, caller)
.map_err(crate::graphql::graphql_app_err)?;
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -52,12 +80,17 @@ impl MutationRoot {
updated_at: now,
deadline: input.deadline,
};
db.create_task(&task)
// Return the stored row, not the pre-insert input: create_task
// normalizes the assignee (whitespace-only becomes NULL/open), and a
// response that shows the raw input lies about the reservation.
let stored = db
.create_task(&task)
.await
.map_err(crate::graphql::graphql_db_err)?;
Ok(AgentTaskType::from(task))
Ok(AgentTaskType::from(stored))
}
#[graphql(complexity = "50 + child_complexity")]
async fn claim_task(
&self,
ctx: &Context<'_>,
@@ -77,16 +110,25 @@ impl MutationRoot {
.claim_task(&id, &assignee_did)
.await
.map_err(crate::graphql::graphql_db_err)?;
// Report the stored assignee form like REST, so subscribers see the
// same identity whichever API surface ran the transition.
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(assignee_did.as_str())
.to_string();
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did: assignee_did,
by_did,
at: Utc::now().to_rfc3339(),
});
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn complete_task(
&self,
ctx: &Context<'_>,
@@ -110,15 +152,24 @@ impl MutationRoot {
.await
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
return Err(async_graphql::Error::new(
"only the task assignee can complete it",
));
}
let task = db
.finish_task(&id, "completed", input.result.as_deref())
.finish_task(&id, "completed", input.result.as_deref(), caller)
.await
.map_err(crate::graphql::graphql_db_err)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(by_did.as_str())
.to_string();
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -129,6 +180,7 @@ impl MutationRoot {
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn fail_task(
&self,
ctx: &Context<'_>,
@@ -151,16 +203,25 @@ impl MutationRoot {
.await
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
return Err(async_graphql::Error::new(
"only the task assignee can fail it",
));
}
let reason = input.reason.unwrap_or_default();
let task = db
.finish_task(&id, "failed", Some(&reason))
.finish_task(&id, "failed", Some(&reason), caller)
.await
.map_err(crate::graphql::graphql_db_err)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(by_did.as_str())
.to_string();
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -186,6 +247,22 @@ mod tests {
.join("; ")
}
/// The `id` scalar returned by a successful `createTask` mutation.
fn created_id(resp: &Response) -> String {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::Object(task) = obj.get("createTask").expect("createTask key")
else {
panic!("createTask not an object");
};
let async_graphql::Value::String(id) = task.get("id").expect("id key") else {
panic!("id not a string");
};
id.clone()
}
/// N2: every GraphQL mutation requires a verified signer and binds the acting
/// DID to it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate.
@@ -353,4 +430,339 @@ mod tests {
errors(&resp)
);
}
/// #496 (GraphQL): createTask applies the same repo-ownership gate as the
/// REST handler — a stranger naming a hosted repo is rejected without
/// persisting anything, while the owner files. The quarantined and
/// unknown-id arms are pinned too, including that each accepted write
/// stores the supplied id verbatim, so this copy of the gate cannot regress
/// silently.
#[sqlx::test]
async fn create_task_rejects_foreign_repo_id(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let owner = "did:key:zGQLTASKOWNERAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zGQLTASKSTRANGERBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
)
};
// Stranger signs as themselves (so the signer binding passes) but
// names the victim's repo → rejected by the ownership gate, leaking
// nothing about the repo.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::api::REPO_OWNER_ONLY_MESSAGE),
"a non-owner must not file under a foreign repo id (shared denial text): {errs}"
);
// The denial must not persist: no task row may carry the victim's
// repo id afterwards.
let stored = state
.db
.list_tasks(None, None, 50)
.await
.expect("list tasks");
assert!(
stored
.iter()
.all(|t| t.repo_id.as_deref() != Some(repo.id.as_str())),
"a denied write must not leave a task under the foreign repo id"
);
// The owner files under their own repo id.
let resp = schema
.execute(Request::new(q(owner)).data(AuthenticatedDid(owner.into())))
.await;
assert!(
errors(&resp).is_empty(),
"the owner should file against their repo: {}",
errors(&resp)
);
// Unknown repo ids stay accepted as opaque labels.
let q_unknown = format!(
r#"mutation {{ createTask(delegatorDid: "{stranger}", input: {{ kind: "build", capability: "repo:write", repoId: "no-such-repo-id" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(q_unknown).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"an id naming no hosted repo stays an opaque label: {}",
errors(&resp)
);
// Read back the row: the unresolved id must be stored verbatim, not
// dropped or rewritten to a repo it never named.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some("no-such-repo-id"),
"an unresolved id must be stored verbatim as a label"
);
// A quarantined repo id is treated as unknown (accepted, never a 403
// that would confirm the id names a real repo). This pins the
// `!quarantined` arm on this copy of the gate.
state
.db
.set_repo_quarantine(&repo.id, true)
.await
.expect("quarantine");
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"a quarantined repo id must not 403 (existence oracle): {}",
errors(&resp)
);
// Read back the row: the quarantined id is stored verbatim as a label
// too, so the accepted write is not silently dropped.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some(repo.id.as_str()),
"a quarantined id must be stored verbatim as a label"
);
}
/// #496 (GraphQL): a repo-lookup fault in createTask's gate must stay
/// opaque. Dropping a column `get_repo_by_id` selects faults that lookup;
/// the resolver must return the fixed DB message and leak no schema text
/// (#250), as `create_task_db_error_message_is_opaque` pins for the
/// agent_tasks fault.
#[sqlx::test]
async fn create_task_gate_repo_lookup_fault_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner = "did:key:zGQLTASKFAULTOWNERAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-fault-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-fault-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
sqlx::query("ALTER TABLE repos DROP COLUMN machine_id")
.execute(&pool)
.await
.expect("drop machine_id");
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{owner}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(owner.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"gate sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("machine_id") && !errs.contains("column"),
"schema text leaked through the gate: {errs}"
);
}
/// Pre-assigned GraphQL claimTask must reject a stranger (no UCAN leak)
/// and admit the reserved assignee.
#[sqlx::test]
async fn claim_task_honors_preassigned_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let reserved = "did:key:zGQLRESERVEDAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let thief = "did:key:zGQLTHIEFBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-reserved".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: Some(reserved.into()),
capability: "repo:write".into(),
ucan_token: Some("gql-ucan-secret".into()),
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ claimTask(id: "task-gql-reserved", assigneeDid: "{actor}") {{ id status ucanToken }} }}"#
)
};
let resp = schema
.execute(Request::new(q(thief)).data(AuthenticatedDid(thief.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("reserved"),
"stranger claim must surface reserved deny distinctly: {errs}"
);
assert!(
!errs.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL errors: {errs}"
);
if let Ok(data) = resp.data.into_json() {
let s = data.to_string();
assert!(
!s.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL data on failed claim: {s}"
);
}
// Parity with REST: failed steal must leave the row untouched.
let still = state
.db
.get_task("task-gql-reserved")
.await
.expect("get")
.expect("task exists");
assert_eq!(
still.status, "pending",
"status must stay pending after steal"
);
assert_eq!(
still.assignee_did.as_deref(),
Some(reserved),
"assignee_did must stay reserved after steal"
);
let resp = schema
.execute(Request::new(q(reserved)).data(AuthenticatedDid(reserved.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim: {}",
errors(&resp)
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["claimTask"]["status"], "claimed");
assert_eq!(data["claimTask"]["ucanToken"], "gql-ucan-secret");
}
/// create_task normalizes a whitespace-only assignee to NULL (open task),
/// so the create response must show the stored row, not echo the input: a
/// client trusting the echoed "reservation" would believe a claimable task
/// is reserved. Before the fix the response was built from the pre-insert
/// struct and returned the padded input verbatim.
#[sqlx::test]
async fn create_task_reports_stored_assignee_not_input(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let caller = "did:key:zGQLCREATORAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let schema = state.graphql_schema.as_ref();
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{caller}", input: {{ kind: "build", capability: "repo:write", assigneeDid: " " }}) {{ id assigneeDid }} }}"#
);
let resp = schema
.execute(Request::new(q).data(AuthenticatedDid(caller.into())))
.await;
let data = resp.data.into_json().expect("json data");
assert!(
data["createTask"]["assigneeDid"].is_null(),
"whitespace-only assigneeDid must come back null (stored shape): {data}"
);
}
/// A task seeded with a whitespace-padded assignee must still be
/// completable by that assignee over GraphQL: REST and Db::finish_task
/// trim the stored value before comparing, and the GraphQL gate must do
/// the same or padded reservations finish only through REST.
#[sqlx::test]
async fn graphql_finish_accepts_padded_stored_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let assignee = "did:key:zGQLPADDEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-padded".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORDDDDDDDDDDDDDDDDDDDDDDDDDD".into(),
assignee_did: Some(format!("\t{assignee}\n")),
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let claim = format!(
r#"mutation {{ claimTask(id: "task-gql-padded", assigneeDid: "{assignee}") {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(claim).data(AuthenticatedDid(assignee.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim a padded reservation: {}",
errors(&resp)
);
let finish = format!(
r#"mutation {{ completeTask(id: "task-gql-padded", byDid: "{assignee}", input: {{}}) {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(finish).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.is_empty(),
"assignee must complete a padded reservation: {errs}"
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["completeTask"]["status"], "completed");
}
}
+814 -35
View File
@@ -1,57 +1,154 @@
use async_graphql::{Context, Object, Result};
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
use std::sync::Arc;
use crate::db::Db;
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
use crate::db::{Db, RepoRecord, MAX_VISIBLE_REPO_PAGE_SIZE, MAX_VISIBLE_TASK_PAGE_SIZE};
use super::types::{AgentTaskType, RefUpdateType, RepoType};
use super::types::{AgentTaskType, RefUpdateType, RepoPageType, RepoType};
fn repo_type(repo: RepoRecord) -> RepoType {
RepoType {
name: repo.name,
owner_did: repo.owner_did,
description: repo.description,
default_branch: repo.default_branch,
created_at: repo.created_at.to_rfc3339(),
}
}
fn repo_cursor(repo: &RepoRecord) -> String {
URL_SAFE_NO_PAD.encode(
serde_json::to_vec(&(&repo.owner_did, &repo.name))
.expect("a pair of strings is JSON serializable"),
)
}
fn parse_repo_cursor(cursor: &str) -> Result<(String, String)> {
// A position contains only public response fields, not a database id or
// authority. Every page re-evaluates visibility for the current caller.
if cursor.len() > 4096 {
return Err(async_graphql::Error::new("invalid repository cursor"));
}
let bytes = URL_SAFE_NO_PAD
.decode(cursor)
.map_err(|_| async_graphql::Error::new("invalid repository cursor"))?;
let position: (String, String) = serde_json::from_slice(&bytes)
.map_err(|_| async_graphql::Error::new("invalid repository cursor"))?;
// PostgreSQL text cannot contain NUL. Reject it as malformed input before
// binding either cursor component, rather than reporting a database error.
if position.0.contains('\0') || position.1.contains('\0') {
return Err(async_graphql::Error::new("invalid repository cursor"));
}
Ok(position)
}
pub struct QueryRoot;
#[Object]
impl QueryRoot {
// DB-backed roots carry a base cost so aliases consume the request budget
// even when each alias selects only one inexpensive response field.
// `repos` performs the same full corpus scan as a max-size `reposPage`
// (list_visible_repos_page with MAX_VISIBLE_REPO_PAGE_SIZE + 1), so it
// is priced as the full page it is, not as a cheap alias.
#[graphql(complexity = "50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity")]
/// Complete visible repository list, up to the documented maximum number
/// of entries. Larger lists must use reposPage; this field returns an
/// error instead of truncating silently.
async fn repos(&self, ctx: &Context<'_>) -> Result<Vec<RepoType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
let repos = db
.list_all_repos_deduped()
.await
.map_err(crate::graphql::graphql_db_err)?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
// threaded onto the context by optional_signature; absent = anonymous.
let caller = ctx
.data::<crate::auth::AuthenticatedDid>()
.ok()
.map(|d| d.0.as_str());
let ids: Vec<String> = repos.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
let mut repos = db
.list_visible_repos_page(caller, None, MAX_VISIBLE_REPO_PAGE_SIZE + 1)
.await
.map_err(crate::graphql::graphql_db_err)?;
Ok(repos
.into_iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, caller)
})
.map(|r| RepoType {
name: r.name,
owner_did: r.owner_did,
description: r.description,
default_branch: r.default_branch,
created_at: r.created_at.to_rfc3339(),
})
.collect())
if repos.len() > MAX_VISIBLE_REPO_PAGE_SIZE {
return Err(async_graphql::Error::new(format!(
"repository list exceeds {} entries; use reposPage with limit and after",
MAX_VISIBLE_REPO_PAGE_SIZE
)));
}
// Preserve the legacy activity ordering for complete, small lists.
repos.sort_by_key(|repo| std::cmp::Reverse(repo.updated_at));
Ok(repos.into_iter().map(repo_type).collect())
}
/// Bounded visible repositories ordered by owner and name. Continue with
/// endCursor while hasNextPage is true. Pages are not a database snapshot.
// Every page sorts the full deduplicated corpus under COLLATE "C"
// (list_visible_repos_page documents the accepted O(total repos) bound),
// so the price charges the scan, not the emitted rows: a limit:1 page
// costs the same as a max-size page, the way `repos` is priced.
#[graphql(complexity = "50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity")]
async fn repos_page(
&self,
ctx: &Context<'_>,
#[graphql(
default = 50,
desc = "Page size from 1 to the documented maximum; other values are rejected."
)]
limit: i64,
after: Option<String>,
) -> Result<RepoPageType> {
if !(1..=MAX_VISIBLE_REPO_PAGE_SIZE as i64).contains(&limit) {
return Err(async_graphql::Error::new(format!(
"limit must be between 1 and {}",
MAX_VISIBLE_REPO_PAGE_SIZE
)));
}
let after = after.as_deref().map(parse_repo_cursor).transpose()?;
let caller = ctx
.data::<crate::auth::AuthenticatedDid>()
.ok()
.map(|d| d.0.as_str());
let db = ctx.data_unchecked::<Arc<Db>>();
let mut repos = db
.list_visible_repos_page(
caller,
after
.as_ref()
.map(|(owner, name)| (owner.as_str(), name.as_str())),
limit as usize + 1,
)
.await
.map_err(crate::graphql::graphql_db_err)?;
let has_next_page = repos.len() > limit as usize;
repos.truncate(limit as usize);
let end_cursor = repos.last().map(repo_cursor);
Ok(RepoPageType {
nodes: repos.into_iter().map(repo_type).collect(),
has_next_page,
end_cursor,
})
}
// Complexity is additive with a base reflecting the collector's fixed scan:
// every request loads the full deduped repo set, the quarantined set, and
// the visibility rules, then walks up to max(limit, 2048) event rows past
// withheld ones, regardless of `limit`. A multiplicative price both
// undercharged limit:1 (51 for that fixed work) and made a request at the
// documented maximum unreachable (two fields at max limit cost 450 > 400).
//
// Decision on the collector floor (review #465, 3rd round): the up-to-2048
// walk is bounded constant overhead that does not scale with `limit`, so
// it is priced in the fixed base, not the limit term; the limit term
// prices only the emitted rows. Pricing the walk at face value in the base
// would put every refUpdates query over the 400 budget and make the field
// unusable, so the base prices that fixed scan as a lump sum instead.
#[graphql(
complexity = "100 + (limit.clamp(0, MAX_VISIBLE_REF_UPDATES) as usize) + child_complexity"
)]
async fn ref_updates(
&self,
ctx: &Context<'_>,
repo: Option<String>,
#[graphql(
default = 20,
desc = "Max 200; larger requests return the newest 200 rows (no continuation cursor)."
desc = "Requests above the documented maximum return the newest rows up to that maximum (no continuation cursor)."
)]
limit: i64,
) -> Result<Vec<RefUpdateType>> {
@@ -105,6 +202,14 @@ impl QueryRoot {
Ok(resolved)
}
// Complexity is additive: list_tasks is a single LIMIT-bounded query whose
// cost does not scale with the selected fields, so the price is the base
// plus the clamped limit plus the child fields. A multiplicative price
// made a request at the documented maximum unreachable (two fields at max
// limit cost 450 > 400).
#[graphql(
complexity = "50 + (limit.clamp(0, MAX_VISIBLE_TASK_PAGE_SIZE) as usize) + child_complexity"
)]
async fn tasks(
&self,
ctx: &Context<'_>,
@@ -112,7 +217,7 @@ impl QueryRoot {
assignee_did: Option<String>,
#[graphql(
default = 50,
desc = "Max 200; larger requests are clamped to 200 (no error). Negative values clamp to 0."
desc = "Requests above the documented maximum are clamped to it (no error). Negative values clamp to 0."
)]
limit: i64,
) -> Result<Vec<AgentTaskType>> {
@@ -120,7 +225,7 @@ impl QueryRoot {
// Clamp before SQL: a negative LIMIT is a client fault that Postgres
// rejects with 2201W, which would otherwise trip the opaque DB path
// and write an error-level log on every probe (#250 review).
let limit = limit.clamp(0, 200);
let limit = limit.clamp(0, MAX_VISIBLE_TASK_PAGE_SIZE);
let tasks = db
.list_tasks(status.as_deref(), assignee_did.as_deref(), limit)
.await
@@ -128,6 +233,7 @@ impl QueryRoot {
Ok(tasks.into_iter().map(AgentTaskType::from).collect())
}
#[graphql(complexity = "50 + child_complexity")]
async fn task(&self, ctx: &Context<'_>, id: String) -> Result<Option<AgentTaskType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
let t = db
@@ -141,12 +247,642 @@ impl QueryRoot {
#[cfg(test)]
mod tests {
use crate::db::{Db, ReceivedRefUpdate, RepoRecord};
use base64::Engine;
use chrono::Utc;
use sqlx::PgPool;
use std::sync::Arc;
const OWNER: &str = "did:key:z6MkOwner";
#[sqlx::test]
async fn repos_legacy_rejects_overflow_and_pages_reach_every_visible_repo(pool: PgPool) {
let db = db(pool).await;
let total = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE + 5;
for index in 0..total {
let name = format!("repo-{index:03}");
db.create_repo(&repo(&name, OWNER, &name, true))
.await
.unwrap();
}
// The SQL helper itself must bound materialization, even for a caller
// accidentally requesting an unlimited page.
assert_eq!(
db.list_visible_repos_page(None, None, usize::MAX)
.await
.unwrap()
.len(),
crate::db::MAX_VISIBLE_REPO_PAGE_SIZE + 1
);
let schema = schema(db);
let legacy = anon(&schema, "{ repos { name } }").await;
assert_eq!(legacy.errors.len(), 1);
assert!(legacy.errors[0].message.contains("use reposPage"));
assert_eq!(legacy.data, async_graphql::Value::Null);
let mut cursor = None;
let mut names = Vec::new();
loop {
let response = schema.execute(
async_graphql::Request::new(
"query($after: String) { reposPage(limit: 50, after: $after) { nodes { name } hasNextPage endCursor } }",
).variables(async_graphql::Variables::from_json(serde_json::json!({"after": cursor}))),
).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
let json = response.data.into_json().unwrap();
let page = &json["reposPage"];
let rows = page["nodes"].as_array().unwrap();
assert!(rows.len() <= 50);
names.extend(
rows.iter()
.map(|row| row["name"].as_str().unwrap().to_owned()),
);
if !page["hasNextPage"].as_bool().unwrap() {
break;
}
let next = page["endCursor"].as_str().unwrap().to_owned();
assert_ne!(cursor.as_ref(), Some(&next));
cursor = Some(next);
assert!(names.len() <= crate::db::MAX_VISIBLE_REPO_PAGE_SIZE);
}
assert_eq!(
names,
(0..total)
.map(|index| format!("repo-{index:03}"))
.collect::<Vec<_>>()
);
}
#[sqlx::test]
async fn repos_legacy_accepts_exactly_the_visible_bound(pool: PgPool) {
let db = db(pool).await;
let base_time = Utc::now();
let total = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE;
for index in 0..total {
let name = format!("repo-{index:03}");
let mut r = repo(&name, OWNER, &name, true);
// Anti-correlate updated_at with name, creation order, and index
// so only ordering by updated_at DESC can satisfy the expectation.
let offset = (index * 37) % total;
r.updated_at = base_time + chrono::Duration::seconds(offset as i64);
db.create_repo(&r).await.unwrap();
}
db.create_repo(&repo("hidden", OWNER, "hidden", false))
.await
.unwrap();
let response = anon(&schema(db), "{ repos { name } }").await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
let repos = response.data.into_json().unwrap()["repos"]
.as_array()
.unwrap()
.clone();
assert_eq!(repos.len(), total);
let names: Vec<_> = repos.iter().filter_map(|r| r["name"].as_str()).collect();
assert!(
!names.contains(&"hidden"),
"hidden repo must be excluded from legacy repos response"
);
let mut expected_indices = (0..total).collect::<Vec<_>>();
expected_indices.sort_by_key(|&idx| std::cmp::Reverse((idx * 37) % total));
let expected_names: Vec<String> = expected_indices
.into_iter()
.map(|index| format!("repo-{index:03}"))
.collect();
assert_eq!(
names,
expected_names
.iter()
.map(|s| s.as_str())
.collect::<Vec<_>>(),
"legacy repos must retain activity ordering (most recently updated first)"
);
}
#[sqlx::test]
async fn repos_page_pagination_crosses_owner_boundary(pool: PgPool) {
let db = db(pool).await;
let owner1 = "did:key:z6MkaOwner1";
let owner2 = "did:key:z6MkbOwner2";
let earlier_owner = "did:web:example.com";
// This owner sorts first only after did:key normalization. Its name
// sorts past the first cursor, so an unqualified OR name predicate
// would incorrectly serve it again on the next page.
//
// Rows are inserted out of order and not in reverse asserted order.
// Assigned IDs do not track name order, and owner2 contains both
// "B-repo" and "a-repo" whose byte order (B < a) differs from their
// case-folded order (a < b), ensuring lower(d.name) cannot satisfy
// the keyset query. Sized to 6 entries so the last page is full-sized
// (limit 2), proving hasNextPage distinguishes a full terminal page.
db.create_repo(&repo("r5", owner2, "c-repo", true))
.await
.unwrap();
db.create_repo(&repo("r9", earlier_owner, "zz-repo", true))
.await
.unwrap();
db.create_repo(&repo("r3", owner2, "B-repo", true))
.await
.unwrap();
db.create_repo(&repo("r8", owner1, "z-repo", true))
.await
.unwrap();
db.create_repo(&repo("r2", owner1, "b-repo", true))
.await
.unwrap();
db.create_repo(&repo("r1", owner2, "a-repo", true))
.await
.unwrap();
let schema = schema(db);
let page1_resp = anon(
&schema,
"{ reposPage(limit: 2) { nodes { name ownerDid } hasNextPage endCursor } }",
)
.await;
assert!(page1_resp.errors.is_empty(), "{:?}", page1_resp.errors);
let p1 = page1_resp.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p1["hasNextPage"], true);
assert_eq!(
p1["nodes"],
serde_json::json!([
{"name": "zz-repo", "ownerDid": earlier_owner},
{"name": "b-repo", "ownerDid": owner1},
])
);
let cursor = p1["endCursor"].as_str().unwrap();
let page2_query = format!(
"{{ reposPage(limit: 2, after: \"{cursor}\") {{ nodes {{ name ownerDid }} hasNextPage endCursor }} }}"
);
let page2_resp = anon(&schema, &page2_query).await;
assert!(page2_resp.errors.is_empty(), "{:?}", page2_resp.errors);
let p2 = page2_resp.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p2["hasNextPage"], true);
assert_eq!(
p2["nodes"],
serde_json::json!([
{"name": "z-repo", "ownerDid": owner1},
{"name": "B-repo", "ownerDid": owner2},
])
);
let cursor = p2["endCursor"].as_str().unwrap();
let page3_query = format!(
"{{ reposPage(limit: 2, after: \"{cursor}\") {{ nodes {{ name ownerDid }} hasNextPage endCursor }} }}"
);
let page3_resp = anon(&schema, &page3_query).await;
assert!(page3_resp.errors.is_empty(), "{:?}", page3_resp.errors);
let p3 = page3_resp.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p3["hasNextPage"], false);
assert_eq!(
p3["nodes"],
serde_json::json!([
{"name": "a-repo", "ownerDid": owner2},
{"name": "c-repo", "ownerDid": owner2},
])
);
}
#[sqlx::test]
async fn repos_page_at_documented_maximum_distinguishes_terminal_page(pool: PgPool) {
let db = db(pool).await;
let total = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE;
for index in 0..total {
let name = format!("repo-{index:03}");
db.create_repo(&repo(&name, OWNER, &name, true))
.await
.unwrap();
}
let schema = schema(db.clone());
let query =
format!("{{ reposPage(limit: {total}) {{ nodes {{ name }} hasNextPage endCursor }} }}");
let resp200 = anon(&schema, &query).await;
assert!(resp200.errors.is_empty(), "{:?}", resp200.errors);
let p200 = resp200.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p200["hasNextPage"], false);
let nodes200 = p200["nodes"].as_array().unwrap().clone();
assert_eq!(nodes200.len(), total);
// Adding row 201 keeps page 1 content identical while flipping hasNextPage to true.
let name201 = format!("repo-{total:03}");
db.create_repo(&repo(&name201, OWNER, &name201, true))
.await
.unwrap();
let resp201 = anon(&schema, &query).await;
assert!(resp201.errors.is_empty(), "{:?}", resp201.errors);
let p201 = resp201.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p201["hasNextPage"], true);
assert_eq!(p201["nodes"], p200["nodes"]);
assert_eq!(p201["endCursor"], p200["endCursor"]);
}
#[sqlx::test]
async fn repos_page_visibility_matches_the_shared_gate(pool: PgPool) {
use crate::db::VisibilityMode;
let db = db(pool).await;
let reader = "did:key:zReader";
for (id, public) in [
("open", true),
("private", false),
("root-deny", true),
("root-reader", false),
("subtree", true),
("root-tie", true),
("odd-star", true),
("malformed-readers", true),
("non-array-readers", true),
("non-string-reader", true),
("quarantined", true),
] {
db.create_repo(&repo(id, OWNER, id, public)).await.unwrap();
}
// Canonical and mirror copies must still collapse before pagination.
db.create_repo(&repo("z6MkOwner/open", "z6MkOwner", "open", true))
.await
.unwrap();
db.create_repo(&repo(
"other-method",
"did:web:z6MkOwner",
"other-method",
false,
))
.await
.unwrap();
db.set_repo_quarantine("quarantined", true).await.unwrap();
for (id, glob, readers) in [
("root-deny", "/", vec![]),
("root-reader", "/**", vec![reader.to_owned()]),
("subtree", "/secret/**", vec![]),
("root-tie", "/", vec![reader.to_owned()]),
("root-tie", "/**", vec![]),
("odd-star", "/*", vec![]),
("malformed-readers", "/", vec![]),
("non-array-readers", "/", vec![]),
("non-string-reader", "/", vec![]),
] {
db.set_visibility_rule(id, glob, VisibilityMode::B, &readers, OWNER)
.await
.unwrap();
}
// The typed setter cannot create malformed JSON, but existing TEXT rows
// can contain it. Such a rule must deny this repo, not abort the page.
// All three deny arms of the reader_dids predicate are pinned here:
// (a) invalid JSON, (b) valid JSON that is not an array, (c) an array
// containing a non-string member. The Rust gate parses reader_dids as
// Vec<String> with unwrap_or_default(), so (b) and (c) also fail to
// parse and deny; the SQL CASE must agree on every shape.
let rows = sqlx::query("UPDATE visibility_rules SET reader_dids = $1 WHERE repo_id = $2")
.bind("not JSON")
.bind("malformed-readers")
.execute(db.pool())
.await
.unwrap();
assert_eq!(rows.rows_affected(), 1);
let rows = sqlx::query("UPDATE visibility_rules SET reader_dids = $1 WHERE repo_id = $2")
.bind("\"just-a-string\"")
.bind("non-array-readers")
.execute(db.pool())
.await
.unwrap();
assert_eq!(rows.rows_affected(), 1);
// The named reader is listed, but the non-string member poisons the
// whole list: both sides must still deny them.
let rows = sqlx::query("UPDATE visibility_rules SET reader_dids = $1 WHERE repo_id = $2")
.bind("[\"did:key:zReader\", 42]")
.bind("non-string-reader")
.execute(db.pool())
.await
.unwrap();
assert_eq!(rows.rows_affected(), 1);
let all = db.list_all_repos_deduped().await.unwrap();
for caller in [
None,
Some(OWNER),
Some("z6MkOwner"),
Some(reader),
Some("zReader"),
Some("did:web:z6MkOwner"),
] {
let mut expected = Vec::new();
for record in &all {
let rules = db.list_visibility_rules(&record.id).await.unwrap();
if crate::visibility::listable_at_root(
&rules,
record.is_public,
&record.owner_did,
caller,
) {
expected.push(record.id.clone());
}
}
expected.sort();
let mut actual = db
.list_visible_repos_page(caller, None, usize::MAX)
.await
.unwrap()
.into_iter()
.map(|record| record.id)
.collect::<Vec<_>>();
actual.sort();
assert_eq!(actual, expected, "caller {caller:?}");
}
let schema = schema(db);
for caller in [None, Some("did:key:zUnauthorized")] {
let query = "{ repos { name } }";
let response = match caller {
None => anon(&schema, query).await,
Some(caller) => authed(&schema, query, caller).await,
};
assert!(response.errors.is_empty(), "{:?}", response.errors);
let mut names: Vec<String> = response.data.into_json().unwrap()["repos"]
.as_array()
.unwrap()
.iter()
.map(|repo| repo["name"].as_str().unwrap().to_owned())
.collect();
names.sort();
assert_eq!(names, ["odd-star", "open", "subtree"]);
}
let query = "{ reposPage(limit: 1) { nodes { name ownerDid } hasNextPage endCursor } }";
let response = anon(&schema, query).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
let json = response.data.into_json().unwrap();
let page = &json["reposPage"];
assert_eq!(
page["nodes"],
serde_json::json!([{"name": "odd-star", "ownerDid": OWNER}])
);
assert_eq!(page["hasNextPage"], true);
let cursor = super::parse_repo_cursor(page["endCursor"].as_str().unwrap()).unwrap();
assert_eq!(cursor, (OWNER.to_owned(), "odd-star".to_owned()));
assert!(!json.to_string().contains("private"));
assert!(!json.to_string().contains("quarantined"));
// Route-level check with unauthorized authenticated caller:
// Excludes private, quarantined, and root-deny repos, matching anonymous behavior.
let all_query = "{ reposPage(limit: 50) { nodes { name } hasNextPage } }";
let unauth_response = authed(&schema, all_query, "did:key:zUnauthorized").await;
assert!(
unauth_response.errors.is_empty(),
"{:?}",
unauth_response.errors
);
let unauth_json = unauth_response.data.into_json().unwrap();
let unauth_names: Vec<&str> = unauth_json["reposPage"]["nodes"]
.as_array()
.unwrap()
.iter()
.filter_map(|n| n["name"].as_str())
.collect();
assert_eq!(unauth_names, vec!["odd-star", "open", "subtree"]);
let unauth_str = unauth_json.to_string();
assert!(!unauth_str.contains("private"));
assert!(!unauth_str.contains("quarantined"));
assert!(!unauth_str.contains("root-deny"));
assert!(!unauth_str.contains("root-reader"));
assert!(!unauth_str.contains("root-tie"));
// Reader caller gets root-reader and root-tie, but still excludes private, quarantined, root-deny.
let reader_response = authed(&schema, all_query, reader).await;
assert!(
reader_response.errors.is_empty(),
"{:?}",
reader_response.errors
);
let reader_json = reader_response.data.into_json().unwrap();
let reader_names: Vec<&str> = reader_json["reposPage"]["nodes"]
.as_array()
.unwrap()
.iter()
.filter_map(|n| n["name"].as_str())
.collect();
assert_eq!(
reader_names,
vec!["odd-star", "open", "root-reader", "subtree"]
);
let reader_str = reader_json.to_string();
assert!(!reader_str.contains("private"));
assert!(!reader_str.contains("quarantined"));
assert!(!reader_str.contains("root-deny"));
assert!(!reader_str.contains("root-tie"));
}
#[sqlx::test]
async fn repos_page_rechecks_cursor_authority_and_exact_boundary(pool: PgPool) {
let db = db(pool).await;
let visible = repo("visible", OWNER, "a-visible", true);
db.create_repo(&visible).await.unwrap();
db.create_repo(&repo("hidden", OWNER, "z-private", false))
.await
.unwrap();
let schema = schema(db);
let first = anon(
&schema,
"{ reposPage(limit: 1) { nodes { name } hasNextPage endCursor } }",
)
.await;
assert!(first.errors.is_empty());
let first = first.data.into_json().unwrap();
assert_eq!(first["reposPage"]["hasNextPage"], false);
let query = format!(
"{{ reposPage(limit: 1, after: \"{}\") {{ nodes {{ name }} hasNextPage endCursor }} }}",
super::repo_cursor(&visible)
);
let owner = authed(&schema, &query, OWNER).await;
assert!(owner.errors.is_empty());
assert_eq!(
owner.data.into_json().unwrap()["reposPage"]["nodes"][0]["name"],
"z-private"
);
let anon = anon(&schema, &query).await;
assert!(anon.errors.is_empty());
assert_eq!(
anon.data.into_json().unwrap()["reposPage"],
serde_json::json!({
"nodes": [], "hasNextPage": false, "endCursor": null
})
);
let unauth = authed(&schema, &query, "did:key:zUnauthorized").await;
assert!(unauth.errors.is_empty());
assert_eq!(
unauth.data.into_json().unwrap()["reposPage"],
serde_json::json!({
"nodes": [], "hasNextPage": false, "endCursor": null
})
);
}
#[tokio::test]
async fn repos_page_rejects_nul_cursors_before_database_access() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
for position in [("did:key:zOwner\0", "repo"), ("did:key:zOwner", "repo\0")] {
let cursor = super::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&position).unwrap());
let query = format!("{{ reposPage(after: \"{cursor}\") {{ hasNextPage }} }}");
let response =
tokio::time::timeout(std::time::Duration::from_secs(1), anon(&schema, &query))
.await
.expect("invalid cursors must be rejected without accessing the database");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "invalid repository cursor");
assert_eq!(response.data, async_graphql::Value::Null);
}
let position = ("did:key:zOwner", "repo");
let cursor = super::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&position).unwrap());
assert_eq!(
super::parse_repo_cursor(&cursor).unwrap(),
(position.0.to_owned(), position.1.to_owned())
);
}
#[tokio::test]
async fn repos_page_rejects_invalid_inputs_before_database_access() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
let over_max = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE + 1;
let expected_limit_message = format!(
"limit must be between 1 and {}",
crate::db::MAX_VISIBLE_REPO_PAGE_SIZE
);
// Each input pins its own rejection message: the loop must not accept
// either message for every query, or a misattributed message (e.g. the
// limit arm returning "invalid repository cursor") stays invisible.
for (query, expected_message) in [
(
"{ reposPage(limit: 0) { hasNextPage } }".to_owned(),
expected_limit_message.clone(),
),
(
"{ reposPage(limit: -1) { hasNextPage } }".to_owned(),
expected_limit_message.clone(),
),
(
format!("{{ reposPage(limit: {over_max}) {{ hasNextPage }} }}"),
expected_limit_message.clone(),
),
(
"{ reposPage(after: \"invalid!\") { hasNextPage } }".to_owned(),
"invalid repository cursor".to_owned(),
),
] {
let response =
tokio::time::timeout(std::time::Duration::from_secs(1), anon(&schema, &query))
.await
.unwrap();
assert_eq!(response.errors.len(), 1);
assert_eq!(
response.errors[0].message, expected_message,
"wrong rejection message for query: {query}"
);
}
// Valid base64 and valid cursor JSON: only the length guard rejects it.
let oversized = super::URL_SAFE_NO_PAD
.encode(serde_json::to_vec(&("did:key:reader", "a".repeat(3100))).unwrap());
assert!(oversized.len() > 4096);
assert!(serde_json::from_slice::<(String, String)>(
&super::URL_SAFE_NO_PAD.decode(&oversized).unwrap()
)
.is_ok());
assert!(super::parse_repo_cursor(&oversized).is_err());
}
#[tokio::test]
async fn parse_repo_cursor_rejects_wrong_shape_json() {
// Valid base64 and valid JSON, but not a (String, String) pair: the
// wrong-shape arm must report "invalid repository cursor", exactly,
// before any database access.
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
for payload in [
serde_json::json!(["only-one"]),
serde_json::json!(["a", "b", "c"]),
serde_json::json!({"owner": "x", "name": "y"}),
serde_json::json!(42),
serde_json::json!(["a", 1]),
] {
let cursor = super::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&payload).unwrap());
let err = super::parse_repo_cursor(&cursor).unwrap_err();
assert_eq!(err.message, "invalid repository cursor");
let query = format!("{{ reposPage(after: \"{cursor}\") {{ hasNextPage }} }}");
let response =
tokio::time::timeout(std::time::Duration::from_secs(1), anon(&schema, &query))
.await
.expect("wrong-shape cursors must be rejected without accessing the database");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "invalid repository cursor");
assert_eq!(response.data, async_graphql::Value::Null);
}
}
#[tokio::test]
async fn longest_repo_name_cursor_round_trips() {
// #465 4th round: a cursor the server emits must always parse back.
// Repo names are bounded at the create/fork/mirror write paths, so the
// longest admissible name must produce a cursor under the 4096-byte
// parse limit.
let name = "a".repeat(crate::db::MAX_REPO_NAME_LEN);
let record = repo("long", OWNER, &name, true);
let cursor = super::repo_cursor(&record);
assert!(
cursor.len() <= 4096,
"emitted cursor must stay under the parse limit: {}",
cursor.len()
);
assert_eq!(
super::parse_repo_cursor(&cursor).unwrap(),
(OWNER.to_owned(), name)
);
}
/// A negative refUpdates(limit) must be accepted and clamped to zero rows,
/// like tasks(limit: -1): the complexity formula's `limit.clamp(0, ...)`
/// prices it at the floor and the shared collector returns before any
/// database access. Mirrors tasks_negative_limit_clamped.
#[tokio::test]
async fn ref_updates_negative_limit_clamped() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
let resp = tokio::time::timeout(
std::time::Duration::from_secs(1),
anon(&schema, "{ refUpdates(limit: -1) { repo } }"),
)
.await
.expect("negative limit must clamp without accessing the database");
assert!(
resp.errors.is_empty(),
"negative limit must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(count(&resp), 0);
}
/// The mirror write path binds peer-supplied names verbatim, so it must
/// enforce the same bound as the local write paths: an overlong peer name
/// would otherwise make reposPage emit an unparseable cursor. The
/// rejection happens before any database access.
#[tokio::test]
async fn mirror_repo_rejects_overlong_name() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let db = Db::for_testing(pool);
let name = "b".repeat(crate::db::MAX_REPO_NAME_LEN + 1);
let err = db
.upsert_mirror_repo("z6MkLong", &name, "/tmp/long", None, false)
.await
.unwrap_err();
assert!(
err.to_string().contains("exceeds"),
"unexpected error: {err}"
);
}
async fn db(pool: PgPool) -> Arc<Db> {
let db = Db::for_testing(pool);
db.run_migrations().await.unwrap();
@@ -475,12 +1211,13 @@ mod tests {
assert_eq!(count_tasks(&resp), 0);
}
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises Max 200).
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises the
/// documented maximum).
#[sqlx::test]
async fn tasks_limit_ceiling_clamped_to_200(pool: PgPool) {
async fn tasks_limit_ceiling_clamped_to_max(pool: PgPool) {
let db = db(pool).await;
let now = Utc::now().to_rfc3339();
for i in 0..201 {
for i in 0..crate::db::MAX_VISIBLE_TASK_PAGE_SIZE + 1 {
db.create_task(&crate::db::AgentTask {
id: format!("task-ceil-{i}"),
repo_id: None,
@@ -501,7 +1238,49 @@ mod tests {
}
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: 5000) { id } }").await;
assert_eq!(count_tasks(&resp), 200, "limit above 200 must clamp to 200");
assert_eq!(
count_tasks(&resp),
crate::db::MAX_VISIBLE_TASK_PAGE_SIZE as usize,
"limit above the maximum must clamp to MAX_VISIBLE_TASK_PAGE_SIZE"
);
}
/// A limit above MAX_VISIBLE_REF_UPDATES must reach the resolver clamped,
/// not be rejected: the complexity formula clamps the priced limit (so the
/// document passes validation) and the shared collector clamps the rows
/// (so at most MAX_VISIBLE_REF_UPDATES come back). Removing the formula
/// clamp prices the raw limit and rejects the document; removing the
/// collector clamp returns unbounded rows. Mirrors
/// tasks_limit_ceiling_clamped_to_max.
#[sqlx::test]
async fn ref_updates_limit_ceiling_clamped_to_max(pool: PgPool) {
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
let db = db(pool).await;
db.create_repo(&repo("r1", OWNER, "widget", true))
.await
.unwrap();
let total = MAX_VISIBLE_REF_UPDATES as usize + 1;
for i in 0..total {
let mut row = ref_row(&format!("ceil{i}"), "z6MkOwner/widget");
row.timestamp = format!("2026-07-01T10:{:02}:{:02}+00:00", i / 60, i % 60);
db.insert_ref_update(&row).await.unwrap();
}
let schema = schema(db);
let query = format!(
"{{ refUpdates(limit: {}) {{ repo }} }}",
MAX_VISIBLE_REF_UPDATES + 4800
);
let resp = anon(&schema, &query).await;
assert!(
resp.errors.is_empty(),
"limit above the maximum must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(
count(&resp),
MAX_VISIBLE_REF_UPDATES as usize,
"limit above the maximum must clamp to MAX_VISIBLE_REF_UPDATES"
);
}
fn count_tasks(resp: &async_graphql::Response) -> usize {
@@ -20,6 +20,7 @@ impl SubscriptionRoot {
/// `if announce` block, `api/repos.rs`). This is a single-point invariant —
/// any new sender to `ref_update_tx` MUST be `announce`-gated, or private-repo
/// ref metadata leaks here to unauthenticated subscribers (#112/#114 class).
#[graphql(complexity = "50 + child_complexity")]
async fn ref_updates(
&self,
ctx: &Context<'_>,
@@ -46,6 +47,7 @@ impl SubscriptionRoot {
})
}
#[graphql(complexity = "50 + child_complexity")]
async fn task_events(
&self,
ctx: &Context<'_>,
+8
View File
@@ -11,6 +11,14 @@ pub struct RepoType {
pub created_at: String,
}
#[derive(SimpleObject)]
pub struct RepoPageType {
pub nodes: Vec<RepoType>,
pub has_next_page: bool,
/// Position after the last returned visible repository; null for an empty page.
pub end_cursor: Option<String>,
}
#[derive(SimpleObject, Clone)]
pub struct AgentTaskType {
pub id: String,
+1
View File
@@ -2638,6 +2638,7 @@ mod tests {
/// type probe carries its oid on stdin rather than in argv, so an oid appears in
/// the log only once an object has already got past its probe, and a healthy
/// object costs two invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log)
.unwrap_or_default()
+261 -51
View File
@@ -7,6 +7,7 @@ mod bootstrap;
mod cert;
mod config;
mod db;
mod durable_outbox;
mod encrypted_pin;
mod error;
mod git;
@@ -51,6 +52,7 @@ use state::AppState;
struct DegradedState {
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
}
/// Two independent counters with no cross-field invariant — atomics, not a
@@ -159,6 +161,7 @@ async fn main() -> Result<()> {
degraded_listener,
node_did.to_string(),
Arc::clone(&db_startup),
config.web_url.clone(),
db_ready_rx,
shutdown_tx.subscribe(),
));
@@ -429,6 +432,8 @@ async fn main() -> Result<()> {
std::time::Duration::from_secs(3600),
200_000,
);
let federated_rate_limiter =
rate_limit::RateLimiter::new_bounded(12, std::time::Duration::from_secs(60), 10_000);
if config.sync_trigger_rate_limit == 0 {
tracing::warn!(
"GITLAWB_SYNC_TRIGGER_RATE_LIMIT=0 — /sync/trigger IP rate limiting disabled"
@@ -467,6 +472,7 @@ async fn main() -> Result<()> {
push_limiter_trust,
sync_trigger_rate_limiter,
peer_write_rate_limiter,
federated_rate_limiter,
shutdown_tx: shutdown_tx.clone(),
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(config.max_concurrent_git_ops)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(
@@ -604,6 +610,13 @@ async fn main() -> Result<()> {
}
let _legacy_cid_sweep = spawn_legacy_cid_sweep(&state, &config);
let _queue_lifecycle_sweep = spawn_queue_lifecycle_sweep(&state, &config);
// NOTE: the due-request worker is spawned AFTER the one-shot startup
// reconcile/drain below, not here: its 5s interval would otherwise tick
// during operator/PoS setup and interleave with boot recovery on backlog
// rows (state gates prevent data loss, but the interleave costs recovery
// latency and widens the reconcile/drain race window).
let _marker_cleanup_worker = spawn_marker_cleanup_worker(&state);
let router = server::build_router(state.clone());
// Re-register the socket bound at startup — same fd, so there was never a
@@ -695,6 +708,63 @@ async fn main() -> Result<()> {
let grace = std::time::Duration::from_secs(config.shutdown_grace_secs);
info!(grace_secs = config.shutdown_grace_secs, "axum server ready");
// #26 Split PR 1: drain any `applied` rows left by a previous
// process that crashed after Git applied a ref but before the
// bookkeeping landed. Runs once, BEFORE the server accepts new
// pushes, so a recovery re-derivation does not race a fresh push.
// Non-fatal: a transient drain failure is logged and the rows
// remain `applied` for the next startup to pick up.
//
// P1-A: the reconcile step runs FIRST and promotes any `prepared`
// or `uncertain` row whose target SHA actually landed on disk.
// This is the path that recovers a ref when the post-receive
// `mark_pending_ref_transitions_applied` call errored or was
// interrupted after `receive_pack` returned Ok, or when
// receive-pack returned Err but some refs may have landed
// (the `uncertain` state). Without this step, the drain (gated
// on `state = 'applied'`) would never see those rows.
//
// P2 (reviewer-1/2 round 3): use the multi-pass reconcile so
// prepared/uncertain rows beyond the first 1000-row page are
// processed in the same startup, rather than waiting for the
// next restart (where they might age out of MAX_RECONCILE_AGE).
match durable_outbox::reconcile_prepared_from_disk_all(
state.clone(),
durable_outbox::DRAIN_PER_PASS_LIMIT,
durable_outbox::DRAIN_MAX_PASSES,
)
.await
{
Ok(0) => {}
Ok(n) => info!(
n,
"reconciled prepared/uncertain -> applied via on-disk ref match"
),
Err(e) => warn!(
err = %e,
"pending ref transition reconcile failed at startup (non-fatal; will retry on next start)"
),
}
match durable_outbox::drain_receive_pack_requests_all(
state.clone(),
durable_outbox::DRAIN_PER_PASS_LIMIT,
durable_outbox::DRAIN_MAX_PASSES,
)
.await
{
Ok(0) => {}
Ok(n) => info!(n, "drained pending ref transitions from prior run"),
Err(e) => warn!(
err = %e,
"pending ref transition drain failed at startup (non-fatal; will retry on next start)"
),
}
// Start the due-request worker only now that boot recovery is done (see
// the note at the other spawns above): its first tick must not interleave
// with the one-shot reconcile/drain on backlog rows.
let _due_request_worker = spawn_due_request_worker(&state);
// `into_make_service_with_connect_info` exposes the socket peer address as
// `ConnectInfo<SocketAddr>` so the push limiter can key on the real client
// when no trusted proxy header applies (see `rate_limit::client_key`).
@@ -727,6 +797,158 @@ async fn main() -> Result<()> {
Ok(())
}
/// U4 (#173): spawn the periodic legacy provider-CID repair sweep. Releases before this
/// version stored the PROVIDER CID (Kubo dag-pb / Pinata CIDv0) in `pinned_cids.cid`,
/// and this version's `/ipfs/{cid}` resolver withholds any row whose stored key is not
/// the raw-content CID. The opportunistic repair on the pin path only fires when a push
/// re-carries the object, which normal git negotiation makes it not do, so those rows
/// need a walk. DETACHED, never on the boot path: the caller keeps serving while this
/// runs, and the sweep's own batch bound plus inter-batch delay keep it off the DB's
/// critical path. Its cursor is durable, so a restart mid-walk resumes instead of
/// rewinding. It re-arms after every run, including a failed one, so it never returns
/// and there is no awaited value to log here; the shutdown watcher below is what ends
/// it.
///
/// A named function rather than an inline block in `main` so the WIRING has a seam a
/// test can call: that the task is spawned at all, that it reads its batch and delay
/// from the config knobs rather than some other field, that the caller is not blocked
/// on it, and that the shutdown watcher actually ends it mid-walk. The sweep's own
/// behavior is covered elsewhere; this is the boot-path half.
fn spawn_legacy_cid_sweep(state: &AppState, config: &Config) -> tokio::task::JoinHandle<()> {
let db = state.db.clone();
let repos_dir = config.repos_dir.clone();
let git_bin = state.git_bin.clone();
let git_timeout = std::time::Duration::from_secs(config.git_service_timeout_secs);
let batch = config.pin_repair_sweep_batch;
let delay = std::time::Duration::from_secs(config.pin_repair_sweep_delay_secs);
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
tokio::select! {
_ = ipfs_pin::run_sweep_rearmed(
&repos_dir, &git_bin, git_timeout, batch, delay,
ipfs_pin::SWEEP_REARM_DELAY, &db,
) => {}
// Shutdown mid-walk simply drops the run; the persisted cursor means the
// next boot picks up where this one stopped.
_ = shutdown_rx.changed() => {}
}
})
}
/// #26 Split PR 1 step 4 — periodic queue-lifecycle purge. Runs on
/// the same detached task pattern as `spawn_legacy_cid_sweep`:
/// tokio::spawn with a shutdown watcher, never on the boot path. The
/// interval is fixed at 24 hours (the spec calls for "one per
/// cluster per day"); the inter-batch delay is implicit in the
/// batch size plus the wall-clock cost of each pass. The drain
/// (`drain_receive_pack_requests_all`) and the purge
/// (`purge_request_queue`) share the same `DRAIN_PER_PASS_LIMIT`
/// budget so a 1000-row purge pass takes roughly the same time as
/// a 1000-row drain pass.
fn spawn_queue_lifecycle_sweep(state: &AppState, config: &Config) -> tokio::task::JoinHandle<()> {
let app_state = state.clone();
let retention_days = config.queue_retention_days;
let batch = config.queue_purge_batch;
let git_timeout = std::time::Duration::from_secs(config.git_service_timeout_secs.min(30));
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(24 * 3600));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
// Consume the immediate first tick: Tokio intervals fire
// immediately on creation, which would run the purge
// concurrently with startup reconcile/drain. The child purge
// is terminal-parent-gated in SQL, but delaying the first
// sweep a full period removes the race entirely.
interval.tick().await;
loop {
tokio::select! {
_ = interval.tick() => {
if let Err(e) = durable_outbox::purge_request_queue(
&app_state.db,
retention_days,
batch,
).await {
tracing::warn!(err = %e, "queue lifecycle purge failed; will retry on next tick");
}
// Drain marker tombstones with the bounded deleter;
// failures retain the tombstone for the next tick.
if let Err(e) = durable_outbox::drain_marker_cleanup_queue(
&app_state,
batch,
git_timeout,
).await {
tracing::warn!(err = %e, "marker cleanup drain failed");
}
}
_ = shutdown_rx.changed() => {
break;
}
}
}
})
}
/// Shutdown-aware background due-request loop. The startup drain runs
/// once; without this, `effects_pending` rows with a future
/// `next_attempt_at` would wait until the next process restart.
/// Polls the indexed due query with bounded batches and failure
/// isolation (one request's error never aborts the batch).
fn spawn_due_request_worker(state: &AppState) -> tokio::task::JoinHandle<()> {
let app_state = state.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(5));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
tokio::select! {
_ = interval.tick() => {
match durable_outbox::drain_receive_pack_requests(
app_state.clone(),
durable_outbox::DRAIN_PER_PASS_LIMIT,
)
.await
{
Ok((0, _)) => {}
Ok((n, _)) => tracing::info!(n, "due request worker drained"),
Err(e) => tracing::warn!(err = %e, "due request worker drain failed"),
}
}
_ = shutdown_rx.changed() => break,
}
}
})
}
/// Dedicated marker-tombstone worker at a shorter cadence than the
/// daily purge, so external Git refs do not linger a full period
/// after their SQL owner is gone.
fn spawn_marker_cleanup_worker(state: &AppState) -> tokio::task::JoinHandle<()> {
let app_state = state.clone();
let git_timeout =
std::time::Duration::from_secs(app_state.config.git_service_timeout_secs.min(30));
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(60));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
tokio::select! {
_ = interval.tick() => {
if let Err(e) = durable_outbox::drain_marker_cleanup_queue(
&app_state,
durable_outbox::DRAIN_PER_PASS_LIMIT,
git_timeout,
)
.await
{
tracing::warn!(err = %e, "marker cleanup worker failed");
}
}
_ = shutdown_rx.changed() => break,
}
}
})
}
fn spawn_shutdown_signal(tx: watch::Sender<bool>) {
tokio::spawn(async move {
#[cfg(unix)]
@@ -869,11 +1091,12 @@ async fn run_degraded_server(
listener: TcpListener,
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
mut db_ready_rx: watch::Receiver<bool>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<()> {
let addr = listener.local_addr().ok();
let router = build_degraded_router(node_did, db_startup);
let router = build_degraded_router(node_did, db_startup, web_url);
info!(?addr, "degraded HTTP server ready");
axum::serve(listener, router)
@@ -890,10 +1113,15 @@ async fn run_degraded_server(
Ok(())
}
fn build_degraded_router(node_did: String, db_startup: Arc<DbStartupStatus>) -> Router {
fn build_degraded_router(
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
) -> Router {
let state = DegradedState {
node_did,
db_startup,
web_url,
};
// Everything answers 503 with the same body — including /health and
// /ready, so peer readiness probes and uptime monitors correctly see a
@@ -925,6 +1153,9 @@ async fn degraded_node_info(State(state): State<DegradedState>) -> impl IntoResp
obj.insert("name".into(), "gitlawb-node".into());
obj.insert("version".into(), env!("CARGO_PKG_VERSION").into());
obj.insert("did".into(), state.node_did.clone().into());
if let Some(web_url) = &state.web_url {
obj.insert("web_url".into(), web_url.clone().into());
}
}
(StatusCode::SERVICE_UNAVAILABLE, Json(body))
}
@@ -1171,6 +1402,7 @@ mod rate_limiter_sweep_tests {
state.close_issue_rate_limiter = RateLimiter::new(10, window);
state.sync_trigger_rate_limiter = RateLimiter::new(10, window);
state.peer_write_rate_limiter = RateLimiter::new(10, window);
state.federated_rate_limiter = RateLimiter::new(10, window);
state.ipfs_rate_limiter = RateLimiter::new(10, window);
let limiters = |s: &crate::state::AppState| {
@@ -1181,6 +1413,7 @@ mod rate_limiter_sweep_tests {
s.close_issue_rate_limiter.clone(),
s.sync_trigger_rate_limiter.clone(),
s.peer_write_rate_limiter.clone(),
s.federated_rate_limiter.clone(),
s.ipfs_rate_limiter.clone(),
]
};
@@ -1199,55 +1432,6 @@ mod rate_limiter_sweep_tests {
}
/// it.
fn spawn_legacy_cid_sweep(state: &AppState, config: &Config) -> tokio::task::JoinHandle<()> {
let db = state.db.clone();
let repos_dir = config.repos_dir.clone();
let git_bin = state.git_bin.clone();
let git_timeout = std::time::Duration::from_secs(config.git_service_timeout_secs);
let batch = config.pin_repair_sweep_batch;
let delay = std::time::Duration::from_secs(config.pin_repair_sweep_delay_secs);
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
tokio::select! {
_ = ipfs_pin::run_sweep_rearmed(
&repos_dir, &git_bin, git_timeout, batch, delay,
ipfs_pin::SWEEP_REARM_DELAY, &db,
) => {}
_ = shutdown_rx.changed() => {}
}
})
}
async fn gossip_ping_round(
db: &Db,
client: &reqwest::Client,
failed_once: &mut HashSet<String>,
peers: Vec<db::PeerRecord>,
) {
{
let current_dids: HashSet<&str> = peers.iter().map(|peer| peer.did.as_str()).collect();
failed_once.retain(|did| current_dids.contains(did.as_str()));
}
for peer in peers {
let ok = ping_peer_readiness(client, &peer.http_url).await;
match peer_ping_db_update(failed_once, &peer.did, ok) {
Some(reachable) => {
if let Err(error) = db.mark_peer_ping(&peer.did, reachable).await {
warn!(did = %peer.did, err = %error, "failed to persist peer readiness");
}
}
None => warn!(
did = %peer.did,
"peer readiness probe failed once; preserving previous federation status"
),
}
}
}
/// Decide whether one readiness sample should update the persisted federation
/// gate. A success is authoritative immediately. A failure must repeat on the
/// next gossip tick before it can mark a peer unreachable, so one transient
/// database hiccup does not hide the peer's repositories for five minutes.
fn peer_ping_db_update(failed_once: &mut HashSet<String>, did: &str, ready: bool) -> Option<bool> {
if ready {
failed_once.remove(did);
@@ -1424,6 +1608,32 @@ fn load_or_create_keypair_at(key_path: &std::path::Path) -> Result<Keypair> {
Ok(kp)
}
async fn gossip_ping_round(
db: &Db,
client: &reqwest::Client,
failed_once: &mut HashSet<String>,
peers: Vec<db::PeerRecord>,
) {
{
let current_dids: HashSet<&str> = peers.iter().map(|peer| peer.did.as_str()).collect();
failed_once.retain(|did| current_dids.contains(did.as_str()));
}
for peer in peers {
let ok = ping_peer_readiness(client, &peer.http_url).await;
match peer_ping_db_update(failed_once, &peer.did, ok) {
Some(reachable) => {
if let Err(error) = db.mark_peer_ping(&peer.did, reachable).await {
warn!(did = %peer.did, err = %error, "failed to persist peer readiness");
}
}
None => warn!(
did = %peer.did,
"peer readiness probe failed once; preserving previous federation status"
),
}
}
}
#[cfg(test)]
mod gossip_ssrf_tests {
use super::{
+692 -3
View File
@@ -7,11 +7,11 @@
//! The node's PeerId comes from an Ed25519 keypair loaded from a persistent
//! key file, so the PeerId is stable across restarts.
use std::collections::{hash_map::DefaultHasher, HashMap};
use std::collections::{hash_map::DefaultHasher, HashMap, HashSet, VecDeque};
use std::hash::{Hash, Hasher};
use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
use std::time::{Duration, Instant};
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
@@ -315,6 +315,17 @@ fn is_zero(v: &u32) -> bool {
*v == 0
}
// Identify is an untrusted address source. Keep enough entries for normal
// multi-homed peers while bounding both one peer and the whole routing table.
const IDENTIFY_ADDRESS_LIMIT: usize = 8;
// Bound canonicalization and sorting too, including rejected/duplicate entries.
const IDENTIFY_REPORT_ADDRESS_LIMIT: usize = 64;
const IDENTIFY_NEW_ADDRESS_LIMIT: usize = 8;
const IDENTIFY_GLOBAL_ADDRESS_LIMIT: usize = 1024;
const IDENTIFY_ADDRESS_WINDOW: Duration = Duration::from_secs(60);
const IDENTIFY_ADDRESS_TTL: Duration = Duration::from_secs(30 * 60);
const IDENTIFY_ADDRESS_CLEANUP_INTERVAL: Duration = Duration::from_secs(60);
/// A ref-update event published to Gossipsub when a push lands.
///
/// The signing bytes are this struct serialized with `sig` set to None (see
@@ -1631,6 +1642,200 @@ fn did_to_kad_key(did: &str) -> kad::RecordKey {
kad::RecordKey::new(&format!("/gitlawb/did/{did}").as_bytes())
}
/// Returns true when `address` was explicitly configured for `peer` through
/// AddKnownPeer. Explicit addresses are retained in Kademlia even when an
/// overlapping Identify lease expires or is evicted.
fn is_explicit_address(
explicit_addresses: &HashMap<PeerId, HashSet<Multiaddr>>,
peer: &PeerId,
address: &Multiaddr,
) -> bool {
explicit_addresses
.get(peer)
.is_some_and(|addresses| addresses.contains(address))
}
#[derive(Debug)]
struct IdentifyAddress {
address: Multiaddr,
expires_at: Instant,
}
#[derive(Debug)]
struct IdentifyPeerAddresses {
addresses: VecDeque<IdentifyAddress>,
window_started: Instant,
new_addresses: usize,
}
#[derive(Debug, Default, PartialEq, Eq)]
struct IdentifyAddressChanges {
added: Vec<Multiaddr>,
refreshed: Vec<Multiaddr>,
removed: Vec<(PeerId, Multiaddr)>,
}
#[derive(Debug, Default)]
struct IdentifyAddressBook {
peers: HashMap<PeerId, IdentifyPeerAddresses>,
address_count: usize,
}
impl IdentifyAddressBook {
fn update(
&mut self,
peer_id: PeerId,
now: Instant,
addresses: &[Multiaddr],
) -> IdentifyAddressChanges {
let mut changes = IdentifyAddressChanges::default();
if self.peers.contains_key(&peer_id) {
changes.removed.extend(self.expire_peer(peer_id, now));
}
let reported = addresses
.iter()
.take(IDENTIFY_REPORT_ADDRESS_LIMIT)
.filter_map(|address| address.clone().with_p2p(peer_id).ok())
.collect::<HashSet<_>>();
if reported.is_empty() && !self.peers.contains_key(&peer_id) {
return changes;
}
let state = self
.peers
.entry(peer_id)
.or_insert_with(|| IdentifyPeerAddresses {
addresses: VecDeque::new(),
window_started: now,
new_addresses: 0,
});
if now.saturating_duration_since(state.window_started) >= IDENTIFY_ADDRESS_WINDOW {
state.window_started = now;
state.new_addresses = 0;
}
// Refresh every retained address in the bounded report before admission.
// Refreshed addresses are reported so the Identify handler can re-admit
// them to Kademlia. Kademlia may have dropped a retained address
// after a failed dial or never kept it for a full k-bucket, and a
// TTL refresh alone would never re-add it.
for existing in &mut state.addresses {
if reported.contains(&existing.address) {
existing.expires_at = now + IDENTIFY_ADDRESS_TTL;
changes.refreshed.push(existing.address.clone());
}
}
let mut new_addresses: Vec<Multiaddr> = reported
.iter()
.filter(|address| {
!state
.addresses
.iter()
.any(|existing| &existing.address == *address)
})
.cloned()
.collect();
// HashSet iteration is deliberately unordered. Sort new candidates so
// an oversized Identify report has deterministic admission behavior.
new_addresses.sort();
for address in new_addresses {
if state.new_addresses >= IDENTIFY_NEW_ADDRESS_LIMIT {
break;
}
let evicted = if state.addresses.len() < IDENTIFY_ADDRESS_LIMIT {
None
} else {
state
.addresses
.iter()
.position(|existing| !reported.contains(&existing.address))
.and_then(|index| state.addresses.remove(index))
};
if let Some(evicted) = evicted {
self.address_count -= 1;
changes.removed.push((peer_id, evicted.address));
} else if state.addresses.len() >= IDENTIFY_ADDRESS_LIMIT {
// Every retained address was refreshed in this report. Keep
// that stable subset and drop surplus new candidates.
break;
}
if self.address_count >= IDENTIFY_GLOBAL_ADDRESS_LIMIT {
// Reject the admission instead of evicting another peer's
// entry: cross-peer eviction lets a flood of new identities
// push honest peers' addresses out of the book and, when the
// evicted address is their last one, out of the Kademlia
// routing table. Per-peer eviction above still applies to the
// reporting peer's own addresses.
break;
}
state.new_addresses += 1;
state.addresses.push_back(IdentifyAddress {
address: address.clone(),
expires_at: now + IDENTIFY_ADDRESS_TTL,
});
self.address_count += 1;
changes.added.push(address);
}
// Do not keep a row when every candidate was rejected: a zero-address
// row is invisible to address_count and would otherwise linger until
// the expiry tick. This matches expire_peer, which drops empty rows.
if state.addresses.is_empty() {
self.peers.remove(&peer_id);
}
changes
}
fn expire(&mut self, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let mut removed = Vec::new();
let peers = self.peers.keys().copied().collect::<Vec<_>>();
for peer_id in peers {
removed.extend(self.expire_peer(peer_id, now));
}
removed
}
fn expire_peer(&mut self, peer_id: PeerId, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let expired = self
.peers
.get_mut(&peer_id)
.map(|state| Self::expire_state(state, now))
.unwrap_or_default();
self.address_count -= expired.len();
if self
.peers
.get(&peer_id)
.is_some_and(|state| state.addresses.is_empty())
{
self.peers.remove(&peer_id);
}
expired
.into_iter()
.map(|address| (peer_id, address))
.collect()
}
fn expire_state(state: &mut IdentifyPeerAddresses, now: Instant) -> Vec<Multiaddr> {
let mut expired = Vec::new();
let mut retained = VecDeque::with_capacity(state.addresses.len());
while let Some(address) = state.addresses.pop_front() {
if address.expires_at <= now {
expired.push(address.address);
} else {
retained.push_back(address);
}
}
state.addresses = retained;
expired
}
}
/// Combined libp2p behaviour.
#[derive(NetworkBehaviour)]
#[behaviour(prelude = "libp2p_swarm::derive_prelude")]
@@ -3960,6 +4165,14 @@ pub async fn start(
// Track in-flight GetRecord queries → reply channels
let mut pending_get_did: HashMap<kad::QueryId, oneshot::Sender<Option<DidRecord>>> =
HashMap::new();
// Keep Identify-derived entries across disconnects so a bootstrap address
// learned through Identify remains usable for redial. The TTL cleanup below
// removes entries that are no longer refreshed without touching explicit
// AddKnownPeer addresses.
let mut identify_addresses = IdentifyAddressBook::default();
let mut explicit_addresses: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
let mut identify_cleanup = tokio::time::interval(IDENTIFY_ADDRESS_CLEANUP_INTERVAL);
identify_cleanup.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
// Start the event loop as a background task
tokio::spawn(async move {
@@ -3979,6 +4192,13 @@ pub async fn start(
_ = ingest_sweep.tick() => {
run_ingest_sweep(&ingest_limiters, &replay_guard, ingest_now()).await;
}
_ = identify_cleanup.tick() => {
for (peer_id, address) in identify_addresses.expire(Instant::now()) {
if !is_explicit_address(&explicit_addresses, &peer_id, &address) {
swarm.behaviour_mut().kademlia.remove_address(&peer_id, &address);
}
}
}
// Graceful shutdown: exit the swarm loop when the
// process-wide signal flips. This drops the Swarm
// which closes all libp2p connections cleanly.
@@ -4113,7 +4333,21 @@ pub async fn start(
identify::Event::Received { peer_id, info, .. }
)) => {
debug!(peer = %peer_id, "identify received");
for addr in info.listen_addrs {
let changes = identify_addresses.update(
peer_id,
Instant::now(),
&info.listen_addrs,
);
for (removed_peer, addr) in changes.removed {
if !is_explicit_address(&explicit_addresses, &removed_peer, &addr)
{
swarm
.behaviour_mut()
.kademlia
.remove_address(&removed_peer, &addr);
}
}
for addr in changes.refreshed.into_iter().chain(changes.added) {
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
}
}
@@ -4150,6 +4384,14 @@ pub async fn start(
}
}
P2pCommand::AddKnownPeer { peer_id, addr } => {
let Ok(addr) = addr.clone().with_p2p(peer_id) else {
warn!(%peer_id, %addr, "dropping known-peer address with a foreign peer suffix");
continue;
};
explicit_addresses
.entry(peer_id)
.or_default()
.insert(addr.clone());
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
}
P2pCommand::Dial(addr) => {
@@ -4201,6 +4443,453 @@ pub async fn start(
mod tests {
use super::*;
fn identify_address(peer_id: PeerId, octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
.with_p2p(peer_id)
.unwrap()
}
fn identify_address_without_peer(octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
}
#[test]
fn identify_empty_reports_do_not_retain_peer_entries() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let foreign = identify_address(PeerId::random(), 1, 10_000);
for _ in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT + 1 {
let peer = PeerId::random();
assert_eq!(
book.update(peer, now, &[]),
IdentifyAddressChanges::default()
);
assert_eq!(
book.update(peer, now, std::slice::from_ref(&foreign)),
IdentifyAddressChanges::default()
);
}
assert!(book.peers.is_empty());
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_fully_rejected_peer_leaves_no_row_at_full_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// Fill the global budget with one-address peers.
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// A fresh peer reporting at saturation has every candidate rejected
// by the global-budget check. The row created by or_insert_with must
// be dropped: a zero-address row is invisible to address_count and
// would otherwise linger until the expiry tick.
let fresh = PeerId::random();
let fresh_address = identify_address(fresh, 2, 30_000);
let changes = book.update(fresh, now, std::slice::from_ref(&fresh_address));
assert!(changes.added.is_empty());
assert!(!book.peers.contains_key(&fresh));
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_update_reports_expired_addresses_as_removed() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
// An update past the TTL expires the stale address through
// expire_peer and reports it in changes.removed so the Identify
// handler can drop it from Kademlia.
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1),
std::slice::from_ref(&address),
);
assert_eq!(changes.removed, vec![(peer, address.clone())]);
assert_eq!(changes.added, vec![address]);
}
#[test]
fn identify_expire_drops_rows_emptied_by_expiry() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
assert!(book.peers.contains_key(&peer));
// Once every address in the row has expired, expire() must drop the
// row itself, matching the empty-row drop in update().
let removed = book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1));
assert_eq!(removed, vec![(peer, address)]);
assert!(!book.peers.contains_key(&peer));
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_report_processing_stops_at_the_input_limit() {
let peer = PeerId::random();
let now = Instant::now();
let foreign = identify_address(PeerId::random(), 1, 10_000);
let accepted = identify_address(peer, 2, 10_001);
let ignored = identify_address(peer, 3, 10_002);
let mut report = vec![foreign; IDENTIFY_REPORT_ADDRESS_LIMIT - 1];
report.push(accepted.clone());
report.push(ignored);
let mut book = IdentifyAddressBook::default();
let changes = book.update(peer, now, &report);
assert_eq!(changes.added, vec![accepted]);
assert!(changes.removed.is_empty());
assert_eq!(book.address_count, 1);
}
#[test]
fn identify_addresses_grow_without_eviction_below_the_peer_cap() {
let peer = PeerId::random();
let now = Instant::now();
let first = identify_address(peer, 1, 10_000);
let second = identify_address(peer, 2, 10_001);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&first));
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&second),
);
assert_eq!(changes.added, vec![second]);
assert!(changes.removed.is_empty());
assert_eq!(book.peers[&peer].addresses.len(), 2);
assert_eq!(book.address_count, 2);
}
#[test]
fn identify_global_budget_rejects_new_admissions() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let peers = (0..IDENTIFY_GLOBAL_ADDRESS_LIMIT)
.map(|index| {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
(peer, address)
})
.collect::<Vec<_>>();
let (peer, refreshed) = &peers[0];
let new_address = identify_address(*peer, 2, 30_000);
let changes = book.update(
*peer,
now + IDENTIFY_ADDRESS_WINDOW,
&[new_address.clone(), refreshed.clone()],
);
// The budget is full: the new address is rejected rather than
// evicting another peer's entry, and the refreshed address keeps its
// slot and is reported for re-admission to Kademlia.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![(*refreshed).clone()]);
assert_eq!(book.peers.len(), IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(
book.peers[peer]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![refreshed]
);
}
#[test]
fn identify_per_peer_eviction_applies_at_full_global_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// One filler peer goes in before the victim so the evicted address
// is not also the globally-oldest insertion. If saturation evicted
// the globally-oldest entry instead of the victim's own oldest,
// the sentinel's address would be the one removed.
let sentinel = PeerId::random();
let sentinel_address = identify_address(sentinel, 200, 19_999);
book.update(sentinel, now, std::slice::from_ref(&sentinel_address));
// The victim peer holds a full per-peer row while the rest of the
// global budget is filled by one-address peers.
let victim = PeerId::random();
let victim_addresses: Vec<Multiaddr> = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(victim, index as u8 + 1, 10_000 + index as u16))
.collect();
book.update(victim, now, &victim_addresses);
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT - IDENTIFY_ADDRESS_LIMIT - 1 {
let peer = PeerId::random();
let address = identify_address(peer, 200, 20_000 + index as u16);
book.update(peer, now, &[address]);
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// The victim reports a ninth address after the rate window rolls over.
// Per-peer eviction must still apply at global saturation: the oldest
// non-reported address is evicted to make room, and the global count
// is unchanged. This ordering only holds because eviction runs before
// the global-budget check; hoisting that check above the eviction
// block makes this test fail.
let ninth = identify_address(victim, 100, 30_000);
let changes = book.update(
victim,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&ninth),
);
assert_eq!(changes.added, vec![ninth]);
assert_eq!(changes.removed, vec![(victim, victim_addresses[0].clone())]);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.peers[&victim].addresses.len(), IDENTIFY_ADDRESS_LIMIT);
// The sentinel was inserted first; per-peer eviction must leave it
// alone. A global-oldest eviction policy would have removed it.
assert_eq!(
book.peers[&sentinel]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&sentinel_address]
);
}
#[test]
fn explicit_address_helper_detects_retained_addresses() {
let peer = PeerId::random();
let other = PeerId::random();
let address = identify_address(peer, 1, 10_000);
let mut explicit: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
// Unknown peer: nothing is explicit.
assert!(!is_explicit_address(&explicit, &peer, &address));
explicit.entry(peer).or_default().insert(address.clone());
assert!(is_explicit_address(&explicit, &peer, &address));
// The same address under a different peer is not retained.
assert!(!is_explicit_address(&explicit, &other, &address));
// A different address for the same peer is not retained.
assert!(!is_explicit_address(
&explicit,
&peer,
&identify_address(peer, 2, 10_001)
));
}
#[test]
fn identify_addresses_are_capped_with_fifo_eviction() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &initial);
assert_eq!(first.added, initial);
assert!(first.removed.is_empty());
let second = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
);
assert_eq!(second.added, vec![replacement]);
assert_eq!(second.removed, vec![(peer_id, initial[0].clone())]);
}
#[test]
fn identify_oversized_reports_keep_the_same_refreshed_subset() {
let peer_id = PeerId::random();
let now = Instant::now();
let mut report = (0..IDENTIFY_ADDRESS_LIMIT + 2)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &report);
assert_eq!(first.added.len(), IDENTIFY_ADDRESS_LIMIT);
assert_eq!(first.removed.len(), 0);
let retained = first.added.clone();
report.reverse();
let second = book.update(peer_id, now + IDENTIFY_ADDRESS_WINDOW, &report);
assert!(second.added.is_empty());
assert!(second.removed.is_empty());
let current = book
.peers
.get(&peer_id)
.unwrap()
.addresses
.iter()
.map(|entry| entry.address.clone())
.collect::<Vec<_>>();
assert_eq!(current, retained);
}
#[test]
fn identify_addresses_rate_limit_new_entries_across_updates() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_NEW_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(book.update(peer_id, now, &initial).added, initial);
assert!(book
.update(peer_id, now, std::slice::from_ref(&replacement))
.added
.is_empty());
assert_eq!(
book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
)
.added,
vec![replacement]
);
}
#[test]
fn identify_addresses_have_a_global_budget_across_peers() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let mut first = None;
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer_id = PeerId::random();
let address = identify_address(peer_id, (index % 254) as u8 + 1, 10_000 + index as u16);
if first.is_none() {
first = Some((peer_id, address.clone()));
}
assert!(book.update(peer_id, now, &[address]).removed.is_empty());
}
let (peer_id, first_address) = first.unwrap();
let address = identify_address(peer_id, 250, 30_000);
let changes = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&address),
);
// The global budget is full, so the admission is rejected and the
// peer's existing entry is left alone.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(
book.peers[&peer_id]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&first_address]
);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_addresses_refresh_and_expire() {
let peer_id = PeerId::random();
let now = Instant::now();
let address = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, now, std::slice::from_ref(&address))
.added,
vec![address.clone()]
);
let refreshed = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let refresh = book.update(peer_id, refreshed, std::slice::from_ref(&address));
assert!(refresh.added.is_empty());
assert!(refresh.removed.is_empty());
// Refreshed addresses are reported so the Identify handler re-adds
// them to Kademlia even though they were not newly admitted.
assert_eq!(refresh.refreshed, vec![address.clone()]);
assert!(book
.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1))
.is_empty());
assert_eq!(
book.expire(refreshed + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, address)]
);
}
#[test]
fn identify_refresh_reports_only_addresses_in_the_report() {
let peer_id = PeerId::random();
let now = Instant::now();
let reported = identify_address(peer_id, 1, 10_000);
let unreported = identify_address(peer_id, 2, 20_000);
let mut book = IdentifyAddressBook::default();
// Admit both addresses up front so the peer holds two.
assert_eq!(
book.update(peer_id, now, &[reported.clone(), unreported.clone()])
.added
.len(),
2
);
// A later report carrying only one of the two addresses must refresh
// just that one: pins the negative direction of the refresh
// predicate, so an address the peer stopped advertising keeps its
// original TTL instead of being refreshed back onto the Kademlia
// re-admit path on every report.
let report_at = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let changes = book.update(peer_id, report_at, std::slice::from_ref(&reported));
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![reported.clone()]);
// The unreported address still expires at its original TTL, while
// the refreshed address survives until its refreshed TTL.
assert_eq!(
book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, unreported)]
);
assert!(book.expire(report_at + Duration::from_secs(1)).is_empty());
assert_eq!(
book.expire(report_at + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, reported)]
);
}
#[test]
fn identify_addresses_reject_foreign_peer_suffixes() {
let peer_id = PeerId::random();
let other_peer_id = PeerId::random();
let address = identify_address(other_peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert!(book
.update(peer_id, Instant::now(), &[address])
.added
.is_empty());
}
#[test]
fn identify_addresses_canonicalize_the_peer_suffix_once() {
let peer_id = PeerId::random();
let raw = identify_address_without_peer(1, 10_000);
let canonical = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, Instant::now(), &[raw]).added,
vec![canonical]
);
}
#[test]
fn dir_mode_replace_authority_is_the_write_bits() {
assert!(!dir_mode_allows_untrusted_replace(0o755));
+1
View File
@@ -912,6 +912,7 @@ mod tests {
/// probe carries its oid on stdin rather than in argv, so an oid appears in the log only
/// once an object has already got past its probe, and a healthy object costs two
/// invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log)
.unwrap_or_default()
+15 -3
View File
@@ -412,7 +412,15 @@ pub fn build_router(state: AppState) -> Router {
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
.route("/api/v1/repos", get(repos::list_repos))
.route("/api/v1/repos/federated", get(repos::list_federated_repos))
.route(
"/api/v1/repos/federated",
get(repos::list_federated_repos)
.route_layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.route_layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.federated_rate_limiter.clone(),
trust: state.push_limiter_trust,
})),
)
.route("/api/v1/repos/{owner}/{repo}", get(repos::get_repo))
.route(
"/api/v1/repos/{owner}/{repo}/commits",
@@ -611,7 +619,7 @@ async fn ready(State(state): State<AppState>) -> axum::response::Response {
async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
let p2p_peer_id = state.p2p.as_ref().map(|h| h.local_peer_id.to_string());
Json(json!({
let mut body = json!({
"name": "gitlawb-node",
"version": env!("CARGO_PKG_VERSION"),
"did": state.node_did.to_string(),
@@ -620,7 +628,11 @@ async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
"auth": "http-signature-rfc9421",
"identity": "ed25519",
"p2p_peer_id": p2p_peer_id,
}))
});
if let Some(web_url) = &state.config.web_url {
body["web_url"] = json!(web_url);
}
Json(body)
}
pub(crate) async fn stats(State(state): State<AppState>) -> Json<serde_json::Value> {
+4
View File
@@ -190,6 +190,9 @@ pub struct AppState {
/// sink as trigger and accepts unsigned requests from known peers, so it is
/// braked too; each peer's distinct IP gets its own bucket.
pub peer_write_rate_limiter: RateLimiter,
/// Per-client-IP federation budget: 12 requests per minute, with at most
/// 10,000 tracked source keys. Shared by router instances and swept below.
pub federated_rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
@@ -351,6 +354,7 @@ impl AppState {
self.ipfs_work_rate_limiter.cleanup().await;
self.sync_trigger_rate_limiter.cleanup().await;
self.peer_write_rate_limiter.cleanup().await;
self.federated_rate_limiter.cleanup().await;
}
/// Trigger graceful shutdown. Idempotent — calling more than once
+781 -9
View File
@@ -14,6 +14,7 @@
use std::collections::HashMap;
use std::path::Path;
use std::path::PathBuf;
use std::sync::Arc;
use gitlawb_core::identity::Keypair;
@@ -435,9 +436,10 @@ async fn process_batch(
.await;
}
Err(e) => {
warn!(repo = %item.repo, origin = %origin_url, err = %e, "repo sync failed");
let _ = db.mark_sync_failed(&item.id).await;
crate::metrics::record_sync_processed("failed");
// Prune vs terminal classification lives in
// `handle_sync_item_error` (defer = stays pending for the
// next tick; anything else = failed).
handle_sync_item_error(db, &item, &origin_url, e).await;
}
}
}
@@ -691,6 +693,284 @@ async fn existing_promisor_state(repo: &str) -> PromisorProbe {
}
}
/// Byte-prefix check for the two node-managed blob namespaces exempt from
/// the fail-closed commit walk (`visibility_pack.rs`): request markers
/// (`refs/gitlawb/requests/`) and issue records (`refs/gitlawb/issues/`).
/// Bytewise so non-UTF-8 refnames (which git accepts) are classified, not
/// lossy-decoded into a false exempt match. Extracted so the predicate is
/// unit-testable; [`prune_non_exempt_gitlawb_refs`] enforces it on import.
fn gitlawb_ref_is_exempt(raw_refname: &[u8]) -> bool {
raw_refname.starts_with(b"refs/gitlawb/requests/")
|| raw_refname.starts_with(b"refs/gitlawb/issues/")
}
/// Parse one `git show-ref` output line into its refname bytes: `<hex-sha>`
/// (40-hex SHA-1 or 64-hex SHA-256, which a sha256 mirror inherits from its
/// origin), one space, then the refname. Empty lines (trailing newline) yield
/// `Ok(None)`; anything else-shaped is an anomaly the caller fails on rather
/// than skips — silently passing an unclassifiable line could leave an evil
/// ref imported. Extracted so malformed input is unit-testable without git.
fn show_ref_refname(line: &[u8]) -> anyhow::Result<Option<&[u8]>> {
if line.is_empty() {
return Ok(None);
}
let Some(sp) = line.iter().position(|b| *b == b' ') else {
return Err(anyhow::anyhow!("git show-ref emitted an unparsable line"));
};
let (sha, raw) = (&line[..sp], &line[sp + 1..]);
let sha_ok = (sha.len() == 40 || sha.len() == 64) && sha.iter().all(|b| b.is_ascii_hexdigit());
if !sha_ok || raw.is_empty() {
return Err(anyhow::anyhow!("git show-ref emitted an unparsable line"));
}
Ok(Some(raw))
}
/// Delete imported `refs/gitlawb/*` refs outside the two exempt subtrees.
///
/// Mirrors import whatever the origin advertises (`+refs/*:refs/*`), and
/// `transfer.hideRefs` is a serve-side knob: a repo's own `git fetch` never
/// consults local hideRefs, and origins hide only `requests/` anyway. So a
/// planted `refs/gitlawb/<other>` blob ref — creatable on any pre-gate node —
/// lands on every mirror and then trips `assert_all_refs_are_commits` closed,
/// wedging serving on each one. Prune at import instead: after clone/fetch,
/// list `refs/gitlawb` and delete every ref outside the exempt subtrees
/// (negative refspecs cannot re-include them, so post-import deletion is the
/// mechanism that works).
///
/// Callers map a transient prune error to [`PruneFailed`] so the batch
/// defers the row (stays `pending`, retried next tick) instead of failing
/// it: a failed row would leave a possibly-wedged mirror unretried until
/// the next peer announcement. Deterministic refusals (identity mismatch,
/// malformed enumeration) map to [`PruneInvalid`], which fails terminally
/// since no retry can clear them.
///
/// Enumeration is `git show-ref` (no pattern), parsed as raw bytes: refnames
/// cannot contain newlines, so one line per ref is lossless even for
/// non-UTF-8 names, which a lossy decode would miss. The `<sha> SP <refname>`
/// structure (not `for-each-ref --format=%(refname)`, which carries no object
/// id) is what lets the parser width-check 40-hex SHA-1 vs 64-hex SHA-256
/// and reject anything else-shaped instead of skipping it.
///
/// Fail-closed throughout: the exit code is classified (only `1` with empty
/// output means "no refs"; anything else non-zero, e.g. 128 on a missing
/// repo, is fatal, since a silent skip would leave an imported evil ref in
/// place while the sync reports success), and any line that is not
/// `<hex-sha> SP <refname>` — 40-hex SHA-1 or 64-hex SHA-256, which a sha256
/// mirror inherits from its origin — fails the sync rather than being
/// skipped into an unpruned wedge.
/// Bounded git spawn for the prune below: `kill_on_drop` plus a hard timeout
/// so a wedged child cannot stall a mirror import or sweep forever. The
/// refname delete runs on exact bytes, so args arrive as `OsStr`, not `str`.
/// Non-PruneInvalid failures (including a timeout) wrap as PruneFailed and
/// defer for retry, matching the transient classification below.
async fn prune_git(
local_path: &Path,
args: &[&std::ffi::OsStr],
label: &str,
) -> anyhow::Result<std::process::Output> {
let fut = async {
// allow-unbounded-git: bounded by the tokio::time::timeout wrapping
// this future below, plus kill_on_drop.
let mut cmd = tokio::process::Command::new("git");
cmd.kill_on_drop(true);
cmd.arg("-C").arg(local_path).args(args);
cmd.output().await
};
tokio::time::timeout(std::time::Duration::from_secs(30), fut)
.await
.map_err(|_| anyhow::anyhow!("{label} timed out"))?
.map_err(|e| anyhow::anyhow!("{label} failed to spawn: {e}"))
}
async fn prune_non_exempt_gitlawb_refs(local_path: &Path) -> anyhow::Result<()> {
let local_str = local_path.to_str().unwrap_or(".");
// Prove the path is itself a repo before trusting the enumeration:
// `git -C` walks up to the nearest ancestor repo, so inside a non-repo
// child (a partially-created mirror) `show-ref` would exit 0 with the
// ANCESTOR's refs and `update-ref -d` would then run against that repo.
// Require `rev-parse --absolute-git-dir` to resolve to the path itself
// (mirrors are bare repos, where the git dir IS the path).
let git_dir_out = prune_git(
local_path,
&[
std::ffi::OsStr::new("rev-parse"),
std::ffi::OsStr::new("--absolute-git-dir"),
],
"git rev-parse",
)
.await?;
if !git_dir_out.status.success() {
let stderr = String::from_utf8_lossy(&git_dir_out.stderr);
return Err(anyhow::anyhow!(PruneInvalid(format!(
"mirror path is not a git repo (rev-parse failed): {stderr}"
))));
}
let actual_dir = PathBuf::from(
String::from_utf8_lossy(&git_dir_out.stdout)
.trim()
.to_string(),
);
let canonical_path = local_path
.canonicalize()
.map_err(|e| anyhow::anyhow!(PruneInvalid(format!("mirror path does not resolve: {e}"))))?;
let canonical_git_dir = actual_dir
.canonicalize()
.map_err(|e| anyhow::anyhow!(PruneInvalid(format!("git dir does not resolve: {e}"))))?;
if canonical_git_dir != canonical_path {
return Err(anyhow::anyhow!(PruneInvalid(format!(
"mirror path {local_str} is not itself a repo (git dir resolves elsewhere); refusing to enumerate"
))));
}
let out = prune_git(
local_path,
&[std::ffi::OsStr::new("show-ref")],
"git show-ref",
)
.await?;
// Classify on the exit code (the `existing_promisor_state` precedent):
// 0 parses, 1 with empty output is an empty repo, anything else is a
// fatal enumeration failure that must not read as "nothing to prune".
match out.status.code() {
Some(0) => {}
Some(1) if out.stdout.is_empty() => return Ok(()),
_ => {
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!(
"git show-ref failed with code {:?}: {stderr}",
out.status.code()
));
}
}
for line in out.stdout.split(|b| *b == b'\n') {
// Malformed lines are deterministic (same git, same output every
// tick): fail terminally via PruneInvalid, not transiently.
let Some(raw) =
show_ref_refname(line).map_err(|e| anyhow::anyhow!(PruneInvalid(e.to_string())))?
else {
continue;
};
if !(raw == b"refs/gitlawb" || raw.starts_with(b"refs/gitlawb/"))
|| gitlawb_ref_is_exempt(raw)
{
continue;
}
// Delete by exact bytes (not the lossy display string below): a
// non-UTF-8 refname would otherwise survive the prune.
let refname_os = os_from_raw_refname(raw);
let shown = refname_os.to_string_lossy().into_owned();
let delete = prune_git(
local_path,
&[
std::ffi::OsStr::new("update-ref"),
std::ffi::OsStr::new("-d"),
refname_os.as_os_str(),
std::ffi::OsStr::new("--no-deref"),
],
"git update-ref -d",
)
.await?;
if !delete.status.success() {
let stderr = String::from_utf8_lossy(&delete.stderr);
return Err(anyhow::anyhow!(
"git update-ref -d {shown} failed (mirror still carries a non-exempt refs/gitlawb ref): {stderr}"
));
}
tracing::info!(
gitlawb_ref = %shown,
"mirror: pruned non-exempt refs/gitlawb ref on import"
);
}
Ok(())
}
/// Lossless bytes → OsString for `update-ref -d`: non-UTF-8 refnames must be
/// deleted by exact bytes. Unix is the serving platform; elsewhere fall back
/// to a lossy decode (best effort where exact bytes are unrepresentable).
#[cfg(unix)]
fn os_from_raw_refname(raw: &[u8]) -> std::ffi::OsString {
use std::os::unix::ffi::OsStringExt;
std::ffi::OsString::from_vec(raw.to_vec())
}
#[cfg(not(unix))]
fn os_from_raw_refname(raw: &[u8]) -> std::ffi::OsString {
String::from_utf8_lossy(raw).into_owned().into()
}
/// Marker for mirror-prune failures: the clone/fetch succeeded, so the
/// mirror is on disk, but a non-exempt `refs/gitlawb/*` ref may still be
/// imported. Local-git conditions are transient (a retry can succeed), while
/// `mark_sync_failed` is terminal until the next peer announcement — and a
/// failed row would leave a possibly-wedged mirror unretried in the
/// meantime. The batch error arm downcasts to this and defers the row
/// (stays `pending`, retried on the next 30s tick, where fetch+prune are
/// idempotent) instead of failing it.
#[derive(Debug)]
struct PruneFailed(String);
impl std::fmt::Display for PruneFailed {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "mirror prune failed: {}", self.0)
}
}
impl std::error::Error for PruneFailed {}
/// Marker for deterministic mirror-prune refusals: the identity check (path
/// is not itself a repo) and malformed `show-ref` output cannot clear on
/// retry — re-running fetch+prune every tick would spin forever on a
/// permanently bad path. Unlike [`PruneFailed`], these fail the row
/// terminally so the condition surfaces instead of looping silently.
#[derive(Debug)]
struct PruneInvalid(String);
/// Preserve a deterministic prune refusal through the clone/fetch call
/// sites: `prune_non_exempt_gitlawb_refs` already returns `PruneInvalid`
/// for identity mismatches and malformed enumeration, and re-wrapping
/// every error as `PruneFailed` would stringify that type away before
/// `handle_sync_item_error` downcasts it — leaving a permanently invalid
/// mirror deferred (fetch+prune every tick) instead of failed. Transient
/// failures keep the `PruneFailed` deferral.
fn wrap_prune_error(e: anyhow::Error) -> anyhow::Error {
match e.downcast::<PruneInvalid>() {
Ok(invalid) => anyhow::Error::new(invalid),
Err(other) => anyhow::anyhow!(PruneFailed(other.to_string())),
}
}
impl std::fmt::Display for PruneInvalid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "mirror prune refused: {}", self.0)
}
}
impl std::error::Error for PruneInvalid {}
/// Classify a mirror sync failure for one queue row. Transient prune
/// failures ([`PruneFailed`]) defer: the mirror is on disk but may still
/// carry a non-exempt refs/gitlawb ref, a transient local-git condition a
/// retry can clear — failing the row would leave it unretried until the
/// next peer announcement, wedging serving in the meantime. Deterministic
/// refusals ([`PruneInvalid`]) and everything else fail the row terminally:
/// retrying a permanently bad path every tick would spin forever instead of
/// surfacing. Extracted so the deferral (not just the prune itself) is
/// unit-testable.
async fn handle_sync_item_error(
db: &Db,
item: &crate::db::SyncQueueItem,
origin_url: &str,
err: anyhow::Error,
) {
if err.downcast_ref::<PruneFailed>().is_some() {
warn!(repo = %item.repo, origin = %origin_url, err = %err, "mirror prune failed; leaving sync row pending for retry");
crate::metrics::record_sync_processed("deferred");
return;
}
warn!(repo = %item.repo, origin = %origin_url, err = %err, "repo sync failed");
let _ = db.mark_sync_failed(&item.id).await;
crate::metrics::record_sync_processed("failed");
}
/// Mirror-clone a repo from a remote URL into a local bare repo.
/// `Promisor` mode adds `--filter=blob:limit=10g`, which marks the repo a git
/// promisor (so a pack with origin-omitted withheld blobs is accepted) while
@@ -715,6 +995,13 @@ async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> an
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!("git clone --mirror failed: {stderr}"));
}
// A fresh mirror imports everything the origin advertises, including any
// planted non-exempt `refs/gitlawb/*` ref; prune before serving from it.
// Prune outcome flows through `wrap_prune_error`: transient failures
// defer (stays pending), deterministic refusals fail terminally.
prune_non_exempt_gitlawb_refs(local_path)
.await
.map_err(wrap_prune_error)?;
Ok(())
}
@@ -735,7 +1022,11 @@ async fn fetch_repo(local_path: &Path, remote_url: &str, mode: MirrorMode) -> an
])
.await?;
match mode {
// The fetch imports whatever the origin advertises via the stored
// `+refs/*:refs/*` refspec; prune non-exempt `refs/gitlawb/*` refs after
// every fetch so one planted origin ref cannot wedge serving on the
// mirror (see `prune_non_exempt_gitlawb_refs`).
let fetch_result = match mode {
MirrorMode::Promisor => {
git_run(&["-C", local_str, "config", "remote.origin.promisor", "true"]).await?;
git_run(&[
@@ -775,7 +1066,13 @@ async fn fetch_repo(local_path: &Path, remote_url: &str, mode: MirrorMode) -> an
git_run(&["-C", local_str, "fetch", "--prune", "origin"]).await
}
}
}
};
fetch_result?;
// Prune after every fetch (see above); outcome flows through
// `wrap_prune_error` like the clone path.
prune_non_exempt_gitlawb_refs(local_path)
.await
.map_err(wrap_prune_error)
}
#[cfg(test)]
@@ -1330,6 +1627,18 @@ mod tests {
.await;
}
/// Use a rooted path without a Windows drive prefix in the remote fixture.
/// The destination still resolves on the temp directory's current drive,
/// while the remote's relative path contains no illegal colon component.
fn absolute_slug_path(path: &Path) -> String {
assert!(path.is_absolute());
path.components()
.filter(|part| !matches!(part, std::path::Component::Prefix(_)))
.collect::<std::path::PathBuf>()
.to_string_lossy()
.replace('\\', "/")
}
#[sqlx::test]
async fn process_batch_rejects_slug_escaping_repos_dir(pool: PgPool) {
// The verified escape from #272: `PathBuf::join` discards everything
@@ -1345,12 +1654,13 @@ mod tests {
// fails on but leaves the parent it created, so assert on both.
let outside = TempDir::new().unwrap();
let escape_dir = outside.path().join("nest");
let slug = format!("a/{}/escape", escape_dir.display());
let escape_path = absolute_slug_path(&escape_dir);
let slug = format!("a/{escape_path}/escape");
let escape_target = escape_dir.join("escape.git");
// Serve the composed URL for real, so a run without the guard genuinely
// clones outside the root rather than merely failing at git.
let rel = format!("a{}/escape", escape_dir.display());
let rel = format!("a{escape_path}/escape");
let (_remote, peer_url) = rooted_remote(&[&rel]);
let did = "did:key:z6MkAttacker";
@@ -1470,6 +1780,7 @@ mod tests {
// ── canonical containment before the git call (issue #272) ───────────────
/// Every ref in `repo`, as one string, for a before/after comparison.
#[cfg(unix)]
fn refs_of(repo: &Path) -> String {
let out = Command::new("git")
.args(["-C", repo.to_str().unwrap(), "for-each-ref"])
@@ -1802,6 +2113,7 @@ mod tests {
/// starvation tests is fixed rather than dependent on how fast the loop
/// runs. Two rows enqueued in the same microsecond would otherwise order
/// arbitrarily.
#[cfg(unix)]
async fn enqueue_at(db: &Db, pool: &PgPool, repo: &str, did: &str, enqueued_at: &str) {
enqueue(db, repo, did).await;
sqlx::query("UPDATE sync_queue SET enqueued_at = $1 WHERE repo = $2")
@@ -1995,7 +2307,8 @@ mod tests {
let outside = TempDir::new().unwrap();
let escape_dir = outside.path().join("nest");
let slug = format!("a/{}/gitlawb-probe", escape_dir.display());
let escape_path = absolute_slug_path(&escape_dir);
let slug = format!("a/{escape_path}/gitlawb-probe");
let escape_target = escape_dir.join("gitlawb-probe.git");
// Two things this fixture must get right or the test is green for the
@@ -2006,7 +2319,7 @@ mod tests {
// run without the guard genuinely clones outside the root instead of
// just failing at git. Db::upsert_peer cannot seed this row: it gates on
// is_public_http_url, which rejects file://.
let rel = format!("a{}/gitlawb-probe", escape_dir.display());
let rel = format!("a{escape_path}/gitlawb-probe");
let (_remote, peer_url) = rooted_remote(&[&rel]);
let did = "did:key:z6MkAttacker";
seed_local_peer(&pool, did, &peer_url).await;
@@ -2092,4 +2405,463 @@ mod tests {
assert!(object_count(&mirror) > 0, "mirror has no objects");
assert_eq!(sync_status(&pool, "z6Mkfoo/hello").await, "done");
}
/// The import-exemption predicate is bytewise over both node-managed
/// namespaces and nothing else. Non-UTF-8 bytes never match (git accepts
/// them, so a lossy decode could false-exempt). Narrowing either prefix
/// turns these red.
#[test]
fn gitlawb_import_exemption_covers_both_namespaces_by_bytes() {
for exempt in [
&b"refs/gitlawb/requests/550e8400-e29b-41d4-a716-446655440000"[..],
&b"refs/gitlawb/issues/abc123"[..],
] {
assert!(
gitlawb_ref_is_exempt(exempt),
"import must keep {}",
String::from_utf8_lossy(exempt)
);
}
for pruned in [
&b"refs/gitlawb/evil"[..],
&b"refs/gitlawb"[..],
&b"refs/gitlawbfoo"[..],
&b"refs/heads/main"[..],
&b""[..],
// Raw 0xFF byte: never an exempt prefix, always pruned.
&b"refs/gitlawb/\xffvil"[..],
] {
assert!(
!gitlawb_ref_is_exempt(pruned),
"import must prune {}",
String::from_utf8_lossy(pruned)
);
}
}
/// End-to-end prune on a scratch bare repo: a planted non-exempt
/// `refs/gitlawb/*` blob ref is deleted, exempt `requests/`/`issues/`
/// blob refs and content refs survive. Removing the prune call from
/// `clone_repo`/`fetch_repo` leaves the evil ref in place, which is what
/// wedges serving on mirrors — this test plants exactly that state and
/// proves the prune removes it.
#[test]
fn prune_removes_non_exempt_gitlawb_refs_and_keeps_exempt() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("m.git");
std::fs::create_dir_all(&work).unwrap();
// A real commit for the content branch (branch refs must point
// at commits; the evil/exempt refs below point at blobs).
std::fs::write(work.join("f.txt"), b"x\n").unwrap();
let run_work = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&work)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
run_work(&["init", "-q"]);
run_work(&["config", "user.email", "t@t"]);
run_work(&["config", "user.name", "t"]);
run_work(&["add", "f.txt"]);
run_work(&["commit", "-qm", "init"]);
assert!(
Command::new("git")
.args([
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap()
])
.current_dir(td.path())
.status()
.unwrap()
.success(),
"git clone --bare failed"
);
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&bare)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
let run_out = |args: &[&str]| {
let out = Command::new("git")
.args(args)
.current_dir(&bare)
.output()
.unwrap();
assert!(out.status.success(), "git {args:?} failed");
out.stdout
};
run(&["init", "-q", "--bare", "."]);
std::fs::write(bare.join("blob-body"), b"marker\n").unwrap();
let blob = String::from_utf8_lossy(&run_out(&["hash-object", "-w", "blob-body"]))
.trim()
.to_string();
// The clone carries the content branch; plant the evil ref and
// both exempt refs (all blob-pointing, like node metadata).
run(&["update-ref", "refs/gitlawb/evil", &blob]);
run(&["update-ref", "refs/gitlawb/requests/r1", &blob]);
run(&["update-ref", "refs/gitlawb/issues/i1", &blob]);
prune_non_exempt_gitlawb_refs(&bare)
.await
.expect("prune succeeds");
let raw_refs = run_out(&["for-each-ref", "--format=%(refname)"]);
let refs = String::from_utf8_lossy(&raw_refs);
let set: std::collections::HashSet<&str> = refs
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
assert!(
!set.contains("refs/gitlawb/evil"),
"non-exempt blob ref must be pruned, still present: {set:?}"
);
for keep in ["refs/gitlawb/requests/r1", "refs/gitlawb/issues/i1"] {
assert!(set.contains(keep), "{keep} must survive the prune: {set:?}");
}
assert!(
set.iter().any(|r| r.starts_with("refs/heads/")),
"content branch ref must survive the prune: {set:?}"
);
});
}
/// `show-ref` line parser: 40-hex SHA-1 and 64-hex SHA-256 lines yield
/// the refname, empty lines yield None, and anything else-shaped errors
/// (the caller fails the sync rather than skipping into an unpruned
/// wedge). Reverting to a 40-only width check turns the sha256 case red.
#[test]
fn show_ref_line_parser_accepts_both_hash_widths_and_rejects_junk() {
// 40-hex SHA-1 line.
let sha1_line = format!("{} refs/gitlawb/evil", "a".repeat(40));
assert_eq!(
show_ref_refname(sha1_line.as_bytes()).unwrap(),
Some(b"refs/gitlawb/evil".as_slice())
);
// 64-hex SHA-256 line (a sha256 mirror inherits the width).
let sha256_line = format!("{} refs/gitlawb/evil", "b".repeat(64));
assert_eq!(
show_ref_refname(sha256_line.as_bytes()).unwrap(),
Some(b"refs/gitlawb/evil".as_slice())
);
// Non-UTF-8 refname passes through as exact bytes.
let mut raw = vec![b'c'; 40];
raw.extend_from_slice(b" refs/gitlawb/\xffvil");
assert_eq!(
show_ref_refname(&raw).unwrap(),
Some(b"refs/gitlawb/\xffvil".as_slice())
);
// Trailing newline residue.
assert_eq!(show_ref_refname(b"").unwrap(), None);
// No space, short sha, non-hex sha, empty refname: all errors.
let bads: Vec<Vec<u8>> = vec![
b"nospaceshere".to_vec(),
format!("{}refs/x", "a".repeat(40)).into_bytes(),
[b'z'; 40]
.iter()
.chain(b" refs/x".iter())
.copied()
.collect(),
format!("{} ", "a".repeat(40)).into_bytes(),
format!("{} refs/x", "a".repeat(41)).into_bytes(),
];
for bad in &bads {
assert!(
show_ref_refname(bad).is_err(),
"malformed show-ref line must error, not skip"
);
}
}
/// Pruning a path that is not a repo errors. Two shapes: a missing path
/// fails `rev-parse` outright (exit 128), while a plain subdirectory of
/// a repo would otherwise enumerate the ancestor's refs — covered by the
/// identity test below. Either way the prune refuses instead of pruning
/// the wrong repo or silently skipping an evil ref.
#[test]
fn prune_on_non_repo_path_errors() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let err =
prune_non_exempt_gitlawb_refs(Path::new("/tmp/gitlawb-definitely-not-a-repo-xyz"))
.await
.expect_err("non-repo path must fail the prune, not skip it");
assert!(
err.to_string().contains("not a git repo")
|| err.to_string().contains("does not resolve"),
"unexpected error shape: {err:#}"
);
});
}
/// Identity check: pruning a plain subdirectory of a repo refuses instead
/// of enumerating the ancestor. `git -C` on the child exits 0 from
/// `show-ref` with the ANCESTOR's refs, so without the
/// `rev-parse --absolute-git-dir` identity check the delete loop would
/// enumerate and delete the wrong repository's refs. Deleting the check
/// makes the prune succeed and removes the ancestor's evil ref — both
/// assertions go red.
#[test]
fn prune_on_non_repo_child_of_repo_refuses_and_touches_nothing() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let td = TempDir::new().unwrap();
let ancestor = td.path().join("ancestor.git");
std::fs::create_dir_all(&ancestor).unwrap();
let run_ancestor = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&ancestor)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
run_ancestor(&["init", "-q", "--bare", "."]);
// Blob target for the planted ref (mirrors the node-metadata shape).
std::fs::write(ancestor.join("blob-body"), b"marker\n").unwrap();
let out = Command::new("git")
.args(["hash-object", "-w", "blob-body"])
.current_dir(&ancestor)
.output()
.unwrap();
assert!(out.status.success(), "git hash-object failed");
let blob = String::from_utf8_lossy(&out.stdout).trim().to_string();
run_ancestor(&["update-ref", "refs/gitlawb/evil", &blob]);
// A plain child directory: not itself a repo.
let child = ancestor.join("not-a-repo");
std::fs::create_dir_all(&child).unwrap();
let err = prune_non_exempt_gitlawb_refs(&child)
.await
.expect_err("pruning a non-repo child must refuse, not enumerate the ancestor");
assert!(
err.to_string().contains("not itself a repo"),
"refusal must name the identity mismatch, got: {err:#}"
);
// The ancestor's evil ref is untouched.
let refs_out = Command::new("git")
.args(["for-each-ref", "--format=%(refname)"])
.current_dir(&ancestor)
.output()
.unwrap();
let refs = String::from_utf8_lossy(&refs_out.stdout);
assert!(
refs.lines().any(|l| l.trim() == "refs/gitlawb/evil"),
"ancestor evil ref must survive the refused prune"
);
});
}
/// sha256 mirror: a 64-hex `show-ref` line parses and the planted ref is
/// pruned. A 40-only parser skips every line and leaves the evil ref.
#[test]
fn prune_parses_sha256_show_ref_and_prunes() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let td = TempDir::new().unwrap();
let bare = td.path().join("m256.git");
std::fs::create_dir_all(&bare).unwrap();
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&bare)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
let run_out = |args: &[&str]| {
let out = Command::new("git")
.args(args)
.current_dir(&bare)
.output()
.unwrap();
assert!(out.status.success(), "git {args:?} failed");
out.stdout
};
run(&["init", "-q", "--bare", "--object-format=sha256", "."]);
std::fs::write(bare.join("blob-body"), b"marker\n").unwrap();
let blob = String::from_utf8_lossy(&run_out(&["hash-object", "-w", "blob-body"]))
.trim()
.to_string();
assert_eq!(
blob.len(),
64,
"sanity: sha256 repo must mint 64-hex objects"
);
run(&["update-ref", "refs/gitlawb/evil", &blob]);
run(&["update-ref", "refs/gitlawb/issues/i1", &blob]);
prune_non_exempt_gitlawb_refs(&bare)
.await
.expect("prune succeeds on a sha256 repo");
let raw_refs = run_out(&["for-each-ref", "--format=%(refname)"]);
let refs = String::from_utf8_lossy(&raw_refs);
let set: std::collections::HashSet<&str> = refs
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
assert!(
!set.contains("refs/gitlawb/evil"),
"evil ref must be pruned from sha256 mirror: {set:?}"
);
assert!(
set.contains("refs/gitlawb/issues/i1"),
"exempt ref must survive: {set:?}"
);
});
}
/// Prune failures defer the queue row (stays `pending` for the next
/// tick); deterministic prune refusals and any other sync error fail it
/// terminally. Deleting the `downcast_ref::<PruneFailed>` arm restores
/// terminal failure and turns the first assertion red.
/// Prune failures defer the queue row (stays `pending` for the next
/// tick); deterministic prune refusals and any other sync error fail it
/// terminally. Deleting the `downcast_ref::<PruneFailed>` arm restores
/// terminal failure and turns the first assertion red.
///
/// The wrapper below is what the clone/fetch call sites run through:
/// a `PruneInvalid` from the helper must still downcast as
/// `PruneInvalid` afterwards (not stringified into `PruneFailed`),
/// and anything else must arrive as `PruneFailed`.
#[test]
fn wrap_prune_error_preserves_deterministic_type() {
let invalid = wrap_prune_error(anyhow::anyhow!(PruneInvalid("x".to_string())));
assert!(
invalid.downcast_ref::<PruneInvalid>().is_some(),
"deterministic refusal must survive the call-site wrap"
);
assert!(
invalid.downcast_ref::<PruneFailed>().is_none(),
"deterministic refusal must not also read as transient"
);
let transient = wrap_prune_error(anyhow::anyhow!("boom"));
assert!(
transient.downcast_ref::<PruneFailed>().is_some(),
"generic failure must arrive as transient"
);
assert!(
transient.downcast_ref::<PruneInvalid>().is_none(),
"generic failure must not read as deterministic"
);
}
#[sqlx::test]
async fn prune_failure_defers_row_other_failures_fail_it(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let item = |id: &str| crate::db::SyncQueueItem {
id: id.to_string(),
repo: "z6Mkfoo/hello".to_string(),
node_did: "did:key:z6MkOrigin".to_string(),
ref_name: "refs/heads/main".to_string(),
new_sha: "0".repeat(40),
cid: None,
status: "pending".to_string(),
enqueued_at: chrono::Utc::now().to_rfc3339(),
};
async fn status_of(pool: &sqlx::PgPool, id: &str) -> String {
let row: (String,) = sqlx::query_as("SELECT status FROM sync_queue WHERE id = $1")
.bind(id)
.fetch_one(pool)
.await
.unwrap();
row.0
}
// Seed three pending rows directly (enqueue_sync mints random ids).
for id in ["sync-prune-defer", "sync-prune-invalid", "sync-other-fail"] {
sqlx::query(
"INSERT INTO sync_queue (id, repo, node_did, ref_name, new_sha, cid, status, enqueued_at)
VALUES ($1, $2, $3, $4, $5, NULL, 'pending', $6)",
)
.bind(id)
.bind("z6Mkfoo/hello")
.bind("did:key:z6MkOrigin")
.bind("refs/heads/main")
.bind("0".repeat(40))
.bind(chrono::Utc::now().to_rfc3339())
.execute(state.db.pool())
.await
.unwrap();
}
handle_sync_item_error(
&state.db,
&item("sync-prune-defer"),
"http://origin.example",
anyhow::anyhow!(PruneFailed("boom".to_string())),
)
.await;
assert_eq!(
status_of(state.db.pool(), "sync-prune-defer").await,
"pending",
"prune failure must leave the row pending for retry"
);
handle_sync_item_error(
&state.db,
&item("sync-other-fail"),
"http://origin.example",
anyhow::anyhow!("boom"),
)
.await;
assert_eq!(
status_of(state.db.pool(), "sync-other-fail").await,
"failed",
"non-prune failure must fail the row"
);
// Deterministic refusal: no retry can clear it, so it fails
// terminally instead of spinning every tick forever.
handle_sync_item_error(
&state.db,
&item("sync-prune-invalid"),
"http://origin.example",
anyhow::anyhow!(PruneInvalid("not itself a repo".to_string())),
)
.await;
assert_eq!(
status_of(state.db.pool(), "sync-prune-invalid").await,
"failed",
"deterministic prune refusal must fail the row, not defer forever"
);
}
}
+75
View File
@@ -0,0 +1,75 @@
//! Native Git fixtures shared by timing tests on Unix and Windows.
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// The fixture delays or hangs in its own process, so the production watchdog
/// can terminate it without depending on a shell or descendant process support.
pub(super) enum Behavior<'a> {
Delay(u64),
Hang,
HangOid(&'a str),
HangRepo(&'a str),
LogTypes(&'a Path),
}
pub(super) struct GitShim {
_directory: tempfile::TempDir,
executable: PathBuf,
}
impl Deref for GitShim {
type Target = Path;
fn deref(&self) -> &Path {
&self.executable
}
}
pub(super) fn create(name: &str, behavior: Behavior<'_>) -> GitShim {
static COMPILED: OnceLock<tempfile::TempDir> = OnceLock::new();
let compiled = COMPILED.get_or_init(|| {
let directory = tempfile::tempdir().expect("native Git fixture directory");
let source = directory.path().join("git_shim.rs");
std::fs::write(&source, include_str!("../tests/fixtures/git_shim.rs"))
.expect("write native Git fixture source");
let executable = directory
.path()
.join(format!("git-shim{}", std::env::consts::EXE_SUFFIX));
let output =
std::process::Command::new(std::env::var_os("RUSTC").unwrap_or_else(|| "rustc".into()))
.arg("--edition=2021")
.arg("-Dwarnings")
.arg(&source)
.arg("-o")
.arg(executable)
.output()
.expect("compile native Git fixture with the installed Rust toolchain");
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
directory
});
let directory = tempfile::Builder::new()
.prefix(name)
.tempdir()
.expect("Git fixture instance");
let file_name = format!("git-shim{}", std::env::consts::EXE_SUFFIX);
let executable = directory.path().join(&file_name);
std::fs::copy(compiled.path().join(file_name), &executable).expect("copy native Git fixture");
let config = match behavior {
Behavior::Delay(milliseconds) => format!("delay\n{milliseconds}"),
Behavior::Hang => "hang\n".to_owned(),
Behavior::HangOid(oid) => format!("hang-oid\n{oid}"),
Behavior::HangRepo(repo) => format!("hang-repo\n{repo}"),
Behavior::LogTypes(path) => format!("log-types\n{}", path.display()),
};
std::fs::write(directory.path().join("config"), config).expect("configure native Git fixture");
GitShim {
_directory: directory,
executable,
}
}
+814 -64
View File
@@ -28,6 +28,9 @@ use gitlawb_core::identity::Keypair;
use crate::auth::AuthenticatedDid;
use crate::state::AppState;
#[path = "test_git_shim.rs"]
mod git_shim;
/// Build an [`AppState`] over a real, migrated Postgres pool (from `#[sqlx::test]`).
/// Runs the schema migrations first, because the per-test database starts empty.
///
@@ -116,6 +119,7 @@ fn build_state(db: Arc<crate::db::Db>, pool: PgPool) -> AppState {
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new_bounded(12, Duration::from_secs(60), 10_000),
shutdown_tx: tokio::sync::watch::channel(false).0,
// Generous — no test drives the handler-level shed (git_permit is unit-tested).
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
@@ -558,11 +562,12 @@ mod tests {
/// #174 (SC1, load-bearing): a saturated READ pool must NOT shed an
/// authenticated push — the write pool is a separate budget. Read pool at zero,
/// write pool with capacity: the push proceeds PAST admission (it then errors on
/// the placeholder DB, but crucially it is not a 503). Route git-receive-pack
/// the closed DB, with db_unavailable rather than overloaded). Route git-receive-pack
/// back to the read pool and this goes red — that is the isolation proof.
#[tokio::test]
async fn git_receive_pack_not_shed_by_exhausted_read_pool() {
let mut state = test_state_lazy();
state.db.pool().close().await;
// Read pool exhausted as if a flood of anonymous clones held every slot.
state.git_read_semaphore = Arc::new(tokio::sync::Semaphore::new(0));
// Write pool keeps its default capacity from test_state_lazy.
@@ -584,10 +589,13 @@ mod tests {
.await
.unwrap();
assert_ne!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"an exhausted READ pool must not shed a push — the write pool is a separate budget (#174)"
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(
body["error"],
crate::error::DB_UNAVAILABLE_CODE,
"the push must clear admission and reach the deliberately closed database"
);
}
@@ -1036,6 +1044,345 @@ mod tests {
);
}
/// #496: create_task must reject a caller-supplied repo_id that names a
/// hosted repo the caller does not own. A stranger naming the victim's repo
/// gets 403, leaking nothing about the repo and persisting nothing; the
/// owner files (201). Repo-less tasks stay allowed; an id naming no hosted
/// repo (unknown, or quarantined, hidden as if it did not exist) is stored
/// verbatim as an opaque label, and the 403-vs-201 split between a hosted
/// id and an unknown one is itself a one-bit oracle. This is a write gate:
/// task reads are caller-independent and not repo-scoped on this base.
#[sqlx::test]
async fn create_task_rejects_foreign_repo_id(pool: PgPool) {
let owner = "did:key:zTASKREPOOWNERAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zTASKREPOSTRANGERBBBBBBBBBBBBBBBBBBBBBB";
let state = test_state(pool).await;
let repo = seed_repo(owner, "task-gate-repo");
state.db.create_repo(&repo).await.expect("seed repo");
let router = || {
Router::new()
.route(
"/api/v1/tasks",
axum::routing::post(crate::api::tasks::create_task),
)
.with_state(state.clone())
};
let post_as = |signer: &str, body: String| {
router().oneshot(signed_request_as(
signer,
Method::POST,
"/api/v1/tasks",
Body::from(body),
))
};
let body_with = |signer: &str, repo_id: Option<&str>| {
let repo_field = repo_id
.map(|id| format!(r#","repo_id":"{id}""#))
.unwrap_or_default();
format!(
r#"{{"kind":"build","capability":"repo:write","delegator_did":"{signer}"{repo_field}}}"#
)
};
let created_id = |bytes: &[u8]| -> String {
let v: serde_json::Value = serde_json::from_slice(bytes).expect("task json");
v["id"].as_str().expect("task id").to_string()
};
// Stranger filing under the victim's repo id → exact 403.
let resp = post_as(stranger, body_with(stranger, Some(&repo.id)))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::FORBIDDEN,
"a non-owner must not file tasks against a foreign repo id"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8_lossy(&bytes);
assert!(
text.contains(r#""error":"forbidden""#),
"denial must keep the forbidden envelope: {text}"
);
assert!(
text.contains(crate::api::REPO_OWNER_ONLY_MESSAGE),
"denial must carry the shared owner-denial text (same as GraphQL): {text}"
);
// The denial must not persist: no task row may carry the victim's
// repo id afterwards — a 403 that still writes is the injection.
let stored = state
.db
.list_tasks(None, None, 50)
.await
.expect("list tasks");
assert!(
stored
.iter()
.all(|t| t.repo_id.as_deref() != Some(repo.id.as_str())),
"a denied write must not leave a task under the foreign repo id"
);
// Owner filing under their own repo id → 201.
let resp = post_as(owner, body_with(owner, Some(&repo.id)))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::CREATED,
"the owner must be able to file tasks against their repo"
);
// Repo-less task → 201 (unaffected).
let resp = post_as(stranger, body_with(stranger, None)).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::CREATED,
"repo-less task creation must keep working"
);
// Unknown repo id → 201 as an opaque label.
let resp = post_as(stranger, body_with(stranger, Some("no-such-repo-id")))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::CREATED,
"an id naming no hosted repo stays an opaque label"
);
// Read back the row: the unresolved id must be stored verbatim, not
// dropped or rewritten to a repo it never named.
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let task = state
.db
.get_task(&created_id(&bytes))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
task.repo_id.as_deref(),
Some("no-such-repo-id"),
"an unresolved id must be stored verbatim as a label"
);
// Quarantined repo id → treated as unknown (201), never a 403 that
// would confirm the id names a real repo.
state
.db
.set_repo_quarantine(&repo.id, true)
.await
.expect("quarantine");
let resp = post_as(stranger, body_with(stranger, Some(&repo.id)))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::CREATED,
"a quarantined repo id must not 403 (existence oracle)"
);
// Read back the row: the quarantined id is stored verbatim as a label
// too, so the accepted write is not silently dropped.
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let task = state
.db
.get_task(&created_id(&bytes))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
task.repo_id.as_deref(),
Some(repo.id.as_str()),
"a quarantined id must be stored verbatim as a label"
);
}
/// #496 (REST): a repo-lookup fault in the create_task gate must render the
/// module's opaque `db_error` envelope, not the raw sqlx text. Dropping a
/// column `get_repo_by_id` selects faults that first lookup; the 500 body
/// stays fixed and names no column (#250/#226).
#[sqlx::test]
async fn create_task_gate_repo_lookup_fault_is_opaque(pool: PgPool) {
let owner = "did:key:zTASKFAULTOWNERAAAAAAAAAAAAAAAAAAAAAAAAA";
let state = test_state(pool.clone()).await;
let repo = seed_repo(owner, "task-gate-fault-repo");
state.db.create_repo(&repo).await.expect("seed repo");
let router = || {
Router::new()
.route(
"/api/v1/tasks",
axum::routing::post(crate::api::tasks::create_task),
)
.with_state(state.clone())
};
let body = format!(
r#"{{"kind":"build","capability":"repo:write","delegator_did":"{owner}","repo_id":"{}"}}"#,
repo.id
);
// Drop a column the lookup's SELECT names so get_repo_by_id faults.
sqlx::query("ALTER TABLE repos DROP COLUMN machine_id")
.execute(&pool)
.await
.expect("drop machine_id");
let resp = router()
.oneshot(signed_request_as(
owner,
Method::POST,
"/api/v1/tasks",
Body::from(body),
))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::INTERNAL_SERVER_ERROR,
"a faulted repo lookup must be a 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8_lossy(&bytes);
assert!(
text.contains(r#""error":"db_error""#),
"gate DB fault must keep the opaque envelope: {text}"
);
assert!(
text.contains(crate::error::DB_ERROR_MESSAGE),
"gate DB fault must carry the fixed message: {text}"
);
assert!(
!text.contains("machine_id") && !text.contains("column"),
"schema text leaked through the gate: {text}"
);
}
/// #496 (REST): the gate's second lookup (`is_repo_quarantined`) runs through
/// the same opaque helper. Dropping the column only it selects leaves
/// `get_repo_by_id` succeeding, so this pins that second call site too.
#[sqlx::test]
async fn create_task_gate_quarantine_lookup_fault_is_opaque(pool: PgPool) {
let owner = "did:key:zTASKFAULT2OWNERAAAAAAAAAAAAAAAAAAAAAAAA";
let state = test_state(pool.clone()).await;
let repo = seed_repo(owner, "task-gate-fault2-repo");
state.db.create_repo(&repo).await.expect("seed repo");
let router = || {
Router::new()
.route(
"/api/v1/tasks",
axum::routing::post(crate::api::tasks::create_task),
)
.with_state(state.clone())
};
let body = format!(
r#"{{"kind":"build","capability":"repo:write","delegator_did":"{owner}","repo_id":"{}"}}"#,
repo.id
);
// `quarantined` is not in get_repo_by_id's SELECT, so that lookup still
// succeeds and the fault lands on is_repo_quarantined.
sqlx::query("ALTER TABLE repos DROP COLUMN quarantined")
.execute(&pool)
.await
.expect("drop quarantined");
let resp = router()
.oneshot(signed_request_as(
owner,
Method::POST,
"/api/v1/tasks",
Body::from(body),
))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::INTERNAL_SERVER_ERROR,
"a faulted quarantine lookup must be a 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8_lossy(&bytes);
assert!(
text.contains(r#""error":"db_error""#),
"quarantine DB fault must keep the opaque envelope: {text}"
);
assert!(
text.contains(crate::error::DB_ERROR_MESSAGE),
"quarantine DB fault must carry the fixed message: {text}"
);
assert!(
!text.contains("quarantined") && !text.contains("column"),
"schema text leaked through the gate: {text}"
);
}
/// #496 (REST): the create_task INSERT goes through the same opaque helper,
/// so a write fault must not serialize sqlx text either. A repo-less task
/// reaches the INSERT without a gate lookup; dropping a column the INSERT
/// names faults it.
#[sqlx::test]
async fn create_task_insert_fault_is_opaque(pool: PgPool) {
let signer = "did:key:zTASKINSERTFAULTAAAAAAAAAAAAAAAAAAAAAAAA";
let state = test_state(pool.clone()).await;
let router = || {
Router::new()
.route(
"/api/v1/tasks",
axum::routing::post(crate::api::tasks::create_task),
)
.with_state(state.clone())
};
let body =
format!(r#"{{"kind":"build","capability":"repo:write","delegator_did":"{signer}"}}"#);
sqlx::query("ALTER TABLE agent_tasks DROP COLUMN status")
.execute(&pool)
.await
.expect("drop status");
let resp = router()
.oneshot(signed_request_as(
signer,
Method::POST,
"/api/v1/tasks",
Body::from(body),
))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::INTERNAL_SERVER_ERROR,
"a faulted INSERT must be a 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8_lossy(&bytes);
assert!(
text.contains(r#""error":"db_error""#),
"INSERT fault must keep the opaque envelope: {text}"
);
assert!(
text.contains(crate::error::DB_ERROR_MESSAGE),
"INSERT fault must carry the fixed message: {text}"
);
assert!(
!text.contains("status") && !text.contains("column"),
"schema text leaked through the INSERT: {text}"
);
}
/// N3: get_tree gates on the REQUESTED subtree, not the repo root. A caller
/// denied a withheld subtree is rejected there (404) but passes the gate on a
/// non-withheld path (so the rejection is path-scoped, not repo-wide).
@@ -1186,6 +1533,451 @@ mod tests {
);
}
/// Pre-assigned tasks must not be stealable: a stranger claiming a pending
/// task reserved for another agent must fail, and must not receive the
/// UCAN. The reserved assignee still claims successfully.
#[sqlx::test]
async fn claim_task_honors_preassigned_assignee(pool: PgPool) {
let delegator = "did:key:zCLAIMDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAA";
let reserved = "did:key:zCLAIMRESERVEDBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let thief = "did:key:zCLAIMTHIEFCCCCCCCCCCCCCCCCCCCCCCCCCCCCC";
let state = test_state(pool).await;
let mut task = seed_task("task-reserved", delegator);
task.assignee_did = Some(reserved.to_string());
task.ucan_token = Some("ucan-secret-for-reserved-only".into());
state.db.create_task(&task).await.expect("seed");
let router = || {
Router::new()
.route(
"/api/v1/tasks/{id}/claim",
axum::routing::post(crate::api::tasks::claim_task),
)
.with_state(state.clone())
};
let uri = "/api/v1/tasks/task-reserved/claim";
// Thief signs as themselves and asks to claim — must fail, UCAN stays put.
let resp = router()
.oneshot(signed_request_as(
thief,
Method::POST,
uri,
Body::from(format!(r#"{{"assignee_did":"{thief}"}}"#)),
))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::FORBIDDEN,
"stranger must not steal a pre-assigned task, got {}",
resp.status()
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8_lossy(&body);
assert!(
!text.contains("ucan-secret-for-reserved-only"),
"UCAN must not leak on a failed steal: {text}"
);
let still = state.db.get_task("task-reserved").await.unwrap().unwrap();
assert_eq!(still.status, "pending");
assert_eq!(still.assignee_did.as_deref(), Some(reserved));
// Reserved assignee claims successfully and receives the UCAN.
let resp = router()
.oneshot(signed_request_as(
reserved,
Method::POST,
uri,
Body::from(format!(r#"{{"assignee_did":"{reserved}"}}"#)),
))
.await
.unwrap();
assert!(
resp.status().is_success(),
"reserved assignee must claim, got {}",
resp.status()
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let text = String::from_utf8_lossy(&body);
assert!(
text.contains("ucan-secret-for-reserved-only"),
"reserved assignee must receive UCAN: {text}"
);
assert!(text.contains("\"status\":\"claimed\"") || text.contains("claimed"));
}
/// Open (unassigned) pending tasks remain first-claimer-wins after the
/// pre-assignment gate.
#[sqlx::test]
async fn claim_task_open_still_first_claimer_wins(pool: PgPool) {
let delegator = "did:key:zOPENDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAA";
let claimer = "did:key:zOPENCLAIMERBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let state = test_state(pool).await;
state
.db
.create_task(&seed_task("task-open", delegator))
.await
.expect("seed");
let router = Router::new()
.route(
"/api/v1/tasks/{id}/claim",
axum::routing::post(crate::api::tasks::claim_task),
)
.with_state(state.clone());
let resp = router
.oneshot(signed_request_as(
claimer,
Method::POST,
"/api/v1/tasks/task-open/claim",
Body::from(format!(r#"{{"assignee_did":"{claimer}"}}"#)),
))
.await
.unwrap();
assert!(
resp.status().is_success(),
"open task must still be claimable, got {}",
resp.status()
);
let got = state.db.get_task("task-open").await.unwrap().unwrap();
assert_eq!(got.status, "claimed");
assert_eq!(got.assignee_did.as_deref(), Some(claimer));
}
/// ASCII-blank assignees (`ASSIGNEE_BLANK`: space, tab, LF, CR) must be open
/// at the claim SQL blank branch. Seeds via raw INSERT so create_task
/// normalization is not in the path.
#[sqlx::test]
async fn claim_task_ascii_blank_reservations_are_open(pool: PgPool) {
let delegator = "did:key:zBLANKDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAA";
let claimer = "did:key:zBLANKCLAIMERBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let state = test_state(pool.clone()).await;
let now = chrono::Utc::now().to_rfc3339();
for (id, blank) in [
("task-blank-empty", ""),
("task-blank-tab", "\t"),
("task-blank-space", " "),
("task-blank-lf", "\n"),
("task-blank-cr", "\r"),
] {
sqlx::query(
"INSERT INTO agent_tasks (id, repo_id, kind, status, delegator_did, assignee_did, capability, ucan_token, payload, result, created_at, updated_at, deadline)
VALUES ($1,NULL,'build','pending',$2,$3,'repo:write',NULL,NULL,NULL,$4,$4,NULL)",
)
.bind(id)
.bind(delegator)
.bind(blank)
.bind(&now)
.execute(&pool)
.await
.unwrap_or_else(|e| panic!("seed {id}: {e}"));
let claimed = state
.db
.claim_task(id, claimer)
.await
.unwrap_or_else(|e| panic!("{id} must be open: {e}"));
assert_eq!(claimed.status, "claimed", "{id}");
assert_eq!(claimed.assignee_did.as_deref(), Some(claimer), "{id}");
}
}
/// create_task must null whitespace-only assignees and return that shape
/// (fail-on-remove for the blank filter).
#[sqlx::test]
async fn create_task_normalizes_whitespace_assignee(pool: PgPool) {
let delegator = "did:key:zCREATENORMDELEGAAAAAAAAAAAAAAAAAAAAAAAA";
let state = test_state(pool).await;
let mut task = seed_task("task-create-norm", delegator);
task.assignee_did = Some(" \t\n ".into());
let stored = state.db.create_task(&task).await.expect("create");
assert!(
stored.assignee_did.is_none(),
"create_task must return normalized NULL assignee, got {:?}",
stored.assignee_did
);
let got = state
.db
.get_task("task-create-norm")
.await
.unwrap()
.unwrap();
assert!(got.assignee_did.is_none(), "row must store NULL assignee");
}
/// Claiming a reserved task must keep the stored DID form so exact-match
/// list filters still find it. Also covers bare→full claim + list by bare
/// (claimer full DID must not be required for list equality).
#[sqlx::test]
async fn claim_task_keeps_stored_assignee_did_form(pool: PgPool) {
let delegator = "did:key:zFORMDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAA";
let bare = "zFORMRESERVEDBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let full = format!("did:key:{bare}");
let state = test_state(pool).await;
let mut task = seed_task("task-form", delegator);
task.assignee_did = Some(bare.to_string());
state.db.create_task(&task).await.expect("seed");
let claimed = state
.db
.claim_task("task-form", &full)
.await
.expect("claim with full DID");
assert_eq!(
claimed.assignee_did.as_deref(),
Some(bare),
"reserved claim must keep the stored bare-key form"
);
let listed_bare = state
.db
.list_tasks(Some("claimed"), Some(bare), 10)
.await
.unwrap();
assert!(
listed_bare.iter().any(|t| t.id == "task-form"),
"delegator filtering by bare key must still find the task"
);
let listed_full = state
.db
.list_tasks(Some("claimed"), Some(&full), 10)
.await
.unwrap();
assert!(
!listed_full.iter().any(|t| t.id == "task-form"),
"exact list by claimer's full DID must miss the bare stored form"
);
}
/// Padded non-blank reservation must survive claim unchanged (exact form).
#[sqlx::test]
async fn claim_task_keeps_padded_reservation_exact(pool: PgPool) {
let delegator = "did:key:zPADDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let key = "zPADRESERVEDBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let padded = format!(" {key}\t");
let full = format!("did:key:{key}");
let state = test_state(pool.clone()).await;
let now = chrono::Utc::now().to_rfc3339();
sqlx::query(
"INSERT INTO agent_tasks (id, repo_id, kind, status, delegator_did, assignee_did, capability, ucan_token, payload, result, created_at, updated_at, deadline)
VALUES ($1,NULL,'build','pending',$2,$3,'repo:write',NULL,NULL,NULL,$4,$4,NULL)",
)
.bind("task-padded")
.bind(delegator)
.bind(&padded)
.bind(&now)
.execute(&pool)
.await
.expect("seed padded");
let claimed = state
.db
.claim_task("task-padded", &full)
.await
.expect("claim padded reservation");
assert_eq!(
claimed.assignee_did.as_deref(),
Some(padded.as_str()),
"claim must not BTRIM a non-blank reservation"
);
let listed = state
.db
.list_tasks(Some("claimed"), Some(&padded), 10)
.await
.unwrap();
assert!(
listed.iter().any(|t| t.id == "task-padded"),
"exact list by original padded form must still find the task"
);
let finished = state
.db
.finish_task("task-padded", "completed", None, &full)
.await
.expect("padded assignee must finish after trim match");
assert_eq!(finished.status, "completed");
}
/// Direct coverage for the DB-layer finish_task assignee Rust gate (handlers
/// 403 before calling in, so HTTP tests alone leave this unproven).
#[sqlx::test]
async fn finish_task_rejects_non_assignee_at_db(pool: PgPool) {
let delegator = "did:key:zFINISHDELEGATORAAAAAAAAAAAAAAAAAAAAAAAA";
let assignee = "did:key:zFINISHASSIGNEEBBBBBBBBBBBBBBBBBBBBBBBBB";
let stranger = "did:key:zFINISHSTRANGERCCCCCCCCCCCCCCCCCCCCCCCCC";
let state = test_state(pool).await;
state
.db
.create_task(&seed_task("task-finish-gate", delegator))
.await
.expect("seed");
state
.db
.claim_task("task-finish-gate", assignee)
.await
.expect("claim");
let err = state
.db
.finish_task("task-finish-gate", "completed", None, stranger)
.await
.expect_err("stranger must not finish");
assert!(
err.to_string().contains("assignee") || err.to_string().contains("finishable"),
"expected assignee denial, got {err}"
);
let still = state
.db
.get_task("task-finish-gate")
.await
.unwrap()
.unwrap();
assert_eq!(
still.status, "claimed",
"row must stay claimed after denial"
);
let done = state
.db
.finish_task("task-finish-gate", "completed", None, assignee)
.await
.expect("assignee finishes");
assert_eq!(done.status, "completed");
}
/// Fail-on-remove for `finish_task`'s `AND assignee_did=$5`: a concurrent writer
/// that changes the stored assignee between the production read and UPDATE must
/// leave the task claimed (production `Db::finish_task` path).
#[sqlx::test]
async fn finish_task_sql_assignee_predicate_is_load_bearing(pool: PgPool) {
let delegator = "did:key:zFINISHSQLDELEGAAAAAAAAAAAAAAAAAAAAAAAAA";
let assignee = "did:key:zFINISHSQLASSIGNBBBBBBBBBBBBBBBBBBBBBBBB";
let other = "did:key:zFINISHSQLOTHERCCCCCCCCCCCCCCCCCCCCCCCCC";
let state = test_state(pool.clone()).await;
let id = "task-finish-sql";
state
.db
.create_task(&seed_task(id, delegator))
.await
.expect("seed");
state.db.claim_task(id, assignee).await.expect("claim");
let pool_bg = pool.clone();
let id_bg = id.to_string();
let other_bg = other.to_string();
let (locked_tx, locked_rx) = tokio::sync::oneshot::channel();
let hog = tokio::spawn(async move {
let mut tx = pool_bg.begin().await.expect("begin");
sqlx::query("SELECT assignee_did FROM agent_tasks WHERE id = $1 FOR UPDATE")
.bind(&id_bg)
.fetch_one(&mut *tx)
.await
.expect("lock row");
locked_tx.send(()).ok();
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
sqlx::query(
"UPDATE agent_tasks SET assignee_did = $2 WHERE id = $1 AND status = 'claimed'",
)
.bind(&id_bg)
.bind(&other_bg)
.execute(&mut *tx)
.await
.expect("race assignee");
tx.commit().await.expect("commit");
});
locked_rx.await.expect("row lock held before finish");
let err = state
.db
.finish_task(id, "completed", None, assignee)
.await
.expect_err("assignee slot race must fail production finish_task");
hog.await.expect("hog task");
assert!(
err.to_string().contains("not in claimed state"),
"expected finish denial, got {err}"
);
let still = state.db.get_task(id).await.unwrap().unwrap();
assert_eq!(still.status, "claimed");
assert_eq!(still.assignee_did.as_deref(), Some(other));
}
/// Fail-on-remove for the claim UPDATE assignee-slot re-check: a concurrent
/// writer that changes `assignee_did` after the pre-read must block claim.
#[sqlx::test]
async fn claim_task_update_assignee_slot_recheck_is_load_bearing(pool: PgPool) {
let delegator = "did:key:zCLAIMSLOTDELEGAAAAAAAAAAAAAAAAAAAAAAAAA";
let reserved = "did:key:zCLAIMSLOTASSIGNEEBBBBBBBBBBBBBBBBBBBBBBBBB";
let other = "did:key:zCLAIMSLOTOTHERCCCCCCCCCCCCCCCCCCCCCCCCC";
let state = test_state(pool.clone()).await;
let id = "task-claim-slot";
let now = chrono::Utc::now().to_rfc3339();
sqlx::query(
"INSERT INTO agent_tasks
(id, repo_id, kind, status, delegator_did, assignee_did, capability,
ucan_token, payload, result, created_at, updated_at, deadline)
VALUES ($1, NULL, 'build', 'pending', $2, $3, 'repo:write',
NULL, NULL, NULL, $4, $4, NULL)",
)
.bind(id)
.bind(delegator)
.bind(reserved)
.bind(&now)
.execute(&pool)
.await
.expect("seed reserved");
let pool_bg = pool.clone();
let id_bg = id.to_string();
let other_bg = other.to_string();
let (locked_tx, locked_rx) = tokio::sync::oneshot::channel();
let hog = tokio::spawn(async move {
let mut tx = pool_bg.begin().await.expect("begin");
sqlx::query("SELECT assignee_did FROM agent_tasks WHERE id = $1 FOR UPDATE")
.bind(&id_bg)
.fetch_one(&mut *tx)
.await
.expect("lock row");
locked_tx.send(()).ok();
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
sqlx::query(
"UPDATE agent_tasks SET assignee_did = $2 WHERE id = $1 AND status = 'pending'",
)
.bind(&id_bg)
.bind(&other_bg)
.execute(&mut *tx)
.await
.expect("race assignee");
tx.commit().await.expect("commit");
});
locked_rx.await.expect("row lock held before claim");
let err = state
.db
.claim_task(id, reserved)
.await
.expect_err("assignee slot race must fail production claim_task");
hog.await.expect("hog task");
assert!(
err.to_string().contains("not found or already claimed"),
"expected SQL re-check denial, got {err}"
);
assert!(
err.downcast_ref::<crate::db::TaskReservedForOtherAssignee>()
.is_none(),
"must prove the UPDATE predicate, not the Rust pre-check: {err}"
);
let still = state.db.get_task(id).await.unwrap().unwrap();
assert_eq!(still.status, "pending");
assert_eq!(still.assignee_did.as_deref(), Some(other));
}
/// Adversarial-review GATE-2 (create_pr): opening a PR requires read access.
/// A non-reader is denied on a private repo before any PR is created; the
/// owner is allowed.
@@ -4353,6 +5145,7 @@ mod tests {
node_did: owner.to_string(),
signature: "sig".to_string(),
issued_at: Utc::now().to_rfc3339(),
version: 1,
})
.await
.expect("seed private cert");
@@ -10409,14 +11202,10 @@ mod tests {
seed_legacy_pin(&pool, &bare, &fx.public_oid, Some(&repo.id)).await;
// A git that takes 300ms per invocation (the read makes two: type, then content).
let slow_git = std::env::temp_dir().join(format!("gl-slow-git-{short}"));
std::fs::write(&slow_git, "#!/bin/sh\nsleep 0.3\nexec git \"$@\"\n").expect("write shim");
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&slow_git, std::fs::Permissions::from_mode(0o755))
.expect("chmod shim");
}
let slow_git = git_shim::create(
&format!("gl-slow-git-{short}"),
git_shim::Behavior::Delay(300),
);
let ticks = std::sync::Arc::new(AtomicUsize::new(0));
let ticker = {
@@ -11363,19 +12152,6 @@ mod tests {
);
}
/// Write an executable `git` stand-in and return its path.
fn write_git_shim(name: &str, script: &str) -> std::path::PathBuf {
let path = std::env::temp_dir().join(name);
std::fs::write(&path, script).expect("write the git shim");
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755))
.expect("chmod the git shim");
}
path
}
/// F6 scenario 1 (#173 round 13): one hung candidate must not starve the rows behind
/// it in the same pass. `DiscoveryCtx` is loaded once per pass, so before the per-row
/// slice every source-less row in a pass shared ONE deadline: the first row's wedged
@@ -11415,23 +12191,9 @@ mod tests {
// The type stage feeds the oid on STDIN (`cat-file --batch-check`) and the
// content stage puts it in argv, so the stand-in has to look in both places.
let git_bin = write_git_shim(
let git_bin = git_shim::create(
&format!("gl-hung-git-{short}"),
&format!(
"#!/bin/sh\n\
if [ \"$2\" = \"--batch-check\" ]; then\n\
\x20 oid=$(cat)\n\
\x20 case \"$oid\" in\n\
\x20 {hung_oid}) sleep 30; exit 1 ;;\n\
\x20 esac\n\
\x20 printf '%s\\n' \"$oid\" | git \"$@\"\n\
\x20 exit $?\n\
fi\n\
case \"$*\" in\n\
\x20 *{hung_oid}*) sleep 30; exit 1 ;;\n\
esac\n\
exec git \"$@\"\n"
),
git_shim::Behavior::HangOid(&hung_oid),
);
let stats = tokio::time::timeout(
@@ -11516,10 +12278,7 @@ mod tests {
// Wedges on every invocation, so no row can ever be repaired and the only
// question left is what each one COSTS.
let git_bin = write_git_shim(
&format!("gl-spent-git-{short}"),
"#!/bin/sh\nsleep 30\nexit 1\n",
);
let git_bin = git_shim::create(&format!("gl-spent-git-{short}"), git_shim::Behavior::Hang);
let stats = tokio::time::timeout(
std::time::Duration::from_secs(60),
@@ -12123,10 +12882,7 @@ mod tests {
seed_legacy_pin(&pool, &src, oid, None).await;
}
let git_bin = write_git_shim(
&format!("gl-starve-git-{short}"),
"#!/bin/sh\nsleep 30\nexit 1\n",
);
let git_bin = git_shim::create(&format!("gl-starve-git-{short}"), git_shim::Behavior::Hang);
tokio::time::timeout(
std::time::Duration::from_secs(120),
@@ -12194,9 +12950,9 @@ mod tests {
let (raw_cid, provider_cid) = seed_legacy_pin(&pool, &src, &fx.public_oid, None).await;
// Wedges only inside the position-nine repo, which the sweep enters by cwd.
let git_bin = write_git_shim(
let git_bin = git_shim::create(
&format!("gl-mid-git-{short}"),
"#!/bin/sh\ncase \"$(pwd)\" in\n */midcand9.git) sleep 30; exit 1 ;;\nesac\nexec git \"$@\"\n",
git_shim::Behavior::HangRepo("midcand9.git"),
);
let first = tokio::time::timeout(
@@ -12358,19 +13114,9 @@ mod tests {
let log = std::env::temp_dir().join(format!("gl-ali-log-{short}"));
let _ = std::fs::remove_file(&log);
let git_bin = write_git_shim(
let git_bin = git_shim::create(
&format!("gl-ali-git-{short}"),
&format!(
"#!/bin/sh\n\
if [ \"$2\" = \"--batch-check\" ]; then\n\
\x20 oid=$(cat)\n\
\x20 printf '%s %s\\n' \"$oid\" \"$(basename $(pwd))\" >> {log}\n\
\x20 printf '%s\\n' \"$oid\" | git \"$@\"\n\
\x20 exit $?\n\
fi\n\
exec git \"$@\"\n",
log = log.display()
),
git_shim::Behavior::LogTypes(&log),
);
let mut traversal = crate::ipfs_pin::DiscoveryTraversalState::default();
@@ -16912,6 +17658,7 @@ mod tests {
node_did: "did:key:zNode".into(),
signature: "sig".into(),
issued_at: "2026-01-01T00:00:00Z".into(),
version: 1,
};
state.db.insert_ref_certificate(&cert).await.unwrap();
@@ -16947,6 +17694,7 @@ mod tests {
node_did: "did:key:zNode".into(),
signature: "sig".into(),
issued_at: "2026-01-01T00:00:00Z".into(),
version: 1,
};
state.db.insert_ref_certificate(&cert).await.unwrap();
@@ -17686,6 +18434,7 @@ mod tests {
node_did: "did:key:zNode".into(),
signature: "sig".into(),
issued_at: "2026-01-01T00:00:00Z".into(),
version: 1,
};
state.db.insert_ref_certificate(&cert).await.unwrap();
@@ -19360,6 +20109,7 @@ mod tests {
node_did: "did:key:zNODE".into(),
signature: "sig".into(),
issued_at: issued_at.to_string(),
version: 1,
}
}
+160 -7
View File
@@ -38,19 +38,139 @@ pub fn fire_event(
event: &str,
payload: serde_json::Value,
) {
let repo_id = repo_id.to_string();
let event = event.to_string();
tokio::spawn(async move {
fire_event_async(db, http_client, &repo_id, &event, payload).await;
});
fire_event_occurrence(db, http_client, repo_id, event, payload, None, None);
}
async fn fire_event_async(
/// Occurrence-keyed variant: at most one delivery per
/// `(request_id, ref_name, hook)` is spawned; concurrent executors
/// collapse via the `webhook_deliveries` ledger. Best-effort on crash
/// (ledger is claimed before spawn), which the PR scope allows.
///
/// For durable outbox usage, call `claim_webhook_delivery_before_spawn` first
/// to claim deliveries synchronously, then call this with `skip_claim=true` and
/// the claimed hook IDs to only spawn HTTP POST for claimed hooks.
pub fn fire_event_occurrence(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
) {
let repo_id = repo_id.to_string();
let event = event.to_string();
let request_id = request_id.map(|s| s.to_string());
let ref_name = ref_name.map(|s| s.to_string());
tokio::spawn(async move {
fire_event_async_occurrence(
db,
http_client,
&repo_id,
&event,
payload,
request_id.as_deref(),
ref_name.as_deref(),
false,
None,
)
.await;
});
}
/// Variant for durable outbox: only spawn HTTP POST for previously claimed hooks.
#[allow(clippy::too_many_arguments)]
pub fn fire_event_occurrence_with_claimed_hooks(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
claimed_hook_ids: Vec<String>,
) {
let repo_id = repo_id.to_string();
let event = event.to_string();
let request_id = request_id.map(|s| s.to_string());
let ref_name = ref_name.map(|s| s.to_string());
tokio::spawn(async move {
fire_event_async_occurrence(
db,
http_client,
&repo_id,
&event,
payload,
request_id.as_deref(),
ref_name.as_deref(),
true,
Some(claimed_hook_ids),
)
.await;
});
}
/// Claim webhook delivery before spawning HTTP POST.
/// Returns `Some(hook ids)` successfully claimed, or `None` when the hook
/// list itself could not be read (transient DB error, unknown membership).
/// A `None` caller must fall back to best-effort send rather than dropping
/// the delivery: children are still deleted downstream, so an empty claim
/// set on DB error would lose the webhook permanently.
/// This synchronous claim phase must be awaited before child deletion to prevent
/// permanent webhook loss on crash between deletion and claim.
pub async fn claim_webhook_delivery_before_spawn(
db: Arc<Db>,
repo_id: &str,
event: &str,
request_id: Option<&str>,
ref_name: Option<&str>,
) -> Option<Vec<String>> {
let mut claimed_hooks = Vec::new();
// Only claim when we have occurrence context (request_id + ref_name)
if let (Some(req), Some(r)) = (request_id, ref_name) {
let hooks = match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(err = %e, "failed to list webhooks for event {event}");
return None;
}
};
for hook in hooks {
let delivery_id = crate::db::deterministic_id(&["webhook", req, r, &hook.id]);
match db
.claim_webhook_delivery(&delivery_id, req, repo_id, event)
.await
{
Ok(true) => claimed_hooks.push(hook.id),
Ok(false) => continue, // Already claimed
// Send fallback (legacy best-effort): a transient claim
// error must not drop the delivery while children are
// still deleted. The in-task claimer this replaced sent
// anyway on claim error; keep that here.
Err(e) => {
tracing::warn!(err = %e, hook_id = %hook.id, "webhook claim failed; sending anyway");
claimed_hooks.push(hook.id);
}
}
}
}
Some(claimed_hooks)
}
#[allow(clippy::too_many_arguments)]
async fn fire_event_async_occurrence(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
skip_claim: bool,
claimed_hook_ids: Option<Vec<String>>,
) {
let hooks = match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
@@ -76,8 +196,36 @@ async fn fire_event_async(
let client = Arc::clone(&http_client);
let event_name = event.to_string();
let bytes = payload_bytes.clone();
let delivery_id = uuid::Uuid::new_v4().to_string();
// Stable occurrence delivery key when request context exists;
// random UUID otherwise (legacy callers without occurrence).
let delivery_id = match (request_id, ref_name) {
(Some(req), Some(r)) => crate::db::deterministic_id(&["webhook", req, r, &hook.id]),
_ => uuid::Uuid::new_v4().to_string(),
};
// Claim before spawn so concurrent executors collapse to one
// delivery per occurrence. Failures to record fall back to
// sending (legacy best-effort) rather than dropping.
// Skip claim if already done by caller (durable_outbox path).
if !skip_claim && request_id.is_some() {
match db
.clone()
.claim_webhook_delivery(&delivery_id, request_id.unwrap_or(""), repo_id, event)
.await
{
Ok(true) => {}
Ok(false) => continue,
Err(_) => {}
}
} else if skip_claim {
// Skip hooks that weren't claimed by the caller
if let Some(ref claimed) = claimed_hook_ids {
if !claimed.contains(&hook.id) {
continue;
}
}
}
let db_for_mark = db.clone();
tokio::spawn(async move {
// Policy skip, not a delivery failure: the stored URL predates the
// create-time gate. The "invalid_url" label is documented on
@@ -92,6 +240,7 @@ async fn fire_event_async(
return;
}
let delivery_for_mark = delivery_id.clone();
let signature = hook.secret.as_deref().map(|s| sign_payload(s, &bytes));
let mut req = client
@@ -113,6 +262,10 @@ async fn fire_event_async(
"http_error"
};
crate::metrics::record_webhook_delivery(result_label);
// Any HTTP response proves the delivery fired; mark
// sent so the ledger never claims an unsent delivery.
// Network errors stay pending for stale-reclaim.
let _ = db_for_mark.mark_webhook_sent(&delivery_for_mark).await;
tracing::info!(
url = %hook.url,
event = %event_name,
+59
View File
@@ -0,0 +1,59 @@
use std::io::{Read, Write};
use std::process::{Command, Stdio};
use std::time::Duration;
fn main() -> std::io::Result<()> {
let executable = std::env::current_exe()?;
let config = std::fs::read_to_string(executable.parent().unwrap().join("config"))?;
let (mode, parameter) = config.split_once('\n').unwrap_or((&config, ""));
let args: Vec<_> = std::env::args_os().skip(1).collect();
if mode == "delay" {
std::thread::sleep(Duration::from_millis(parameter.parse().unwrap()));
}
let batch = args.get(1).is_some_and(|arg| arg == "--batch-check");
let input = if batch {
let mut bytes = Vec::new();
std::io::stdin().read_to_end(&mut bytes)?;
Some(bytes)
} else {
None
};
let cwd = std::env::current_dir()?;
let hang = mode == "hang"
|| (mode == "hang-repo" && cwd.file_name().is_some_and(|name| name == parameter))
|| (mode == "hang-oid"
&& (args.iter().any(|arg| arg == parameter)
|| input
.as_ref()
.is_some_and(|bytes| String::from_utf8_lossy(bytes).trim() == parameter)));
if hang {
std::thread::sleep(Duration::from_secs(30));
std::process::exit(1);
}
if mode == "log-types" && batch {
let mut log = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(parameter)?;
writeln!(
log,
"{} {}",
String::from_utf8_lossy(input.as_ref().unwrap()).trim(),
cwd.file_name().unwrap().to_string_lossy()
)?;
}
let mut child = Command::new("git")
.args(&args)
.stdin(if input.is_some() {
Stdio::piped()
} else {
Stdio::inherit()
})
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()?;
if let Some(bytes) = input {
child.stdin.take().unwrap().write_all(&bytes)?;
}
std::process::exit(child.wait()?.code().unwrap_or(1));
}
+488 -20
View File
@@ -416,8 +416,8 @@ fn f3_second_writer_leased_until_reap() {
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
);
let receive = repos_production
.find("smart_http::receive_pack(")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack");
.find("smart_http::receive_pack_raw")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack_raw");
assert!(
lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \
@@ -509,8 +509,8 @@ fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so
/// the two cannot drift apart.
/// must sit inside `if disposition.spawn_tail`, and `release` must consume the same
/// disposition (`disposition.release_ok`) so the two cannot drift apart.
///
/// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
@@ -519,7 +519,9 @@ fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
/// `guard.release(` and the ordering assertion fails; take it out of the
/// `if push_succeeded` block and the failed-push assertion fails.
/// `if disposition.spawn_tail` block and the failed-push assertion fails; gate
/// `spawn_tail`/`release_ok` on the outcome commit and
/// `post_git_disposition_replication_carve_out` fails with it.
#[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs");
@@ -534,41 +536,507 @@ fn inv22_replication_tail_spawns_at_the_durability_boundary() {
.expect("split always yields a first chunk");
let success_flag = production
.find("let push_succeeded = receive_result.is_ok();")
.expect(
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()",
);
.find("post_git_disposition(")
.expect("U5 gate missing: the tail's disposition must be bound before the gate");
let gate_open = production
.find("if push_succeeded {")
.find("if disposition.spawn_tail {")
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded");
let spawn = production
.find("tokio::spawn(post_receive_replication_tail(")
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production
.find(".release(push_succeeded)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate");
.find(".release(disposition.release_ok)")
.expect("U5 gate stale: release must consume the same disposition as the tail gate");
let touch = production
.find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
let webhook = production
.find("webhooks::fire_event(")
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks");
// #26 Split PR 1 step 3 — the webhook fan-out moved into
// `durable_outbox::apply_request_effects`, which the live
// handler calls inline (the recovery drain calls the same
// function on the next startup). The gate now pins that the
// handler is wired to the executor, not to a per-ref inline
// webhook call.
let effects_executor = production
.find("apply_request_effects(&state, &request_id)")
.expect("U5 gate stale: git_receive_pack no longer calls apply_request_effects inline");
assert!(
success_flag < gate_open && gate_open < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \
"U5 gate bypassed: the tail must be spawned inside `if disposition.spawn_tail`, or a \
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
!production[gate_open + "if disposition.spawn_tail {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if disposition.spawn_tail` block, so a \
rejected push now spawns a tail"
);
assert!(
spawn < release && spawn < touch && spawn < webhook,
spawn < release && spawn < touch && spawn < effects_executor,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \
and the effect executor, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements"
);
}
/// #26 Split PR 1 step 3 — drain + handler share a single effect
/// executor. The live handler (api/repos.rs) and the recovery
/// drain (`durable_outbox::drain_receive_pack_requests_with`) both
/// call `apply_request_effects`, so the per-ref effects fan-out
/// lives in exactly one place. The v29 per-ref walk
/// (`derive_one`, `drain_pending_ref_transitions_all`,
/// `lookup_accepted_ordinal`) is dead code: any caller reintroduced
/// would be the per-ref walk the step-3 PR removed. This gate
/// fails if a call site slips back in or the new seam is bypassed.
#[test]
fn inv26_step3_live_and_drain_share_apply_request_effects() {
let repos = src("api/repos.rs");
let outbox = src("durable_outbox.rs");
// The live handler calls `apply_request_effects`. Split the
// file at the test attribute so test code can't satisfy the
// gate by itself.
let production_repos = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
production_repos.contains("apply_request_effects(&state, &request_id)"),
"live handler must call `apply_request_effects(&state, &request_id)`; \
reverting to a per-ref inline fan-out splits live and recovery"
);
// The durable intent insert closes the pre-outbox crash window this
// PR exists to fix. Wrapping it in `if false` must turn this gate
// red; drain-only tests cannot prove the handler creates the rows.
assert!(
production_repos.contains("insert_receive_pack_request_with_children("),
"live handler must persist durable intent via \
`insert_receive_pack_request_with_children`; disabling it must fail this gate"
);
// The drain's per-request seam calls the same executor. Test
// code lives below `mod drain_tests`, so split there too.
let production_outbox = outbox
.split("\nmod drain_tests {")
.next()
.expect("split always yields a first chunk");
assert!(
production_outbox.contains("drain_receive_pack_requests_with"),
"drain seam `drain_receive_pack_requests_with` must exist; \
removing it forces a per-ref walk back into the drain"
);
assert!(
production_outbox.contains("apply_request_effects"),
"durable_outbox production code must define `apply_request_effects`; \
removing it splits the executor between live and recovery"
);
// The drain's per-request walker is wired to the executor.
// The closure body of `drain_receive_pack_requests_with` is the
// only call site — if a future change calls `derive_one`
// instead, this assertion fires.
let drain_seam_open = production_outbox
.find("pub async fn drain_receive_pack_requests_with<F, Fut>")
.expect("drain seam must be defined in the production half");
let drain_seam_close = production_outbox[drain_seam_open..]
.find("\n}\n")
.expect("drain seam body must close");
let drain_seam_body = &production_outbox[drain_seam_open..drain_seam_open + drain_seam_close];
assert!(
drain_seam_body.contains("apply_request_effects"),
"drain_receive_pack_requests_with must call apply_request_effects; \
wiring it to a per-ref helper reintroduces the v29 walk"
);
// The deleted per-ref drain must have zero call sites in the
// production half. A regression that re-adds a caller would
// bring back the per-ref fan-out.
assert!(
!production_outbox.contains("drain_pending_ref_transitions_all("),
"deleted `drain_pending_ref_transitions_all` must have zero production call sites; \
a per-ref walk is reintroduced"
);
assert!(
!production_outbox.contains("derive_one("),
"deleted `derive_one` must have zero production call sites; \
the per-ref fan-out is reintroduced"
);
assert!(
!production_outbox.contains("lookup_accepted_ordinal("),
"deleted `lookup_accepted_ordinal` must have zero production call sites; \
the per-ref ordinal lookup is reintroduced"
);
assert!(
!production_repos.contains("derive_one("),
"deleted `derive_one` must have zero live-handler call sites"
);
assert!(
!production_repos.contains("drain_pending_ref_transitions"),
"deleted per-ref drain functions must have zero live-handler call sites"
);
}
/// #26 Split PR 1 step 4 — the periodic queue-lifecycle purge is
/// wired in `main.rs` and the contract is enforced by the `idx_receive_pack_requests_completed_at`
/// partial index from v30.
///
/// Assertions:
/// 1. `main.rs` calls `purge_request_queue` once at boot (well, on
/// the spawn-task interval) and the spawn function exists.
/// 2. The DB helpers `purge_completed_receive_pack_requests` and
/// `purge_completed_pending_ref_transitions` exist in `db/mod.rs`.
/// 3. The config knobs `queue_retention_days` and `queue_purge_batch` exist
/// on `Config`.
/// 4. `quarantined` is NOT in the purge WHERE clause (step 4 does not
/// introduce the state, but the invariant holds for the future).
#[test]
fn inv26_step4_queue_lifecycle_purge_is_wired() {
let main_src = src("main.rs");
assert!(
main_src.contains("spawn_queue_lifecycle_sweep"),
"main.rs must spawn the periodic queue-lifecycle purge"
);
assert!(
main_src.contains("purge_request_queue"),
"main.rs must call purge_request_queue on the periodic sweep"
);
let db_src = src("db/mod.rs");
assert!(
db_src.contains("purge_completed_receive_pack_requests"),
"db/mod.rs must expose purge_completed_receive_pack_requests"
);
assert!(
db_src.contains("purge_completed_pending_ref_transitions"),
"db/mod.rs must expose purge_completed_pending_ref_transitions"
);
let config_src = src("config.rs");
assert!(
config_src.contains("queue_retention_days"),
"Config must expose queue_retention_days"
);
assert!(
config_src.contains("queue_purge_batch"),
"Config must expose queue_purge_batch"
);
}
/// #26 Split PR 1 step 5 — the marker gate and the retry-bound
/// quarantine are wired end-to-end. A missing or hash-mismatched
/// marker on disk quarantines the request; a Retry over the bound
/// does too. This gate pins every load-bearing seam, against the
/// production half of each file (the test modules name the same
/// identifiers in their own harnesses).
///
/// Assertions:
/// 1. `reconcile_prepared_page` calls `mark_request_quarantined`
/// and `mark_children_rejected_for_quarantined_parent` on
/// marker-gate failure.
/// 2. The drain's `EffectsOutcome::Retry` arm checks
/// `effects_max_attempts`.
/// 3. `git::store::read_ref` exists in `git/store.rs`.
/// 4. `db::mark_request_quarantined`,
/// `db::mark_children_rejected_for_quarantined_parent`,
/// `db::get_receive_pack_requests_by_ids`, and
/// `db::request_state::QUARANTINED` all exist.
/// 5. The handler writes the marker ref BEFORE calling
/// `smart_http::receive_pack_raw` (the durability window).
#[test]
fn inv26_step5_marker_quarantine_and_bound_are_wired() {
let outbox = src("durable_outbox.rs");
let store = src("git/store.rs");
let db = src("db/mod.rs");
let repos = src("api/repos.rs");
// Split at the TEST MODULE for the production-only assertions.
let production_outbox = outbox
.split("\nmod drain_tests {")
.next()
.expect("split always yields a first chunk");
let production_repos = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (1) The reconcile's marker gate quarantines via the DB helpers.
assert!(
production_outbox.contains("mark_request_quarantined"),
"reconcile_prepared_page must call mark_request_quarantined on marker-gate failure"
);
assert!(
production_outbox.contains("mark_children_rejected_for_quarantined_parent"),
"reconcile_prepared_page must call mark_children_rejected_for_quarantined_parent \
so quarantined parents cancel their children"
);
// The gate reads the marker ref and compares against the parent's
// `request_bytes_hash` via `git::store::read_ref` /
// `git::store::marker_value_for`. Reverting either reintroduces
// the DoS window the marker gate exists to close.
assert!(
production_outbox.contains("git::store::read_ref"),
"reconcile_prepared_page must read the marker ref via git::store::read_ref"
);
assert!(
production_outbox.contains("marker_value_for"),
"reconcile_prepared_page must compute the expected marker value via \
git::store::marker_value_for"
);
// (2) The drain's `EffectsOutcome::Retry` arm checks the bound.
assert!(
production_outbox.contains("effects_max_attempts"),
"drain_receive_pack_requests_with must consult effects_max_attempts on Retry"
);
// (3) `git::store::read_ref` is the read seam the gate depends on.
assert!(
store.contains("pub fn read_ref("),
"git::store::read_ref must exist; the marker gate reads through it"
);
assert!(
store.contains("pub fn marker_value_for("),
"git::store::marker_value_for must exist; the marker gate computes the expected \
value with it (and the live handler writes the value via the same helper)"
);
// (4) DB-side seams the gate depends on.
assert!(
db.contains("pub async fn mark_request_quarantined"),
"Db::mark_request_quarantined must exist"
);
assert!(
db.contains("pub async fn mark_children_rejected_for_quarantined_parent"),
"Db::mark_children_rejected_for_quarantined_parent must exist"
);
assert!(
db.contains("pub async fn get_receive_pack_requests_by_ids"),
"Db::get_receive_pack_requests_by_ids must exist (avoids N+1 in the marker gate)"
);
assert!(
db.contains("pub const QUARANTINED: &str = \"quarantined\""),
"request_state::QUARANTINED must be defined"
);
// (5) The handler writes the marker ref BEFORE the durability
// boundary (smart_http::receive_pack_raw). Severing the
// ordering re-opens the marker-gate DoS window for live pushes.
let marker_write = production_repos
.find("git::store::marker_value_for")
.expect("U5 gate stale: the live handler no longer computes the marker value");
let receive_raw = production_repos
.find("smart_http::receive_pack_raw")
.expect("U5 gate stale: git_receive_pack no longer calls smart_http::receive_pack_raw");
assert!(
marker_write < receive_raw,
"U5 gate bypassed: the marker ref must be written BEFORE receive_pack_raw so the \
reconcile's gate has evidence of the live push"
);
}
/// Issue #26 split 1 — namespace gate and pre-git refusal lifecycle.
///
/// The push gate (repos.rs) must deny:
/// 1. Any ref-update line that fails UTF-8 decode (the raw body is passed verbatim
/// to git receive-pack, and git accepts refnames with bytes >= 0x80; the gate
/// never sees these refs, no durable child row exists, and under requests/issues/
/// the lossy for-each-ref decode still matches the narrowed exemption).
/// 2. The bare `refs/gitlawb` name (the `starts_with("refs/gitlawb/")` check misses it).
///
/// Every explicit pre-git refusal must call `refuse_receive_pack_before_git` to
/// terminalize the intent aggregate in one transaction, or a dropped future between
/// the intent insert and `receive_pack` strands a `received` + `prepared` aggregate
/// that is never purged, never promotable, and poisons its tuple.
///
/// The visibility exemption (visibility_pack.rs) was narrowed to `requests/` and
/// `issues/` only. The push gate is the premise that narrowing relies on (the comment
/// at visibility_pack.rs:340), so tests must prove:
/// - A `refs/gitlawb/<other>` ref pointing at a blob is denied on push.
/// - A stray `refs/gitlawb/<other>` blob ref planted in the pack path fails closed.
///
/// Source gates below pin the wiring; behavioral proofs live next to the code:
/// `parse_ref_updates_refuses_non_utf8_command_line` and
/// `internal_namespace_gate_covers_bare_name_and_subtree` (api/repos.rs),
/// `planted_non_exempt_gitlawb_blob_ref_fails_closed` (git/visibility_pack.rs),
/// and the mirror import prune
/// `prune_removes_non_exempt_gitlawb_refs_and_keeps_exempt` (sync.rs).
#[test]
fn issue_26_namespace_gate_and_refusal_wiring() {
let repos = src("api/repos.rs");
let vis = src("git/visibility_pack.rs");
// P1: fail the push on ref-update lines the handler cannot decode.
// The gate at parse_ref_updates must refuse when any command line in the
// ref-update section is undecodable UTF-8. The check is an Err arm on
// `from_utf8(data)` that returns BadRequest, so removing it allows non-UTF-8
// refs to bypass the namespace gate.
assert!(
repos.contains("ref-update line is not valid UTF-8; push refused"),
"P1 gate missing: parse_ref_updates must refuse when any ref-update line is \
non-UTF-8 (git accepts bytes >= 0x80; the gate never sees them, no durable \
child exists, and the lossy for-each-ref decode still matches the exemption)"
);
// P1 (cont): the bare `refs/gitlawb` name must also be caught. The
// predicate is extracted as `ref_is_internal_namespace` so it is
// unit-testable; pin the bare-name arm inside it and its use at the
// push gate (a `starts_with("refs/gitlawb/")`-only check misses the
// bare name, and it lands on repos with no internal refs yet).
assert!(
repos.contains("ref_name == \"refs/gitlawb\"")
&& repos.contains(".find(|r| ref_is_internal_namespace(r))"),
"P1 gate missing: ref_is_internal_namespace must match the bare \
'refs/gitlawb' name and the push gate must consult it \
(starts_with('refs/gitlawb/') misses it, and it lands on repos with \
no internal refs yet)"
);
// P1 (cont): every explicit pre-git refusal must call refuse_receive_pack_before_git
// to terminalize the intent. Exactly 6 occurrences: the five inline call
// sites (lease cap, per-source cap, write pool, acquire timeout, acquire
// error) plus the drop-guard's own pre-git spawn. The post-git guard phase
// calls mark_receive_pack_interrupted (a different name), so deleting any
// one inline site drops the count to 5 and turns this red. The parse and
// namespace refusals correctly carry no call: they return before the
// intent insert, so there is no aggregate to terminalize.
assert_eq!(
repos.matches("refuse_receive_pack_before_git(").count(),
6,
"P1 wiring: exactly 6 refuse_receive_pack_before_git calls expected (5 \
inline refusal sites + guard pre-git spawn); deleting any inline site \
must turn this red"
);
// Pin the five inline sites by their reason literals, so a site that stops
// terminalizing (or is renumbered away) fails by name, not just by count.
for reason in [
"repo write-lease waiter cap reached",
"admission per-source cap reached",
"git write pool saturated",
"acquire_write timed out",
"acquire_write failed",
"handler dropped before git started",
] {
assert!(
repos.contains(reason),
"P1 wiring: refusal reason literal missing: {reason}"
);
}
// P1 (cont): the intent drop-guard must be phase-aware. It is armed before
// the insert resolves (covering the cancellation window), flipped to
// git-started immediately before the receive_pack await (a drop from there
// is a mid-git disconnect, terminalized via mark_receive_pack_interrupted
// because the Err arm never runs on a drop), and disarmed once the outcome
// commit resolves. Assert the ordering, not just presence: the phase flip
// must precede the receive_pack call.
assert!(
repos.contains("IntentDropGuard::new") && repos.contains("notify_git_started()"),
"P1 gate missing: IntentDropGuard must exist and flip to git-started phase \
before receive_pack (a dropped future between intent insert and git start \
strands a received+prepared aggregate)"
);
let flip_pos = repos
.find("notify_git_started()")
.expect("notify_git_started call must exist");
let git_pos = repos
.find("receive_pack_raw_with_reflog(")
.expect("receive_pack call must exist");
assert!(
flip_pos < git_pos,
"P1 ordering: notify_git_started() must precede the receive_pack call, \
or a mid-git drop is misclassified as a pre-git shed"
);
assert!(
repos.contains("_intent_guard.disarm()"),
"P1 gate missing: the guard must disarm once the outcome commit resolves, \
or every push spawns a wasteful cleanup txn at scope end"
);
// A sync Drop cannot await: cleanup is spawned via the current runtime
// handle with an off-runtime fallback. A bare tokio::spawn panics
// mid-unwind with no runtime (off-runtime drop, process teardown).
assert!(
repos.contains("Handle::try_current()"),
"P1 gate missing: guard Drop must spawn via Handle::try_current with an \
off-runtime arm, not a bare tokio::spawn"
);
// P2: the post-git drop arm must exist and be wired. The guard's
// GIT_STARTED phase spawns `mark_receive_pack_interrupted` (parent to
// `rejected_at_git`, children to `uncertain` — the Err arm never runs on
// a drop). Deleting that spawn keeps every count above green, so pin the
// method name; its behavior is proved by
// `post_git_interruption_marks_uncertain_and_terminalizes_parent`.
assert!(
repos.contains("mark_receive_pack_interrupted("),
"P2 wiring: the guard's post-git phase must spawn \
mark_receive_pack_interrupted (a mid-git drop otherwise strands \
received+prepared with no Err arm to mark them uncertain)"
);
// The guard must commit computed fates rather than a blanket uncertain
// once the report is parsed, and only disarm on a resolved commit: pin
// both the stash call and the `outcome_commit_ok` gate. Making disarm()
// unconditional strands the commit-failure path and misfiles proven
// rejections as uncertain.
assert!(
repos.contains("set_fates(ComputedFates"),
"P2 wiring: computed fates must be stashed on the guard once parsed, \
or a post-commit drop can only apply a context-free uncertain"
);
assert!(
repos.contains("if outcome_commit_ok"),
"P2 wiring: disarm must be gated on outcome_commit_ok; unconditional \
disarm strands the persistent-commit-failure path"
);
// The live-handler `AwaitingSiblings` arm is unreachable through the
// current outcome builder (documented at the arm itself) and therefore
// has no behavioral test; pin its presence so removing it is conscious.
// The drain consumer IS behaviorally driven (sibling-wait tests).
assert_eq!(
repos.matches("EffectsOutcome::AwaitingSiblings").count(),
1,
"live AwaitingSiblings arm must exist (defensive symmetry with the \
drain consumer); deleting it must turn this red"
);
// P2: pin the mirror prune call sites. The prune helper is called from
// both `clone_repo` and `fetch_repo` (plus its own definition) — deleting
// either call leaves the direct-helper test green while mirrors import
// evil refs again. Count occurrences of the call with its argument.
let sync_src = src("sync.rs");
assert_eq!(
sync_src
.matches("prune_non_exempt_gitlawb_refs(local_path)")
.count(),
2,
"P2 wiring: prune must be called from both clone_repo and fetch_repo; \
deleting either call must turn this red"
);
// Both call sites must route through `wrap_prune_error`, which preserves
// deterministic `PruneInvalid` refusals instead of stringifying every
// error into transient `PruneFailed` (an every-tick deferral spin for
// permanently invalid mirrors). Pin the exact call shape at both sites.
assert_eq!(
sync_src.matches(".map_err(wrap_prune_error)").count(),
2,
"P2 wiring: both prune call sites must use wrap_prune_error; inline \
rewrap must turn this red"
);
// P2: the visibility exemption narrowing removed a serving path. The exemption
// is now scoped to `requests/` and `issues/` only (visibility_pack.rs:371), and
// the comment at :340 states the push gate is the premise. Tests must prove the
// gate works, which the P1 findings above covered (non-UTF-8 line, bare name,
// refusal wiring). This check verifies the exemption narrowing landed.
assert!(
vis.contains(
r#"!r.starts_with("refs/gitlawb/requests/") && !r.starts_with("refs/gitlawb/issues/")"#
),
"P2 gate missing: the visibility exemption must be narrowed to requests/ and \
issues/ only (the push gate is the premise; any other refs/gitlawb/* ref is \
denied on push and must fail closed in the pack path)"
);
}
+2
View File
@@ -26,6 +26,8 @@ clap = { version = "4", features = ["derive", "env"] }
dirs = "5"
reqwest = { workspace = true }
uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
urlencoding = "2"
alloy = { version = "1", default-features = false, features = [
"contract",
+541 -21
View File
@@ -4,6 +4,7 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -34,7 +35,12 @@ pub enum CertCmd {
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show a specific ref certificate and verify its signature
/// Show a specific ref certificate and verify its signature.
///
/// `--verify` requires `--expect-node <did>`: the queried node's
/// self-reported DID is not a trust anchor, because a node serving a
/// forged certificate also controls its own self-report. Callers
/// must name the issuer they trust.
Show {
/// Repository in <owner>/<repo> or <repo> format
repo: String,
@@ -49,13 +55,17 @@ pub enum CertCmd {
#[arg(long)]
dir: Option<PathBuf>,
/// Exit non-zero unless the Ed25519 signature verifies AND the
/// issuing node matches the queried node (or --expect-node)
/// issuing node matches --expect-node (which --verify requires:
/// without a caller-supplied anchor there is no trusted issuer to
/// verify against)
#[arg(long)]
verify: bool,
/// Expected issuing node DID for --verify. A valid signature alone
/// only proves the cert is internally consistent — signed by whatever
/// key it names — so --verify also anchors the issuer to a DID you
/// trust: this value when given, else the queried node's DID.
/// trust. The queried node's self-reported DID is deliberately NOT
/// accepted as a fallback anchor: the node that served the cert can
/// serve a matching self-report.
#[arg(long, requires = "verify")]
expect_node: Option<String>,
},
@@ -152,6 +162,24 @@ async fn cmd_show(
let node_did = cert["node_did"].as_str().unwrap_or("?");
let signature = cert["signature"].as_str().unwrap_or("?");
let issued_at = cert["issued_at"].as_str().unwrap_or("?");
// #26 Split PR 3: read the wire-format version. A MISSING key
// selects legacy v1 (an old server that predates the field).
// A PRESENT value must be a JSON integer that fits in u32 and
// names a version this client supports — otherwise the response
// declares a format we cannot represent and we refuse to verify
// rather than guess the payload shape. Reviewer 2 finding:
// collapsing `null`, a string, a float, or an overflow integer
// onto v1 would let a well-signed v1 signature pass `--verify`
// against a server that explicitly said "version 2 (or 99, or
// 4294967297)" — that is the exact mismatch the field exists
// to prevent. parse_cert_version is the load-bearing parser
// that distinguishes missing-key (legacy v1) from invalid-value
// (unsupported).
let parsed_version = parse_cert_version(cert.get("version"));
let version_display: String = match &parsed_version {
Ok(v) => v.to_string(),
Err(reason) => format!("unsupported ({reason})"),
};
println!("Ref Certificate: {cert_id}");
println!(" Ref: {ref_name}");
@@ -160,6 +188,7 @@ async fn cmd_show(
println!(" Pusher: {pusher}");
println!(" Node DID: {node_did}");
println!(" Issued at: {issued_at}");
println!(" Version: {version_display}");
println!(" Signature: {signature}");
println!();
@@ -169,9 +198,31 @@ async fn cmd_show(
// This proves the cert is internally authentic — signed by the key it
// names; the node-DID comparison below covers *which* node that is.
let repo_id = cert["repo_id"].as_str().unwrap_or("");
let verdict = verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
);
let verdict = match parsed_version {
// v1 is the only version this client verifies. The v1
// signed payload is the 7-field canonical form with no
// `version` key — see gitlawb-node/src/cert.rs. A future
// v2+ cert has a different signed payload shape (the
// version field becomes part of the signed JSON per
// `v2_signing_payload`), so a v2+ cert from a server this
// client does not know about must NOT be silently verified
// as v1.
// Reviewer 2: refuse rather than guess; the client and
// server must agree on the version.
// parse_cert_version admits exactly one Ok value: 1 (missing key and
// integer 1 both land there; 2, floats, strings, overflow and every
// other spelling come back Err with their own reason). A separate
// `Ok(v != 1)` arm existed here and was DEAD CODE — a test aimed at it
// could never catch drift in this match (review round 2) — so the
// match now mirrors the parser's real contract, which the
// `parse_cert_version_truth_table` test pins.
Ok(_) => verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
),
Err(ref reason) => Err(format!(
"cert declared a version this client cannot represent ({reason}); refusing to verify"
)),
};
println!("Signature verification:");
match &verdict {
@@ -181,7 +232,11 @@ async fn cmd_show(
);
}
Err(reason) => {
println!(" INVALID — {reason}");
// A version refusal means no cryptographic check ran, so it must
// not print as an INVALID signature: operators and log scrapers
// would read that as a failed Ed25519 verification. INVALID is
// reserved for failures out of verify_signature itself.
println!(" {} — {reason}", signature_verdict_label(&parsed_version));
}
}
@@ -198,29 +253,39 @@ async fn cmd_show(
},
Err(e) => Err(e.to_string()),
};
let current_node_did = current.as_ref().ok().cloned();
for line in did_check_report(&current, node_did) {
println!("{line}");
}
if require_valid {
if let Err(reason) = verdict {
anyhow::bail!("certificate signature did not verify: {reason}");
// Round 3: do not claim the signature failed when it was
// never checked. A version refusal means the payload
// shape is unknown, so no signature verification ran.
anyhow::bail!("certificate could not be verified: {reason}");
}
// A valid signature proves internal consistency only: the payload was
// signed by whatever key the certificate itself names. A hostile
// source can mint a keypair, put its DID in node_did, and self-sign.
// --verify therefore also anchors the issuer to a trusted DID:
// --expect-node when given, else the DID of the node being queried.
let expected = expect_node.as_deref().or(current_node_did.as_deref());
match expected {
// --verify therefore also anchors the issuer — and the anchor must be
// a DID the CALLER trusts, which is exactly what --expect-node is.
// The DID the queried node reports at `/` is NOT an anchor: the same
// node that served a forged certificate can serve a matching
// self-report, and with that fallback a wholly forged cert printed
// VALID and exited 0 (review round 2, demonstrated by execution). So
// --verify without --expect-node refuses to claim a trusted issuer at
// all, rather than laundering the server's self-assertion into one.
match expect_node.as_deref() {
Some(expected) if expected == node_did => {}
Some(expected) => anyhow::bail!(
"certificate is signed by {node_did}, but the expected issuer is {expected} — \
a valid signature alone proves internal consistency, not a trusted issuer"
),
None => anyhow::bail!(
"cannot anchor the issuer: node info is unreachable and no --expect-node was given"
"--verify needs --expect-node <did> to anchor the issuer: the signature is \
valid, but it only proves the cert is self-consistent with the DID it \
names ({node_did}); the queried node's self-reported DID is not a trust \
anchor"
),
}
}
@@ -268,6 +333,18 @@ fn did_check_report(current: &std::result::Result<String, String>, node_did: &st
}
}
/// The stdout label for a failed verification verdict. A `parsed_version`
/// error means `verify_signature` never ran, so the refusal must print as
/// UNSUPPORTED VERSION rather than INVALID, which operators and log
/// scrapers would read as a failed Ed25519 check.
fn signature_verdict_label(parsed_version: &std::result::Result<u32, String>) -> &'static str {
if parsed_version.is_err() {
"UNSUPPORTED VERSION"
} else {
"INVALID"
}
}
/// Rebuild the node's canonical signing payload (field order must match
/// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the
/// certificate's Ed25519 signature against the key embedded in `node_did`.
@@ -314,6 +391,65 @@ fn verify_signature(
.map_err(|_| "Ed25519 signature does not match the signed payload".to_string())
}
/// Parse the wire-format `version` field of a certificate response.
///
/// Semantics (Reviewer 2):
/// - MISSING key (`None`) → `Ok(1)`. An old server predates the field, so
/// the cert is by definition the v1 (pre-versioning) shape; the v1
/// verify path is the only path that can possibly match the bytes.
/// - PRESENT key must be a JSON integer that fits losslessly in `u32`
/// and equals a version this client supports. `null`, a string,
/// a float, an overflow integer, or any version other than 1 all
/// yield `Err`. Collapsing any of these onto v1 would let a
/// well-signed v1 signature pass `--verify` against a response
/// that explicitly declared a format this client cannot represent
/// — which is exactly the mismatch the version field exists to
/// prevent.
///
/// The supported-version set is hard-coded to `{1}` because PR 3 ships
/// only v1 verification; bump this set (and the verdict branch) when
/// v2 verification lands.
fn parse_cert_version(value: Option<&Value>) -> Result<u32, String> {
let v = match value {
None => return Ok(1),
Some(v) => v,
};
// serde_json::Value::as_u64 rejects strings, floats, booleans,
// nulls, arrays, and objects — but it silently truncates floats
// that are whole numbers (e.g. 2.0 → 2). We must reject floats
// explicitly so a server cannot smuggle a v2 cert through the
// float shape.
if v.is_f64() {
return Err(format!(
"version is a JSON number with a fractional part ({v}); expected an integer"
));
}
let n = v.as_u64().ok_or_else(|| {
// null / string / bool / array / object — anything that is
// not a JSON integer.
format!("version is {v}; expected a JSON integer")
})?;
// Lossy narrowing: refuse anything that does not fit in u32.
// u32::MAX is 4_294_967_295; serde_json only goes up to u64.
if n > u32::MAX as u64 {
return Err(format!(
"version {n} does not fit in u32 (max {})",
u32::MAX
));
}
let n = n as u32;
if n != 1 {
return Err(format!(
"this client supports cert version 1 only; server returned {n}"
));
}
Ok(1)
}
async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str) -> Result<String> {
if id.len() >= 36 {
return Ok(id.to_string());
@@ -811,7 +947,15 @@ mod tests {
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let got = cmd_show("alice/r".to_string(), id, server.url(), None, true, None).await;
let got = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(kp.did().as_str().to_string()),
)
.await;
assert!(got.is_ok(), "valid sig + matching issuer must pass");
cert.assert_async().await;
}
@@ -827,10 +971,17 @@ mod tests {
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
let err = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(other.did().as_str().to_string()),
)
.await
.unwrap_err()
.to_string();
assert!(err.contains("expected issuer"), "got: {err}");
cert.assert_async().await;
}
@@ -848,7 +999,7 @@ mod tests {
.await
.unwrap_err()
.to_string();
assert!(err.contains("cannot anchor the issuer"), "got: {err}");
assert!(err.contains("--expect-node"), "got: {err}");
cert.assert_async().await;
}
@@ -889,7 +1040,7 @@ mod tests {
.await
.unwrap_err()
.to_string();
assert!(err.contains("signature did not verify"), "got: {err}");
assert!(err.contains("does not match"), "got: {err}");
cert.assert_async().await;
}
@@ -973,4 +1124,373 @@ mod tests {
assert!(report.contains("verify offline"), "got: {report}");
assert!(!report.contains("WARNING"), "got: {report}");
}
/// #26 Split PR 3: a missing `version` field on a cert defaults
/// to 1, so an old server's response is forward-compatible with
/// a new client. The v1 verify path is taken, and a
/// well-signed v1 cert round-trips through `verify_signature`.
///
/// Reviewer 1 finding: the previous shape only parsed the field
/// and never called `verify_signature`, so the test could not
/// fail if the verdict branch was wired to the wrong path. This
/// version signs and verifies the full round-trip.
#[test]
fn missing_version_defaults_to_1_and_verifies() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
// Simulate the JSON a v1 server returns: no `version` key.
let cert_json = serde_json::json!({
"id": "test-id",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig.clone(),
"issued_at": "2026-07-22T00:00:00+00:00",
// no `version` field
});
// Forward-compat: the parse yields v1 (the only version this
// client verifies), and the verdict arm matching Ok(1) feeds
// the v1 payload through verify_signature end to end.
assert_eq!(parse_cert_version(cert_json.get("version")).unwrap(), 1);
let verdict = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(
verdict.is_ok(),
"missing version must take the v1 verify path: {verdict:?}"
);
}
/// #26 Split PR 3: an explicit `version: 1` on a cert is the v1
/// verify path. A v1 cert with an explicit version verifies the
/// same as a v1 cert without one — round-trip through
/// `verify_signature`.
#[test]
fn explicit_version_1_takes_v1_path() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
// Simulate a v1 cert with an explicit `version: 1` field.
let cert_json = serde_json::json!({
"id": "test-id",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig.clone(),
"issued_at": "2026-07-22T00:00:00+00:00",
"version": 1,
});
// The parse arms the Ok(1) verdict branch, which calls
// verify_signature on the v1 payload. A signature mismatch
// here would mean the verdict branch was wired to the
// wrong path or the payload drifted.
assert_eq!(parse_cert_version(cert_json.get("version")).unwrap(), 1);
let verdict = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(
verdict.is_ok(),
"explicit version: 1 must take the v1 verify path: {verdict:?}"
);
}
/// #26 Split PR 3 + Reviewer 2: parse_cert_version is the
/// load-bearing gate that distinguishes a missing-key (legacy
/// server) from a present-but-malformed value. The four cases
/// below pin every cell of that truth table; collapsing any
/// pair would let a server advertise "version 2" (or 99, or
/// 4294967297, or "two") while the client runs the v1
/// signature path on a different signed payload — a hostile
/// misconfiguration that `--verify` must not silently accept.
#[test]
fn parse_cert_version_truth_table() {
// MISSING key → legacy v1 (the v1 path is the only one that
// could match pre-versioning bytes).
assert_eq!(
parse_cert_version(None).unwrap(),
1,
"a missing version field is the legacy v1 path"
);
// Explicit 1 → v1.
assert_eq!(
parse_cert_version(Some(&serde_json::json!(1))).unwrap(),
1,
"explicit version 1 is the v1 path"
);
// Explicit 2 → Err, NOT Ok(1). Reviewer 2: a v1 signature
// must not verify against a server that said "version 2".
assert!(
parse_cert_version(Some(&serde_json::json!(2))).is_err(),
"explicit version 2 must not collapse to v1"
);
// u32 overflow (2^32 + 1) → Err, NOT Ok(1). The previous
// `.map(|v| v as u32)` silently truncated this to 1.
let overflow = serde_json::json!(u64::from(u32::MAX) + 1);
assert!(
parse_cert_version(Some(&overflow)).is_err(),
"a version that overflows u32 must not truncate to 1"
);
// Non-numeric: a string, a float, null, bool — every shape
// that is not a JSON integer is rejected. None of these
// may collapse to v1.
for bad in [
serde_json::json!("two"),
serde_json::json!(2.0), // serde_json::Value::is_f64 is true for this
serde_json::json!(2.5), // obviously fractional
serde_json::json!(true),
serde_json::json!(null),
serde_json::json!([2]),
serde_json::json!({"v": 2}),
] {
assert!(
parse_cert_version(Some(&bad)).is_err(),
"non-integer version {bad} must not collapse to v1"
);
}
}
/// A version refusal must not print as INVALID — no cryptographic
/// check ran, so labelling it INVALID would tell operators an Ed25519
/// verification failed when the payload shape was the disqualifier.
/// Removing the label branch fails this test.
#[test]
fn signature_verdict_label_distinguishes_refusal_from_bad_signature() {
assert_eq!(
signature_verdict_label(&Err("version 2 not supported".to_string())),
"UNSUPPORTED VERSION"
);
assert_eq!(signature_verdict_label(&Ok(1)), "INVALID");
}
/// #26 Split PR 3, round 2: a `version: 2` certificate must fail
/// `--verify` THROUGH `cmd_show` itself, not through a copy of its
/// match. The prior form of this test duplicated the verdict match in
/// the test body and aimed at an `Ok(v != 1)` arm the real parser can
/// never produce, so gutting `cmd_show`'s rejection kept every test
/// green (demonstrated by the reviewer by execution). Driving the
/// command end to end is what makes this a guard: the mock serves a
/// cert whose SIGNATURE IS VALID for v1 bytes but which declares
/// version 2, and the command must exit Err with the version named —
/// the v1 verify path must not launder it through.
#[tokio::test]
async fn cmd_show_verify_rejects_v2_even_with_valid_v1_signature() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
// Sign a v1-shaped payload so the ONLY disqualifier is the version.
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let cert_json = serde_json::json!({
"id": "0123456789abcdef0123456789abcdef0123",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
"version": 2,
});
let mut server = mockito::Server::new_async().await;
// 36-char id short-circuits resolve_cert_id; "owner/repo" input
// short-circuits resolve_repo — the cert GET is the only required
// route. `/` may be probed for the contextual DID line; serving it
// keeps the test about the verdict, not about network noise.
let _cert = server
.mock(
"GET",
"/api/v1/repos/o/r/certs/0123456789abcdef0123456789abcdef0123",
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_json.to_string())
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(serde_json::json!({ "did": node_did }).to_string())
.create_async()
.await;
let err = cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
Some(node_did.clone()),
)
.await
.expect_err("a version-2 cert must fail --verify even with a valid v1 signature");
let msg = format!("{err:#}");
assert!(
msg.contains("version"),
"the failure must name the version mismatch, not a generic invalid signature: {msg}"
);
}
/// Round 2's second demonstrated escape: with no --expect-node, the old
/// fallback anchored the issuer to the DID the QUERIED NODE reports at
/// `/` — which a forger controls along with the cert, so a wholly forged
/// self-signed certificate passed --verify with exit 0. `cmd_show` with
/// --verify and no --expect-node must now refuse to claim a trusted
/// issuer, even when the served cert's signature is valid and the node's
/// self-report matches it.
#[tokio::test]
async fn cmd_show_verify_without_expect_node_refuses_self_asserted_anchor() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let cert_json = serde_json::json!({
"id": "0123456789abcdef0123456789abcdef0123",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
});
let mut server = mockito::Server::new_async().await;
// P3 (reviewer round 2): both mocks are required to be
// hit for the test to mean anything. `mockito 1.7`'s
// `assert_on_drop` is off by default, so the previous
// `_cert` and `_root` bindings would silently let a
// regression that skipped the root fetch pass.
// `.expect(2)` pins each mock's call count (the cert mock
// is hit twice because the test runs `cmd_show` twice) and
// `.assert_async()` at the end fires if any are unmet.
let cert_mock = server
.mock(
"GET",
"/api/v1/repos/o/r/certs/0123456789abcdef0123456789abcdef0123",
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_json.to_string())
.expect(2)
.create_async()
.await;
// The forger's matching self-report — the exact shape
// that used to convert into a trust anchor. The guard's
// whole point is to make the test FAIL if a future
// regression stops calling `/` here: `.expect(2)` +
// `.assert_async()` makes that dependency visible.
let root_mock = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(serde_json::json!({ "did": node_did }).to_string())
.expect(2)
.create_async()
.await;
let err = cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
None,
)
.await
.expect_err("--verify without --expect-node must not mint a trust anchor from the node's self-report");
let msg = format!("{err:#}");
assert!(
msg.contains("--expect-node"),
"the refusal must tell the caller how to anchor trust: {msg}"
);
// Positive control: the same cert with the DID explicitly trusted
// passes, so the guard is about the anchor, not the signature.
cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
Some(node_did),
)
.await
.expect("explicitly anchored --verify of a valid v1 cert must pass");
// P3 (reviewer round 2): confirm the mocks were actually
// hit. Without this, a regression that skipped the cert or
// root fetches would still produce a passing test (the
// `expect_err` arm succeeds, the positive control succeeds)
// and the test would mean nothing.
cert_mock.assert_async().await;
root_mock.assert_async().await;
}
}
+177 -36
View File
@@ -3,7 +3,7 @@
//! Checks:
//! 1. Identity — ~/.gitlawb/identity.pem exists, DID parseable
//! 2. Registration — ~/.gitlawb/ucan.json exists (registered with a node)
//! 3. GITLAWB_NODE — env var is set to a non-localhost URL
//! 3. GITLAWB_NODE — env var points at a public node (unset is advisory)
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.twigpine.com)
//! 5. git helper — git-remote-gitlawb is in PATH
//! 6. git — git is installed
@@ -80,7 +80,6 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
});
let mut checks = Vec::new();
let mut all_ok = true;
// ── 1. Identity ───────────────────────────────────────────────────────
let pem_path = dir.join("identity.pem");
@@ -141,32 +140,16 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
match std::env::var("GITLAWB_NODE") {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Ok(v) if is_loopback_url(&v) => {
checks.push(Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
));
}
Ok(v) if !v.is_empty() => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
_ => {
checks.push(Check::fail(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
"export GITLAWB_NODE=https://node.twigpine.com",
));
}
}
checks.push(gitlawb_node_env_check(
std::env::var("GITLAWB_NODE").ok().as_deref(),
));
// ── 4. Node connectivity ──────────────────────────────────────────────
let client = NodeClient::new(&args.node, None);
// GITLAWB_NODE="" binds args.node to "" through clap, and check 3 groups
// empty with unset. The probe must apply the same normalization or it
// would dial "" and Fail while the env row only Warns.
let node_url = effective_node_url(&args.node);
let client = NodeClient::new(node_url, None);
match client.get("/").await {
Ok(resp) if resp.status().is_success() => {
let info = resp.json::<serde_json::Value>().await.unwrap_or_default();
@@ -175,7 +158,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
let short_did = did.chars().take(30).collect::<String>();
checks.push(Check::pass(
"node",
format!("{} — v{version} ({short_did}…)", args.node),
format!("{} — v{version} ({short_did}…)", node_url),
));
// Capability drift: a node newer than this CLI may require features
// (RFC 9421 signing, iCaptcha) the CLI doesn't speak.
@@ -194,14 +177,14 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
Ok(resp) => {
checks.push(Check::fail(
"node",
format!("{} returned HTTP {}", args.node, resp.status()),
format!("{} returned HTTP {}", node_url, resp.status()),
"check GITLAWB_NODE env var or try gl register --node <url>",
));
}
Err(e) => {
checks.push(Check::fail(
"node",
format!("{} unreachable: {e}", args.node),
format!("{} unreachable: {e}", node_url),
"check your internet connection or set GITLAWB_NODE",
));
}
@@ -285,23 +268,21 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
CheckState::Fail => "✗",
};
println!(" {icon} {:<24} {}", check.label, check.detail);
if matches!(check.state, CheckState::Fail) {
all_ok = false;
}
}
println!();
let has_failures_now = has_failures(&checks);
let has_issues = checks
.iter()
.any(|c| matches!(c.state, CheckState::Fail | CheckState::Warn));
if !has_issues {
println!("Everything looks good. Run `gl quickstart` to create your first repo.");
} else {
if all_ok {
println!("Setup looks good with some warnings:");
} else {
if has_failures_now {
println!("Some checks failed. Suggested fixes:");
} else {
println!("Setup looks good with some warnings:");
}
for check in &checks {
if matches!(check.state, CheckState::Fail | CheckState::Warn) {
@@ -311,14 +292,75 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
}
println!();
if !all_ok {
if has_failures_now {
println!("Run `gl quickstart` for a guided setup.");
}
}
// #357: a non-zero exit is the only signal scripting and CI have. Warn-only
// rows stay exit 0 (iCaptcha offline, version drift, shell-alias shadowing
// are all non-fatal); Fail-class rows (identity, registration, node
// reachable, git-remote-gitlawb present, git present) flip to exit 1.
// Direct `process::exit` rather than returning Err: the prose above is the
// user-facing summary, and an anyhow Error frame would just duplicate it.
if has_failures(&checks) {
std::process::exit(1);
}
Ok(())
}
/// Classify the `GITLAWB_NODE` env var for `gl doctor`. Extracted so unit
/// tests can pin the Warn tier directly (#357): an unset env is advisory, not
/// a failure, because `--node` defaults to `https://node.twigpine.com` and the
/// CLI works without it. Flipping this back to `Check::fail` must break a
/// test, not just change prose.
fn gitlawb_node_env_check(env_val: Option<&str>) -> Check {
match env_val {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Some(v) if is_loopback_url(v) => Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
),
Some(v) if !v.is_empty() => Check::pass("GITLAWB_NODE", v.to_string()),
// `--node` defaults to `https://node.twigpine.com` and the CLI works
// fine without the variable, so an unset env is advisory, not a
// failure. Warn-class keeps the exit code 0 for a stock working
// install; this used to be Fail, which the obvious "return non-zero
// on any failure" change would have flipped to exit 1 on every
// default-config install (#357).
_ => Check::warn(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
format!("export GITLAWB_NODE={PUBLIC_NODE}"),
),
}
}
/// The URL the connectivity probe should dial. `GITLAWB_NODE` set but empty
/// (or whitespace-only) binds `args.node` to `""` through clap rather than
/// to the `default_value`, and check 3 already groups empty with unset, so
/// the probe applies the same normalization: an empty effective URL means
/// the public node, same as unset.
fn effective_node_url(node: &str) -> &str {
if node.trim().is_empty() {
PUBLIC_NODE
} else {
node
}
}
/// True when at least one check is Fail-class. Warn-only runs (iCaptcha offline,
/// version drift, shell-alias shadowing, GITLAWB_NODE unset) keep exit 0;
/// this is the gating predicate for #357's exit-status fix.
fn has_failures(checks: &[Check]) -> bool {
checks.iter().any(|c| matches!(c.state, CheckState::Fail))
}
/// Check if a binary name exists anywhere on PATH.
/// True when the rc file contains a real `unalias` command naming `gl` —
/// not a comment, and not a longer word like `unalias global`. Ordering
@@ -644,4 +686,103 @@ mod tests {
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("GitHub API returned HTTP 403"));
}
/// #357: an all-Ok run does not flip the exit code to non-zero — a healthy
/// install must keep returning 0 so scripts like `gl doctor && gl push`
/// keep working.
#[test]
fn exit_predicate_is_false_for_all_ok() {
let checks = vec![
Check::pass("identity", "ok"),
Check::pass("registration", "ok"),
Check::pass("git", "ok"),
];
assert!(!has_failures(&checks));
}
/// #357: Warn-class rows (iCaptcha offline, version drift, shell-alias
/// shadowing, GITLAWB_NODE unset on a default-config install) are NOT
/// exit-status failures — they are still printed to the user but the
/// process exits 0. This is the rule that the previous "any
/// Fail-or-Warn row → exit 1" change would have broken for stock
/// installs that simply have not set the env var.
#[test]
fn exit_predicate_is_false_for_warn_only() {
let checks = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export GITLAWB_NODE=..."),
Check::warn("iCaptcha", "offline", "check connectivity"),
Check::warn("version", "drift", "upgrade"),
Check::warn("shell alias", "omz git plugin", "unalias gl"),
];
assert!(!has_failures(&checks));
}
/// #357: a single Fail-class row trips the exit code. The set of Fail
/// rows is whatever `run` writes through `Check::fail`: identity missing
/// or unparseable, registration missing/malformed, node unreachable,
/// git-remote-gitlawb absent, git absent.
#[test]
fn exit_predicate_is_true_when_a_fail_row_is_present() {
let checks = vec![
Check::pass("registration", "ok"),
Check::fail("identity", "missing", "gl identity new"),
Check::pass("git", "ok"),
];
assert!(has_failures(&checks));
let mixed_warns = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export"),
Check::fail("node", "unreachable", "check network"),
Check::warn("iCaptcha", "offline", "check"),
];
assert!(has_failures(&mixed_warns));
}
/// #357: `GITLAWB_NODE` unset must be Warn, not Fail. This pins the
/// re-tier so flipping it back to `Check::fail` breaks the suite — the
/// previous `has_failures`-only tests built their own `Check::warn` rows
/// and could not see the re-tier at all.
#[test]
fn gitlawb_node_env_unset_is_warn() {
let check = gitlawb_node_env_check(None);
assert!(matches!(check.state, CheckState::Warn));
assert!(check.detail.contains("not set"));
let empty = gitlawb_node_env_check(Some(""));
assert!(matches!(empty.state, CheckState::Warn));
}
/// `GITLAWB_NODE=""` binds `args.node` to `""` through clap, and check 3
/// groups empty with unset. The probe must dial the same URL unset
/// would — removing the normalization makes check 4 probe `""` and Fail
/// while the env row only Warns, so an empty env install exits 1.
#[test]
fn effective_node_url_treats_empty_and_blank_as_unset() {
assert_eq!(effective_node_url(""), PUBLIC_NODE);
assert_eq!(effective_node_url(" "), PUBLIC_NODE);
assert_eq!(effective_node_url("\t\n"), PUBLIC_NODE);
assert_eq!(
effective_node_url("http://127.0.0.1:7545"),
"http://127.0.0.1:7545"
);
}
#[test]
fn gitlawb_node_env_set_is_pass() {
let check = gitlawb_node_env_check(Some("https://node.twigpine.com"));
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("https://node.twigpine.com"));
}
#[test]
fn gitlawb_node_env_loopback_is_pass() {
for url in ["http://127.0.0.1:7545", "http://localhost:7545"] {
let check = gitlawb_node_env_check(Some(url));
assert!(
matches!(check.state, CheckState::Ok),
"loopback {url} must be Ok, not Warn/Fail"
);
}
}
}
+5 -1
View File
@@ -142,7 +142,11 @@ pub async fn run(args: MirrorArgs) -> Result<()> {
println!();
println!("✓ Mirror complete: {name}");
println!(" Clone: git clone {gitlawb_url}");
println!(" View: https://twigpine.com/{owner_short}/{name}");
// Only print View: when the node advertises a web_url — self-hosted nodes
// without a web front-end would otherwise produce a 404 link (#370).
if let Some(web_url) = crate::repo::fetch_node_web_url(&args.node).await {
println!(" View: {web_url}/{owner_short}/{name}");
}
Ok(())
}
+405 -2
View File
@@ -5,7 +5,7 @@ use clap::{Args, Subcommand};
use serde_json::{json, Value};
use std::path::PathBuf;
use crate::http::NodeClient;
use crate::http::{sanitize_node_msg, NodeClient};
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -274,6 +274,126 @@ async fn resolve_owner_did(_node: &str, dir: Option<&std::path::Path>) -> Result
Ok(did.split(':').next_back().unwrap_or(&did).to_string())
}
/// Fetch `GET /` from the node and return the `web_url` if the node advertises one.
///
/// The View: link is a nice-to-have, so every failure mode degrades to `None`
/// rather than failing the enclosing command — but failures are surfaced as
/// stderr warnings (request error, non-success HTTP status with the status,
/// malformed advertised value), since they all mean the node itself is
/// misbehaving or misconfigured, which the user would otherwise never learn.
/// A successful response that lacks a usable `web_url` (field missing or
/// blank) omits the link silently: self-hosted nodes without a web front-end
/// are expected to not advertise one (#370).
///
/// The body is treated like every other caller-chosen node reply (INV-6): the
/// read is capped and an accepted `web_url` must be free of control/bidi bytes
/// — it reaches the terminal verbatim through the View: line.
pub(crate) async fn fetch_node_web_url(node: &str) -> Option<String> {
let info_client = NodeClient::new(node, None);
let info_resp = match info_client.get("/").await {
Ok(resp) => resp,
Err(err) => {
eprintln!("warning: node info request failed ({err}); skipping View link");
return None;
}
};
let status = info_resp.status();
if !status.is_success() {
eprintln!("warning: node info request returned {status}; skipping View link");
return None;
}
// An info reply is a DID, a few URLs and counts — 8 KiB is well past what
// the shape needs; anything longer is hostile or broken (peer.rs precedent).
let raw_body = crate::http::read_body_capped(info_resp, 8 * 1024).await;
match web_url_from_info_body(&raw_body) {
Ok(url) => url,
Err(reason) => {
eprintln!("warning: {reason}; skipping View link");
None
}
}
}
/// Classify a capped `GET /` body for the `View:` advertisement. Split out of
/// `fetch_node_web_url` so tests can pin the classification contract directly:
/// a truncated or half-read body is indistinguishable from a node that
/// advertised nothing, so it must warn rather than silently omit the link.
fn web_url_from_info_body(
raw_body: &crate::http::CappedBody,
) -> std::result::Result<Option<String>, String> {
if raw_body.truncated || raw_body.read_failed {
return Err("node info body was oversized or could not be read in full".to_string());
}
let info: Value = match serde_json::from_str(&raw_body.text) {
Ok(json) => json,
Err(_) => return Err("node info body was not valid JSON".to_string()),
};
// Missing field / non-string degrade to None without warning — same contract
// as before; only transport- and advertisement-level failures warn there.
let raw = match info["web_url"].as_str() {
Some(v) => v,
None => return Ok(None),
};
let trimmed = raw.trim().trim_end_matches('/');
if trimmed.is_empty() {
return Ok(None);
}
// A present-but-malformed value means the node is misconfigured; say so
// instead of quietly dropping the link. Mirrors the node-side boot
// validation: must be an absolute http(s) URL with no query or fragment —
// the link is built by string-appending `/{owner}/{repo}` to it. Control
// and bidi-format bytes are rejected outright (not stripped): they have no
// legitimate place in a base URL and would reach the terminal through the
// View: line — including when they arrive percent-encoded (INV-6).
let malformed_reason: Option<String> = if trimmed
.chars()
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c))
{
Some("contains control or bidi characters".to_string())
} else {
match trimmed.parse::<url::Url>() {
Err(_) => Some("not an absolute URL".to_string()),
Ok(parsed) if !matches!(parsed.scheme(), "http" | "https") => {
Some("not http(s)".to_string())
}
Ok(parsed) if parsed.query().is_some() || parsed.fragment().is_some() => {
Some("contains a query or fragment".to_string())
}
// userinfo is rejected because the `View:` line prints the
// value verbatim; a remote advertisement carrying
// `user:pass@host` would land that credential in terminal
// scrollback and CI logs.
Ok(parsed) if !parsed.username().is_empty() || parsed.password().is_some() => {
Some("contains a username or password".to_string())
}
Ok(parsed)
if parsed
.host_str()
.unwrap_or("")
.chars()
.chain(
percent_encoding::percent_decode_str(parsed.path())
.decode_utf8_lossy()
.chars(),
)
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c)) =>
{
Some("contains control or bidi characters, even percent-encoded".to_string())
}
Ok(_) => None,
}
};
match malformed_reason {
// The reason string is ours; the advertised value is not — defang it
// exactly as it would have been defanged had it been accepted.
Some(reason) => Err(format!(
"node advertised a malformed web_url ({:?}, {reason})",
sanitize_node_msg(trimmed)
)),
None => Ok(Some(trimmed.to_string())),
}
}
async fn cmd_create(
name: String,
description: Option<String>,
@@ -311,7 +431,11 @@ async fn cmd_create(
println!("✓ Created repository: {name}");
println!(" Clone: git clone {gitlawb_url}");
println!(" HTTP: {clone_url}");
println!(" View: https://twigpine.com/{owner_short}/{name}");
// Only print View: when the node advertises a web_url — self-hosted nodes
// without a web front-end would otherwise produce a 404 link (#370).
if let Some(web_url) = fetch_node_web_url(&node).await {
println!(" View: {web_url}/{owner_short}/{name}");
}
if let Some(desc) = payload["description"].as_str().filter(|s| !s.is_empty()) {
println!(" Desc: {desc}");
}
@@ -848,6 +972,285 @@ mod tests {
.unwrap();
}
#[tokio::test]
async fn test_fetch_node_web_url_with_web_url() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"https://example.com","did":"did:key:z6Mk"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url.as_deref(), Some("https://example.com"));
}
#[tokio::test]
async fn test_fetch_node_web_url_trims_trailing_slash() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"https://example.com/"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url.as_deref(), Some("https://example.com"));
}
#[tokio::test]
async fn test_fetch_node_web_url_without_web_url() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:z6Mk"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
#[tokio::test]
async fn test_fetch_node_web_url_empty_string() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":""}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
#[tokio::test]
async fn test_fetch_node_web_url_whitespace_only() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":" "}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
#[tokio::test]
async fn test_fetch_node_web_url_server_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(500)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
/// A node advertising a present-but-malformed web_url (not an absolute
/// http(s) URL) must not yield a View link — and must warn, since a
/// malformed advertisement means the node is misconfigured (#370).
#[tokio::test]
async fn test_fetch_node_web_url_malformed_value_is_rejected() {
for bad in [
"gitlawb.com",
"not a url",
"ftp://files.example.com",
// Query/fragment corrupt the appended /{owner}/{repo} path.
"https://example.com?a=1",
"https://example.com/#faq",
] {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(format!(r#"{{"web_url":"{bad}"}}"#))
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(
web_url, None,
"malformed web_url {bad:?} must not render a View link"
);
}
}
/// A transport-level failure (connection refused) degrades to None but is
/// no longer silent: the request error is surfaced on stderr so an
/// unreachable node isn't indistinguishable from one without a web_url.
#[tokio::test]
async fn test_fetch_node_web_url_connection_refused_warns() {
// Port 1 on localhost is reserved (tcpmux) and refuses connections.
let web_url = fetch_node_web_url("http://127.0.0.1:1").await;
assert_eq!(web_url, None);
}
/// A hostile node can smuggle ANSI/bell/bidi controls inside a web_url that
/// still passes http(s) URL parsing — those bytes would reach the terminal
/// verbatim through the View: line. The advertised value must be rejected
/// outright: no control byte may survive into the returned string.
#[tokio::test]
async fn test_fetch_node_web_url_rejects_control_bytes() {
for bad in [
"https://example.com/\x1b[31mred", // ANSI CSI escape in path
"https://example.com\x07", // bell
"https://ex\u{202e}ample.com", // bidi override (RLO)
"https://example.com/\u{200f}", // RLM format char
"\x1b]0;title\x07https://evil.example", // OSC title-set prefix
] {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
// serde_json escapes the control bytes so the body is VALID
// JSON — a raw literal would make parse fail and the test
// would pass without reaching the control check at all.
.with_body(serde_json::json!({"web_url": bad}).to_string())
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(
web_url, None,
"web_url with control bytes {bad:?} must be rejected"
);
}
}
/// An oversized or half-read `GET /` body must warn, not silently
/// classify as "no web front-end": a cut body fails JSON parse, which is
/// indistinguishable from a node that advertised nothing without the
/// truncated flag. Removing the flag check makes this test pass
/// vacuously through the parse-fail arm.
#[test]
fn web_url_from_info_body_warns_on_truncated_reads() {
for (body, label) in [
(
crate::http::CappedBody {
text: r#"{"web_url":"https://x"#.repeat(2000),
truncated: true,
read_failed: false,
},
"truncated",
),
(
crate::http::CappedBody {
text: r#"{"web_url":"https://x"#.to_string(),
truncated: false,
read_failed: true,
},
"read_failed",
),
] {
let err = web_url_from_info_body(&body)
.expect_err(&format!("{label} body must warn, not silently classify"));
assert!(
err.contains("oversized or could not be read"),
"{label}: got {err}"
);
}
// A clean body missing web_url still classifies as a silent None.
let clean = crate::http::CappedBody {
text: r#"{"did":"did:key:x"}"#.to_string(),
truncated: false,
read_failed: false,
};
assert_eq!(web_url_from_info_body(&clean).unwrap(), None);
}
/// Percent-encoded bidi must not slip past the literal scan: a node
/// advertising `https://example.com/%E2%80%AE` serves wire bytes with no
/// Cf code point, but a browser decodes it into a reordering character
/// on the rendered View: link (INV-6).
#[test]
fn web_url_from_info_body_rejects_percent_encoded_bidi() {
let body = crate::http::CappedBody {
text: r#"{"web_url":"https://example.com/%E2%80%AE"}"#.to_string(),
truncated: false,
read_failed: false,
};
let err = web_url_from_info_body(&body).expect_err("percent-encoded bidi must be rejected");
assert!(err.contains("bidi"), "got {err}");
}
#[tokio::test]
async fn test_fetch_node_web_url_validates_after_trimming() {
// Trailing slash is trimmed before validation; result stays usable.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"http://localhost:8080/ui/"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url.as_deref(), Some("http://localhost:8080/ui"));
}
/// A remote node that advertises `web_url` with userinfo (`user@`,
/// `user:pass@`) must be dropped: the value is reprinted on the `View:`
/// line, and a credential embedded in the URL would land in terminal
/// scrollback and CI logs. Mirrors the node-side boot validation.
#[tokio::test]
async fn test_fetch_node_web_url_rejects_userinfo() {
for bad in [
"https://[email protected]",
"https://user:[email protected]",
] {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(format!(r#"{{"web_url":"{}"}}"#, bad))
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(
web_url, None,
"web_url with userinfo {bad:?} must be rejected"
);
}
}
/// A non-success status must still return None (View link is cosmetic) but
/// the status surfaces as a user-visible stderr warning instead of being
/// swallowed silently (#370 review).
#[tokio::test]
async fn test_fetch_node_web_url_non_success_warns_with_status() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(503)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
// The warning itself goes to stderr; mockito can't capture it here, so
// this asserts only the None contract. Manual check: run against a
// 503-ing node and observe "node info request returned 503" on stderr.
}
#[tokio::test]
async fn test_cmd_create_server_error() {
let dir = TempDir::new().unwrap();
+138
View File
@@ -0,0 +1,138 @@
//! End-to-end test for the `View:` line printed by `gl repo create` (#370).
//!
//! `cmd_create` is the load-bearing #370 path: it is the one place the CLI
//! prints a `View:` line, and the only thing keeping it from being a
//! hardcoded `https://gitlawb.com/...` 404 is `fetch_node_web_url`. The
//! helper-level tests in `repo.rs` cover the helper, not the command path —
//! if `cmd_create` went back to a hardcoded constant, every helper test
//! would still pass.
//!
//! This integration test drives the real `gl` binary against a mockito
//! server that answers both `POST /api/v1/repos` and `GET /`. Asserting on
//! the subprocess's stdout is straightforward: bytes are bytes, with no
//! libtest / gag / in-process capture race.
//!
//! CARGO_BIN_EXE_gl is set by Cargo for integration tests of a `[[bin]]`
//! in the same package, so the test always picks up the in-tree build.
use std::process::Stdio;
use tempfile::TempDir;
use tokio::io::AsyncWriteExt;
use tokio::process::Command;
fn gl_bin() -> String {
std::env::var("CARGO_BIN_EXE_gl").expect("CARGO_BIN_EXE_gl is unset for this test")
}
async fn write_identity(dir: &TempDir) {
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
let path = dir.path().join("identity.pem");
let mut f = tokio::fs::File::create(&path).await.unwrap();
f.write_all(pem.as_bytes()).await.unwrap();
}
/// The trailing key segment of the freshly-generated identity. Mirrors the
/// in-source `resolve_owner_did` test helper but lives in this integration
/// file because that helper is `pub(crate)`.
async fn owner_short(dir: &TempDir) -> String {
let pem = tokio::fs::read_to_string(dir.path().join("identity.pem"))
.await
.unwrap();
let kp = gitlawb_core::identity::Keypair::from_pem(&pem).unwrap();
let did = kp.did().to_string();
did.split(':').next_back().unwrap_or(&did).to_string()
}
async fn run_gl_create(node_url: &str, dir: &TempDir) -> String {
let output = Command::new(gl_bin())
.arg("repo")
.arg("create")
.arg("myrepo")
.arg("--private")
.arg("--branch")
.arg("main")
.arg("--node")
.arg(node_url)
.arg("--dir")
.arg(dir.path())
.env("NO_COLOR", "1")
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.await
.expect("failed to spawn `gl repo create`");
assert!(
output.status.success(),
"`gl repo create` failed: stderr:\n{}\nstdout:\n{}",
String::from_utf8_lossy(&output.stderr),
String::from_utf8_lossy(&output.stdout),
);
String::from_utf8(output.stdout).expect("`gl` wrote non-UTF-8 to stdout")
}
#[tokio::test]
async fn cmd_create_view_url_tracks_node_advertisement() {
let dir = TempDir::new().unwrap();
write_identity(&dir).await;
let owner = owner_short(&dir).await;
// Case 1: node advertises a web_url. The `View:` line must use the
// advertised origin and the resolved owner/short name.
{
let mut server = mockito::Server::new_async().await;
let _m_create = server
.mock("POST", "/api/v1/repos")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"name":"myrepo","clone_url":"gitlawb://did:key:z6Mk/myrepo"}"#)
.create_async()
.await;
let _m_info = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"https://git.example","did":"did:key:z6Mk"}"#)
.create_async()
.await;
let server_url = server.url();
let out = run_gl_create(&server_url, &dir).await;
let view_line = out
.lines()
.find(|l| l.trim_start().starts_with("View:"))
.unwrap_or_else(|| panic!("View: line missing from stdout:\n{out}"));
assert!(
view_line.contains(&format!("https://git.example/{owner}/myrepo")),
"View: line must use the advertised origin, got: {view_line:?}"
);
}
// Case 2: node does NOT advertise a web_url. The `View:` line must be
// absent, so a self-hosted node without a web front-end never produces
// a 404 link (#370).
{
let mut server = mockito::Server::new_async().await;
let _m_create = server
.mock("POST", "/api/v1/repos")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"name":"myrepo","clone_url":"gitlawb://did:key:z6Mk/myrepo"}"#)
.create_async()
.await;
let _m_info = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:z6Mk"}"#)
.create_async()
.await;
let server_url = server.url();
let out = run_gl_create(&server_url, &dir).await;
assert!(
!out.lines().any(|l| l.trim_start().starts_with("View:")),
"View: line must be absent when node omits web_url; got:\n{out}"
);
}
}
+48
View File
@@ -0,0 +1,48 @@
//! #357: pin the `gl doctor` exit-code wiring, not just the predicate.
//!
//! The unit tests for `has_failures` never touch `std::process::exit(1)` or
//! the `GITLAWB_NODE` Warn tier. Deleting the exit gate outright kept
//! `cargo test -p gl doctor::` green, so this binary probe is the only thing
//! that breaks when the wiring is gutted.
use std::process::Command;
#[test]
fn doctor_exits_nonzero_when_fail_rows_present() {
let dir = tempfile::tempdir().expect("temp dir");
// Use an unroutable node so the "node unreachable" check is Fail-class.
// The temp dir has no identity.pem / ucan.json, so identity + registration
// are also Fail — at least one Fail must flip the process to exit 1.
let bin = env!("CARGO_BIN_EXE_gl");
let output = Command::new(bin)
.args([
"doctor",
"--dir",
dir.path().to_str().unwrap(),
"--node",
"http://127.0.0.1:1",
])
// Inherit PATH but neutralize GITLAWB_NODE so the test is deterministic
// across machines that have it set (otherwise it would be Pass).
.env_remove("GITLAWB_NODE")
// iCaptcha probes a real URL by default; keep it unreachable too so
// it stays Warn and does not affect the Fail gating.
.env("GITLAWB_ICAPTCHA_URL", "http://127.0.0.1:1")
.output()
.expect("run gl doctor");
// The command should have exited 1 because at least one Fail row exists.
// If the `std::process::exit(1)` gate is removed, this becomes 0 and the
// test fails — that is the regression it pins.
assert!(
!output.status.success(),
"gl doctor must exit non-zero when Fail rows are present; stdout: {}\nstderr: {}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
assert_eq!(
output.status.code(),
Some(1),
"exit code must be 1, not some other non-zero"
);
}
+1
View File
@@ -8,6 +8,7 @@ Step-by-step guide to staking $GITLAWB, registering your node on-chain, and earn
- A wallet with at least **10,000 $GITLAWB** (minimum stake) plus a small amount of ETH on Base for gas
- Docker or Rust 1.91+ (for running the node process)
- PostgreSQL 16+ — the node uses `pg_input_is_valid`, which exists only on PostgreSQL 16 and later. The bundled `docker-compose.yml` pins `postgres:16-alpine`; if you supply your own `DATABASE_URL`, the server must be 16+.
- A public HTTP URL (your-host.com) — can be a VPS, Fly.io app, or anything reachable. A Fly.io config is provided at `infra/fly/fly.toml` (deploy from the repo root with `fly deploy -c infra/fly/fly.toml`)
---
+76
View File
@@ -0,0 +1,76 @@
# GraphQL repository pagination
`repos` returns the complete visible repository list when it contains at most
200 entries. Larger lists return a GraphQL error directing the caller to
`reposPage`; they are never silently truncated.
Use `reposPage` for larger lists:
```graphql
query Repositories($after: String) {
reposPage(limit: 50, after: $after) {
nodes {
name
ownerDid
}
hasNextPage
endCursor
}
}
```
Start with `after: null`. When `hasNextPage` is true, pass the returned
`endCursor` as `after` on the next request. Stop when `hasNextPage` is false.
An empty page has a null `endCursor`. The default limit is 50 and the maximum
page size is 200; limits below 1 or above 200 are rejected (see Query budgets
below). Malformed cursors, including decoded owner or name strings containing
NUL, return `invalid repository cursor` before database access.
Pages are ordered by normalized owner DID and repository name. Visibility and
mirror deduplication are applied in the database before limiting the page;
hidden and quarantined repositories do not occupy page slots or create a
continuation signal. Cursors contain a position from the last returned visible
repository, not a permission grant. Each request checks the current caller's
visibility independently. Keep the same caller while traversing a list.
Malformed reader lists in a repository's root visibility rule deny access to
non-owners for that repository while other visible repositories remain listable.
Pagination is not a snapshot: concurrent renames, ownership changes, or visibility
changes can alter subsequent pages. Treat cursors as opaque and restart from the
first page when a fresh complete traversal is needed. Small legacy `repos`
queries retain their activity ordering.
## Query budgets
Every GraphQL document is validated against two budgets before any resolver
runs. Both apply per document (per request), not per client: `/graphql` has
no per-caller rate limiting, so a rejected document can always be retried
smaller. The constants live in `crates/gitlawb-node/src/graphql/mod.rs`.
- **Complexity budget: 400 per document** (`GRAPHQL_MAX_COMPLEXITY`). Each
root field carries a base cost plus the clamped page size plus the selected
child fields:
- `repos`: 50 + `MAX_VISIBLE_REPO_PAGE_SIZE` + child fields. `repos`
always scans the full visible corpus, so it is priced as the full page
it is.
- `reposPage`: 50 + `MAX_VISIBLE_REPO_PAGE_SIZE` + child fields. Every
page request sorts the full deduplicated corpus, so the price does not
depend on `limit`: seven `reposPage(limit: 1)` aliases cost the same as
seven max-size pages.
- `refUpdates`: 100 + clamped `limit` (`MAX_VISIBLE_REF_UPDATES`) + child
fields. The 100 base covers the collector's fixed scan (the deduplicated
repo set, the quarantined set, the visibility rules, and the bounded
up-to-2048-row walk past withheld rows); the limit term prices only the
rows returned.
- `tasks`: 50 + clamped `limit` (`MAX_VISIBLE_TASK_PAGE_SIZE`) + child
fields.
- `task`, mutations, subscriptions: 50 + child fields.
A document whose total exceeds 400 is rejected with
`Query is too complex.` before any resolver runs, so an over-budget
document needs fewer aliases or fewer child fields. A smaller `limit` does
not help `reposPage` or `repos`: both are priced at the full
`MAX_VISIBLE_REPO_PAGE_SIZE` scan regardless of `limit`.
- **Depth cap: 14 per document** (`GRAPHQL_MAX_DEPTH`). Documents nested
deeper than 14 selections are rejected with `Query is nested too deep.`