Compare commits

...
378 Commits
Author SHA1 Message Date
beardthelionandAyush7614 42cc893fb3 fix(gl): make doctor exit non-zero on Fail-class checks (#357) (#547)
* fix(gl): make doctor exit non-zero on Fail-class checks (#357)

gl doctor printed per-row status and 'Some checks failed' prose, but its
exit code was always 0. A diagnostic whose exit status cannot express
failure is a trap waiting for the first 'gl doctor && ...' to land
somewhere it should not (#357).

Three changes:

1. Re-tier the 'GITLAWB_NODE unset' check from Fail to Warn. The CLI's
   --node flag defaults to https://node.gitlawb.com (PUBLIC_NODE) and the
   CLI works fine without the env var, so an unset env is advisory, not
   a failure. Without this re-tier the obvious 'return non-zero on any
   failure' change would have flipped gl doctor to exit 1 on a stock
   working install, which is exactly the regression the issue warns
   against.

2. Route the exit-code decision through a new has_failures(&[Check])
   helper: exit 1 if and only if at least one row is Fail-class.
   Warn-class rows (iCaptcha offline, version drift, shell-alias
   shadowing, GITLAWB_NODE unset) keep the process at exit 0 because
   those conditions are still printed to the user and the obvious
   '&& pipeline' pattern stays valid. Use std::process::exit(1) rather
   than returning Err so anyhow's error frame does not duplicate the
   user-facing summary that already prints.

3. Three unit tests pin the new predicate against the three regimes:
   all-Ok, warn-only, and a single Fail row tripping the exit. The
   warn-only case is the regression guard for the re-tier: if anyone
   flips GITLAWB_NODE back to Fail, exit_predicate_is_false_for_warn_only
   fails too, so the two halves of the fix cannot drift apart.

Verified end-to-end against the public gitlawb node:
  * unset GITLAWB_NODE, missing identity/registration  -> exit 1
  * unset GITLAWB_NODE, full healthy install           -> exit 0 (warn)
  * missing git-remote-gitlawb                         -> exit 1

* fix(gl): pin doctor exit wiring and GITLAWB_NODE Warn tier

- Extract GITLAWB_NODE env classification into gitlawb_node_env_check()
  so unit tests can pin the Warn tier directly; the previous
  has_failures-only tests built their own Check::warn rows and could not
  see a re-tier back to Fail.

- Add three unit tests for the extracted helper (unset/empty is Warn,
  set is Pass, loopback is Pass).

- Add integration probe crates/gl/tests/doctor_exit.rs that drives the
  real binary with a temp --dir and --node http://127.0.0.1:1 and asserts
  exit 1. Deleting the std::process::exit(1) gate keeps the unit suite
  green but breaks this probe, closing the gap reviewer noted.

Co-authored-by: review fix for #391

* fix(gl): align GITLAWB_NODE operator copy with PUBLIC_NODE

* fix(gl): dial the public node when GITLAWB_NODE is empty

Check 3 groups an empty env with unset (Warn), but clap still binds
args.node to "" and the connectivity probe dialed it verbatim, so a
GITLAWB_NODE="" install showed a Fail row and exited 1. effective_node_url
normalizes empty or whitespace-only input to PUBLIC_NODE before the
probe so empty matches unset on check 4.

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-08 04:42:15 +00:00
0189990ac9 fix(node): bound federated repository aggregation (#546)
* fix(node): bound federated repository aggregation

* fix(node): propagate federated peer truncation

* fix(node): Bound federation latency and preserve truncated peer pages.

* fix(node): complete federation review follow-ups

Register the bounded peer-query fixture in the writer ledger, name federation bounds, and share the state-owned limiter across routers and the cleanup sweep. Cover exhausted-bucket rejection before database access.

* fix(node): pin federated response contract end to end (#466 review)

Address beardthelion's P2: the federated envelope was only exercised at
unit level, so hardcoding "truncated": false, deleting the failed-peer
truncated flag, or dropping node_did from enrich_federated_repo each left
all nine federated tests green.

Add four route tests driving build_router and asserting on the real HTTP
200 body:
- envelope fields (count, nodes_queried, truncated) and local annotations
  (node_url, node_did, local) for a seeded public repo
- truncated == true when the peer table overflows MAX_FEDERATED_PEERS
  (unreachable peer URLs, no live peer needed)
- truncated == true on the failed-fetch path with the table under the bound
- peer annotations (node_url, node_did, local: false) via a mockito peer

* fix: strengthen federated response bound tests per review

- db/mod.rs: add writer-ledger entries and disposition docs for the
  three federated route tests touching the peers table
- api/repos.rs: seed the envelope test repo under a non-node owner DID
  so node_did must equal the actual node DID; pin the outbound peer
  query to exact limit=200&offset=0; make the peer-table-overflow test
  use a reachable stub for every peer so the count bound is the only
  possible reason for truncated=true, and assert nodes_queried

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: Marlowe <[email protected]>
2026-11-08 02:24:53 +00:00
beardthelion 61d8a4f8f1 fix(core): compute created-skew in checked arithmetic (#358) (#545)
created= is attacker-supplied i64 with no range clamp. now - created
overflows on i64::MIN (panic in debug, wrap in release), and the wrap
case made abs() return i64::MIN, which is not > 300, so a skew of 2^63
seconds read as fresh. Treat an unrepresentable difference as the
largest possible skew.
2026-11-07 23:24:17 +00:00
b33562f1a9 fix(node): bound Identify-derived Kademlia addresses (#544)
* fix(node): bound Identify-derived Kademlia addresses.

* fix(node): preserve refreshed Identify addresses.

* fix(node): Log rejected known-peer addresses for diagnosis.

* fix(node): bound Identify report processing and refresh eviction

Avoid retaining peers with empty or rejected address reports, bound canonicalization to a fixed input prefix, and move refreshed addresses behind stale addresses in global eviction order. Add regressions for each bound and below-cap growth.

* fix(node): address CodeRabbit majors on Identify address bounds

- Reject Identify address admissions once the global address budget
  is full instead of evicting another peer's entry. Cross-peer
  eviction let a flood of new identities push honest peers'
  addresses out of the book and, when the evicted address was their
  last one, out of the Kademlia routing table. Per-peer eviction of
  the reporting peer's own addresses is unchanged.
- Re-add refreshed Identify addresses to Kademlia, not just newly
  added ones. Kademlia may have dropped a retained address after a
  failed dial or never kept it for a full k-bucket, and a TTL
  refresh alone would never re-admit it.

* fix(node): address #462 review findings from beardthelion

- remove the write-only insertion_order bookkeeping (field,
  remove_insertion_token, and its call sites) and collapse the repeated
  peers.get/get_mut lookups in IdentifyAddressBook::update into a single
  entry binding
- drop a peer row at the end of update() when every candidate was
  rejected, so zero-address rows no longer linger until the expiry tick
- add a regression test proving per-peer eviction still applies at full
  global budget (fails if the global check is hoisted above eviction)
- extract the explicit-address retention predicate into the tested
  is_explicit_address helper used by both Kademlia removal sites

* fix(node): pin row-removal and expiry guards in Identify address book

Address beardthelion's round-two P3 findings on #462:

- Add test that a fully-rejected fresh peer at full global budget
  leaves no peers row (pins the empty-row drop in update()).
- Add test that update() past IDENTIFY_ADDRESS_TTL reports the stale
  address in changes.removed (pins expiry reporting).
- Add test that expire() drops rows emptied by expiry (pins the
  sibling drop in expire_peer).
- Distinguish per-peer from cross-peer eviction in the saturation
  test: insert a sentinel peer before the victim and assert its
  address survives, so a global-oldest eviction policy would fail.

* fix(node): pin refresh predicate negative direction per beardthelion round 3

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: Marlowe <[email protected]>
Co-authored-by: cairn-intern <[email protected]>
2026-11-07 22:49:33 +00:00
beardthelionandGravirei d3845ced69 fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#121) (#543)
* fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#134)

Add auth rejection to list_pins and list_anchors handlers. The pin index
spans the entire node and would expose metadata for every object ever
pushed; anonymous callers must not see it.

- Require AuthenticatedDid extension in both handlers, return 401 when absent
- Add server.rs regression test for anonymous rejection through build_router
- Fix closed-pool tests to pass authenticated requests (test 503 path)

* fix(node): enforce authentication on /api/v1/ipfs/pins and /api/v1/arweave/anchors routes

* fix(node): format arweave routes for consistency in build_router function

* fix(node): gate scoped /api/v1/arweave/anchors reads on repository visibility

Closes the review follow-up on #134: a signed non-reader could pass the
auth-layer check on `?repo=<owner>/<name>` and obtain private-repo anchor
metadata (ref names, old/new SHAs, CIDs, irys tx ids). The auth check
proved only that the signature was valid; it did not bind the request to
the canonical repo-read policy.

* Parse the slug through `validate_repo_slug` (same helper the sync path
  uses) and call `authorize_repo_read` before the SQL query. Missing
  repos, quarantined mirrors, and signed non-readers all collapse to the
  standard 404 (`repo_not_found`).
* The unscoped global listing stays auth-only (#121 narrowing; the wider
  permissionless-identity enumeration is the #136 class, out of scope).
* Clamp the limit to `[0, 200]` so `?limit=-1` no longer reaches Postgres
  as `LIMIT -1`.

Five new `build_router` regressions cover the full production contract
through the real `optional_signature` middleware: scoped unsigned 401,
scoped owner 200, scoped non-reader 404 with no anchor metadata leaked
in the body, scoped missing-repo 404 indistinguishable from non-reader,
and `?limit=-1` clamped to an empty 200.

* fix(node): keep /api/v1/ipfs/pins out of the CID resolver's per-IP flood brake

Follow-up to #134: the previous reordering put `/api/v1/ipfs/pins` and
`/ipfs/{cid}` under the same `rate_limit_by_ip` + `IpRateLimiter` layer
stack, so both routes shared the resolver's flood brake bucket. That
bucket is documented as the once-per-request brake for `GET /ipfs/{cid}`
because the resolver can drive bounded-but-expensive repo walks. The
pin listing is a single `list_pinned_cids()` call and has no reason to
share the bucket. Sharing meant /ipfs/{cid} traffic could exhaust the
bucket and 429 the pins endpoint, and signed pin polling could exhaust
the bucket and 429 legitimate CID reads.

Compose `ipfs_routes` from two sub-routers with independent policies,
then merge:
  * `/ipfs/{cid}` keeps `optional_signature` + `rate_limit_by_ip` + the
    `IpRateLimiter` extension. Same behavior as before #134.
  * `/api/v1/ipfs/pins` gets only `optional_signature`. Auth-required
    (handler denies with 401), but no rate limit — it has nothing to
    flood.

Two new `build_router` regressions exercise the bucket independence
through `TrustedProxy::XForwardedFor`:
  * 5 signed pin requests against a size-2 CID bucket all 200, then a
    `/ipfs/{cid}` from the same IP also 200 (pins did not debit the
    bucket).
  * A `/ipfs/{cid}` exhausts a size-1 CID bucket (200, then 429), then
    a signed pin from the same IP is 200 (the bucket does not gate
    pins).

All 65 tests in `server::tests::` + `test_support::tests::ipfs_*`
still pass, including the existing 429 tests for the CID resolver.

* fix(node): normalize scoped anchor SQL filter to stored repo slug

After authorize_repo_read succeeds, the SQL filter used the raw ?repo=
string (format!("{owner}/{name}")). Anchor rows are stored with
normalize_owner_key(owner_did)/name (short key), so a caller using
?repo=did🔑.../name passed authz but got anchors: [] — a false
empty page that broke callers that legitimately pass the full DID
form. Build the filter from record.owner_did normalized and
record.name instead, matching the shape the seeder uses.

* All three findings addressed on bug_fix_2:

  P2 — signed_scoped_anchors_full_did_form_matches_stored_slug_through_build_router now seeds
  a second repo under a different owner and asserts irys-other-repo-tx is absent. Negative
  control: flipping the production filter to None made the test go RED with count: 2 showing
  the cross-repo leak.

  P3 — let short = owner_did.split(':').next_back().unwrap() replaced with let short =
  crate::db::normalize_owner_key(&owner_did).to_string(). Reader and writer now share one
  convention.

  P3 — PR #134 description updated: pins bullet now describes the actual pre-merge layering
  (and cites the axum-layer-vs-merge-pitfall memory entry), and round 2's stored-slug
  normalization is added as its own bullet with the regression's coverage.

  5/5 anchor-related tests pass, cargo fmt --check clean.

* fix(node): pass SignatureBodyLimit state to optional_signature layers

* test(node): complete listing-denial regressions for the new gates

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 21:46:53 +00:00
beardthelionandGravirei bac49f150a fix(node): owner-gate create_task repo_id against hosted repos (#541)
* fix(node): owner-gate create_task repo_id against hosted repos (#496)

* fix(node): address review on #496 gate — opaque lookup errors, no-persist asserts, GraphQL arm pins, honest oracle comment

* fix(node): address P3 review on #496 gate — shared denial text, honest oracle wording, db-fault pins

- Give the REST and GraphQL copies of the create_task owner gate one denial
  sentence: add REPO_OWNER_ONLY_MESSAGE, used by require_repo_owner and the
  REST forbidden() envelope, and pin it from both surface tests.
- Carry the corrected one-bit-oracle wording into the GraphQL gate comment and
  the REST test docstring (drop the "oracle-free" claim).
- Fault-inject the gate's repo lookups: drop machine_id (get_repo_by_id) for
  REST and GraphQL, and quarantined (is_repo_quarantined) for REST, asserting
  the fixed opaque db_error envelope and no schema text.

* docs(node): stop claiming caller-scoped task reads in #496 gate comments

The create_task gate is a write gate. Unknown/quarantined repo ids are stored
verbatim, and the REST/GraphQL task-list paths are caller-independent on this
base, so the comments must not imply those ids are read-scoped to the
delegator/assignee. Describe only the enforced ownership check; note the
one-bit oracle split without asserting a read contract.

* fix(node): mirror AppError's 503 split and opaque INSERT in create_task

- db_error now sends pool-class failures to 503 db_unavailable like
  AppError::Db (db_unavailable made pub(crate)); query faults stay the opaque
  500. Unit tests cover the split and context-wrapped sqlx errors.
- Route the create_task INSERT through db_error instead of serializing
  e.to_string(); a REST fault-injection test pins the opaque envelope.
- The deny/allow matrix reads back the unknown and quarantined tasks to prove
  the id is stored verbatim, and drops no-leak asserts on the fixed forbidden
  envelope that could never carry the repo id.

* test(node): pin stored-verbatim repo_id on GraphQL gate arms

The REST deny/allow matrix reads each accepted task back and asserts the
stored repo_id is the supplied id; the GraphQL unknown/quarantined arms only
asserted no errors, so a GraphQL regression that dropped, normalized, or
rebound the id stayed green. Read those tasks back by the mutation's returned
id and assert the stored label matches.

* test(node): pin createTask repo-ownership marker in the GraphQL authz fence

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 21:31:50 +00:00
beardthelionandGravirei b4a3122500 fix(core): reject zero threshold, canonicalize did:key, document replay (#509) (#540)
* fix(core): reject zero threshold, canonicalize did:key, document replay (#509)

* test(core): drop error echoes from assert messages flagged by CodeQL

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 15:48:05 +00:00
beardthelionandeuxarist 0cdb4bcc7c fix(api,sync): Delimit positional arguments in git clone mirror invocations (#539)
* fix(api,sync): Delimit positional arguments in git clone mirror invocations

Pass "--" before positional path/URL arguments in Command invocations
for "git clone --mirror" across repos.rs, sync.rs, and gl mirror.rs.

Fixes #374

* style(gl): Format args array in mirror.rs

Fix cargo fmt line-length break for git clone --mirror arguments.

Refs #376

* fix(node): Pin the git clone delimiter against remote-argument injection

The delimiter change had no test, so nothing stopped it being reverted as
cosmetic. What it guards is real: git parses a remote beginning with a
dash as an option, and --upload-pack=<cmd> is one git hands to a shell.

No path reaches that today. Peer URLs are gated by is_public_http_url, so
the origin clone_repo composes can only begin with http:// or https://;
fork paths are joins under repos_dir; and clap rejects a leading-dash
positional in gl mirror. The delimiter is what removes git's dependence
on those three unrelated gates staying correct.

Assert git reports the whole argument as a repository it cannot find,
which is only true once the delimiter forces it to be a path.
Undelimited, git consumes it as an option and fails against the
destination instead, never naming the injected string, so the assertion
is red before the fix and green after.

Refs #374

* fix(sync,gl): Delimit positional arguments in fetch_repo and gl clone

Carry the positional argument delimiter into fetch_repo when setting the
remote origin URL and into setup_partial_clone across all clone branches.
Correct the error string quoted in the clone regression test docstring.

Refs #374

* test(gl,node): Pin the argv delimiter on every git sink this PR changed

The delimiter contract is duplicated across five direct `git` argv
constructions, but only `sync::clone_repo` had a test that went red when
`--` was deleted. Removing or misplacing the delimiter in any other sink
left the suite green, so the hardening rested on review rather than on
the tests.

Each changed sink now has a subprocess-facing regression driven through
real git: the three `setup_partial_clone` shapes (plain, `--branch`, and
the sparse/promisor arm), `fetch_repo`'s `remote set-url`, `gl mirror`,
and the fork clone.

Two of these could not carry the obvious assertion:

- `git_global` and `git_run` both format failures as
  `git {args:?} failed: {stderr}`, so the injected value is already in
  the message via the debug-printed argv. Asserting that the error
  merely contains it passes with `--` deleted. The clone tests assert on
  git's own single-quoted `repository '<value>'` instead, which only
  appears when git read the value as a repository.
- `remote set-url` does not fail on a delimited option-shaped value, it
  stores it. The `fetch_repo` test asserts the stored URL rather than an
  error: undelimited, git exits 129 with `unknown option` and leaves the
  previous URL in place.

`gl mirror` and `fork_repo` needed their argv extracted to be testable.
`mirror::run` loads a keypair and contacts a node before reaching the
clone, and bails with a message that interpolates the source whether or
not the delimiter is present. The extraction keeps the production
`.status()` call, so a large mirror clone still streams git's progress
to the terminal instead of being captured.

Every one of the five was confirmed red with its delimiter removed and
green with it restored.

Refs #374

* fix(gl): Route the last three clone warnings through emit_warning

`emit_warning` sanitizes a line before writing it, which strips ANSI
escape and bidi characters from untrusted text. Three warning sites in
the blob-recovery paths still used a bare `eprintln!` while interpolating
`oid`, which comes from node JSON or an Arweave gateway manifest, so
those escapes reached the user's terminal raw.

Refs #374

* style(gl): elide the lifetime in mirror_clone_args

* test(gl,node): pin LC_ALL=C on git-subprocess stderr assertions

---------

Co-authored-by: euxarist <[email protected]>
2026-11-07 12:38:49 +00:00
0e5a1608e2 feat(node): persist upstream mirror configuration and transition state (#538)
* feat(node): persist upstream mirror transition state

Origin-Session: local-d6a143 | Codex | 12 prompts

* fix(node): reject null mirror transition state

Origin-Session: local-d6a143 | Codex | 14 prompts

* fix(node): address mirror state review findings

Origin-Session: local-d6a143 | Codex | 16 prompts

* fix(node): close mirror review blockers

Origin-Session: local-d6a143 | Codex | 20 prompts

* fix(node): align peer URL validation message

Origin-Session: local-d6a143 | Codex | 20 prompts

* fix(node): close remaining mirror review findings

Origin-Session: local-d6a143 | Codex | 21 prompts

* test(node): track the Db pool field change in the mirror-state migration tests

* fix(node): revalidate stored webhook URLs at delivery and cover the subdomain repeated-dot case

* docs(node): document the invalid_url webhook delivery outcome

---------

Co-authored-by: KRATOS <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-07 03:50:15 +00:00
beardthelionandGravirei 37947208c5 docs: state that GITLAWB_ENFORCE_OWNER_PUSH defaults to true (#537)
* docs: state that GITLAWB_ENFORCE_OWNER_PUSH defaults to true

PR #330 flipped the code default to true and updated .env.example, the
README env-var table, the AWS guide, the macOS app, and docs/RUN-A-NODE.md,
but two prose mentions in operator-facing docs still describe the
opposite. SECURITY.md line 57 ("Repository write authorization defaults"
section) and README.md line 69 (Known limitations) both say the flag
"defaults to `false`", which contradicts the live runtime and the rest
of the same files. Reviewer 2 closed #224 with this as the P2 finding
to fix in the config-and-policy split.

The correct text already exists in the README env-var table; the prose
restatements were missed. Bring the prose into line with the code.

- SECURITY.md: rewrite the section to lead with the default being true
  and describe `false` as the explicit rolling-upgrade override.
- README.md: rewrite the known-limitations bullet to state the default
  is true and the `false` opt-out is a rolling-upgrade only.
- Add `crates/gitlawb-node/tests/owner_push_default_docs.rs`: a
  regression guard that reads the four surfaces (clap default, .env.example,
  README known-limitations prose, README env-var table, SECURITY.md) and
  asserts each one says the default is `true`. Reverting either doc edit
  turns one named test red; flipping `default_value_t` back to `false`
  turns the env example test red. Each surface is checked independently
  so a stale doc lies alone.

No code change. No new migration. No dependency change. Compiles clean,
clippy clean under `-D warnings`.

Refs: closes the P2 finding from the review of #224.

* fix(docs): clarify default value consistency for GITLAWB_ENFORCE_OWNER_PUSH

* docs(readme): drop the default-open roadmap line and the em-dash punctuation

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 00:38:41 +00:00
beardthelion 5ed3381a6b fix(gl): truncate node-supplied display strings at a char boundary (#350) (#536)
* fix(gl): truncate node-supplied display strings at a char boundary (#350)

Every timestamp/id/SHA the CLI prints was cut with &s[..N] or
&s[..N.min(s.len())]. The first panics on a short value, the second
still panics when byte N is inside a multi-byte char. A shared
text::truncate cuts at a char boundary and covers all sites, including
two the issue did not name (node events, agent did) and the same shape
in bounty, changelog, ipfs pin listing, and the merge-commit print.

* fix(gl): bound truncate by bytes, not chars (#460)

Address PR review feedback: char_indices().nth(max) capped characters,
so a multibyte char near the cut could exceed the byte limit. Start at
max.min(len) and walk back to the nearest char boundary instead.

* fix(gl): bound the bounty leaderboard did suffix at a char boundary
2026-11-06 20:38:05 +00:00
beardthelion 9b915b09a7 fix(attest): bound did:key method-id length before base58 decode (#360) (#535)
gitlawb-core added the same guard in 73bf132c; the second copy of the
parser in gitlawb-attest never got it, so an untrusted signer string of
arbitrary length reaches a decoder whose cost is quadratic in input.
2026-11-06 18:14:06 +00:00
77ac1037cb fix(icaptcha-client): parse GITLAWB_ICAPTCHA_INSECURE as truthy only (#227) (#533)
* fix(icaptcha-client): parse GITLAWB_ICAPTCHA_INSECURE as truthy only

Presence checks treated =0/=false/empty as enabled. Only explicit 1/true
now enable loopback HTTP trust relaxation (#227).

* fix(icaptcha-client): address #246 review (trim test, fmt, lock)

Pin trim with a whitespace truthy case, recover from a poisoned
ICAPTCHA_ENV_LOCK, document trim + warn visibility, and cargo fmt.

* fix(icaptcha-client): do not log raw ICAPTCHA_INSECURE value

The non-truthy warn path previously recorded the full trimmed env
value. That is arbitrary deployment input; report only that the
setting is invalid (#246 review).

* fix(icaptcha-client): pure insecure flag helpers for #246 review

Warn on present non-Unicode GITLAWB_ICAPTCHA_INSECURE (fail closed).
Pass insecure mode into is_https_with / resolve_solver_url_with so
tests no longer mutate process-global env (removes InsecureEnv race).

* fix(icaptcha-client): cover real env read + fix warn spacing

Add a serialized test that drives icaptcha_insecure_enabled() against
the process env so presence-check / typo / always-true mutations fail.
Collapse the mid-sentence space run in the non-truthy warning (#246).

* fix(icaptcha-client): put the non-truthy warning on one literal line

---------

Co-authored-by: Ayush7614 <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-06 17:26:16 +00:00
db98b3957a fix(attest): make verify_all lenient under RequireAll (#359) (#532)
* fix(attest): make verify_all lenient under RequireAll (#359)

Registry::verify_all propagated every per-attestation error via ?, so a
single malformed or unknown attestation aborted the whole batch. Under
Policy::RequireAll that is a DoS vector: any third party can attach an
attestation, so an unrelated attacker/spam/v1 could block an otherwise
valid cert.

Under RequireAll, drop entries that fail to verify instead of aborting;
the required_types check remains the gate and only counts fully_verified
entries, so a required type present only as a malformed attestation still
fails with RequiredMissing. AcceptKnown and RejectUnknown stay strict.

Adds tests for both: a malformed extra is dropped without aborting, and a
malformed-only required type is still rejected.

Closes #359

* test(attest): lock the strict batch policies with an error-propagation test

AcceptKnown and RejectUnknown were only exercised per-attestation, so a
cleanup applying RequireAll's filter_map(Result::ok) shape to the strict
arms would fail open with every test green. Drive both policies through
verify_all with a wrong-cert-hash member and assert the batch surfaces
the underlying error. Also correct the RequireAll doc: unknown valid
types are retained as unverified, not dropped — only real Err paths are.

---------

Co-authored-by: andreolf <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-06 16:54:42 +00:00
beardthelionandAyush7614 8c9e4adc2d fix(node): canonicalize and 404-on-no-match in remove_label (#344) (#530)
* fix(node): canonicalize and 404-on-no-match in remove_label (#344)

add_label stored labels trimmed and lowercased, but remove_label passed the
path segment through as-is, so a label stored as 'bug' did not match
DELETE /repos/{owner}/{repo}/labels/Bug. Worse, remove_label's DB call
discarded the row count and the handler always returned 200 with
{removed: true}, including when the delete matched nothing — the
no-op-rendered-as-success shape the rest of the API rejects.

Move trim+lowercase and charset/length validation into a shared
canonicalize_label helper used by both handlers, and have
db::remove_label return the affected row count so the handler can return
404 when no label was deleted. add_label's behavior is unchanged
(idempotent semantics preserved); the only externally observable change is
that mixed-case and surrounding-whitespace path segments now match the
stored label, and a delete that matches no row returns 404 instead of
{removed: true}.

Tests:
  * remove_label_canonicalizes_and_404s_on_no_match exercises the
    mixed-case path, the trimmed path, the empty-after-trim rejection,
    the disallowed-charset rejection, and the no-op-returns-404 case.
  * remove_label_denies_non_owner re-pins the 403 owner-gated shape
    (matches AGENTS.md: owner-gated mutations return 403 after the repo
    lookup, distinct from the not-found shape).

* test(node): pin len>50 arm in remove_label canonicalize

The helper's length branch had no coverage — 51 chars must 400 and
50 chars must pass validation (404 when absent). This pins the arm
lifted into canonicalize_label and decouples the no-match 404 from the
double-delete above, as requested in review of #390.

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-06 15:08:34 +00:00
beardthelion 8eedb935c3 fix(node): gate bounty mutations on repo read before status checks (#529)
* fix(node): gate id-keyed bounty mutations on repo read before status checks

submit, approve, cancel, and dispute on /api/v1/bounties/{id}/* looked the
bounty up and ran status and participant checks without gating on the
bounty's repo read visibility. A signed stranger holding an id got a
distinguishable 400/403 on a private-repo bounty where an absent id gets
404, so the routes were an existence and lifecycle oracle.

Insert the same authorize_repo_read block get_bounty and claim_bounty
already carry, right after the lookup, so a denied caller sees the
not-found shape either way. Participant checks are unchanged; on a public
repo a stranger still reaches them.

Closes #341

* Address PR review feedback (#448)

- assert the exact not_found body shape for both held and absent bounty
  ids in the mutation-gate helper, so a denial carrying lifecycle or
  participant detail fails the test
2026-11-06 15:06:01 +00:00
beardthelion 47c6e6ce22 fix(node): fail pin_git_object when the add response carries no Hash (#528)
* fix(node): fail pin_git_object when the add response carries no Hash

A 200 from /api/v0/add without a parseable Hash fell back to the
caller-computed raw CID, recording an address the backend never
confirmed stored. For payloads above the backend's chunk threshold
(262144 bytes on Kubo) that raw CID is not even a stored block, so
every later /api/v0/cat on it fails permanently: get_encrypted_blob
errors on every fetch and mirrors retry the envelope on every sync.

A Hash-less 200 is now an add error carrying a truncated body excerpt.
The delaying_endpoint test fixture, which relied on the fallback by
returning an empty 200 body, now answers with a conformant NDJSON line.

* fix(node): reject a non-CID Hash in the add response

Address PR review feedback (#453): a string-valued Hash that does not
parse as a CID would be recorded into encrypted_blobs.cid and fail on
every later cat, the same failure class as a missing Hash. Validate the
returned Hash before accepting it and cover the malformed case with a
regression test.

* test(node): return real CIDs from the add-response test mocks

The pin_git_object CID validation rejects non-CID Hash strings, and a
dozen test_support fixtures answered 200 with placeholder values like
"bafyprovider" or "x". Replace them with a valid CID so the fixtures
model a healthy backend; no test asserts on the literal value.

* fix(node): bound the invalid Hash echoed in the add error
2026-11-06 06:32:13 +00:00
beardthelionandMystic 317a1550c2 fix(node): scope bounty aggregates to anonymously-listable repos (#477) (#527)
* fix(#477): scope bounty aggregates to anonymously-listable repos

bounty_stats and agent_bounty_stats ran unfiltered aggregates over all
bounties, leaking private-repo bounty activity to anonymous callers.

Mirror the stats() pattern from server.rs (#104):

1. Batch-load all deduped repos + visibility rules (2 SQL round-trips)
2. Filter to listable_at_root(rules, is_public, owner_did, None)
3. Pass visible (owner, name) pairs into SQL aggregates via EXISTS/unnest
   with owner key normalization on both sides (BOUNTY_OWNER_CASE_SQL).
4. Normalize claimant DID in agent_bounty_stats_visible and group by
   normalized claimant in bounty_leaderboard_visible (BOUNTY_CLAIMANT_CASE_SQL).
5. Remove orphaned unfiltered aggregate methods to pass dead_code lints.
6. In tests, seed completed bounty claimant state via the real claim/approve
   path, and verify root-rule-decided visibility overrides.

Aggregates are intentionally anonymous-scoped for all callers (matching the
#104 stats pattern).

Fail-closed: DB errors collapse the visible set to empty, so all counts
return 0 — an under-count never leaks existence.

Fixes #477

* style(node): apply rustfmt to the bounty visibility change

* test(node): pin bounty aggregate visibility for mirror rows

* docs(node): record why mirror rows are handled in the bounty visibility filter

* perf(node): keep the claimant index usable in per-agent bounty stats

* fix(node): return DB errors from bounty stats instead of zero counts

---------

Co-authored-by: Mystic <[email protected]>
2026-11-06 03:06:15 +00:00
JATMNandClaude Opus 5.5 30637193a6 fix(node): resolve issue ids from git refnames, not request input (#526)
* fix(node): resolve issue ids from git refnames, not request input

resolve_issue_id answered an exact match by echoing the request-path id
back to callers, so get_issue and close_issue spliced caller input into
`git cat-file blob refs/gitlawb/issues/<id>`. #521 restricted the input
to [A-Za-z0-9-], but the value handed to git was still the request
string (CodeQL rust/command-line-injection #83).

Resolve exact and prefix lookups through one for-each-ref listing and
return the refname git reports. An exact refname still wins over longer
ids that share it as a prefix; unique-prefix, ambiguous, not-found, and
revision-syntax behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(node): keep a dangling exact issue ref from winning over longer ids

for-each-ref lists a ref even when its blob is missing, so the exact-id
precedence added in the previous commit picked a dangling exact ref over a
readable longer sibling: resolve_issue_id returned the dangling id where
main reported an ambiguous prefix, get_issue then returned None, and
close_issue hit its expect().

Let the exact id win only when `git cat-file -e` finds its object, run on
git's own refname so no request input returns to the argv. Otherwise fall
through to the unchanged prefix rules, matching main for every layout.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-06 03:04:20 +00:00
JATMNandClaude Opus 5.5 a23c81d75a fix(deps): bump follow-redirects and override ws in scripts/ (#525)
Resolve Dependabot alerts for the ANS-104 golden-vector tooling in
scripts/:

- follow-redirects 1.16.0 -> 1.16.2 (lockfile only, within axios's
  ^1.16.0 range): GHSA-mrwp-f3q8-3g9p, GHSA-q3jh-wrc6-jgq8,
  GHSA-8r9p-f939-6h3c.
- ws 8.18.0 -> 8.22.1 via an npm override. @ethersproject/providers
  5.8.0 (latest v5) pins ws to exactly 8.18.0, so a lockfile bump
  alone cannot move it: GHSA-96hv-2xvq-fx4p, GHSA-58qx-3vcg-4xpx.

elliptic (GHSA-848j-6mx2-7j84, low) has no patched release and is
left as is.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-06 01:00:41 +00:00
beardthelion 5bb61cc97f ci(release): verify release provenance for backfill publishes (#523)
* ci(release): gate publish jobs on the release environment, anchor the tag check, pin npm

Closes #234. workflow_dispatch runs the selected ref's YAML and npm trusted
publishing matches on the workflow filename, not the ref, so a dispatched
non-main ref could reach the OIDC publish with no approval. Declare
environment: release on all three publish jobs (npm-publish, docker,
docker-manifest) so the environment's deployment-branch rule gates every
publish surface, not just the OIDC one.

Also close the two residuals the issue carries: the release-tag case glob
was right-unanchored (accepted v1.2.3.4.5, v1_x, v1-any-branch-name, bare
v1), now an extglob v+([0-9]).+([0-9]).+([0-9]); and the OIDC publish ran
npm@^11.5.1, a floating range, now pinned to [email protected].1.

The environment itself must exist with a deployment-branch rule limited to
main (and required reviewers if desired) for the gate to bite; npmjs.com
can also scope each package's trusted publisher to environment "release"
for a registry-side check that survives a workflow edit.

* ci(release): qualify tag checkouts and keep moving tags forward-only

Closes #271, closes #238.

actions/checkout resolves an unqualified ref as a branch before a tag, so
a branch named v0.7.1 shadows the tag and the release builds unreviewed
content. Qualify all four release-tag checkouts as refs/tags/..., which
takes checkout's unconditional tag path.

docker-manifest applied :latest and :X.Y unconditionally, so a backfill
of an older tag repointed them at older code. The manifest now always
publishes the immutable :X.Y.Z and applies the moving tags only when the
resolved version is not older than the registry's latest release. The
same shape existed on the npm side: publish moves the latest dist-tag to
whatever it publishes, so a backfilled older version would have stolen
it; older backfills now publish under the backfill dist-tag instead.

The manifest also asserted nothing about the digest set it stitches, so
a wrong-arity or misnamed artifact would ship under :latest. Assert two
digests of 64 lowercase hex chars before imagetools create.

Extend the resolver test to pin both publish steps byte-for-byte and to
reject any unqualified tag-checkout ref, so these guards cannot regress
silently.

* ci(release): gate every job on the release environment and verify tag provenance

Two residuals from review of the dispatch-publish surface:

- environment: release only covered the three registry-publish jobs. Declaring
  it on all eight jobs means a dispatch of unmodified YAML on a non-main ref
  is denied by the environment's deployment rules before any step runs.
- The resolver shape-checked vX.Y.Z but did not prove the tag was produced by
  the release flow. A write-access user could plant a v99.99.99 tag on
  unreviewed content and dispatch a backfill against it. In CI (GH_TOKEN
  present) the resolver now requires a GitHub release for the tag and a
  main...tag compare status of identical or behind, so the tag's commit must
  already be reachable from main.

* ci(release): qualify the provenance compare and fail closed on npm lookup errors

Second adversarial pass found two residuals:

- The provenance check compared main...<tag> with an unqualified refname,
  the same branch-shadows-tag ambiguity the refs/tags/ checkout prefix was
  added to avoid. Resolve the tag through commits/refs/tags/<tag> to a
  commit SHA and compare that instead.
- npm view ... || true mapped every lookup failure to "no latest", so a
  registry outage during an older backfill would publish under --tag
  latest and move the dist-tag backward. E404 still means first publish;
  any other failure now aborts before a dist-tag is chosen.

* ci(release): check release author, decide npm dist-tag per package, close the test gap

Third adversarial pass:

- Provenance now also requires the release to be authored by the release
  automation account, so a hand-created release on a valid tag does not
  qualify for backfill.
- The npm dist-tag decision is per package. A partial publish can leave
  platform packages ahead of the wrapper, and a single lookup on
  @gitlawb/gl would then downgrade an ahead package's latest tag.
- The environment gate test now emits one line per job and marks jobs with
  no declaration MISSING, so a job added without environment: release can
  no longer pass silently.

* ci(release): anchor docker moving tags to the registry floor, not the release pointer

GitHub's latest-release pointer is operator-mutable and is not a bound on
what :latest serves. List the package's versions on ghcr and refuse to
move :X.Y or :latest below the newest X.Y.Z tag there. A 404 means the
package does not exist yet, so the first publish applies all tags; any
other lookup failure aborts. The version filter is grouped with || true so
an empty list does not trip pipefail.

* ci(release): require backfill release assets to be automation-uploaded

The npm backfill republishes GitHub release assets, which are mutable for
anyone with contents: write. Require every asset on the release to have
been uploaded by github-actions[bot], matching the release-author check.
Not a hard bound, but it removes the direct edit path and leaves a review
signal when a hand-uploaded asset blocks a backfill.

* ci(release): pin the verified commit and the exact asset bytes end to end

The provenance checks ran at resolve time but the consumers re-resolved
mutable names afterward, leaving a move-the-tag or replace-the-asset race:

- Checkouts now use the commit SHA the resolver emitted (tag_commit),
  falling back to the qualified tag ref.
- The resolver captures the release's name/id asset pairs once, checks all
  uploaders against them, and the lay-in step downloads by asset id. An
  asset deleted and reuploaded between resolve and download gets a new id
  and fails closed.
- The npm-publish comment now names the third trusted-publisher component
  (environment), which is what binds the OIDC token to the gate.

* ci(release): run the resolver on the producer path and hash homebrew inputs locally

- release-binaries and web-sync now resolve the release tag and check out
  the verified commit SHA, closing the tag-move race on the job that
  produces the release assets, not just the jobs that consume them.
- homebrew-bump resolves the tag, downloads the tarballs by captured asset
  id, and computes sha256 locally instead of trusting the mutable
  *.tar.gz.sha256 sidecar assets.
- The step pin now covers the lay-in, formula, and release-tag checkout
  blocks so edits to the trust boundary force a deliberate fixture update.

* ci(release): bind the tag to the commit its release was created against

Reachable-from-main alone accepts a tag moved to any other main commit.
The release object records its creation-time target_commitish, so require
tag_commit == release.targetCommitish: a moved tag now fails closed, and
the SHA emitted for checkouts is provably the released commit.

* ci(release): force bash for the release-binaries resolver step

The matrix includes windows-latest where run: defaults to pwsh, and the
resolver body is bash.

* ci(release): base64 release-asset names in the resolver map

Release asset names are attacker-controlled text. A crafted name
carrying whitespace could smuggle a fake uploader into the third
positional field and pass the automation-only check. Emitting names
base64-encoded keeps each record in three real fields and makes the
name-to-id lookup immune to injection.

* Address PR review feedback (#447)

- floor the docker minor-line tag at its own X.Y max so an
  in-line-newest backfill advances :X.Y without moving :latest

* Address PR review feedback (#447)

- serialize all release runs under one concurrency group; the
  ref-keyed group let dispatches from different refs interleave

* ci(release): freeze docker moving tags on dispatch, retire the registry floor

* ci(release): make the resolver test's gh stub discriminate on the full call

* ci(release): assert the pinned npm version took effect

* ci(release): surface missing-asset errors from the formula sha() subshell

* ci(release): consume resolver outputs instead of release-please outputs post-resolve

* ci(release): document the concurrency group's pending-slot replacement

* ci(release): share the image output and bound the step scanners to jobs

The docker-manifest rel step now emits image= the same way the docker
job's does, and both manifest steps consume steps.rel.outputs.image
instead of recomputing the lowercase ghcr path.

The three whole-workflow awk scanners now ignore everything before
jobs: and flush at job boundaries, so an `on.push.branches` list item
can no longer open a pseudo-step that swallows top-level YAML, and a
step record can no longer absorb the next job's header lines.

* ci(release): close the review-found evasions in the release checks

Review of the follow-up diff found the structural assertions could be
satisfied or evaded without the behavior they guard:

- the gh stub now pipes a fixture response through real jq with the
  caller's own -q on every arm, so a resolver that hardcodes its
  expected fields is detected
- the moving-tag scan keys on the outcome (a push mechanism plus a
  mutable-tag reference in decommented step text) and requires the
  event+ref guard on the carrier's if: line, instead of binding one
  argv spelling and accepting comment-borne guard text
- a new assertion pins the on.push.branches / workflow_dispatch
  triggers, which no step-level check could see
- the npm check detects install/i and -g/--global variants, requires a
  non-empty install set, and reads the pin comparison from code, not
  comments
- the stale-reads scan covers first step lines, job-level env/if lines,
  and jobs without a resolver step
- the resolver test asserts the GHAE heredoc terminator, clears ambient
  GH_TOKEN/GITHUB_REPOSITORY on non-stub calls, and prints the captured
  resolver stderr when a pass-expected case fails
- sha() exits on a failed download, since errexit is off inside its
  command substitution and a truncated file would otherwise be hashed

* ci(release): floor the moving tags against the releases list

The event gate alone cannot bound a stale run: a re-run of an older
push run replays its stored needs outputs under the original event and
ref, and would move :latest and :X.Y backward to that run's version.
The gated step now derives the newest published release from the live
releases list (contents:read, drafts and prereleases excluded) and
refuses when this run's version is not the maximum. The check reduces
to VERSION == max, because the run's own release already exists by the
time the step runs.

The all-steps assertion now requires the carrier to carry the floor as
well as the event+ref guard, and the red-check spec gains a
gated-but-floorless carrier mutation to prove the clause binds.

* ci(release): pin the rebranded Homebrew formula text in the resolver-steps test

* ci(release): keep npm stderr out of the dist-tag lookup

The Publish step captured `npm view ... dist-tags.latest` with 2>&1, so a
warning on stderr from a successful lookup became part of registry_latest and
could make a new release compare as older than the registry latest, publishing
it under the backfill dist-tag instead of latest.

Send stderr to a temp file and read it only when the lookup fails. E404 still
means the package was never published, and any other failure still fails
closed. The pinned copy of the block in the resolver test is updated to match,
and a stubbed-npm test covers a warning on success, a clean lookup, a genuine
backfill, E404 and a non-E404 failure.

* ci(release): bound the moving tags against every release page

The Move floating tags step floored a stale re-run against the semver maximum
of the first page of the releases list only. That list is ordered by creation,
not semver, so once history passes one page the true newest version could sit
on a later page and a stale re-run would move :latest and :X.Y backward.

Read every page with gh api --paginate and take the semver maximum in bash.
The gh call stays its own command substitution so a failure on any page still
fails closed, and an empty list still yields no floor. The pinned copy in the
resolver test is updated to match, and a stub-gh test serving two pages covers
a stale run whose newer release is only on page 2, the run that is the newest,
a gh failure, and an empty list.

* ci(release): fail closed when a release asset has no uploader login

The resolver ran its automation-only asset check only when the collected
uploader list was non-empty. A null uploader login made jq emit an empty third
field, the list collapsed to nothing, and provenance passed although no asset
was shown to be automation-uploaded.

Substitute a sentinel for a missing login so every asset row carries a value
and a null, missing, or login-less uploader fails the existing comparison. The
base64 name and id map is unchanged, and a release with no assets behaves as
before. The resolver test covers an all-null uploader, a null login, a missing
uploader key, a bot asset mixed with a null one, and the all-bot release.
2026-11-05 21:51:35 +00:00
beardthelionandGravirei 2c341f71ac fix(node): ANS-104 codec and recovery policy types (#26 split 2/4) (#524)
* fix(node): ANS-104 transport, three-outcome probe, verify endpoint (#26 split 2/4)

Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split
into four narrow PRs. This is Split PR 2 (Arweave transport and
verification).

The P1 finding the reviewer assigned to this split: the legacy
`anchor_item_present` mapped `BAD_REQUEST` and `GONE` to `Ok(false)`,
and the recovery code interpreted false as permission to reclaim the
row and pay for another upload. Neither 400 nor 410 proves a
previously paid item was never accepted: 410 can describe an
artifact that existed but is no longer served, and 400 can be
produced by gateway, proxy, or routing failure. The concrete
failure sequence the reviewer named: bundler accepts the item,
the response or terminal DB write is lost, recovery probes the
persisted item_id, the gateway returns 400 or 410, and the node
pays for a second immutable item for the same transition.

Fix: model the probe result as three outcomes, and authorize a
re-upload only on a trustworthy, protocol-defined absence.

NEW MODULE crates/gitlawb-node/src/ans104.rs
  - DataItem: the on-wire ANS-104 shape (signature, owner, target,
    anchor, tags, data — all base64url-encoded except target/anchor).
  - deep_hash: the canonical signing input per the spec. The
    signature is over a SHA-256 mix of the dataitem/list/map
    discriminators, the signature type, owner, target, anchor, the
    recursively-hashed tag list, and the data payload. A
    hand-rolled sha256 of the JSON body would produce a hash no
    Arweave gateway would recognize; the deep-hash is the only
    correct form.
  - sign_data_item: Ed25519 over deep_hash.
  - verify_data_item: parses the owner, checks the signature against
    an expected pubkey, and reports a specific failure reason on
    each branch.
  - 7 unit tests covering round-trip, flipped signature, wrong
    expected key, mutated data, wire-shape round-trip, deep-hash
    stability, and the empty-tags edge case.

NEW MODULE crates/gitlawb-node/src/arweave_v2.rs
  - ProbeOutcome: Present (2xx, body parses as ANS-104, sig
    verifies), DefinitivelyAbsent (404 with a known JSON body
    shape), Indeterminate (everything else). The reviewer's
    three-outcome model.
  - ProbeOutcome::permits_reupload: the recovery policy. ONLY
    DefinitivelyAbsent authorizes a paid re-upload. Indeterminate
    keeps the outbox non-terminal; Present skips re-payment.
  - probe_anchor_item: the gateway probe. 2xx bound to a different
    item id is Indeterminate, not Present. 2xx with a body that
    does not parse as ANS-104 is Indeterminate. Oversized bodies
    are Indeterminate (defense against body-stuffing).
  - read_capped_body: enforces PROBE_MAX_BODY_BYTES (1 MiB) on the
    full body, with a Content-Length fast-path that rejects
    before reading.
  - verify_anchor: the full path the public verify endpoint takes.
    Fetches the data item, parses it, verifies the Ed25519
    signature against the persisted node_did, decodes the data
    payload as JSON, and reports the result.
  - 15 unit tests pinning each classification boundary:
    400/410/5xx/transport/2xx-non-json/2xx-bad-sig/2xx-different-
    owner are all Indeterminate; 404 with empty or known-JSON
    body is DefinitivelyAbsent; 2xx with a valid signed item is
    Present; the re-upload policy is exhaustive.

NEW DB METHOD crates/gitlawb-node/src/db/mod.rs
  - get_arweave_anchor_by_item_id: looks up an existing
    arweave_anchors row by its id (which is the Irys tx id, the
    same value the Arweave gateway uses to serve the data item).
    Used by the public verify endpoint to fetch the persisted
    node_did so the envelope signature can be verified against it.

NEW CONFIG crates/gitlawb-node/src/config.rs
  - GITLAWB_ARWEAVE_GATEWAY_URL: the Arweave gateway the probe
    reads from. Defaults to https://arweave.net; operators can
    point it at a private mirror.

NEW HTTP ENDPOINT crates/gitlawb-node/src/api/arweave.rs
  - GET /api/v1/arweave/anchors/verify/{item_id}: the public
    verify endpoint. Returns {item_id, status, verified,
    owner_did, data_payload, error} where status is one of
    "verified" / "definitively_absent" / "indeterminate".
  - 3 endpoint-level tests covering the verified, 404, and 400
    shapes via the live router, asserting the public surface
    surfaces the three outcomes.

WIRE-UP crates/gitlawb-node/src/server.rs
  - The verify route is registered alongside the existing
    /api/v1/arweave/anchors list route.

NOT IN THIS SLICE (the bundler upload, the next commit):
  - The handler that builds a DataItem from a ref-cert and POSTs
    it to the bundler is not in this PR. The ANS-104 module is
    ready to be called by it; the cert/push-event flow in PR 1
    persists the row that the bundler upload would consume.
  - PR 1's recovery drain produces an `anchor_jobs` row keyed on
    the (repo, ref, old, new) tuple; the next slice reads that row,
    builds the ANS-104 data item, POSTs to the bundler, and writes
    the resulting tx id to arweave_anchors.

Compiles clean, 1113 tests pass with 0 regressions, clippy clean
under -D warnings, fmt clean.

Cross-PR overlap (declared in the PR description):

  - #134 (anchors auth): composes. The verify endpoint reads from
    arweave_anchors; #134's auth layer applies unchanged. The
    probe model is server-side and unauthenticated by design.
  - #285 (advisory-lock session affinity): independent. The probe
    is read-only.
  - #306 (Content-Digest on signed requests): independent. The
    probe does not sign requests.
  - #314 (small-order Ed25519): composes. The verify path calls
    Did::to_verifying_key which already enforces the small-order
    check from #314.
  - #324 (libp2p keypair persistence): independent.
  - #325 (gossip ref-update auth): independent. The verify path
    is for the public Arweave gateway, not gossip.
  - #382 (replication withheld-subtree trees): independent.

* fix(node): address reviewer findings on #26 split 2/4 (transport)

- P1 (DB lookup): get_arweave_anchor_by_item_id now filters on
  irys_tx_id (the externally-routable item id) rather than the
  internal UUID primary key. The production writer stores a fresh
  UUID in `id` and the Irys response / ANS-104 derived id in
  `irys_tx_id`; the old filter would always 404. New
  integration tests exercise the production writer path with
  distinct values for the two columns.
- P1 (verify-gate): verify_anchor now gates on repo read via
  authorize_repo_read. The route is split in server.rs: the
  public list_anchors stays ungated (#134 tracks that surface),
  the verify route gets optional_signature per the team memory
  `axum-layer-vs-merge-pitfall.md`. Denials on row-lookup, gate,
  and missing repo all collapse to the same opaque 404 so the
  public endpoint does not leak anchor-row existence.
- P1 (ANS-104 deep-hash): rewritten to the verified SHA-384
  recursive list/blob algorithm (matches `arbundles` JS
  reference). deep_hash returns [u8; 48]; signature is over the
  raw 48-byte digest. Three reference vectors from
  `Irys-xyz/arbundles/src/__tests__/deepHash.spec.ts` are
  bit-exact-asserted as the interop canary per the team memory
  `self-roundtrip-tests-do-not-prove-interop.md`.
- P1 (artifact-identity): v2 verify now derives the protocol id
  via DataItem::id() (base64url(SHA256(signature))) and requires
  equality with the requested URL item_id. A node key signs
  many data items, so a valid signature only proves who signed
  the response — not that the served item is the one the caller
  asked to verify. The team memory
  `verify-against-artifact-id-not-signer.md` is the policy.
- P1 (verifier can't read this head's writes): verify_anchor
  now accepts BOTH the v2 ANS-104 format and the v1 raw-JSON
  shape the live path on this branch actually writes. The v1
  path does a field-equality check on repo, ref_name, old_sha,
  new_sha, node_did against the persisted row (no signature in
  v1; Irys storage plus the JSON parse are the integrity
  guarantee).
- P2 (ProbeOutcome in result): AnchorVerifyResult now carries
  outcome: ProbeOutcome. The HTTP handler maps the structured
  field to the status string instead of parsing the human-
  readable error message — any future wording change in
  arweave_v2.rs can no longer silently reclassify a
  DefinitivelyAbsent as Indeterminate.
- P2 (empty 404 body): classify_404 returns Indeterminate for
  bodyless 404s. The team memory
  `distinguish-unknown-from-empty.md` is the policy: a
  bodyless 404 from a proxy is not a proof of absence, and the
  recovery policy (permits_reupload -> true on
  DefinitivelyAbsent) authorizes a paid, irreversible re-upload.
- P2 (streaming cap): read_capped_body no longer uses
  resp.bytes() (which buffers the full response before checking
  size). It now reads chunk-by-chunk and aborts the moment the
  cumulative byte count crosses the cap, so a chunked response
  without Content-Length cannot force unbounded allocation.

* fix(node): address three reviewer findings on #26 split 2/4 (round 2)

* fix(node): address five reviewer findings on #26 split 2/4 (round 3)

* fix(node): address reviewers round-4 on #26 split 2/4 (interop)

* fix(node): address coderabbit on #26 split 2/4 (raw tags, vint overflow, golden key)

* fix(node): enforce artifact-id in probe_item on #26 split 2/4

* fix(node): require expected owner for probe Present on #26 split 2/4

* fix(node): narrow #26 split 2/4 to codec and policy types

* fix(node): silence dead-code for deferred slice API on #26 split 2/4

* fix(node): doc and bounds-check cleanup in ans104 on #26 split 2/4

* docs(node): point tests doc at Ed25519 golden script on #26 split 2/4

* fix(node): strict owner lengths, sign-before-hash on #26 split 2/4

* fix(node): keep ans104 tags and field widths consistent on hash and encode

DataItem keeps both the decoded `tags` and the original Avro `raw_tags`, both
public. After a parse, changing `tags` left deep_hash and verify_data_item
using the stale raw bytes, and to_binary wrote a header tag count that no
longer matched the payload, so the frame failed to parse back.

Take the tag block for hashing and encoding from one helper that decodes
`raw_tags` and requires it to equal `tags`, and fail closed otherwise. The
header count comes from the same validated pairs. Parsed multi-block and
size-prefixed items still re-encode byte-exact and keep their signatures.

deep_hash now also rejects a non-empty target or anchor that is not 32 bytes,
as to_binary already does, so sign_data_item fails instead of signing an item
that cannot be encoded. The legacy non-base64url anchor fallback is kept.

The module and signature-type docs now say binary parsing accepts sigtypes 1
to 4 while signing and verifying are Ed25519 only on this slice.

* build(scripts): declare the golden script's direct npm dependencies

ans104_golden_ed25519.mjs imports bs58, @noble/ed25519 and base64url, and
ans104_golden.mjs imports base64url, but package.json listed only arbundles, so
they resolved only through hoisting. Declare them at the versions already
locked, update the lockfile root entry (no resolved version or hash changes),
and note the regeneration command in the golden output file.

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-05 21:12:46 +00:00
beardthelion 911be4d9fe fix(node): validate ref-update SHAs are hex, not just length 40 (#433)
* fix(node): validate ref-update SHAs are hex, not just length 40

parse_ref_updates accepted any 40-byte string as an object id, so a
peer-supplied pkt-line with non-hex content was forwarded into
RefUpdateEvents on gossip, branch_cid rows, and sync-notify JSON as a
canonical-looking identifier. Require [0-9a-fA-F] on both fields.

Closes #398

* chore: refresh PR head
2026-11-05 16:57:42 +00:00
beardthelion df87b150fd fix(gl): pass the mirror temp path to git as an OS string (#434)
* fix(gl): pass the mirror temp path to git as an OS string

The clone destination went through to_str().unwrap(), so a TMPDIR with
non-UTF-8 bytes panicked before git started. Pass the Path directly and
add a Unix regression test cloning into a non-UTF-8 directory name.

Closes #417

* test(gl): cover a non-UTF-8 TMPDIR end to end via a child process

* chore: refresh PR head
2026-11-05 16:52:25 +00:00
beardthelion 0366f18458 fix(node): reconcile public-read messaging with per-repo enforcement (#439)
* fix(node): reconcile public-read messaging with per-repo enforcement

GITLAWB_PUBLIC_READ is read nowhere except a startup warning, while
read enforcement actually happens per repository through is_public and
path-scoped visibility rules. The flag's help text, the startup
warning, .env.example, and the readiness audit all still claimed
private-read enforcement was unwired.

The help and warning now state the flag is reserved and inert, and
point at the real per-repo control. A config test asserts the rendered
help neither claims enforcement is missing nor omits that the flag
does nothing.

Closes #338.

* docs: drop the stale private-read priority entry

* chore: refresh PR head
2026-11-05 16:46:47 +00:00
beardthelion c2e175d222 fix(git-remote): stop waiting on EOF for a delete-only push (#442)
* fix(git-remote): stop waiting on EOF for a delete-only push

* fix(git-remote): consume negotiated push-options and push-cert sections

* fix(git-remote): frame signed pushes correctly, cert before options and no command list

* chore: refresh PR head
2026-11-05 16:39:29 +00:00
beardthelionandKevin Codex cd27756134 ci(triage): match test as a name segment, not just a whole component (#277)
* ci(triage): match test as a name segment, not just a whole component

#202 landed the namespaced-attribute fix, so main already detects
#[tokio::test] and #[sqlx::test]. What it still misses is a third-party
harness whose final path component carries `test` as an underscore
segment. #[test_case(1)] and #[wasm_bindgen_test] both fall through and
wrongly earn the PR a needs-tests label.

Widen the final component to (?:[A-Za-z0-9]+_)*test(?:_[A-Za-z0-9]+)*,
keeping the leading ::, the r# form, and the whitespace tolerance #202
added. Differential over 972 real file-patches from 400 commits: no
verdict changes either way, so this is a forward-looking net for
harnesses the tree does not use yet, not a fix for a current mislabel.

Segment rather than substring, on purpose. addsInlineTest suppresses the
label when it matches, so a false positive is the quiet failure:
#[contest] or #[latest] would clear needs-tests on a PR that added no
tests and nobody would notice. The cost is the harness names with no
underscore, such as #[rstest], which stay unmatched and produce the loud,
author-correctable failure instead.

The segment classes are [A-Za-z0-9]+ rather than \w+ so that `_` is only
ever the literal delimiter. \w+ contains `_`, which makes the repetition
ambiguous and backtracks exponentially. That matters here specifically
because the workflow triggers on pull_request_target, so f.patch is
fork-controlled text on a privileged runner: with \w+, `#[test` followed
by a long `_a` run took 71ms at 22 characters and roughly quadrupled
every two more, which stalls the triage job for any contributor who can
open a PR. The character-class form measures linear across every
adversarial input tried.

* fix(ci): allow Rust token separators before test attribute delimiters

The needs-tests inline detector required ] or ( immediately after horizontal
whitespace, so valid spellings like #[test /* rationale */] or a path split
across added patch lines still triggered needs-tests. Match block comments,
line breaks, and split-line closers instead.

* ci(triage): scan attribute closers token-aware, bound to adjoined added lines

The two-regex fallback matched a path-only attribute line and a ]/( line
anywhere else in the patch, so unrelated records could complete each other
and silently clear needs-tests; the mandatory separator regex also dropped
legal //-to-newline and nested block comments, relabeling real tests.
Replace both with one forward token scan that starts on the attribute's
added line, continues only through immediately adjoined added lines under
a hard bound, and answers 'no inline test' on anything uncertain.

Fence the detector for extraction and add scripts/test-pr-triage-detect.mjs,
run by a new pr-checks job, since pull_request_target executes the target
branch's workflow and can never exercise the proposed body.

* fix(ci): skip test attributes inside raw strings, block comments, and strings

patchAddsInlineTest had no Rust lexical state, so a #[test_case] inside a
raw string (r#"..."#), block comment, or regular string literal, followed
by ] or ( on an adjoining added line, falsely cleared needs-tests. That is
the silent failure direction the detector is supposed to fail loud against,
and it is newly reachable because the underscore-segment regex introduced
by this branch matches forms the old test\b did not.

A pre-pass now walks every patch line (context and added, not removals)
tracking raw strings (r#*", br#*", cr#*"), block comments (/* */ with
nesting), regular strings (which can span newlines in Rust), and line
comments. Added lines that start inside any of these are skipped as
candidates. The pre-pass resets at hunk headers, is O(n) in patch length,
and only sees what the patch shows: a non-code region opened before the
first context line of a hunk is invisible, and the candidate is still
scanned (conservative, fails toward the loud label).

Also corrects the failure-direction comment in test-pr-triage-detect.mjs:
false positives suppress needs-tests (they set touchedTests, which clears
the label), while false negatives apply it (the loud, corrigible
direction). The previous text had them backwards.

Adds 7 regression cases: raw-string, block-comment, no-hash raw-string,
and multiline regular-string false positives (reject), plus raw-string,
block-comment, and regular-string closing before a real test attribute on
the next added line (accept). Revert-check confirmed the pre-pass is
load-bearing for both the nonCodeStart skip and the inString carry:
disabling either makes the corresponding reject cases go RED, restoring
makes all 31 pass.

* fix(ci): recognize char literals so their contents cannot open string state

The lexical pre-pass in patchAddsInlineTest missed real tests when an
earlier added line contained a double-quote char literal like '"' or
b'"'. The scanner saw the " inside the char token and set inString,
which persisted to the next line and suppressed a following #[test].
Add bounded char-literal recognition: on seeing ', look ahead 2-4 chars
(or to } for \u escapes) for a closing '. If found, skip the whole
literal. If not, it is a lifetime or label, and the apostrophe is
skipped as a regular char. Not a skip-to-next-' rule; lifetimes and
labels fall through unchanged.

* fix(ci): bound the \u{ escape scan to prevent quadratic stall

The char-literal pre-pass scanned to the next } with no bound when it
saw \u{, so a fork-controlled patch repeating '\u{ thousands of times
made the triage job quadratic (21s for 30000 reps). Bound the scan to
20 chars past the { (a valid \u{...} holds at most 6 hex digits). Add a
runtime probe for the malformed-escape shape.

* chore: refresh PR head

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-05 16:33:22 +00:00
beardthelion 5479fd59b8 fix(icaptcha): return unsolved instead of blocking or overflowing (#444)
* fix(icaptcha): return unsolved instead of blocking or overflowing

interactive_prompt waited on stdin().read_line even when stdin was an
open, silent non-TTY (an orchestrator holding the pipe open), which
blocked obtain_proof forever. Check IsTerminal first and return None so
the caller surfaces the "couldn't auto-solve" error.

The deterministic solvers evaluated prompts with unchecked i64
arithmetic; service-controlled literals could push add/sub/mul/div/abs
past the range, panicking in debug builds or wrapping to a wrong answer
in release. Use checked arithmetic throughout and return None on
overflow.

Fixes #345

* chore: refresh PR head
2026-11-05 16:01:31 +00:00
beardthelion 131bb95c68 fix(node): scope the refs listing by the canonical repo slug (#388)
* fix(node): scope the refs listing by the canonical repo slug

`list_refs` built its `branch_cids` filter from the raw URL path segments
while the only production writer stores the canonical
`normalize_owner_key(owner_did)/name` slug. `get_repo` resolves both
`did:key:zX` and bare `zX` to the same record, so a request in the full
DID form passed the read gate and then matched zero rows, returning 200
with an empty ref list. A denial shaped like a success is worse than an
error: a caller cannot tell an empty repository from one it queried under
a form the table does not carry.

Bind the record `authorize_repo_read` already returns and build the filter
from it, so the queried resource is the one that was authorized. This is
the existing pattern in `api/events.rs` and in this file's own push path.

Two regressions, each proven to fail without the line it guards. The first
drives both owner-DID forms through `build_router` and asserts they return
the same refs, with a second repo whose refs must not appear; widening the
query to match any owner turns that half red. The second covers the deny
direction, which the endpoint had never had: a private repo answers an
anonymous caller with the same 404 a missing repo gets and puts no CID in
the body. Without it, removing the read gate outright left the suite green.

* chore: refresh PR head
2026-11-05 15:40:50 +00:00
beardthelion 0303222eae fix(node): gate replica register/unregister on repo read visibility (#445)
* fix(node): gate replica register/unregister on repo read visibility

list_replicas already applied authorize_repo_read, but the two
mutations looked the repo up directly: a non-reader of a private repo
could register or remove their own replica and receive the replica
count in the response. Route both handlers through authorize_repo_read
so a denied caller gets the same 404 as a missing repo and nothing is
written. Self-registration and self-removal are unchanged for
authorized readers.

Fixes #435

* test(node): assert denial bodies leak no replica or repo data

CodeRabbit nitpick: the new replica-mutation denial tests only checked
the 404 status. The path instructions for this file require non-leaking
body assertions on denied responses, matching the sibling
list_replicas_is_read_visibility_gated test. Assert the denied PUT and
DELETE bodies contain neither the submitted replica URL nor the repo id,
in both the handler-level test and the real-signature e2e.

* chore: refresh PR head
2026-11-05 15:20:09 +00:00
beardthelion 2450bbd9e9 fix(core): cap the blinded recipient scan in open_blob (#419) (#440)
* fix(core): cap the blinded recipient scan in open_blob

* fix(core): reject over-cap recipient lists at seal too
2026-11-05 05:24:52 +00:00
beardthelion c97695a259 Add a completeness fence over the GraphQL mutation surface (#219)
* test(node): add a completeness fence over the GraphQL mutation surface

Gate the GraphQL mutation surface the way REST is gated, as a SEPARATE guard with
its own bucket table (a guard over a heterogeneous set must classify per member,
INV-1). every_graphql_mutation_has_its_gate asserts the mutation set is complete
(every mutation classified) and that each carries its gate marker.

Completeness is derived from async-graphql's own introspection: the field names
come from the engine as structured data, so no source-text or SDL-format shape
(a brace or async fn in a comment or string, a whitespace/generic/skip variant, a
brace in a rustdoc description) can hide a live mutation. A new unregistered
mutation, including a repo-write, fails the build.

The marker check (does each resolver body call its gate) is a best-effort static
backstop, bounded to each resolver's own comment-masked, string-aware body. It is
explicitly not the completeness guarantee and not load-bearingness: a marker inside
a string literal is a documented vacuity residual, and the adversarial sqlx tests in
graphql/mutation.rs are the real proof the gates reject.

Verified by execution: an unregistered mutation reddens (unclassified), a removed
did_matches gate reddens (missing marker), the clean tree is green; no DB, no
feature flag.

* docs(node): reconcile GraphQL mutation-auth claims with the shipped state

README's known limitations, the OSS-readiness live-network blockers, and the
maintainer roadmap still listed GraphQL mutation auth as an open item, which
contradicts #87: GraphQL mutations already require a verified signer bound to the
acting DID. Reconcile each to the shipped reality while keeping the genuinely-open
nuance: the GraphQL mutation surface is the agent-task queue, repo writes stay
REST-only, and no owner-gated repo-write mutation is exposed over GraphQL. The
every_graphql_mutation_has_its_gate fence forces any new mutation to be classified
in its table; owner-gate and REST parity stay the INV-1 review check.

* fix(node): source the GraphQL mutation fence from SDL, not introspection

Resolves jatmn's #219 P1. The every_graphql_mutation_has_its_gate completeness
fence enumerated mutation fields via async-graphql introspection, which filters
out `#[graphql(visible = false)]` fields. Those fields stay executable, so a
future hidden mutation would need no registry row and no marker while both
set-equality checks stayed green -- defeating the fence's guarantee that no live
mutation goes unclassified. (Confirmed by execution: a synthetic visible=false
mutation is absent from introspection, executable, and present in schema.sdl().)

Source completeness from the engine's own SDL instead. A new pure helper
mutation_field_names_from_sdl parses schema.sdl() with async-graphql's parser
AST (async_graphql::parser::parse_schema) and reads the mutation type's fields;
SDL includes visible=false fields and the AST parse is immune to a brace in a
field description. Introspection is kept as a cross-check: any field present in
SDL but hidden from introspection now fails named as a forbidden hidden
mutation. The stale doc comments (which claimed introspection could not hide a
field) are corrected.

A self-checking version-safety test builds a synthetic schema with a
visible=false mutation and asserts the SDL-sourced set includes it while
introspection omits it -- so if a future async-graphql ever filters SDL by
visibility, that test goes RED rather than the gap reopening silently.

Load-bearing verified by execution: stubbing the SDL enumeration to empty turns
four tests RED, including the fence itself; restoring returns green. Only
api/mod.rs changed; the mutation resolvers are untouched. `cargo test -p
gitlawb-node --bins` green (495); clippy and fmt clean.

* fix(node): make the SDL mutation-fence wiring load-bearing; admit deprecated

An adversarial review of the SDL switch found three gaps:

- The fence's completeness source (SDL) and its hidden-mutation cross-check were
  not load-bearing: because the real MutationRoot has no visible=false field,
  reverting `fields` back to introspection and dropping the cross-check left the
  whole suite green (INV-21). Extract the sourcing + cross-check into a generic
  `assert_mutation_fence<Q,M,S>` that the real fence AND a synthetic hidden schema
  both drive. A new #[should_panic] test (fence_rejects_a_hidden_mutation) feeds a
  visible=false mutation that IS registered, so only the cross-check can fire;
  reverting the SDL source or deleting the cross-check now changes/removes that
  panic and turns the test RED (verified).

- The introspection cross-check used bare `fields { name }` (GraphQL default
  includeDeprecated:false), so a `#[graphql(deprecation = "...")]` mutation --
  executable and present in SDL -- would false-red as a hidden visible=false
  mutation. Use `fields(includeDeprecated: true)`; a new test
  (fence_admits_a_deprecated_mutation) pins it, going RED without the flag.

- schema_lists_exactly_the_registered_mutations still sourced from introspection
  and its comment called introspection "the source of truth", contradicting the
  fix. Retargeted at the SDL set with a corrected comment.

Also folds the two introspection navigations into one generic helper
(introspection_mutation_fields) and points the version-safety test at it, so
there is a single introspection query. cargo test -p gitlawb-node --bins green
(497); clippy and fmt clean. Only api/mod.rs changed.

* fix(node): rebase #219 and require signer gate in the GraphQL fence

Rebase onto current main and skip the obsolete 0.5.1 lockfile commit.
The mutation fence now requires both require_signer(ctx)? and did_matches(
in every registered resolver, and each_mutation_requires_and_binds_signer
covers all four task mutations for unsigned and DID-mismatch denial.

* docs(node): reconcile GraphQL fence docs with SDL discovery

Restore the historical OSS audit blocker text and add a dated status
update for #219. README and the maintainer roadmap now describe the
SDL-sourced fence and signer-plus-binding markers.

* ci: trigger the PR checks on a new head
2026-11-05 04:43:52 +00:00
beardthelionandKevin Codex 6b2c53bbb8 fix(gl): status-check the client read/write surfaces so a node denial surfaces as an error, not a fake result (#123) (#186)
* fix(gl): status-check the client read/write surfaces so a node denial surfaces as an error, not a fake result (#123)

Convert the gl CLI + MCP read/write surfaces to status-check the node's
response before parsing, so a denial (401/403/404) or a degraded 5xx surfaces
as an error instead of being rendered as an empty list or a fake success. The
shared http::read_json helper checks the status first and bounds the error-body
read (a hostile node can't stream an arbitrarily large error body to exhaust
the client), replacing the prior parse-before-status pattern across agent,
bounty, cert, changelog, issue, mcp, peer, pr, protect, repo, star, status,
sync, task, visibility, and webhook. A no_parse_before_status regression gate
locks the converted set in.

Rebased onto current main (squash of the prior branch). main had independently
gated several of these same surfaces, so the overlapping arms are reconciled to
keep BOTH fixes rather than let either regress:

- repo::cmd_label_list keeps main's get_maybe_signed (read-visibility gating:
  public repos stay anon-listable, a private-repo owner signs) carrying
  read_json on top; the PR's get_authed is dropped so the visibility gate holds.
- mcp webhook_list keeps main's get_signed and its signing-keypair-DID owner
  resolution (NOT resolve_owner's node DID) — list_webhooks is owner-gated AND
  auth-required, so the PR's plain get()+resolve_owner would have 401'd then
  403'd — with read_json layered for the bounded error read.
- webhook create/list/delete keep main's get_signed + resolve_owner_repo_pair
  structure and its tests, with read_json swapped in for the manual
  resp.json() so the error body is bounded. main's superseded resolve_owner
  helper and its duplicate tests are dropped.

gl: 388 unit tests + the no_parse_before_status gate green; fmt + clippy clean.

Rebased again onto current main (2026-08-11), which had moved 238 commits.
main independently landed read_body_capped and sanitize_node_msg into http.rs,
so this branch's duplicate definitions in sync.rs are dropped and read_json is
layered on main's helpers instead. main's peer.rs also grew its own announce
handling (remote_announce_failure, local_add_report/local_add_refusal, capped
read, sanitized message) which supersedes this branch's read_json conversion
there; main's side is kept whole and only peer::cmd_list, which main had not
gated, still takes read_json.

fix(gl): report why the node-DID comparison was skipped in cert show

The rebase onto main dropped did_check_report, since main's cmd_show had
been rewritten to do real Ed25519 verification and collapsed every
node-info failure into one "could not fetch current node info" NOTE.

Put the tri-state reporting back on top of main's structure: the signature
verdict and the --verify issuer anchoring are untouched, but the node-DID
comparison now names the reason it could not run and points at the offline
verification path, and "carried no DID" stays distinct from "the lookup
failed". The lookup goes through read_json so a denial yields a reason
rather than an opaque None, and it stays fail-soft: a node-info hiccup must
not turn a successfully displayed certificate into an error exit.

The DID extraction is split out as did_from_node_info so its guard is
testable. An empty did rejects rather than flowing into the comparison,
where it would print a mismatch WARNING against a DID the node never
claimed. That guard had no coverage: removing it leaves every pre-existing
test green and only the new one goes red.

fix(gl): close two completeness-gate escapes; surface the node's error key

Review of the rebased branch found the #186 completeness gate could not fail
on the two shapes that matter most. Both were reproduced by seeding the
regression and watching the gate stay green.

An unguarded parse escaped entirely. The rule flagged a parse followed by
.is_success(), which describes one bad ordering and says nothing about a read
with no status check at all -- the more dangerous revert, since it renders a
denial body as a result. Inverted it to require a status check above the parse,
which also covers a check spelled as_u16() >= 400. The lookback is 24 lines
because sync.rs's hand-rolled Trigger arm puts its .status() 19 lines up; it
should shrink if that arm is ever routed through read_json.

Membership in the scanned set keyed on the text "read_json" rather than a call
site, so a handler fully reverted off read_json kept its membership on a
leftover comment. Still-derived meant the pinned-vs-derived equality never
fired and the deconversion shipped green, dropping the cap and the sanitizer.
Now counts call sites on non-comment lines.

read_json read only `message`, but the task API answers with `error` alone
({"error":"task not found"} in gitlawb-node/src/api/tasks.rs), so every gl task
denial rendered as the generic "request failed". Falls back to `error` through
the same sanitizer and cap; `message` still wins when both are present, which
is the shared AppError envelope. sync.rs already read both.

Also covers cmd_show --verify, which had no test at all: the issuer anchoring
is the security-bearing half of the command, since a valid signature only
proves the certificate is self-consistent and a hostile node can self-sign one.
The certificates carry real signatures, otherwise --verify would bail before
reaching the anchoring and the assertions would be vacuous.

Each fix verified by reverting it: both gate escapes go red, dropping the error
fallback fails the two new read_json tests, and removing the anchoring block
fails three of the five --verify tests.

* test(gl): pin the stats mock hits and the unsigned MCP webhook_list denial

Closes the two items left open on the last review round.

The bounty-stats denial tests were already load-bearing: breaking either
mock's route regex turns both red, because mockito answers an unmatched
route with 501 and neither test's asserted status is 501. That proof is
incidental to mockito's fallback status, though, so a reworded assertion
would silently drop it. Assert the hit directly with expect(1) +
assert_async.

MCP webhook_list keeps get_maybe_signed rather than aligning with the CLI's
get_signed. An agent driving the MCP tool may legitimately have no local
identity, and issuing the request so the node's own denial reaches the
caller is more useful than failing client-side before the node is asked.
That is only defensible if the denial actually surfaces, so pin it: no
identity plus an explicit owner must issue an UNSIGNED request and turn the
node's 401 into an error carrying the status and the node's reason, not an
empty webhook list.

The new test points at an empty temp dir instead of passing None. None
falls back to the ambient ~/.gitlawb/identity.pem, which exists on a dev box
and not on CI, so the request would have been signed here and unsigned
there; the first run of this test failed exactly that way before the dir was
pinned.

Verified by execution: swapping get_maybe_signed for get_signed reds the new
test on "must name the status", and breaking either stats mock route reds
its test. gl: 412 unit tests + the no_parse_before_status gate green; fmt
and clippy clean.

* docs(gl): describe the client-contract rules in plain English

The comments carried internal shorthand identifiers that resolve to nothing
for a reader outside this repo. Replace each with the property it names: a
denial must surface as an error rather than render as an empty or fabricated
result, and an error body must be read under a cap and sanitized before it
reaches the terminal. Comment-only; no behavior change.

Verified after the edit because the completeness gate reads source text to
decide which handlers it scans: reverting a converted handler to a bare
.json().await still turns it red. gl: 412 unit tests green, fmt and clippy
clean, cargo check --workspace --locked clean.

* test(gl): accept only a full non-success guard above a raw parse

A 404-only equality upstream counted as status evidence, so reverting a
read_json call behind agent.rs's special-404 hint to resp.json().await?
kept the fence green while a 500's denial body rendered as a result.
Require .is_success() or an as_u16() range comparison in the lookback,
and pin the distinction with a mutation regression that performs that
exact revert on the real agent.rs source.

* test(gl): drop the traced result from the cert assertion diagnostics (#186)

CodeQL reported seven high-severity "Cleartext logging of sensitive
information" alerts at cert.rs:497, 540, 584, 667, 708, 803 and 862, and the
aggregate check has been red on the branch since. All seven are #[cfg(test)]
assertions that format a whole Result<()> through {result:?} or {got:?}.
CodeQL traces a value associated with resolve_cert_id into the command error
and treats the assertion's panic diagnostic as a logging sink.

The alerts are test-only. The single mod tests opens at cert.rs:328 and runs to
EOF, so all seven sit inside it, and the traced certificate id is already
printed unconditionally in production at cert.rs:145 because it is a public
identifier. That makes these repeated instances of one test-diagnostic pattern
rather than seven distinct vulnerabilities.

Each message now carries static context naming the scenario instead. The
failing test name and source line still identify what broke, and every
assertion predicate is unchanged, so nothing is weakened into vacuity. The
production cleartext-logging query stays enabled: nothing is suppressed or
reconfigured.

One {got:?} at cert.rs:900 is deliberately left alone. CodeQL did not flag it,
its value comes from did_from_node_info over hardcoded literal JSON with no
resolve_cert_id taint, and the interpolation is load-bearing there because it
names which of three loop cases failed.

Verified: cargo test -p gl --bin gl cert, 20 passed. fmt clean.

* test(gl): restore the nine peer denial tests dropped in the rebase (#186)

The 2026-08-11 rebase squashed the branch and lost nine tests. Counting
`async fn cmd_add_` in peer.rs per head: 6 at 06863da0, 3791c40c, 9ed2947d and
ad0e35ad, then 0 at 21ba974c and 0 at fba90038. The production fail-soft at
peer.rs:173-180 survived; only the tests pinning it went.

That matters because the rule this PR's own round 3 established is that a
carve-out with no test that goes red when someone makes it fatal is an
incomplete sweep, not a documented decision. The peer-add public-URL lookup has
been in exactly that state for three weeks.

Recovered from e179cd22, not 5eff6060: the earlier commit carries only four of
the seven items, while e179cd22 has all of them plus three same-class denial
tests for cmd_list, cmd_ping and cmd_resolve that were lost the same way.
Neither commit is an ancestor of HEAD.

Nothing needed adapting. cmd_add's signature and its node-info block are
byte-identical between the two heads. The announce arm has since moved to
remote_announce_failure, which still formats the status the denied-announce
test asserts on, and the local-add arm has moved to local_add_report, which
still returns Ok either way, so both local-add tests hold. The named import
list gains cmd_list, cmd_ping and cmd_resolve.

Proven load-bearing rather than assumed: rewriting the fail-soft match into a
fatal `?` turns cmd_add_announces_fallback_url_when_node_info_denied and
cmd_add_falls_back_on_malformed_node_info red, and leaves the pre-existing
a_peer_failure_past_the_read_cap_still_reports_its_status green. That last
point is the reason these are not redundant with what was already there: the
surviving test drives a 503 whose body is past the read cap, so it pins the
cap and ordering interaction, not the fallback.

Verified: cargo test -p gl --bin gl peer, 25 passed. fmt clean.

* fix(gl): route the status repo sections through read_json and pin the file (#186)

`gl status` was hardened in this PR with section_unavailable_line and
trust_line, so a denied section prints "unavailable ({status})" instead of
rendering as empty. But it was hardened by hand rather than through read_json,
so status.rs carried no marker, never entered the completeness gate's derived
set, and was never scanned. Deleting the helper call would have restored the
denial-as-empty bug with the gate still green: the one file written to fix this
bug was the one converted file the gate did not cover.

The two repo-section parses now route through read_json and status.rs is
pinned in CONVERTED_IN_186. Exactly two source lines change.
section_unavailable_line takes the response by reference and reads only its
status, so read_json sits underneath it with no restructuring, and the outer
if-let shape still stops one denied section from aborting the others. All 35
status tests pass unchanged, including the eight denial tests and their exact
rendered strings.

trust_line stays hand-rolled on purpose. It needs a three-way split (2xx,
exactly 404, other non-2xx) that read_json collapses into a single Err, and it
already carries a literal status check the scan can see.

A second pinned const naming hand-rolled guards was considered and rejected. It
would enforce only that a symbol is spelled somewhere in the file: deleting the
println while keeping the call would stay green. A structural anchor already
exists, which is read_json, and the repo's own guard-design note is explicit
that a name-keyed filter must never be the membership predicate when one does.

Proven, not assumed. Pinning status.rs before converting it turned the gate red
naming the file. Reverting even one of the two read_json calls afterwards turns
it red again, on the unguarded check rather than the pin, because the issues
parse has no full status guard in its lookback window. The one gap worth
naming: deleting both section_unavailable_line call sites while keeping
read_json still scans green, and the denied section would then print nothing
rather than a false "no open pull requests". Probed it, and clippy catches that
case as a dead function under -D warnings, so CI would fail. That backstop
holds only while the helper has no other caller.

Verified: gate 2 passed, cargo test -p gl --bin gl status 35 passed. fmt clean.

* fix(gl): let gl pr view finish rendering when a sub-fetch is denied (#186)

Converting cmd_view's sub-fetches to read_json turned a tolerated 404 into a
mid-render abort. The header prints first, then the reviews fetch is bare `?`,
so a denial leaves a half-rendered PR, exit 1, and the comments section never
runs. Before the conversion the error body parsed, the section was skipped and
the rest of the command still rendered, so this is a regression this PR
introduced rather than pre-existing behavior.

Both sub-fetches now degrade: a denial prints "Reviews unavailable ({status})"
or "Comments unavailable ({status})" and rendering continues. That is the shape
this PR already invented one file over for gl status's denied sections, so the
two commands now behave the same way. read_json stays in use, so pr.rs stays in
the gate's derived set and the pinned list is untouched.

The header fetch stays fatal. With no header there is nothing to render
partially, and softening it would resurrect denial-as-success for the whole
command. cmd_view_header_denial_stays_fatal pins that, and it passed before
this change as well as after, which is the point: it is the must-not case.

mcp.rs's pr_view arm has the same fetch coupling and is deliberately left
alone. It emits a single JSON result, so there is no partial-output problem, a
whole-call error is the correct fail-closed answer for a machine consumer, and
inventing a partial-result shape is a design change nobody asked for.

Test-first and observed: cmd_view_continues_when_reviews_are_denied and
cmd_view_continues_when_comments_are_denied both failed before the change and
pass after. Each asserts the comments mock was hit with expect(1), which is the
load-bearing part: the mock can only be hit if rendering continued past the
denied fetch.

Verified: cargo test -p gl --bin gl, 473 passed. Gate 2 passed. fmt and clippy
clean.

* ci: trigger the PR checks on a new head

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-05 04:20:10 +00:00
beardthelion fc87586411 fix(node): hold fork to the same repo-name rules as every other creation route (#412)
* fix(node): hold fork to the same repo-name rules as every other creation route

fork_repo built its clone destination with store::repo_disk_path, a raw
join with no validation, while create_repo and the sync mirror path both
go through validated_repo_disk_path. Its own name check is
`.chars().all(|c| c.is_alphanumeric() || c == '-' || c == '_')`, which is
Unicode-aware and has no length or leading-character rule, so it admits
four shapes validate_repo_name refuses: the empty string (all() is
vacuously true on it), a non-ASCII alphanumeric such as "cafe" with an
acute e, a leading '-', and a name over the 100-char bound.

The empty name was the reachable defect. Driven through the production
router with a real Ed25519 signature, a `{"name":""}` fork returned 201
Created with a repo row whose name is "" and a clone_url ending in
`/.git`, materialized at `<repos_dir>/<owner_slug>/.git`. It now returns
400 and creates neither the row nor the directory.

This is a deliberate narrowing, not only an empty-name fix: the other
three shapes above are now 400 on fork where they previously reached the
raw join. None of them could ever have been created through create_repo,
which fails at repo_store.init on the same validator, and no existing row
can carry such a name (create_repo validates before its DB insert, and
mirror rows go through validate_repo_slug), so no repo becomes
unforkable. A fork that defaults to source.name is unaffected.

The traversal shape of the same join is NOT reachable and is not what
this changes: forker_did comes from AuthenticatedDid, which auth/mod.rs
inserts only after Did::to_verifying_key resolves a multibase-decodable
did:key, so it can carry neither ".." nor "/".

repo_disk_path is now cfg(test). It has no production caller left, and
the attribute turns the next one into a compile error rather than a
silent bypass. Tests keep it because a fixture that must escape repos_dir
cannot be built with the validated form.

Two tests. A unit test on the mounted handler, and an end-to-end test
through server::build_router with real signatures that covers all four
refused shapes plus a positive control asserting an ordinary name still
forks to disk and to a row. Both are mutation-verified in both
directions: reverting to the raw join reddens the refusals, and forcing
fork to always refuse reddens the control.

Closed #272 fixed this class on the sync route and pointed at
repo_disk_path as the sanitizing convention; it is in fact the
unvalidated join, and the fork route was never scoped.

* fix(node): validate the fork name before spending the proof

The barrier landed after `proof.consume` and after `repo_store.acquire`, so a
name the character allowlist admits but `validate_repo_name` refuses burned a
valid iCaptcha proof and paid a source-repo download before its 400. The
handler's own header comment promises the opposite: a fork rejected for a bad
name never burns a proof. Reported by CodeRabbit on the review of a7257dce.

It now runs with the other admissibility checks, above both.

The regression test pins the ORDER without iCaptcha plumbing. Seed a repo row
whose bytes are not on disk: late validation lets the acquire fail first and
the caller sees a 500 from git, so only early validation can produce the 400
the test asserts. Verified load-bearing by degrading the early call to
`unwrap_or_else`, which reddens it.

Both name rules are kept. CodeRabbit also suggested deleting the character
allowlist as a strict subset of `validate_repo_name`, and that is not the
relation between them: the allowlist rejects a dot, so `v1.2.3` and `my.repo`
are refused today while `validate_repo_name` accepts both, and the allowlist
accepts a non-ASCII alphanumeric that `validate_repo_name` rejects. Removing
it would newly admit dotted fork names, which is a behaviour change and not a
cleanup. A comment now records why the two coexist.

The empty-name fixture also generates its owner DID per run. It writes to a
fixed `/tmp/<owner_slug>`, so a shared constant let two concurrent `cargo
test` processes delete each other's repo mid-test (reported by Greptile).
2026-11-05 03:53:40 +00:00
beardthelionandKevin Codex 288ad92fe8 fix(node): bound the request body the signature middleware buffers before it verifies (#262)
* fix(node): bound the request body the signature middleware buffers before it verifies

`require_signature` buffered the whole request body with an unlimited
`body.collect()` before parsing a single header, and nothing upstream capped it.
On the git write routes the 2 GB `RequestBodyLimitLayer` is applied to the inner
router while `add_auth_layers` wraps it, and a later `.layer()` is the outer
service, so the middleware ran first and saw the raw body. The other signed
groups carry no limit layer at all and lean on axum's 2 MB `DefaultBodyLimit`,
which the extractors enforce through a request extension this middleware never
consults, so that default was bypassed too. `optional_signature` forwards to the
same code whenever a caller merely presents signature headers, so the reach
extended to `/graphql`, `/ipfs/{cid}` and the other optional-signature groups.

Measured against the real router: an unsigned push with `GITLAWB_MAX_PACK_BYTES`
set to 64 KiB drained the full 4 MiB body and only then returned 401. No DID, no
signature, and no repo were required, and in front of it sits only a per-IP
arrivals-per-hour counter, no timeout layer and no connection cap.

The collect now runs through `http_body_util::Limited`, so reading stops at the
limit instead of after the fact, and an over-limit request gets 413 rather than
the 400 reserved for a genuinely unreadable body. The limit is per route group:
`add_auth_layers` takes it and passes it to the middleware as state, the git
groups keep `max_pack_bytes` so large pushes are unaffected, and the rest get a
constant equal to axum's default because that is already what their extractors
enforce. Verification, content-digest, clock-skew and the request reconstruction
are untouched.

The regression test asserts bytes actually polled out of a streaming body, not
the status code, because a status assertion passes whether or not the body was
drained. Neutralizing the limit turns it red; handing every group the git limit
turns the non-git test red; returning 400 instead of 413 turns the status test
red.

This bounds per-request buffering only. Total resident memory is still
O(connections x limit); a read timeout and a connection cap are what close that
and are deliberately not in this change.

* test(node): prove a signed push under the limit lands on disk, and an over-limit one writes nothing

The body-limit guard was covered only by tests that reject. Nothing proved it
still admits what it should, so the fix could have passed every test by refusing
everything, and nothing would have caught a middleware that truncated or
corrupted the body it hands downstream.

These drive a real push: a source repo with a 300 KiB incompressible blob, an
empty bare repo at the path the store resolves, and the byte-for-byte
receive-pack request body captured off the wire from an actual `git push`. The
body is captured rather than generated, because `git send-pack --stateless-rpc`
emits remote-curl's extra RPC framing (a wrapping length plus a stray flush
before PACK) and `git receive-pack --stateless-rpc` rejects it with a pack
signature mismatch.

The positive case sets the cap just above the body and asserts the ref moved and
the tree resolves, not merely that the status was 200. That distinction is
load-bearing: under a truncation mutation the status stays 200, because git
reports the failure inside report-status, and only the on-disk assertion goes
red. Running the whole suite under that mutation leaves exactly one test failing,
so this is the only thing standing between a corrupted body and a green build.

The negative case sets the cap just below the body and asserts 413 with neither
the ref nor the commit present afterwards, so a rejected push cannot half-write.
Its existence check peels to ^{commit}, since rev-parse echoes any well-formed
40-hex string back and would have passed vacuously.

* test(node): give the unresolvable-DID middleware test the signature body limit state

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-05 03:10:35 +00:00
beardthelion 86c544a577 fix(core): reject small-order Ed25519 keys at DID resolution and before X25519 (#314)
* fix(core): reject small-order Ed25519 keys at DID resolution and before X25519

A did:key can encode a small-order (weak) Ed25519 point. VerifyingKey::from_bytes
only decompresses, so such a key resolved cleanly, and x25519_public converted it
to the all-zero Montgomery u. X25519 against u = 0 is the all-zero shared secret
for every scalar, so the per-recipient wrap in seal_blob could be rebuilt with no
secret at all. Since one content key is wrapped per recipient, a single weak
recipient exposed the blob for every recipient.

Reject in Did::to_verifying_key, which is the choke point every consumer that
resolves a DID shares, so a weak key becomes unresolvable and the existing
fail-closed recipient handling catches it without a new code path. Guard
x25519_public as well so the primitive is safe for a caller that obtained the key
some other way.

The attacker test performs the real attack rather than asserting an absence: with
either guard removed it recovers the plaintext and fails printing it.

* fix(attest): mirror the small-order key rejection in its own did:key parser

gitlawb-attest parses did:key itself in verifying_key_from_did_key and never
routes through gitlawb-core's Did. It cannot: gitlawb-core is a dev-dependency
here, so the choke-point rejection is structurally unreachable from this crate
and has to be mirrored rather than inherited.

Signature verification is already strict, so this is defense in depth against a
future consumer of this parser that does not verify strictly, not the closing of
an exploitable path.

One consequence worth naming: verify_rejects_weak_key_signature now observes
Error::Did rather than Error::Signature, because the weak signer is refused at
resolution before verify_strict is reached. That test is retargeted accordingly.
It no longer exercises verify_strict, which remains the second layer for
malleability that does not involve a weak public key (a small-order R under an
honest key); this fixture cannot construct that case.

* test(node): pin that a weak recipient DID falls into the existing fail-closed arm

Because a small-order did:key no longer resolves, it lands in the unresolved set
and plan_seal returns the #47 SkipUnresolvable arm that already refuses to seal
to a partial recipient set. Nothing is sealed for the blob and the existing
bounded operator warning fires. No new branch, no new SealPlan variant, no new
log site.

The must-not-over-reject control asserts an all-legitimate set still seals.

* fix(gl): stop telling a did:key that did:key is unsupported

did_resolve reported every resolution failure as "only did:key is supported
without a resolver". That is right for a method with no local resolver and
misleading for a did:key that parsed fine and failed for its own reason, which
now includes a small-order key.

Split the message on whether the DID is a did:key and surface the underlying
error when it is. Extracted as a pure helper so both branches are unit-testable
without an MCP client harness.

* fix(review): restore verify_strict coverage the parser guard displaced

Refusing a small-order key at DID resolution means a weak signer can no longer
reach attestation.rs's verify_strict through verify_signature, which left that
line with no test at all: downgrading it to the non-strict verify kept the whole
attest suite green. Split the signature check into verify_sig_with_key so a test
can drive it directly with a weak key. Check order in verify_signature is
unchanged; only the seam is new.

Also from review:
- auth/mod.rs sent a did:key that failed on its own key material away with
  "only did:key is supported in alpha", the same misleading hint the gl MCP
  tool was just fixed for. Made it conditional on is_did_key so the two
  parallel surfaces say the same thing.
- The guard comments claimed a small-order point converts to the all-zero
  Montgomery u. That holds for order 1 and 2 only; order 4 and 8 are
  annihilated by the scalar clamping instead. is_weak covers the whole torsion
  set, and mixed-order points are correctly still accepted.
- did.rs doc comment now states the new rejection.
- encrypt.rs: extracted the repeated weak-key fixture into a test helper and
  switched to the file's existing Err(anyhow!(..)) rejection idiom.
- gl tests derive the did:key discriminator from a real parsed Did instead of
  hardcoding the bool, so inverting it at the call site now fails.

* fix(attest): cap the did:key method-id before decoding, mirroring core

The parser here mirrors gitlawb-core's small-order rejection because it cannot
call into it (gitlawb-core is a dev-dependency of this crate). It was mirroring
only one of the two guards that sit three lines apart in the function being
copied: core also caps the method-id at 64 bytes BEFORE the multibase decode,
because base58 decoding is quadratic in its input.

signer comes off an attacker-supplied attestation and verify_signature reaches
this parser after computing the JCS bytes, so an oversized method-id bought a
large decode for a short request. An ed25519 did:key method-id is a fixed 48
characters, so the bound is slack rather than a behavior change.

The test asserts the length error specifically rather than any error: a
multibase error there would mean the cap ran after the decode and bought
nothing.

* fix(node): surface stored envelopes sealed to a now-unresolvable recipient

plan_seal short-circuited on a matching stored tag before resolving the
recipient set, so a blob sealed to a small-order did:key before the
choke-point guard landed kept its old CID and was silently treated as
unchanged. Resolve the set on the match and surface a now-unresolvable
reader as SkipUnresolvableStored so the operator sees the exposure.

* fix(attest): anchor VerifiedAttestation.signer to the key that verified

Registry::verify discarded the VerifyingKey returned by verify_signature
and copied the artifact's signer string into the result. The parser is
canonical so the strings agree for honest artifacts and a mismatched field
already fails verification, but the consumer-facing value now provably
comes from the key that verified rather than a raw artifact field.

* fix(node): bound the echoed DID in the unresolvable-did error and pin the hint branches

The keyid is unauthenticated request input and was reflected into the
response message at full length. Bound the echo to 96 chars and extract the
two-branch hint selection into a testable helper with both branches pinned.

* test(gl): build the weak did:key fixture from its identity-point bytes

The hardcoded WEAK_DID_KEY constant trips secret scanners, which cannot
tell a public DID from an API key. Deriving the DID from the compressed
identity point states why the fixture is weak and matches the house pattern
used in the core, attest, and node crates. ed25519-dalek is added as a
dev-dependency (it was already a workspace dependency).

* fix(core): skip a low-order ephemeral on the envelope open side

open_blob built the X25519 box from the header-supplied eph with no
small-order check, so a crafted entry forced the all-zero shared secret and
unwrapped for any reader. Skip entries whose eph decompresses to a
small-order point; honest entries in the same envelope still open.

* fix(core): test the exchange result, not the ephemeral's encoding

The open-side guard decompressed the ephemeral to Edwards and asked
is_small_order, treating a failure to decompress as safe. That reports safe
for any low-order input that is not a valid Edwards encoding. Measured across
the seven standard low-order encodings: six caught, u = p-1 missed. Its
exchange with any reader is still the all-zero shared secret, so an entry
built on it unwraps for everyone and the attacker controls the plaintext.

Check the result instead: an ephemeral is rejected when the exchange with
this reader's own scalar is all-zero. That is the property the guard exists
to enforce, and it holds for every low-order input regardless of encoding.

The new must-not test needed two attempts. The first passed with the guard,
without it, and with the original pre-fix guard alike, because a blanket
`use chacha20poly1305::aead::Aead` in the test resolved the wrap's encrypt
through a different aead version than open_blob decrypts with, building a
ciphertext that could never open for any reason. Every call in it now names
its trait.

* docs(core): anchor open-side guard comments to exchange-result invariant

The seal-side comment still named is_low_order_montgomery and the
low-order vector test doc claimed a decode-path filter that does not
exist. Both now describe yields_all_zero_shared_secret.

* test(core): use generated keys in poisoned-envelope fixtures

Replace hard-coded test scalars that CodeQL flags as cryptographic
material. The all-zero shared-secret attack does not depend on which
scalar builds the attacker box.

* test(core): use Default::default() for zero-point fixtures flagged by CodeQL

CodeQL flags [0u8; 32] literals that flow to a cryptographic key
argument, even in test fixtures where the value is the all-zero
Montgomery point (a public low-order test vector, not a secret).
Default::default() produces the same bytes without a literal, clearing
the alert without changing behavior.

* test(core): generate body nonces with OsRng in poisoned-envelope fixtures

CodeQL flags hard-coded byte arrays used as nonces, even in test
fixtures where the value is a fixed test vector. Generate with OsRng
instead, matching the production seal path.

* chore: sync Cargo.lock after merging main (ed25519-dalek edge for gl)
2026-11-05 01:49:28 +00:00
beardthelionandKevin Codex 206e0827ad fix(node): bound replay of authenticated gossip ref-update events (#334)
* feat(node): add the freshness check and replay guard, not yet wired

Builds the two layers that bound replay of an authenticated ref-update, with
nothing calling them yet; the ingest path is wired in the next commit.

The freshness check is deliberately two-directional rather than an absolute
delta: an abs() window admits an event stamped up to the window ahead and pins
its seen-set slot until the clock catches up. Producers were enumerated before
settling the unparseable arm; the sole production publish site emits RFC-3339,
so an unparseable timestamp is refused rather than admitted.

The replay guard keys on SHA-256 of the canonical signing bytes, not the raw
wire bytes, because one signature verifies against many encodings and only the
canonical form collapses them to a single key. A golden digest is frozen for
the pre-version artifact, and the same constant is asserted for its v-injected
twin, so the collapse is pinned rather than described. Reservations settle
through a drop guard so only a confirmed entry outlives the ingest call, which
keeps a transient write failure from permanently burning an event's slot.

Replayed and StaleTimestamp are separate outcomes because they diagnose
different conditions, a mesh replay against a broken clock or a healing
partition, and folding them would be the same observability lie the unsigned
shed variant already exists to avoid.

* fix(node): bound replay of authenticated gossip ref-update events

Wires the freshness check and the replay guard into ingest, immediately after
signature verification and above the author debit. The lower placement, just
above the writes, stops the duplicate row and the duplicate sync but still lets
every replay drain the victim author's budget, which is the harm the defect
names: a captured signature replayed 500 times empties the victim's window and
their next genuine push is refused.

The guard runs on the verified path only. Unsigned event bytes are predictable,
so applying it there would let an attacker pre-send a victim's expected event
and have the genuine one dropped as a replay, which is a censorship primitive
rather than a defense.

A replay flood still costs a parse and one Ed25519 verify, because the guard
has to sit below verification for the reason above. What it removes is the
peer_exists round trip, the victim's author debit, the ref-update row and the
sync enqueue.

The existing author-budget test signed one event and ingested the same bytes
five hundred times, so it had to re-sign per iteration to keep exercising the
budget under a shared guard. Giving it a fresh guard per call would have kept
it green while gutting the property it exists to prove. Its over-budget probe
needed the same treatment, since the replay gate sits above the author gate and
would have refused the burst's last bytes before the budget assertion ran.

Both saturation tests now take a shared lock so each keeps an exact assertion
on a process-wide counter; a lower bound would stay green if the fail-open
branch ever double-counted.

* fix(review): bound the saturated guard's rescan and the log it writes

At capacity the inline sweep ran on every event, so a full O(capacity) retain
happened under the lock once per message and reclaimed nothing when nothing had
expired. The guard that bounds replay became a CPU amplifier in exactly the
state an attacker drives toward. The sweep is now rate limited to once a second;
the periodic sweep still reclaims on its own cadence, so only the redundant
rescans go away.

An unparseable timestamp was echoed into the refusal detail verbatim. That value
is attacker-controlled and arbitrary length, so it reached a warn! as both a
log-injection and an unbounded-size sink. Only that arm needs sanitizing; the
other two ran through the parser first.

The capacity rationale cited a count of registered DIDs, which this same file
says elsewhere an attacker mints freely through the announce path, so it was not
a bound at all. It now cites the bound that is real: reaching saturation costs a
hundred thousand durable rows and as many sync enqueues inside one retention
horizon, which the database makes loud.

ingest_now was read twice per ingest while its own doc comment claimed the two
layers share one reading, which is the invariant the retention derivation rests
on. Now read once and passed to both.

Also: the Unparseable outcome is driven through ingest rather than only as a
pure function, the periodic sweep is observable without a live swarm, the
restart exposure is written down where the rest of the tradeoffs already are,
and the saturation-counter lock covers every test that can reach Saturated
rather than the two I first found.

* test(node): close the replay guard's remaining proof gaps, and fix the sibling window

Seven tests for the six gaps a review found. The reservation's release path was
reasoned rather than executed: the drop guard was only ever proven through an
early refusal, while the case its own doc comment names, a transient write
failure burning the event's slot, was never driven. Both write directions now
are. An expired entry at capacity must be replaced in place rather than answer
Saturated, confirm on an already-swept entry is pinned as a deliberate no-op,
and the single-critical-section shape is now driven concurrently rather than
asserted sequentially. Restart behavior was documented but untested; a fresh
guard readmits a seen event and the freshness window still bounds it, which is
the composition that makes the restart exposure finite.

check_created in gitlawb-core used a symmetric abs() window, so a request
stamped 299 seconds ahead was accepted and a signer could roughly double a
signature's effective validity by stamping forward. It is now two comparisons
like the gossip path, 300s late and 60s early, with the error naming the
direction so a fast peer and a slow one need different operator action. There
was no future-direction test at all; there are now five covering both. Every
caller was checked and none depended on the symmetry.

Adding a freshness window obliges auditing the siblings, which is how this one
surfaced: the repo argued both ways in two files for a week.

* fix(core): stop an attacker-supplied created timestamp from overflowing the window check

created is parsed as an unrestricted i64 straight from the Signature-Input
header, so the sender picks it. At i64::MIN the past-side subtraction overflows,
which panics a debug build and wraps a release one into a value that can read as
inside the window. Confirmed by execution before the fix: 'attempt to subtract
with overflow'.

Saturating subtraction answers correctly at both ends, since a timestamp that
far out is refused by whichever side it saturates toward. A test drives all four
extremes of the type and asserts each is refused by a named direction; reverting
to plain subtraction reddens it on the overflow.

The symmetric abs() form this replaced had the same hazard, so splitting the
window into two comparisons did not introduce it, but it did not remove it
either. Found by a cross-family review pass after six same-family reviewers and
I had all read the line.

* fix(review): reconcile the replay guard with the unsigned-admission split

Integration only, from rebasing this branch onto #325's current head. #325 now
returns IngestOutcome::UnsignedAdmitted where it used to return Accepted for an
unsigned rolling-upgrade admission, and this branch predates that split.

Three sites. The seen-set bypass test asserted Accepted on both unsigned
deliveries and on the stale-timestamp case; both now expect UnsignedAdmitted,
and the properties under test are unchanged, that unsigned bytes are admitted
twice rather than deduplicated and that the freshness window does not reach
them. The warn-only-on-admission test's budget-spent case gained the
ReplayGuard argument the signature now takes, with a fresh guard because that
case drives an unsigned event the replay block skips.

The ingest match also grows an arm rather than changing one: `None if unsigned`
returns UnsignedAdmitted without settling a reservation, since the replay block
is gated on `verified` and an unsigned admission never holds one.

* fix(node): render rejected gossip timestamps bounded before logging

DateTime::parse_from_rfc3339 accepts arbitrarily many fractional-second
digits, so a signed timestamp can parse successfully and still carry
megabytes of attacker-chosen wire bytes into the TooOld / TooFarFuture
refusal details, which the swarm loop logs verbatim. Only the
parser-error branch sanitized. The root cause was treating successful
parsing as proof the wire representation is safe to expose.

The TooOld and TooFarFuture variants now carry the PARSED instant and
the detail renders its canonical to_rfc3339() form - bounded by this
build's formatter and control-free by construction - while the
unparseable arm keeps sanitize_for_log on the wire string. All three
arms go through one freshness_refusal_detail helper so the invariant
has a single home, plus a regression test driving an overlong but
parseable fractional stamp through both directions.

* test(node): pass the replay guard to the post-fork shed tests

The two ingest tests added on the target branch after this PR forked
(an_over_budget_author_is_shed_without_a_peer_lookup and
the_early_author_check_does_not_track_an_unseen_did) still call
ingest_ref_update with the pre-guard signature. Thread a fresh
ReplayGuard through each call, matching every other test site.

* test(node): drive the live-mesh publish and refusal path end to end

Two real swarms on QUIC loopback with real signatures and real Postgres:
a signed ref-update from one node is admitted and stored by the other, a
re-delivery of the same signed content under a malleated wire encoding is
refused by the replay guard, and an unsigned event is refused under
require_signed. Both refusals are asserted in the warn stream, and a
control event landing after them proves the mesh still delivers.

This closes the seams ref_update_publish_args names as needing a live
swarm: the PublishRefUpdate select! arm dispatching to sign+publish, and
require_signed reaching the receive path. It needs a test-only command
variant, PublishRawRefUpdate, since the production publish path signs and
serializes itself and can never put a malleated or unsigned event on the
wire.

Still uncovered and named in the test's doc comment: the run_ingest_sweep
select! arm on its 300-second interval, and the main.rs half of the flag
wiring above p2p::start.

* fix(node): harden the gossip rejection path against unauthenticated bytes

Two findings on the pre-auth reject path:

- An unsigned event under require_signed was resolved and slug-validated
  before the flag refused it, so a hostile flood bought DID resolution and
  a rejection reason built from attacker-chosen fields under only the
  loose pre-parse brake. The unsigned refusal now runs immediately after
  the version gate, before any field is read.

- The swarm loop logged IngestOutcome::Rejected and WriteFailed reasons
  verbatim, and several producers embed wire fields (the claimed node_did
  in resolve failures, for one). A signed event carrying a control-bearing
  node_did reached the operator log with escape sequences intact and no
  length bound. Both warn sites now render through sanitize_for_log, so
  every present and future Rejected producer is bounded and control-free
  at the one sink.

The live-mesh test now also publishes a signed event with a 4096-char
control-bearing node_did and asserts the warn stream shows the refusal
with no escape byte and no run past the sanitizer ceiling. A unit test
pins the new ordering by asserting the exact fixed refusal for an
unsigned event with a hostile node_did.

* fix(node): dedupe the ref-update row on the replay key, not on a random id

The reservation releases on a failed write so a republish can repair the
half that never landed. But each ingest minted a fresh row id, so a retry
after a partial failure (row stored, queue write failed) stored the same
signed content a second time. A replay in a degraded state could repeat a
landed side effect until a budget refused it.

The row id for a verified event is now derived from the replay key, so the
retry's insert dedupes on the existing ON CONFLICT instead of duplicating.
The queue entry still retries, which is the repair the release exists for.
Unsigned events keep a random id: they carry no replay key by design, and
deduplicating unauthenticated content would let an attacker pre-send a
victim's expected event and have the genuine one dropped.

Two existing tests pinned the duplicate write and now assert the deduped
shape: the partial-failure republish repairs the queue entry while the
row stays at one, and the restart readmission's residual effect is a
duplicate queue entry (a refetch), not a second record. The new test
asserts both the retry reaches the writes (WriteFailed, not Replayed) and
the row count stays at one.

* test(node): pass a libp2p identity to p2p::start in the gossip delivery tests

* ci: trigger the PR checks on the retargeted base

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-04 23:04:03 +00:00
421f03ac46 fix(node): reject revision syntax in issue ids and drop service passthrough (#521)
* fix(node): reject revision syntax in issue ids and drop service passthrough

resolve_issue_id spliced the request-path issue id straight into a git
revision (`cat-file -e refs/gitlawb/issues/<id>`) and a for-each-ref
pattern, and on an exact match echoed the raw input back as the
canonical id. Revision and glob syntax such as `<uuid>^{blob}`, `~0`, or
`*` therefore resolved to an existing issue and was returned to callers
(and stored by create_issue_comment) as if it were the id. Accept only
ASCII alphanumerics and '-' and report anything else as not found.

service_to_command fell through to returning the caller's service string
for an unknown service. Every caller validates first, but the mapping
now yields only the two fixed subcommands and errors otherwise, so an
unrecognized value can never reach git's argv.

Addresses CodeQL rust/command-line-injection alerts #83 and #80.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(node): cover service_to_command allowlist and passthrough rejection

Direct unit test ensures unknown services and option-shaped inputs cannot
map to git argv even if HTTP-layer validate_service is bypassed in future.

Co-authored-by: Cursor <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Cursor <[email protected]>
2026-11-04 22:44:09 +00:00
JATMNandClaude Opus 5.5 e2ae21795a chore: rebrand user-facing gitlawb references to Twigpine (#520)
* chore: rebrand user-facing gitlawb references to Twigpine

Rename the product name in docs, README, CLI help/output, MCP tool
descriptions, macOS app strings, installers, npm metadata and license
headers from gitlawb/Gitlawb to Twigpine. Point GitHub links at the
renamed Twigpine org (and fix the release workflow's repository guard,
which still matched Gitlawb/node), and switch the public node hosts and
website/install URLs to twigpine.com.

Protocol and operator identifiers are intentionally unchanged
(did:gitlawb, gitlawb://, signing domains, GITLAWB_* env vars, crate
and binary names, X-Gitlawb-* headers, icaptcha and peer-network URLs);
they need a follow-up migration.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs: point image pulls at ghcr.io/twigpine/node and fix macOS DMG name

The release workflow publishes the container image to
ghcr.io/${GITHUB_REPOSITORY,,}, which is ghcr.io/twigpine/node since the
org rename; the old ghcr.io/gitlawb/node path no longer resolves for
anonymous pulls. Point the RUN-A-NODE docker run example, the OSS
readiness audit's claim about it, and the macOS app's generated compose
file at the published path.

The README listed the macOS DMG as "Twigpine Node.dmg", but
build-macos-app.sh writes GitlawbNode-macOS.dmg (kept on purpose), so
document the real file name.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(infra): keep the AWS security group description to avoid replacement

`description` on aws_security_group is ForceNew, so renaming it forced a
replace of the in-use group on the next apply of an existing deployment.
Restore the original text and note why it must stay until a separate
infra migration.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(macos): regenerate the app's compose file instead of treating it as a repo's

Config.repoPath fell back to the data directory when no repo clone was
found, and DockerCompose then picked up the compose file it had generated
there as if it belonged to a repo. The generator therefore only ran on
first launch, so upgraded installs kept their original compose, including
the old ghcr.io/gitlawb/node image, which no longer serves anonymous pulls.

repoPath now returns nil when there is no clone, and the app regenerates
its compose file whenever no repo compose is in use. Project directory and
named volumes are unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(infra): default the AWS node image to ghcr.io/twigpine/node

release.yml publishes to ghcr.io/${GITHUB_REPOSITORY,,}, which is now
ghcr.io/twigpine/node; ghcr.io/gitlawb/node denies anonymous pulls, so the
default made new deployments fail to start and the upgrade command fail
on existing ones. On an existing deployment `terraform apply` only updates
the upgrade SSM document in place (user_data drift is ignored); the new
image is pulled on the next upgrade_command.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(macos): keep operator edits to the generated compose, migrate only the image

e04efe4 regenerated the app's compose file on every access, overwriting
edits operators make to pass settings the generated environment block
omits (e.g. GITLAWB_OPERATOR_STRICT_MODE). Generate the file only when it
is missing; when it exists, keep it and rewrite only references to the
retired ghcr.io/gitlawb/node image, keeping tags and digests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(macos): only migrate complete ghcr.io/gitlawb/node references

Without a left boundary the migration also rewrote the retired path when
it appeared inside a longer reference, such as a registry mirror
(registry.example.com/ghcr.io/gitlawb/node), pointing it at a path the
mirror may not have.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 21:58:29 +00:00
beardthelionandKevin Codex ed76adfe29 fix(node): fence a repo publish on the write attempt that owns it (#285)
* fix(node): close the advisory-lock probe's session if it is dropped mid-acquire

A cancelled .await does not cancel an already-sent SQL statement, so a
pg_try_advisory_lock whose future is dropped still takes the lock
server-side while the caller abandons the result, leaving nothing to
release it. The connection then returns to the pool holding the lock and
wedges that repo until sqlx recycles the session.

Introduce LockProbe, which owns the connection across the in-flight
try-lock and closes it in its own Drop if it is still held. close_on_drop
is a one-way setter, so the arming lives in Drop rather than being set up
front and cleared on success; disarming is Option::take, which is what
into_conn does once an acquire is actually observed. This is now the only
place that issues pg_try_advisory_lock.

The committed gate drops a probe without taking its connection, which is
the state a cancellation leaves behind, and polls a standalone observer
until the lock frees. Deterministic on purpose: the timing sweep that
found this window leaks roughly 1 in 600, which is not something a CI
gate can rest on. Observed RED before this change with the lock still
held for the full 10s window.

Refs #279

* fix(node): add a dedicated, lazily-connected advisory-lock pool

Pinning a connection for the lock's lifetime is only safe if those
connections come from somewhere other than the pool serving ordinary
request handlers, otherwise a push burst starves every other query. Add
GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS (default 32) and a Db::lock_pool
builder, with the sizing tradeoff documented on the field and in
.env.example: every in-flight write pins one connection here, so the
value is a hard ceiling on simultaneous writes node-wide.

The pool connects lazily on purpose. The main pool must connect eagerly
because it runs migrations, which is why it needs connect_db_with_retry's
backoff and degraded-server handoff; that function is not a generic retry
helper and the lock pool is built well after the db-ready handoff has
already resolved. A lazy pool has no startup work, so it adds no new way
for the process to fail to boot and needs no second copy of that
machinery. If Postgres is unreachable when the first write arrives, that
write fails on the pool's own acquire timeout, like any other
database-backed request.

Pure configuration, so no proof-first cycle: the knob is covered by a
parse/default/reject-zero test. Db::lock_pool has no caller until the
guard wiring lands, hence the temporary dead_code attribute.

Refs #279

* fix(node): hold the lock-owning connection in RepoWriteGuard (#279)

Postgres advisory locks are session-scoped: only the backend that took one
can release it. acquire_write took the lock through fetch_one(&pool) and
release unlocked through execute(&pool), two independent checkouts, so the
unlock usually landed on a session that held nothing and returned false.
Measured on main: two writers on one node and the same repo BOTH acquired,
50 of 50 sequential cycles leaked, and 100 writes left 100 orphaned
advisory locks on the server.

The guard now owns the PoolConnection that took the lock, drawn from the
dedicated lock pool, and releases on that same session. The retry loop
probes through LockProbe so a cancellation mid-acquire cannot strand the
lock, and hands the connection back before each backoff so a spinner on a
contended repo does not pin a slot while idle.

Pool exhaustion is deliberately not retried. It is a different condition
from lock contention, and retrying it would spend all 60 attempts on a
capacity problem unrelated to this repo while reporting it as someone else
holding the lock.

Both #279 acceptance tests were observed RED first: the exclusion test
admitted the second writer, and the leak test reported 1 lock held where 0
was required. Both GREEN after. Full crate suite 516 passed.

Db::pool() is removed because this change was its only caller.

Refs #279

* fix(node): free the advisory lock when a write guard dies without releasing

A guard can exit without reaching release(): an early ? on the pre-write
download, a panic, or an axum handler future cancelled when the client
disconnects. Its session still holds the lock, so returning that
connection to the pool would block every future write to the repo until
sqlx recycles it, which on the 0.8.6 defaults is ten minutes idle or
thirty minutes lifetime. Close the session instead and let Postgres free
the lock at session end.

One hazard found by writing the teardown test rather than by reasoning:
PoolConnection::drop spawns onto the runtime for both closing and
returning, and panics outright when no runtime handle exists. That panic
would fire inside a Drop and abort the process during unwind. It is not
introduced by this commit, it comes with owning a PoolConnection at all,
but this is where it becomes reachable. So Drop checks for a runtime
first and, with none, leaks the handle deliberately rather than panicking:
the process is already exiting and socket teardown ends the session, which
is what frees the lock at exit anyway.

Observed RED before the fix, with the lock still held for the full 10s
poll window against a standalone observer on a no-reap pool, and the
teardown case panicking in sqlx-core connection.rs:208. Proven
load-bearing after: neutering the close_on_drop call turns the drop test
RED again. The must-not case (a released guard reuses its backend pid
across four writes on a pool sized 1) passes in both states, so the
signal is specific to the abandoned-guard path.

Full crate suite 519 passed.

Refs #279

* fix(node): read the advisory unlock's result instead of discarding it

pg_advisory_unlock reports "you did not hold this lock" as a false RETURN
VALUE plus a server WARNING, never an error, so let _ = execute(...) could
not distinguish a real release from a no-op. Three blindnesses stacked in
those four lines: execute discards the row, the boolean lives in the row,
and let _ discarded the Result too.

Read it through fetch_one into (bool,). A false means this session's lock
state is not what we believe it is, so the connection stays in the guard
for Drop to close rather than being handed back to the pool as clean. Only
a confirmed unlock returns it. A query error gets the same treatment, since
the lock must not outlive a session we can no longer reason about.

Note this is the only unlock site: the pre-write download's error path
returns through the guard, so Drop covers it and there is no second place
to keep in sync.

RED before: the connection came back on the same backend pid after an
unlock that returned false. GREEN after, and proven load-bearing by
treating false as success, which turns it RED again. The must-not case (a
normal release still reuses its backend) passes in both states, so this
does not over-close the happy path.

Full crate suite 520 passed.

Refs #279

* fix(node): bound the object-storage transfers that run under the write lock

Two transfers happen while the per-repo advisory lock is held: the archive
download inside acquire_write, which runs after the lock is taken and
before the guard exists, and the upload inside release. Both were free
before the guard pinned a lock-pool connection, because the lock's
connection went back to the pool immediately. Now an unbounded stall holds
a lock-pool slot for its whole duration, so enough concurrent stalls deny
every write on the node with no reaping path. Add
GITLAWB_LOCK_HELD_TRANSFER_TIMEOUT_SECS (default 300) and route both
through it.

A timed-out upload is UNKNOWABLE rather than failed, so the timeout arm
takes no compensating action: the PUT may well have landed. The lock is
released either way, which trades a narrow last-writer-wins window for not
wedging the repo behind a stalled transfer. Deliberate, and stated here
rather than discovered later.

Note what is NOT bounded by this knob: acquire_fresh's download. It runs
before any lock is taken, so it holds nothing. An earlier revision of this
change put the bound there by mistake, because both functions contain an
identical download call and the first match won.

Two disclosures.

Coverage: the committed tests cover the bound mechanism, not the wiring.
Driving a genuinely stalled transfer through acquire_write needs either the
object-store abstraction (out of scope) or a process-global
AWS_ENDPOINT_URL_S3 mutation, which would make the suite order-dependent
under the concurrent runner. That a stalled transfer is bounded is
therefore verified by reading, not by execution.

Behavior: on timeout with a local copy present, the download falls into the
pre-existing self-healing fallback and the write proceeds against that
local copy. This widens the conditions reaching that path from
corrupt-or-unreachable to include merely-slow, so a stale tree could now be
written and re-uploaded on a slow link. Kept consistent with the existing
failed-download behavior rather than inventing new semantics here; changing
it is its own decision.

Full crate suite 522 passed.

Refs #279

* fix(node): authorize before taking the write lock in close_issue

close_issue acquired the per-repo advisory lock, then ran the
owner-or-author check, then returned 403 with the lock still held. That was
harmless only because the lock excluded nothing. Making it work, as the
preceding commits do, turns this ordering into a denial primitive: any
caller with repo read access could take the write lock on demand and be
refused the write, while a legitimate writer burned its 60-attempt retry
budget against a lock held by someone with no write authorization. On a
public repo that is every permissionless identity. This series creates the
exposure, so it closes it in the same series.

The owner check is cheap and moves above the lock outright. The author
fallback needs the issue's git-JSON blob, since there is no author column,
so it now reads through acquire() rather than acquire_write(): an issue's
author is set at creation and never changes, so reading it outside the lock
races nothing, and only the mutation needs exclusion.

Two deliberate behavior choices. A non-owner whose authorship cannot be
established gets 403 rather than 404, so this route does not tell an
unauthorized caller whether an issue exists. And the owner's existing
404-for-a-missing-issue path is preserved by re-reading under the guard,
which the mutation wants anyway.

Swept the other three acquire_write sites: merge_pr owner-gates at :200
before acquiring at :214, the repo write path checks did_matches at :916
before :933, and create_issue's read gate is legitimate because that caller
IS authorized for the action it performs. close_issue was the only one with
the wrong order.

RED before: with an independent session holding the repo's lock, a
stranger's request sat in the retry loop until the 3s deadline fired
(Elapsed), which is the wedge itself. GREEN after, refused immediately.
Proven load-bearing: disabling the pre-lock refusal restores the Elapsed.

Full crate suite 523 passed.

Refs #279

* fix(node): address the review findings on the advisory-lock series

Seven fixes from a seven-reviewer pass. Eight of the nine substantive
findings were defects introduced by this branch, and two contradicted
claims made in its own earlier commit messages.

P0, the worst of them. The under-lock download's timeout was folded into
the same Err as a corrupt archive, so it fell through to the
use-the-local-copy fallback and the write proceeded. Two ways that
destroys data: we do not know whether we hold the latest tree, so
re-uploading can overwrite another node's newer archive; and the abandoned
download's extraction runs in an uncancellable spawn_blocking that ends in
remove_dir_all + rename over the same path, so git would have been running
against a directory a background task was about to delete. Now the
Option is branched rather than collapsed: a genuine fetch error keeps the
local-copy fallback, a timeout refuses the acquire and lets the guard's
Drop free the lock.

LockProbe closed its connection on ordinary contention, so a 60-attempt
spinner tore down 60 backends -- the exact behavior an earlier commit
claimed it avoided. The test written to guarantee that asserted the
HOLDER's lock count, which cannot observe the probe's own connection, so it
passed throughout. The correct predicate turned out to be narrower than the
first attempt at this fix: not "we saw an answer" but "we positively know
nothing was acquired," because a true answer means the lock IS held and
dropping without handoff leaks it exactly as a cancellation would. The
first attempt reopened U1's leak and U1's gate test caught it.

tigris.exists() ran unbounded inside the lock-held span, so the knob's
promise was not kept and the docs were wrong about the shape. The HEAD and
the download now share one budget, so worst-case occupancy is one budget
per span rather than two.

close_issue's pre-lock author read used acquire(), whose fast path returns
on a cached directory and never contacts object storage, so on a multi-node
deploy an issue author would be refused their own issue. Now acquire_fresh,
which refreshes without taking the lock -- the property the comment
already claimed. That comment also asserted authorship is immutable, which
a reviewer disproved by force-pushing a forged blob at refs/gitlawb/**;
reworded to the real justification (a pre-check whose mutation re-reads
under the guard). The underlying pushability is pre-existing and larger
than this branch.

The retry loop had no wall-clock cap, reaching ~360s against a 120s proxy
idle timeout that was itself lowered from 600s after held connection slots
caused a production outage. Capped at 90s, with the bail message naming the
real bound.

All four acquire_write call sites stringified their error, destroying the
anyhow downcast that error.rs documents explicitly ("without this, every
database outage surfaces as a 500 instead of a 503"). PoolTimedOut is in
that 503 set, so a saturated lock pool told clients not to retry something
transient. Now propagated.

Tests: six that the plan required and the first pass never wrote. The R4
pool-isolation test (named in the plan as proving the pool split, entirely
absent), close_issue's owner and non-owner-author twins (INV-21c, a
two-principal gate with only its deny arm covered), a waiter-does-not-block-
an-unrelated-repo case, and a config test for the transfer knob. The
runtime-teardown test no longer returns green when DATABASE_URL is unset.

Drop's no-runtime branch now detaches via leak() instead of mem::forget:
PgConnection has no Drop impl, so the socket closes synchronously and the
lock frees immediately rather than at process exit.

Evidence. The probe predicate is proven load-bearing in BOTH directions:
forcing always-close reddens the churn regression, forcing never-close
reddens the cancellation gate, and only the correct predicate satisfies
both. The author twin reddens when the fallback is removed.

One honest limit: the author twin does NOT redden when acquire_fresh is
reverted to acquire. With tigris disabled in tests the two calls are
identical, so that fix is correct by reading, not by execution -- the same
seam that leaves the transfer bound's wiring untested.

Full crate suite 529 passed, clippy -D warnings clean.

Refs #279

* fix(node): log lock-pool saturation in the request path, not via readiness

Completes the shed-in-path half of the availability story. F7 already made a
pool timeout a retryable 503 by letting the sqlx downcast through; this adds
the operator-facing half, with the pool's own size/idle counters so an
incident can distinguish "the pool is full" from "the database is gone"
without reproducing it.

Deliberately NOT a readiness probe. /ready gates Fly routing with no
fail-open, so failing it on a saturated pool would pull this node's READS
out of service too and push its write load onto peers carrying the same
load. That is the downward spiral AWS's health-check guidance names and the
SRE Book documents independently. A lock-pool readiness probe would also add
nothing on the reachability axis, because both pools are built from the same
database_url, so the existing app-pool ping already answers it.

The error is still returned via .context() rather than replaced, because
anyhow preserves downcastability through context layers and the 503 mapping
depends on it.

Cost is bounded by construction: one line per failed acquire, and a failed
acquire has already paid a multi-second pool timeout.

Refs #279

* fix(node): refuse a write when object storage is unknowable, and shed contention as 503

Four defects a seven-reviewer pass found in the previous two commits, three of
them cases where one arm of a match contradicted its neighbour.

The under-lock refresh read a failed Tigris HEAD as "no archive"
(`exists().await.unwrap_or(false)`), skipped the refresh, and wrote against a
possibly-stale tree, then re-uploaded over it. That is the outcome the timeout
arm twenty lines below explicitly refuses. A failed HEAD and a failed download
leave us knowing different things, so they no longer share a branch: only a
download failure after a successful HEAD establishes that the local copy is a
sound thing to fall back to and re-upload. A HEAD failure now refuses the write.

Lock contention that ran out the acquire deadline surfaced as a 500 carrying the
owner slug and repo name in the client-visible body, while pool exhaustion
fifteen lines up returned a deliberate 503. Contention is transient and ordinary,
so it now maps through a typed `RepoBusy` to a retryable 503 with a fixed body;
the detail stays in the log at the raise site.

`lock_not_taken` was assigned after the try-lock answered, so an error left a
previous `true`-derived value standing and could return a lock-holding session to
the pool. It is now cleared before the statement is sent.

`leak()`'s no-panic property in the guard's no-runtime Drop branch depended on
`min_connections == 0` without saying so; a future tuning change would have
silently re-armed a panic inside Drop. Made explicit with the reason.

Two tests that did not bind:

  - `waiter_on_one_repo_does_not_block_another` proved only that two lock keys do
    not collide. It now samples the pool counters across more than two backoff
    cycles. RED at 0/50 samples with `drop(probe)` moved after the sleep, the
    exact defect it names, which the previous version survived.
  - the exhaustion test's `Ok(Err(_)) | Err(_)` was satisfied by its own outer
    timeout. It now requires `PoolTimedOut` and asserts the slots are accounted
    for. RED with the pool sized N+1.

`contended_acquire_sheds_as_repo_busy_not_internal_error` is new and drives the
deadline path for real; the deadline became a field so it does not wait 90s. RED
at 500-vs-503 with the downcast removed.

The acquire deadline's docstring claimed a total under the 120s proxy idle
timeout, which the 300s under-lock transfer bound in the same function
contradicts. It bounds the wait only, and now says so. The backoff is clamped to
the remaining budget.

* fix(node): re-authorize close_issue under the guard instead of only re-checking existence

The guarded re-read matched `Ok(Some(_))` and discarded the blob, so the
authorization decision rested entirely on the pre-lock read. A comment above
claimed the opposite. That is not a narrow window: `acquire_write` re-downloads
the archive after locking, so the tree that gets mutated is routinely not the one
the author was read from. With owner-push enforcement defaulting to false and
branch protection covering only `refs/heads/*`, `refs/gitlawb/issues/*` is
pushable, so a forged author blob landing between the two reads was honored.

The blob is already in hand under the guard, so re-asserting owner-or-author
costs a deserialize. A non-owner whose issue has vanished gets 403 rather than
404, matching the pre-check's existing refusal to reveal existence.

`owner_can_still_close_after_the_reorder` seeded the owner as their own issue's
author, which made it unable to fail: with the owner check disabled the author
fallback granted the close and the test stayed green. It now seeds a third party,
so only the owner arm can grant. RED with `is_owner = false`.

Also drops the claim that the author twin covers the acquire-vs-acquire_fresh
distinction. `RepoStore::for_testing` hardcodes `tigris: None`, so the two calls
are identical in every test here and reverting that line leaves the twin green.
Separating them needs an object-storage seam. Stating the gap beats asserting
coverage that does not exist.

* style(node): rustfmt the new contention test

Whitespace only; cargo fmt --check gates the push.

* fix(node): shed an unrefreshable write as a retryable 503 with a fixed body

The two refusal arms added for the under-lock transfer bound returned bare
anyhow errors, so AppError's From impl (which downcasts only sqlx::Error and
RepoBusy) landed them in Internal: a 500 internal_error whose body was the
error string. Two defects in one. A transient object-storage failure told the
client not to retry, and the timeout arm's message interpolated the owner slug
and repo name straight into the response body, contradicting the fixed-body
policy the RepoBusy arm sets six lines above it.

RepoUnavailable follows RepoBusy exactly: a fieldless type raised with the
operator detail in a context string, downcast to its own rung, mapped to a 503
whose body interpolates nothing. The detail stays in the log at the raise site.

The timeout arm logs at error! where its sibling logs at warn!, deliberately: a
300s stall pinned a lock-pool slot for five minutes and is the condition the
transfer bound exists to surface, so it must keep paging when the handler
classifier demotes the ordinary blip case.

* fix(node): refuse a fresh-copy write when the archive HEAD cannot be read

acquire_fresh collapsed a failed HEAD into "no archive exists" via
unwrap_or(false) and served the local copy, while the sibling path under the
lock already refused on the same condition. A push that hit a storage blip got
an advertisement built from a possibly-stale tree, uploaded a whole pack, and
was then refused by acquire_write for the reason the advertisement had already
swallowed. The authorship pre-check on close_issue read the same way, which is
an infrastructure failure resolving toward a denial.

A failed HEAD now raises RepoUnavailable, so both callers surface it as the
retryable 503 rather than a stale success. The download-failure fallback is
unchanged: a present-but-unreadable archive is still self-healed from local.

info_refs keeps its map_err for every other failure so the read path's error
vocabulary does not move; only the typed error takes the From chain.

A test-only TigrisClient constructor pointed at a closed port makes both
refusals executable, so this is no longer verified by reading. It also proves
the under-lock arm end to end, which the earlier draft had recorded as an
untestable gap.

* fix(node): log expected transient acquire failures at warn, not error

Both acquire call sites logged every failure at error, including RepoBusy,
which the raise site already logs at warn. Ordinary write contention paged.
The previous commit widened the problem: info_refs now raises RepoUnavailable
on a storage blip, so that site would have started paging on the condition
this series just classified as transient and retryable.

A classifier over the two typed refusals picks the level at both sites,
mirroring the startup path's permanent-vs-transient split. Anything it cannot
classify still logs at error, so an unknown failure keeps paging.

* fix(node): log the read failure the close_issue pre-check folds into a denial

The authorship pre-check treated a get_issue I/O error and a genuinely absent
issue as the same None, with no log line. The client answer is deliberately
identical, since a caller who cannot write must not learn whether the issue
exists, but the two are not the same event and an operator had no way to tell
a real authorization denial from a filesystem or parse failure behind it.

Splitting the arm leaves the 403 exactly where it was and makes the read
failure visible in the log.

* test(node): poll for the closed session instead of sleeping a fixed 300ms

The release-invariant test slept 300ms for a Drop-spawned close before
comparing backend pids, which is a coin flip on a loaded CI runner. It now
polls pg_stat_activity for the captured pid on a standalone connection, the
same discipline poll_until_free documents: a pooled observer would be handed
the session under measurement and hide the effect.

The conversion was checked against the failure it exists to catch rather than
assumed. Pooling the session on an unlock that returned false makes the test
fail after the poll deadline, not hang, and no_reap_pool disables idle timeout
and max lifetime so nothing but the close under test can retire that backend.
A generous deadline would have been the same defect as the sleep.

* fix(node): raise a failed fresh download with no local copy as RepoUnavailable

acquire_fresh already refused a failed Tigris HEAD as RepoUnavailable, so the
handler layer mapped it to a retryable 503. A failed GET on the same path still
returned a bare anyhow error, which the info/refs map_err closure stringified to
AppError::Git and answered 500. One endpoint therefore told the client a
transient object-storage blip was permanent or retryable depending on which call
failed, and close_issue's pre-check inherited the same split through its bare ?.

Raise it at the source instead of at each consumer: From<anyhow::Error> for
AppError already downcasts RepoUnavailable out of the context chain, so both
callers pick up the retryable mapping without touching either.

The new test drives HEAD 200 with GET 500, which is the exact state the refusal
is for: archive present per HEAD, GET failed, no local copy to fall back on.

* test(node): assert the second writer sheds as RepoBusy instead of timing out

two_writers_on_the_same_repo_are_not_both_admitted wrapped the second
acquire_write in an 8-second outer timeout and only checked the future had not
finished. That passes for any stall, including lock-pool saturation or a slow
CI box, so it could not tell a working shed from an unrelated hang, and it cost
8 seconds on every suite run.

Use with_lock_acquire_deadline and assert the typed RepoBusy downcast while the
first guard is still held, matching what contended_acquire_sheds_as_repo_busy_
not_internal_error already does. It now also fails loudly if a second writer is
admitted, which the timeout version could not distinguish.

* fix(node): read the close_issue author pre-check from a non-mutating snapshot

The non-owner author fallback refreshed through acquire_fresh, which publishes
into the live repo directory: its extract step removes the existing directory
and renames the new one into place. That runs with no write lock held, so any
signed non-owner could trigger a directory swap underneath an in-flight
guarded write on the same path.

read_snapshot downloads to a throwaway temp dir and hands back a RepoSnapshot
that removes it on drop, so the pre-check still sees fresh data and the live
path is never touched. download_to grows a publish flag to serve both shapes
from one path.

The wedge invariant still holds: a stranger is refused without waiting on the
write lock.

* fix(node): bound every await in the lock-acquire loop by the deadline

The remaining budget was checked before the pool checkout but bounded neither
the checkout nor the pg_try_advisory_lock query that follows it. A checkout
starting just under the deadline could wait out the pool's own acquire timeout,
and a slow query could be accepted after the budget was spent, so the advertised
wall-clock cap held only on the fast path.

Both awaits now run under the remaining budget and shed as RepoBusy when it runs
out. The probe's Drop closes its session, which cannot hold a lock it never
confirmed taking, so the query-timeout arm is a plain shed.

* test(node): add an S3 mock that really enforces conditional writes

The fence work landing next is only as good as what the tests can observe, and
a mock that answers 200 to every PUT would make the whole suite vacuous. This
one holds the object and its ETag, refuses a mismatched If-Match and an
If-None-Match "*" over an existing object with 412, and mints a fresh ETag per
successful PUT so two byte-identical archives never share a token.

Capture-then-replay is deliberate rather than parking a handler and hoping it
resumes: when tokio drops an SDK future the client can tear the connection down
and cancel the server task with it. Replaying what arrived models the arm that
matters (body fully transmitted, commit decided later) with no timing in it.

Six tests pin the semantics in both directions so a hollowed mock cannot hide.

* fix(node): stop pretending a held lock fences a late upload

The timeout arm claimed that keeping the advisory lock protected a successor
from an abandoned PUT. It does not. release takes mut self, so the guard drops
the moment it returns and Drop closes the session; measured on this branch, a
successor took the same repo's lock 5ms after release returned while the PUT
was still in flight.

The comment and warn now say what is actually true: the outcome is unknowable,
the PUT may still land, the lock releases normally, and a conditional upload is
what keeps a late publish from overwriting a successor's archive.

Two tests replace the claim. Session disposition is the observable that
separates the two shapes, so the unlock is pinned to run and be confirmed on the
guard's own session with the connection returned to the pool, checked by backend
pid. Successor admission is pinned too, but noted as not what proves the point,
since the lock frees within milliseconds either way.

* feat(node): let the storage client express and detect a conditional write

upload takes an UploadPrecondition (IfMatch, IfAbsent, Unconditional) rather
than an Option<String>, so the absent case and the deliberate no-fence case are
distinguishable at the type level and a caller cannot lose the fence by passing
None. head_etag reads the current ETag alongside exists, which keeps exists and
its other callers untouched.

A failed precondition has to be classified off the raw HTTP status: PutObjectError
models no PreconditionFailed variant, so a 412 arrives as Unhandled with nothing
useful on it. 412 is always a lost precondition and 409 is one under IfAbsent.
404 deliberately is not: no archive delete exists on this line, so a 404 on a
conditional PUT means a wrong bucket or endpoint, and reporting that as retryable
would send clients into a loop against a permanent fault.

The three background uploads outside the write guard now publish with IfAbsent.
They fire only where the archive is expected absent, and leaving them
unconditional would defeat the fence from the side: init uploads an empty bare
repo, so a push landing just before it could have its archive replaced by that
empty one. A refusal there means someone else already published the key, which is
logged as the correct outcome rather than a failure.

* fix(node): fence the release publish so an abandoned upload cannot win

acquire_write now reads the archive's ETag under the advisory lock and the guard
carries it, so release publishes conditionally on the generation it actually
refreshed from. An upload abandoned by an earlier writer no longer overwrites a
successor's archive: the store rejects it, because the ETag it was written
against is gone.

A refused precondition gets exactly one supersede-retry, never a loop. The
distinction that makes this sound is that a 412 is a definite answer, unlike the
timeout arm where nothing is knowable, and the retrying writer still holds the
lock, so whatever landed underneath was written without one and its tree is not
the authority. Two losses in a row refuse instead of escalating.

That retry is what keeps ordinary pushes working now that init publishes
create-only: a first push racing init's upload of the empty repo loses once and
then wins, rather than surfacing a 503 on the most common operation there is.

release returns a must-use outcome and the four publishing handlers propagate it
before any trust bump, webhook, or success body, so a publish the store refused
reads as a retryable 503 instead of a 201. The three release(false) sites
deliberately do not map it: they publish nothing, and a 503 there would shadow
the 403 or 404 the route means to return.

The download-failure fallback also publishes fenced now. That arm knows the
stored generation (its HEAD succeeded, only the GET failed), so publishing
unconditionally from it would reintroduce the same overwrite.

* test(node): drive the abandoned-upload race end to end, and probe the real backend

The headline test is the one that had to exist. Writer A's release is parked past
its transfer bound and returns with the outcome unknowable, B acquires and
publishes, and A's captured PUT is then replayed: the store answers 412 and B's
archive survives. The create-only variant covers the arm whose real-world failure
is silent rather than loud.

The control is what makes those attributable. With no interleaved B, an
abandoned-then-replayed PUT whose generation still matches lands. Without it the
headline would only show that replays get rejected, not that staleness is what
rejects them.

Header assertions come last in all three, so a lost fence reds on the outcome it
is about rather than on a wire-format check.

A mock cannot prove Tigris honors any of this, and the vendor requires a
Single-region or Multi-region bucket for conditional operations, so against a
Global or Dual-region bucket the fence is a silent no-op. The credentials-gated
probe checks both arms against the real endpoint and cleans up unconditionally,
including when an assertion fails, which is the case it exists to catch. It
accepts 412 or 409 on the create-only arm because both mean the precondition was
enforced and both are already classified as a loss.

* fix(node): reconcile F-series tests and test seams with the rebased guard shape

The rebase onto main carried main's #174 F-series guard tests forward. The
branch's guard replaces main's locked/released/test_pre_unlock_gate Drop
mechanics with close-on-drop (runtime) and leak (off-runtime), so two tests
that asserted the replaced mechanics are reconciled: the off-runtime disposal
test now asserts the leak observable (the slot never returns to idle) and the
detached-unlock-returns-connection test is dropped, its invariant covered by
the U-series drop-frees-the-lock gates. The pre-unlock gate seam is restored
(test-only) so the mid-unlock cancellation tests keep their deterministic
park. ipfs.rs tests are updated to the branch's sync test-client constructor
and 4-arg RepoStore::new.

Refs #279

* fix(node): address the 2026-08-10 review round on the advisory-lock series

- P1: serve the receive-pack advertisement from a non-mutating snapshot
  instead of acquire_fresh, so the unlocked advertisement cannot delete or
  swap the live repo directory under a concurrent guarded write (the stale-ETag
  success ordering). acquire_fresh loses its only production caller and is
  removed.
- P1: read-gate, rate-limit, bound, and cancellation-clean the close_issue
  pre-lock snapshot, so a signed non-owner cannot drive unbounded Tigris
  downloads and blocking extractions, and an abandoned extraction no longer
  leaks its temp dir.
- P1: classify raw 409/412 responses as a lost conditional write via
  SdkError::raw_response() in both upload and the publish supersede-retry, so
  an unparsable error body cannot acknowledge a write that was never published.
- P2: wrap the cold-cache under-lock download failure in RepoUnavailable so it
  sheds as a retryable 503 like the HEAD arm, not a permanent 500.
- P2: surface a refused create-only fork upload as PreconditionLost and refuse
  the fork, so an orphan archive cannot shadow a fork's DB record.

Refs #279

* fix(node): record why the close-issue read gate cannot carry a synced row

The gate added for the pre-lock snapshot reads a repo row's own visibility
rules and public flag. A row synced from a peer is stored public and carries
none of the owner's rules, so for that class the gate can only return allow.

Refusing instead would deny the repo's real owner and the issue's real author
on any node whose only copy is a synced one, and it would not buy the
protection it appears to, because a synced row's recorded owner comes from the
peer that sent it. Every other read gate in the API reaches the same verdict
for such a row, so this is left as is and stated at the call site rather than
special-cased here.

The test that covered the gate seeds a locally created repo and cannot observe
this. Adds one that pins the property directly: the gate allows an arbitrary
caller on a synced row, and the handler refuses that caller anyway, because the
owner-or-author check is what decides this route.

* fix(node): address jatmn review on merged advisory-lock head

Align .env.example db pool with validate floor, move snapshot cleanup into
the blocking extract task, remove fork clones on PreconditionLost, restore
upload_site_reached test plumbing, and use owner DIDs in receive-pack tests
after #330 owner-push default.

* fix(node): cap close_issue snapshot work on the read pool

Take a git_read_semaphore permit before the non-owner author snapshot so
parallel close attempts cannot each drive a full archive extraction while still
under the hourly rate bucket. Format owner-DID test helpers.

* fix(node): mark close_issue mirror visibility gate for push hook

Annotate list_visibility_rules with mirror-rows-handled so the pre-push
surface detector accepts the synced-row read gate.

* fix: unblock pre-push after main merge

Read CappedBody.text in gl whoami (#381 shape), restore legacy CID sweep
wiring dropped in the merge, route rate-limiter cleanup through AppState,
and allow build_lock_pool for test-only callers.

* fix(node): carry transient acquire failures through every write handler

Extend acquire_write_app_error so RepoBusy and RepoUnavailable map to
fixed-body 503s on issue and pull paths, matching receive-pack. Fix the
flaky u2 drain exhaustion test via a drain_faults seam and add handler
regressions for contention and storage unavailability on create_issue.

* fix(node): drop unused logcap helper after drain_faults seam

* fix(node): route fork disk paths through validated_repo_disk_path

CodeQL flagged remove_dir_all on a user-derived path in fork_repo. Use the
same three-layer barrier as RepoStore::local_path and reject names the
validator refuses before clone.

* fix(node): harden release-upload outcomes and replication tail gating

Treat timed-out or failed archive uploads as non-durable in into_result,
gate pinata/gossip on confirmed publish durability while keeping inv22 spawn
order, split lock-pool vs main DB pool validation, map LockPoolBusy through
From<anyhow::Error>, and attach Retry-After on repo write contention errors.

* fix(node): close jatmn round on publish authority and durability gating

Revoke under-lock refresh swaps when the advisory lock ends, fail closed on
unrecorded release outcomes before Pinata/gossip, compensate fork archives when
create_repo fails, recompute acquire backoff from the live deadline, and route
publish swaps through validated_repo_disk_path for CodeQL.

* fix(node): harden issue close/create and fork publish paths on #285

Move close_issue rate limiting after the read gate so denied readers see 404
not 429, roll back local issue refs when publish refuses, re-check swap
authority under publish_lock, propagate fork upload failures before DB insert,
and update lock-pool config help text.

* fix(node): close write-attempt ownership gaps on advisory-lock release

Make publish swaps claim-exclusive via CAS, invalidate the local read cache
on definite publish refusals, harden fork create compensation and clone
cleanup, and keep snapshot temp dirs owned through the async handoff.

* fix(node): gate replication tail on publish durability and split close_issue limits

Wait for a confirmed release outcome before post_receive_replication_tail runs
any walks or coalescing work. Give close_issue its own per-IP rate bucket so
it cannot drain receive-pack quota. Add regression tests for both paths.

* fix(node): add ValidatedRepoDiskPath for CodeQL and reconcile tail durability

Introduce a validated-path newtype with the same inline join and component
walk main uses, so path-injection sinks only accept sanitised repo paths.
Reconcile the publish-durability gate with the disconnect-safe tail spawn via
PublishDurabilitySlot and explicit UploadUnknowable handling.

* fix(node): close three jatmn findings on advisory-lock PR

Retry fork create confirmation before compensating or returning, schedule
background recovery when the lookup stays unavailable, install publish
durability outcomes before the recorded flag, and size the lock pool for
non-push mutations (default 40).

* fix(node): verify adversarial findings on advisory-lock PR

Fail closed when publish durability never records, bound pg_advisory_unlock
with the same transfer budget as upload, and cap close_issue pre-lock snapshots
at git_acquire_timeout_secs instead of the write-lock transfer bound.

* fix(node): compensate under the write lock and harden tail durability handoff

Run create_issue rollback through release_compensating so issue refs are
deleted only while the advisory lock is still held and only on definite
publish refusals, not UploadUnknowable. Replace the tail durability slot's
async try_lock drop with a std mutex so cancellation during polling cannot
skip installing UploadUnknowable.

* fix(node): refuse stale under-lock refresh and gate tail durability on release

Under-lock GET failures now always shed as RepoUnavailable, even when a local
copy exists, so a transient download error cannot publish over a newer stored
generation. PublishDurabilitySlot only synthesizes UploadUnknowable after
release starts, and receive-pack marks that boundary before spawning the tail.

Adds regression tests for both paths; inv22 F4 gate accepts the release wrapper.

* style(node): rustfmt advisory-lock gap-closure commits

* fix(node): satisfy clippy on stale-local download test path join

* feat(node): add a publish attempt/stage boundary

The four lifecycle findings on this branch share one root cause: a resource
was tracked by a coarse state ("release started", "the path exists", "a row
with this name exists") when the safety question was about the IDENTITY and
the PUBLICATION STAGE of one specific write attempt.

This is the vocabulary that makes those questions answerable:

- `PublishAttemptId` — minted before the request is built, carried with the
  bytes as object user metadata, read back off the store to decide whether
  what is published is THIS attempt's work. That turns "did my request
  succeed", which a lost response makes undecidable, into "are the published
  bytes mine", which the store can answer.
- `PublishStage` / `PublishStageCell` — how far one attempt got, observable
  from OUTSIDE the future doing the work. A cancelled handler never returns
  an outcome, so the stage it had reached is the only thing that can classify
  it.
- `UploadError` split by what a failure ENTITLES A CALLER TO DO rather than
  by which HTTP status came back. Destructive compensation is licensed only
  by `proves_not_published()`.

Deliberately free of any object-storage type. #79 deletes `git/tigris.rs` and
replaces it with a BlobStore layer; this module is what survives that swap,
with each backend supplying only its own error classifier.

* feat(node): stamp uploads with an attempt id and preserve dispatch ambiguity

`upload` now reports its progress into a `PublishStageCell` and stamps every
PUT with an attempt id in object user metadata, so both of the questions a
cancelled or unanswered publish raises become answerable:

- `PreparingArchive` is marked before the blocking compression and
  `PutDispatched` immediately before `send()`. The conditional PUT is not
  constructed until compression returns, so a caller abandoned in that window
  definitely never attempted publication — a fact nothing could previously
  observe, because the only report was the return value of a future that no
  longer existed.
- `attempt_landed()` HEADs the key and compares the stored attempt id, which
  is what lets a client whose response was lost recover its own committed
  write instead of guessing.

`UploadError::Other` is replaced by `NotPublished` (proven never committed)
and `Ambiguous` (may have been dispatched and may have committed). The single
backend-aware classifier is `classify_put_failure`: a 4xx is an answer the
server gave before storing anything and proves non-publication; a 5xx does
not, and neither does a timeout, a dispatch failure, or a response the SDK
could not read. `SdkError` is `#[non_exhaustive]`, so the fallback arm is the
cautious one.

`delete` is replaced by `delete_if_attempt_matches`, which reads the attempt
off the object and fences the DELETE with `If-Match` on the generation it
came from. A second name lookup before an unconditional delete would only
narrow the window in which a successor's object can be erased; the
conditional delete closes it.

* fix(node): key durability, cache validity and fork cleanup on the attempt

Closes the four lifecycle findings by consuming the attempt/stage boundary at
the sites that were deciding on coarse state.

P1 — cancellation before PUT dispatch is not publish durability.
`PublishDurabilitySlot` is armed on the guard's publish STAGE rather than on
a "release started" flag, and its `Drop` classifies an abandoned handler by
how far the attempt actually got: a cancellation during compression records a
definite refusal, a cancellation after dispatch records unknowable, and a
cancellation after the store acknowledged records `Released` (which the flag
could never say). `publish_durability_confirmed` now requires `Released`, so
the detached tail no longer does IPFS/Pinata/P2P/GraphQL/Arweave/peer work
for refs that exist only in a rejected local write. A guard with no storage
backend seeds `NoBackend`, keeping Tigris-less deployments' tails running.

P1 — an unknowable generation is quarantined, not served as ordinary cache.
`release` writes a sidecar marker naming the unresolved attempt beside the
live tree (never inside it: it would be tarred into the next archive), and
`acquire`'s existing-path fast path reconciles it before serving. Only the
store confirming it holds that attempt lifts the quarantine; anything else is
a retryable `RepoUnavailable`. The tree is deliberately NOT deleted — the PUT
may have landed and this can be the only local copy. A confirmed publish, an
under-lock refresh, and cache invalidation each clear the marker, so the
quarantine is a bounded refusal rather than a standing outage.

The release-side timeout arm also stops treating every stall alike: a bound
that expires at `PreparingArchive` is a definite non-publication, and a bound
that expires after dispatch spends one short, separately bounded HEAD trying
to reconcile the attempt before falling back to unknowable.

P1 — fork confirmation and cleanup bind to the creating attempt. The DB row
id is minted up front and used as the attempt id, so the object's metadata,
the clone's sidecar stamp and the row all name one attempt.
`confirm_fork_repo_row` looks the row up BY ID and reports Ours / Foreign /
Absent, so a concurrent create, mirror registration or retry can no longer be
returned as this request's own commit. Every destructive step —
`compensate_fork_archive`, its background retries, and `ForkCloneGuard::drop`
— is conditional on the resource still belonging to this attempt, so recovery
that observed `None` before a successor committed can no longer erase that
successor's archive or directory afterwards.

P2 — response-loss ambiguity is preserved rather than compensated. Guarded
writes map an ambiguous publish to `UploadUnknowable` + quarantine instead of
`UploadFailed` + cache invalidation + the caller's undo, so `create_issue` no
longer deletes an issue ref whose archive is durable. Fork creation
reconciles by attempt id: a create-only PUT that committed and lost its
response is RECOVERED and the row inserted, and an unresolved one keeps its
only local clone and refuses retryably. The fork name is no longer fenced
behind the attempt's own orphan.

Tests, each RED-checked by reverting its guard:

- slot_drop_during_compression_records_a_definite_non_publication
- slot_drop_after_dispatch_records_unknowable
- slot_drop_after_the_store_acknowledged_records_released
- slot_drop_with_no_storage_backend_records_released
- publish_durability_confirmed_accepts_only_released
- publish_durability_confirmed_refuses_quickly_after_unrecorded_slot_drop
- receive_pack_cancelled_during_compression_publishes_nothing_and_runs_no_tail
  (+ receive_pack_that_completes_its_publish_still_runs_the_tail as control)
- a_bound_that_expires_before_dispatch_is_a_definite_failure
- an_unresolved_publish_quarantines_the_tree_and_a_later_read_refuses
- a_quarantined_tree_is_served_once_the_store_confirms_the_attempt
- the_next_write_clears_an_inherited_quarantine
- a_confirmed_publish_leaves_the_tree_readable (control)
- a_guarded_write_whose_response_is_lost_is_not_compensated
- a_put_that_commits_and_loses_its_response_is_ambiguous_and_reconcilable
- a_closed_response_with_no_http_status_is_ambiguous
- upload_classifies_404_as_a_definite_non_publication
- upload_classifies_500_as_ambiguous_not_definite_failure
- a_generation_that_moves_between_the_head_and_the_delete_is_not_deleted
- an_attempts_own_object_is_deleted_and_a_foreign_one_is_not
- fork_confirmation_never_claims_a_concurrent_attempts_row
- fork_confirmation_recognizes_this_attempts_own_committed_row
- fork_confirmation_reports_absent_when_nothing_owns_the_name
- fork_clone_guard_leaves_a_successors_mirror_alone
- fork_recovery_resuming_after_a_successor_deletes_neither_object_nor_path
- fork_compensation_removes_what_this_attempt_still_owns
- fork_publish_that_loses_its_response_stays_recoverable
- a_fork_whose_publish_lost_its_response_is_recovered_not_fenced
- a_fork_whose_publish_is_unresolved_keeps_its_clone_and_refuses_retryably
- an_ordinary_fork_publishes_and_commits (control)
- mock_round_trips_the_attempt_metadata_a_put_stamped

The S3 mock gains attempt metadata, conditional DELETE, a commit-then-lose-
the-response mode, a delivered-but-not-committed mode, and a per-key object
store (fork creation touches the source and fork keys in one request, and a
single slot would have let an assertion about one silently read the other).
The compression seam is a condvar gate so no test holds a guard across an
await.

* fix(node): settle the cache contract on the abandoned path and the attempt

The publication boundary this branch introduced was applied at call sites
that each had to be remembered, and five were not. Close the eight gaps
that survived, grouped by what actually failed rather than by symptom.

The marker's lifecycle was answered by the wrong event in three places.
The swap that replaces a live tree now owns the clear, so a cache-miss
download no longer leaves a stale marker that fences the archive it just
installed, and the under-lock refresh no longer lifts a quarantine on a
HEAD that downloaded nothing. init clears a predecessor's marker.

A cancelled handler never reached release, so it applied the tail
contract and not the cache contract. RepoWriteGuard::drop now settles the
tree it abandoned, keyed on how far the publish actually got, and a guard
that handed out its path for writing marks the tree whatever stage it
reached. A marker naming no attempt is a definite non-publication rather
than a permanent refusal: it invalidates and the next read serves the
stored generation, so an abandoned first push cannot wedge a repo at 503.
Only an unreadable sidecar still fails shut.

Both of the store's live-path hand-outs now go through one confirming
helper, so read_snapshot can no longer return an unconfirmed tree as an
ordinary snapshot. The gate binds ahead of the lazy-migration spawn,
because a quarantined tree backfilled to object storage by an IfAbsent
PUT would publish exactly the refs the marker exists to withhold. The
marker no longer rests on a single fs::write; an unwritable sidecar falls
back to an in-memory record that reads consult.

Fork cleanup claims ownership by renaming the stamp before it touches the
tree, so a successor that claims the same path cannot lose its directory
to a predecessor's check-then-unlink, and the directory-absent path no
longer unlinks a stamp it does not own. A create-only precondition loss
whose reconciliation HEAD fails is refused retryably instead of reported
as a permanent name conflict, and its clone is released so the retry can
re-clone rather than finding the destination occupied.

The pin sweep skips a tree whose publish is unresolved, so provider CIDs
are not rewritten from refs that may never become durable. That is the
sidecar marker, distinct from the operator-level quarantined row flag.

26 tests, each written and observed failing before its fix. Suite 1211
passed, 0 failed, 1 ignored; fmt, clippy -D warnings and --locked clean.

* fix(node): settle the abandoned tree at the disconnect, not at reaper exit

The settlement that marks an abandoned write ran from RepoWriteGuard's
Drop, and on a client disconnect that guard is held by the Arc clone
riding the admission guard into the detached reaper, so it ran only once
the process group had been reaped. Reads take no advisory lock and
consult only the sidecar, so for the width of that window a concurrent
fetch was served the abandoned refs off the live tree with no marker
beside it. Confirmed by execution before the fix: no marker on disk,
acquire returning the live path, and the abandoned ref still on the tree
it served.

Only the lock release has to wait for the reaper; the settlement does
not. TreeSettlement is a token carrying the path, the publish stage and
the hand-out flag, holding the exhaustive stage match that used to live
in Drop. The handler takes it out of the guard before the guard is
shared with the reaper, so on a disconnect the handler future drops
first and the marker lands at the disconnect instant while the reaper
still holds the lock. A successful release disarms it. Drop still
settles a guard nobody took the token from, so the arms have one home.

The regression test reads inside the window and asserts the abandoned
refs are not served. It samples pg_locks from an independent session
first, so a run where the reaper had already finished reports itself
inconclusive rather than passing for the wrong reason.

Suite 1212 passed, 0 failed, 1 ignored. The five settlement mutations
are load-bearing, including one that reverts this split and reddens the
window test.

* fix(node): let exactly one classifier settle an abandoned tree

Moving settlement into a token the handler holds left two classifiers
able to run for one write. release() classifies the tree and only then
awaits the unlock, while the handler disarms the token after that await
returns, so a cancel in the unlock gap dropped a still-armed token and
settled a second time. After a definite refusal that is a quarantine
marker written beside the tree release had just deleted, which sends
every later read down the refuse-and-reconcile path for a repo whose
state was already resolved. Observed before the fix: the tree gone, the
marker there.

The arm is now an AtomicBool shared between the guard and its token.
release stores false at the classification point, above the unlock
await, and settle takes the arm with compare_exchange, so whichever
classifier gets there first is the only one that acts.

tree_settled stays, and now has a test that says why: after the handler
takes the settlement the arm is still true, so tree_settled is the only
thing stopping the guard's own Drop, which on a disconnect runs in the
detached reaper, from settling again at reaper exit.

The reap-window test was passing for weaker reasons than it claimed. Its
fake git took SIGTERM and let the group die, so the reaper could free
the advisory lock before the probe read; it now traps TERM and re-parks
so only SIGKILL ends it. It asserts the marker exists at the disconnect
instant rather than only mentioning it, and the served-refs half no
longer passes when acquire fails for an unrelated reason.

Suite 1214 passed, 0 failed, 1 ignored. Nineteen mutations load-bearing,
including one that reverts the release-side disarm and one that reverts
the settlement split.

* test(node): use the test's own pool now that Db::pool is gone

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-04 21:38:41 +00:00
beardthelion 19a7ce2bbd fix(node): drop withheld-subtree trees from the replication pin set (#382)
* fix(node): drop withheld-subtree trees from the replication pin set

A tree reachable only under a denied path survived both pin filters because
trees were never in the blob universe, so public IPFS still learned child
filenames and blob oids. Filter trees the same way as blobs on the delta
withheld set and the full-scan allow-list.

Closes #172.

* chore: re-trigger code quality check
2026-11-04 20:54:43 +00:00
beardthelion 189aad96e5 fix(node): require Content-Digest on a signed request (#306)
* fix(node): require Content-Digest on a signed request

require_signature rebuilds the RFC 9421 signing string from the request
itself, and it read the Content-Digest header with a .unwrap_or("")
fallback. When the header was absent, the covered content-digest component
came out as the empty string on both the signing side and the verifying
side, so the Ed25519 check passed over a string that committed to nothing
about the body. The comparison that would have caught it was guarded on the
presence of that same absent header, so it was skipped. The two defects
compose: a signed request could carry any body at all, and the handler
behind the middleware saw it as authenticated.

The header is now required. An absent Content-Digest is a 400 with
missing_content_digest, and the digest-versus-body comparison is
unconditional, since presence is established before it runs. There is no
longer a path through the middleware on which the body goes uncompared.

Refusing is safe for every client that already works. sign_request computes
the digest with no branch and always returns it in SignedHeaders, and all
eight production call sites attach it as a Content-Digest header: the three
in gl's HTTP client, the fetch and push requests in git-remote-gitlawb, the
node's peer announce, the sync push in sync.rs, and the sync notify in
api/repos.rs. So no conforming client regresses. What the refusal removes is
the case where a caller signed an empty digest and sent a body no signature
covered.

HttpSignature::parse now also refuses a covered component that appears more
than once, per RFC 9421 section 2.1. A repeated identifier says nothing the
single one did not, and it lengthens the signing string a verifier has to
build from a list the caller chooses.

* fix(core): linear duplicate check and reject component parameters

Use a HashSet for covered-component duplicate detection so hostile
Signature-Input headers cannot force quadratic work before rejection.
Reject RFC 9421 component parameters during parse since only bare names
are supported end to end.

* chore: re-trigger code quality check
2026-11-04 20:33:16 +00:00
beardthelion 6233bd9d85 fix(node): validate git branch refs to close option injection (#379)
* fix(node): validate git branch refs to close option injection

PR branch refs and a repo's default_branch were stored from request bodies
with no ref validation, then interpolated into single git argv elements:
git diff {target}...{source} (branch_diff / branch_diff_names), and
git worktree add ... {target} / git merge {source} (merge_branch). A value
beginning with '-', e.g. --output=/tmp/x, is parsed by git as an option
rather than a revision, so it becomes an arbitrary file write. get_pr_diff
takes an optional identity, so on a public repo the trigger is unauthenticated;
planting the PR needs only read access, and the write happens at the withhold
check before the visibility gate.

Defense is applied at two layers:
- Storage boundaries: create_pr validates source_branch and the resolved
  target_branch; create_repo validates default_branch (which becomes a PR's
  target when the PR omits one). These fail fast with 400 and keep junk out
  of the DB.
- The sink: branch_diff, branch_diff_names, and merge_branch reject an
  option-shaped ref before building the git argv, so the property holds for
  every caller and every row, including legacy rows and any future writer
  that skips the boundary check.

The shared validator is crate::git::store::validate_git_ref (git
check-ref-format rules, leading-dash rejection as the core), re-exported as
crate::api::validate_git_ref for the boundary handlers. No -- delimiter is
used: the arguments are revisions, and -- there reinterprets them as
pathspecs.

Both boundary guards and the sink guard are mutation-proven load-bearing.
resolve_head is unaffected (it prefixes refs/heads/); fork_repo takes no
branch from the request.

* fix(node): delegate branch ref validation to git check-ref-format

Rebase onto current main and tighten validate_git_ref to call
git check-ref-format --branch so symbolic names like HEAD and
trailing-dot components are rejected at storage boundaries and
git sinks. Add boundary and legacy-row tests for HEAD, feature.,
and feature/x.

* fix(node): reject fully qualified refs and close review test gaps

Reject refs/ prefixes after check-ref-format so tags and qualified
head names cannot be stored as PR branch fields. Add sink tests for
option-shaped merge source, HEAD target, and poisoned default_branch
resolution; assert get_pr_diff fails on poisoned diff rows.

* fix(node): allow-unbounded-git for check-ref-format validation

Stateless ref-name validation holds no repo concurrency permit.

* style: cargo fmt

* fix(node): reject revision shorthands and split ref-validation errors

git check-ref-format accepts @ and pseudoref names that git later treats as
symbolic revisions, so merge could succeed without merging the source branch.
Reject those explicitly at the shared validator and add sink/boundary tests.

Spawn failures from check-ref-format now surface as git_error (500) instead of
bad_request (400) at create_pr/create_repo boundaries.

* test(node): close ce-review gaps on git ref validation

Add sink tests for @{-1} and pseudoref poisoned rows at diff/merge,
HTTP tests that spawn failure returns git_error (500) not bad_request,
and expand API boundary cases for remaining pseudorefs and qualified refs.

Introduce GITLAWB_TEST_VALIDATE_GIT_BIN test seam with a mutex so parallel
sqlx tests do not race on the override.

* fix(node): prefix sink git argv with refs/heads and close shorthand gap

Pass stored branch short names to diff/merge/worktree as refs/heads/{name}
so revision shorthands cannot retarget another ref namespace at the sink.
Reject heads/, tags/, and remotes/ prefixes at storage boundaries, extend the
pseudoref denylist, and route boundary validation through AppState::git_bin
instead of a process-global test env override that raced parallel tests.

* fix(node): attach merge worktree to the target branch

git worktree add with refs/heads/{name} checks out detached HEAD, so a
successful merge did not advance refs/heads/{target}. Use the branch name
for the worktree and keep refs/heads/ qualification on merge revisions.
Add an end-to-end regression for diverged branches.

* chore: cargo fmt on merge ref-advance test

* fix(node): bind merge worktree to the exact local target branch

validate_git_ref proves branch-name grammar only; git's revision DWIM can
resolve the same short name in another namespace (refs/tags/{name}), so a
PR targeting a tag-shadowed name checked out a detached HEAD, merged onto
it, discarded the commit at cleanup, and the unchecked plain rev-parse
echoed refs/heads/{target} back as a bogus merge SHA — marking the PR
merged and firing pull_request.merged with no branch advanced.

Close the resolution class instead of denylisting names: (1) require the
exact refs/heads/{target} via show-ref --verify before creating the
worktree, (2) verify the worktree's symbolic HEAD is attached to exactly
that ref before merging, (3) accept the merge only if rev-parse --verify
{target}^{commit} exits zero. Add end-to-end regressions for a same-named
tag with no local branch (merge fails, PR stays open, no merged event) and
for a tag/branch coexistence (the local branch advances, the tag stays).

* fix(node): recover a leftover merge worktree instead of wedging the endpoint

merge_branch cleaned up a leftover _merge_worktree only when the directory
still existed. The state git worktree add actually refuses is the opposite: a
registration under $GIT_DIR/worktrees whose directory is gone. It fails with
"missing but already registered" (exit 128), and once a repo is in that state
every later merge on it fails, permanently.

The state is self-inflicted. remove_worktree runs git worktree remove and then
removes the directory unconditionally, so one failed removal leaves the
registration behind.

Two changes. Run the cleanup unconditionally at entry, which also makes the
bail-without-cleanup paths self-healing on the next merge. And pass --force
twice: a single --force refuses a LOCKED worktree ("cannot remove a locked
working tree"), and git worktree prune skips locked registrations too, so with
one flag a locked leftover survives the whole cleanup and git worktree add
then fails with "missing but locked worktree".

Enumerated the leftover states that can reach this code (registered or not, x
directory present / gone / locked / corrupt admin dir, x gc.worktreePruneExpire
default and never). remove -f -f plus the directory removal clears all of them,
and adding a separate git worktree prune changes no outcome in any of them, so
there is no prune call here.

Four tests, one per leftover shape that reaches this code. Against
merge_branch: the plain stale registration, the locked variant, and an
unregistered leftover DIRECTORY (git worktree remove refuses that one with "is
not a working tree", so the directory removal is what clears it). Plus an
end-to-end pass through the merge endpoint on the locked state, asserting the
PR row reaches "merged", the reported merge_sha IS the new refs/heads/{target},
and the merge leaves no worktree behind for the next one. The e2e fixture
asserts git really refuses the state before asserting recovery, so a future git
cannot make it vacuous.

* chore: re-trigger code quality check
2026-11-04 17:55:12 +00:00
beardthelionandKevin Codex 76ca04f355 fix(node): close three host-class gaps in the public-URL gate (#346)
* fix(node): close three host-class gaps in the public-URL gate (#339)

is_public_http_url is the shared gate for every operator-supplied outbound
URL: peer announce, webhook creation, and the boot-time peer prune. Three
host forms got past it that the checks around them plainly meant to exclude.

A name with no dot never matched the `.local` / `.internal` suffix rules, so
`http://internal/` and `http://wpad/` were accepted and left to the resolver
to complete from its search domain. The dotless rule sits on the branch where
the host did not parse as an IP literal, which keeps bracketed IPv6 (also
dotless) on the accepting path; the existing 6to4 and NAT64 accept cases pin
that.

fec0::/10 fell through the IPv6 arm because the link-local mask does not
cover it: 0xfec0 & 0xffc0 is 0xfec0, not 0xfe80. RFC 3879 deprecated the
range, which argues for rejecting it rather than ignoring it.

Trailing root dots were stripped once, so `localhost..` reduced to
`localhost.` and matched neither the equality nor the suffix check. Stripping
to fixation closes that.

Each of the three is covered by a test that reddens when its own production
line is reverted, and the dotted forms stay as negative controls so the
suffix checks remain proven.

* fix(gl): consume CappedBody text in whoami error path after main merge

* chore: re-trigger code quality check

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-04 17:20:36 +00:00
beardthelionandKevin Codex c301a68161 fix(node): persist the libp2p identity instead of deriving it from the node DID (#324)
* feat(node): add a persistent libp2p identity key file

Add p2p_key_path (--p2p-key-path / GITLAWB_P2P_KEY, default
~/.gitlawb/p2p.key) with a resolver mirroring resolved_key_path, and
load_or_create_p2p_keypair, which generates an Ed25519 keypair on first
start, persists it 0600, and loads it thereafter. Mirrors the existing
load_or_create_keypair idiom for the node identity PEM.

A corrupt or unreadable key file is a hard error naming the path rather
than a silent regeneration, so a disk problem cannot quietly rotate the
node's network identity.

Not yet wired into p2p::start; that follows.

* fix(node): load the libp2p identity from the persistent key file

p2p::start now takes the Ed25519 keypair loaded by
load_or_create_p2p_keypair instead of computing one from the node DID,
so a node's network identity is generated once from the OS RNG and kept
on disk rather than recomputed from a public value on every start.

The node DID parameter is gone from start; the call site loads the key
first and continues without p2p if the key file cannot be read, matching
how a swarm-start failure is already handled.

The gossipsub message_id_fn is untouched and keeps its own hasher.

* fix(node): create the p2p key with 0600 at creation and reject a loose one

Open the key file with create_new and the mode set at creation, then
fsync, instead of writing it and narrowing the mode afterwards. The
secret is never on disk under a wider mode, an interrupted start cannot
leave it readable, and the exclusive open also refuses a pre-existing
entry at the path and makes a concurrent start take the key that landed
rather than clobber it.

Refuse to load a key file whose mode grants group or other access, and
name the observed mode so the operator can fix it. Report an empty key
file as empty rather than surfacing a protobuf decode error that blames
a missing rsa feature.

Pin GITLAWB_P2P_KEY onto the mounted volume in the Docker and fly
configs and document it, so the key does not depend on home-directory
resolution to land on persistent storage.

* fix(node): publish the p2p key atomically and pin its directory

Write the key to a scratch file in the same directory and hard-link it
onto the final path. The bytes are durable before any name points at
them, so a crash cannot leave a partial key that fails to load on the
next start and takes the node off the network until someone reads the
logs. A concurrent reader can no longer observe a half-written file
either, since the final name appears complete or not at all.

hard_link rather than rename: rename replaces its destination silently,
so refusing to clobber an existing key would depend on a check followed
by a separate rename, and a concurrent start can land in that gap.
hard_link is atomic and refuses an occupied path, including a symlink,
which it does not follow.

Create the key directory 0700 and tighten it when an existing one grants
group or other access. A 0600 key under a writable directory can still be
replaced or unlinked. Tightening rather than refusing to start, because
existing installs already have 0755 there and refusing would take p2p
down on all of them through a path that only warns.

Formatting on the branch is swept up here; it was already failing
cargo fmt --check before this change.

* style(node): comma instead of a dash in the key-load warning

House style avoids em dashes in text we write. The swarm-failure warning
beside it predates this branch and is left alone.

* fix(node): require the p2p key path to name a directory

A bare filename in GITLAWB_P2P_KEY put the key in whatever directory the
process started from, and the directory guard was skipped entirely on that
path: Path::parent returns Some("") for a bare filename, which the caller
filtered out before ever reaching ensure_key_dir. The key file was created
0600 inside a directory that kept whatever mode it already had.

Config::validate now rejects a p2p key path that names no directory, so the
node says so at boot instead of starting with a key it cannot protect. That
placement is the point: an error raised in the p2p start path is logged and
stepped over, leaving the node running without p2p and reporting healthy.

The check is lexical on the tilde-resolved path. canonicalize would fail on a
parent that does not exist yet, which is the shipped ~/.gitlawb default and
every container's first boot, and comparing against the working directory
would reject /data/p2p.key under the image's WORKDIR, an absolute directory
the operator did name.

Three sites answered the parent question differently, which is how the gap
arose: one filtered the empty case out, one already normalized it, and one
opened "" and silently skipped its fsync. They now share key_parent, and
Config::validate calls it rather than adding a fourth answer.

load_or_create_p2p_keypair also refuses a path naming no directory. That is a
backstop behind the config gate, not the gate, so a later caller that skips
validation cannot quietly restore the old behaviour.

* test(node): run the key-permission probe in its own process

The probe zeroes the umask so the assertion means something: under a
restrictive ambient umask the bits are masked to 0600 regardless of whether
the code pins the mode, and the check passes either way.

Zeroing it in the shared test process is the problem. umask is process-global
and cargo runs these tests on threads, so any test creating a file in that
window inherits 000. Measured before this change: an unrelated concurrent
test's file was created 0666.

The probe now runs in a child process, where the zeroed umask cannot reach a
sibling and dies with the child. The parent is an ordinary test that runs
concurrently with everything else. Double-gated with #[ignore] plus an env
check so a bare --ignored sweep does not zero the umask in the shared process
after all.

The parent asserts the child ran exactly one test and that it passed, not
just that it exited 0. A libtest filter matching nothing runs zero tests and
still exits 0, so without that assertion a renamed fixture would read as a
green permission check while asserting nothing. Verified by pointing the
filter at a name that does not exist and watching the parent fail.

* docs(node): scope the p2p key permission claim and note the rotation

The old wording said the key file is "created with owner-only permissions"
without qualification, which is only true on Unix: every permission path in
p2p/mod.rs is cfg(unix), so on other platforms the file inherits whatever the
directory gives it and nothing is enforced. Say what is actually enforced and
where.

Also document what operators now have to do rather than leaving them to
discover it:

- GITLAWB_P2P_KEY must name a directory, since a bare filename is refused at
  startup.
- The PeerId rotates once on the first start after upgrading, so a
  GITLAWB_P2P_BOOTSTRAP multiaddr pinning a peer's old id with a /p2p/<PeerId>
  suffix needs updating or dropping. Suffix-less addresses and the HTTP seed
  list are unaffected.
- If the node reports tightening a loose key directory, the key that was in it
  should be treated as possibly exposed and deleted so a fresh one is
  generated.

* fix(review): close a `..` escape from the p2p key-path check

Two reviewers found the same hole independently: the check rejected a path
naming no directory, but a relative parent that walks back out through `..`
named one and still landed in the working directory. `a/../p2p.key` and
`./keys/../p2p.key` resolve to the cwd itself and `../p2p.key` resolves above
it, so all three put the key exactly where the check exists to keep it out of,
and had ensure_key_dir chmod that directory to 0700 on the way. Verified by
running the paths through the predicate and printing where each parent lands.

The rule is now that a relative key path must name a directory and must not
walk back out: at least one Normal component, no ParentDir. `..` inside an
absolute path stays accepted, since it cannot depend on where the process
started. The predicate moves into names_no_usable_directory next to key_parent,
and the config gate and the load_or_create_p2p_keypair backstop both call it, so
they cannot drift apart.

Also fixes two smaller gaps found in the same pass:

- The permission fixture could report "1 passed" while asserting nothing. Its
  env gate returns early, and an early return is a passing test, so a renamed
  variable would look green. It now prints a sentinel after its assertions and
  the parent requires it. Confirmed by pointing the child at a different
  variable and watching the parent fail.
- A GITLAWB_P2P_KEY starting with `~/` is refused when no home directory
  resolves, instead of creating a literal `~` directory relative to wherever
  the node happened to start.

The backstop had no test, so it has one now, along with a both-directions test
for the predicate. That test cleans up after itself: with the guard removed it
really does write a key next to the source, which broke a later run once.

* fix(review): reject `..` in an absolute p2p key path too

The previous commit closed this for relative paths and exempted absolute ones,
reasoning that an absolute path cannot depend on the working directory. That is
true and it is not the hazard. `key_parent` hands `ensure_key_dir` the lexical
parent, so `/data/keys/../p2p.key` chmods `/data` rather than the `keys`
directory the path appears to name, and `/data/../p2p.key` run as root would
try to tighten `/` to 0700. The exemption also had a test asserting the first of
those was fine, so the gap was written down as intended behaviour.

`..` is now rejected wherever it appears. An absolute path's root counts as
naming a directory, so `/p2p.key` still validates and `/data/keys/p2p.key` is
unaffected.

Found by a second-model review pass after the in-process reviewers had cleared
the relative half.

* fix(node): scrub the serialized p2p private key from memory

Two buffers held the private key in its protobuf form and dropped without
scrubbing: the encoding produced when a new identity is generated, and the file
contents read back on every subsequent start. Both are now Zeroizing, matching
what gitlawb-core already does for its own key material.

Scope worth being honest about: this scrubs our copies of the serialized form,
not the libp2p Keypair itself, which owns the secret for the process lifetime
and exposes no way to zeroize it. The gain is that the encoded bytes do not
outlive the write and the read.

zeroize was already in the tree through gitlawb-core, so this promotes it to a
direct dependency of gitlawb-node and adds no packages; the lockfile change is
the one line recording that.

* fix(node): validate the p2p key path before securing its parent

Reject filesystem-root parents, directory-valued paths, and symlinks before
any directory is created or chmodded. Create missing ancestors at the ambient
mode and pin only the nominated key directory to 0700. Refuse symlink reads
with O_NOFOLLOW, propagate directory durability sync failures, and document
the loose-key rejection path for operators.

* fix(node): satisfy clippy single-element-loop on p2p path test

* fix(node): tighten the p2p key storage contract before mutation

Anchor parent-directory checks on symlink_metadata, reject ~/ escapes and
non-regular existing keys with a bounded read, treat directory AlreadyExists
as a first-boot race, and skip key-path validation when P2P is disabled.

* fix(node): refuse foreign-owned p2p key paths and cap reads on all platforms

Check directory and key ownership before chmod or load, walk ancestors for
foreign control, bound non-Unix reads to MAX_P2P_KEY_BYTES, and gate unix-only
fixture tests without panicking on other targets.

* fix(node): hold the p2p key directory open across its trust checks

Resolve the key path once, then do every check and mutation through an
O_DIRECTORY|O_NOFOLLOW handle on the parent (openat/fstat/fchmod/linkat),
so nothing can be swapped between the ownership check, the tighten, and
the key open. Key reads go through one bounded, non-blocking loader on
every platform; new keys are generated in an unlinked scratch file and
published atomically, with concurrent first boots converging on the
winner's identity. The storage contract is indexed by two matrix tests:
filesystem shapes in p2p (each refusal proven side-effect free against a
recursive snapshot) and path spellings plus the p2p-disabled guarantee
in config.

* fix(node): walk the p2p key ancestor chain descriptor-anchored

Replace the path-following metadata walk (foreign_ancestor_error) and the
post-walk create_dir_all gap with a single descriptor-anchored walk that
verifies and creates the chain together. Missing components are created at
0700 relative to the verified parent and re-verified; AlreadyExists is a race
the winner must pass. Non-unix builds keep create_dir_all.

* fix(node): refuse group-writable and non-directory p2p key ancestors

The ancestor walk now refuses any component with group or world write bits
unless sticky, and any intermediate component that is a symlink or another
object type. Tests drive the walk through real boots: 0770/0775 ancestors
refused before any key IO, 0755 and 1777-sticky ancestors accepted with stable
reload, and an intermediate symlink refused with its target untouched. Both
guards proven load-bearing by neuter revert-checks.

* fix(node): verify the cwd anchor for relative p2p key paths

A bare-relative key path is now walked from a verified working directory:
the cwd is opened as '.' with O_NOFOLLOW|O_DIRECTORY, fstat-verified for
ownership and write bits before any relative component is touched, and a
writable or foreign-owned cwd is refused before anything is created. The
refusal names the working directory path. Child-process fixtures prove both
directions without mutating the shared process cwd.

* test(node): stop rendering uids in p2p assert-message strings

CodeQL flags the uid-interpolating assert messages as cleartext logging to a
console sink. Every assert message in the p2p test module now avoids
interpolating uid values or the uid-bearing error text, while the substantive
assertions (the refusal names both uids, the ownership check fires before the
chmod) stay intact and are verified by the same assertions.

* test(node): add walk rows to the p2p key-storage contract matrix

The matrix now proves the ancestor-walk refusals table-driven: a
group-writable (0775) ancestor, a world-writable non-sticky (0777) ancestor,
and an intermediate symlink component are each refused with the no-mutation
snapshot discipline, while the existing boot rows still reload to a stable
PeerId. The fchmod-on-create path pins newly created ancestors to 0700
regardless of the ambient umask.

* docs(node): note the verified working directory for relative p2p key paths

README, .env.example, and the clap help for GITLAWB_P2P_KEY now state that a
relative key path is anchored at a verified working directory, and that an
unsafe cwd or ancestor is refused with p2p off while HTTP stays up. Also
normalizes two pre-existing em dashes in the README settings table.

* fix(node): pin and verify the mode of every key object this process creates

POSIX applies the process umask to every requested creation mode, so the
0700 passed to mkdirat and the 0600 passed to openat were requests rather
than results. Under a mask that strips owner bits the created directory
landed 0000 and the no-follow reopen failed EACCES before the repairing
fchmod was reached, so that half failed loudly. The scratch key was worse:
it was published unreadable, the boot that created it succeeded on its
already-open descriptor, and only the next boot lost p2p, behind a health
check that still reported healthy.

Every object this process creates now goes through one rule: create, pin
the mode on the object just created, reopen, and verify the achieved mode
by fstat. The directory pin is issued by name off the verified parent
before the reopen, because a directory that landed 0000 cannot be opened
at all, so a pin that waits for the reopen is unreachable in exactly the
case it exists for. A race winner is verified as-is and never chmodded,
which is the rule the ancestor walk already applied.

Pinned's field is private to the pin module and no constructor is
exported, so a descriptor that skipped the pin cannot be turned into a
key-directory handle or reach publication. That is what keeps the rule
from depending on review.

Two adjacent corrections the same defect exposed. The leaf mode predicate
was supersets-only: mode & 0o077 != 0 asks whether anything is granted
beyond the owner, so 0000, 0100 and 0400 all passed it while being
unusable, and an inherited setgid bit was judged on the wrong bit width. A
loose directory is still tightened; an over-closed one is now refused with
its remedy rather than widened, matching how an over-closed key file is
already handled. And an unreadable key or key directory now names its own
cause: every open failure previously reported a refused symlink, which is
what an operator saw for a key masked at creation.

The matrix runs first boot and fresh-process reload across umask 0000,
0022 and 0777, crossed with missing and existing ancestors and leaf, with
concurrent creators and an injected write failure. It runs in child
processes because umask is process global, and it requires at least one
successful concurrent creator so a row where every creator fails cannot
pass. All 21 rows were RED under umask 0777 before this change.

* fix(node): create and publish the node identity key through the pinned path

The node identity PEM had its own storage flow: create_dir_all with no
mode, then write, then set_permissions. That is the exists-then-write-then-
chmod sequence INV-23 prohibits, and it sits in the same ~/.gitlawb the p2p
key uses, so the umask defect the previous commit fixed for one key was
still live for the other.

Measured on the current code, as a non-root uid, all three RED:

  umask 0000  directory landed 0777, world-writable, holding the key
  umask 0022  directory landed 0755, world-traversable
  umask 0777  directory landed 0000 and the write failed with EACCES

The third is the one that mattered most: load_or_create_keypair runs before
the listener binds, so the node exited there and no p2p code was reached at
all. The umask-independence guarantee could not be demonstrated on the
shipped default while this stood.

The directory is now created through the same pin helper, at a verified
0700, and the PEM is published through the same scratch-then-link path at a
verified 0600. An existing directory that grants access beyond the owner is
tightened, which closes the other half of the same gap: a 0600 key inside a
0755 directory is still replaceable by anyone who can write that directory.
An existing key is loaded untouched and never chmodded.

Deliberately not the full ensure_key_dir. That carries the ancestor trust
walk, and importing its refusals onto a path that never had them would turn
an unsafe but currently booting deployment into a boot failure on upgrade.
Only the immediate parent goes through the pin helper, which verifies the
grandparent it is about to chmod a child of; ancestors above that keep the
existing create_dir_all behavior. The one new refusal is a group or world
writable non-sticky grandparent, which is the case where another local user
can replace the node's identity outright.

* fix(node): split p2p key config validation from live storage validation

The key-path validator answered two different questions and its callers
disagreed about what to do with the answer. Bare names, `..`, a trailing
separator and the filesystem root are properties of the configured value,
and Config::validate refuses them before the listener binds. The same
function also stat'd the key path and its parent, so a symlinked parent, a
non-directory parent or an unreadable parent exited the node as invalid
configuration, while the identical class of fault found one layer later in
load_or_create_p2p_keypair only logged and left HTTP serving.

That split was invisible from the outside and the docs described only one
half of it. Measured against the binary before this change: a symlinked
parent, a regular-file parent and an unreadable parent all exited 1 before
bind, which is the opposite of what README and .env.example promise.

Validation is now lexical only, behind a P2pKeyConfigError so the two
domains cannot be confused at a call site. Every live storage fact is left
to the load path, which already re-establishes each one on a descriptor it
opened rather than a pathname it stat'd: O_NOFOLLOW on the key open, a
regular-file check by fstat, and ELOOP or ENOTDIR at the leaf. The verdicts
are unchanged; what changes is that one policy now decides all of them.

Two supporting changes. The whole p2p port gate moves ahead of the database
connect, both arms together, because connect_db_with_retry retries forever
and left the disabled arm unreachable without a database; only p2p::start
still needs the pool. And a failed key load is now logged at error with a
stable event name and mirrored into a gauge, because the policy this commit
settles on is to keep serving HTTP with a green health check while the node
is off the p2p network, and that is only defensible if the outage is
visible to something other than a human reading startup logs.

A new integration test drives the real binary with no database and proves
both domains at the process boundary: nine lexical spellings exit before
binding, ten storage faults serve HTTP with the failure logged, and
GITLAWB_P2P_PORT=0 reaches its disabled log without touching the key tree.
Every row asserts a before/after snapshot, because a refusal that mutates
storage on its way out is its own defect and a returned error cannot show
it. Three of those rows were RED before this change.

A symlinked key directory also reports as a symlink again. Linux returns
ENOTDIR rather than ELOOP when O_NOFOLLOW meets O_DIRECTORY, so the errno
alone cannot separate a symlink from a regular file and the message had
regressed to naming the wrong cause.

* docs(node): state the two-domain key-path policy and how to see a degrade

README and .env.example promised that p2p stays off while HTTP keeps
serving when the key cannot be loaded, including for an unsafe ancestor or
working directory. Measured against the binary, that was false for a
symlinked parent, a non-directory parent and an unreadable parent, each of
which exited before binding. The preceding commit made the code match the
promise; this makes the promise precise.

All three operator surfaces now name the split the same way. A value that
cannot name a securable key file is refused before the node binds. Anything
wrong with the storage itself degrades, and the sentence says how to notice:
the node logs p2p_identity_key_load_failed at error and sets the matching
metric, /health stays green in that state, and fixing the storage and
restarting restores the same PeerId. A node quietly off the network is the
cost this policy accepts, so the docs point at the signal rather than
leaving an operator to find it in startup logs.

Two corrections while here. The key directory is tightened only when it
grants access beyond its owner; one closed too far is refused with a
chmod 700 remedy rather than widened, so the sentence no longer implies the
node will open up a directory an operator locked down. And the existing
advice to delete a key after a tightening warning is now scoped to the
loosening case it was written for: an over-closed directory is refused, not
tightened, and nothing in it was ever exposed, so following that advice
would have meant deleting a safe key and taking a second PeerId rotation
that breaks every pinned bootstrap multiaddr.

* test(node): pin the degrade signal and wait for both startup lines

Two gaps in the boundary test, both found by extending the mutation spec to
cover this commit's own guards.

The degrade rows asserted only the prose message, so renaming the event
field or dropping the level would have passed while every alert built on
the operator docs broke and the node still looked healthy. The rows now
pin the error level and the p2p_identity_key_load_failed name the docs tell
operators to watch.

The rows also waited on a single log line, which is a race: the degraded
server logs "ready" from a spawned task while the p2p gate logs its verdict
from the main task, so whichever the reader stopped at first left the other
uncaptured. It passed when the test binary ran directly and when the row ran
alone, and failed only once other work shifted the scheduling. Reads now
wait for every needle before returning.

The fix stays on the test side on purpose. Logging the p2p verdict before
the degraded server is spawned would also make the order deterministic, but
that is production sequencing changed for a test's convenience.

* fix(node): keep key publication transactional and path-policy split

Bare GITLAWB_KEY filenames publish into the working directory without
chmodding it. A directory this process created is removed if pin or
verify fails. Ancestor walks use search descriptors so a safe 0111
component is traversable. Scratch unlink is part of publish success and
is fsynced. Rotation advice is limited to writable or readable exposure,
not ordinary 0755 tightening.

* fix(node): refuse a writable cwd for a bare identity key

Publishing identity.pem into cwd must not chmod cwd, but a 0600 key is still replaceable if that directory is group or world writable. Apply the same trusted-parent check used for nominated key directories before the handle is returned.

* fix(node): load identity without following pathnames

Create already published through a directory handle. Load still used exists-then-read, which followed a symlink at the key and opened the grandparent by pathname. Open both no-follow, treat /identity.pem like a cwd publish so it reaches the filesystem, and open a search-only grandparent with the walk flags.

* fix(node): parse key paths, leave identity parents, repair special modes

A terminal `.` in GITLAWB_P2P_KEY was accepted then retargeted by Path onto
the parent directory. Identity creation imported p2p's 0700 pin onto any
existing named parent. A setgid-only 2700 key directory skipped repair and
disabled P2P. Darwin ancestor walks still required directory-list permission.

* fix(node): type-check the non-unix p2p scratch writer

The shared scratch-then-link path names pin::Pinned on every target, but
the pin module was unix-only, so gitlawb-node did not compile off Unix.

* fix(node): refuse identity path spellings that Path would retarget

GITLAWB_KEY had the same terminal-dot hole as GITLAWB_P2P_KEY, and its
tilde expansion did not refuse a suffix that escaped home. Bare filenames
stay legal.

* fix(node): create every missing identity parent on one pinned walk

The identity key path split one creation transition across two policies.
`create_dir_all` built every ancestor above the immediate parent at the
ambient umask, then `create_dir_pinned_at` ran `verify_trusted_parent` on
the last of those and refused it. Under umask 0002 that ancestor lands
0775, so the phase that made it handed it to the phase that rejects it:
the node exits before binding and the failed boot leaves the directory
behind at 0775, so the next attempt starts from state the first one made.

Replace the split with one descriptor-anchored walk. Pass 1 walks up from
the key's parent until a component opens; that descriptor is the anchor, and
it is the same fd the next step creates into. Pass 2 walks back down calling
`create_dir_pinned_at` per component, each one pinned 0700 and verified at
its achieved mode before it becomes the parent of the next, so from the
anchor down no pathname is resolved twice. Pre-existing ancestors are
adopted as-is and never chmodded, since GITLAWB_KEY is a file path rather
than a dedicated-directory setting; write-authority is still required on the
directory each component is created in.

A boot that fails past the walk now removes exactly what it created,
deepest first, through unlinkat with AT_REMOVEDIR against the descriptor
each entry was made in. A race-adopted directory is never in that list, and
AT_REMOVEDIR refuses a non-empty directory, so a leaf another boot has
published into survives. The accumulator is an out-parameter so a mid-walk
failure and a post-walk failure reach the same single rollback arm.

Two consequences worth naming. A symlink at the deepest existing ancestor is
refused when the missing suffix reaches it, where `create_dir_all` followed
it. Ancestors ABOVE that anchor are still resolved by pathname and their
symlinks still followed, unchanged from before this walk, because O_NOFOLLOW
binds only the final component of a path-based open; closing that would mean
an openat chain from the root and the full ancestor policy that GITLAWB_KEY
paths deliberately do not get. The doc comment now says so rather than
claiming more than the code does. And a losing concurrent first boot can
remove an intermediate another boot adopted but has not yet written into,
turning that boot's start into an ENOENT failure; both still fail closed.

Also replace the root-adjacent test. It proved a pathname property by
running the full mutating path against the real filesystem root and
assuming the runner lacks permission. Under euid 0, common in build
containers, `/` is accepted (root-owned, 0755, no group or other write
bit), the production path returns Ok, the test's panic arm fires before its
cleanup assertion, and a real private key is left at /identity.pem, after
which the leading exists() check makes every later run a silent skip. The
routing is now asserted lexically through a named predicate, with no
filesystem access and no dependence on euid.

The matrix covers one, two and three missing levels under umasks 0000,
0002, 0022 and 0777, a pre-existing 0755 ancestor that must stay 0755, a
relative multi-level path, a symlinked deepest ancestor, refusal without
partial mutation, and a populated leaf whose removal must be refused and
reported. Created-directory modes are asserted by exact equality, and each
success row reloads in a fresh process and requires the same DID.

* test(node): race real concurrent first boots of the identity key

The rollback added with the pinned walk can remove an intermediate that a
concurrent boot has already adopted, and until now that was reasoned about
rather than executed. Nothing in the suite started two real first boots
against the same key path.

Add a `concurrent-first-boot` layout to the self-exec fixture and a driver
that races several children through `load_or_create_keypair_at` on one
missing multi-level path. A two-phase file rendezvous puts every child
inside a spin loop before any of them is released, so the spread across the
racing call is one loop iteration rather than process-startup jitter. Three
arms: unaided boots, one injected write failure against one unaided boot,
and four injected failures against one, which is what actually opens the
window.

The assertions are properties of every legal outcome, never one
interleaving, so the test is deterministic even though the race is not. A
boot that claims success has a parseable 0600 key on disk, all successes
agree on the DID, surviving directories are exactly 0700 with no scratch
residue, and if no key was published then nothing claimed success and a
retry must still work. Observation counters are printed, not asserted, so a
run that never hits the window says so rather than passing as proof.

Two things the race showed that reading could not. The documented window is
real: a boot was refused with "No such file or directory" on its publish
after a loser rolled back the leaf it had adopted. But the more common shape
is one hop earlier, the loser removing an adopted intermediate so the ENOENT
lands on the next component's creation instead. And under enough concurrent
rollbacks every boot can fail, not merely the losers, so several nodes
starting against a shared first-boot path can all need a restart. Two-child
racing never opened the window at all in 500 iterations.

* fix(node): resolve identity key path components one at a time

`O_NOFOLLOW` binds only the final component of a path-based `open(2)`, which
is all the kernel offers for a multi-component pathname. Three sites resolved
a whole configured path in one open and so guarded only the last position:
the load path, the existing-named-parent fast path, and the walk that creates
missing parents. Every interior symlink was followed, and `verify_trusted_parent`
then attested the directory that was reached rather than the path taken to it.

That is enough to substitute the node's whole identity, not merely misplace
its key. Load and create resolve the configured pathname identically, so
repointing one interior link redirects both. Point it at a directory holding
another PEM and the node boots as that DID while its real key sits untouched
on disk and nothing is logged as wrong. Point it at an empty directory and
the node mints a fresh identity and reports a first boot. Neither needs the
attacker to read anything: ownership and mode checks still pass, because they
run on the object at the end of the path.

Resolve each component with `openat` and `O_NOFOLLOW` against the descriptor
of the one before it, from `/` for an absolute path or the working directory
for a relative one, so a symlink is refused at every position. Interior
components open search-only and the final component keeps the caller's flags,
so a legitimate 0111 ancestor still resolves and the leaf descriptor is still
the one that gets fsynced and chmodded.

Components are adopted with no ownership or mode judgment. The only new
refusal is a symlink on the key path: a shared 0775 volume, a foreign-owned
ancestor and `/etc` all keep working, because `GITLAWB_KEY` is a file path
rather than a dedicated-directory setting. `verify_trusted_parent` stays
exactly where it was, on the descriptor a child is created in. The p2p key
path is untouched; it already walked component by component.

This is pre-existing rather than a regression: the same probe against the
branch point produces the same result. It is fixed here because closing it
replaces the walk this branch just added, so splitting it would mean
reviewing that code twice.

Covered by rows for an interior symlink with the next level present, an
absolute target escaping the base, a two-link chain, the fast path, a
relative configured path, and both load-path variants asserting on the DID
the node ends up presenting rather than on a mode or an error string. The
cases that were already refused are pinned too, including a symlink at the
immediate parent and a dangling link, neither of which had a test before.

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-04 17:02:15 +00:00
beardthelion 8a98cab243 test(node): cover the component walk in validated_repo_disk_path (#411) (#441)
* test(node): cover the component walk in validated_repo_disk_path

* chore: re-trigger code quality check
2026-11-04 16:38:52 +00:00
JATMNandClaude Opus 5.5 623b600835 fix(deps): bump libp2p to pick up hickory 0.26 and serde_with 3.22 (#518)
hickory-proto/hickory-resolver 0.25.2 were pulled in only by libp2p-dns
0.44. libp2p-dns 0.45 moves to hickory 0.26 but needs libp2p-core 0.44
and libp2p-identity 0.3, so the node's direct libp2p crates move to the
matching release line (core 0.44, gossipsub 0.50.1, identify 0.48,
identity 0.3, kad 0.49, quic 0.14, swarm 0.48, dns 0.45). multiaddr goes
to 0.19 to match libp2p-core and avoid compiling two copies. No source
changes were needed.

Lockfile: hickory-proto/resolver 0.25.2 -> 0.26.3 (GHSA-q2qq-hmj6-3wpp /
RUSTSEC-2026-0119, GHSA-6f2x-v7q7-m7m5), serde_with 3.18.0 -> 3.22.0
(GHSA-7gcf-g7xr-8hxj).

With the fix in, drop the RUSTSEC-2026-0118/0119 ignores from
.cargo/audit.toml as #76 requires.

Closes #76

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 16:37:21 +00:00
beardthelionandKevin Codex 9bda2e103c fix(node): authenticate gossip ref-update events before writing them (#325)
* feat(node): sign and verify gossip ref-update events

Add an optional sig field to RefUpdateEvent carrying a base64 Ed25519
signature over the event's own fields, plus signing_bytes, the single
producer of signing input on both sides, and sign/verify helpers.

The field is optional and skipped when absent, following the owner_did
precedent from #144, so an event from a signing node still deserializes
on a peer that predates this change. A nested envelope would have broken
that and is deliberately not used.

Verification resolves the key from the claimed node_did, so a signature
that does not bind the DID it claims is refused. Non-did:key and
unresolvable DIDs are refused with the peers-table denial sentences,
built from PeerWriteDenied rather than retyped so the two surfaces
cannot drift apart.

A golden test pins the signing input: a field reorder changes what every
peer signs, which no round-trip test can see.

Helpers are not wired into the ingest or emit paths yet.

* fix(node): authenticate gossip ref-update events before writing them

The gossipsub ingest path accepted a ref-update from any peer and wrote
it, while the HTTP route carrying the same payload checks the signature
against the claimed DID, requires a known peer, and validates the repo
slug. Extract the path into ingest_ref_update and apply the same three
gates in that order, so both transports answer the same input the same
way.

A present signature must verify in either mode. An absent one is refused
when GITLAWB_REQUIRE_SIGNED_PEER_WRITES is set and otherwise accepted
with a warning naming the flag, so peers that predate signing keep
federating until the fleet is ready. The DID-method refusal runs right
after parse in both modes and reuses the peers-table denial sentences.

Rejection writes nothing: neither a ref-update row nor a sync-queue
entry. Those are separate sinks and the tests assert them separately.

from_peer's comment now says it records the forwarder, not the author.

* fix(node): bound gossip ingest per source and sign outbound events

Brake the ingest path at 60 events per minute per forwarding peer,
checked ahead of the parse and the signature work. Verifying a signature
is the expensive step, so a brake behind it would sell an unauthenticated
flood the CPU the brake exists to protect. Rejection writes nothing to
either sink.

Sign events on the way out, so a fleet running with enforcement on can
verify what its peers publish. A signing failure warns and skips the
publish rather than emitting something a verifying peer would drop.

The emit-to-ingest round trip is asserted end to end, which is what
proves both sides agree on the signing input by execution rather than by
inspection.

* test(node): cover the did-method gate and correct overclaiming comments

The did-method gate had no load-bearing test: deleting it left the suite
green, because the only non-did:key case was signed, so verification
resolved the DID internally and returned the same refusal. Add the case
only the gate decides, an unsigned non-did:key event from a registered
peer with enforcement off, and correct the comment that claimed the
other test covered it.

Assert the limiter's source bound, which no test pinned, and make the
backward-compat test able to see an added field by parsing under a
deny_unknown_fields reader as well as a permissive one.

Say plainly at the peers check that authentication is not authorization:
a self-registered did:key passes it, so what the gate buys is blocking an
unregistered DID and, with the signature, blocking impersonation of an
existing peer. Drop the claim that the unsigned-acceptance warning is
covered by a test, because it is not.

An outcome of Accepted now means the row was stored; a failed write
reports itself instead of reading as success.

* fix(node): charge the ingest budget to the author, not the mesh edge

A single 60-per-forwarder budget was both too tight and pointed at the
wrong principal. Pushes emit one gossip event per ref, so a push touching
more than 60 refs lost its tail with no signal to the publisher, and
because gossipsub re-shares mesh-wide, a flood routed through an honest
neighbour spent that neighbour's budget on every receiver.

Split it. A loose pre-parse bound on the forwarding peer still caps the
CPU spent before anything is parsed or verified, which is all a
free-to-mint identity can honestly buy. A tighter bound on the
authenticated node_did sits after verification and the peer gate, so the
two durable writes are charged to whoever actually authored them, and it
is sized to admit a large push whole.

Both rejections now name the budget and its window, so a dropped
ref-update says which bound refused it rather than vanishing.

Replace the per-event full peers scan with a keyed lookup on the same
column; upsert_peer shares it.

A push above the author bound still loses its tail, now loudly. The two
transports continue to disagree about events per push: gossip publishes
per ref while sync/notify batches.

* docs(node): the did-method gate runs after the parse, not before it

Two test comments called it the pre-parse did-method gate. The parse is
above it and the gate is below; what is true is that it runs before the
signature branch, in both enforcement modes. The pre-parse label belongs
to the forwarder brake, which is genuinely the first statement, and the
nearby correct usage is where the wrong one came from.

* fix(node): charge the gossip write budget only to a proven author

An unsigned ref-update event's node_did is asserted, not proven. The author
budget was debited on that key in both signature arms, so a flood of unsigned
events naming a registered peer drained that peer's budget and its own genuine
signed updates were shed. The attacker chose the victim.

The author budget is now charged only when the signature verified. Unsigned
compatibility traffic charges a separate forwarder-keyed bucket sized at
GOSSIP_UNSIGNED_SOURCE_MAX_EVENTS (1500), above the per-author cap because a
forwarder aggregates many authors, and below the pre-parse brake; the
compile-time assert now pins all three in one chain. The shed carries its own
IngestOutcome variant so a forwarder is never named as an author.

validate_repo_slug moves above both the signature match and peer_exists, so a
malformed event costs no database round trip rather than merely being charged
to nobody. A test-only peer_exists tally proves that placement, which an
outcome assertion cannot distinguish.

* fix(node): version the signed ref-update format and refuse unknown versions

The signed payload carried no version and verify_ref_update knew one
representation, so the next signed-field addition would partition the mesh
mid-rollout: an upgraded publisher signs bytes an older receiver cannot
reproduce, and the older receiver reports a genuine event as a bad signature.

RefUpdateEvent now carries v, skipped when zero, so a v0 event's wire and
signing bytes stay byte-identical to the versionless form. GOLDEN_SIGNING_BYTES
is unchanged and every signature already issued still verifies. A frozen
artifact signed before the field existed is checked into the suite and verified
without being re-signed, since a signature a test just produced is
self-consistent by construction and proves nothing about compatibility.

An event whose version this build cannot interpret is now refused in its own
words, ahead of every other gate: no field below an unknown version has known
meaning. Without that, a v1 event was admitted outright rather than merely
mislabelled.

* docs(node): describe the real signing handoff at the publish site

The comment claimed signing was future work and the event went out unsigned.
It does not: the swarm loop signs via p2p::signed_publish_bytes immediately
before publishing, and skips the publish entirely when signing fails. Leaving
sig None here is correct, and the comment now says why rather than describing
a state the code left behind.

* docs: GITLAWB_REQUIRE_SIGNED_PEER_WRITES spans both transports

The flag gates gossip ref-update ingest as well as the HTTP announce and sync
routes, but README and .env.example still scoped it to those two routes. An
operator who upgraded the HTTP peers and then enabled it would silently drop
gossip from every publisher still running an old build, with no error visible
on the publisher's side because gossip carries no response.

Documents the cross-transport behavior and the one ordering requirement that
follows: upgrade every gossip publisher before enabling.

* fix(review): count gossip ingest outcomes and sweep the ingest limiters

Every IngestOutcome reached tracing and nothing else, so gossip-ingest health
was invisible on /metrics, the only surface this daemon exposes to a dashboard
or an alert. Adds gitlawb_gossip_ingest_events_total{outcome} alongside the
existing sync-queue and webhook counters, with the label derived from an
exhaustive match so a seventh variant fails to compile rather than shipping
uncounted. The reason strings stay in the log line; they are sender-shaped and
would make the label set unbounded.

The three ingest limiters were built as locals in the swarm task, so
sweep_rate_limiters could not reach them and their keys stayed resident until
the 200k ceiling. The swarm loop now sweeps them on its own interval, and
cleanup iterates a destructured array so a fourth limiter breaks the build
instead of being silently missed, which is the failure sweep_rate_limiters'
own comment records against the /ipfs limiter.

* test(node): close four gaps the review found in this PR's own guards

The frozen signing constants pinned only the all-Some optional shape, but the
sole publish site always emits cert_id None, so no constant covered the
encoding every real event uses. Proven during review: adding
skip_serializing_if to cert_id left the suite green while the signing input
changed for 100% of production events. A second constant now pins the
null-carrying form.

The legacy artifact's provenance assertion claimed re-capturing it from current
code was what the assertion refused. It was not: skip-when-zero means a
regenerated artifact also carries no version key, so the check passed and the
claim was unfounded. A SHA-256 over the frozen bytes does the job the message
promised, and the message now states only what it checks.

WriteFailed was the one outcome variant no test constructed, so the
independent-writes property it exists to express was unexercised. Both
directions are now driven by failing a real write.

The publish arm's body is extracted so the exact bytes handed to gossipsub are
asserted signed and accepted by ingest under enforcement. The select! dispatch
and the require_signed threading from main.rs still need a live swarm and are
named as uncovered rather than implied covered.

* fix(review): close the remaining findings from the round on this PR

The unsigned-acceptance warning fired before two gates that could still drop
the event, so a message the operator reads as an admission could be followed by
a forwarder-budget shed or an unknown-peer refusal. It now fires only once the
event is genuinely admitted, and a test pins that it fires exactly once, which
nothing checked before.

The publish site hardcoded the format version while the ingest gate compared
against the constant, so emitter and gate could drift apart silently. The
publish site now reads the constant.

The known-peer gate changed a federation precondition without documenting it: a
gossip publisher must also be a known peer on the receiving node, via bootstrap
or a prior announce, or its events are dropped in both flag modes. That is now
in the README beside the rollout ordering, derived from what the ingest path
does rather than from another docstring.

Six tests each hand-rolled the same both-modes rejection loop; they now share
one helper. The log-capture helper was left under a tmp_ prefix from the round
that added it and is renamed, since scaffolding vocabulary should not ship.

* fix(review): report unsigned gossip admissions as their own outcome and metric (#325)

An unsigned event that survives the rolling-upgrade window was returned as
IngestOutcome::Accepted, so `gitlawb_gossip_ingest_events_total{outcome="accepted"}`
mixed signature-verified admissions with unauthenticated legacy traffic. An
operator could not tell whether the fleet still relied on the compatibility
allowance or authenticated gossip was healthy.

Add IngestOutcome::UnsignedAdmitted with its own `unsigned_admitted` metric
label, returned when an unsigned event passes every gate and the writes land.
`Accepted` is now reserved for signature-verified events. Propagate through the
swarm-loop logging and the tests that asserted `Accepted` for unsigned
admissions.

Both guards proven load-bearing by execution: reverting the return-site change
goes RED on flag_off_unsigned_known_peer_event_is_accepted ("got Accepted"),
and giving UnsignedAdmitted the "accepted" label goes RED on
every_ingest_outcome_carries_a_distinct_metric_label.

* fix(node): shed an over-budget gossip author before the peer lookup (#325)

The author brake sat below `db.peer_exists`, so a signed author already past its
budget still paid a Postgres round trip before being refused. That is a brake
behind the work it bounds: the lookup is done for a request that was never going
to be admitted, and a signed author can drive it at the pre-parse source rate,
further with peer-id rotation.

The brake now answers first, and the answer is a READ rather than the charge.
`check` inserts a window for a key it has never seen, so probing with it here
would let a flood of self-minted signed DIDs occupy the bounded author map before
`peer_exists` could refuse them, and legitimate new authors would be shed once it
filled. That would have made the remedy worse than the finding. `is_over_budget`
allocates nothing, reports an unseen key as within budget, and leaves the charge
itself below the lookup, unmoved. The only behaviour that changes is WHEN an
already-over-budget author is refused.

Two guards, both proven load-bearing by execution. Removing the early shed makes
`an_over_budget_author_is_shed_without_a_peer_lookup` red on the call counter,
which observes the lookup not happening rather than restating the ordering.
Swapping the read for `check` makes `the_early_author_check_does_not_track_an_unseen_did`
red on the map size, which is the must-not that keeps the brake from becoming its
own memory-fill surface.

Found by a cross-family refute pass on the pushed head.

* chore: retrigger checks after a CodeQL infrastructure failure

The swift analysis job on run 32041989078 failed in "Set up job", before
checkout or any analysis step. GitHub refuses a rerun on default-setup
(dynamic) runs, so a new head is the only way to get a fresh scan.

* fix(deps): bump h2 to 0.4.19 for RUSTSEC-2026-0258

The branch lockfile still pinned h2 0.4.13, so cargo audit failed on
every push while main had already moved on. This is lockfile-only; no
source change in the gossip auth work.

* chore: re-trigger code quality check

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-11-04 16:34:10 +00:00
JATMNandClaude Opus 5.5 151d688387 fix(deps): bump aws-smithy-json to 0.62.7 (GHSA-8ffr-xgwf-xj56) (#516)
Cargo.lock resolved aws-smithy-json 0.61.9 (via aws-sdk-s3 1.119.0) and
0.62.5 (via aws-config and the sso/ssooidc/sts clients). Both are in the
range affected by GHSA-8ffr-xgwf-xj56 / CVE-2026-18140 (<= 0.62.6):
uncontrolled recursion when skipping unknown keys in deeply nested JSON.

0.62.7 requires aws-smithy-runtime-api ^1.12.3, so the AWS SDK moves as
a set to the newest releases that still build on our 1.91 MSRV (they
declare rust-version 1.91.1; later releases need 1.94.1):
aws-config 1.8.18, aws-sdk-s3 1.137.0 (which itself requires
aws-smithy-json ^0.62.7), aws-sdk-sso 1.102.0, aws-sdk-ssooidc 1.104.0,
aws-sdk-sts 1.107.0, aws-sigv4 1.4.5, and matching aws-smithy-* crates.

aws-sigv4 1.4.5 moves to p256 0.13, which drops crypto-bigint 0.4.9 from
the lockfile. Update the RUSTSEC-2026-0387 note in .cargo/audit.toml to
match; the ignore is still needed for crypto-bigint 0.5.5.

Lockfile and audit-config comment only; no manifest or source changes.

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 15:14:11 +00:00
JATMNandClaude Opus 5.5 4d024e01bc fix(deps): resolve cargo audit failures on main (#514)
* fix(deps): resolve cargo audit failures on main

Update quinn-proto 0.11.19, rustls 0.23.46, rustls-webpki 0.103.16,
webpki-roots 1.0.11, zstd 0.13.4, zstd-safe 7.4.0 and zstd-sys 2.1.2 in
Cargo.lock to clear RUSTSEC-2026-0338..0342, 0346..0349, 0381, 0382,
0391, 0398, 0403 and 0410.

Ignore RUSTSEC-2026-0387 (crypto-bigint): the fix is only in 0.7.x, and
no dependent compatible with our tree uses it. alloy-signer (through
2.5.0) pins elliptic-curve 0.13, and aws-sigv4 >=1.5 requires Rust
1.94.1, above the 1.91 MSRV.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(audit): document RUSTSEC-2026-0387 reachability and graph-based removal

Correct the aws-sigv4 boundary (1.6.4 is the first release on
crypto-bigint 0.7; 1.5.x-1.6.3 still need ^0.5.4), mark the ignore as
reachable through the k256 PrivateKeySigner and SigV4a signing paths,
and tie removal to Cargo.lock having no crypto-bigint below 0.7.6.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 12:26:02 +08:00
beardthelion 6588590813 chore: update rustls 0.23.37 → 0.23.45 to resolve RUSTSEC-2026-0285 (#455) 2026-11-04 01:23:10 +00:00
Kevin Codex bfc44f926d Merge pull request #381 from Gitlawb/fix/gl-whoami-capped-body-read
fix(gl): read whoami error bodies through CappedBody.text
2026-08-26 13:26:50 +08:00
beardthelion 65f5989a1d test(node): pin owner-push off in receive-pack lock tests
These lock-path proofs use non-owner proof DIDs. With enforce_owner_push
defaulting on, they fail before receive-pack runs.
2026-08-25 10:27:48 -05:00
beardthelion 4725146db2 fix(gl): read whoami error bodies through CappedBody.text
#173 changed read_body_capped to return CappedBody; peer.rs and sync.rs
were updated but whoami.rs was not, leaving main red after the #223 merge.
2026-08-25 10:04:20 -05:00
Kevin Codex 73fd7473af Merge pull request #330 from Gitlawb/fix/default-enforce-owner-push
fix(node)!: enforce owner-only push by default
2026-08-25 22:06:12 +08:00
Kevin Codex 21c74c65d9 Merge pull request #223 from Gravirei/fix/issue-220-whoami-registered-false-trust-lookup
fix(gl): only report registered: false on 404 in whoami (#220)
2026-08-25 22:04:56 +08:00
Kevin Codex 21f8cb6135 Merge pull request #173 from Gitlawb/fix/issue-135-ipfs-cid-tree-gate
fix(node): gate GET /ipfs/{cid} tree objects so a withheld subtree's structure can't leak (#135)
2026-08-25 22:04:14 +08:00
Gravirei 861cced8f8 chore: update h2 0.4.13 → 0.4.18 to resolve RUSTSEC-2026-0258 2026-08-24 15:10:32 +06:00
Gravirei 39ce655904 fix(gl): sanitize full transport error chain in whoami, remove raw error attachment (#220) 2026-08-23 22:12:19 +06:00
Gravirei d25cd0c6e6 merge: resolve sync.rs conflict — move read_body_capped/sanitize_node_msg to http.rs (#220) 2026-08-23 22:07:46 +06:00
Kevin Codex e4c7458fb3 Merge pull request #332 from anandh8x/agent/docs-correct-security-status
docs: correct security and workspace status
2026-08-16 21:40:30 +08:00
beardthelion 76f630008f fix(node): never hand back a continuation that made no progress
On a resumed request whose visit budget was already spent by the provenance
phase, the scan's top-of-loop visit arm sealed pager.cursor, which at that
moment is the position the caller just sent. The node returned the caller's own
token, verbatim, rung after rung. Three rungs were observed returning an
identical position.

A token looks like progress, so the client keeps going: gl retries to its
resume cap, and every one of those requests re-runs the full provenance phase,
up to seventeen repo acquires and cat-file subprocesses, advancing nothing
before it errors. That is roughly nine anonymous requests worth of work for
none, and it is worse than shedding nothing, because a caller who is told the
ladder is over stops immediately.

A seal now has to be strictly ahead of where the request itself started: a
different candidate is ahead by construction, since only the gated advance can
name one, and the same candidate needs a row past the start row. A request that
started at the front is before everything, so its seals pass untouched.

When the proposer settled at least one row this rung, the existing ceiling arm
already seals that row, and it is strictly ahead because a resumed scan only
walks rows past its cursor. Only a rung that settled nothing sheds without a
token, and that is honest: the spender is the provenance phase, which runs the
same way every rung, so no retry can do better.

The filter sits at the single mint site, where a future call site cannot bypass
it, and it logs the drop as a boolean. record_scan_truncation has already
logged that a position was sealed by then, and a 503 carrying no token next to
that line is the confusion that log exists to prevent.
2026-08-16 02:20:05 -05:00
Vasanthdev2004 888fab6028 fix(infra): serialize the upgrade and write compose through a unique temp path
The compose reinstall added last round is not safe under overlap, and `main`
never wrote compose from this document, so the race arrives with that commit.

Two executions — a double run, a retry while the first is still pulling, two
operators — shared the fixed `/opt/gitlawb/compose.yaml.new`. The second
truncates the file the first is about to install; and once the first renames,
the second's open descriptor keeps writing into the live `compose.yaml` inode.
Both outcomes stay valid YAML, so the `up -d --remove-orphans` on the next line
accepts a blended or truncated service set and deletes whatever fell off — on a
live node, the local postgres or the TLS terminator. Both reviewers reproduced
it independently; one measured a short file in 12 of 12 trials.

The whole step now runs under `flock -n` on /opt/gitlawb/.upgrade.lock and
exits non-zero if the lock is held, so a second run cannot restart against a
file the first is mid-install. The temp file comes from `mktemp`, so concurrent
writers cannot truncate each other, with a trap to clean it up on failure.

The compose body is base64-encoded rather than written through a nested
heredoc. The rendering carries `$${VAR}` passthroughs that must reach the file
unexpanded, which needs a quoted delimiter, and a shell heredoc nested inside an
indented Terraform heredoc depends on the dedent landing that delimiter at
column 0. Decoding one line has neither failure mode and no delimiter can
collide with the content.

Not validated locally: terraform is not installed on this machine, so this
change has had no `fmt`, `validate`, or `plan` run against it.
2026-08-16 11:00:21 +05:30
beardthelion d0635d95b7 fix(node): resume the /ipfs scan per candidate so a starved one is not skipped
A CID that maps to several oids shared one resume slot across every candidate,
and the slot kept the maximum position. An earlier candidate could spend the
probe budget walking past a repo that holds the object for a later one, seal a
position beyond it, and the next rung would resume past a row that candidate
never examined, wrap, and shed tokenless. The client reads an absent token as
the ladder being over, so the object became permanently unretrievable, at stock
config, deterministically on every retry.

The token now names which candidate it is resuming, and the rules that keep
that sound are narrower than they first look:

Only one candidate per request may seal, and which one depends on where the
REQUEST started. On a resumed request it is the resumed candidate alone, since
the shared pager holds only the table suffix from the caller's cursor, so a
later candidate walked a suffix and never saw the front. On a front-started
request it is the first unfinished candidate, since there every candidate walks
from the front and a later candidate's stop is honest coverage. Silencing later
candidates unconditionally would remove the only thing that mints rung 1 when
the first candidate wraps untruncated.

A candidate is finished when its row loop walked every fetched row, or when it
owed no scan at all. Both matter: a properly provenanced candidate never wraps,
so without the second arm the ladder dies every rung. The wrap is witnessed per
candidate at the row loop's own two exits, never by reading the shared pager
flag at the tail, which any short page sets and which would let a candidate that
truncated mid-page look finished and strand the rows it refused.

Finishing a non-final candidate advances the seal to the next one at a front
sentinel and taints, because the tail emits a continuation only when something
tainted; sealing without tainting would suppress the taint and return a
definitive 404 while discarding the token it had just minted.

The keep-the-maximum comparison is gone. With one proposer per request the slot
is written at most once, so an assertion states that directly instead.

The pager stays shared per request. A per-candidate pager would restore the
fan-out the paging exists to remove.
2026-08-15 23:09:36 -05:00
beardthelion 2656718916 feat(core): carry the resumed candidate's identity in the scan token
A CID can map to several git oids, and the ladder needs to name which one it is
resuming. The sealed position gains the candidate's oid hex so a rung resumes
that candidate rather than a position in a list: oids_for_cid is a sorted set,
so an ordinal silently repoints at a different candidate when a pin that sorts
earlier arrives between rungs, while an identity degrades safely to "not found,
restart at the front".

The field is length-prefixed and padded to 64, matching the framing the row
fields already use, because production oids are 40 hex, not 64: repos are
created with --object-format=sha1 and only the test fixtures are sha256. A
fixed 64-byte field would fail every seal on a real deployment and shed a
tokenless 503, which the client reads as the ladder being over. Both widths are
exercised, and a zero-length candidate is rejected at decode so it cannot be
confused with the front-of-table sentinel, which is empty row fields with a
real candidate.

VERSION goes to 3 and the plaintext to 527 bytes, so a token minted under the
old layout opens to None and the caller restarts at the front. Nothing has
minted one outside tests.

The slot carrying the position is a struct rather than a widened tuple on
purpose: a 3-tuple would have pulled the hex into the existing keep-the-maximum
comparison, changing behavior this commit is meant to leave alone.

Token length stays invariant across both oid widths, since length would
otherwise be a side channel for the withheld row it names. That is asserted on
a real seal in gitlawb-core, not on the gl fixtures: nothing in gl seals or
opens a token, so its width constant cannot detect a wrong layout.
2026-08-15 22:20:34 -05:00
beardthelion 366b599331 fix(node): order the CID candidate list so it cannot depend on heap order
oids_for_cid ran a bare SELECT with no ORDER BY, so Postgres was free to return
the candidates in physical heap order. get_by_cid walks those candidates under
one shared probe budget, visit budget and pager, so whichever comes back first
is the one that spends the request's budget: two nodes holding identical data,
or one node before and after an unrelated write, could resolve the same CID
differently and one could shed a 503 where the other serves.

The instability is not hypothetical. An unpin and re-pin of a single object,
which is an ordinary production sequence, moves that row to the end of the heap
and rotates the list.

The sibling pin_sources_for_oid already orders its union for exactly this
reason, and the handler comment next to it leans on that determinism.
2026-08-15 21:56:07 -05:00
beardthelion 1bdefb18c2 chore(node): log every /ipfs scan truncation and correct the helper's docs
Code review of the previous commit turned up three things worth fixing in place.

record_scan_truncation centralized the taint and the seal but logged nothing, so
centralizing actually made a truncation less visible than the scattered inline
taints it replaced: only the visit ceiling logged, and it logs from inside the
gate, before the caller decides whether a position gets sealed. Two identical log
lines could therefore mean "the ladder continues" or "the caller is stranded".
One debug line now carries the reason and whether a position was sealed. It logs
only whether one exists, never its value, since the position names a withheld
row's created_at and id.

The doc comment claimed to be "the one site that records a scan truncation" while
eight other taint sites bypass it. The distinction is real but it is not the one
the comment drew: a ceiling stops the scan and owes the caller a position, while
a transient skip refuses one row and the rows behind it are still walked. Says
that now.

The forward-only rule's stated justification was wrong. A later candidate's
position is still ahead of the token the caller echoed, so letting it win would
not move the ladder backwards; it would shrink each rung toward a single row.
Also records that the comparison is Rust byte order while the pager's keyset runs
under the database collation, which can disagree on a non-C collation, and why
that costs a replay rather than a skipped row.
2026-08-15 19:45:08 -05:00
beardthelion 42e6aabc10 Merge origin/main into fix/issue-135-ipfs-cid-tree-gate
Carries #326 (distinct-signer certificate threshold) onto this head.
2026-08-15 18:38:15 -05:00
beardthelion 69cfbd4b3c fix(node): mint a continuation when a ceiling stops the /ipfs scan mid-page
The probe and visit ceilings taint inside gate_and_serve and returned Skip, so
the row they refused and every row behind it were walked past without a verdict
while the resume position was sealed from pager.cursor, the end of the fetched
page. Two ways that stranded content:

- On the final page, `pager.exhausted` breaks ahead of every mint arm, so the
  shed carried no continuation at all. A tokenless search_incomplete is the
  wrapped-scan answer, which tells `gl ipfs get` its ladder is over, so a holder
  on that page was unreachable on every retry.
- Mid-page, the sealed cursor sat past the refused rows, so the resume skipped
  them. The shipped ladder tests all set page_rows == ceiling, which puts the
  break exactly on a page boundary and hides both.

The ceiling now returns GateOutcome::CeilingStop rather than tainting on its way
out, and the scan loop stops there and seals the row in front of the one that was
refused. record_scan_truncation is the single site that records a truncation:
it taints and seals together, and only ever moves the position forward, so a
later oid candidate re-walking the same rows on a spent budget cannot hand back a
token the caller already echoed. The wrapped-scan tail no longer clears a seal,
since a ceiling can stop a resumed scan part way through the last page.
2026-08-15 18:38:09 -05:00
Vasanthdev2004 579eee07bc docs(infra): correct the ignore_changes comment for the new upgrade path
The lifecycle comment still listed image_tag among the values needing a deliberate instance replacement. It is rendered into compose.yaml, which the upgrade command now reinstalls, so apply plus upgrade rolls it out. Only the values user-data writes into .env at first boot stay replacement-only.
2026-08-15 14:12:14 +05:30
Vasanthdev2004 de4d932609 docs(infra,macos): make the opt-out reachable and stop the AWS guide contradicting itself
All four are in the deployment half added last round; none is in the Rust.

The AWS guide told operators two different things
--------------------------------------------------
"Changing configuration" still said editing `image_tag` does not affect a running
instance and routed to `-replace`, while the upgrade section thirty lines above
now reinstalls the rendered compose, which carries `image_tag`. The same
paragraph told operators to edit `.env` and restart — which the new "Why the
upgrade rewrites the compose file" section explains is silently ineffective for
any key the installed compose does not already name.

Split by which file carries the change rather than deleted, because the
distinction is real: values rendered into `compose.yaml` (`image_tag`, ports,
`domain_name`, `db_host`, `icaptcha_*`) roll out with `apply` + `upgrade_command`;
values user-data writes into `.env` at first boot (`public_url`,
`bootstrap_peers`, integrations) do not propagate from tfvars at all and need an
edit on the instance or a replacement. The third case is the one that bites: a
setting the installed compose does not yet name is read from `.env` and dropped,
so it needs the upgrade first.

`--remove-orphans` stopped being inert
---------------------------------------
The flag is not new, but until the upgrade rewrote `compose.yaml` it reconciled
against the instance's own file, so the only orphans were hand-started
containers. It now reconciles against a fresh rendering whose service set is
conditional — `postgres` only when `db_host` is empty, `caddy` only when
`domain_name` is set. Setting `use_rds` or clearing `domain_name` and then
running what the README calls a pull-and-restart stops and removes the local
postgres or the TLS terminator. That is what those variables mean, but it happens
on the upgrade rather than on apply, so it is now stated next to the
Terraform-owned warning. Data survives either way; both bind-mount under
/mnt/data.

The macOS setting had no writer
--------------------------------
`enforceOwnerPush` was persisted, loaded, emitted into `.env`, and forwarded by
the generated compose — and nothing set it. `SettingsWindow` had no state and no
assignment, so the only way to reach the opt-out on that launch path was hand
editing config.json in Application Support, which no document mentions. Editing
`.env` does not work there either: `writeEnvFile()` rebuilds it from `Config` on
every start. It now has a Toggle beside "Sync repos from peers", following the
pattern already in that view.

Three lines the change made untrue
-----------------------------------
The handler test asserted "with no configuration at all" while setting
`enforce_owner_push` explicitly — the right thing for the test, the wrong claim
for the message. The cross-tenant docstring still described three distinct pusher
DIDs after A and B were given `n1`'s owner. And OSS-READINESS-AUDIT listed
unprotected branches not enforcing owner checks as a live blocker; owner checks
now cover every branch, and only the UCAN half of that sentence still stands.

Also dropped the `#[allow(dead_code)]` on `test_state_with`, redundant since the
same commit added its call site.
2026-08-15 11:47:36 +05:30
anandh8x 0b47e1fc08 docs: clarify visibility limits 2026-08-15 10:32:24 +05:30
anandh8x 505c452378 docs: correct security and workspace status 2026-08-15 10:30:51 +05:30
Kevin Codex 50d3cbbe97 Merge pull request #326 from euxaristia/fix/cert-threshold-duplicate-signatures
fix(core): Count distinct signer DIDs in certificate threshold check
2026-08-15 10:49:11 +08:00
beardthelion a7e8e35030 fix(review): make the coupled sites findable and pin two more properties
Code review found the doc comments carrying claims the code does not
support and two coupling points documented from one side only.

The floor and the resolver's walk_cap are not textual twins: the floor
reads the constant, the resolver reads the AppState seam, and they agree
only because every construction seeds one from the other. Say that, and
give walk_cap the back-reference it lacked. The lifted node_verifies
helper no longer named the middleware it mirrors, so an edit to
require_signature would not find either copy.

The request-target clause pins @path, not @method or content-digest: a
301, 302 or 303 still rewrites a signed POST to a bodyless GET while the
signature headers ride along. Record that rather than implying the hop is
safe. Add the query-removed and fragment-only matrix rows, and a mutant
pinning that scan-phase walks are not charged to the work bucket, which
nothing covered.
2026-08-14 19:21:44 -05:00
beardthelion dc5daa14b4 fix(node): size the ipfs work floor for provenance walks plus one full search
The floor reserved one complete legacy search per window, probes plus page
tolls, but the provenance visibility walk debits the same bucket before the
fallback runs and its cap is charged per phase. With a route limit set below
the floor the provenance phase spent from the budget the search was promised,
the fallback 429d short of its configured reach, and the retry re-paid the
same walk charges, so a readable holder past that point stayed unreachable.

Add the walk term, min(MAX_HISTORY_WALKS_PER_REQUEST, ipfs_max_repos_walked),
which is textually the resolver's own walk_cap so the two move together. The
ladder fixture pins repos-walked to 1 to keep the page toll, not the new term,
as the thing binding it.
2026-08-14 17:43:48 -05:00
beardthelion 1c86ee22d6 test(node): pin require_signature to the request-target it received
The verifier already rebuilds @path from the URI it was sent, which is why
the client-side redirect bug surfaced as a 401 rather than as a bypass.
Nothing pinned that, so a change to the reconstruction could drop the query
half or collapse it to a constant and only the clients would notice.

Drive the production router fixture with a signature made over one path and
a request on another, and again with a query mismatch, plus an identically
signed control. No pre-fix RED is obtainable here by construction, so each
case is proven load-bearing by injecting the defect it names.
2026-08-14 15:48:46 -05:00
beardthelion 92bb55441d test(clients): prove the refused redirect never reaches the node verifier
The two same-origin follow tests drove exactly the path-changing hop the
predicate now refuses, and their mocks only proved a target was reached,
never that the signature verified there.

Rewrite both into refusal tests and run the real gitlawb-core verification
over the request the target actually received, so the recorded verdict is
what the assertion speaks about. Each refusal is paired with a positive
control that verifies, so an empty verdict slot is attributable to the
refusal rather than to a dead harness.
2026-08-14 15:10:52 -05:00
beardthelion 5c0d6f3252 fix(core): refuse a redirect that rewrites the signed request-target
may_follow compared host, port and scheme only, so a same-origin hop that
normalized a trailing slash or a query was followed. The signature binds
@path as the client sent it and the node rebuilds it from the URI it
received, so that hop left the signature covering a target nobody asked
for and the read 401d.

Add the request-target clause and repoint the seven matrix rows that rode
on a path change, so each keeps pinning the host or port property it
exists for rather than going false on the path alone.
2026-08-14 15:10:44 -05:00
Vasanthdev2004 1f0d9ff368 fix(node,infra,macos): make the owner-push opt-out reach every launch path
The previous round forwarded GITLAWB_ENFORCE_OWNER_PUSH in three compose
templates. Two of those are not the file their runtime actually reads, so the
documented rolling-upgrade opt-out did not work where it mattered.

Existing AWS instances could not opt out
-----------------------------------------
`aws_instance` ignores user_data drift, so an instance keeps the compose file it
was created with, and the published SSM upgrade only ran `docker compose pull &&
up -d`. Compose passes only the variables named in a service's `environment:`
block, so a key added to the template never reached a running instance: an
operator following our own instructions wrote GITLAWB_ENFORCE_OWNER_PUSH=false
to /opt/gitlawb/.env, upgraded, and got the new default anyway — silently, with
their CI pushers refused.

The upgrade document now reinstalls the rendered compose file before restarting.
It is idempotent (identical bytes on an already-current instance) and it makes
the upgrade a migration rather than a restart. compose.yaml is Terraform-owned
and overwritten; per-instance settings belong in .env, which is untouched.
infra/aws/README.md documents that `terraform apply` must precede the upgrade,
since the SSM document embeds the rendering.

The packaged macOS app read a different compose file
-----------------------------------------------------
The bundled Resources/docker-compose.yml is not what a packaged app runs: with
no source checkout selected, `generateComposeFile()` writes its own template,
whose environment block did not carry the key. `Config.writeEnvFile()` also
regenerates .env on every start, so a hand-added value was discarded. Between
them, a packaged-app user had no way to opt out at all.

`enforceOwnerPush` is now a persisted app setting, emitted into .env on every
start, and `generateComposeFile()` forwards it with the same interpolation as
the bundled resource — one definition, three consistent sites.

A test inherited the environment it was supposed to assert about
-----------------------------------------------------------------
`default_config_refuses_a_non_owner_push` built state through `test_state`, which
parses `Config` from the process environment — 47 fields come from GITLAWB_*
variables. On a host exporting GITLAWB_ENFORCE_OWNER_PUSH=false, which is exactly
what the rolling-upgrade guidance tells operators to set, it constructed a
disabled state, let the push through to git, and failed for a reason unrelated to
the code under test.

Renamed to `enforced_owner_push_refuses_a_signed_non_owner` and given an explicit
config through a new `test_state_with`, so it states the configuration it is
about. Verified by running it both with and without the variable exported. The
shipped default keeps its own env-independent proof off the parser declaration,
in config::tests — a parser question and an authorization question should not
share a failure mode. `test_state` now carries a warning about the hazard, since
any of those 47 fields can skew a test the same way.
2026-08-14 23:35:50 +05:30
Vasanthdev2004 2f0174cc04 fix(node): close review findings on the owner-push default
Every item below was verified against the code before changing it; two of them
were confirmed by running the probe the reviewer described.

A test the fixture sweep missed, and that could not fail
--------------------------------------------------------
`f1_write_cap_is_inert_without_a_resolvable_source_key` seeded `z6f1none` and
pushed as `z6MkF1NoKeyPusher…`, so the owner gate rejected it before it reached
the per-caller cap the test exists to measure. Its assertion was
`!matches!(r, Err(Overloaded(_)))`, which a `Forbidden` satisfies, so it reported
ok while measuring nothing — and it is not `#[cfg(unix)]`, so it did that inside
every green run. Confirmed by inverting the assertion to expect `Forbidden`: it
passed, proving the request stopped at the gate.

The pusher is now the owner, and the assertion is positive: reaching git on the
missing on-disk repo (`Err(Git(_))`) is proof the request cleared both the gate
and the cap. The sibling `f1_write_cap_key_is_per_source_not_per_repo` carried
the same weak negative in its second assertion and is tightened the same way.

The default was barely tested, and what tested it read the environment
---------------------------------------------------------------------
`Config::parse_from` consults the process environment, so on a host exporting
`GITLAWB_ENFORCE_OWNER_PUSH=false` the old `enforce_owner_push_defaults_to_true`
reported the operator's setting rather than what this crate declares — it failed
outright, which would also break any contributor who sets the variable. It is
replaced by two tests that are env-independent and behavioural:

  * `enforce_owner_push_is_declared_true_independent_of_the_environment` reads
    the declared default off `Config::command()`.
  * `default_config_refuses_a_non_owner_push` drives the handler on an untouched
    `test_state` config and asserts 403 — authenticated by a real RFC 9421
    signature, so 403 rather than 401 is what makes it a statement about
    authorization.

Both were checked by reverting `default_value_t` to `false`: the handler test
reports 500 where it wants 403, and the declared test reports `["false"]`.

The cross-tenant test lost its second tenant
--------------------------------------------
The sweep collapsed `u1_parked_push_does_not_shed_another_pusher_behind_the_same
_ip` onto one identity, leaving the test named for cross-tenant behaviour with a
single tenant and two docstrings describing a scenario it no longer ran. `n2` is
now owned by `z6u1nat2` and its push signs as that owner, so two distinct tenants
share one edge IP again — which is the property under test, since the per-source
key is the resolved peer address and never the DID.

Four comments outlived their premise
-------------------------------------
`repos.rs` (3 sites) and `.env.example` still said owner enforcement defaults
off. The IP-keying rationale they support is unchanged and still correct — a DID
farm defeats a DID key — so only the dead premise is rewritten: the cap is
load-bearing because the rate limiter bounds arrival rate rather than occupancy,
because the gate can be turned off for a rolling upgrade, and because one owner
can open many concurrent pushes to their own repo.

Operators on the container path got none of this
-------------------------------------------------
`GITLAWB_ENFORCE_OWNER_PUSH` is now forwarded in `docker-compose.yml`, the macOS
app's bundled compose, and `infra/aws/compose.yaml.tftpl`, defaulting to `true`
to match the node. Without it the rolling-upgrade opt-out this PR documents in
three places was unreachable for anyone running compose.

A cutover this project never promised
--------------------------------------
The README paragraph announcing a 15 September 2026 flip of
`GITLAWB_REQUIRE_SIGNED_PEER_WRITES` is removed. That flag is not changed here,
the date appears nowhere on main, and a PR about a different flag is not where
that commitment would be made. The table row for `GITLAWB_ENFORCE_OWNER_PUSH`
carries the part that belongs.

A docstring that credited the wrong thing
------------------------------------------
The field and test docs claimed the value-taking action is what makes
`GITLAWB_ENFORCE_OWNER_PUSH=false` resolve to `false`. It is not: the env form
resolved under the presence-only declaration too. What the change buys is
`--enforce-owner-push false` parsing at all, rather than erroring as an
unexpected argument. Both docstrings now say so.
2026-08-14 20:43:43 +05:30
Vasanthdev2004 7de6f7cf5d test(node): derive the P2 pusher from the repo it targets
`p2_push` signed every push as one shared `P2_PUSHER` constant, but its two
callers own different repos — `z6p2tail` and `z6p2fail`. Pinning the constant to
either owner leaves the other rejected by the owner-push gate, so the identity
now follows the `owner` argument the helper already takes, the same way the
burst test derives it.

The rejection surfaced as `async fn resumed after completion` rather than a 403:
a denied push completes on the first poll, `p2_logged` never observes
`receive-pack`, and the polling loop then polls the finished future again.
2026-08-14 18:31:00 +05:30
Vasanthdev2004 586896d8b2 test(node): push as the repo owner in the remaining lease and scan fixtures
Fourteen concurrency tests seed a repo through `f4_state_with_repo` or
`p2_parked_release_state` and then push under an unrelated DID, which the
owner-push gate now rejects before the machinery each test asserts on is ever
reached. They are transport and lease tests, not authorization tests, so each
now signs as the owner of the repo it targets.

Two needed more than a swapped constant:

- `receive_pack_burst_scans_serialized_and_both_pushes_succeed` pushes to two
  repos with different owners, so the identity is derived per push from the
  owner already threaded through the closure.
- `u1_parked_push_does_not_shed_another_pusher_behind_the_same_ip` used three
  distinct pusher DIDs, which owner-only push makes impossible. The distinction
  was narrative: the per-source key is the resolved peer IP and never the DID,
  so three concurrent pushes from one edge IP contend exactly as before. The
  comment now says so rather than leaving the reader to infer it.

The gate keeps its own coverage in `enforced_rejects_non_owner_with_forbidden`
and `enforced_rejects_missing_did_with_forbidden`, so nothing here removes the
assertion that a non-owner push is refused.

These tests are `#[cfg(unix)]`, so a Windows checkout cannot run them; they were
missed in the previous round because the Windows suite runs 717 of the 825 tests
Linux runs.
2026-08-14 18:00:36 +05:30
Vasanthdev2004 cdbda19c29 fix(node)!: enforce owner-only push by default
A `did:key` signature is authentication, not authorization: the method is
self-certifying, so any party can generate a keypair, derive its DID and sign.
With `GITLAWB_ENFORCE_OWNER_PUSH` defaulting to `false`, `owner_push_rejection`
short-circuited and every `git-receive-pack` carrying any valid signature was
accepted — including pushes to a repository the signer does not own, and
including private ones, on every branch not explicitly protected.

The gate itself already worked when switched on. What was missing is that no
reachable default configuration switched it on, so a node started with nothing
configured accepted a push from anyone. That default is what this changes.

The argument now takes a value so there is a way back: `--enforce-owner-push
false` and `GITLAWB_ENFORCE_OWNER_PUSH=false` both disable it, while the bare
`--enforce-owner-push` form still means `true`. Without this the flag would be
presence-only (`ArgAction::SetTrue`), which with a `true` default would leave
operators no way to opt out during a rolling upgrade.

Three concurrency tests pushed as non-owners in order to reach the shedding
logic they actually assert on. They now push as the repo owner, so they keep
exercising the shipped default rather than opting out of the gate.

Docs updated in README.md, .env.example and docs/RUN-A-NODE.md, including the
delegated-key caveat below. README also carries a dated cutover for
`GITLAWB_REQUIRE_SIGNED_PEER_WRITES`, which stays `false` for now so live peers
can finish upgrading.

BREAKING CHANGE: `git-receive-pack` now rejects a push whose authenticated DID
is not the repo owner, returning 403 before any ref update is applied.
Delegated and CI keys count as non-owners: a UCAN `git/push` capability is
verified but not yet honored for authorization, so an agent pushing under its
own DID cannot push while this is on. Set `GITLAWB_ENFORCE_OWNER_PUSH=false`
during a rolling upgrade, or have automation push as the repo owner, until
scoped collaborator / UCAN-delegated push rights land.
2026-08-14 17:22:23 +05:30
Vasanthdev2004 5a7a77b8dc fix(node): gate two Unix-only tests so the crate compiles on Windows
Two tests build a fake git as a `/bin/sh` script, mark it executable through
`std::os::unix::fs::PermissionsExt`, and reap the hung `rev-list` with
`libc::kill(SIGKILL)`. None of that exists on Windows, and neither test was
cfg-gated, so `cargo test -p gitlawb-node` failed to compile there with six
errors before running anything. A Windows checkout could not run a single test
in the crate, including the ones that are platform-independent.

Gate both with `#[cfg(unix)]`. The attribute is a no-op on Linux, so CI keeps
running them exactly as before; only the Windows build changes, from "does not
compile" to "runs the platform-independent tests".

Refs #228
2026-08-14 17:22:23 +05:30
beardthelion c82a70ab21 fix(core): put the redirect predicate behind a feature so core stays lean
The dependency allowlist caught this: sharing the predicate pulled url, and
behind it idna and the icu crates, into the closure of the one crate that is
supposed to stay embeddable. Both clients that need the predicate already parse
URLs, so they opt in and nothing else pays for it.

The test dependency is not optional, so the origin matrix still runs under a
bare cargo test -p gitlawb-core. Gating the module on the feature alone would
have left those tests silently unbuilt, which is the failure the module is
there to prevent.
2026-08-13 18:55:58 -05:00
beardthelion ba44a018ec fix(core): share the origin-scoped redirect rule with the git helper
The previous commit stopped the CLI carrying request signatures to a host the
node redirected it to, and left the other signing client alone. That one is the
git remote helper, which is what actually runs on clone, fetch and push, and it
had no redirect policy at all. A push signs from its first request, so a node
answering 302 handed out a working credential, and on a 307 or 308 the pack
body went with it. The rule now lives in core and both clients use it, so the
two cannot drift apart again.

Three guards were not holding what they claimed. Bounding the retry wait by the
remaining deadline could be deleted with every test still green, which the
timing shows was four real seconds of overshoot past the bound the docs
advertise; the assertion was loose enough to pass either way and is now tight
enough to fail. The refusing-gateway warning was asserted through a test-only
mirror rather than the write the user sees, so deleting the write kept the test
green. And both no-hint exceptions, the definitive not-found and the wrapped
scan, were only ever reached without a token in hand, which is the one state
where they do nothing.

A response whose body failed partway through the read still replayed the held
token for every remaining attempt, spending the whole ladder on one position and
then reporting that the object might lie further on. A body cut by the read cap
was already terminal for that reason; a body cut by a broken connection is the
same unreadable state and is terminal now too.

A successful fetch no longer buffers the whole object before writing a byte of
it, so a node cannot make the client hold an arbitrary amount of memory inside a
time limit that was never a memory limit. The remaining silent skips in the
recovery loops now say what went wrong, including the manifest fetch and the
body reads, so a failure there is reported rather than looking like an absent
blob.

Also: the redirect chain bound is off by one against reqwest's own, has a test,
and its comment no longer claims to prevent something the request timeout
already prevents.
2026-08-13 17:34:14 -05:00
beardthelion de3f230940 fix(gl): stop carrying request signatures to a host the node redirected us to
The node client followed redirects with no policy, and reqwest strips only the
classic credential headers when a redirect crosses origins. RFC 9421 signature
headers are not on that list, so a node answering 302 could hand a caller's
signature to another host, which then served whatever bytes it liked for a
content address that never hashed to them. Following continuations made it
worse by turning one signed request per invocation into as many as nine. The
signature covers the method, the path and the body digest but not the
authority, and the node accepts any signature inside a five minute skew with no
nonce ledger, so one harvested header reads a caller's private paths at any
node in the fleet.

Redirects are now scoped to the origin that issued them. Same-origin ones still
follow, so an http to https upgrade or a trailing-slash normalization keeps
working, and a cross-origin one stops rather than errors, so the caller sees
the 3xx through the status path it already has. A downgrade back to http on the
same host is refused too: the origin matches, but the signature would leave in
cleartext, which is the same leak by a slower route. A custom policy replaces
reqwest's built-in chain limit, so the limit is restated rather than lost.

The recovery path in clone had the same silence the CLI just lost: it skipped a
blob on any non-success without saying why, so a truncated scan, a permission
denial and a dead gateway were one indistinguishable outcome. It now names the
status or the transport error, capped and stripped of control characters since
the text comes from a caller-chosen gateway, and continues as before.

Reading a response body no longer conflates three endings. The cap cutting the
body, the body finishing, and the read failing partway were all one silent
break, so a failed read surfaced as an error message that stopped after the
colon. Each is now reported, and a body that could not be read says so.

Coverage for three paths that were previously argued rather than run: a
transport failure mid-ladder still hands back the resume token, a scan that
ends before any ceiling still answers a definitive not-found, and a response
whose body stalls after its headers is still ended by the client's own
deadline.
2026-08-13 16:10:04 -05:00
beardthelion 8c39cfdcb3 fix(review): keep the resume token on every terminal that still holds one
Review of the scan-continuation work found the ladder surfacing its token on
three terminals and dropping it on three others: a mid-ladder 5xx that is not
an overload, a truncation whose offered continuation fails validation while a
good token is still held, and a transport error. In each case the user is told
to start over, and a bare re-run restarts at the first row and re-spends the
node's per-IP budget, which is the failure the token exists to prevent. It is
surfaced on those paths now, still never on a definitive 404 or on the node's
deliberate scan-wrapped signal, and a rejected token is still never echoed
back.

A body cut at the read cap used to change the answer. Truncated mid-JSON it
failed to parse, lost its error code, and fell to the transient-overload arm,
which retried the old position for every remaining attempt and then reported
that the object might lie beyond the rows scanned when nothing past the first
page had been. The cap now reports that it cut, and a cut body is terminal.

Three guards were doing nothing. Mutation testing showed the Retry-After clamp
could be deleted with every test still green, so a node advertising a day-long
wait would have parked the client inside a single sleep that neither the
attempt cap nor the deadline can interrupt. The rejection of a malformed
caller-supplied token was equally untested, though the same property is proven
on the node-offered side. The withheld-blob ladder asserted only that each rung
was not a success, which any early exit satisfies without the rule ever being
evaluated; it now witnesses that the scan reached the end of the table.

The inter-attempt wait is bounded by the time left on the deadline, so a run
can no longer overshoot the bound it advertises. That drops the wait out of the
worst case entirely rather than shrinking it: the last attempt starts before
the deadline and the client's total timeout covers that request through the end
of its body, so a run is over by the deadline plus that timeout. The docs said
about 95 seconds and now say about 90.

They also said a large download already streaming is never cut off part way.
That was wrong. The scan deadline does not cover the download, but the client's
timeout is a total request deadline running until the body finishes, so a slow
transfer is cut off. Both prose sites now say what the code does.

Also: stdout is flushed before the command returns, so a closed pipe fails
loudly instead of yielding a truncated object with a success status.
2026-08-13 15:09:45 -05:00
beardthelion eac4d43c67 docs(gl): describe how gl ipfs get resumes a truncated scan, and its limits
Document the behaviour rather than the mechanism: that an object pinned before
the node recorded its repo is found by scanning the inventory, that the scan
stops at the node's ceilings, and that the command follows the resume token the
node hands back instead of reporting a false not-found.

State the bounds in values, not constant names, and state honestly what each
one covers. The deadline bounds the search: every attempt gets the time left on
it to produce response headers, and it deliberately does not cover downloading
an object once found, so a large blob already streaming is never cut off. The
error-body read runs outside it under the client's own timeout, which is why
the longest a single run can take is about 95 seconds rather than the flat
deadline, and the in-code comment now derives that instead of asserting it.

Say what is not promised, too. A rate limit ends the ladder because its window
is an hour and its Retry-After cannot be honored inside one invocation, and the
node's per-IP fanout brake can end a ladder well short of the cap, so automatic
resumption is not a guarantee that the object will be reached.

Where a bound stops the ladder, the printed invocation is the only thing that
makes progress: a bare re-run restarts at the first row, reproduces the same
truncation and spends the per-IP budget again.
2026-08-13 14:11:59 -05:00
beardthelion 181f6178a0 fix(gl): accept the resume token the incomplete-scan message tells you to pass
The bounded terminals printed "gl ipfs get <cid> --scan <token>", but the
command took no --scan argument, so following that advice failed on an
unexpected argument and the resume input existed only behind a private seam.
The flag now exists and forwards to the same validation a node-supplied token
gets, which is what makes surfacing the token worth anything: without it a
stopped ladder restarts from the front on every rerun.

Three test fixes ride along, all found by running the mutation matrix rather
than by reading.

The deadline scenario asserted its message "must name the deadline" while its
fixture CID was bafkreideadline, so any message echoing the CID satisfied it.
Under a mutant that removed the deadline the ladder ran to the cap and that
assertion still passed. The CID is renamed so the assertion can only pass when
the message genuinely names the deadline, and the mutant now reddens on it
instead of on the call count.

The default classification arm was certified only for the token-less case: no
fixture reached it with a token already held, so degrading it to retry was an
equivalent mutation against the whole suite. A scenario now drives an
unrecognized code mid-ladder with a token in hand and asserts it is terminal.

The wired path from the argument to the resume input had no coverage; only the
seam was tested. A scenario now goes through the function clap dispatches to,
so dropping the argument turns it red.
2026-08-13 14:02:34 -05:00
beardthelion 9a8d190492 fix(gl): follow scan continuations in gl ipfs get, bounded three ways
A truncated legacy CID scan answers 503 with a sealed continuation token, and
the command turned that into a hard error, so content past a scan ceiling was
unreachable from the first-party client. It now follows the continuation
automatically.

Three bounds, each protecting a different resource. The attempt cap bounds the
work asked of the node. A wall-clock deadline bounds the caller's own time and
is passed into each request as that request's timeout, so a slow-but-alive node
cannot outlast it. The per-wait clamp bounds a single Retry-After, so a hostile
value cannot stall the client. There is deliberately no non-advancing-token
check: tokens are nonce-randomized ciphertext, so two seals of one position are
never byte-equal and a client cannot detect a wedged ladder by comparing them.

Classification happens at one site with no wildcard to retry. Status is checked
before the body is read. A rate limit is terminal on status alone, since its
window is an hour and its Retry-After cannot be honored inside one invocation.
An overload mid-ladder is retried on the still-held token, because that shed
releases its permit when the request ends and the node advertises a retry. Any
other status or code is terminal through the default arm, including an overload
on the first request, where no token is held and there is nothing to resume.

Every terminal message is capped and sanitized before it reaches the terminal,
and a rejected token is never echoed back; the bound is named instead. Tokens
are accepted only as non-empty base64url within a generous ceiling, so a future
change to the sealed layout cannot silently disable resume.

Where a bound stops the ladder, the still-held token is printed with the
invocation that resumes from it, rather than wording that implies a bare re-run
makes progress.

gl suite 332 green, fmt and clippy clean.
2026-08-13 13:47:41 -05:00
beardthelion 1d37cb1ce9 test(gl): cover the scan-continuation resume ladder in gl ipfs get
The node answers a truncated legacy CID scan with a 503 carrying a sealed
continuation token, and gl ipfs get turns every non-success into a hard error,
so content reachable only past a scan ceiling cannot be fetched with the
first-party client at all. The old test asserted that terminal 503 as the
desired behaviour, and its fixture body predates the token, so it is replaced
rather than adjusted.

Thirteen scenarios pin the contract: a resume that completes, the attempt cap,
a wedged server that keeps minting fresh tokens for the same position, a
missing or malformed token, a first-request overload, the signed resume, the
wall-clock deadline, a mid-ladder rate limit that is terminal, a mid-ladder
overload that is retried, the token length boundary pair, the token surfaced at
every bounded terminal, and a caller-supplied continuation that starts from the
token rather than the front.

Hostile-input scenarios assert the surfaced message is capped and stripped of
control and bidi characters and that a rejected token is never echoed back.
Ladder fixtures answer Retry-After: 0 so the clamped waits are zero and the
whole gl suite still runs in under a second in real time.

The seam signatures land here so the tests compile and fail on their
assertions: cmd_get delegates to cmd_get_inner, which threads a deadline, an
attempt cap, and an optional starting continuation. Its body is unchanged and
still bails on any non-success, so all fourteen new tests fail on the property
each one names.
2026-08-13 13:13:29 -05:00
beardthelion 21e6873683 fix(node): bound the legacy scan's query by the remaining row budget
GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS accepts any value from one upward, but the
pager always asked the database for a full page and the ceiling was only
checked afterwards, in the scan loop. With the ceiling below the page size an
anonymous request still selected and rule-loaded a whole page while holding
walk admission, and any ceiling that is not a page multiple overshot by up to
a page. Compute the remaining budget first and ask for min(page, remaining),
so the page size stays a batching detail rather than an implicit floor under an
operator ceiling.

The exhaustion inference moves with it. A short page used to prove the table
was exhausted; once the limit can be shortened by the budget it proves nothing,
and marking the pager exhausted breaks at the top-of-loop arm that sits ahead
of every ceiling arm, taints nothing, and mints no continuation. That path ends
in a definitive 404 for content that exists, so the predicate now compares the
returned rows against the limit actually sent.

The budget computation sits above the timeout opener so the deadline-wrapping
guard's lookback window is unaffected, and the limit is read from the AppState
ceiling the row-ceiling arm reads, not the config field, so the arm and the
query cannot disagree.

Full suite 1059 green, fmt, clippy, and --locked clean.
2026-08-13 12:39:05 -05:00
beardthelion a60026bbfa test(node): pin the legacy scan's DB-facing row selection to the configured ceiling
The row ceiling is enforced in the scan loop, after a full page has already
been fetched and rule-loaded, so a cap below the page size is ignored for the
first fetch and any cap that is not a page multiple overshoots by up to a page.

The existing coverage could not catch that. Both row-ceiling tests asserted
rows <= ceiling + one page, which passes whether or not the query is bounded,
and rows-returned cannot distinguish a bounded query from a full fetch that is
trimmed afterwards. Add a scan_limit seam recording the limit actually sent to
the database, tighten both tolerant assertions to exact counts, and add two
tests that pin the limit: a ceiling below the page size and a ceiling that is
not a page multiple.

Also add the fail-open fixture. A repo evaluated with a partial rule set is
served as if it had no rules, so the boundary repo is seeded public at root
with a path-scoped rule and a real blob on disk, pinned with NULL provenance so
the scan path's rule map is the one consulted, and asserted never served across
its own ladder. Its positive control serves the same blob with the rule absent,
so a failure there is attributable to the rules rather than to a probe or a
budget skip.

Both new tests fail against current code on the scan_limit assertion.
2026-08-13 11:54:41 -05:00
beardthelion 75c0e282af test(node): probe the production advisory-lock key, not a stale copy
Both write-lock tests hand-copied advisory_lock_key's derivation. The copy
still used DefaultHasher, so once the key moved to SHA-256 (#210) the probe
checked a key nobody held. The disconnect test then failed asserting the
lock was held, and the success test passed asserting a release it never
actually observed.

Import the production function, as the acquire-deadline test already does,
so the two cannot diverge again.
2026-08-13 09:55:24 -05:00
beardthelion f4f0f4bff0 Merge remote-tracking branch 'origin/main' into fix/issue-135-ipfs-cid-tree-gate 2026-08-13 09:44:35 -05:00
beardthelion e6cfb588c2 fix(review): bound token key, rule bytes, and the discovery load
An external review found three costs that outlive the request or the boot
that paid for them.

The scan token was sealed with a key minted per boot, so a restart or a
rolling deploy made every outstanding continuation undecryptable and the
handler reads an unopenable token as absent. A caller walking a large
inventory could be sent back to the front indefinitely. The key is now
derived from the node's Ed25519 seed with HKDF, so it is the same after a
reload, under a distinct salt and info string so it is not the signing
key and cannot be turned against signatures. Rotation is a version bump in
the info string. An earlier comment rejected derivation on the grounds
that surviving restarts bought nothing; that is exactly what it buys, and
the comment now says so.

The rule byte ceiling was summed after the page had already been fetched,
so it truncated the request without bounding the work: the allocation and
the transfer had happened. The budget now bounds the query itself. The cut
lands on a repo boundary, which is not cosmetic: a mid-repo cut leaves a
partial rule set, and a partial rule set at the gate is indistinguishable
from a repo with no rules, so it would fail open and serve an object those
rules withheld. Only fully loaded repos are gated. The query always admits
the first rule-carrying repo of a page, so the bound is the budget plus
one repo rather than the budget; without that a single oversized repo
would put the cut at the cursor and every retry would reproduce the same
page.

The old check carried a not-exhausted condition so a short final page
could not turn a complete scan into a permanent shed. That condition is
gone because the taint no longer keys on page length; it keys on the query
having left repos unloaded. A short final page that fits produces no cut
and still 404s, and one that is cut is honestly incomplete and resumable.

Discovery paged the whole repo table and stat'd every warm repo to pick a
sixteen candidate window. Its comment called that acceptable because the
cost was paid once, which was true when the sweep ran once per boot. This
round made the sweep periodic, so an upgraded node with one unrepairable
row paid a full table scan on every run, forever. The load now stops once
the window is full. It collects one candidate past the cap so that running
out still distinguishes a warm set that fits from one that does not, which
is what the reset arm needs now that the total count is no longer known.
Ordering is unchanged, so an attacker who mints repos still cannot move
the boundary.

Verified by mutation: a per-boot key strands a continuation across a
restart, using the signing seed directly defeats the domain separation,
summing bytes after the fetch materializes the whole page, paging to
exhaustion buys pages the window cannot use, and reading the fit under cap
from the window length loses the reset.
2026-08-12 23:38:15 -05:00
beardthelion 342dd48d24 fix(review): mint a continuation at every ceiling and widen the token
Code review found two ways the scan ladder still dead-ends, both the same
shape the round has been fixing: a truncation that sheds without a token,
which this design reads as "the ladder is over", so the caller restarts at
the front forever and everything past that point is unreachable.

The continuation was wired into the row and rules ceilings but not the
probe and visit ceilings, and those two bind first on any inventory with
root-readable repos, long before the row ceiling. Every fixture seeded
root-denying repos, so no probe was ever charged and no test could reach
the arms that were missing it. All four breaks stop at the same cursor, so
all four now carry it.

The token's field width was 128 bytes against a repo id the write path
admits at 356 (255 owner, a slash, 100 name); a plain full DID with a long
name already passes 128. A repo over the width at a truncation boundary
failed the seal and shed tokenless, and repo names are peer-controlled, so
the boundary was steerable rather than unlucky. Fields are now sized
separately, 64 for the timestamp key and 384 for the id, and the version
byte moves so older tokens fail open to absent instead of mis-parsing.
Tokens stay constant length at 668 characters.

The sweep no longer exits for good on a failed pass. Making it periodic
was the point of the round, and returning on the first error meant one
deadlock disabled repair until the next boot, silently. It re-arms on a
longer interval, escalates the log once failures are consecutive, and
backs off again when a run repairs nothing so a table that will never
repair stops costing 64 object reads every five minutes.

The rule ceiling counts retained bytes instead of rule rows and is checked
where the rules are actually fetched, since a page can carry unboundedly
many rules and rule count is the wrong unit for a memory bound. It skips a
short final page: without that, a complete scan of an absent object turns
into a permanent 503.

Two docs contradicted the code and are corrected: the walk cap is charged
per phase now, so a request can spend twice the knob, and the README's
retry bound holds when the row ceiling binds.

Verified by mutation: reverting either ceiling's continuation strands the
holder, narrowing the token width fails the seal at the boundary,
restoring the terminal return stops the sweep, counting rules instead of
bytes lets a page overshoot, and removing the idle backoff keeps a
fruitless table under the base interval.
2026-08-12 19:12:24 -05:00
Kevin Codex 96d8123e0f Merge pull request #320 from euxaristia/fix/like-prefix-escaping
fix(node): Escape LIKE wildcards in certificate prefix search
2026-08-13 06:22:26 +08:00
Kevin Codex c380f9cee5 Merge pull request #215 from Gravirei/fix/issue-210-stable-advisory-lock-key
fix(repo_store): use SHA-256 for stable advisory-lock key (#210)
2026-08-13 06:21:50 +08:00
beardthelion 0d7711c089 fix(node): bound the pre-walk CID resolve on its own short budget
get_by_cid takes the global walk permit and the per-source permit before
resolving the CID, and that lookup was clamped only by the 600s request
budget. A well-formed CID with no pinned row does no probe and no walk,
but against a stalled pool it held a scarce walk slot for the full
window, so requests from enough distinct sources could reject every real
retrieval at admission while nothing was walking.

The lookup now runs under its own budget, defaulting to ten seconds and
clamped by the request budget, so a larger value degrades rather than
extends. Admission still comes first. Resolving before admission was the
other way to fix this and it trades one amplification for another: taking
the read out from behind the gate lets unadmitted anonymous callers stack
concurrent queries, and shedding a flood cheaply before it reaches the
database is what admission-first is for.

Only that one await moves. Everything after it stays on the request
budget because it runs interleaved with work the caller has already been
admitted for: the per-oid source lookup runs after real probing from the
second candidate on, the marker pair only on a provenance miss, and the
pager fetches between pages that cost probes. A short clock anchored at
admission would be long spent by then, so widening it would shed
requests that are making progress. The region comment now carries that
assignment per await.

The shed names the knob an operator can actually turn, separately from
the request-budget shed, so the two are distinguishable in the body and
the log.

Verified by mutation: reverting the clamp overruns the short budget, and
anchoring one short clock across the whole permit-held region sheds a
slow but progressing walk. The second leaves the first test green, which
is what makes it a boundary case rather than a second removal.
2026-08-12 16:59:02 -05:00
beardthelion ae96ffa467 fix(node): budget provenance and fallback walks separately
MAX_HISTORY_WALKS_PER_REQUEST is MAX_PIN_SOURCES + 1, which is exactly
enough for the first pinner plus every recorded additional source. Those
seventeen sources each spend a walk reaching their deny when they are
root-readable but path-scoped, so the walk budget is gone by the time the
at-cap fallback reaches a public source that record_pin_source dropped at
its sixteen-row cap. If that source needs a walk too it is skipped at the
ceiling and the request returns 503 search_incomplete on every retry: a
public object nobody can ever fetch.

The counter is now per phase. gate_and_serve already knows which phase it
is in, so provenance and fallback each get their own budget against the
same cap, and the fallback's capacity no longer depends on what the
provenance phase spent. Total walk work per request stays bounded at
twice the cap, a constant, so the amplification story holds.

Raising the shared ceiling was the obvious alternative and does not work:
the adversary picks how many provenance slots exist, since they are
grindable repo ids filling pin_repo_sources, so cap + N denials recreate
the exhaustion for any constant N.

The test uses real repos with real path-scoped rules that each spend a
real walk, proven by the walk log. The existing buried-source test seeds
repos that do not exist, which never enter the walk block at all, and
that is why it could not see this.

The must-not assertion needed changing to mean anything. A CID is
content-addressed, so every holder of it has byte-identical content and
comparing the served bytes against the public repo's object cannot show
which repo served. The gate is pinned instead by a second object held
only by a path-denying repo, with the fallback armed for it, asserted to
404 for an anonymous caller.

Verified by mutation: re-merging the counters restores the deterministic
503; capping the fallback at what the provenance phase left over is a
reserve that still couples them and fails the same way; and dropping the
fallback cap entirely unbounds the total. Each reddens exactly one of the
two tests, so neither is standing in for the other.
2026-08-12 16:27:23 -05:00
beardthelion 22f4db94b2 fix(node): bound the legacy CID scan and let the caller carry its position
A denial-only inventory paged through the whole repo table. gate_and_serve
returns Skip on quarantine and on the root visibility deny before either
walk.probes or walk.visits increments, and those two counters were the
pager's only stop conditions, so an anonymous caller holding a CID from
the public pins index could read every row and its rules while the global
IPFS walk admission stayed held for the request budget.

Three parts, and the shape of each is set by a way an earlier attempt was
wrong.

A row ceiling, with a rules companion. Rows scanned bounds the DB reads
and the pager's row retention; it does not bound rules, since the rules
query has no LIMIT and an owner picks how many rules their repos carry, so
that gets its own ceiling. Both taint into the existing retryable 503 and
never a 404.

A ceiling alone would stop there and strand content. The scan starts from
a NULL cursor over (created_at, id) ASC, so a bare ceiling cuts the same
oldest prefix on every request and a holder past it is unservable for
good, which is the defect this PR already carries a finding about. So the
503 hands back the position. Server-side alternatives were tried and
dropped: a node-global cursor lets concurrent callers skip each other's
windows, and a per-caller map lets a key farm evict an honest caller
mid-ladder. State the caller carries has neither problem.

The token is sealed, not encoded. It names a row the caller may not be
allowed to see, and id derives from the owner DID, so plaintext would leak
a withheld repo's existence, creation time and owner. XChaCha20-Poly1305
with a fresh nonce per seal, the canonical CID as associated data, an
expiry inside the sealed bytes, and a fixed-width plaintext so token
length cannot vary with the row it names. Undecryptable, expired,
malformed and foreign-CID tokens are one indistinguishable case: absent,
start at the front.

The wrap taint keys on the pager reaching exhaustion while resumed, not
on any break site, so a token pointing at the last row still sheds a
retryable 503 instead of falling through to the 404 tail.

Pages are charged to the per-IP work bucket. Without that the ceiling
bounds one request and nothing across requests, and the denial path pays
nothing, which is the per-request-free-allowance shape already rejected
once on this branch. The derived work-budget floor grows by the same page
term so a deep scan cannot self-throttle mid-ladder.

A toll shed renders 429 rather than 503, which reverses the tail's
documented precedence for this one case. The tail comment names the
exception: the caller's own bucket stopped the scan, not the node's
search, and a 503 would invite an immediate retry into an empty bucket.

Four existing tests hand-size the work limiter to their exact probe count
and now also pay for a page; each gained one token and a comment showing
the arithmetic. Production is unaffected because the floor grew by the
same term.

Verified by mutation: removing the ceiling unbounds the scan; truncating
without tainting turns it into a 404; emitting a token but never resuming
never reaches the holder; dropping the toll makes pages free; encoding
the token as base64 plaintext leaks the withheld fields; dropping the CID
from the associated data honors a cross-CID resume; a variable-width
encoding makes token length vary with the row; and a constant nonce makes
two seals identical.
2026-08-12 15:49:51 -05:00
euxaristia 3993fd1f5d fix(core): count distinct signer DIDs in certificate threshold check
satisfies_threshold counted raw signature entries against the maintainer
list, so a certificate carrying the same maintainer's signature duplicated
across two entries satisfied a 2-of-N threshold with only one real
signer. Dedupe by collecting valid signers into a set before comparing
against the threshold, so each maintainer counts once regardless of how
many times their signature appears in the certificate.
2026-08-12 15:44:06 -04:00
beardthelion 9babb5ece0 fix(node): record a discovered source and its marker in one commit
Discovery recorded the source and armed the fallback in two independent
best-effort writes. If the insert landed and the marker did not, the row
was left raw CIDv1 with a nonempty, below-cap, unmarked source set. Later
sweeps skip such a row, and get_by_cid reads that set as complete and
suppresses the fallback scan, so once the transient error cleared, a
public duplicate of an object whose recorded holder is private stayed
unreachable for good.

record_discovered_pin_source does both writes in one transaction: the
capped source insert, the gated failure-row delete, and the sentinel that
arms the fallback. The sentinel is unconditional because one holder found
by a bounded warm-only probe never proves the set complete.

The method ends in an explicit commit, so an elapsed deadline is a
definite non-write rather than an unknown. A failed record still leaves
an empty source set, which the resolver's own empty-set signal routes to
the fallback, so the degradation posture is unchanged and now has a test.

The fault injection is a BEFORE INSERT trigger on pin_source_failures,
not a dropped table. Dropping it would fail the delete inside
record_pin_source's own transaction, roll that back, and produce the same
empty set as the fix, so the test would pass without ever reaching the
half-state.

Verified by mutation: splitting the transaction back into two writes
404s the public copy, and dropping the sentinel leaves the fallback
unarmed.
2026-08-12 14:13:17 -05:00
beardthelion 4fdfa1cedb fix(node): rotate discovery candidates across traversals
Discovery probed the first sixteen warm candidates of a stable
(created_at, id) ordering on every pass, so an object held only by the
seventeenth repo was never repaired and every boot repeated the same
work. The probe cap bounds expensive reads, which is right; it was also
silently bounding which candidates are ever considered.

A persisted continuation now rotates the window. Four choices carry it,
each forced by a way an earlier draft was wrong:

Keyset, not offset. An offset indexes into a list whose length anyone can
change, so candidates entering below the boundary shift the window off the
holder. A key names a position that survives insertions. created_at is
server-stamped, so minted repos only ever append.

Per traversal, not per pass. The row cursor also advances per pass, so
per-pass rotation gives row R the window index (t*m + j) mod W and a row
sees only W/gcd(m, W) of the W windows. Sharing one window per traversal
makes coverage a per-row property.

Advanced only over probes that started with a live deadline. A probe
against a spent deadline is charged a read but never opens the repo, so
counting it would step the window over candidates nobody examined.

Persisted from inside sweep_pass, next to the row cursor. The run-end site
never executes: spawn_legacy_cid_sweep drops the future on SIGTERM.
Traversal state is owned by the wrapper because the dead-read cap can pause
a run mid-traversal and a later run ends it.

The sweep re-arms on a timer instead of running once per boot, so coverage
is wall-clock rather than a reboot count, and returns only on a failing
pass query. The unconditional sleep is what keeps a repaired table from
spinning.

Verified by mutation: no rotation strands the holder; an offset
continuation shifts the window; advancing to the window end both skips an
unread holder and burns an unprobed window; per-pass advancement drifts
one row's window from another's; no re-arm leaves the holder unreached; a
run-scoped accumulator applies the wrong arm after a cap pause; and
deleting the v26 entry fails the upgrade path.
2026-08-12 13:46:03 -05:00
beardthelion 22560b0872 fix(node): give each discovery row its own slice of the pass budget
DiscoveryCtx held one deadline for a whole sweep pass, so a single hung
candidate on the first source-less row spent it for every row behind that
one. Those rows then called repair_legacy_provider_cid with an expired
deadline, came back retryable without a meaningful probe, and were charged
reads for work they could not do. CID order is stable, so the same first
row consumed the budget on every boot and the rows behind it starved
permanently.

Each row now probes under min(now + git_timeout / 4, pass_deadline), so at
least four rows per pass get a live deadline and no row can take more than
a quarter. A row that starts with the pass deadline already spent returns
PassBudgetSpent: no probe, zero reads charged, retryable, and surfaced on
SweepStats so a run can say it happened. Charging those reads would burn
MAX_DEAD_ROW_READS_PER_RUN on probes that cannot succeed.

The field is renamed to pass_deadline because that is what it is:
load_discovery_ctx runs once per pass and a run loops passes. Naming it
for the run would have stated a bound the code does not hold.

The divisor is not a bound on a single probe. A row's slice is still
shared by up to MAX_LEGACY_DISCOVERY_PROBES candidates, and the doc says
so rather than implying a guarantee.

Verified by mutation: collapsing the row deadline back to the pass
deadline starves the second row, and folding the spent-budget arm into
ordinary retryable accounting charges reads that must stay free.
2026-08-12 12:20:19 -05:00
beardthelion 650460acc0 test(node): cover the pre-provenance upgrade row end to end
The sweep's source-less discovery had no fixture built from a real
pre-migration row. Every existing test seeded its pin through the modern
path, which cannot reproduce the shape an upgraded node actually carries:
a row written when pinned_cids.repo_id and pin_repo_sources did not exist
yet, so its source set is empty with no other signal.

The fixture un-applies migrations 19 and 20, inserts the provider-keyed
pin naming only (sha256_hex, cid, pinned_at), then migrates forward. That
ordering is what makes it real; seed_legacy_pin binds repo_id, the column
being proven absent, so the insert is inline.

It brackets the repair rather than asserting an end state: the anonymous
GET is driven before the sweep and asserted not to serve, then after the
sweep asserted to return the object's bytes. The marker assertion queries
pin_source_failures directly for the "" sentinel instead of reading the
pin_sources_incomplete bool, which cannot distinguish the sentinel from a
repo id that record_pin_source would later clear.

Verified by mutation: short-circuiting the sources.is_empty() discovery
arm restores the pre-fold fall-through and the test goes red on the
repair assertion.
2026-08-12 11:56:56 -05:00
Gravirei b01d17b7f3 docs(repo_store): pin colon-free owner_slug contract on advisory_lock_key
The domain separation owner_slug + ':' + repo_name has no length prefix, so
it is injective only while owner_slug cannot contain a colon. State that on
the pub(crate) function and guard it with a debug_assert, since a raw DID
would collide (('did:key:abc','x') and ('did','key:abc:x') hash the same).
2026-08-12 13:07:50 +06:00
beardthelion 4c665fbca8 fix(node): port discovery's candidate load onto the paged repo query
#173 removed Db::list_all_repos in favour of a keyset-paged
list_repos_page_for_scan, so this branch stopped compiling on rebase. One call
site, load_discovery_ctx, ported with its reviewed semantics intact.

The quarantine drop gets simpler rather than different: each page row carries its
own quarantined flag, so the separate list_quarantined_repos query is gone and
the filter runs over rows the pass already read. Private non-quarantined repos
still stay in the list. Warm-only via validated_repo_disk_path inside
spawn_blocking is untouched, so nothing here reaches repo_store.acquire and a
repair pass still cannot become a bulk restore, and an unsafe path is still
dropped with the same warn and stays terminal.

Oldest-first by (created_at, id) now comes from the query's ORDER BY (index
backed by migration v25) instead of a client-side sort, and pages concatenate in
that order, so the explicit sort_by is redundant and removed. The anti-grinding
reason for that order is unchanged and the doc comment keeps it: repo_id derives
from a grindable owner DID, so an id sort would let someone bury the true holder
past the probe cap. The probe cap still bounds expensive reads rather than
candidates considered, and no cap on candidates was added.

This pager runs to exhaustion, unlike the resolver's, which drives the same query
and stops when its probe or visit budget is spent. The comment says why: the
resolver is an anonymously reachable route holding scarce walk admission, while
this is background maintenance on a timer with no caller and no permit
contention that needs the whole warm candidate set to settle a row. Paging is per
pass, matching load_discovery_ctx's existing once-per-pass lifetime.

Refs Gitlawb/node#321.
2026-08-11 22:46:29 -05:00
beardthelion bc202368e1 fix(node): charge discovery's probes against the per-run read budget
The two halves of round 12 left a hole between them. The cursor rewind became
unconditional on reaching the end of the table, so every completed run re-walks
every row, and MAX_DEAD_ROW_READS_PER_RUN is what stops that from costing
O(rows) object reads on every boot. But row_read_attempted was only ever set in
the provenance loop, so a source-less row, the one shape discovery exists for,
spent up to MAX_LEGACY_DISCOVERY_PROBES reads and contributed nothing. The
per-row cap bounded one row; nothing bounded the run.

discover_legacy_row now reports the probes it spent and the caller charges them
whenever the row was not repaired.

Per probe rather than per row, because at 16 probes a row the budget would
otherwise admit 16 times the reads it names.

Including the retryable arm, which is where discovery differs from the
provenance loop. There a retryable read is against a repo the row names as a
holder and is expected to succeed once that repo warms. Discovery probes repos
the row does not name and re-derives its candidate list from scratch each run,
so an unrepaired row costs the same reads again on the next boot whatever its
outcome was. The cap-reached arm is retryable by design, to stop a grindable
repo id burying the true holder, and leaving it uncharged would leave exactly
the steerable path uncharged.
2026-08-11 22:08:10 -05:00
beardthelion bd0bef1476 fix(node): carry discovery onto the per-(oid, repo) incompleteness marker
Rebasing onto the #173 head that made the marker per (object, repo) broke
three assumptions this branch was written against.

Discovery re-marked the set incomplete after recording a discovered source,
because a successful record_pin_source used to clear the whole per-object
boolean. It no longer does, so that call went from compensating for a clear to
being the only thing arming the resolver's fallback. It now marks against the
unknown-repo sentinel rather than the repo it just recorded, which would claim
that repo failed when it is the one that succeeded. The sentinel is what the
v24 migration carries pre-upgrade markers under and means the same thing here:
the bounded warm-only probe may have missed a holder and nobody knows which.

Two tests asserted the superseded behavior and are inverted rather than
deleted, which is what their own comments asked for. GAP 2 documented the
per-object marker being cleared by any later record; that residual is closed,
so the test now proves discovery's marker survives one. The cold-candidate test
proved the cursor was not rewound; the rewind is unconditional on completion
now, so it asserts the cost that still matters, that re-walking the row reads
no object bytes and restores nothing.
2026-08-11 22:08:10 -05:00
beardthelion 25cab14024 test(node): pin the per-object marker's third-repo residual
record_pin_source's doc already names this residual and accepts it: the
pin_sources_incomplete marker is one boolean per object rather than per
(object, repo), so a genuine record from a third repo clears a marker that a
different repo's failed record set. Its two neighbouring cases are tested,
the same repo re-recording and a no-op insert that must not clear, and this
one sat between them described in prose and guarded by nothing.

The test drives the real path rather than writing rows. Repo A first-pins the
object. Repo B genuinely holds it and pushes, its skip-branch record fails on
the definite-error arm, and the code marks the set incomplete. An anonymous
read is then SERVED from B, which is the assertion that matters: it proves the
unrecorded holder is reachable while the marker stands. Repo C pushes, its
record succeeds, and the clear fires inside C's transaction. The identical
read now 404s, with the preload counter proving the fallback scan never ran.

Nothing here changes behavior. It converts an accepted tradeoff from a comment
into a guard, so widening it takes a deliberate edit rather than going
unnoticed. The doc comment says the residual fails closed, and that holds:
the provenance loop and the fallback scan both serve through the same
gate_and_serve, so clearing the marker only ever removes a gated search and
can never serve an object the caller could not already read. The window is
exactly one to fifteen sources, because at cap the count-guarded insert is a
no-op and the marker survives.

If someone implements the per-(oid, repo) marker the doc proposes, this test
is expected to go red. It should be updated to the new behavior, not deleted.

Proven load-bearing: removing the clear from record_pin_source leaves the
second read at 200 and turns the test red on the marker assertion.
2026-08-11 22:08:10 -05:00
beardthelion d88c0c726c fix(node): repair source-less legacy pins by bounded additive discovery
The boot sweep resolved each legacy row's bytes only through
pin_sources_for_oid. A row predating provenance has repo_id NULL and no
pin_repo_sources entry, so the source set is empty, the `for repo_id in
sources` body never runs, and the cursor advances past it. list_pinned_cids
also filters those rows out on is_raw_cidv1, so they were both unadvertised
and unrepairable: nothing on the node would ever fix them. The sweep exists
precisely for pre-provenance rows, and that was the shape it could not
handle.

For a source-less row, discovery now probes a bounded set of warm local repos
for the object and, on a hit, repairs the key through the existing
repair_legacy_provider_cid.

What it records is the part worth reading closely. It calls record_pin_source
additively and then mark_pin_sources_incomplete. It does NOT call
backfill_pin_provenance, and repo_id stays NULL. Reading identical bytes out
of a repo proves that repo HOLDS the object, not that it is the source: forks,
a vendored file, a shared LICENSE blob and the empty tree all collide across
repos. Since backfill_pin_provenance guards on AND repo_id IS NULL, the first
claim written is permanent, so an exclusive claim would let probe order decide
which repo's visibility rules gate a public read surface, with no correction
path. It would also disable the resolver's own fallback: needs_scan is
sources.is_empty() || at_cap || incomplete, so a non-empty below-cap unmarked
set reads as complete and stops the scan that would have found a servable
copy. The additive record plus the incomplete marker keeps that fallback
alive and makes nothing permanent.

Candidates are quarantined-filtered. list_all_repos is a bare SELECT over
repos with no quarantine or visibility filter, while the resolver's own legacy
scan loads list_quarantined_repos and gates on it, so discovery loads the same
set and drops those ids. A quarantined repo is hidden from every reader, so it
must not become a recorded source. Private non-quarantined repos stay in the
list: an additive source record does not widen what the resolver will serve,
since every source is gated independently.

The candidate list and the warm filter load once per pass, lazily, on the
first source-less row, and the is_dir sweep runs under spawn_blocking rather
than inline on a worker. Nothing goes through repo_store.acquire, so a cold
repo is never pulled back from remote storage; a repair pass must not become
a bulk restore.

Two classification rules matter more than they look. A cold or quarantined
candidate does NOT mark the row retryable. The candidate list is every repo on
the node rather than the row's holders, and sweep_legacy_provider_cids rewinds
the cursor whenever any retryable skip occurred, so counting a cold candidate
would make every source-less row retryable on every run on a cold-storage
node: the cursor would rewind forever, the sweep would never drain, and each
pass would pay full discovery cost. Only a read error on a warm candidate is
evidence about the row. Conversely, cap exhaustion IS retryable rather than
terminal, and candidates are ordered oldest-first by (created_at, id) rather
than by id: repo_id derives from the owner DID, which anyone can grind, so a
first-N-wins cap over an id-sorted list would let someone bury the true holder
permanently behind cheap registrations.

MAX_LEGACY_DISCOVERY_PROBES bounds the expensive unit, a bounded object read
from a warm repo. A candidate dropped cold, quarantined, or unsafe costs no
probe. Its doc states the contract it encodes rather than its resemblance to
the resolver's own cap, which happens to share the value under a different
contract.

The source record is best-effort and says so at the site rather than claiming
a later pass heals it. Once the key is rewritten the row is raw CIDv1, so the
cost gate skips it free and nothing revisits it; the healing path is the
resolver's needs_scan fallback, which the incomplete marker keeps available.

The no-remote-fetch property is asserted from both sides. On the effect side,
a cold candidate's disk path still does not exist after two full runs, with a
control proving the sweep declined an available copy rather than finding
nothing to take. On the call side, a source scan of this module (comments and
string literals stripped, four anti-vacuity assertions) rejects the
fetch-capable call shapes, and a mutation planting a real acquire in the probe
loop turns it red. What that pair does not cover, and the test says so: a
fetch reached transitively through git::store or db, and git's own promisor
lazy-fetch on a partial clone. Neither shape exists in the tree today.

Concurrency around the marker is executed rather than assumed. Discovery
records then marks, in that order, because a successful record_pin_source
clears the marker. Under a proven lock-wait interleaving, a concurrent record
committing before discovery's mark leaves the row marked, and two concurrent
passes over one row converge on a single source with the marker set. Both the
swap and the drop of that pair are red-checked.

One residual is pinned by a test rather than papered over, and it is not
introduced here. pin_sources_incomplete is one boolean per object, not per
(object, repo), so a genuine pusher whose record_pin_source commits strictly
AFTER discovery's mark clears it, leaving a non-empty below-cap unmarked set
that tells needs_scan to stop scanning. No ordering inside this sweep can
prevent that, and record_pin_source's own doc already states the same residual
for its existing call sites; closing it needs a per-(oid, repo) marker, which
is a change to the marker's shape. It is bounded in practice: the clearing
writer really holds the bytes, so every source in the set is servable, and for
these rows it cannot be a regression, since before discovery ran they were
keyed on a provider CID and were neither served nor advertised.

Tests: fifteen regressions driving the sweep entry points. The repair, cheap
second pass, probe cap and multi-holder cases were observed RED first. The
must-not cases carry mutations instead, since a guard against something
happening has no honest pre-fix red: quarantined holder never serves, a cold
candidate is neither fetched nor materialized and does not rewind the cursor,
an unsafe candidate name is dropped without aborting the pass while a later
safe candidate still repairs, a raw-CIDv1 row never enters discovery, and a
provenanced row still resolves through the source loop alone. Every guard
carries a red-check mutation with an attributed message, including three on
the design itself: one adds backfill_pin_provenance beside the additive record
and must redden the no-exclusive-claim assertion, one neutralizes the
incomplete marker, and one swaps the record and mark order.
2026-08-11 22:08:10 -05:00
beardthelion 4985e38af2 fix(ipfs): page the legacy CID scan instead of materializing every repo
GITLAWB_IPFS_MAX_LEGACY_PROBES bounded the probe fan-out, but nothing bounded
the load that ran before the first probe. On a legacy (NULL-provenance) or
capped/incomplete provenance set, get_by_cid called list_all_repos, cloned every
id, fetched visibility rules for the whole set, and fetched the node's entire
quarantine list, then retained all three for the request. /ipfs/{cid} is
anon-reachable and CIDs are enumerable from the unsigned pins index, and an
object pinned from MAX_PIN_SOURCES repos is permanently at cap, so every later
anonymous GET for it paid that whole preload with the scarce walk permits held,
even at a probe budget of 1.

The selection is now keyset-paged on (created_at, id) ASC, with the quarantine
flag returned per row so the separate whole-node quarantine query is gone. Both
key columns are immutable, so paging is exact: a repo touched mid-scan cannot
cross a page boundary and go unvisited, and updated_at is attacker-bumpable,
which would have let a caller sort their own repos ahead of the true holder.
The pager lives outside the oid loop, so its cursor and its accounting are per
request rather than per candidate. Before fetching another page the scan checks
that a row on it could still reach a verdict; a spent probe or visit budget
stops the read and TAINTS, so a truncated search still sheds the retryable 503
and never a false 404. A page of pure denials is deliberately not a stop
condition, since a cheap denial costs no probe and a public object buried behind
private repos must still be reached. Both page queries stay clamped to the
remaining request budget, and the rules query still fails closed.

A new v25 migration adds idx_repos_created_at_id. Bounding what the handler
materializes is only half the property: repos carried no index in the new keyset
order, so Postgres seq-scanned the table and top-N sorted it to return every
page while the walk admission was held, which moves the same O(rows) cost onto
the database rather than removing it. Measured on 50k rows: 951 shared buffers
and ~44ms per page without the index, versus an Index Only Scan at 7 buffers and
~0.09ms with it, and the keyset predicate pushed down as an Index Cond instead
of filtering after a scan. Nothing names the index in any query text, so its
entry says why it exists and a test asserts its presence structurally.

Two order-dependent tests were reseeded rather than left green: the buried-row
and acquire-taint cases encoded their ordering through updated_at, which no
longer determines iteration. Both were re-confirmed RED under their documented
mutations after the reseed. Three regressions land with the fix. One drives a
one-probe, one-row-per-page request against three candidate repos and asserts it
materializes one row and runs one query, which a query counter alone could not
have caught. One seeds two distinct source-less oids under a single CID and pins
the per-request property directly: the second candidate re-reads the pages the
first paid for, so the whole request costs one pass rather than one per
candidate. The third covers the index, including the INV-7 upgrade path.

Refs Gitlawb/node#173.
2026-08-11 21:40:19 -05:00
euxaristia 1d3987bba6 fix(review): make the LIKE escape independent of the SQL parser mode
Spelling the escape character as `ESCAPE '\'` puts a backslash inside an
ordinary SQL string literal. Under the legacy `standard_conforming_strings=off`
setting that backslash escapes the closing quote, so the literal runs on and the
statement fails to parse. The failure happens before the bound parameters are
considered, so every prefix lookup breaks, including prefixes that contain no
metacharacter at all, and short certificate IDs stop resolving entirely.
Connection settings come from outside the process, so that mode can arrive from
a database- or role-level setting.

Use `!` as the LIKE escape character instead, escaping `%`, `_`, and `!` in the
bound pattern. The statement then holds no backslash and parses the same either
way, without needing a second layer of backslash escaping. A backslash in the
prefix becomes an ordinary character, which is the literal match the search
wants anyway.

Cover it with a test that runs the production query through a second pool that
requests the legacy mode at connection startup, so a future return to a
parser-mode-dependent literal is caught rather than only wildcard handling under
the default configuration.

Refs #319
2026-08-11 14:09:13 -04:00
beardthelion 9758bc7303 docs(node): name the knob that actually sets the legacy-probe budget
The comment beside ipfs_max_legacy_probes told operators to tune
GITLAWB_IPFS_MAX_REPOS_WALKED, but AppState::ipfs_legacy_probe_budget reads
GITLAWB_IPFS_MAX_LEGACY_PROBES. The knob it named is the separate cap on
expensive visibility walks, so anyone following the guidance next to this line
adjusted the walk cap and left the legacy scan fan-out at its default.
2026-08-11 12:23:08 -05:00
beardthelion 5575da4b65 fix(node): make the pin-source incompleteness marker per (object, repo)
pinned_cids.pin_sources_incomplete was one boolean per object, so any
successful source record cleared it, including a genuine record from a repo
unrelated to the failure. The resolver reads a cleared marker as "every source
is recorded", drops the scan fallback, and 404s an anonymous caller whose only
servable copy is the one whose record failed. The missing source is a property
of an (object, repo) pair, so v24 stores it as one and only a later record from
that same repo clears it.

Pre-upgrade markers carry over with an empty sentinel repo id that no real
record matches, so they keep their fallback instead of being cleared by the
next unrelated record. pinned_cids.pin_sources_incomplete is left in place,
unread, so a rollback to the previous release still finds its markers.

Also adds the handler-level coverage the size-probe commit could not: a test
seam drops the object between the type probe and the size probe, and the
request tails to a retryable 503 rather than a definitive 404. The seam is
keyed on (repo path, oid) because fixture oids are content-derived and shared
across tests, and an oid-only key deleted objects out from under two unrelated
tests running in parallel.
2026-08-11 12:20:39 -05:00
beardthelion ed6355ed2c fix(node): stop the /ipfs size probe reporting a fault as a missing object
object_size_bounded mapped every non-timeout failure to Ok(None), which
gate_and_serve reads as a verified absence and does not taint the search for.
A corrupt object, an unreadable pack, or a failed spawn therefore ended the
search cleanly and handed an authorized caller a definitive 404 instead of the
retryable 503 tail, even though the type probe had already reported the object
present. Confirmed by execution against the old code: a corrupt loose object
returned Ok(None) where the healthy read returned Ok(Some(n)).

The probe now returns Result<u64, ProbeError>, the vocabulary the type and
content stages either side of it already use: a reaped child is Transient, and
everything else is classified by store readability, Transient when the store
cannot be read and Deterministic when it can. There is no absence value left,
so the collapse is unrepresentable rather than fixed at one call site, and the
gc-between-stages case is classified like any other fault.

ServedRead::Gone lost its only producer and is removed with it.

Covered by two store tests, each asserting a healthy read first so the fault
case cannot pass on a broken fixture. Not covered: the handler path that turns
the fault into a 503, because that call site hardcodes "git" rather than
state.git_bin, so no shim can be injected there.
2026-08-11 11:38:59 -05:00
beardthelion e4aaae8c52 fix(node): bound the fruitless reads one sweep run will spend
Rewinding on every completed walk means a later run re-attempts the object
read for every row whose bytes are permanently gone, so a node that collected
dead pins from a deleted repo or a force-pushed history paid O(dead rows) git
invocations on every boot with no decay. A run now stops after 64 such reads
and keeps its cursor, so the next boot resumes past what it already walked and
the table is still covered across boots.

Also corrects two pieces of prose that described the superseded round 11
behavior: the SweepStats::retryable_skips field doc still claimed that field
drives the rewind, and the new below-cursor test's comments still said a clean
run parks its cursor.

The pass-failure test waited on a fixed 100ms sleep for the first pass to
finish before renaming the table out from under the walk. It now waits on the
cursor that pass writes, so a runner slow enough to push the first pass past
the sleep no longer fails the assertion for the wrong reason.
2026-08-11 11:11:34 -05:00
Gravirei aa68d97a8d refactor(repo_store): expose advisory_lock_key pub(crate) and drop test copy
The deadline test hand-copied the advisory-lock key derivation, and that
copy drifted once the key moved to SHA-256, silently breaking the lock
collision the test is built around. Import the production function
instead so the key cannot diverge again.
2026-08-11 17:59:33 +06:00
beardthelion 1b44b12b46 fix(node): rewind the pin repair sweep cursor on every completed walk
A clean sweep run parked its cursor at the table's maximum sha256_hex and
later runs read only rows above it, so a provider-CID row written below that
point by another node during a rolling upgrade was never swept. The resolver
withholds such a row's advertised key, leaving the object unretrievable with
nothing left to repair it.

The round 11 rewind could not cover this: it fired only when a run skipped a
retryable row, and the run that parks the cursor is by definition clean. The
rewind is now unconditional on reaching the end of the table. A run that stops
on a pass error keeps its position instead, so a node whose DB fails part-way
through does not restart the walk from the beginning on every boot.

The cost is one ordered scan per run, plus one repair attempt per run for a
row that is unrepairable in principle. Rows already carrying the canonical raw
key cost a codec decode and no object read, so a node that has finished
repairing pays the scan and nothing more. The terminal-skip test now asserts
that bounded cost rather than the row going unread.
2026-08-11 04:33:47 -05:00
beardthelion 9ac3fd3f6b Merge origin/main into fix/issue-135-ipfs-cid-tree-gate
#254 landed after this branch was last pushed and it moves the same surface.
It converts get_by_cid's sqlx failure arms from AppError::Internal to a bare
Into<AppError> conversion, so a connection-class failure downcasts to
AppError::Db and answers 503 db_unavailable instead of a 500 (#251).

Both textual conflicts were spurious alignments rather than real overlaps.
Git matched this branch's provenance path against main's legacy-scan preload
because both regions end the same way, and oids_for_cid does not exist on main
at all. Resolved to this branch's structure, which already contains main's
preload in its own place with all three of its queries clamped.

The semantic change is the part that mattered and it was applied by hand
afterwards, not inherited from the resolution. All ten clamped DB awaits in
get_by_cid now use the bare conversion. Taking this branch's side wholesale
would have silently reverted #251 across every one of them, including the four
this branch added, since the arm shape they were written to mirror predated
#254.

Suite green on the merged tree with fmt, clippy, and metadata --locked clean.
2026-08-10 22:46:18 -05:00
euxaristiaandcairn-code eeafa1fa53 fix(node): Escape LIKE metacharacters in certificate prefix search
Short-ID resolution passes a caller-provided prefix into a LIKE pattern. Any '%', '_' or '\' in that prefix is interpreted as a LIKE wildcard rather than a literal character, so lookups can match certificates outside the requested prefix.

Escape the three metacharacters in the prefix and add an explicit ESCAPE '\' clause so prefix matching is always literal.

Add a regression test covering literal '%', '_' and '\' prefixes.

Co-Authored-By: cairn-code <[email protected]>
2026-08-10 23:38:05 -04:00
Kevin Codex 0e2328bd80 Merge pull request #254 from Ayush7614/fix/251-db-outage-503-ipfs-arweave
fix(node): map ipfs/arweave DB outages to 503 via bare ? (#251)
2026-08-11 09:54:29 +08:00
Kevin Codex 10e0840dd9 Merge pull request #312 from Gitlawb/fix/shim-nonblocking-accept
fix(git-remote): normalize the shim's accepted socket to blocking
2026-08-11 09:53:00 +08:00
beardthelion 9984e08212 fix(node): bound the pin loops' DB work by the batch deadline, both lanes
api/repos.rs holds the global pin_semaphore across ipfs_pin::pin_new_objects
and across pinata::pin_new_objects. Inside both, batch_budget_gate only gates
between objects and the git read was already clamped, but the DB calls were
bare awaits. One stalled query parked the permit past every budget, and once
all pin permits were so held, post-push IPFS and Pinata replication stopped
for every repository on the node. The pool defers rather than sheds, so the
queue behind it only grows.

The bound set is derived callee-transitively from each budgeted region's call
graph, not from the shape of the fix. A grep for db./retry_db_record over the
loop bodies cannot see a DB await behind a call site, and that is not
hypothetical: it missed repair_legacy_provider_cid, whose own cid_for_oid and
repair_legacy_provider_cid awaits run under the permit from both lanes while
its deadline argument bounds only the spawn_blocking git read. Kubo: six
direct sites plus those two transitive ones. Pinata: eight direct sites, two
more than Kubo because its post-upload path has its own record_pin_source and
incomplete marker. Completeness here is the enumeration, scoped to the call
graph at this head, not a compile-enforced guarantee.

db_bounded takes the ABSOLUTE deadline, so a slow predecessor cannot hand a
later call a fresh budget. What its Elapsed arm MEANS is a property of the
operation, not of the timeout. A multi-statement transaction whose tx.commit()
is never reached definitively did not land: record_pin_source is
pool.begin(), INSERT, conditional UPDATE, tx.commit(), so a future cancelled
while the INSERT waits on a lock cannot have committed. A single autocommit
statement may well have landed server-side after the client future was
dropped, since tokio cancels the future but not the statement Postgres is
running; mark_pin_sources_incomplete and record_pinata_cid are that shape.

Each site maps its elapsed arm on that basis. Reads skip the object.
Provenance and backfill warn and continue, since the backfill is idempotent
under its repo_id IS NULL guard. A timed-out record_pin_source writes the
incomplete marker, exactly as its definite-error arm does, because the source
definitively was not recorded and leaving the set incomplete AND unmarked is
the state this change exists to prevent: /ipfs would read a non-empty
below-cap source set as complete and 404 a copy the repo would serve. The
arms stay separate so the warn still distinguishes a stalled batch from
scattered per-object failures.

The durability writes take a floored remainder. batch_budget_gate only
guarantees PIN_READ_FLOOR before an object starts and the add gets the whole
remainder, so a successful add can finish with ~0 budget left; an unfloored
bound would fail a write that today completes in milliseconds. That leaves
bytes in Kubo with no pinned_cids row, and on the Pinata side it is worse,
because pinned.push is unconditional there, so repos.rs would build cid_map
and gossip publish_ref_update for a CID that /ipfs will 404. The floor is
re-taken from now at each call, so graces chain within one iteration: the
worst case is deadline+4s on the Kubo skip branch and deadline+6s on the
Pinata add path, about 5% of a 120s PIN_BATCH_BUDGET. It does not stack per
object, because batch_budget_gate breaks at the next iteration's first
statement.

pin_new_objects_gated now takes the batch budget instead of hardcoding
PIN_BATCH_BUDGET. The permit-release regression asserts a bound that IS the
budget, so with the constant inlined it cost 120s of wall clock; passing it
drops the test to 3.4s. Production passes the same constant.

Both twin doc comments are updated together. Each previously stated that the
DB round-trips were untimed inside the budgeted region, which is now false.
The Pinata permit's total hold across pinata_object_list_for_refs stays
unbounded and is called out as such: repos.rs acquires the permit before that
pre-loop walk, so this change closes the DB-stall trigger, not the git-walk
one.

Tests: thirteen regressions, the stall cases observed RED first with the bare
await sitting on the lock until the outer wrap fired. Every guard is proven
load-bearing by a mutation carrying an attributed message, so a RED is pinned
to the named property rather than to seam removal.

Three properties needed a fixture built specifically to bind them, because
the obvious test passed either way. The record floor: a zero remainder still
landed the write, since tokio polls the inner future before the timer and a
local UPDATE round-trips inside one tick, so the marker's table is now locked
and released inside the grace window, which fails without the floor and lands
with it. The absolute deadline: every single-object stall test passes under a
per-call duration too, so a helper unit test now runs two calls against one
shared deadline and asserts the second is cut off by the remainder its
predecessor left, with the mutation granting a fresh duration equal to the
budget rather than larger than it. The Pinata post-upload elapsed arm: it had
been changed in lockstep with the two covered sites but nothing executed it,
so a fixture now lets the upload succeed and stalls only the record that
follows.
2026-08-10 20:36:04 -05:00
beardthelion bad82b3cd9 fix(node): clamp every admission-held DB await in the CID resolver
get_by_cid builds its walk admission (the global permit plus the
per-caller permit) and then made several DB awaits with no deadline while
those scarce permits were held. A stalled pool or a blocked query pinned
every walk slot for the whole stall, so later /ipfs requests shed capacity
503s long after GITLAWB_IPFS_REQUEST_BUDGET_SECS had elapsed. Any
unauthenticated caller can reach it.

The clamped set is derived from the admission-held region itself, not from
the two sites the review named. Walking the handler body found four bare
awaits:

  oids_for_cid, pin_sources_for_oid, pin_sources_at_cap,
  pin_sources_incomplete

The last two were not named in review. Already clamped and left alone: the
three per-source lookups and the legacy-scan preload trio. Two negatives
worth recording: gate_and_serve is the only callee reached from inside the
region and holds no DB awaits of its own, so the region does not extend
transitively, and ipfs_work_rate_limiter.is_throttled is in-memory, not a
round trip. Ten DB awaits in the body, zero bare after this change.

Each clamp takes the arm shape the per-source lookups already use, so a
timeout returns the budget-tainted retryable 503 and the early return
RAII-drops the admission. budget_shed and remaining are hoisted above the
oids_for_cid await so the provenance-path sites share one definition; the
legacy-scan preload keeps its own in its nested scope.

Tests: four stall regressions, one per clamped site. The first two lock the
table their query reads and were observed RED first, with the bare await
sitting on the lock until the 10s outer wrap fired.

The marker pair needed a test-only seam rather than a lock. Both queries read
tables that earlier queries in the same request touch first, so a LOCK TABLE
aimed at them stalls an earlier clamp instead, and budget exhaustion has the
same problem because an earlier query elapses before the pair is reached. The
remaining window between pin_sources_for_oid returning and the pair running
is a single get_repo_by_id round trip, measured at 0.14 to 0.17 ms, which is
too small to time a lock into without flaking. So there is a cfg(test) stall
armed per target, placed INSIDE the timeout wrapper so the clamp is what
fires. Every seam item is cfg(test), including the function itself, so a
non-test caller would not compile, and a full release build is clean.

Each of the four asserts the same contract: a 503 inside the budget, the
budget taint in the body, the walk permit returned, and a follow-up request
admitted rather than capacity-shed. All four clamps are proven load-bearing
by reverting the clamp alone with an attributed message, and the mutations
cross-isolate: reverting one turns only its own test red, so each test
stalls at its own query rather than a sibling's.
2026-08-10 20:35:36 -05:00
beardthelion 208fd808b6 Merge remote-tracking branch 'origin/main' into fix/issue-135-ipfs-cid-tree-gate
# Conflicts:
#	crates/gitlawb-node/src/db/mod.rs
2026-08-10 15:08:30 -05:00
beardthelion d4e5ecf299 fix(review): bound the probe's channel waits and stop overstating the handshake
Cross-model review round, findings corroborated by the in-process reliability
reviewer.

Both channel waits were unbounded. If the reader thread never reached its read,
or never returned after the peer close, the test wedged the binary instead of
failing, and cargo test has no per-test timeout to rescue it. Both are now
recv_timeout with bounds far above any real scheduling delay, so hitting one is
a genuine defect report rather than a flake, and the failure names which wait
stalled.

The handshake paragraph also overstated what the handshake buys. It listed "the
reader thread not yet scheduled to reach the read at all" among the states the
handshake removes, which reads as though the whole scheduling window is closed.
It is not: the handshake removes thread spawn from the window, and a reader
descheduled between its own Started send and its read is still a false-green
path. That path is now stated plainly instead of being left to the following
paragraph to correct.

Re-verified after the change: probe RED 50/50 under a deleted
set_nonblocking(false) at full-suite parallelism, RED on the named message under
a deleted set_read_timeout, 50/50 green unmutated, fmt and clippy clean.
2026-08-10 13:55:53 -05:00
beardthelion 4d9bd5a18a fix(review): assert the read-timeout postcondition and drop the socket clone
Two findings from review of the probe test.

The helper has two postconditions and only one was guarded. Deleting
set_read_timeout left every test green while a stalled peer would park the
shim's only accept-loop thread forever, since handle_conn runs inline in that
loop. std exposes read_timeout() as a getter, so this half is checkable exactly
with no timing component. Confirmed load-bearing: deleting the line reds the
probe on the new message.

The probe also normalized one socket and then read from a try_clone of it, so
it only proved anything if blocking mode is shared across the two. On Linux it
is, because try_clone is dup(2) and O_NONBLOCK lives on the shared open file
description. On Windows try_clone goes through WSADuplicateSocketW and WSASocketW
(std os/windows/io/socket.rs), which mints a new socket descriptor, and whether
FIONBIO follows it is a question this test would have depended on without being
able to answer. It would have failed silently and green, on the one platform
where the underlying defect is real. The clone was unnecessary, so the reader
now owns the socket the helper normalized and the question does not arise.

Also correct two overstatements in the probe's own doc comment. It called
itself deterministic while its verdict is a 250ms window, and it described the
residual false-green path as "the few instructions between the signal and the
call" when the window is a full cross-thread round trip: signal, read, return,
channel send, and this thread waking. Both now say what the code does.

Re-verified after the change: probe RED 50/50 under a deleted
set_nonblocking(false) at full-suite parallelism, RED on the named message under
a deleted set_read_timeout, and 50/50 green unmutated. The end-to-end test came
in at 39/50 on this run against the first mutation, a third sample alongside 43
and 47, so its doc comment now reports the spread instead of two numbers.
2026-08-10 13:46:43 -05:00
beardthelion 6f58f35c95 test(git-remote): make the non-blocking-accept guard deterministic
The regression test added alongside the shim fix could pass with the fix
removed. It synchronized on a channel send that happens after accept() but
before handle_conn runs, so the request bytes could already be buffered by
the time the shim read them, and the read then succeeded on a non-blocking
socket. Measured at 43 and 47 of 50 full-suite runs catching a deleted
set_nonblocking(false), so it went false-green roughly one run in eight.

Extract the normalization into normalize_accepted_stream and assert its
postcondition directly, with no client and no request-arrival race. The new
probe puts a connected socket into non-blocking mode, calls the helper, and
checks that a reader parks instead of returning WouldBlock. Blocking mode has
no getter to query: std exposes none and Winsock has no FIONBIO getter at all,
so the property has to be observed through behavior.

The reader signals Started immediately before its read and the main thread
waits for that signal unbounded before opening the 250ms window. Without the
handshake the window would also be satisfied by a thread that has not been
scheduled yet, which would reintroduce the same silent hole one level down.
Peer close comes after the assertion, never before; closing first unblocks the
read and fails the test even when the helper is correct.

Also correct a platform claim this PR had in five places. The bug is not
Windows-only. An accepted socket inherits the listener's flag on BSD-derived
stacks, so macOS and the BSDs are exposed too; Linux is the exception because
std uses accept4 there, which does not inherit file status flags
(rust-lang/rust#67027). A macOS contributor could hit this today while the
comments said it was impossible. The macOS half is documented rather than
executed, and the comments now say so.

Verification, full-suite runs at default parallelism:
  helper body mutated (set_nonblocking(false) deleted): probe RED 50/50
  call site mutated (helper call deleted):              probe GREEN 0/50,
                                                        integration test 45
                                                        and 46 of 50
  unmutated:                                            50/50 green

The call-site row is the probe's blind spot, measured rather than assumed: it
calls the helper directly, so only the integration test notices a deleted
call, and it does so at a sampled rate rather than reliably. That residual is
recorded in the integration test's doc comment.
2026-08-10 12:20:48 -05:00
Gravirei c68e1b07f8 fix(test): match SHA-256 advisory-lock key in receive-pack deadline test
The test reproduces acquire_write's advisory-lock key derivation locally so
a held session lock can stall the handler. repo_store's advisory_lock_key
now uses SHA-256, so the DefaultHasher copy no longer collided and the
stall never happened. Update the local copy to the SHA-256 derivation.
2026-08-10 22:09:10 +06:00
Gravirei 9f340eac5e Merge upstream/main into fix/issue-210-stable-advisory-lock-key
Resolve conflicts: keep the SHA-256 advisory-lock key change alongside
upstream's connection-affine RepoWriteGuard rework and its F1/F3b/F4/U8
test suite; RUN-A-NODE.md keeps both the updated Public URL row and the
rolling-upgrade advisory-lock row.
2026-08-10 21:31:38 +06:00
beardthelion 8d7f95fd63 fix(git-remote): normalize the shim's accepted socket to blocking (#192)
start_shim sets the listener non-blocking so its accept loop can poll the
stop flag. On unix accept() hands back a blocking socket, so that never
showed; on Windows the accepted socket inherits the listener's FIONBIO
state and std does not reset it, so every connection arrived non-blocking.

handle_conn reads with read_line(..).unwrap_or(0), which turns the
resulting WouldBlock into "peer closed" and returns without writing a
response, dropping the connection and aborting the client mid-request. The
shim accepts as soon as the handshake completes, strictly before the
request bytes arrive, so the window was open on every connection and
closed only because the 5ms accept poll usually lagged the client. That is
what made real_git_withheld_shaped_first_post fail on windows-latest with
os error 10053 and zero POSTs recorded.

Normalizing at handle_conn entry also restores the 30s read timeout on the
next line, which a non-blocking socket silently made meaningless.

The regression test forces the Windows socket shape explicitly, so it is
load-bearing on every platform rather than only the one that exhibits the
bug, and it establishes the accept-before-request interleaving with a
handshake rather than a sleep: a timing-based version cannot flake red but
can flake green under load, which would make the guard vacuous.
2026-08-10 10:14:43 -05:00
Ayush7614 c29a91af6b fix(node): map ipfs/arweave DB outages to 503 via bare ? (#251)
Rebased onto current main. Keep budget-timeout wrappers on get_by_cid;
convert sqlx failures with Into<AppError> (not AppError::Internal) so
pool closed/timeouts become 503 db_unavailable. Same for list_pins and
arweave anchors, with closed-pool regressions.
2026-08-10 20:19:51 +05:30
Kevin Codex 241b36633b Merge pull request #247 from Ayush7614/fix/226-opaque-internal-errors
fix(node): opaque AppError::Internal and AppError::Db HTTP bodies (#226)
2026-08-10 21:43:42 +08:00
Kevin Codex 425ebf40e7 Merge pull request #309 from euxaristia/fix/strict-ed25519-verification
fix(core): enforce strict RFC 8032 Ed25519 signature verification
2026-08-10 21:41:57 +08:00
beardthelion ea2219b2cb fix(node): bound the legacy-repair read by the pin batch deadline
repair_legacy_provider_cid built its own deadline from git_timeout and never
consulted the batch deadline, but both pin loops call it with a pin_semaphore
permit held. At shipped defaults that is a 600s git_service_timeout_secs against
a 120s PIN_BATCH_BUDGET, so one legacy row whose cat-file wedges could hold a
global pin slot for five times the budget the batch is supposed to cost and
starve every other repo's pin work. The loop's budget gate cannot help: it runs
at the top of the next iteration and cannot preempt a call already in flight.

The helper now takes the deadline rather than deriving it, since its two kinds
of caller can afford different holds. Both pin loops clamp to
min(batch deadline, now + git_timeout). The boot sweep holds no permit and has
no batch to overrun, so it keeps the plain git_timeout.

The Kubo side has had this since the repair landed and the Pinata side inherited
it from the lockstep work in the previous commit, so both get a test: a wedged
cat-file with a 60s git_timeout and a 2s batch budget must return on the batch
order. Both were RED at 62s before the clamp and are load-bearing by mutation.

Also stops the read-bound comments implying both arms bind in a default
deployment. At shipped defaults the batch deadline is always the tighter one;
the git_timeout arm is what an operator who lowers that knob gets.
2026-08-10 07:43:53 -05:00
euxaristiaandcairn-code 1fcbbbd89c fix(attest): use strict Ed25519 verification with regression test
Co-Authored-By: cairn-code <[email protected]>
2026-08-10 07:38:40 -04:00
euxaristiaandcairn-code 871b8610cf test(core): cover strict Ed25519 verification
Add a regression test for verify_strict: a signature forged under a weak (small-order) public key satisfies the verification equation but must be rejected. The non-strict verify path accepts it.

Co-Authored-By: cairn-code <[email protected]>
2026-08-10 07:38:40 -04:00
euxaristiaandcairn-code a6c13be15b fix(core): use strict Ed25519 signature verification
Use verify_strict from ed25519-dalek for identity signature checks, matching the crate's recommended RFC 8032 strict verification mode.

Co-Authored-By: cairn-code <[email protected]>
2026-08-10 07:38:40 -04:00
Ayush7614 af821e09bf test(node): disposition peer-readiness fixture writers in ledger
Same #248 test-only INSERT INTO peers fixtures as on #254; record them
in LEDGER so test (stable)/beta pass on the rebased #247 head.
2026-08-10 15:51:49 +05:30
Ayush7614 d192028ee1 fix(node): opaque AppError::Internal and AppError::Db HTTP bodies (#226)
Rebased onto current main. Keep Overloaded/UnresolvableDid arms; make
Internal and non-unavailable Db responses opaque with server-side logs,
and pin exact JSON in unit + anonymous route tests.
2026-08-10 15:51:49 +05:30
beardthelion 4681f54327 fix(node): bring the Pinata pin path back into lockstep with the Kubo one
Two gaps remained after the main merge closed the unbounded-read one. The Kubo
skip branch repairs a legacy provider-CID row to the raw-content resolver key on
re-push and the Pinata skip branch did not, so a Pinata-first node left those
rows unresolvable until the deferred sweep caught them. And the Pinata object
read was bounded only by the batch deadline, so one wedged cat-file could hold
the pin permit for the whole budget.

Both are fixed by threading git_timeout into pinata::pin_new_objects, which it
needed anyway: repair_legacy_provider_cid takes a Duration, and the read bound
is now min(batch deadline, now + git_timeout), the same shape the Kubo side
uses. repair_legacy_provider_cid becomes pub(crate).

The returned pairs still diverge on purpose and the twin docs say why: Kubo omits
a record-failed pin because its return is log-only, Pinata keeps it because its
return feeds the announcement cid_map. Pinata's return behavior is unchanged.

Four tests drive pinata::pin_new_objects rather than the Kubo twin, because a
repair call merely present in pinata.rs proves nothing about the Pinata lane:
the repair runs on the skip branch, the codec cost gate reads no bytes for a
row that is already canonical, a failed repair does not break the batch, and the
upload path never runs the repair. The two must-not guards have no obtainable
pre-fix RED, so they are proven by mutation instead: removing the cost gate and
moving the repair onto the upload path each turn one red.
2026-08-10 02:14:10 -05:00
beardthelion f839fb9246 fix(node): stop reporting a Kubo pin whose DB record never landed
On record_pinned_cid_with_source exhausting its retries, pin_new_objects warned
and then pushed the object onto the returned vector anyway, so the post-push log
told operators an object was pinned when the durable index had no row for it and
the resolver could not serve that CID. The push is now conditional on the record
succeeding.

The twins deliberately diverge here and the old comment said the opposite, that
the push mirrored pinata to keep them structurally identical. The Kubo return is
log-only, so dropping a record-failed pin costs nothing. The pinata return is a
real input: api/repos.rs builds the sha-to-cid map from it, which drives
upsert_branch_cid and the p2p ref-update gossip, so omitting rows there would
silently drop branch-to-CID records. Both comments now say so, since the next
reader's instinct will be to re-align them.

Test asserts the record-failed object is absent from the return while the add
mock still fired, and that a second object is still attempted rather than the
batch breaking. Verified RED against the unconditional push before the fix, and
load-bearing by re-injecting it.
2026-08-10 01:34:04 -05:00
beardthelion 8f3a52175a docs(node): correct the walk-cap and db-pool operator docs
Two doc drifts jatmn flagged on #173, both docs-only.

The /ipfs walk cap is min(internal ceiling, GITLAWB_IPFS_MAX_REPOS_WALKED),
where the internal ceiling is MAX_PIN_SOURCES + 1 = 17. config.rs, README, and
.env.example all described a max() floor instead, so an operator reading them
would think the default of 64 was the effective ceiling and that raising the
knob bought headroom. Neither is true. The runtime is not changed: two tests set
the knob to 1 and depend on the cap actually being 1, so wiring the documented
floor would make the knob inert and break them.

db_max_connections has defaulted to 48 for a while and four comments still said
20. Two of them did arithmetic on it. main.rs and repo_store.rs justified the
separate advisory-lock pool by "20 is below 32", which is now false and made the
separation look like a consequence of the defaults; it is structural, since a
push holds its connection for the whole receive-pack at any pool size. Also adds
a RUN-A-NODE troubleshooting entry for the boot failure an in-place upgrade hits
when it is still configured with the old pool size of 20.
2026-08-10 01:24:46 -05:00
beardthelion 9f74811269 Merge origin/main into fix/issue-135-ipfs-cid-tree-gate
#174 landed on main at 5981257, so the integration merge this branch carried is
now redundant with main's own history. Five files conflicted, and two more
auto-merged into states that did not compile.

store.rs: main migrated the bounded-read family from a per-call timeout:
Duration to a shared deadline: Instant. The auto-merge took main's
batch_check_probe alongside this branch's object_type_bounded, which compose,
and produced two read_object_bounded definitions with incompatible signatures
and no conflict marker. Took main's convention throughout and re-applied this
branch's additions on top: object_size_bounded, the wedged-child reaper test,
and the doc enrichments.

ipfs_pin.rs: the pin read is bounded by the earlier of the batch deadline
(#174) and the object's own git_timeout (#173). Neither implies the other and
both are load-bearing: with only the batch deadline, one wedged cat-file holds
the pin permit for the whole 120s budget; with only git_timeout, a batch of
slow reads runs past the budget. Each direction was verified by injecting it
and observing the RED.

api/repos.rs: kept main's choice of the literal "git" over state.git_bin for
the pin path's cat-file, since that knob is the walk binary and tests point it
at a fake. The parameter stays so the seam is still drivable from a test, and
the reason now sits at the call site.

main.rs: dropped main's free sweep_rate_limiters, superseded by
AppState::sweep_rate_limiters, which sweeps the same limiters plus
ipfs_work_rate_limiter. Both call sites already used the method.

db/mod.rs: dispositioned main's two new peers-table test writers in the
completeness ledger.

Migration numbering is unchanged. v17 is byte-identical on both sides, and
main's own reservation comment already allots 18-23 to this work.
2026-08-10 01:08:46 -05:00
Kevin Codex 225644a3ea Merge pull request #248 from MikeTomlin19/agent/fix-peer-readiness-176
fix(node): use readiness for peer liveness
2026-08-10 12:17:20 +08:00
Kevin Codex 6668525f41 Merge pull request #286 from Gitlawb/fix/icaptcha-release-versioning
fix(release): bump icaptcha-client with the rest of the workspace
2026-08-10 12:11:19 +08:00
Kevin Codex 5fdf155c7f Merge pull request #192 from Gitlawb/fix/issue-117-multiround-fetch-deadlock
fix(git-remote): drive multi-round fetch as a v0 stateless-RPC client loop (#117)
2026-08-10 12:10:26 +08:00
Kevin Codex 8d38043a4c Merge pull request #290 from Gitlawb/fix/273-peer-authority-gate
fix(node): bind a peer row to its DID so only its keyholder can repoint it (#273)
2026-08-10 12:02:39 +08:00
beardthelion a53c151879 chore: sync Cargo.lock after rebase onto main 2026-08-09 22:39:31 -05:00
beardthelion 01971f7e11 test(git-remote): make the Windows Job Object mandatory in run_bounded
Resolves jatmn's #192 P2. run_bounded's clean-exit teardown only terminated
the job when one had been established; when assign_job returned None it did
nothing, so a git-remote-gitlawb helper descendant that outlived the `git
fetch` leader while holding the inherited stdout/stderr pipe write-ends stalled
the reader joins on the helper's ~300s HTTP timeout, past the bound run_bounded
promises. The timeout path's reap_tree (taskkill /T) fallback cannot rescue the
clean path: once the leader has cleanly exited it no longer roots the
descendant in the parent-PID tree, so taskkill /T cannot reach the orphan.

The Job Object is the only reliable Windows teardown, so make it mandatory:
assign_job now returns JobHandle and panics per-step if the job cannot be
established, and both teardown paths unconditionally terminate it. Nested
unnamed jobs with KILL_ON_JOB_CLOSE are standard on the shipped
x86_64-pc-windows-msvc / CI runners, so this costs no coverage; it converts a
silent ~300s hang into a loud, actionable failure and strengthens the bound
(KILL_ON_JOB_CLOSE kills the descendant at latest when JobHandle drops).
reap_tree, whose only Windows use was the removed None arm, is narrowed to
cfg(all(not(unix), not(windows))).

The change is entirely cfg(windows). Verified by cross-compiling the tests for
x86_64-pc-windows-gnu (compiles clean, clippy -D warnings clean, no dead_code
from the reap_tree narrowing) and by the unchanged Linux git-remote-gitlawb
suite (green). The Windows runtime teardown is reasoned-not-run here; CI's
Windows job is the runtime gate.
2026-08-09 22:39:31 -05:00
beardthelion 659ee00d7d test(git-remote): tear down the fetch tree with a Job Object on Windows (#192)
run_bounded's clean-exit descendant reap was Unix-only: on Windows a
git fetch leader that exits cleanly leaves the git-remote-gitlawb helper
blocked in its ~300s HTTP request holding the inherited pipes, and
taskkill /T can't reach a descendant once the leader is gone, so the
reader joins blew past the promised bound. The child is now assigned to
a Job Object at spawn (the analog of the Unix process_group(0)), and the
job is terminated on both the timeout and clean-exit paths, which kills
the whole tree regardless of leader liveness. KILL_ON_JOB_CLOSE is the
safety net. The clean-exit regression is now cross-platform so it
exercises this on a future Windows CI lane (#228).

Unix tests unchanged and green; the Windows path is compile-verified for
x86_64-pc-windows-msvc (runtime coverage lands with #228's CI lane).
2026-08-09 22:39:31 -05:00
beardthelion 63cb52b253 test(git-remote): bind the withheld-path test to the expected ACK/NAK rejection
The test accepted any nonzero git exit (with posts>=1) as the recorded #191
withheld rejection, so a truncated response or a later HTTP failure after the
first POST could pass green. Assert the specific `expected ACK/NAK` signature so
only that break passes, and pin LC_ALL=C so the diagnostic stays the untranslated
English string on a git build with l10n installed. (#192 F6)
2026-08-09 22:39:31 -05:00
beardthelion 5a5d4e1b43 fix(node): count fetch completion from the response outcome, not the request
The gitlawb_fetches_total gate keyed on whether the request carried `done`,
which both undercounts a no-done completion (a client can finish a flush-
terminated have round with `ACK <oid> ready` + pack and no `done` -> counted 0)
and, on the filtered path, counts a rejected response per POST (the pre-#191
two-POST negotiation the client refuses was counted twice).

Derive completion from the response instead. upload_pack / upload_pack_excluding
now return whether they served a pack; response_served_pack detects the PACK
magic in both side-band-64k and raw framing. The plain path counts when the
response delivered a pack (catches no-done, skips negotiation-only rounds so an
N-round fetch counts once); the filtered path stays gated on the finalizing
`done` round until #191 makes that negotiation valid. Also tighten
request_finalizes_fetch to reject 0003 framing and trailing bytes after the
`done` pkt-line. (#192 F1/F2 + CodeRabbit repos.rs:761)
2026-08-09 22:39:31 -05:00
beardthelion c45ec9605d fix(node): make metrics::init race-safe with Once
init() did an unsynchronized REGISTRY.get() check then OnceLock::set(...).
expect(...) for each metric, so two concurrent callers both passed the check
and the loser panicked at 'set INFO once'. Two tests call init(), so the
parallel test runner could hit this. Guard the body with std::sync::Once so it
runs exactly once. Verified: a 32-thread barrier stress-test panics before the
fix and passes after (run in isolation; not committed since the process-global
registry makes the race non-deterministic in the shared-binary suite). (#192)
2026-08-09 22:21:00 -05:00
beardthelion 8c04043fb5 fix(git-remote): reject malformed pkt-line length headers instead of flushing
A non-UTF-8 or non-hex 4-byte length prefix was collapsed to 0000 via
unwrap_or, so a garbled header was handled exactly like a flush -- masking
corrupted/desynchronized protocol input and letting accumulated have lines
go out as a synthesized flush round. Propagate the parse error instead;
a real 0000 still parses as a valid flush. (#192)
2026-08-09 22:21:00 -05:00
beardthelion bf4d8cd9d4 test(git-remote): bound the fetch reader joins when the leader exits before a pipe-holder
run_bounded only tore down the process group on the deadline path. When the git
leader exits on its own but a descendant (the git-remote-gitlawb helper mid-HTTP
request) outlives it holding the inherited stdout/stderr write-ends, the reader
joins blocked on the helper's ~300s timeout, past the deadline run_bounded
promises. Close the group on the clean-exit path too, guarded by a kill(-pgid, 0)
liveness probe so a recycled pid is never signalled (the process-group ID stays
reserved while any member is alive). Adds a leader-exits-first fixture and a
regression test.

Verified by execution: with the clean-path reap removed the new test blocks ~10s
on the descendant (RED); with the reap it returns promptly.
2026-08-09 22:21:00 -05:00
beardthelion 1c690c125c fix(node): count a completed fetch once per fetch, not per stateless-RPC POST
git_upload_pack recorded a fetch and observed a pack size on every successful
POST. In HTTP stateless-RPC the plain serve path streams a pack only on the
finalizing `done` round, so an N-round incremental fetch was reported as N
completed fetches. The filtered path (upload_pack_excluding) instead serves a
self-contained full pack on the single POST that reaches it, with no `done`, so
gating purely on `done` would drop those private (path-scoped) fetches to zero.

Count once on the POST that actually completes a fetch: the `done` round on the
plain path (request_finalizes_fetch parses pkt-lines and fails closed on a
malformed body), or the one filtered-serve POST (served_filtered_pack).
should_count_fetch captures the decision; a completed fetch is exactly one such
POST, so a multi-round negotiation is never double-counted.

The observe_pack_size call still measures the request body, not the served pack;
that mislabel predates this change and is left as a follow-up.

Verified by execution: a three-round plain fetch and a no-`done` filtered serve
each record exactly one completion; reverting the gate to count per POST makes
the plain fetch record 3, and breaking the decision to AND drops both to 0.
2026-08-09 22:21:00 -05:00
beardthelion 22ec66a846 test(git-remote): make the run_bounded harness portable to the shipped windows target (#192)
The large-output and descendants tests re-invoke the test binary as their
fixtures instead of spawning sh/seq, so they run on x86_64-pc-windows-msvc;
the non-unix timeout path tears down the whole fetch tree with taskkill /T
before joining the drainers instead of killing only the leader. Also folds
the receive-pack POST block into post_pack_round (review nitpick), removing
the duplicated status-check/forward logic.
2026-08-09 22:21:00 -05:00
beardthelion 4201ce4d58 test(git-remote): reap the whole fetch process group and make the withheld guard load-bearing (#192)
Address jatmn's two P2 review findings on real_git_fetch.rs.

F1 (INV-22): run_bounded killed only the top-level `git fetch` on the
timeout path, so the git-remote-gitlawb helper descendant kept the
inherited stdout/stderr pipe write-ends open and the reader joins blocked
on the helper's ~300s HTTP timeout instead of the 30s deadline. Spawn the
child with process_group(0) and, on timeout, tear the whole group down
(SIGTERM, ESRCH poll, SIGKILL escalation, bounded cap, then reap) before
joining, mirroring gitlawb-node/src/git/smart_http.rs. libc is added as a
cfg(unix) dev-dep; non-unix keeps the child.kill() fallback so the
always-on test still compiles. New regression run_bounded_reaps_descendants_holding_the_pipe:
a backgrounded sleep inherits the pipe and outlives a 1s-deadline leader;
reverting to a leader-only kill turns it RED (elapsed ~10s).

F2 (INV-21): the non-success branch of real_git_withheld_shaped_first_post
accepted any `git fetch` failure and only printed the POST count, so a
failure before the first POST (broken advertisement, helper lookup,
connection) passed as the expected withheld rejection. Assert posts >= 1
before accepting the nonzero exit. Forcing a pre-POST failure (0 POSTs)
turns the assertion RED.
2026-08-09 22:21:00 -05:00
beardthelion 1cb930cbd5 fix(git-remote): abort fetch on EOF without POSTing; harden the real-git test harness (#192)
F1: a nonempty have-batch terminated by EOF means git aborted mid-negotiation, but
the fetch loop folded RoundEnd::Eof into the Done arm, appended a synthetic
0009done, and POSTed it — so a cancelled private fetch still issued an authorized,
signed upload-pack request and made the node generate and stream a full pack to a
client that had gone away. Split the arm: only a real done pkt finalizes; a
nonempty EOF terminates without a POST. The nonempty_eof_batch_posts_as_final_round
test that locked in the wrong behavior is flipped to assert no POST.

F2: fetch_with_helper piped stdout+stderr but only polled try_wait, so a fetch
emitting more than an OS pipe buffer (~64 KiB) blocked git before it exited and the
harness reported a false 30s deadlock. Drain both streams on reader threads while
enforcing the deadline, so the deadline measures a real protocol hang only.

F3: the integration repos were removed only on the success path (predictable
pid/counter names), so a timeout/panic leaked real git repos and could poison a
later run. Root them in a RAII tempfile::TempDir that drops on unwind.

RED->GREEN: nonempty_eof_batch_aborts_without_posting (pre-fix POSTs a synthetic
done → the aborted-negotiation POST fails against the single-response server; fixed
makes one request, no POST). repos_are_cleaned_up_on_unwind (temp dir gone after a
panic unwinds past the guard). Full git-remote-gitlawb suite 42 unit + 3 integration
green, fmt + clippy clean.
2026-08-09 22:21:00 -05:00
beardthelion c4609d464a test(git-remote): close branch-coverage gaps in the fetch loop (#117)
Adds executed coverage for the negotiation branches the first pass left
unrun: an invalid pkt-line length (rejected, not underflowed), a non-EOF
read error (propagated, not swallowed as end-of-stream), an immediately
empty upload-pack request (skips the POST), a nonempty have-batch
terminated by EOF (POSTed as a final done round), and an empty
receive-pack body (skips the POST). Each was mutation-checked to fail
without its guard, so the coverage is load-bearing rather than vacuous.
2026-08-09 22:21:00 -05:00
beardthelion 68ba49267c fix(git-remote): skip content-free flush rounds in the fetch loop (#117)
A code-review pass found the negotiation loop would POST once per bare `0000`
flush that carried no haves, so a malformed `wants + N*0000 + EOF` stream
amplified into N signed POSTs. Real git never sends a content-free mid-negotiation
flush and the peer is upstream of the local git process, so this was not
reachable in practice, but it left the loop unbounded per input. Skip a round
that carries no new haves (an empty flush) and only POST rounds with have content
or a terminator, which also makes the loop provably bounded by git's finite have
set. The fresh-clone done-with-no-haves round still POSTs.
2026-08-09 22:21:00 -05:00
beardthelion 89c0ad597f test(git-remote): real-git integration proof of the multi-round fetch loop (#117)
A committed integration test drives a real `git fetch` through the built helper
against a real `git upload-pack --stateless-rpc`, so the stateful-to-stateless
ACK bridging is proven by execution rather than reasoned. The main.rs mock tests
cannot falsify it: a pre-scripted Cursor never reacts to the server's ACKs. An
in-test shim replicates the node's v0 serving; the fetch is forced to at least
two negotiation rounds (a fixture that resolved in one round fails the test) and
the resulting object graph is verified.

The second scenario drives the withheld-blob shape (a full pack on the first
POST, as upload_pack_excluding does) through real git and records the outcome:
real git rejects a pack where it expected an ACK continuation ("expected
ACK/NAK, got ..."), so a multi-round fetch of a withheld repo does not complete.
The helper forwards and terminates cleanly rather than hanging, so this is a
node-side concern, not a helper defect, left as a follow-up per the plan's
Withheld-Path Decision. The test guards the helper's non-hang behavior and
surfaces the break if the assumption ever changes.
2026-08-09 22:21:00 -05:00
beardthelion a64b14150d fix(git-remote): drive multi-round fetch as a v0 stateless-RPC client loop (#117)
git-remote-gitlawb advertised `connect`, so git spoke the stateful native
protocol, but handle_connect collapsed the exchange into one GET plus one POST.
A multi-round fetch (more than ~32 overlapping commits) deadlocked: git sent a
flush-terminated have-batch and blocked for an ACK/NAK the helper never sent,
while the helper blocked reading for a `done` git never sent.

Turn Phase 2 into a per-round stateless-RPC client loop. read_upload_pack_round
returns one round at a time (at a flush, done, or EOF) instead of buffering past
flushes waiting for `done`; negotiate_upload_pack captures the wants once and
POSTs a self-contained request per flush-terminated batch: wants, one flush,
every have accumulated so far, and exactly one terminator. Each ACK/NAK is
streamed back to git so it advances, until the done round returns the pack. The
node's `git upload-pack --stateless-rpc` keeps no state between POSTs, so wants
and all prior haves are re-sent every round and no intermediate flush survives
into a body (which would truncate the negotiation server-side).

Signing is preserved per round: every POST carries the Phase-1 decision (signed
after the 404 escalation for a private repo, anonymous for a public one), and a
mid-negotiation denial surfaces through the sanitized error path rather than
reading as an empty or successful fetch. The receive-pack path is unchanged.

Covers the deadlock repro, multi-round accumulation on the wire, per-round
signing for private and public fetches, mid-negotiation denial surfacing, the
withheld-shaped forwarding path, and single-round non-regression.
2026-08-09 22:21:00 -05:00
mike 337442e449 test: pin gossip readiness persistence 2026-08-09 23:12:12 -04:00
mike 74f7554894 fix(node): bound peer failure tracking 2026-08-09 23:10:59 -04:00
mike 6bd8f41212 fix(node): harden peer readiness state 2026-08-09 23:10:59 -04:00
mike 5af14c203d fix(node): bound legacy readiness fallback 2026-08-09 23:09:55 -04:00
mike 547f9aed10 fix(node): preserve legacy peer compatibility 2026-08-09 23:09:55 -04:00
mike 1a841a99e2 fix(node): use readiness for peer liveness 2026-08-09 23:09:55 -04:00
Kevin Codex c10ccf1f05 Merge pull request #255 from Ayush7614/fix/250-opaque-graphql-db-errors
fix(node): opaque GraphQL DB error messages (#250)
2026-08-10 10:24:15 +08:00
Kevin Codex 7b365b0dc3 Merge pull request #249 from MikeTomlin19/agent/harden-release-tag-239
fix(ci): reject unsafe release dispatch tags
2026-08-10 10:22:10 +08:00
Kevin Codex 218d6f169a Merge pull request #214 from Gravirei/fix/issue-211-icaptcha-mock-consumed-guard-gaps
test(icaptcha): harden mock-consumed guard with proxy observer and blackhole tripwire
2026-08-10 10:20:24 +08:00
Kevin Codex 817698a0d9 Merge pull request #209 from Gitlawb/docs/agents-md
docs(agents): add AGENTS.md carrying the authorization rules PRs most often break
2026-08-10 10:16:53 +08:00
Kevin Codex e969dca718 Merge pull request #245 from Gitlawb/test/icaptcha-empty-key-filter
test(icaptcha-client): bind new()'s empty-API-key filter (#216)
2026-08-10 10:12:53 +08:00
Kevin Codex 1989fcca11 Merge pull request #202 from euxaristia/fix/pr-triage-async-test-detection
fix(ci): recognize namespaced test attributes in pr-triage's needs-tests check
2026-08-10 10:04:53 +08:00
Kevin Codex 5981257be5 Merge pull request #174 from Gitlawb/fix/served-git-concurrency-cap
feat(node,git): cap concurrent served git ops with a 503 load-shed (#62)
2026-08-10 10:00:06 +08:00
beardthelion 91ff1b7237 docs(node): correct the read-pool comment and list the served-git concurrency knobs
The git_read_semaphore doc claimed it bounds both info/refs advertisements,
but the receive-pack advertisement draws from git_push_advert_semaphore
(api/repos.rs), so an operator sizing the read pool for both would undercount
their read capacity.

README's settings table also omitted the served-git concurrency env vars that
.env.example already documents: MAX_CONCURRENT_GIT_OPS, MAX_CONCURRENT_GIT_PUSHES,
MAX_CONCURRENT_READS_PER_CALLER, and MAX_CONCURRENT_PIN_TASKS.
2026-08-06 23:27:03 -05:00
beardthelion dac29e550c docs(env): stop claiming the pin pool bounds retained memory
GITLAWB_MAX_CONCURRENT_PIN_TASKS caps how many pin loops run at once, not how
much object-id list memory is retained. On the local IPFS path the caller
materializes the list before pin_new_objects_gated acquires, so a parked loop
still holds its full list and the parked count is capped only per repo by
EncryptInflight. config.rs and the pin_new_objects_gated docstring already say
this; the env template still said the knob bounds the memory.
2026-08-06 21:54:31 -05:00
Gravirei a27a267871 fix(gl): require non-empty error message and use ephemeral port in whoami tests (#220) 2026-08-06 13:18:29 +06:00
Gravirei b4007a5e00 fix(gl): assert whoami 404 verdict and preserve transport error cause (#220) 2026-08-06 11:58:11 +06:00
Gravirei a329174175 fix(gl): assert 404 verdict and harden transport error handling in whoami (#220) 2026-08-06 11:57:23 +06:00
Gravirei fd78cfa3ad fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:15 +06:00
Gravirei 3cc15cf724 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei 16cc3b37e8 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei 3f41df5868 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei b0e5cc0ee0 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei 12bc759ea6 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei 10e3492825 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei c10f2ca255 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:39:07 +06:00
Gravirei 75b0df8f1c fix(whoami): update registered status handling on 404 response 2026-08-06 11:39:07 +06:00
Gravirei 3add5356e3 fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:38:47 +06:00
Gravirei fdd11030dc fix(gl): only report registered: false on 404 in whoami (#220) 2026-08-06 11:38:47 +06:00
Gravirei dc6b19ddc4 test(icaptcha): tag the observer flush by source port, not by counter tick
The drain barrier sampled target = count + 1 before connecting the flush and
waited for any next increment. A leak accepted after that sample and before
the flush was counted satisfied the wait; the test then subtracted the flush
as if it were that leak, so leak_count could be 0 with a real leak present
(false green).

Tag the flush connection instead: the accept loop records each accepted
connection's source port, and drain waits until the flush's own source port
is recorded (the flush is never counted). Because accept() is FIFO, when the
flush is recorded every connection that completed its handshake before it is
recorded too, so the post-drain count still covers all egress in flight.
Verified with a probe that reproduces the reported scenario (leak accepted
before the flush is now counted) before removing it.
2026-08-06 11:36:03 +06:00
Gravirei 89e10d0ff1 fix(repo_store): use SHA-256 for stable advisory-lock key (#210)
DefaultHasher is not frozen by the Rust standard and has shifted across
toolchain versions, so the same (owner_slug, repo_name) produced a
different i64 advisory-lock key on different builds and broke
cross-machine write-exclusion for the same repo.

Hash with SHA-256 instead and pin the result with a golden-value test,
plus a one-axis-at-a-time differential test so a regression that drops
either parameter is caught. Document the SHA-256 re-keying caveat on
acquire_write and in the RUN-A-NODE.md operator checklist (drain writes
or cut over single-node during a rolling upgrade).
2026-08-05 19:26:44 +06:00
Gravirei 59378ea4e3 test(icaptcha): use [::1] for the blackhole tripwire fixture
The tripwire bound a hard-coded 127.0.0.2 alias, which macOS does not route
by default (only 127.0.0.1), so the test could not run on apple-darwin
targets.  Switch the fixture to [::1] (IPv6 loopback), which is present by
default on Linux, macOS, and Windows and is NOT covered by NO_PROXY
(127.0.0.1, localhost).

The negative control still holds: hyper-util's proxy matcher has no implicit
loopback bypass — a host is only proxied-around when literally listed in
NO_PROXY (verified in the pinned hyper-util source).  Both the positive
control (direct reach of [::1]) and negative control (proxy intercept) still
pass.
2026-08-05 14:31:53 +06:00
Gravirei f741c72103 test(icaptcha): address round-3 review findings on guard and tripwire
- Tripwire: fail loudly (expect) when 127.0.0.2 cannot be bound, instead of
  printing SKIP and returning green with zero coverage of the negative
  control.  Linux routes 127/8 as loopback; macOS needs an explicit alias,
  and the message says so.
- Observer: restore a bounded drain/ack barrier (Mutex + Condvar).  The
  detached accept thread signals after each accept; the test opens a flush
  connection and blocks until it is acknowledged.  accept() is FIFO, so every
  leak connection that completed its handshake before the flush is counted
  before the flush.  Leaks that start their handshake after the drain remain
  the irreducible window, now documented as such.
- support: document that arm_blackhole / disarm_proxy_env mutate process-wide
  env vars and are safe only because each consuming binary has exactly one
  test; a future second test must be serialized against them.
2026-08-05 14:16:40 +06:00
Gravirei 366120e081 test(icaptcha): remove unbounded stabilise polling from mock-consumed observer
The stabilise() polling loop cannot prove no more connections are coming,
only that none arrived in the last 100ms. A carefully timed background
leak could schedule its connection after stabilise stabilises and the test
would pass despite a live egress.  Replace with a simple counter read
before and after obtain_proof.  Connections accepted during the synchronous
call are counted; a leak whose TCP handshake completes after the counter
is read is outside the assertion window — this is now documented as a
known limitation of the single-threaded test harness.

Also narrow the fire-and-forget claim: the observer catches leaks that a
closed-port blackhole would miss (because it counts connections regardless
of error propagation), but the timing race inherent in a detached accept
thread means the guarantee is best-effort for post-return handshakes.
2026-08-05 14:06:53 +06:00
Gravirei 68a68a30f9 test(icaptcha): fix tripwire fixture read deadlock 2026-08-05 14:06:53 +06:00
Gravirei a1b96813ba test(icaptcha): address review findings on mock-consumed guard (#211) 2026-08-05 14:06:53 +06:00
Gravirei 419c2bd230 test(icaptcha): harden mock-consumed guard with proxy observer and blackhole tripwire (#211)
Replace the closed-port proxy with a ProxyObserver (TCP listener with
acknowledged accept-loop flushing) so fire-and-forget leaks whose error
is discarded are caught and the count is synchronized before assertion.
Add a blackhole tripwire asserting obtain_proof fails against 127.0.0.2
with the proxy armed, narrowed to a connect-level error to prevent
vacuous passes. Share arm_blackhole across both test files via a
tests/support module.

Document the observer's coverage scope: it only witnesses leaks on the
single exercised happy path; error/retry branches and non-proxy
transports are outside its guarantee.
2026-08-05 14:06:53 +06:00
mike dfe52dc8dd test(ci): pin release resolver checkouts 2026-08-05 01:22:10 -04:00
mike bc0895299c test: snapshot resolver whitespace 2026-08-05 01:22:10 -04:00
mike fa5cd65031 test: pin release resolver steps 2026-08-05 01:22:10 -04:00
mike 4c1d7814d4 test(ci): bind resolver job mutation 2026-08-05 01:22:10 -04:00
mike af5a3888a8 test(ci): harden release workflow guard 2026-08-05 01:22:10 -04:00
mike ab653e9b94 test(ci): stabilize release bypass fixture 2026-08-05 01:22:10 -04:00
mike fe07564837 test(ci): harden release tag drift guards 2026-08-05 01:22:10 -04:00
mike eac0cf7bca test(ci): enforce release tag invariants 2026-08-05 01:22:10 -04:00
mike 62513c8555 test(ci): cover remaining invalid release tags 2026-08-05 01:22:10 -04:00
mike 1b65f70403 fix(ci): add tested release tag resolver 2026-08-05 01:22:10 -04:00
mike 0b990d687f fix(ci): reject unsafe release dispatch tags 2026-08-05 01:22:10 -04:00
beardthelion 5b54814929 fix(node): stop reporting an unreadable object store as readable, and bound the pin permit
Two review findings from the last round.

The object-store readability probe swallowed EACCES on objects/pack exactly
like the benign NotFound of a loose-only store, and never checked the loose
fan-out at all. A present object behind an unreadable directory therefore
looked absent: git cat-file printed a clean missing, the probe said the store
was fine, and /ipfs/{cid} answered 404 for content that exists. The same bool
drives classify_store_fault, so an unreadable store also came back as a
terminal 500 rather than a retryable 503. Only NotFound stays benign now, and
the probe additionally opens the object's own fan-out path. That check is
scoped to the oid (validated as 40 or 64 hex before any path component is
derived) rather than sweeping all 256 directories, so an unrelated broken
directory cannot veto every lookup in the repo, and the caller pays one open
instead of a directory drain it does not need. An unreadable directory now
fails closed: absence cannot be certified through something we cannot read, so
a missing object there sheds a retryable 503 instead of a 404.

Both pin loops held the shared pin permit across a synchronous, unbounded
git read, so a wedged filesystem pinned a slot in a pool that defers rather
than sheds, and blocked a runtime worker while it did. The read now runs
through the bounded, process-group-reaped helper under spawn_blocking, against
the batch deadline, and the Pinata loop gets the batch budget it never had.
The budget gate is shared between the two loops so they cannot drift, and it
declines to start a read it cannot pay for rather than spawning a child only
to reap it.

A fault stops the batch only when the object store is genuinely unreadable,
where every remaining object would fail the same way. An object-scoped fault
skips that object and continues, because one corrupt object should not cost a
healthy batch, and the recovery path re-offers the same list and would break at
the same place forever.
2026-08-04 17:44:59 -05:00
beardthelion 8ccf2b12e4 docs(agents): drop retired ungated exceptions; fix forged-cert test rule
Jatmn's #209 round: known_ungated is empty on main and the three listed
handlers are gated, so the guide must not teach them as exceptions. Restore
the signature/certificate section with a trusted positive case and a forged
rejection case instead of blessing a forged "valid" drive.
2026-08-04 14:18:26 -05:00
beardthelion c8a6941941 docs(agents): correct three inaccurate claims flagged in review
Scope the denial-test guidance to each route's actual authorization
rule instead of a blanket non-owner assertion, since several
documented routes (star_repo, replica actions, bounty actions,
author-based closing) intentionally authorize non-owners.

Name the known_ungated exceptions (list_webhooks, list_replicas,
list_protected_branches) instead of implying full repo-scoped read
coverage.

Describe the listed jobs as workflow checks that run on every PR
rather than merge-blocking CI, matching the live main branch ruleset
(one required review, no required status checks).
2026-08-04 14:17:41 -05:00
beardthelion 6c9d109c1b docs(agents): add AGENTS.md carrying the authorization rules PRs most often break
Encode the contributor-facing gate rules (the three owner-gate forms in
use and the by-design non-owner exceptions, path-scoped reads and the
listings contract, denial semantics, deny-test expectations, migration
versioning, client denial surfacing, MSRV and the exact CI gate set) in
a root AGENTS.md that coding agents load automatically, and point
CONTRIBUTING.md at it. Enforcement labels state only what is on main
today.
2026-08-04 14:17:40 -05:00
Ayush7614 726ba3361b fix(node): fail-closed GraphQL AppError mapping (#255)
Whitelist eight safe AppError variants; opaque Git and anything else.
Add a source-scrape guard for map_err sites, a real sqlx context chain
test, and a tasks(limit) ceiling pin at 200.
2026-08-04 12:00:07 -05:00
euxaristia 13f6948b3e fix(ci): use anchored regex for pr-triage needs-tests detection
Drop the head-file fetch and Rust lexer. Match test attributes on
added lines only, with horizontal-whitespace anchoring, covering bare
#[test], #[cfg(test)], namespaced forms (#[tokio::test(...)]), optional
leading ::, and raw-identifier spellings. needs-tests is advisory, so
the simpler matcher is enough.
2026-08-04 12:00:07 -05:00
Ayush7614 5077883ad1 fix(node): surface safe AppError variants in GraphQL (#255)
Keep Db/Internal opaque; pass through NotFound/BadRequest/etc. and document
the tasks limit clamp in the schema.
2026-08-04 12:00:07 -05:00
euxaristia 8319c20660 fix(ci): make bounded scan inconclusive, bound blob size, handle no-newline marker, recognize raw-ident test
Addresses jatmn's review: an exhausted 15-file scan or an oversized/
unreadable blob no longer asserts "no test found" (needs-tests was
being applied on unverified evidence), blobs over 5MB are skipped
before decoding, the "\ No newline at end of file" patch marker no
longer miscounts as an added context line, and #[r#test] /
#[tokio::r#test] raw-identifier attributes are now recognized.
2026-08-04 12:00:07 -05:00
Ayush7614 030c5bd3a4 fix(node): address #255 review (clippy, sqlx-only opaque)
Use function-pointer map_err to clear redundant_closure. Opaque only
when a sqlx::Error is in the anyhow chain so claim/finish business
messages survive. Clamp GraphQL tasks(limit); add mutation coverage.
2026-08-04 12:00:07 -05:00
euxaristia fa51d428cb fix(ci): stop bounded scan from aborting triage, use blob API, tighten matching
Addresses jatmn's second round of review on pr-triage's needs-tests check:
hitting the per-PR fetch cap now logs and stops scanning instead of
throwing (which was skipping label reconciliation entirely), file content
comes from the Git blob API instead of the Contents API to avoid the
encoding: none gap for 1-100MB files, the added-line check now requires a
non-whitespace attribute character (not just added whitespace) on the
added line, stripped literals leave a placeholder so they can't glue
adjacent tokens into a false match, and the namespaced-test regex now
accepts a leading :: for absolute-path attributes.
2026-08-04 12:00:07 -05:00
Ayush7614 6b1e1b229b fix(node): opaque GraphQL DB error messages (#250)
Route query and mutation DB failures through a shared helper that
logs the cause chain server-side and returns a generic client
message, so unauthenticated POST /graphql cannot leak sqlx text.
2026-08-04 12:00:07 -05:00
euxaristia 24952b16be fix(ci): handle nesting, multiline attributes, precise terminal matching, and bound Contents API scans 2026-08-04 12:00:07 -05:00
euxaristia 51f72ee3be fix(ci): lex the full head file so state survives across diff hunks
A diff hunk carries no lexical context from lines outside it, so scanning
patch text alone could misclassify an added line that sits inside a comment
or raw string opened earlier in the file (or outside the hunk's context
window). Fetch the head-version file content via the contents API and lex it
in file order, checking only the lines the diff actually added.
2026-08-04 12:00:07 -05:00
euxaristia e458cc2d84 fix(ci): robustly strip comments/strings and match parameterized tests 2026-08-04 12:00:07 -05:00
euxaristia 474b42cfc5 fix(ci): update pr-triage needs-tests regex to anchor attributes and cover parametrized tokio::test 2026-08-04 12:00:07 -05:00
euxaristia 6452e996f8 fix(ci): recognize namespaced test attributes in pr-triage's needs-tests check
The needs-tests detection regex only matched a literal #[test] or #[cfg(test)
immediately after #[, so it missed namespaced test attributes like
#[tokio::test] or #[async_std::test]. Any PR whose only new tests were async
(the common pattern in this codebase, e.g. crates/gl/src/agent.rs) got falsely
labeled needs-tests even when tests were added in the same commit.

Reproduced against PR #198: it added two #[tokio::test]-only tests in the same
commit as the source change, and the triage bot still commented needs-tests.

Widens the regex to also match any attribute ending in ::test], verified
against representative patches including PR #198's actual diff.

Closes #201
2026-08-04 12:00:07 -05:00
beardthelion 753a13b093 test(node): widen the leader-reap test deadline past the child's start race
The `test (beta)` lane failed on 17f9eda reading `leader.pid`, NotFound, while
`test (stable)` passed the same commit. The test armed a 100ms watchdog deadline
and the fake git records its pid as its first statement, so a runner slow enough
to delay `/bin/sh` past 100ms gets the whole process group torn down before the
pid is ever written.

Reproduced by delaying the fake git's first statement by 300ms, which yields the
identical NotFound panic at the same line. With the deadline at 2s the same
injection passes at 0.3s, 1.0s and 1.8s of start delay, so the race window moves
from 100ms to 2s.

The property under test is unchanged: a pre-fix build stands the watchdog down on
drain and blocks on `child.wait()` for the child's full `sleep 30`, which still
trips the 10s recv budget. Confirmed by injecting that exact defect, which goes
red on the budget assertion rather than on some other failure.
2026-08-04 11:31:05 -05:00
beardthelion 5df7d2afb3 test(gl): pin the local-add print selection, not just its helper
local_add_refusal had six unit tests and the wiring had none, so
replacing the whole match at the call site with an unconditional "Added
to local peer list." left every test green. That is the shape the store
already names: a helper test plus a status-mapping test does not prove
the call site.

Extract the selection into local_add_report, following the same
print-selection-helper precedent used for the certificate DID hint, and
assert both directions: a refusal never yields the success line, and a
200 never yields the warning. Collapsing the match to always return the
success line now turns that red. The call site is left as a two-line
dispatch carrying no logic.
2026-08-03 20:38:53 -05:00
beardthelion 8313c10c5b test(node): make the peers-write scan see a statement split across lines
The scan matched a verb-plus-table needle per line, so it was whitespace
and case sensitive. The identical statement written across lines walked
straight past it, and that form is how the longer queries in this file
are already written, so it is the shape a new writer most likely takes.

Normalize the whole file before matching, mapping offsets back to line
numbers so each statement is still attributed to the function issuing
it. The scan is split into a helper so the property can be asserted
against a synthetic source: the tree contains no multi-line peers write
today, so a test that only ran against the real tree would prove nothing.
Fixtures derive from the needles at runtime for the reason the needles
themselves do, which is that a literal here is found by the scan of this
very file.
2026-08-03 20:38:53 -05:00
beardthelion d6434b287b fix(node): scope the unproven DID check to the insert, not to the DID
The check ran before anything looked at whether the row existed, so it
refused a pure liveness refresh of a row already in the table. Nothing is
protected by that: the row is already there, and an identical-URL refresh
changes no authority. What it did instead was freeze every peer a
deployed database already holds from before this gate, since upsert_peer
validated nothing then and the handler accepted any parseable DID. Those
rows take a 400 on every unsigned announce and cannot be corrected by a
signed one either, because no key resolves.

Now the DID is judged only when no row exists, which is the only place
refusing it prevents anything. The read-then-write race is benign: if a
row appears in between, the write becomes an update and the existing
repoint guard still refuses any http_url change.
2026-08-03 20:38:53 -05:00
beardthelion 895fcf05d8 fix(gl): check the peer's status before parsing its announce reply
The capped read means a peer answering with a body past the cap leaves
truncated JSON behind, and parsing that with ? before the status check
turned a 503 into "invalid JSON response", losing the status the user
needs. A non-JSON error page from a caller-chosen peer URL does the same.
The failure check now runs first and falls back to the body text when
there is no message field. Bound by a test that drives the command
against a mock peer, since reordering the two statements compiles and
passes every unit test in the file.
2026-08-03 20:38:53 -05:00
beardthelion 8b3aa1decf fix(gl): defang and bound node-supplied text before it reaches the terminal
The peer-add refusal warning added earlier in this branch copied the
node's message straight into stderr, and both announce bodies were read
unbounded, so a hostile --node endpoint, a compromised local node, or a
MITM on plain http could put ANSI or OSC escapes into the user's
terminal. The accepted-announce block had the same hole on the success
path, where the DID and URL come from a peer whose URL the caller chose.

sanitize_node_msg and read_body_capped move from sync.rs to http.rs,
which already owns the wire boundary, and both paths in peer.rs now use
them. Only the display is sanitized: the values forwarded to the local
node stay verbatim, since that is data rather than terminal output.
2026-08-03 20:38:53 -05:00
beardthelion e8f091e1c4 fix(node): give the unresolvable-DID refusal its own error code
The split reported the cause in the message but left the code as
bad_request, so a client telling this apart from the other validation
failures on the announce route had to substring-match prose. The auth
middleware already answers unresolvable_did for the same failure on a
request's keyid; the peer-write path now answers the same way. Status is
unchanged at 400, and an unsupported DID method keeps bad_request.
2026-08-03 20:38:53 -05:00
beardthelion df2a1797d1 fix(node): report an unresolvable did:key distinctly from an unsupported method
The unproven insert path folded every to_verifying_key failure into
UnsupportedDidMethod, so an oversize, malformed, or non-ed25519 did:key
was told its DID method is unsupported, which is false and leaves the
caller nothing to act on. Split the arm by cause: a parse or resolution
failure now returns UnresolvableDid carrying the underlying reason, in
the same wording the signed path already uses, while a foreign method
keeps methodNotSupported. Every input stays refused in the 400 class.
2026-08-03 20:38:53 -05:00
beardthelion ebb93e3231 test(gl): pin the local-add refusal warning to the node's production wording
The refusal test fed local_add_refusal an invented message, so it could
never go red when the node's wording changed. Drive it with the Display
of PeerWriteDenied::UnprovenRepoint and assert the same substring the
node-side pin asserts against a real router response.
2026-08-03 20:38:53 -05:00
beardthelion b6ebaa4b1e fix(gl): surface the node's refusal instead of reporting a peer add that did not happen
The announce route now answers 403 when an unproven caller tries to
repoint an existing peer. gl peer add discarded the local node's reply
with let _ and printed the success line unconditionally, so a refused add
reported as a completed one. The refusal is split into a small function so
the path is assertable, and it is tested for a 403 with a message, a
refusal with no body, and the accepted case. The local add stays best
effort: a transport failure warns rather than failing the command, which
is what the surrounding code already assumed.

The README described the staged rollout as accepting unsigned peers on
these routes, which is no longer the whole rule. With signed peer writes
off, an unsigned announce may still register an unseen peer and refresh a
row whose URL is unchanged, but changing an existing peer's http_url needs
a signature from that peer's own DID, and only a did:key can be registered
unsigned.
2026-08-03 20:38:53 -05:00
beardthelion 73bf132c1a fix(node): resolve the key on the unproven insert path, and bound the decode
The gate restricting unproven inserts to did:key checked the method label
only. Did::from_str validates that the method is one of key, web, or
gitlawb and never inspects the key material, so did:key:notarealkey passed
and created exactly the row nobody can ever correct that the gate exists
to prevent. It now resolves the key the way the signature path does.

Six test fixtures using invented did:key strings stopped inserting once
the gate became real. That failure is the evidence it now reads key
material rather than a prefix, and it means those fixtures were exercising
a path no real DID could take. They carry derivable values now.

Resolving the key put a quadratic base58 decode on a route that accepts
anonymous callers by default. Measured through the router, a 64,000
character did:key cost 8.05s against 4.6ms for the previous label check,
on a tokio worker with no spawn_blocking, and a 1.5 MB body reaches the
handler. The bound sits in to_verifying_key so it also covers the
HTTP-signature keyid, which reads an untrusted string through the same
function. An ed25519 method-id is a fixed 48 characters, so 64 is slack.

Its test asserts the error names the length rather than merely being an
error, because an oversized id fails either way and a check for is_err()
would pass with the guard removed, having already paid the cost.

Three last_ping_ok assertions were vacuous: the seed helper never granted
reachability, so they compared false against false and would have held
against code that never cleared the flag. The helper grants it now and
asserts the seed took. Nothing was hiding behind them.
2026-08-03 20:38:53 -05:00
beardthelion 7cdfb843ff test(node): bind the peer-authority gate and ledger its writers (#273)
Drives the must-not set through the production router with real RFC 9421
signatures rather than injected extensions, so the signed path is exercised
as a caller would reach it, in both require_signed_peer_writes modes.

Covers the allowed cases and every rejection: unsigned insert of an unseen
DID, unsigned repoint refused with the row byte-identical after, a
signature from the row's own DID accepted, a signature from a different
DID refused, an identical-URL re-announce refreshing only last_seen, and a
did:web announce refused with no row created.

The signature-from-a-different-DID case is the RUSTSEC-2022-0009 shape,
where libp2p-core accepted a valid signature without checking it derived
the claimed identity. It asserts the key is bound to the row rather than
to anything the request carries.

The announce handler's keyid-mismatch branch had no test at all and
returned 400. It now returns 403 and is tested: a caller denied for want
of proof of control over the targeted row gets the same response class
whether or not it presented a signature.

The completeness ledger derives its set from every writer of the peers
table rather than from upsert_peer's callers, because a caller scan cannot
see a future writer that issues its own SQL. A second table records the
call-site authority for main.rs's bootstrap announce-back, which issues no
SQL and so can never appear in the writer ledger; its authority choice has
no runtime test in this change and is recorded as such.

The type system carries the guard: the authority parameter has no default,
so a new caller cannot omit it. The scan is the backstop for a raw
statement that bypasses upsert_peer entirely, and stays minimal rather
than copying the handler-directory apparatus, which does not apply to a
scan over SQL literals.
2026-08-03 20:38:53 -05:00
beardthelion 14481df288 fix(node): bind a peer row to its DID at the upsert_peer boundary (#273)
upsert_peer ended in an unconditional ON CONFLICT DO UPDATE SET http_url,
so any unauthenticated caller could repoint any peer. A repointed row
steers the sync worker's git remote, the post-receive notify fan-out,
trigger_sync, and the public GET /api/v1/resolve/{did}, none of which
consult the reachability flag that #270's stopgap resets.

An unsigned write may now insert an unseen DID but may never change an
existing row's http_url; changing one requires a verified signature from
that row's own DID. The gate is at the db boundary rather than in the
announce handler because there is a second writer: the bootstrap
announce-back in main.rs reads a peer's own JSON response body with no
proof of anything, and now declares itself unproven, which makes it
insert-only by construction.

The proven variant carries the DID it proves and upsert_peer compares it
against the row being written. A bare proven/unproven flag would say that
something was proven without saying which DID, leaving the real check in
the handler and reproducing RUSTSEC-2022-0009, where a valid signature was
accepted without checking it derived the claimed identity.

An unproven insert is restricted to did:key. Only did:key can ever
authenticate here, so a did:web or did:gitlawb row created unsigned would
become unwritable by anyone once this rule lands.

The unproven path derives its refusal from rows_affected, because an
ON CONFLICT DO UPDATE with a guarding WHERE reports zero rows rather than
raising, and a silent no-op is the fail-open this gate exists to prevent.
The refusal is a typed error the handler maps to 403, downcast out of the
anyhow chain the way sqlx errors already are, so it does not render as a
500.

Observed at c83cbc5 before the fix: an unsigned announce naming an
existing peer returned 200 OK and left the attacker's URL stored. That
test is the RED this change turns green.

Two of #270's reachability tests now drive the proven path. Under this
rule an unproven repoint is refused, so they can no longer express a URL
change through it; they assert the same behavior as before.
2026-08-03 20:38:53 -05:00
beardthelion 6a405f65f1 fix(release): scope the annotation check to the [package] table
The coverage gate matched `# x-release-please-version` anywhere in a manifest,
so an annotated `version` under any other table satisfied it. A crate could
leave the version cargo actually reads unannotated, pass the gate, and still
freeze at release time, with release-please rewriting the decoy line instead:
the exact failure this gate exists to catch.

Scope the match to the [package] table.
2026-08-03 09:45:17 -05:00
beardthelion 730432d0cd test(icaptcha-client): bind new()'s empty-API-key filter (#216)
IcaptchaCfg::new normalizes an empty GITLAWB_ICAPTCHA_API_KEY to None
via .filter(|s| !s.is_empty()) before storing api_key. No test drove
the empty-string value through new(), so a regression would attach an
empty bearer token to the challenge and answer requests and ship green.

The new test drives an empty key through new() on a trusted operator
origin and asserts it normalizes to None, with a contrast arm showing
a real key still attaches. Mutation check: removing the filter fails
this test and no other.
2026-08-03 09:05:14 -05:00
beardthelion 9bba8f9934 test(node): restore the ipfs-max-repos-walked range test the #174 merge dropped
The merge kept the knob and its clap range but carried over only the
default-value assertion, so nothing asserted that 0 and 1048577 are
rejected. Restored verbatim from the #174 side, beside its
ipfs_max_repo_visits sibling.

Load-bearing: widening the range to 0..=2_097_152 turns it red.
2026-08-03 01:16:52 -05:00
beardthelion a0e0e0bf1b Merge remote-tracking branch 'origin/fix/served-git-concurrency-cap' into fix/issue-135-ipfs-cid-tree-gate 2026-08-03 00:52:56 -05:00
beardthelion 17f9edad47 Merge remote-tracking branch 'origin/main' into fix/served-git-concurrency-cap 2026-08-03 00:20:03 -05:00
beardthelion aeb6895141 test(node): stop the repos.rs structural gates splitting at the wrong marker
Three gates scanned the "production half" of api/repos.rs by splitting at the
first cfg(test) attribute. Adding test-only items above the code they check
moved that split point above every line being scanned, so all three failed
looking for production code that was still right there.

They now split at the test module, which is what they meant. This is the same
defect already fixed in the ipfs gate during the merge, and no split-on-the-
attribute site remains.

Both behavior-bearing gates were re-proven load-bearing: F2 goes red when the
Pinata object set is re-derived before the pin permit is taken, and the tail
gate goes red when release stops consuming the same success flag. F3 is a
presence check whose marker cannot be removed by a single compiling edit, so it
is not mutation-proven here.
2026-08-02 20:16:58 -05:00
beardthelion d1fc9bc835 fix(node): retry the coalesced drain's re-read instead of losing the push
The drain read the repo row and its visibility rules once each. Either read
failing dropped that coalesced push's pins and recovery copy for good: the repo
read returned None on error, and the rules read collapsed through ok(), which
makes the repo read as not announceable. Both fail closed, so nothing leaked,
but nothing sweeps the lost work up either. A single database blip was enough.

resolve_drain_object_list now refreshes through a bounded retry: three
attempts, 50ms doubling, and either read failing is transient. A missing repo
row is the one terminal answer, so it releases at once and spends no retry
budget. Exhaustion logs at ERROR with the repo id and attempt count, so giving
up is observable rather than silent.

The coalescing loop is untouched. It already has the shape the retry needs: a
refresh that gives up skips the pin call, and the next finish_or_take_pending
picks up anything that coalesced while the retry was running.

Eight restored tests cover it, written and observed red against the seam-only
tree first. The five the plan predicted red came up red for the reasons it
predicted, and the three controls stayed green. All eight are mutation-proven:
no-retry, an off-by-one bound, gone burning budget, a rules error read as
empty, an unfiltered candidate set, a duplicated refresh, break-on-giveup, and
a pending slot that never drains each turn their own test red.
2026-08-02 20:16:58 -05:00
beardthelion 8557a12e59 test(node): add a test-only fault seam for the drain's two re-reads
A real Postgres pool will not fail on demand, so the drain's error arms have no
way to be driven from a test. This ports the injection table the removed
requeue loop used, keyed on the task's repo id, which is a fresh uuid per test
so parallel cases cannot cross-inject.

The two reads now go through drain_get_repo and drain_list_rules, whose bodies
consult the seam under cfg(test) and then make the same call as before. Error
handling is untouched: the repo read keeps its arms, the rules read keeps its
ok() collapse, and no retry exists yet. The suite is unchanged at 856, which is
the point.

inject and counters carry a dead_code allow until the tests that call them
land.
2026-08-02 18:50:03 -05:00
beardthelion 979d3c45bb test(node): restore the four u3 drain tests onto the coalescer
These covered post-push work being requeued rather than dropped, and were
removed in the merge because they drove the replaced requeue loop. The
behaviour they check is still there, so they re-seam onto the coalescer without
any production change.

The seam moved in three ways. The inflight key is the repo identity key, not
the row id. A coalescing push now merges its (old, new) tip pairs into the
pending slot, so a test that passes an empty vec merges nothing and the drain
never laps: every coalesce carries real commit oids. And the full-scan arm is
reached through the pending-slot overflow, so the leak test coalesces 1025
pairs and asserts the slot degraded to FullScan before the run.

run_encrypt_pin_task_for_test builds the task context the way the production
spawn site does and takes a real owner and name, because the drain re-fetches
by owner and name; a blank name would resolve Gone on every lap.

All four are mutation-proven: drain-does-nothing, stale-empty-rules,
full-scan-skips-the-fail-closed-filter, and a leaked inflight key each turn
their own test red.
2026-08-02 18:39:33 -05:00
beardthelion 3ca35d216d test(node): restore three guard tests the #174 merge dropped
The merge deleted coverage that no longer compiled against the reconciled
designs, rather than re-seaming it. Three of those tests have counterparts in
the merged tree after all:

unlock_error_connection_close_is_bounded and
write_guard_dropped_off_runtime_disposes_the_connection came back with
close_conn_bounded and the Drop backstop, which the lock-pool design needed
after all (the after_release hook cannot free a lock when the unlock errors on
a live session). Both now observe the store's derived lock pool rather than the
pool handed to for_testing, which is where the guard's connection actually
lives.

get_by_cid_caps_repos_walked_knob_bounds_the_walks was orphaned when its body
was spliced into a sibling test. It drives GITLAWB_IPFS_MAX_REPOS_WALKED, which
is the binding cap only now that the gate takes the tighter of the two walk
ceilings.

The one test not restored is write_guard_release_when_not_locked: the guard has
a single construction site and it sits below the lock-or-bail, so a guard that
never took the lock cannot exist.
2026-08-02 18:39:33 -05:00
beardthelion e5c1da2bc6 Merge #174 into the CID tree-gate branch
Reconciles the served-git concurrency cap with the /ipfs CID tree gate. The two
branches independently reworked the same three seams, so most of this is picking
one mechanism per seam rather than taking a side per file.

/ipfs walk admission: keeps this branch's provenance-first resolver and its lazy
legacy scan, but drops the detached serve task in favour of #174's shared
Arc<WalkAdmission>, cloned into each spawn_blocking. Both close the permit-release
bypass; the Arc does it without leaving an abandoned request's full legacy scan
running against a held slot. Ported in with it: the request-budget gate at all four
stages with each child deadline clamped to the remainder, the walk timeout computed
inside the closure so queue delay is charged, taint sources in place of a single
truncated flag, and the deterministic-fault 500 gated on nothing else having tainted.

The two per-request ceilings are now composed rather than one shadowing the other.
The visit ceiling was unwired by the merge and the walk cap answered to only one of
the two knobs; both are honoured, the tighter one winning.

Object probe: object_type_bounded takes a Duration and returns ProbeError, so this
branch's call shape keeps #174's absence-vs-unreadable-store discriminator. A clean
`missing` from cat-file is byte-identical whether the object is absent or the pack
is unreadable, and the discriminator is store readability, not git's wording.

Write lock: the advisory-lock pool's after_release hook does not cover an unlock that
errors on a live session. Measured: a poisoned connection returned to the pool still
held the lock 15s later, because the hook's own unlock_all fails the same way. So the
guard now disposes of that connection, in release and in the Drop backstop.

Post-push replication goes to #174's coalescer. This branch's requeue_faults
integration suite has no equivalent seam on that design and is not carried over; it
is recoverable from the pre-merge head.

Also clamps the provenance path's three per-source lookups to the request budget.
They run while the walk permits are held, which is the stall hazard the preload
clamp already covered, and the structural guard found them because it now requires
every occurrence to be wrapped instead of assuming exactly one.

1346 tests pass across the workspace; fmt and clippy are clean under --locked. The
four re-anchored structural guards were mutation-tested and all four go red on the
defect they name.
2026-08-02 16:09:57 -05:00
beardthelion eb697364bf docs(node): correct the ceiling margin, it is 5.85x not an order of magnitude
Both doc comments describing GIT_SERVICE_TIMEOUT_SECS_MAX said the 100-year
ceiling sits "an order of magnitude" under the ~584-year limit of a
u64-nanosecond Instant. The real margin is 3,153,600,000 seconds against
18,446,744,074, a factor of about 5.85. An order of magnitude means ten.

Worth noting how it spread, since the fix is trivial and the pattern is not.
The phrase was written once on the constant, then copied into the
ipfs_request_budget_secs doc by the commit that was at that moment correcting
a different false claim in the same file. Nobody re-ran the division at either
site, because quoting an existing sentence feels like citation rather than
assertion.

The direction of the error is the part that matters: a vague "well under"
would have been true and would have invited a reader to check. The
precise-sounding "order of magnitude" was false and discouraged checking.

Comments only; the diff carries no executable line, checked by filtering it
for non-comment changes. fmt and clippy clean under the CI form.
2026-07-31 02:09:32 -05:00
beardthelion bc826df2ec fix(review): correct the budget-knob claims and mirror the range in operator docs
Review round on the three preceding commits.

The knob's doc said a value past the ceiling "aborts every /ipfs/{cid}
request in a release build". That is wrong, and three reviewers plus an
outside model landed on it independently: the ceiling is 100 years, the
Instant addition does not actually overflow until roughly 584 years on a
u64-nanosecond Instant, so values in between are representable and are
rejected as a deliberate safety margin, not because they panic. Reworded
to say that, since a confidently wrong comment is worse than the vague
one it replaced.

Also on the knob:
- GIT_SERVICE_TIMEOUT_SECS_MAX has two consumers now, so its own doc says
  so, and notes this knob derives only the Instant addition and not the
  lease-steal multiply.
- The "clears every disable setting" phrasing overclaimed: u32::MAX sits
  above the ceiling. Now names the documented sentinels instead.

Operator-facing docs: the sibling knob's ceiling commit updated both
README.md and .env.example in the same change, and this one had not. Both
now carry the accepted range, matching that precedent.

House style: five comment lines added by the preceding commits used em
dashes, which this project does not allow. Four were carried over from the
text being edited and one was copied from the sibling test. All replaced.
Checked mechanically over the added lines rather than by eye.

Two comments restated an argument in full immediately after pointing at
the canonical site for it. Trimmed to the pointer plus the local delta.

Scoping correction worth recording: the two preceding commits claimed "no
behavior change". That holds for repos.rs, state.rs, main.rs and
test_support.rs, but a config.rs field doc IS clap --help output, so the
pin-knob doc edit changes operator-visible help text. No runtime behavior
changes either way.

Verification: full suite 733 passed, fmt clean, clippy clean under the CI
form. The range guard was separately proven load-bearing in both
directions by mutation: reverting to range(1..) goes RED on the rejection
assertions, and tightening the ceiling to 500_000_000 goes RED on the
disable-value assertions.

Known residual, not fixed here: the bound is parse-side only, so a Config
built by direct construction rather than through clap can still hold an
unrepresentable value.
2026-07-31 01:24:46 -05:00
beardthelion 050b6d2832 docs(node): state what the permit-less load-shed peek actually delivers
The three handler peeks said they shed "before any DB/disk work", which
reads as an admission bound on the DB window. It is not one. The peek
holds no permit, and permits are only held from the authoritative acquire
below, after the visibility and rate gates, so a burst arriving while
permits are free all proceeds into the DB and none of it sheds. What the
peek actually gives is a cheap 503 for one request once the pool has
ALREADY filled, sparing that request's DB work.

The gate ordering that makes it a peek is deliberate and stays: a denied
or rate-limited request must consume no slot, and one source must not
hold global slots through the DB/visibility window. Reordering to bound
the window would undo that. So this corrects the claims and changes no
behavior; bounding the DB window would need an admission mechanism the
peek is not.

test_support.rs's receive-pack docstring additionally called the peek
"the permit" and "the first statement". Both were wrong at this head: it
is a non-holding peek, and the authoritative write-pool acquire sits
after the per-repo lease. Corrected along with the shed claim. The
"remove the block and this goes red" sentences stay, since those tests
drive an already-saturated pool, which is the case the peek does handle.

Sweep over "before any DB", "before DB", "pre-DB". Left alone with
reason: api/ipfs.rs:142 describes a HELD walk-admission acquire, so its
claim is true; graphql/mutation.rs:201 and test_support.rs:528 are about
auth ordering, unrelated; the repos.rs test docstrings at 4194/4213/4556/
4618/4682 describe test mechanics against a saturated pool, which is the
case the peek delivers.

No behavior change: the diff is comments and docstrings only, checked by
filtering the diff for non-comment lines.
2026-07-31 00:59:51 -05:00
beardthelion f91b224bab docs(node): state what the pin-admission pool actually bounds
The pin pool's docs claimed it bounded "how many MB-scale lists are held
at once across all repos". It does not. On the local IPFS path the caller
materializes the object list before pin_new_objects_gated acquires, so a
task parked waiting for a permit is still holding its full list, and how
many tasks park is capped only per repo by EncryptInflight. Across
distinct repos that retained memory is unbounded, which is the gap a
reviewer raised against this claim.

Correct the claim at each site rather than implying the pool closes an
exposure it cannot see. The pool does bound concurrent pin loops, and
therefore lists held while being pinned; that part was always true and is
kept. The residual is now named where each claim lives, including on the
config knob, so an operator does not read the knob as a memory cap it
cannot deliver.

Bounding the retention is a real change to the capture shape (the Pinata
twin already avoids it by acquiring before it derives) and is not
attempted here.

Sites corrected: pin_new_objects_gated's docstring and its F2b empty-list
comment, the empty-list test's docstring, AppState::pin_semaphore and
AppState::encrypt_inflight field docs, the max_concurrent_pin_tasks knob
doc, and the construction comment in main.

Sweep over pin_semaphore, F6, and "held at once". Left alone with reason:
ipfs_pin.rs 16/31/174-200/518 describe the loop that genuinely holds the
permit, so their claims are about hold duration and are true; repos.rs
1029-1036 and 2276-2294 describe the Pinata path, which acquires before
materializing, so its "at most pin_semaphore permits' worth exist at once"
holds; repos.rs 5898 claims concurrent loops are bounded, which is the
true concurrency claim; the F6 hits in api/ipfs.rs are that file's own
review numbering for walk-permit ordering, unrelated to this pool;
constructions and test fixtures carry no claim.

No behavior change: the diff is comments and docstrings only, checked by
filtering the diff for non-comment lines.
2026-07-31 00:57:53 -05:00
beardthelion cf7fe46df2 fix(node): bound GITLAWB_IPFS_REQUEST_BUDGET_SECS by representability
ipfs_request_budget_secs parsed range(1..), so every positive u64 was
accepted, but get_by_cid derives the request deadline as
Instant::now() + Duration::from_secs(that). That addition is an explicit
overflow check rather than a debug-only one, so an oversized value aborts
every /ipfs/{cid} request in a release build instead of setting a very
long budget. The route is anon-reachable, so an operator typo is felt
publicly.

Reuse GIT_SERVICE_TIMEOUT_SECS_MAX, the ceiling the sibling knob already
carries: the bound is representability, not a policy view of a sane
budget, so the large "set it very large to disable" values operators were
told to use still parse. Reject rather than clamp, matching the other
bounds in this config.

Execution evidence:
- RED before the change: the new test failed with "3153600001 is past the
  representable ceiling and must be rejected at parse time" while its
  disable-value assertions passed, so the test is not vacuous in either
  direction.
- GREEN after: config::tests 13 passed, including the sibling
  git_service_timeout regression test.

Sweep for the same defect class, re-run with a widened shape because the
original grep missed a site. Passes: Instant::now() sites; from_secs()
fed by config fields; config fields reaching a Duration through an
arithmetic expression; config fields reaching a tokio timer constructor.
The arithmetic and timer passes each return exactly one hit,
heartbeat_interval_hours (config.rs 150, main.rs 916, operator.rs 161),
which has no range at all and multiplies by 3600 before the Duration.
That knob is a separate defect on a path this branch does not touch, so
it is recorded for a follow-up rather than widened into here. Every other
Instant addition derives from the already-bounded git_service_timeout_secs
or from a constant.
2026-07-31 00:56:04 -05:00
beardthelion 33063baf5a fix(node): set the git_service_timeout ceiling by representability, not tidiness (#174)
A second-model pass on the previous two commits caught an upgrade footgun in the
first and stale prose in the second.

The 365-day ceiling was tighter than the defect required. The help text has
always told operators to set this very large to disable the bound, and clap
accepted every positive u64, so values like 999999999 (~31 years) are working
production "off" settings that neither overflow `* 2 + 60` nor exceed an Instant
deadline. A node running one of those would have failed to start after this
upgrade, over a value that was never the defect. The ceiling moves to 100 years,
which clears every such setting and stays an order of magnitude under the
~584-year limit of a u64-nanosecond Instant, the tightest representation on any
platform we build for. Every value that worked before still parses; only the
ones that would have panicked are rejected. The test now pins the pre-existing
disable values alongside the rejections so the distinction cannot quietly
regress.

Clamp-and-warn was the suggested remedy and is not what landed: it would keep
accepting a value the node cannot honor and silently substitute a different one,
where the range says what is representable and fails fast, matching how every
other bound in this file behaves.

The lease-bound comment and its test still described the pre-ceiling world. Both
now say what is true, and the test deliberately keeps u64::MAX rather than moving
to the ceiling as suggested: Config is reachable by direct construction, and at
the ceiling the arithmetic does not overflow, so a test pinned there would pass
with the saturation removed and prove nothing. Re-confirmed load-bearing after
the rewording.
2026-07-30 18:07:16 -05:00
beardthelion b231d32703 fix(node): bound git_service_timeout_secs to a range its deadlines can represent (#174)
Tracing the lease-bound overflow to its other consumers turned up a worse
instance of the same defect. This PR routed git_service_timeout_secs into
build_filtered_pack and blob_paths, and both build a deadline as
`Instant::now() + Duration::from_secs(..)`. That addition panics on overflow in
RELEASE as well as debug, so a large-but-accepted value crashes the serve path
rather than merely disabling a timeout.

Hardening each arithmetic site as it is found would leave the next one open, so
the bound goes where the value enters: clap now rejects anything above 365 days
at parse time, which keeps every derived duration in range at once and fails
fast at boot instead of at the first path-scoped clone. A year is many orders of
magnitude above any real clone or push, so it remains the practical way to
disable the bound.

The test pins both derivations at the maximum and rejects the value one past it
and u64::MAX. RED when the upper bound is dropped from the value_parser.
2026-07-30 17:28:36 -05:00
beardthelion 6032fd2d95 fix(node): saturate the derived lease steal bound instead of overflowing it (#174)
GITLAWB_GIT_SERVICE_TIMEOUT_SECS accepts every positive u64, so deriving the
per-repo write-lease steal bound as `* 2 + 60` panics every push in a debug
build across the top of the accepted range, and in release wraps to a short
Duration that would let a waiter steal a live push's lease.

Saturating keeps the knob's documented meaning: a timeout that large means the
service bound is effectively off, so the steal bound should be effectively
infinite rather than tiny.

The regression drives git_receive_pack rather than the arithmetic, so the
call-site wiring is what is under test. RED before at repos.rs:1733 with
"attempt to multiply with overflow", GREEN after; re-confirmed load-bearing by
reinstating the unchecked expression.
2026-07-30 17:28:25 -05:00
beardthelion feefaf4f9e fix(node): dispose the write-guard connection when the Drop unlock errors (#174)
Review finding from jatmn, round nine. RepoWriteGuard::release closes the
connection when pg_advisory_unlock returns Err (F3b): the await resolved, so
the session is alive and still holds the lock, and returning that connection
to the pool hands the next caller a lock nobody tracks. The Drop backstop
carried the same hazard and only logged.

Both arms are fixed.

The spawned detached unlock now closes the connection through
close_conn_bounded on the error path, instead of letting the async block end
and return it to the pool.

The off-runtime arm was worse than the finding described. It dropped an
already-taken connection with no unlock attempted at all, and this turned out
to panic rather than leak: PoolConnection::drop calls rt::spawn for its
return-to-pool task, whose no-runtime fallback panics with "this
functionality requires a Tokio context". So any off-runtime guard drop
panicked in a destructor today. It now detaches, which gives up the pool slot
and yields a plain PgConnection whose drop closes the socket, ending the
session that holds the lock. Drop cannot await, so detach is the whole
disposal available here. Confirmed against the vendored sqlx 0.8.6 source
rather than assumed, and min_connections is 0 everywhere in this repo.

Three tests, modeled on the release F3b cases, on a pool with the idle reaper
disabled so a reaped session cannot pass the assertion for us. Observed RED
first:

  write_guard_dropped_off_runtime_disposes_the_connection ... FAILED
    panicked at sqlx-core-0.8.6/src/pool/connection.rs:208:
    this functionality requires a Tokio context
    then: dropping a write guard off a Tokio runtime must not panic
  write_guard_drop_with_failing_unlock_does_not_return_the_connection ... FAILED
    timed out waiting for the connection whose detached unlock errored to be
    closed rather than returned to the pool still holding the session lock
  write_guard_drop_with_successful_unlock_keeps_the_connection ... ok

The success case is green on both sides on purpose: it is the must-not case,
so it only ever catches a widening of disposal onto the healthy path.

GREEN after, and all 12 guard tests pass, so the release coverage did not
regress. Both new guards mutation-proved load-bearing, each red on its own
message, 2/2.

Full suite 729 passed, 0 failed. fmt and clippy clean.
2026-07-30 13:41:11 -05:00
beardthelion fad18bc923 docs(node): stop the per-source cap comments claiming a bound the keying lacks (#174)
Both new comments said saturating the pool takes ~8 distinct source IPs,
each also rate-limited. Neither half holds for an IPv6 caller with a routed
/64: the sub-cap and the rate limiter both key on rate_limit::client_key,
which with TrustedProxy::None falls through to the full peer address with no
prefix folding, so one caller has 2^64 keys. max_keys does not compensate,
since keys are freed on permit drop and live keys never approach the 100,000
default.

The comments now say what the cap actually bounds, concurrent slots per
resolved client key, and name the residual.

The keying fix stays deferred as a design call, and that deferral is
defensible: full-IP keying predates this PR, so these caps help IPv4 and
single-address callers without regressing anything. Landing a comment that
overstates the guarantee is not defensible, in a round that corrects a
weaker overclaim one screen away.

The per_source_push_cap docstring added earlier in this round repeated the
same claim, so it is corrected here rather than shipping fresh.

Comments only, no behaviour change. check, fmt clean.
2026-07-30 13:41:11 -05:00
beardthelion 1501bd1bfd docs(node): drop a backstop claim with no backstop behind it (#174)
The pinata_object_list_for_refs docstring said a visibility tightening that
lands after tail-start is covered by the reconciliation sweep. No such sweep
exists; state.rs says so directly, that a dropped job would be lost forever
precisely because there is none.

The claim is dropped and nothing replaces it. An earlier draft wanted to
substitute a next-push claim, which no site establishes either, and swapping
one unbacked claim for another is the same defect.

Four pre-existing sites make the same claim (repos.rs:30, push_delta.rs at 8,
189 and 281 with warns at 349 and 358). They predate this PR and are left for
a follow-up rather than widening a nine-round diff.
2026-07-30 13:41:11 -05:00
beardthelion 6ef2effe77 test(node): bind the per-source cap test to the production derivation (#174)
advert_per_caller_cap_sizing_is_never_zero opened with a local closure
re-implementing (pushes / 8).max(1), so it bound nothing. Production derives
that expression twice, and the second site was added after the test was
written. per_source_push_cap now lives in rate_limit.rs next to
PerCallerConcurrency, the type the value feeds, and both main.rs sites and
the test call it.

The finding was confirmed by execution before the fix. Deleting .max(1) from
both production sites with the closure still in place:

  injected (2 sites), re-running ... still green
    VACUOUS  the guard's own defect was injected and nothing failed
    0/1 load-bearing

And after, deleting .max(1) from the helper:

  injected (1 site), re-running ... red, on the expected message
    LOAD-BEARING  matched 'advert cap must be >= 1 for pushes=1'
    1/1 load-bearing

The first attempt at that second proof came back RED-WRONG-REASON against
'minimum write pool must derive cap 1, not 0': the loop's assertion for
pushes=1 fires first and aborts before that line is reached. Same property,
earlier assertion.

No mutation residue. Test green, fmt and clippy clean.
2026-07-30 13:41:11 -05:00
beardthelion d9f8f414eb test(node): bind the repo-key boundary test to the separator it names (#174)
separator_prevents_the_owner_name_boundary_collision asserted that
("a","b_c") and ("a_b","c") produce different keys. Those differ under any
join at all, bare included (ab_c against a_bc), so deleting the separator
from repo_identity_key left the test green. It could not fail for the reason
it named.

The inputs are now ("a","bc") against ("ab","c"), both abc under a bare
join. The test docstring said the separator was \0; the code joins with /.
Corrected. The docstring on repo_identity_key itself carried the same wrong
collision example, so it is corrected too.

Proven by mutation rather than argued, joining with no separator:

  old inputs, injected, re-running ... still green
    VACUOUS  the guard's own defect was injected and nothing failed
    0/1 load-bearing

  new inputs, injected, re-running ... red, on the expected message
    LOAD-BEARING  matched 'owner/name boundary must not be ambiguous'
    1/1 load-bearing

Separator restored, no mutation residue. All 6 repo_identity_key_tests pass.
2026-07-30 13:41:11 -05:00
beardthelion 35f71e515d fix(node): charge the /ipfs walk budget for blocking-queue delay (#174)
The walk's deadline clamp was computed on the async side, before
spawn_blocking was called, then used inside the closure. Time spent queued
for a blocking thread went uncharged, so the walk ran with a budget larger
than the request's true remainder. The clamp is now derived inside the
closure at task start.

This is the sibling of the upload-pack site fixed in 28a6ca4, and unlike
that one it sits on the anonymously reachable route.

A queue delay that eats the whole remainder saturates the clamp to zero and
the bounded walk fails closed through the existing taint arm, no verdict,
which is the safe direction.

Testing gap, recorded at the site rather than papered over: the queue-delay
path is reasoned, not executed. Observing it needs a runtime with the
blocking pool pinned and parked, and the sqlx::test harness gives no seam
for that. 28a6ca4 shipped with the same gap for the same reason, verified
against its diff and message rather than assumed. The tests in this file
saturate the admission semaphores, never the blocking pool, so there was
nothing to reuse. An assertion on the arithmetic would only restate the
implementation.

cargo test ipfs::: 24 passed, 0 failed.
2026-07-30 13:41:11 -05:00
beardthelion 03dde22e98 fix(node): bound the IPFS pin batch and let a slow healthy upload finish (#174)
pin_new_objects runs under a pin_semaphore permit and that pool defers rather
than sheds. Both sinks in ipfs_pin.rs built a bare reqwest::Client::new(),
which carries no timeout, so a silent endpoint parked the pool indefinitely.
Both now use the shared no-redirect client from build_http_client() through a
module-local OnceLock, rather than the hand-rolled twin its docstring forbids.

The client alone does not bound the hold: without a ceiling the loop is O(N)
with N chosen by the pusher. A 120s batch budget is taken once at loop start
and gated at the top of every iteration, so no object's work begins with zero
budget left. The remainder is also passed to each add as a per-request
timeout, which is what lets one large healthy upload run past the client's 10s
default without letting the batch run forever. RequestBuilder::timeout
replaces the client value for that request only and leaks nothing to other
callers, verified by execution against the pinned reqwest.

An earlier version of this commit classified a dead endpoint by
reqwest::Error::is_timeout() || is_connect() and broke the batch. That was
wrong in both directions and is gone. It missed the ordinary dead-peer shapes,
since a peer closing mid-request yields IncompleteMessage and a peer RST
mid-upload yields ConnectionReset, both observed with is_timeout and is_connect
false, so a load-balancer idle timeout evaded it entirely. And it fired on a
slow but alive endpoint, because .timeout() is a total deadline covering body
upload, so a healthy multi-megabyte add was read as a dead endpoint and
abandoned the rest of the push. A wall-clock bound needs neither judgement.

The docstring now states what is bounded and what is not, rather than implying
the hold is closed. Not bounded: the git read, since store::read_object shells
two bare Command::output() calls with no timeout or process-group reaping and
this loop does not run it under spawn_blocking; the DB round-trips; and the
pool itself, since the Pinata replication task holds the same semaphore across
a full git re-derivation with no deadline. Truncation is stated plainly: a
batch stopped at the deadline leaves the rest unpinned and nothing sweeps them
up, so recovery is opportunistic through a later full-scan push.

Tests written first, both RED by assertion rather than by a harness timeout:

  pin_new_objects_stops_the_batch_at_its_deadline ... FAILED
    the batch must stop partway, not pin all five and not stall on the
    first: pinned 5
  pin_new_objects_does_not_abandon_the_batch_on_a_slow_but_alive_endpoint ... FAILED
    a slow but progressing endpoint must pin both objects: an upload past
    the client's 10s default is not a dead endpoint
    left: 1, right: 2

Both guards mutation-proved load-bearing against present-but-wrong mutants,
each red on its own message: the loop-top break widened to continue, and the
per-request timeout replaced with None.

Also bounds encrypted_pin.rs's pin_git_object call, where a hang wedged a
repo's EncryptInflight key. Its Drop backstop covers unwind, not hang.

Full suite 730 passed, 0 failed. fmt and clippy clean.
2026-07-30 13:41:11 -05:00
beardthelion a261f7d289 Merge branch 'main' into fix/served-git-concurrency-cap
Brings the branch onto post-#243 main so cargo metadata --locked passes
against the committed tree.
2026-07-30 10:25:07 -05:00
beardthelion 4f424693b8 fix(release): bump icaptcha-client with the rest of the workspace
icaptcha-client became a workspace member in #138, after #130 last wrote
release-please-config.json, and nothing updated extra-files. release-please
rewrites a version line only when the line carries the annotation AND the
manifest is listed, so the crate was skipped every release and sat at 0.4.0
while the other five walked to 0.7.1.

Nothing broke: every intra-workspace path dep is bare, so no version
requirement could go unsatisfied; no crate is published to a registry; and
the crate reads no CARGO_PKG_VERSION of its own. The cost was the workspace
Cargo.toml comment asserting a lockstep rule the config did not keep, which
is what the next person adding a crate would read and copy.

Add the missing entry, and gate the rule so it cannot silently lapse again.
The gate derives its required set from `cargo metadata`, so a new crate reds
CI whether or not its author knows the gate exists, and it covers all three
release-silent shapes: a member absent from extra-files, a member whose
version line lost the annotation, and an extra-files path that is no longer
a member.

Next release moves icaptcha-client 0.4.0 to the current workspace version in
one step. Nothing depends on that version, so the jump is cosmetic.
2026-07-30 08:54:33 -05:00
beardthelion afa65e0a59 fix(node): skip an /ipfs absence re-probe the budget cannot pay for (#62)
The absent-CID path disambiguates a clean `missing` by re-running the probe,
but the re-probe took the same deadline with no check that any budget was left.
A first probe that nearly exhausted the budget still spawned a second child
that could only be reaped, and the watchdog's SIGTERM grace plus SIGKILL settle
carried the whole call to roughly twice the budget. Measured on
object_type_bounded with a 1s budget and a probe that burns 0.9s of it: two
spawns and 2021ms elapsed.

That is the same ~2x occupancy shape this PR exists to remove, on a different
handler, and it is worse here than on the served-git path: /ipfs/{cid} carries
only optional_signature, and an absent CID drives this branch once per repo, so
an unauthenticated caller spraying well-formed absent CIDs pays the overshoot
per repo. It also degraded an honest absence into a retryable 503.

Re-probe only when the remaining budget covers what the first probe actually
took, since the re-probe runs the same command. Below that, taint to Transient
rather than spawn: an inconclusive disambiguation is not an absence verdict, so
it must not become a false 404 either. The same 1s case is now one spawn and
finishes inside the budget.

Both introduced by this PR (d39fc8d, the F5 disambiguation); neither
object_type_bounded nor the re-probe exists at the merge base, so this is not a
pre-existing condition. The companion test pins that an ample budget still
re-probes, so the check cannot silently disable the disambiguation it guards.
2026-07-29 22:56:47 -05:00
beardthelion b89dec4ce6 fix(node): charge blocking-queue delay against the shared upload-pack deadline (#62)
The shared deadline computed the walk's budget on the async side, before the
task was queued on the blocking pool. withheld_blob_oids_bounded starts its own
clock when it actually runs, so a budget measured at queue time handed the walk
a full budget from whenever the pool got to it: the read permit was held for
queue delay PLUS the budget, and only the serve phase saw the outer deadline
already spent.

Computing the remainder inside the closure charges that delay against the shared
deadline, which is what makes the ~1x bound in the docs true rather than
approximate. Not a regression from the previous commit (the old code passed a
constant git_timeout, so the delay went uncharged there too, and the combined
hold was queue delay plus two full budgets), but the claim this PR now prints in
README and .env.example should be exact.

The two shared-deadline tests still pass and still fail on revert; they cover
the budget arithmetic, not the queue-saturation path. Reproducing that needs a
runtime with the blocking pool pinned and parked, which the sqlx::test harness
does not give a seam for, so the queue-delay path is reasoned, not executed.
2026-07-29 22:11:43 -05:00
beardthelion 16f4d31900 docs: normalize README punctuation and record the two admission doc changes (#62)
Two table rows needed correcting for the changes in this PR: the
GITLAWB_GIT_SERVICE_TIMEOUT_SECS row now says the classification walk and the
pack serve share one deadline on the path-scoped upload-pack path, so the value
bounds them combined; and the GITLAWB_IPFS_REQUEST_BUDGET_SECS row no longer
claims the probe and content-read subprocesses are unbounded, while keeping the
object-store-readability sweep named as a live residual.

The other thirteen changed lines are punctuation only, no wording changes: em
dashes replaced with colons, commas, or sentence breaks, and one pair of curly
quotes straightened. The repo's prose-style hook validates a whole file rather
than a diff, so these pre-existing characters were rejecting every edit to
README.md, including the two rows above.
2026-07-29 21:48:49 -05:00
beardthelion 9bfe7d5d30 docs(node): correct stale admission-bound comments and document derived caps (#62)
Four prose corrections on this PR's surfaces, no behavior change.

The receive-pack advertisement no longer draws from the write pool, it draws
from a dedicated advert pool, but five comments still described the old
shape: the info/refs per-source block, the cap construction in main.rs, the
State field doc, and a test doc plus its assertion message. The state.rs and
main.rs ones mattered most because they contradicted the operator text being
written in this same commit.

The /ipfs probe and content-read subprocesses each run under their own
deadline now (the lesser of git_service_timeout_secs and the remaining
budget, reaped by process-group teardown), but four surfaces still claimed
they had no duration bound past the pre-start budget check. The correction
stays scoped rather than declaring the route safe: the probe's
object_store_readable check is a synchronous filesystem sweep with nothing to
reap, so a wedged filesystem can still hold the walk slot past the deadline,
and that residual is now stated where the old warning used to be. A grep for
the full phrase misses two of these surfaces, since one wraps across doc-comment
lines and one says "duration clamp" instead, so the sweep uses the fragment.

Both push-side per-source concurrency caps are derived as
max_concurrent_git_pushes / 8 with a floor of 1 and have no environment
variable, so neither was documented anywhere an operator reads. The write-side
one is load-bearing for push availability: it is acquired before the global
write permit, and owner enforcement defaults off, so without it one host
minting disposable did:key identities could monopolize the write pool. They
are described in .env.example beside the read cap rather than in README's
table, which is keyed by variable name.

The hung-walk assertion in visibility_pack printed {err}, showing only the top
anyhow context and dropping the underlying io error. A beta-lane failure
surfaced as "failed to spawn git for-each-ref" with no errno, which made it
undiagnosable. {err:#} prints the chain. The underlying flake is still
undiagnosed: it did not reproduce in 123 local runs and the stable lane passed
on the same commit, so this only improves the next occurrence.
2026-07-29 21:44:53 -05:00
beardthelion 1219c21469 fix(node): share one deadline across the withheld walk and the pack serve (#62)
The path-scoped git-upload-pack branch spent git_service_timeout_secs twice:
once on the withheld-blob classification walk, then a second full budget on
the serve. One clone could hold a read-pool permit for ~2x the configured
budget, which is the occupancy hole this PR otherwise exists to close.

Thread one Instant deadline through both phases, mirroring
fail_closed_full_scan_objects and build_filtered_pack: the walk runs against
the remainder and the serve gets what the walk left, computed after the
blocking task's await so it cannot pick up a fresh budget. A walk that
consumes the whole budget saturates the serve to zero, which surfaces as
GitServiceTimeout -> 504 rather than running unbounded. Under-serving one
slow clone is the safe direction versus over-holding the pool.

Both serve arms are covered because a fix that reached only the plain arm
would leave the ~2x hold reachable by any clone of a repo that actually
withholds a blob. Each test was observed RED before the change (status 200,
the serve completing on a fresh budget) and GREEN after, and reverting the
filtered arm alone flips only its own test, so neither guard is a proxy for
the other.

The knob's meaning changed for this path, so .env.example now says the value
bounds the two phases combined rather than each separately.
2026-07-29 21:44:28 -05:00
beardthelion fed460cf4b docs(node): record the Tigris ordering residual the tail reorder introduces
Spawning the replication tail above guard.release() also moves it above the
Tigris upload, so a ref can be announced while the shared durable copy is still
the old one, and a disconnect there cancels the upload while the detached tail
still pins and announces.

Accepted rather than fixed. Upload-then-announce was never guaranteed: release
tolerates a failed upload by design, warning and continuing to the unlock, so an
announce over a stale Tigris copy was already reachable before this reorder, and
it self-heals through acquire_fresh's local fallback plus the next push. The
alternative, detaching release and the tail together, would return 200 before
the durable copy lands, which changes the client contract more than the window
it closes.
2026-07-29 14:31:51 -05:00
beardthelion 9308b9cc27 test(node): record why the handler path cannot probe the stranded-lock case
A handler-level "the next acquire_write succeeds" check does not discriminate:
it still passes with the RepoWriteGuard::Drop backstop disabled. Measured with
the backstop off, the lock is held at the drop and reads free about two seconds
later from a session outside the pool. #[sqlx::test] builds its pool with
idle_timeout(1s), so the guard's connection is reaped once the handler future
drops it, the session ends, and postgres frees the lock with no help from the
code under test. acquire_write retries for far longer than that window, so it
waits the release out and reports success either way.

write_guard_release_cancelled_mid_unlock_frees_the_lock is the real proof: it
probes from a connection held out of the pool, 400ms after the drop, inside that
window rather than past it.
2026-07-29 13:46:33 -05:00
beardthelion 3ced62fab0 fix(node): spawn the replication tail above guard.release() (#174)
The post-receive tail was spawned after `guard.release(...)`, which on a
successful push awaits the Tigris upload and then the advisory unlock while
the handler future is still tied to the client connection. The pack has
already landed on disk by then, so a disconnect in that window left a durable
push whose pins, recovery copy and announce were never spawned: the dropped
tail F2 closed, one step earlier in the handler.

Spawn it above `release`, gated on an explicit `receive_result.is_ok()`. The
`?` below the spawn can no longer be what keeps a rejected push from pinning
and announcing a half-applied repo, so the success check is now a named flag
that `release` consumes too. The tail is read-only on the repo directory and
holds neither the lease nor the advisory lock, so overlapping it with the
upload waits on nothing the handler still owns.

Tests: a handler-level disconnect driven through a parked `release` (new
store-level test seam), plus the must-not direction, a failed receive-pack
that spawns no tail. The U5 source gate now binds the new boundary: spawn
inside `if push_succeeded`, above `guard.release(push_succeeded)`.
2026-07-29 12:37:39 -05:00
beardthelion 7e5e6aabd0 fix(node): keep the repo identity key printable
The key is logged as the `repo` field on the lease waiter-cap shed and the
steal-bound warning (state.rs), and it was joined with a NUL. Those are the two
messages an operator reads to find a contended repo, and a NUL-hostile log sink
truncates the field at the separator, rendering two different repos' warnings
identically. Before this PR that field was a UUID or a mirror id, both printable,
so the NUL was a regression this change introduced.

Join with '/' instead. It cannot occur in the owner slug by construction - the
slug is owner_did.replace([':','/'], "_") - and it mirrors the shape of the disk
path the key exists to reproduce. A test asserts the key carries no control
characters so this cannot regress silently.

Rotating the U2 lease test's row to a UUID id is not incidental: the mirror path
that seeds it uses 'owner_short/name' as the row id, which now coincides with the
identity key and would leave the test unable to tell the two keys apart. The
rotated id is also the real shape of the bug. Re-verified RED against the reverted
call site after the change.
2026-07-29 06:32:40 -05:00
beardthelion 7fa12bebff style: rustfmt the repo_identity_key tests 2026-07-29 03:04:58 -05:00
beardthelion ade53cbf47 fix(node): spawn the replication tail at the durability boundary
The post-receive replication tail was spawned at the end of git_receive_pack, after
touch_repo, the trust-score update, per-ref certificate issuance and webhook fan-out.
A client or proxy disconnect anywhere in that window dropped the tail, and with it
this push's pins, recovery copy, and announcements - even though the push was already
durable on disk.

Spawn it immediately after the receive_result ? instead. That is the first point where
the push is known to have landed: guard.release() a few lines up runs on failure too,
so anchoring there would spawn a tail for a push git rejected, pinning and announcing
a half-applied repo - the state release() deliberately refuses to upload to Tigris,
and one a hostile pusher can produce on demand by aborting a pack mid-transfer.

RefUpdate gains Clone so the tail takes the parsed updates while the certificate and
webhook loops keep iterating their own copy.

This makes the tail run concurrently with certificate issuance rather than after it,
so a ref can be announced before its signed certificate exists. Accepted, and stated
here rather than left for a reviewer to find: cert issuance already fails open (errors
are logged and skipped) and the gossip event carries cert_id: None regardless, so no
announce consumer reads a certificate out of it.

Evidence is an ordering gate in inv22_gates.rs rather than a cancellation-race test,
matching the F3 gate's instrument: forcing a disconnect to land inside the cert/webhook
window is timing-dependent and would be flaky, while the ordering is precisely what
changed. The gate is mutation-verified in both directions - moving the spawn back below
the webhooks fails the upper bound, hoisting it above the ? fails the lower one.
2026-07-29 03:04:30 -05:00
beardthelion 19291dfd64 fix(node): hold the encrypt-scan permit in its closure and run info/refs under git_bin
Two single-site wiring fixes from the same review round.

withheld_recipients_gated held its git_encrypt_semaphore permit in the async frame
and then awaited spawn_blocking, so dropping the future released the permit while the
uncancellable walk kept running. Every other post-receive scan helper moves the permit
into the closure - replication_withheld_set says so in as many words - so this aligns
the last site with the F4 contract. The regression drops the future mid-walk and
asserts the pool stays at zero while the git child is still alive.

git_info_refs passed a literal "git" to smart_http::info_refs while upload-pack and
receive-pack both pass state.git_bin, so no fake-git harness could drive the
advertisement path. The regression points git_bin at a fake emitting a distinctive
advertisement and asserts it reaches the response body; before the fix the body came
back from real system git (agent=git/2.43.0).
2026-07-29 02:55:20 -05:00
beardthelion 7d32a79e96 fix(node): hold /ipfs walk admission through the blocking walk, not the handler future
The global and per-source walk permits were handler locals, so a client disconnect
released them the moment the future dropped while the uncancellable spawn_blocking
probe/walk/read kept running against a slot the pool now considered free. A disconnect
flood could admit replacements while prior blocking git children still occupied
threads and PIDs, weakening GITLAWB_MAX_CONCURRENT_IPFS_WALKS and
GITLAWB_IPFS_WALK_PER_SOURCE on a surface any unauthenticated caller can reach.

Hold the pair in an Arc and clone it into each of the three blocking closures. The
permits release when the last clone drops, which collapses both failure directions
into one case: a disconnect leaves the abandoned closure holding the slot, and a
panicking closure leaves the handler holding it. No loop arm changes and the
JoinError arms keep today's taint-and-continue.

A move-and-return shuttle was implemented and rejected. It is correct only if every
arm that continues the loop re-binds the returned value, which the compiler cannot
enforce - the absent-object arm Ok(Ok(None)) => continue, taken on nearly every
iteration of a random-CID scan, drops it - and it forces a second decision about the
panic arms whose least-bad answer would let one broken repo shed every other owner's
/ipfs lookups.

The regression strengthens the existing admission test: it now asserts the slot is
still held after the future is dropped and before the child is killed, with a
child-still-alive precondition so it cannot pass vacuously. Only the walk site runs
under state.git_bin and can be pinned that way; the probe and read deliberately shell
to real git, so a source-scan gate in inv22_gates.rs binds all three clone sites
instead. Removing any one clone turns that gate red.
2026-07-29 02:46:51 -05:00
beardthelion 99449cf00b fix(node): key the write lease and encrypt coalescing on the stable disk identity
repo_write_leases and encrypt_inflight keyed on record.id, but RepoStore::local_path
and the pg advisory lock key on the sanitized owner slug plus repo name, and the bare
repo on disk is reused when a repo is deleted and recreated under the same slug. The
row id rotates across that recreate, so both in-process serializers stopped serializing
exactly where the shared objects/ directory still needed them: a disconnected push's
reaper could still be tearing down while a new push took a different key and proceeded.

Add repo_identity_key(owner_did, repo_name) and use it at both sites. Three details are
load-bearing and carry tests: the sanitization is repo_disk_path's replace([':','/'],"_")
rather than normalize_owner_key, which folds a did🔑 prefix and would disagree with
the disk path; the separator is a NUL, without which owner 'a' + name 'b_c' and owner
'a_b' + name 'c' collide; and both call sites pass record.owner_did / record.name rather
than the request path segments, since get_repo normalizes DID aliases and the raw
segments would mint two keys for one directory.

The sanitization is deliberately not injective. Two did:web spellings can fold to one
slug, which would misroute encrypt coalescing if it were reachable, but repos.disk_path
is NOT NULL UNIQUE and holds exactly this derivation, so two such rows cannot coexist.
A test pins that fold in place: making the key injective is what would let two owners
sharing one objects/ directory push concurrently.
2026-07-29 02:39:09 -05:00
beardthelion ea84dc5927 fix(node): write coalesced-drain pin metadata under the re-fetched repo id
resolve_drain_object_list already re-fetches the repo row by owner/name and
deliberately reads visibility rules under record.id, because a delete+recreate
under the same slug gives the row a new id while the bare repo on disk is
reused. The encrypt write a few lines later still took ctx.repo_id, frozen when
the task was spawned, so recovery-pin metadata for a coalesced drain landed
against the deleted row and was unreachable from the live one.

Return the re-fetched id from the resolver and pass it explicitly to
pin_and_encrypt_objects. The parameter is explicit rather than read from ctx so
the two callers stay honest about which id they mean: the snapshot iteration
passes its own spawn-time id, the drain passes the fresh one.

The Arweave anchor is unaffected - EncryptedManifest carries no repo id and
builds its slug from owner and name, both stable across the rotation.
2026-07-29 02:16:04 -05:00
beardthelion 799759f1fc fix(node): sweep the /ipfs rate limiter and document the lease waiter cap
The 300s cleanup task swept the DID, create-IP, push, sync-trigger and
peer-write limiters but not ipfs_rate_limiter, so its expired keys sat
until the map hit max_keys and the inline capacity sweep reclaimed them.
Drive the sweep off AppState in a named sweep_rate_limiters so a limiter
added to the state and forgotten here fails a test instead of leaking
keys, and cover all six with a regression that goes red when any one is
dropped from the sweep.

GITLAWB_REPO_LEASE_MAX_WAITERS was also missing from .env.example and the
README settings table, leaving operators without a way to discover or
tune the cap behind the "repo is busy with another push" 503.
2026-07-28 16:01:30 -05:00
beardthelion c8e8de2949 fix(node): stop a failed walk from taking a pin permit, and bound the close on an errored unlock
Three findings from the review round.

Moving the Pinata gate to the rules-only predicate was necessary, because a
coalesced push has no walk of its own to consult. But it also meant a push whose
OWN walk failed now acquired a global pin permit and ran a second full
re-derivation that failed the same way. Measured on a repo whose rev-list always
fails: twelve git children and three walk attempts, against four and one before.
With eight pin slots that defer rather than shed, eight such pushes stall pins for
every repo on the node. The gate now excludes a non-coalesced push whose own walk
failed, leaving the coalesced path on the rules-only predicate it needs, and the
pin helper returns before taking a permit for an empty object list.

The connection close on the failing-unlock path had no deadline, and release is
awaited inline while the write lease and both admission permits are still held.
The condition that produces that branch is a connection whose last statement
errored, and a blackholed socket to Postgres is one way to get there, so the await
could pin three admission resources for a TCP timeout. It now runs under a five
second bound; on expiry the future is dropped, which closes the socket and ends
the session anyway, which is what frees the lock. What is tested is the deadline.
That sqlx's close is the thing that stalls in production is reasoned from its
source, not reproduced, and the test says so.

The two tests for that path waited a fixed four hundred milliseconds before
asserting. Replacing the sleep with polling was not enough on its own: the test
pool sets a one second idle timeout, so a connection returned to the pool is
reaped shortly after and the session ends without any help from the code under
test. Both tests passed under mutation once the sleep was long enough to reach
that reaper, which means a straightforward flake fix would have quietly gutted
them. They now run against a pool with no idle reaper, so release is the only
thing that can end the session, and both mutations go red again.

Adds tokio's test-util as a dev-dependency for paused-time assertions. Production
dependencies are unchanged and the build stays --locked clean.
2026-07-28 09:14:54 -05:00
beardthelion de325f9f46 fix(node): bound lease waiters per repo, and stop spending cross-repo budget on a per-repo wait
The previous attempt moved the per-source write permit above the lease park to
bound buffered pack bodies. Two reviewers demonstrated that it denies far more
than the source doing the parking. A source parked on one repo's lease was shed
on repos with no contention at all, and because the trusted-proxy setting
defaults to trusting nothing and keying on the socket peer, every pusher behind a
proxy, NAT, or CI runner pool resolves to one key: four parked pushes shed every
push from every pusher to every repo for up to twenty-one minutes. That is the
node-wide shed the permit was moved below the lease to prevent in the first
place, reintroduced at an eighth of the threshold.

The permit goes back below the lease. What is actually unbounded is the waiter
set, so that is what is bounded now, per repo, with a shed past the cap.

The cap counts live waiters, never the entry refcount. The refcount includes the
holder, so capping it would let a lease whose owner died without unwinding pin a
slot forever, which is the permanent wedge the steal timeout exists to survive.
The count is claimed in the same critical section as the refcount and held by a
guard scoped to the cancellable wait, so a cancelled or shed waiter gives it back.
An uncontended acquire takes the fast path and spends no waiter budget; that path
does not barge a queued waiter, which was checked over two thousand rounds rather
than assumed.

Two tests now guard the defect that was just removed: a source parked on one repo
is still served on another, and a parked push does not shed an unrelated pusher
sharing its resolved key. Restoring the old ordering turns both red. Capping on
the refcount instead of live waiters turns the steal-backstop test red. Holding
the waiter slot past the wait turns the fast-path test red.

What this bounds is the parked count, not the parked bytes. At a two gigabyte
pack limit the byte figure stays large, and the total across repos, whose
dominant term is the buffering the signature middleware does before any admission
point at all, is tracked in #263. Past the cap, same-repo concurrency sheds
instead of queueing; that is the trade the knob buys.
2026-07-28 08:01:51 -05:00
beardthelion 1b1a9bab70 test(node): observe the write-cap shed as state, not as a wall-clock deadline
The two F1 tests asserted "the capped source shed rather than parked" by racing a
ten second timeout, and drove the holding push in ten millisecond slices. Under
`cargo test --workspace` that is a race against the scheduler rather than a test:
a reviewer saw two of five runs go red, and under load the pair stretched from
1.8s to 19.2s while the fake git holding the lease only lived for 10s, past which
every assertion in both tests is meaningless.

Shed versus park is a question of which path ran, so both are now read as state.
A shed is `Err(Overloaded)`. A park is a second reference on the lease entry,
exposed by a test-only `refs_for`; `acquire` takes that reference synchronously
before the cancellable wait, so a waiter is visible the instant it reaches the
lease, and it stays visible because the steal bound is 1260s. The tests spawn
their pushes and poll until one of the two mutually exclusive states appears, so
a starved machine iterates more rather than failing.

Every wall-clock assertion is gone. The worst of them was a sleep followed by
`!is_finished()`: a runtime that had not yet polled that task satisfied it
vacuously, which means the constant-key mutation could have gone green under
load, and the discriminator is the whole reason that test exists.

All three mutations still go red, now in seconds rather than by timeout expiry:
the per-caller acquire below the lease, the global permit above it, and the
per-caller key replaced with a constant.

Also fixes a third flake found on the way: the acquire-deadline test's follow-up
push inherited the deliberately short two second acquire timeout, which has to
cover a real advisory-lock round trip, so its `Overloaded` was indistinguishable
from the drained-pool `Overloaded` the assertion was actually testing for.

The sqlx `databases_pkey` collisions seen alongside this are machine residue, not
a leak from these tests: sqlx derives the test database name from a hash of the
test path and does cleanup-then-insert without a lock, so any two concurrent test
binaries running the same path collide. The orphan count here held at 53 across
every run.
2026-07-28 02:40:37 -05:00
beardthelion eb0460ae81 Merge remote-tracking branch 'origin/main' into fix/served-git-concurrency-cap 2026-07-28 00:21:00 -05:00
beardthelion d93f78400c fix(node): coalesce a repeat push before it parks on the encrypt pool, not after
The post-push tail resolved the withheld set and materialized the object list
before it ever consulted the per-repo coalescer, so rapid pushes to one repo each
parked on the encrypt semaphore and repeated the scan before discovering they
should have coalesced. The gate now runs first, keyed on the cheap
listable-at-root predicate that `replication_withheld_set` already computes from
rules alone before it touches the semaphore.

Three things made this less mechanical than it looks, all of them ways a naive
move would have shipped a regression.

The withheld set had a second consumer. It also gated the Pinata pin and the
branch-to-CID upsert, so a coalesced push that skipped the walk would have pinned
nothing and recorded no CID, while a test asserting merely that the announce
spawn ran would have stayed green. That gate moves to the announce predicate too.

Coalescing must not return from the tail. The announce spawn sits below the
coalescer in the same task, and returning early would drop gossip, the GraphQL
broadcast, Arweave anchoring and peer notify, which are per-push and
non-idempotent. A flag skips the encrypt section and falls through.

The publish paths are gated on announce and iterate the ref updates, not the
object list, so handing the tail the cheap predicate would have made a repo whose
walk is failing start gossiping and anchoring, losing the fail-closed property.
`pinata_object_list_for_refs` already recomputes the real set inside the pin
permit, so it now returns that verdict alongside the list and the tail uses it.
No additional walk is introduced.

Moving the gate earlier also opens a window where a push can coalesce while the
admitted one is still walking. If that walk then fails, the admitted task now
still spawns the drain with an empty snapshot so those tips are consumed, rather
than dropping the guard and discarding them. Note a released guard and a dropped
guard are indistinguishable by the in-flight map, since Drop clears the key
either way, so that case is asserted on drained work rather than on the map being
empty.

Each of the four is pinned by a mutation: walking anyway, returning early,
leaving the Pinata gate on the withheld set, and dropping instead of draining
each turn exactly one test red.
2026-07-28 00:05:04 -05:00
beardthelion 72fe5cb3bf fix(node): end the session when the advisory unlock errors, instead of pooling a locked connection
`release` discarded the unlock's result and set `released` regardless, so an
unlock that errored while its session stayed alive (a statement timeout, an admin
cancel) returned a connection to the pool still holding the repo's advisory lock,
with the Drop backstop already disarmed by that flag.

The error is now captured. On failure the connection is taken and closed, which
ends the session and makes Postgres drop the lock server-side, and only then is
the guard marked released, because the lock went with the session rather than
with the unlock. `close()` rather than `detach()` since we are already in an async
fn and can wait for the teardown deterministically.

Disposal is the only mechanism on this path. It cannot coexist with an armed Drop
backstop: disposal needs the connection taken, and Drop returns early when there
is none, while leaving the connection in place so Drop can use it just puts the
locked connection back in the pool when that spawned task ends. The Drop doc now
says so rather than claiming to cover a case it cannot.

The cancellation path is untouched. It was already correct, since `released` is
set after the await, and its test still passes unmodified.

The new tests assert against `pg_locks` from a connection held out of the pool
before the acquire, never by calling `acquire_write` a second time. That shortcut
would pass whether or not the fix is present, because the pool can hand back the
same connection and session advisory locks are re-entrant. The unlock error is
injected by leaving the pinned connection in an aborted transaction, which errors
the next statement while keeping the session and its lock alive, and the test
asserts that precondition before releasing.

Not covered: that the failure path still skips the Tigris upload. The test store
builds with no Tigris client, so the upload site is unobservable here without a
new seam, and that block is untouched by this change.
2026-07-27 23:02:15 -05:00
beardthelion 5b46796b36 fix(node): take the per-source write permit before parking on the repo lease
A push buffers its whole pack before the handler runs, then parks on the per-repo
write lease for up to git_service_timeout_secs * 2 + 60, which is 1260s at
defaults. Both admission permits were taken after that park, so nothing accounted
for waiters: any number of requests could sit on one contended repo, each holding
a fully buffered body, invisible to both semaphores.

The per-source sub-cap now runs before the park. It already exists, is sized at a
quarter of the push pool, and is keyed on the resolved source IP, so parked bodies
are bounded at four per source without a new knob or a new cap to tune.

The global permit deliberately stays below the lease. Moving it up is what
f02c5e1 fixed: a lease-blocked waiter holding a global slot sheds pushes to every
other repo node-wide. A per-source key does not have that shape, since exhausting
it denies only the source doing the exhausting, so the two permits belong on
opposite sides of the park and the comment there now says why.

The load-bearing test is not that a capped source sheds, which any cap satisfies,
including the global one this rejects. It is that a different source still parks
and is served while the first is at its cap. Keying the permit on a constant
leaves the shed test green and turns only that one red. Moving the global permit
above the lease turns the f02c5e1 guard red.

Bounded per source, not globally. A distributed set of sources can still each hold
their four, and the unbounded-total term is the pre-auth buffering in #263 rather
than the lease.
2026-07-27 22:45:48 -05:00
beardthelion 219fbdbdba test(node): cover the sweep's boot wiring and prove an interrupted push never reaches the upload
Two claims from earlier commits were argued rather than executed. Both are now
observed.

The legacy-CID sweep's spawn sat inline in main() where no test could reach it,
so the batch size, the inter-batch delay, the detachment and the shutdown select
were all reasoned. The block moves verbatim into `spawn_legacy_cid_sweep`, which
a test calls directly. It seeds rows whose keys are already canonical, so the
cost gate skips each and the persisted cursor reads out walk progress cleanly,
then asserts the cursor stops at the configured batch boundary, the call returns
without blocking, the task is still alive inside its delay, and shutdown ends it.
Never spawning, reading the batch or delay from the wrong knob, or removing the
shutdown branch each turns it red. What remains reasoned is the single call line
inside main(), down from a twenty-six line block.

The claim that a disconnect-interrupted push never uploads to Tigris was
structural: the tests build the store with no Tigris client, so a counter inside
the client arm could never move and would have proved nothing. The counter now
sits at the upload decision, before the client is consulted, and the disconnect
test asserts zero while the success test asserts exactly one. That positive
control is what makes the zero meaningful, and it is itself load-bearing:
releasing the success path with failure semantics drops it to zero, and adding a
Drop impl that uploads raises the disconnect count to one.

The counter observes reaching the upload site, not an S3 request. TigrisClient
wraps a concrete SDK client with no injectable endpoint, and plumbing one through
would be out of proportion to the risk here.
2026-07-27 21:44:31 -05:00
beardthelion d85416127b fix(node): shed 503 from the issue and PR write paths when the lock pool is exhausted
The lock-pool work gave `acquire_write` a distinct exhaustion error and mapped it
to 503 with Retry-After, but only on the push handler. The three other callers,
both issue writes and the PR merge, still turned it into a generic git error and
returned 500. Those are exactly the callers that hold no concurrency permit, so
they are the likeliest to meet an exhausted pool in the first place.

All three now go through the push handler's helper rather than repeating the
mapping, so the log level and the message cannot drift between the four sites.
Each has a test that occupies the single connection of a one-slot pool with a
guard on a different repo, so the failure is pool capacity rather than
advisory-lock contention, and asserts status 503 plus retry-after through
IntoResponse. Reverting the mapping at one site turns only that site's test red.

Worth knowing for review, and left alone deliberately: close_issue takes the
write guard before its author check, so this shed is reachable by any
authenticated caller whether or not they could close the issue. That ordering
predates this change and reordering the handler is not this commit's business,
but it does mean the 503 discloses that the repo's write pool is busy.
2026-07-27 21:14:55 -05:00
beardthelion ef9b7165cf fix(node): keep the incompleteness marker until a source is really recorded, and let the sweep retry
Four follow-ups from an adversarial pass over the previous commits.

The marker clear ran unconditionally inside `record_pin_source`'s transaction,
but its INSERT is `ON CONFLICT DO NOTHING` guarded by the cap, so it records
nothing when the repo is already a source. That is the common case, not a rare
one: the pin path calls `record_pin_source` for every already-pinned object, and
on a requeue pass that is the whole-repo enumeration. So a re-push from the repo
that was already recorded cleared a marker set by a different repo's failure and
the public copy went back to 404ing. The clear is now gated on the insert
actually adding a row. A genuine record from a third repo still clears the
marker, which needs a per-(oid, repo) table to close and is documented rather
than papered over.

The sweep advanced its cursor before every skip, which stops a wedge but made a
transient skip permanent: once the walk reached the end of the table the cursor
parked there and no later boot read another row. On a Tigris-backed node the
common skip is "repo is not on local disk", so the sweep would repair whatever
happened to be warm at boot and then never run again, and the new advertise
filter turned those rows from "advertised but 404" into "never advertised, never
repaired". Repairs now report whether a skip was retryable or terminal, and a run
that skipped anything retryable rewinds the cursor once, after the walk, so the
next run re-walks. A permanently unrepairable row is terminal and does not
trigger a rewind, so this cannot become a hot loop.

The sweep also reached its repo paths through the raw join helper, skipping the
three-layer traversal barrier `RepoStore::local_path` applies. A test with a repo
row named `../../escapee` showed it reading and repairing bytes from outside
`repos_dir`. That barrier is now extracted and shared, so the two callers cannot
drift. And the repair's synchronous `git cat-file` moved under `spawn_blocking`,
matching the existing treatment of the withheld walk, so a wedged read no longer
parks a worker thread for the full service timeout.

Finally the resolver's two source-set queries ran ahead of the work-budget peek
that exists to shed a spent caller before doing work. The peek now runs first. A
throttled caller whose set turns out complete gets 429 where it used to get 404,
which is the honest answer since its search never ran, and it removes an oracle
that let a throttled caller tell a complete source set from an incomplete one.
2026-07-27 18:29:01 -05:00
beardthelion 752fd19b6d fix(node): stop the lock pool from being occupied by spinners and racing the reaper
Three regressions from the previous four commits, each demonstrated before being
fixed.

The advisory-lock retry loop checked its connection out before spinning, so a
caller that lost the lock held a pool connection through all sixty one-second
attempts. The old pool-per-query shape returned it between iterations, so a
spinner occupied nothing. That matters more than the pool sizing suggested:
`acquire_write` has three non-push callers (two in issues, one in pulls) that
hold no concurrency permit, and the issue routes carry no rate limit, so any
self-minted DID could fire enough concurrent closes at one repo to strand the
pool and fail authenticated pushes on every repo. The checkout now happens per
attempt and a losing attempt returns the connection before sleeping, so only the
winner retains one. Lock-pool exhaustion also carries its own error now and sheds
503 with Retry-After instead of surfacing as a 500.

Holding the lock on one connection then meant a client disconnect released it
early: the handler future dies mid-receive-pack, the guard drops without
`release`, and `after_release` frees the lock while the detached reaper is still
giving the process group its SIGTERM grace. A second writer was admitted 90ms
after the drop, well inside that window, which is the invariant smart_http
already documents. The write guard now rides the AdmissionGuard into the reaper,
the same seam the admission permits already use, so the lock outlives the group.
The success path still reclaims the guard and releases it synchronously, and a
dropped guard performs no Tigris upload, so an interrupted push cannot publish a
half-applied repo.

The requeue loop's exit on an exhausted re-read discarded a push that coalesced
during the retry window: `requeue_or_release` had cleared the dirty bit without
marking the guard released, so `Drop` removed the key and no pass was ever
attempted. It now falls through with nothing to replicate and lets the tail's
atomic check-and-clear decide. A lap only happens when a push actually coalesced
and each pays a full bounded re-read, so a sustained outage still terminates,
which is tested with ten thousand injected faults under a watchdog rather than
argued.

Also clamps the lock pool's derived size, which previously followed
max_concurrent_git_pushes up to its million-connection ceiling, and documents the
node's total connection budget on that knob.
2026-07-27 17:47:56 -05:00
beardthelion 412d04e12d fix(node): sweep legacy provider-CID pins instead of waiting for a re-push
`repair_legacy_provider_cid` had exactly one trigger: the already-pinned skip
branch inside `pin_new_objects`, which a push only reaches when it re-carries the
object. Normal git negotiation omits objects the node already has, so on an
upgraded node that push generally never arrives. Meanwhile `list_pinned_cids`
kept advertising the stored provider CID that this branch's `/ipfs/{cid}`
deliberately withholds, so the node handed out CIDs it would then refuse to
serve, and previously pinned data stayed unresolvable.

Two halves, both needed. A detached background sweep walks `pinned_cids` in
bounded batches with an inter-batch delay, resolves each row's repo from its
recorded provenance, and reuses the existing repair (and its cost gate, so a row
already keyed on a raw CIDv1 reads no bytes). Migration v16 adds a single-row
cursor table so a restart mid-walk resumes rather than rewinding; the cursor
advances on every row read, before any skip, so an unrepairable row cannot wedge
the walk. A row whose bytes are gone, whose provenance is NULL, or whose repo is
not on local disk is left exactly as it is. And `list_pinned_cids` now omits rows
whose key is not a raw CIDv1, so the window before the sweep catches up
advertises nothing the resolver withholds.

The sweep resolves paths directly rather than through `repo_store.acquire`, which
would pull cold repos back from Tigris and turn a repair pass into a bulk
restore. Defaults are deliberately quiet: 64 rows per batch, 60s between batches.

Ten tests cover the repair, the bytes-gone and unrepairable arms, the cost gate,
the batch bound, cursor resumption, both states of one row through the advertise
filter, and the degenerate empty and no-legacy-rows tables. Removing the filter,
the batch bound, the cursor advance, the bytes-gone early return, or the
inter-batch sleep each turns a test red.

One pre-existing test used placeholder resolver keys that the new filter drops;
its values are now real raw CIDv1s with every assertion unchanged.
2026-07-27 15:53:26 -05:00
beardthelion ff189ceab9 fix(node): record a failed pin-source write so the resolver keeps its fallback
The resolver treated a non-empty, below-cap pin-source set as proof that every
source had been recorded, and skipped the bounded scan on that basis. But
`record_pin_source` is best effort at every call site: the ipfs_pin sites were
warn-only once their retries exhausted, and both pinata sites had no retry at
all. An object first pinned from a private repo and later pushed from a public
one whose source write failed keeps a set naming only the private source, so
`GET /ipfs/{cid}` returned 404 for an object the public repo would have served.

Migration v15 adds `pinned_cids.pin_sources_incomplete`, NOT NULL DEFAULT FALSE
so every existing row reads as complete. A record that fails outright sets it; a
later successful record clears it, inside the same transaction as the insert so
the two paths cannot drift. `needs_scan` now ORs the marker in, short-circuiting
after the existing empty and at-cap checks so it costs nothing on the serve path.
Pinata's two bare calls now share `retry_db_record` with ipfs_pin. The ipfs_pin
first-pin path additionally records the CID and its source in one transaction;
Pinata's remains two retried calls.

INV-10 is preserved: a complete set still tail-404s with no preload, asserted on
the existing preload counter. The fallback is not an authorization bypass, and
the test for that asserts the scan actually ran before checking the denial, so
the denial cannot pass vacuously. Dropping the marker term from the gate, forcing
it true, or removing the clear each turns a test red.

The marker is per-object rather than per-(object, repo): a successful record from
one repo clears a marker set by another's failure. That is the conservative
direction, since the marker only ever adds the fallback back.
2026-07-27 14:40:03 -05:00
beardthelion 2961ba73ec fix(node): retry the requeue re-read so a coalesced push is not silently dropped
`requeue_or_release` clears the dirty flag atomically with the decision to loop,
which is correct and stays as it is. The problem was what the pass did next: a
failed `get_repo_by_id` fell into an Err arm that yielded no rules, and
`list_visibility_rules(...).ok()` collapsed its own error the same way. With no
rules, `replication_withheld_set` returns `(false, None)`, the pass skipped its
work entirely, and the loop exited having consumed the dirty bit that stood for
the coalesced push. There is no reconciliation sweep to re-derive it, so that
push's objects were never pinned and its withheld blobs never sealed.

`requeue_refresh_state` now separates the three cases the old code collapsed.
`Ok(Some)` proceeds. `Ok(None)` means the repo really is gone and releases
without touching the retry budget. `Err` from either read is transient and is
retried with doubling backoff up to three attempts. A rules-read error is no
longer indistinguishable from a repo that has no rules.

Exhausting the retries still drops the pass, which is the pre-existing residual,
but it now logs at error level with the repo id and attempt count instead of
disappearing behind a warning. Six tests cover the retried-then-lands path, the
bounded-exhaustion path, repo-gone, the rules-read arm, freshness of the applied
rules, and the coalescing property that had to survive. Reverting the Err arm,
restoring the `.ok()`, raising the bound, or downgrading the error log each turns
a test red.
2026-07-27 13:52:44 -05:00
beardthelion a4f7d5dffb fix(node): own the repo write lock on one connection so cancellation cannot leak it
`acquire_write` took `pg_try_advisory_lock` through the shared pool, then awaited
the Tigris exists/download path before constructing `RepoWriteGuard`. Two bugs
fell out of that. The `tokio::time::timeout` this branch added around the call
can drop the future inside that window, after the lock is held and before the
only unlock (`RepoWriteGuard::release`) exists, wedging every later push to the
repo. And because a session advisory lock belongs to the connection that took
it, running the lock and the unlock through the pool lets them land on different
connections: the unlock returns false and leaks, while a competing acquire that
draws the holding connection re-enters the lock and two writers run against the
same repo concurrently.

`RepoStore` now owns a dedicated lock pool built with an `after_release` hook
that runs `pg_advisory_unlock_all()`. `acquire_write` checks out one connection,
takes the lock on it, and moves it into the guard, which unlocks on that same
connection. sqlx's `PoolConnection::drop` spawns `return_to_pool`, which runs the
hook, so a connection dropped by cancellation still clears its locks; the unlock
is asynchronous with respect to the drop, which the tests account for. The pool
is separate from the main query pool because a push holds its connection for the
whole receive-pack and `db_max_connections` (20) sits below
`max_concurrent_git_pushes` (32).

Seven tests cover the cancellation window, the same-session release, and the
serialization the pool-mismatch bug was breaking. Both the cancellation
regression and the concurrent-writer case were observed red before the fix, and
removing the `after_release` hook or moving the unlock back to the pool turns
them red again.
2026-07-27 13:22:45 -05:00
beardthelion cc41d37bef fix(node): bound the post-push pin and repair reads so a wedge cannot pin the coalescing key
A cross-model adversarial pass found that run_post_push_replication holds the
per-repo coalescing key until requeue_or_release, but pin_new_objects reached it
only after two unbounded git reads: the U7 legacy-repair read and the
pre-existing pin read, both plain Command::output with no teardown. A wedged
cat-file on a stuck backend hung the task forever, so the key was held until
process death and later pushes only marked it dirty without spawning a
replacement (the same class this PR closed on the /ipfs serve path).

Add read_object_bounded (the bounded twins under one shared deadline) and use it
at both sites; a timeout skips that object and lets the task proceed to
requeue_or_release. Adds a wedge-reap regression proven load-bearing two ways.
2026-07-20 21:06:10 -05:00
beardthelion 9cef54ea18 fix(review): share one deadline across the /ipfs size+content reads
Code review found each candidate's size and content reads were each granted a
full git_service_timeout, so a single served candidate could hold the /ipfs
walk permit for 2x the timeout. Share one deadline across both stages (mirrors
build_filtered_pack) so the read holds admission for one timeout total, and
correct the admission-bound comment: the worst case is O(candidates x timeout)
bounded by the walk concurrency cap, not a single deadline.
2026-07-20 20:27:35 -05:00
beardthelion 15bdc5e64a fix(node): opportunistically repair legacy provider-CID pins
Before this PR the pin path stored the provider CID (Kubo dag-pb / Pinata) in
pinned_cids.cid, which the new resolver recomputes-and-404s, while
/api/v1/ipfs/pins still advertises the stale value. Repair a legacy row in the
already-pinned skip branch: gate on a cheap stored-CID codec check (only a
non-raw-codec CID reads bytes, so raw rows keep the DB-only skip cost),
recompute the raw CID, and rewrite it while stashing the old value in a new v14
legacy_provider_cid column (distinct from pinata_cid, which gates the Pinata
pin-skip). Rows whose bytes are gone stay withheld. Documents that the deferred
one-shot sweep, not this opportunistic path, fully retires the window.
2026-07-20 20:11:42 -05:00
beardthelion c04c4a5da4 fix(node): retry pin-source and pinned-CID recording on transient errors
The three warn-only record sites in the detached post-push task dropped a
transient DB failure silently, leaving a permanently incomplete pin-source set
that makes the resolver 404 a valid public copy. Wrap them in a bounded retry
(inserts are idempotent via ON CONFLICT DO NOTHING); on exhaustion the warn
still fires so behavior degrades to today's. Documents the residual crash/outage
window that only a reconciliation sweep retires.
2026-07-20 19:50:23 -05:00
beardthelion 20c338eca5 fix(node): split /ipfs work accounting off the route rate limiter
The route middleware charged ipfs_rate_limiter once per request while the
legacy scan and provenance walk charged the same bucket per probe/walk, so a
one-probe request cost two tokens and GITLAWB_IPFS_RATE_LIMIT=1 admitted the
request at the route then 429'd it at the pre-scan peek.

Add a bounded ipfs_work_rate_limiter, move the in-scan, per-walk, and pre-scan
charges onto it, and leave the route limiter as the pure once-per-request
brake. The work-budget capacity derives from the route limit with a floor of
one full legacy-probe budget per window (no new operator knob), so a
default-config deep search never self-throttles. Adds it to the periodic sweep,
updates all three AppState construction sites, and rewrites the contradictory
limiter comments.
2026-07-20 19:41:57 -05:00
beardthelion 86fc2763a3 fix(node): wire GITLAWB_IPFS_MAX_REPOS_WALKED to the legacy-probe budget
The knob was parsed and documented as the per-request /ipfs fan-out cap but
never read in production: AppState seeded the legacy-probe budget from a fixed
256 constant, so setting the knob did nothing and an operator could still incur
256 acquire/probe operations per request.

Seed ipfs_max_legacy_probes from the knob (default raised to 256 to preserve
shipped behavior); leave the history-walk ceiling on its MAX_PIN_SOURCES+1
constant so a provenanced request is never truncated into a false 503. Updates
the README and .env.example to the shipped behavior and adds cap-honored and
plumbing tests.
2026-07-20 19:17:12 -05:00
beardthelion 6222775bb1 fix(node): requeue coalesced pushes instead of dropping them
Per-repo coalescing dropped every push that arrived while a repo's encryption
task was in flight, and no reconciliation ever processed it, so a withheld
blob added by the coalesced push had its recovery copy permanently absent. The
drop also silently skipped the coalesced push's local-IPFS pin work.

EncryptInflight becomes a dirty-flag map; the detached task loops, and the
atomic check-and-clear sits at the task tail, unconditional on the
has_path_scoped_rule gate and walk success (a public or rules-free repo would
otherwise exit before it ran). Each requeue re-reads repo state fresh and
re-enumerates the pin half through the fail-closed full scan, never bare
list_all_objects, so a coalesced rule change is honored and no withheld or
dangling object leaks. At-most-one-task-per-repo is preserved.
2026-07-20 19:04:55 -05:00
beardthelion b881447b59 fix(node): own /ipfs admission for the life of the walk
get_by_cid held the global walk permit and the per-source permit as handler
locals, so dropping the request future released both while the spawn_blocking
probe, visibility walk, and object read kept running; a cancel-spam or timeout
client could exceed MAX_CONCURRENT_IPFS_WALKS and the per-source cap, and the
bare cat-file probe/read children had no teardown at all.

Move the gated serve pipeline into a detached task that owns an AdmissionGuard,
awaited by the handler, so admission releases only after the bounded work is
gone; back the /ipfs probe and read with run_bounded_git twins for
process-group teardown. Adds cancel-mid-walk, timeout-reap, and cancel-spam
regressions plus two load-bearing INV-22 rows. Also reflows one U1 test line to
satisfy cargo fmt.
2026-07-20 18:34:09 -05:00
beardthelion de2ad5a83d fix(node): hold read admission through the filtered-pack reaper
The path-scoped upload-pack branch kept the read and per-caller permits as
handler locals and called upload_pack_excluding with no AdmissionGuard, so a
cancelled filtered clone released admission while KillGroupOnDrop was still
reaping the git group, bypassing the read and per-caller concurrency caps.

drive_git_child now returns the disarmed guard on success so one guard rides
both build_filtered_pack stages (rev-list then pack-objects); the handler builds
the guard from the permits as the plain path does. Adds a disconnect regression
and an INV-22 gate row, both proven load-bearing.
2026-07-20 17:54:49 -05:00
beardthelion 6afa236e65 docs(node): correct three lease/replication comments from code review (#174)
- steal_after: the reclaim is NOT a guarantee that only a leaked lease is
  reclaimed. A waiter's timeout starts at acquire(), not at the FIFO head, so a
  same-repo backlog whose cumulative wait exceeds steal_after can steal while an
  earlier waiter still writes; correctness rests on the retained pg advisory
  lock (which serializes the stealer at acquire_write), not on the bound.
- pinata_object_list_for_refs: recomputes from the tail-start rules snapshot,
  not a fresh read at pin-worker time, so a rule tightened AFTER tail-start is
  not reflected (matches the old retained-list behavior; reconciliation sweep is
  the backstop). Removed the overstated 'rule tightened since the push is honored'.
- RepoWriteGuard::Drop: note that on runtime shutdown the detached unlock task
  may be dropped before it polls, bounded by pool teardown / connection close.
2026-07-20 12:02:46 -05:00
beardthelion e1ab240c7a fix(node): don't 500 an /ipfs miss when a transient taint co-occurs (#174 F5 review)
The deterministic-fault terminal 500 was checked before the transient
truncated_by 503, unconditionally. When one repo is corrupt (deterministic)
while a DIFFERENT repo is transiently skipped in the same scan, the requested
CID may live in the transient repo -- a retryable 503 is correct, but the
handler returned a terminal 500 and a conformant client won't retry, hiding
retrievable content until the unrelated corrupt repo is repaired.

Gate the 500 on truncated_by.is_empty(): it fires only when a deterministic
fault is the SOLE reason nothing served; a co-occurring transient taint falls
through to the retryable 503. A pure deterministic fault still terminally 500s
an absent lookup by design. Adds a co-occurrence regression test.
2026-07-20 11:54:09 -05:00
beardthelion e383047d71 fix(node): take the write permits after the per-repo lease, not before (#174 F3 review)
Code review found a DoS the F3 lease introduced: git_receive_pack acquired the
global and per-source write permits before block-waiting on the per-repo lease,
so a second same-repo push that parked on the lease pinned a scarce global
write-pool slot (1 of 32) for up to steal_after while sending zero bytes. A few
hostile sources stacking same-repo pushes could hold every slot on idle
lease-waiters and shed 503 on every push to every other repo node-wide.

Move both permit acquisitions to after the lease acquire so a lease-blocked
waiter pins no write-pool slot, and keep a cheap non-holding availability peek
before the DB lookup so a push flood on a saturated pool still sheds 503 without
touching Postgres (the authoritative held permit is taken after the lease).
Adds a blocked-waiter-holds-no-permit regression test.
2026-07-20 11:54:09 -05:00
beardthelion 9f4a8c7d2f fix(node): serialize a disconnected push's repo until its group is reaped (#174 F3)
On client disconnect, RepoWriteGuard::Drop released the per-repo advisory
lock immediately (it holds no child/pgid), while KillGroupOnDrop's detached
reaper was still tearing down the git process group for ~4s. A second same-node
push could then acquire the repo and race the first's still-writing objects/
dir and Tigris upload.

Add an in-process per-repo write lease (block-and-wait, keyed like the advisory
lock) that SUPPLEMENTS the retained cluster-wide PG lock. The lease rides into
the disconnect reaper via a clone on the write-path AdmissionGuard (None on all
read paths) and a second clone held across guard.release() for the clean-path
Tigris upload, so it frees only after the group is reaped. A bounded-wait steal
reclaims a leaked lease (block-and-wait has no coalesce degradation). The lease
is always taken before the PG lock and released after, so the two serializers
cannot invert. Deterministic fake-git race test + inv22 gate row.
2026-07-20 10:58:37 -05:00
beardthelion b897e27cf2 fix(node): bound post-receive Pinata replication memory (#174 F2)
The detached Pinata task moved each push's full object-id list into the
closure and parked it on pin_semaphore.acquire_owned(); a tokio semaphore
bounds active holders, not the waiter queue, so a slow Pinata backend let N
queued pushes each retain an MB-scale OID list (unbounded memory/tasks).

Capture only the small ref-update tuples and re-derive the object set via the
existing reaped/deadline-bounded git helpers AFTER the pin permit is acquired,
so retained memory is O(ref tuples), not O(object lists). Per-ref effects are
unchanged (still one spawn per push, no coalescing or shedding, so no dropped
announcements). Adds an inv22 gate row forbidding a retained object list across
the acquire, plus re-derivation-equivalence and reaping tests.
2026-07-20 10:19:50 -05:00
beardthelion e866584784 fix(node): clamp the initial /ipfs metadata queries to the request budget (#174 F6)
list_all_repos and list_visibility_rules_for_repos ran as bare awaits while
holding the scarce IPFS walk permits, so a query blocked in Postgres pinned the
slots past GITLAWB_IPFS_REQUEST_BUDGET_SECS and shed later requests as 503s.
Wrap both in the remaining request-deadline budget; on timeout the RAII permits
drop and the handler sheds a retryable budget 503. Fails closed: a timeout on
the visibility-rules query denies the request rather than serving an unfiltered
listing.

Appends an inv22 gate row asserting both queries stay deadline-wrapped.
2026-07-20 10:00:40 -05:00
beardthelion 48dbb75ac6 fix(node): classify /ipfs object probe with a version-stable signal (#174 F5)
Probe existence via `git cat-file --batch-check` (structured `<oid> missing`
on exit 0) instead of matching English `cat-file -t` prose, so a git wording
change cannot flip a miss into an error or vice-versa. A bad-config/corrupt
repo on a readable store is now a terminal, non-retryable 500 (not a false 404,
and not a retryable 503 that would fan out cat-file retries); a transient
unreadable store stays 503. Error bodies are opaque (no raw git stderr).

Layers onto the existing object_store_readable + re-probe disambiguation.
2026-07-20 09:36:09 -05:00
beardthelion 671cee184f fix(node): make RepoWriteGuard::release cancellation-safe (#174 F4)
Await pg_advisory_unlock while the pooled connection is still owned by
self; take it and set released only after the await resolves. A cancel
mid-unlock now leaves the Drop backstop armed (conn still Some), so the
session-level advisory lock is no longer leaked onto a pooled connection.

Adds a cancel-mid-unlock regression test (via a test-only pre-unlock gate)
and an append-only inv22 gate row asserting the take/released orderings.
2026-07-20 09:14:01 -05:00
beardthelion cc9615961f fix(node): bound concurrent pin-loop memory; document the per-repo task cap (#174 F6)
EncryptInflight bounds the outstanding post-push encryption-task set to one per
repo, but its state.rs doc implied a global cap it does not deliver, and each pin
loop (ipfs_pin + pinata pin_new_objects) holds a full per-push object-id list
while walking it — so N distinct authenticated first-pushes could hold N MB-scale
lists at once, unbounded.

Add a global pin_semaphore (GITLAWB_MAX_CONCURRENT_PIN_TASKS, default 8) and gate
BOTH pin loops through it: they DEFER when the pool is full, never drop a pin.
Tighten the state.rs comment to state the bound is PER-REPO, with the cross-repo
residual (throttled by auth + rate limits, memory bounded by the pin permit)
called out honestly.

Also correct .env.example GITLAWB_DB_MAX_CONNECTIONS (20 -> 48): with the #174 F1
boot validation, 20 is below max_concurrent_git_pushes(32)+headroom(8) and would
fail startup — closing the INV-24 doc gap from the F1 change.

Tests: pin_new_objects_gated_defers_when_pin_pool_exhausted (RED without the
permit acquire); max_concurrent_pin_tasks range/default.
2026-07-18 11:13:32 -05:00
beardthelion e489968fde fix(node): disambiguate absence from an unreadable object store in the /ipfs probe (#174 F5)
object_type_bounded treated any bare `fatal:` cat-file exit as absence. A packed
object whose pack/idx is transiently unreadable (permissions, or a mid-repack
race) emits `could not get object info` — byte-identical to a genuine miss — so a
present object could return Ok(None) and the /ipfs scan a definitive 404 instead
of the intended retryable 503. The stderr cannot separate the two (the CID is
pre-validated, so that string is also the only legitimate absence path), so the
disambiguation must be out of band.

On the collided fatal, probe object_store_readable (objects/ listable, every
pack/idx openable). Unreadable -> Err (taint -> 503). Readable -> re-probe once;
still absent -> Ok(None), now present -> the type. This narrows but does not close
the concurrent-repack TOCTOU (the check samples a different instant), stated
honestly rather than claimed deterministic.

Test: object_type_bounded_unreadable_pack_is_error_not_absence packs a real repo,
chmod 000 the pack, and asserts Err (root-guarded). RED with the readability check
disabled: the present blob returned Ok(None).
2026-07-18 10:57:16 -05:00
beardthelion c866ccb5c8 fix(node): share one whole-scan deadline across both full-scan phases (#174 F4)
fail_closed_full_scan_objects ran phase 1 (replicable_blob_set_bounded) then
phase 2 (all_blob_oids) with a FRESH Instant::now() + timeout for phase 2, under
one held scan permit. A large-but-successful phase 1 plus a full phase 2 held the
permit ~2x the configured budget instead of the one whole-scan budget.

Compute one deadline before phase 1 and share it: phase 2 takes the shared
Instant directly, phase 1 takes the remaining-until-deadline. Total occupancy is
now ~1x. The completeness cost is honest: a repo whose phase 1 nears the budget
under-pins this push (fail closed) rather than over-holding, so size the budget so
both phases normally fit.

Test: full_scan_shares_one_deadline_across_both_phases drives a slow-but-
successful phase 1 that eats the budget; the shared deadline reaps phase 2 and the
scan fails closed (empty). RED before: phase 2's fresh budget completed and kept
the non-blob candidate.
2026-07-18 10:49:18 -05:00
beardthelion fab61e4188 fix(node): bound the /ipfs cat-file probe and content read (#174 F3)
The GET /ipfs/{cid} object-type probe and content read ran bare synchronous
Command::output() on the async worker, budget-checked only before they started.
A hung or corrupt object store therefore pinned a Tokio runtime worker and both
held IPFS admission permits indefinitely, so enough requests exhausted the route
despite GITLAWB_IPFS_REQUEST_BUDGET_SECS. The walk beside them already used
spawn_blocking + the reaped bounded runner; these two stages were left bare.

Extract run_bounded_git_raw (returns ExitStatus/stdout/stderr so callers can
classify exit codes; run_bounded_git stays a thin bail-on-nonzero wrapper for the
walk callers) and add store::object_type_bounded / read_object_content_bounded on
it, preserving object_type's Ok(None)-on-absence vs Err classification so the
serve path's 404-vs-503 semantics are unchanged. Run both /ipfs stages in
spawn_blocking under a deadline clamped to min(git timeout, remaining budget),
with the IPFS permits held across the awaited spawn_blocking.

Test: get_by_cid_hung_probe_is_reaped_and_sheds_503 hangs git on a FIFO alternates
with no feeder; the reaped probe sheds a 503 in bounded time (GREEN), while a
neutralized deadline blocks the handler forever (RED). The existing probe-error /
corrupt-repo tests still assert the absence-vs-error classification at the handler.
2026-07-18 10:36:16 -05:00
beardthelion 6df972d294 fix(node): own the post-receive replication tail in a detached task (#174 F2)
The post-receive replication tail (withheld/candidate/full-scan resolution, then
the encrypt+recovery spawn and the announce spawn) ran inline in the request
future and parked on git_encrypt_semaphore before the durable try_begin gate. A
client/proxy disconnect while parked dropped the future and silently lost this
push's pins, recovery copy, and announcements, with no reconciliation sweep to
recover them (state.rs documented this exact residual).

Move the whole tail into an independently owned tokio task and return the git
response immediately, so a disconnect can no longer drop the parked work. Each
push owns its own tail, including its own always-spawned announce, so the
announce is never brought under the per-repo encrypt coalescing (which would
drop a coalesced push's per-ref announcements). Update state.rs to retire the
disconnect-loses-work residual.

Test: receive_pack_landed_push_returns_without_parking_on_scan_pool asserts the
handler returns 200 while the scan pool is held (RED with the tail inline: the
request future parks and the timeout fires; GREEN detached). The burst
serialization test is retimed to poll for scan completion since the withheld
walk is now detached too.
2026-07-18 10:14:27 -05:00
beardthelion 1b3a5f6064 fix(node): make write-acquire cancellation-safe and connection-affine (#174 F1)
acquire_write took the pg advisory lock, then awaited Tigris, then built the
guard. An outer tokio::time::timeout firing during the Tigris await dropped the
future with the lock held and no guard, so no unlock ran and later pushes to the
repo spun 60s. release() also unlocked on an arbitrary pooled connection, not the
one that locked (session advisory locks are connection-affine), so even the
normal release was unreliable.

Pin a PoolConnection in RepoWriteGuard, build the guard before the lock query,
run the lock loop through it, unlock on that same connection in release(), and
add a Drop backstop that spawns a detached affine unlock (Handle::try_current so
an off-runtime drop logs instead of panicking). Two distinct-session tests
(drop-without-release, affine release) go RED without the fix, GREEN with it.

Pinning a connection per active write is a DB-pool DoS unless the pool clears
the concurrent-write cap: the shipped default (20) was below
max_concurrent_git_pushes (32). Raise the db_max_connections default to 48 and
add Config::validate() to reject db_max_connections < max_concurrent_git_pushes
+ 8 headroom at boot; wire it into startup.
2026-07-18 09:51:10 -05:00
beardthelion 2eaf8b1086 fix(review): state the probe/read stages' unbounded-hang residual honestly; net the acquire-taint found path
The budget-knob docs claimed the overshoot was bounded by one unclamped
probe; an unclamped subprocess is not a bound, so the README,
.env.example, config help, and module comments now say plainly that a
hung git probe holds the walk slot for the hang's duration. New
regression: an acquire-tainting ghost row ahead of a healthy public copy
must not stop the scan; the later found short-circuit still serves 200.
2026-07-17 20:42:48 -05:00
beardthelion ec36e7f2d7 fix(review): close the round's review findings on the #174 fix series
object_type now distinguishes repo-level git failures (not-a-repository,
error:-prefixed corruption) from genuine object absence, so a corrupt
repo taints the /ipfs scan into the retryable 503 instead of vouching
for a definitive 404; the drain reads visibility rules by the freshly
fetched record id, closing a latent fail-open on id divergence. The four
budget gates collapse into one stage-labelled helper, the three scan
sites share state::acquire_scan_permit (inv22 tripwire repointed and
sever-verified), and the Tigris stall tests use a local accept-and-park
listener instead of a non-routable address.
2026-07-17 20:16:04 -05:00
beardthelion 79910fc169 fix(node): requeue coalesced post-push work via an in-lock pending slot and loop-drain (#174)
try_begin now merges a losing push's tip pairs into the in-flight key's
pending slot in the same critical section as the presence check, bounded
at 1024 pairs with a full-scan marker on overflow (force_full_scan
threads to the resolver; empty tips never encode the marker). The task
loop-drains: finish_or_take_pending either hands back the batch with the
key retained, or removes the key and disarms the guard atomically, so a
successor task's key survives the guard drop. Drains re-fetch rules
fresh and replay the tail pipeline under the U4 helper gates only; the
loop holds no task-level pool permit, so a hot repo cannot deadlock the
pool at size 1.
2026-07-17 19:17:17 -05:00
beardthelion 2259a248ce fix(node): admit the post-receive git scans to the encrypt pool, defer-not-shed (#174)
replication_withheld_set's walk arm, resolve_candidates_for_push's git
stages, and fail_closed_full_scan_objects each take a
git_encrypt_semaphore permit before their spawn_blocking and carry it
inside the closure, so a push burst holds at most pool-many concurrent
scans and a started scan finishes with its permit. Deletion-only pushes
and repos without path-scoped rules never park. Contention defers the
landed push's tail (logged with queue_wait_ms), never errors it; the
acquire sites document the accepted disconnect-during-park residual.
2026-07-17 18:37:40 -05:00
beardthelion 00618603ad fix(node): bound an admitted /ipfs request's total lifetime with a shared budget (#174)
GITLAWB_IPFS_REQUEST_BUDGET_SECS (default 600) is captured as one
deadline at handler entry: every acquire and expensive walk is clamped
to the remaining budget, every stage checks remaining before starting,
and exhaustion taints the scan into the retryable 503. A walk in flight
is never aborted at the tokio layer (that would free the walk permit
while the blocking thread still runs); its clamped git deadline plus
group teardown ends it. Docs state the residual overshoot: the in-flight
stage's kill/reap slack plus the unclamped probe subprocesses.
2026-07-17 18:03:49 -05:00
beardthelion d3ead53aee fix(node): make /ipfs scan verdicts honest: walk-scoped cap, visit ceiling, 503 on truncation (#174)
A request-scoped taint list replaces the silent folds: any repo skipped
without a verdict (acquire timeout, probe or read failure, walk failure,
cap or ceiling exhaustion) turns the terminal into a retryable 503 with
Retry-After, and 404 is reserved for a scan in which every candidate
reached a verdict. The walk cap now counts only expensive allowed-set
walks and skip-continues on exhaustion so a plain public copy past the
cap still serves; a new GITLAWB_IPFS_MAX_REPO_VISITS knob (default 1024)
bounds the acquire+probe cost class and stops the scan, documented with
its worst-case object-store fetch count.
2026-07-17 17:39:03 -05:00
beardthelion bda9cb102e fix(node): hold filtered-pack admission through disconnect teardown (#174)
drive_git_child hands the AdmissionGuard back on success so
upload_pack_excluding can thread it across the rev-list and pack-objects
stages; on timeout or error it is dropped post-reap, and on future-drop
it rides the detached reaper as before. The upload-pack handler builds
the guard once ahead of the withheld/plain branch, so a client
disconnect mid-filtered-serve no longer frees the read-pool and
per-source permits while the git process group is still being reaped.
2026-07-17 17:05:28 -05:00
beardthelion 1e2f0d1dd6 fix(node): close the /ipfs pin-source griefing hole via a bounded scan fallback
An attacker who pins a public object from MAX_PIN_SOURCES repos before the
legitimate public source registers fills pin_repo_sources (record_pin_source is
first-N-wins and drops later sources silently); the buried public source is then
never reached because a non-empty provenance set suppressed the legacy scan, so
anon GET /ipfs/{cid} 404s forever for a public object (re-breaks F1).

U1: db::pin_sources_at_cap reports whether pin_repo_sources is at MAX_PIN_SOURCES
for an oid (the only observable signal that a servable source may have been
dropped, since the write cap never overshoots).

U2: get_by_cid falls back to the bounded legacy scan on a provenance miss when the
set is empty OR at_cap. The scan gates every repo through the real per-caller gate,
so it finds the buried public copy. A complete (non-full) set still fast-404s, so
ordinary denials never fan out to O(repos) (INV-10/F3); the fallback honors the
is_throttled peek.

Regression ipfs_cid_buried_public_source_still_serves_via_scan_fallback: 404 before,
200 after; pin_sources_at_cap_flips_at_max covers the boundary.
2026-07-17 10:15:56 -05:00
beardthelion 107abf9145 fix(node): align ipfs_pin return with pinata; add Retry-After to SearchIncomplete
- ipfs_pin::pin_new_objects returns the provider Hash (not the raw resolver key),
  matching the pinata twin's contract; the DB cid stays the raw content CID. The
  return is logging-only here, so this is drift-avoidance, not a functional fix.
- AppError::SearchIncomplete now carries Retry-After: 1 like Overloaded, so both
  retryable 503s from the /ipfs handler advertise the retry hint consistently.
2026-07-17 10:15:56 -05:00
beardthelion bb3ecc8f30 fix(node): tie the /ipfs walk ceiling to MAX_PIN_SOURCES + 1
A provenanced object has a bounded source set (first-pinner + up to
MAX_PIN_SOURCES additional). With the walk ceiling at 16 == MAX_PIN_SOURCES, an
authorizing public source sorting after 16 path-scoped denials was never reached:
the ceiling truncated the search and returned a false 503 for a readable object.
Raise the ceiling to MAX_PIN_SOURCES + 1 so the whole bounded provenance set is
always tried; the legacy scan stays bounded by MAX_LEGACY_PROBES_PER_REQUEST.
2026-07-17 10:15:56 -05:00
beardthelion dbadd243b4 fix(node): bound the /ipfs cat-file probes under the walk permit; backfill pinata provenance
P2-E: the object-type probe and the F6 size+read ran bare git cat-file inside
spawn_blocking while the /ipfs walk permit was held, so a wedged cat-file could
pin the global walk slot for the request's life. Bound both under
git_service_timeout_secs; on timeout free the slot (truncated -> retryable 503)
and skip the repo.

P2-D: the pinata already-pinned branch now backfills NULL first-pinner
provenance in lockstep with the ipfs_pin skip branch (consistency; the object
was already resolvable via the pin_repo_sources union).
2026-07-17 10:15:56 -05:00
beardthelion 332a5785ca fix(node): close grok P1 findings on the #173/#174 IPFS integration
P1-A: gate_and_serve held the /ipfs walk permit across a bare repo_store.acquire;
wrap it in git_acquire_timeout_secs (mirroring #174's own handler) so a cold/hung
Tigris acquire cannot pin the global walk slot. On expiry skip the repo and mark
the search truncated (retryable 503, not a false 404).

P1-B: the local-IPFS pin path recorded Kubo's provider Hash as pinned_cids.cid;
for objects above the block size that is a dag-pb root that does not hash the raw
content, so GET /ipfs/{cid} listed then 404'd them under the F2 integrity check.
Record the locally-computed raw-content CID, mirroring the pinata twin.
2026-07-17 10:15:56 -05:00
beardthelion b6bc69898c feat(node): integrate the #173 IPFS CID tree-gate onto the #174 walk-concurrency base
Rebased onto #174 (fix/served-git-concurrency-cap). #173's incremental history
cannot replay commit-by-commit because #174 rewrote the same handler files in
parallel; this single commit carries the fully-integrated tree (identical to the
verified merge result), with the follow-up fixes as separate commits on top.

Brings in #173: GET /ipfs/{cid} CID->oid resolution via pinned_cids, per-caller
path-scoped blob/tree/commit-tag gating (#135/#173 F1-F6), and the pin-source
provenance table. Adapts #173's tree/commit-tag walks to #174's run_bounded_git
so the held /ipfs walk-concurrency permit is duration-safe (F5).
2026-07-17 10:15:56 -05:00
beardthelion cdc62a703a fix(node): reject unsupported git service before the read slot; document the new admission knobs (#174)
P2-1: git_info_refs treated every service other than git-receive-pack as a read op
and did the DB/visibility/Tigris work under a read permit, validating the service
string only downstream in smart_http. So an unauthenticated `?service=anything` to
a public repo consumed a read slot + storage work before rejection. Validate the
service is exactly git-upload-pack or git-receive-pack immediately after parsing,
returning 400 before the pre-DB shed and the permit. Handler-layer regression:
with the read pool exhausted, `?service=git-explode` returns 400 (validated first),
not 503 — removing the validation makes it 503 (RED).

U5/INV-24 docs: document the new knobs from U2/U3 in .env.example and the README
env-var table — GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS, GITLAWB_MAX_CONCURRENT_IPFS_WALKS,
GITLAWB_IPFS_WALK_PER_SOURCE, GITLAWB_IPFS_MAX_REPOS_WALKED, GITLAWB_IPFS_RATE_LIMIT.
Each is already enforced on the path it names (U2/U3 tests), so the docs match the
implemented routing.
2026-07-15 13:16:40 -05:00
beardthelion 98ebb47216 fix(node): bound the post-push encryption task set by per-repo coalescing without dropping recovery copies (#174)
After a successful path-scoped push the handler spawns a DETACHED task that runs
pin_new_objects then withheld_recipients_gated, which acquire_owned().await's on
git_encrypt_semaphore — it DEFERS (blocks) when the pool is full. The semaphore
caps active walks but nothing capped how many detached tasks were spawned and
PARKED on that await: N rapid path-scoped pushes spawned N tasks, each holding
cloned object lists/rules/paths/keys — an unbounded parked-waiter set (P2-2).

2a54c15 deliberately chose defer-not-shed because dropping the walk loses the
withheld-blob recovery copy and there is no reconciliation sweep to rebuild it.
So bound the OUTSTANDING-TASK set by per-repo coalescing rather than shedding:
EncryptInflight (a bounded Arc<Mutex<HashSet<repo_id>>>) + try_begin — before the
spawn, if a task for the repo is already in-flight skip the duplicate (the pending
walk covers the newer objects); otherwise spawn and move an RAII
EncryptInflightGuard into the task that removes the key on drop (completion, error,
or panic-unwind). This bounds the set to <=1 pending task per repo and never drops
work; withheld_recipients_gated's defer is unchanged.

The second detached post-push spawn (Pinata replication, also runs pin_new_objects)
is scoped out with rationale: it parks on no semaphore (no unbounded waiter set)
and does per-push per-ref work (branch->CID, gossip, subscription, Arweave,
peer-notify) keyed to this push's ref_updates — coalescing it would DROP a later
push's announcements, a correctness regression.

Regressions (mutation-verified RED->GREEN): the outstanding set is bounded to 1
per repo under saturation (never-coalesce -> 32 tasks RED); a coalesced repo's key
is released when its task ends so it is reprocessed, never permanently skipped
(guard-drop no-op -> repo locked out, recovery copy lost forever, RED — the exact
durability regression the fix prevents); distinct repos each admit; cold-set first
push admits; plus an inv22_gates structural tripwire that fails if the handler gate
is removed.
2026-07-15 13:10:04 -05:00
beardthelion 7c23a66da9 fix(node): gate the /ipfs walk with bounded concurrency admission + per-source cap + route rate limit (#174)
GET /ipfs/{cid} (get_by_cid, publicly reachable via optional_signature) ran
allowed_blob_set_for_caller_bounded in spawn_blocking inside a per-repo loop with
NO global/per-source admission and NO route rate limit — the surface had zero
concurrency or rate control. A permissionless caller could fan out concurrent
full-history git walks (each up to the walk timeout) exhausting blocking-pool
threads and PIDs outside every served-git pool (P1-3).

Acquire a global git_ipfs_walk permit (try_acquire_owned, shed 503) plus a
per-source sub-cap (with_default_max_keys, reject-before-insert, keyed via
client_key/TrustedProxy so XFF can't spoof it) ONCE after CID validation, held as
handler locals across the whole request — including every spawn_blocking walk, so
the slot reflects real blocking-thread occupancy (a tokio timeout cannot cancel
spawn_blocking). None key -> global pool only. Cap repos walked per request so one
request can't serialize N full-history walks. Attach rate_limit_by_ip +
IpRateLimiter to the /ipfs route (mirrors the git/create/peer routers; the
extension is required or the middleware is a silent no-op). Visibility/authz
semantics unchanged.

New knobs: GITLAWB_MAX_CONCURRENT_IPFS_WALKS (32), GITLAWB_IPFS_WALK_PER_SOURCE
(4), GITLAWB_IPFS_MAX_REPOS_WALKED (64), GITLAWB_IPFS_RATE_LIMIT (600/hr, 0
disables) — all range-validated and enforced on-path.

Handler-layer regressions (all mutation-verified RED->GREEN): shed-at-capacity
503 (delete acquire -> falls through), per-source cap sheds same source / admits
another, None-key arm sheds on the global pool, map self-bounds reject-before-insert,
the walk permit is held while the spawn_blocking walk runs (available_permits()==0;
dropping it before the loop -> RED), repos-walked cap (remove break -> 2 walks),
and the route IP rate limit actually fires 429 (drop the Extension -> RED no-op).
2026-07-15 12:42:55 -05:00
beardthelion 90bd9a6270 fix(node): bound the served-git storage-acquisition phase with a release-on-expiry deadline (#174)
The served-git admission permit is acquired, then RepoStore::acquire*/advisory-lock
runs before the bounded git runner starts — with no local deadline. acquire/acquire_fresh
await Tigris HEAD/GET, and acquire_write's advisory-lock loop awaits a per-iteration
pg_try_advisory_lock that can block indefinitely on a hung Postgres pool. Since
GITLAWB_GIT_SERVICE_TIMEOUT_SECS only starts once git spawns, a stalled backend pins
the permit and drains the pool until every later request 503s (P1-2).

Wrap each git-path acquire (git_info_refs read/advert, git_upload_pack, git_receive_pack)
and the /ipfs per-repo acquire loop in tokio::time::timeout(git_acquire_timeout_secs, ..).
On expiry the git paths return AppError::Overloaded (503 + Retry-After); the permit is a
handler-local at that point (moved into the AdmissionGuard only after acquire) so the early
return frees the slot. The /ipfs per-repo acquire keeps its fail-closed `continue` on
timeout (never serves an un-acquired repo). No repo_store.rs signature change: the outer
timeout cancels a mid-sleep/mid-fetch_one future, so no in-loop deadline is needed.

New knob GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS (default 30, range 1..), modeled on
git_service_timeout_secs, kept separate because acquisition and git execution are distinct
cost centers.

Handler-layer regression receive_pack_acquire_deadline_sheds_and_releases_permit: holds the
same pg advisory lock acquire_write derives on a second connection so the real loop must
retry, drives git-receive-pack with the deadline at 2s and write pool size 1, and asserts
503 at ~2s + the permit recovers + a follow-up admits once the lock frees. Removing the
timeout wrapper turns it RED (blocks ~31s to the test ceiling on the ~59s advisory loop).
2026-07-15 12:00:02 -05:00
beardthelion d5de423393 fix(node): hold served-git admission until the process group is reaped on the plain spawn paths (#174)
On the plain (non-path-scoped) info_refs / upload-pack / receive-pack paths the
global + per-source admission permits were handler-locals that dropped the
instant the handler future dropped on a client disconnect, while KillGroupOnDrop
reaps the git process group in a DETACHED task (SIGTERM -> grace -> SIGKILL ->
reap). So a disconnect-spammer admitted replacements while prior groups were
still alive, defeating the process cap and the per-source cap during teardown.
be0cdd6 already fixed this for the path-scoped upload-pack walk; this closes the
residual plain paths (P1-a).

Introduce AdmissionGuard, a move-only holder of the permits, threaded through
info_refs/upload_pack/receive_pack/run_git_service into drive_git_child's
KillGroupOnDrop. On disconnect the guard moves into the detached reaper and drops
only after the group is ESRCH-confirmed reaped; on success/timeout disarm()
returns it for the earliest provably-free drop. The rev-list/pack-objects
visibility-walk callers hold no admission and pass None.

Thread git_bin through upload_pack/receive_pack (they hardcoded "git") so the
plain POST path is drivable by an injected fake git, matching the existing
injectable seam on info_refs/run_git_service/upload_pack_excluding.

Handler-layer regression upload_pack_plain_permit_held_through_group_reap_after_disconnect:
isolates the global read pool (size 1, per-source + rate limiter permissive), a
SIGTERM-ignoring descendant keeps the group alive across the reap window, and
asserts available_permits()==0 held on disconnect + a cross-source replacement
sheds 503, then frees after ESRCH. Reverting the guard-into-reaper move (release
immediately) turns it RED at the held-through-reap assertion. Plus a None-key arm.
2026-07-15 12:00:02 -05:00
beardthelion ca7a7074dd test(node): INV-22 completeness guard for the served-git concurrency surface (#174)
A source-scan tripwire that fails when a new site reintroduces the concurrency-cap
class PR #174 fixed, or when one of the five gates (U1-U5) is removed. INV-22: a
permit held per op recovers only if every path is duration-bounded and reaps the
group before releasing admission, and every detached git task carries admission.

Lives in tests/ (a separate crate) on purpose — a guard scanning the file it lives
in would match its own identifier literals and pass vacuously; scanning src/ from
here keeps every check load-bearing. Checks: run_bounded_git confirms child exit via
child_terminated_without_reaping (U1); KillGroupOnDrop launches the reaper via
Handle::try_current (U2); git_receive_pack acquires state.git_write_per_caller (U4);
the encryption walk runs through withheld_recipients_gated over git_encrypt_semaphore
(U5); and the bounded recipients walk is invoked exactly once — only inside the gated
helper — so a new spawn_blocking bypass (count > 1) fails (P1-e non-bypass).

Proven load-bearing: a synthetic second call site turns the count tripwire RED, and
each named gate maps to a fix whose own regression already goes RED on revert.
2026-07-14 20:33:01 -05:00
beardthelion 68043f5844 fix(node): bound concurrent post-push encryption walks with an admission pool (#174)
After a successful path-scoped push the handler released its write permit and
then ran a DETACHED tokio task whose spawn_blocking(withheld_blob_recipients_bounded)
performed another full-history git walk under no admission. N fast completed
pushes spawned N concurrent full-history walks past GITLAWB_MAX_CONCURRENT_GIT_PUSHES
(which bounds only the in-handler phase), and the per-child bounding from 71389e6
caps each walk's duration but not their count (P1-e).

Add git_encrypt_semaphore, a pool of its own (sized from max_concurrent_git_pushes,
no new knob — Q1) so a long background walk never holds a foreground write slot and
a handler holding a write permit can't self-deadlock. Route the walk through a new
withheld_recipients_gated helper that acquires the pool before the walk and DEFERS
(blocks) when full rather than shedding — dropping the walk would lose the
withheld-blob recovery copy, so durability stays fail-closed.

Regression: encrypt_walk_defers_when_pool_exhausted drives the gated helper with the
pool exhausted and asserts the walk blocks and does not run its rev-list, then runs
once a permit frees. RED before (the walk runs regardless of the pool), GREEN after.
The detached push task calls this exact helper.
2026-07-14 20:28:52 -05:00
beardthelion f592378bdc fix(node): cap concurrent receive-pack pushes per source IP (#174)
The authenticated git-receive-pack POST acquired only the global write semaphore
— no per-source sub-cap, unlike the read and anon-advert pools. Owner enforcement
defaults off, so one host minting disposable did:key identities could open
max_concurrent_git_pushes slow POSTs from a single source IP and 503 every other
source's push; the 600/hour push limiter bounds arrival rate, not in-flight
concurrency (P1-d).

Add git_write_per_caller (a PerCallerConcurrency sized like the advert cap,
max_concurrent_git_pushes/8), and acquire it before the global write permit,
keyed on the resolved source IP via read_caller_key — never the signed DID (a DID
farm defeats a DID key). This required adding the PeerAddr + HeaderMap extractors
to git_receive_pack, which it lacked (without them the key is None and the cap is
inert). Neutralized the shared acquire helper's log wording now that it serves
both read and write paths.

Regression: receive_pack_per_source_write_cap_sheds_capped_source_not_others — a
source at its write sub-cap sheds Overloaded/503 (proving the extractors resolve a
key), while a different source is not shed. RED before (the capped source proceeds
past admission to a git error instead of shedding), GREEN after; the existing
receive-pack advert cap test still passes.
2026-07-14 20:20:49 -05:00
beardthelion 0c767d1585 fix(node): hold upload-pack admission through the walk on disconnect (#174)
On the path-scoped upload-pack path the per-source and global read permits were
handler locals, so a client disconnect dropped the handler future and released
both permits while the awaited spawn_blocking withheld-blob walk kept running
(spawn_blocking can't be cancelled). A permitless caller could disconnect-spam to
exceed the read cap while real git work continued (P1-b).

Move both permits into the blocking task and return them out: on success they
flow back so the serve phase keeps them; on a dropped future the returned tuple
(with the permits) is discarded only when the blocking task completes, so
admission tracks the walk's real duration.

Regression: upload_pack_permit_held_through_walk_after_disconnect drives the
handler to mid-walk (fake git hangs on rev-list), disconnects, and asserts the
global read slot stays held (available_permits == 0). RED before (the slot frees
to 1 the instant the future drops), GREEN after. The test roots the repo store at
its own TempDir so it is isolated per run.
2026-07-14 20:09:40 -05:00
beardthelion b900151468 fix(node): reap the served-git process group on disconnect, not just SIGTERM (#174)
On a client disconnect KillGroupOnDrop sent a lone SIGTERM with no escalation or
reap, so a group member that ignores SIGTERM (a wedged pack-objects, a malicious
helper) survived while the handler released its concurrency permit — letting
disconnect-spam accumulate live git processes and defeat the load shed (P1-c).
The timeout path already ran the full teardown.

Make the disconnect teardown as strong as the timeout path: the guard now owns
the child and, on drop, launches a detached reaper that runs the same
TERM -> grace -> SIGKILL -> reap-the-group sequence (reap_group_on_timeout).
Owning the tokio Child in the reaper keeps it the sole reaper of the leader, so
it never races tokio's own orphan reaper (a raw waitpid(-pgid) would).

Regression: run_git_service_sigkills_a_sigterm_ignoring_child_on_disconnect, RED
before (the descendant survives the lone SIGTERM), GREEN after; all existing
teardown/timeout/disarm tests still pass (the three hand-built-guard tests were
updated to the owned-child shape).

Residual: capacity (the handler's semaphore permit) still releases at disconnect
rather than strictly after the reaper confirms the group dead, so accumulation is
now bounded to ~one teardown window (<=4s) rather than eliminated; gating the
permit on the reap needs threading it through the serve handlers and is left for
a follow-up (it also touches the receive-pack permit lifetime U4 changes).
2026-07-14 19:50:41 -05:00
beardthelion dd3672ef0e fix(node): keep the visibility-walk watchdog armed until the child is reaped (#174)
A group member (or the leader itself) that closes stdout before the deadline made
the stdout drain return EOF early, which stood the watchdog down via done_tx
before it could ever fire; child.wait() then blocked on the still-live child,
pinning the walk thread and its read/write permit past
GITLAWB_GIT_SERVICE_TIMEOUT_SECS. This is distinct from the descendant case
already covered: there the leader sleeps until the deadline so the watchdog does
time out; here the drain-EOF races ahead of it.

Stand the watchdog down only once the child has actually terminated, detected
without reaping (waitid + WNOWAIT) so the leader's pid stays unreaped and its
pgid un-recycled until the watchdog finishes its teardown and is joined. Past the
deadline the watchdog still owns the full SIGTERM -> grace -> SIGKILL sequence.

Regression: run_bounded_git_reaps_a_leader_that_closes_stdout_then_hangs, RED
before (blocks on child.wait past the recv budget), GREEN after; the three
existing teardown tests still pass (no recycle-hazard regression).
2026-07-14 19:27:12 -05:00
beardthelion ec201effb6 test(node): retry the ETXTBSY exec race in build_filtered_pack timeout tests (#174)
build_filtered_pack_times_out_a_hung_rev_list and _a_hung_pack_objects
spawn a freshly-written fake git directly, bypassing the fake_git_run_with_pids
retry the rest of the module uses. Under cargo test fork-storm load a concurrent
worker forks while the fake's write fd is still open, so execve returns ETXTBSY
and the raw io::Error surfaces where the test asserts GitServiceTimeout, failing
test (stable) intermittently.

Route both tests through build_filtered_pack_or_exec_race_retry, which retries
ONLY errno 26 (ETXTBSY) and keeps the outer 10s watchdog per attempt, so a
genuinely-missing async timeout bound still trips the watchdog and a wrong error
type still fails at the assertion. Production paths unchanged.
2026-07-13 16:01:49 -05:00
beardthelion 7671df98c7 test(node): execution-cover the delta-path push-scan timeout wiring (#174)
The finding-2 wiring test covered only list_all_objects (the full-scan fallback).
Add delta_path_exec_fns_time_out_on_a_hung_git covering the common push path:
peeled_object_type (cat-file), rev_list_delta (rev-list), and list_all_objects_with_type
(cat-file, which all_blob_oids delegates to). Each must return within the watchdog
budget on a hung git; reverting any to a bare Command::output() blocks past the recv
budget (RED-verified).
2026-07-13 10:55:57 -05:00
beardthelion 1e1058ca29 docs(node): correct concurrency + timeout docs to match the final git routing (#174)
jatmn P2 (INV-24): the config help, .env.example, and README described routing the
handler no longer does.

- The read pool serves upload-pack and the UPLOAD-PACK info/refs advertisement only;
  the anon receive-pack info/refs advertisement draws from its own dedicated pool
  (sized like the write pool but disjoint), not the read pool and not the write pool.
- The write pool is the authenticated push POST only.
- The per-caller read cap is keyed on the resolved source IP, never the DID: a
  signature does not move a caller off it (the docs claimed a per-DID budget, which
  would tell a NATed authenticated client it escapes the shared cap when it does not).
- git_service_timeout_secs now also bounds the push-side pin-candidate discovery
  (rev-list / cat-file), reaped via process-group teardown, alongside the info/refs
  advertisements and the withheld-blob walk.
2026-07-13 10:46:24 -05:00
beardthelion 5aa37a17a0 fix(node): bound and reap push-candidate git children under the write permit (#174)
Route push_delta's candidate-discovery git children (cat-file / rev-list) through the
bounded run_bounded_git (deadline + process-group SIGKILL teardown) instead of bare
Command::output(). A hung scan no longer pins the git-receive-pack write permit past
the deadline or a client disconnect. Threads git_bin + a shared per-scan deadline
(git_service_timeout_secs) through resolve_candidates_for_push and all_blob_oids.
2026-07-13 10:41:31 -05:00
beardthelion 39fab32e91 fix(node): SIGKILL a SIGTERM-ignoring member of the visibility-walk process group (#174)
jatmn/CodeRabbit P1: the walk watchdog stood down as soon as the group LEADER was
reaped. A leader that exits cleanly on SIGTERM while a background member ignores
SIGTERM and closes its inherited pipes let the main drain unblock, the leader be
reaped, and the watchdog skip its SIGKILL escalation — so the handler returned while
the descendant kept running. jatmn's suggested "continue until ESRCH" could not work:
the main thread reaping the leader is what frees the pgid, so continuing to kill(-pgid)
after that races a recycled group.

Restructure the coordination instead: the main thread now defers reaping the leader
until the watchdog thread returns. The watchdog always escalates through an
unconditional group SIGKILL (no leader-reap short-circuit), and because the leader is
still unreaped while every kill(-pgid) fires, the pgid cannot have been recycled. The
grace before SIGKILL is a fixed budget (only paid on the exceptional timeout path).

Regression run_bounded_git_sigkills_a_sigterm_ignoring_descendant_after_leader_exits:
a leader that exits 0 on SIGTERM spawns an sh -c descendant (its own $$, not a ( )
subshell's parent pid) that ignores SIGTERM and closes its pipes; the descendant must
be dead when the runner returns. RED on the old teardown (survives), GREEN now. The
existing leader-ignores-SIGTERM and hung-walk teardown tests still pass.
2026-07-13 10:11:41 -05:00
beardthelion 4f2a36fcb4 fix(node): compile the bounded visibility git runner on non-Unix targets (#174)
jatmn P1: run_bounded_git was gated #[cfg(unix)], but its callers
(assert_all_refs_are_commits, blob_paths) and the module are compiled on all
targets, so a non-Unix build (the continue-on-error Windows release target) could
not resolve the function.

Add a #[cfg(not(unix))] twin with the same signature and result semantics. It bounds
a single child (no process-group teardown, which is Unix-only): threads feed stdin
and drain stderr, the main thread drains stdout, and a watchdog thread kills the
child at the deadline. The child is shared with the watchdog behind a mutex the main
thread does not hold while draining, so the watchdog can always acquire it to kill,
and killing closes stdout to unblock the drain. Best-effort (reaps only the direct
child), which is why the hardened group-aware path stays Unix-only.

Verified the twin type-checks by compiling its (portable-std) body on the native
target; the full non-Unix link is CI's Windows job.
2026-07-13 09:49:33 -05:00
beardthelion c5d19df8d6 fix(node): give the receive-pack advert its own pool so it can't shed authed pushes (#174)
jatmn P1: the anon-reachable GET info/refs?service=git-receive-pack drew from the
same git_write_semaphore as the authenticated git-receive-pack POST. The per-source
advert cap reserves no global slots, so ~8 sources (each within the per-IP push rate
limit) could hold all 32 write permits across acquire_fresh + the advertisement, and
the next authenticated push then 503s at admission. That is an anonymous caller
shedding an authenticated push across the auth boundary (INV-10).

Add a dedicated git_push_advert_semaphore, disjoint from the write pool. Both the
pre-DB early-shed peek and the held acquire for the receive-pack advertisement now
draw from it; the write pool is left exclusively for the POST. An advert flood can at
worst exhaust the advert pool (each source still capped by git_push_advert_per_caller
and the per-IP push rate limiter); the authenticated push pool is untouched.

Test receive_pack_advertisement_draws_from_dedicated_advert_pool proves at the handler
layer that the advert sheds when its own pool is saturated AND survives when the write
pool is saturated (the reservation). Both flip under revert, verified independently for
the pre-DB peek and the held acquire.
2026-07-13 09:44:53 -05:00
beardthelion e4c14ccd01 fix(node): shed the served-git pool before the DB and cap per-source before it (#174)
CodeRabbit review: the held git_permit was acquired at the top of git_info_refs
and git_upload_pack, before the per-source cap, so one source could occupy the
global pool during the DB/visibility window before its sub-cap rejected the
excess. Move the held git_permit to after acquire_read_caller_permit (still
before acquire_fresh/git, so INV-10's bound on the fresh Tigris acquire holds),
and add a cheap pre-DB early shed (available_permits() == 0 -> 503, holds no
permit) so the #62 shed-before-DB property is preserved without holding a permit
across the DB work.

The receive-pack POST is unchanged (owner-only; its top-of-handler git_permit is
deliberate for acquire_write bounding).

Tests: the two #62 shed-before-DB tests now exercise the explicit early check
(load-bearing: disable it and they fall through to the DB); three new
*_per_source_cap_sheds_with_global_capacity tests prove the per-source sub-cap
sheds a capped source even with global capacity free (load-bearing on the caller
cap). Full workspace green.
2026-07-12 14:40:59 -05:00
beardthelion a4beba2eea docs(node): the withheld-walk is bounded by the service timeout, not a fixed const (#174)
Follow-up to threading the configured timeout into the walk: the walk now honors
GITLAWB_GIT_SERVICE_TIMEOUT_SECS on both the serve and replication paths, so the
README no longer says a fixed internal deadline.
2026-07-12 02:12:37 -05:00
beardthelion b2939ad990 test(node): vet the withheld-walk bound end to end; use the configured timeout (#174)
Close the reasoned-not-run gaps from the code review by making the walk's git
binary and timeout injectable, then driving the missing branches with a real
handler and a fake git instead of reasoning about them.

- Add state.git_bin and *_bounded variants of the walk entry points taking
  (git_bin, timeout); the served handlers (upload-pack serve, receive-pack
  replication and full-scan and encrypt-pin, and the ipfs gate) now pass the
  operator-configured GITLAWB_GIT_SERVICE_TIMEOUT_SECS, so the whole walk is bounded
  by the same budget as the other served-git ops rather than a fixed constant. The
  git_bin-less wrappers stay for the real-git classification tests.

Newly vetted by execution (not reasoning):
- receive-pack replication path is bounded: replication_withheld_set with an injected
  hung git returns within the budget and fails closed, so it cannot pin the write
  permit git_receive_pack holds across it.
- a hung withheld-blob walk on the upload-pack POST returns 504 (real handler, real
  repo on disk, injected hung git), proving the GitServiceTimeout -> git_service_app_error
  wiring end to end.
- the watchdog status-gate: a child that exits successfully is not reported as a
  timeout even when the watchdog fired (mutation-checked: drop the guard -> RED).
- SIGKILL escalation: a SIGTERM-ignoring child is still reaped via SIGKILL and the
  group is gone; a truly uninterruptible D-state child (unreapable by any signal) is
  the documented residual, matching the async teardown.
- the advertisement per-source cap sizing never derives 0.

Full gitlawb-node suite 515 green.
2026-07-12 02:12:08 -05:00
beardthelion 6c7bbaa193 fix(review): cap the receive-pack advertisement per source so anon cannot starve the write pool (#174)
U2 moved the receive-pack info/refs advertisement onto git_write_semaphore to keep
an anonymous read flood from starving the push handshake. But the advertisement is
anon-reachable on public repos and holds its write permit across the slow
acquire_fresh Tigris download, and the only per-source brake on it was the push
RATE limiter, not a concurrency cap. So a multi-source flood of receive-pack
advertisements could hold the write pool's slots across those downloads and shed
authenticated pushes (both the advertisement and the owner-gated git-receive-pack
POST draw from the same pool). U2 thus introduced the first anonymous consumer of
the write pool the state doc promised anon could never reach; the plan's residual
note (no worse than the POST) was wrong, because the POST is owner-gated and the
advertisement is not.

Add git_push_advert_per_caller, a per-source concurrency sub-cap on the receive-pack
advertisement keyed on the resolved source IP (the same PerCallerConcurrency
mechanism U1 uses for reads), sized to an eighth of the write pool so a single
source holds at most that share and saturating the pool takes many distinct source
IPs, each also braked by the per-IP push rate limiter. The upload-pack advertisement
keeps its read-pool per-caller cap; the owner-gated POST is unchanged. Correct the
state doc for git_write_semaphore accordingly.

Handler-layer proof: a source at its receive-pack advertisement cap sheds 503 (RED
before the acquisition, 500-not-503), while a different source and the upload-pack
advertisement are unaffected. Full suite 510 green.
2026-07-12 00:56:24 -05:00
beardthelion 39e1f12255 fix(review): close the withheld-walk watchdog reused-pgid race (#174)
Code review found run_bounded_git's watchdog could return a spurious 504 and
signal a recycled process group. The watchdog runs off a wall clock on its own
thread; done_tx.send() only fires after child.wait() reaps the leader, so a walk
that finished within microseconds of the deadline took the watchdog's Timeout
branch, discarded a fully-captured successful result, and returned GitServiceTimeout
(a 504 for a walk that actually completed). Worse, the Timeout branch SIGTERMed
-pgid unconditionally after the leader was reaped, so a recycled pgid could be
signalled, the exact hazard smart_http guards via disarm-after-wait.

Set a reaped AtomicBool the instant the main thread reaps the child; the watchdog
checks it before every kill and stands down if the leader is already reaped. Gate
the timeout verdict on !status.success(), so a child that exited on its own is never
reported as a timeout even if the watchdog fired late. Add the survived-SIGKILL warn
smart_http's reap already emits, for operator visibility on a wedged (D-state) git.

The hung-walk test stays green (a killed child exits by signal, not success, so it
still surfaces GitServiceTimeout and reaps the group) and the 32 real-git
classification tests stay green (a fast walk is never spuriously killed).
2026-07-12 00:36:42 -05:00
beardthelion 87b6d2d498 docs(node): correct the git-service-timeout coverage note (#174)
GITLAWB_GIT_SERVICE_TIMEOUT_SECS bounds the info/refs advertisement too:
smart_http::info_refs drives it through drive_git_child under this timeout, with a
passing test proving the 504. The old note claimed it does not. It also claimed the
withheld-blob path is unbounded; after the blob_paths seam bound (this PR) the walk
is bounded and reaped, by a fixed internal deadline rather than this env var, so the
line now states that precisely instead of overclaiming this setting covers it.
2026-07-12 00:20:07 -05:00
beardthelion 5204502678 fix(node): bound the withheld-blob walk at the shared blob_paths seam (#174)
The withheld-blob classification walk (blob_paths) fanned out blocking git children
with no deadline and no process-group teardown: git for-each-ref, git cat-file, git
rev-list, a git ls-tree per commit, and an uncounted git rev-parse (via
store::head_commit). A hung or pathologically slow child pinned the caller's
served-git permit for the whole hang, and on client disconnect the spawn_blocking
task and its git children ran on, orphaned. blob_paths is the shared core of five
callers: the upload-pack serve path (holds a read permit) AND, inside
git_receive_pack, the post-push replication and encrypt-then-pin walks (hold the
write permit U2 reserves for pushes). So the same unbounded walk could pin either
pool, and leaving the write-side twin unbounded would have made U2's reservation a
claim that does not match behavior.

Bound every git child at the blob_paths spawn seam on the blocking side: each child
runs in its own process group with a watchdog thread that SIGTERMs (then SIGKILLs)
the group on one shared deadline spanning the whole walk, and retains admission
until the group is reaped. This is the blocking-side counterpart of
smart_http::drive_git_child (spawn_blocking cannot be cancelled by an async
timeout). blob_paths stays sync, so all five callers keep their signatures and the
32 classification tests are unchanged; because every caller funnels through
blob_paths, one seam bounds both the serve and replication paths. The previously
unbounded store::head_commit child becomes a bounded git rev-parse inside the walk.
A walk that hits its deadline carries GitServiceTimeout, which the serve handler now
maps to 504 rather than a generic 500.

Proof: a fake git that hangs on rev-list makes blob_paths return GitServiceTimeout
within the watchdog budget (not block on the child) and the recorded process-group
leader is reaped, not orphaned; neutralizing the watchdog kill makes it hang past
the budget (RED). The 32 real-git classification tests stay green through the
refactor, including detached-HEAD, non-standard-ref, and deleted-in-history cases.
2026-07-12 00:20:07 -05:00
beardthelion d0f96684ed fix(node): draw the receive-pack advertisement from the write pool (#174)
git_info_refs acquired git_read_semaphore for BOTH services, so the push handshake
(GET /info/refs?service=git-receive-pack) competed in the global read pool. An
anonymous clone flood could exhaust that pool and shed a legitimate push with 503
during its required advertisement phase, before it ever reached git_write_semaphore
on the POST. The write pool exists precisely so anonymous reads cannot shed an
authenticated push, but only the POST drew from it.

Select the pool by service: the receive-pack advertisement (phase one of a push)
now draws from git_write_semaphore, like the git-receive-pack POST, so a saturated
read pool cannot starve it. The per-IP push_rate_limiter that already brakes the
advertisement stays as the anti-flood control, and the advertisement stays
reader-visible with no new auth requirement. Because the receive-pack branch is now
a write-path op, it no longer consumes a read per-caller slot.

Handler-layer proofs: with the read pool at zero the receive-pack advertisement
survives while the upload-pack advertisement sheds; with the write pool at zero the
receive-pack advertisement sheds while upload-pack is unaffected; and a receive-pack
advertisement from an IP whose read per-caller budget is full still gets through
(mutation-checked, RED when the skip is neutralized).
2026-07-11 23:54:21 -05:00
beardthelion 45408309f8 fix(node): key the per-caller read cap on source IP, not the signed DID (#174)
read_caller_key returned the authenticated DID when a caller signed, dropping the
source-IP key. Public read routes accept any valid did:key via optional_signature
with no admission step, so one host could mint N disposable DIDs and hold
max_concurrent_reads_per_caller slots under each, multiplying its budget N-fold and
filling the global read pool, while the same host unsigned was capped on its IP.

Key the read sub-cap on the resolved source IP for every caller, signed or not,
mirroring the push path's IpRateLimiter which already throttles on source IP for
this exact DID-farm reason. Drops the now-unused caller_did parameter at both call
sites (git_info_refs, git_upload_pack).

Inverts info_refs_per_caller_cap_keys_on_did_not_ip into
info_refs_per_caller_cap_keys_on_ip_not_did: fill one source IP's slot, then two
requests signed under different DIDs from that same IP both shed 503 (farm
defeated), while a signed request from a different IP keeps its own budget. RED on
the DID-keyed tree, GREEN after.
2026-07-11 23:44:13 -05:00
beardthelion 8816684c71 fix(node,git): bound the rev-list stage under drive_git_child so a hung enumeration can't pin a git permit (#174) 2026-07-11 13:02:24 -05:00
beardthelion 6176fbb697 docs(config): add GITLAWB_MAX_CONCURRENT_GIT_OPS to .env.example (#174)
The read-pool knob was referenced by the push and per-caller entries'
comments but had no example line of its own, so operators couldn't
discover it from the template. Add it with the config default (128).
2026-07-10 16:05:29 -05:00
beardthelion 08fb3a02df fix(node,concurrency): resolve #174 code-review findings
Review of the served-git concurrency cap found no P0/P1; these are the
verified P2 follow-ups.

config: the max_concurrent_git_ops doc overclaimed that "every capped path
is duration-bounded." The rev-list object enumeration in the withheld-blob
path still runs in an uncancellable spawn_blocking, so a stuck rev-list can
hold its slot until git exits. Scope the guarantee to the streaming stages
and name the residual. Also tighten the fairness claim: the receive-pack
advertisement shares the read pool (a shed advertisement is a cheap retryable
GET); only the push POST is on the isolated write pool.

api/repos: add info_refs_per_caller_cap_keys_on_did_not_ip, the missing
handler proof that a signed caller is keyed by its DID, not its source IP.
Filling the DID slot sheds a request from a free IP; collapsing read_caller_key
to its IP arm turns the assertion green-not-503 (mutation-verified RED).

api/repos: extract acquire_read_caller_permit so both read handlers share one
shed path instead of a duplicated match block.

rate_limit: recover from a poisoned PerCallerConcurrency mutex instead of
panicking. The critical section is pure counter arithmetic and cannot poison
the lock, but a panic there would brick the limiter for every caller.

505 tests pass; clippy -D warnings and fmt clean.

Part of #174.
2026-07-10 13:45:38 -05:00
beardthelion b519251e00 docs(node,config): reconcile the concurrency-cap comments with the fix (#62)
The max_concurrent_git_ops and git_service_timeout_secs doc-comments (and .env.example) described the info/refs and withheld-blob paths as unbounded follow-up gaps. Both are now closed (#174): the comments reflect the read/write pool split, the per-caller sub-cap, and that every capped path is duration-bounded with process-group teardown. Verified the pattern-doc pre-ship checklist: every git_permit / write-permit / per-caller site holds only a timeout+teardown git path, and all three size knobs are range(1..=1_048_576).

Closes the #174 work. No behavior change.
2026-07-10 13:20:18 -05:00
beardthelion 0fdaaef3f7 fix(node,git): bound and reap the withheld-blob pack build (#62)
The filtered-pack path ran the whole rev-list + pack-objects build inside a spawn_blocking, so an outer tokio timeout could not cancel the blocking thread and a client disconnect orphaned the git child while the permit freed (jatmn P1b, the second gap path). Split it: rev-list enumeration stays blocking off the runtime (rev_list_keep), but the streaming pack-objects stage now runs under the shared drive_git_child on the async side, so it is duration-bounded (GitServiceTimeout -> 504) and its process group is reaped on disconnect. build_filtered_pack becomes async and takes a git_bin seam + timeout; upload_pack_excluding threads the git_service_timeout through.

A watchdog-bounded test proves a hung pack-objects times out (rev-list fast, pack-objects hangs). The refactor's happy path is covered by the existing filtered-pack correctness and real-git partial-clone/fetch tests, all still green; disconnect/group-teardown is the shared drive_git_child code proven by the run_git_service tests. 504 tests pass.

Part of #174.
2026-07-10 13:16:41 -05:00
beardthelion 9ca626e415 fix(node,git): bound and reap the info/refs advertisement (#62)
info_refs ran a bare Command::output() with no timeout and no process-group teardown, so a hung git pinned its concurrency slot indefinitely and a client disconnect orphaned the child (jatmn P1b). Extract the timeout + process_group(0) + KillGroupOnDrop core from run_git_service into a shared drive_git_child, and route info_refs through it with an injectable git_bin. A hung advertisement now aborts with GitServiceTimeout (mapped to 504); disconnect reaps the group.

run_git_service's teardown tests all pass through the shared core (proving the group teardown info_refs inherits), the real-git filter tests cover the advertisement happy path, and a new watchdog-bounded test proves a hung advertisement times out. 503 tests pass.

Part of #174.
2026-07-10 13:09:34 -05:00
beardthelion 222fb570a3 fix(node,git): per-caller concurrency sub-cap on the read pool (#62)
Adds PerCallerConcurrency, a bounded-keyed in-flight limiter (distinct from the request-rate RateLimiter) so no single caller monopolizes the served-git read pool. Each caller (per-DID when signed via optional_signature, else per-source-IP via client_key) may hold at most max_concurrent_reads_per_caller concurrent reads; over that it sheds 503. The key map is self-bounding (a key is dropped when its in-flight count hits zero) with a reject-before-insert max_keys backstop so a key farm can't grow it (INV-15). Applied in git_upload_pack and both info/refs advertisements, acquired after the visibility gate so a denied request never consumes a slot (KTD7).

Primitive unit-tested (cap + self-bounding + reject-before-insert) and mutation-checked. Handler-layer SC2: same-caller sheds while a different caller passes, proven on BOTH git_info_refs and git_upload_pack with independent mutation probes; plus a None-key bypass test. Per-source-IP keying is trust-config dependent, documented on the config knob. 502 tests pass.

Part of #174.
2026-07-10 12:50:46 -05:00
beardthelion c97fe3a551 fix(node,git): isolate authed pushes in a dedicated write pool (#62)
git-receive-pack now draws from a separate git_write_semaphore (max_concurrent_git_pushes) instead of the shared pool, so a flood of anonymous reads can no longer shed an authenticated push at admission (jatmn P1a). The shared field is renamed git_read_semaphore and continues to gate upload-pack and both info/refs advertisements. The write permit stays above acquire_write so it precedes the Tigris fresh-acquire (INV-10).

Handler-layer tests: write-pool shed (503), and a cross-boundary proof that an exhausted read pool does NOT shed a push; both mutation-checked (routing receive-pack back to the read pool flips each RED). 497 tests pass.

Part of #174.
2026-07-10 12:29:30 -05:00
beardthelion 2c6cef6ad0 feat(node,config): add write-pool and per-caller read-cap knobs (#62)
Add max_concurrent_git_pushes (default 32) and max_concurrent_reads_per_caller (default 16), both clap range(1..=1_048_576) so an oversized value is a clean CLI error, not a Semaphore::new boot panic. The per-caller knob documents that per-source-IP keying is only as granular as GITLAWB_TRUSTED_PROXY. Wiring lands in the following commits; these are the config surface for the #174 concurrency-fairness fix.

Resolves jatmn P1a/P1b groundwork on #174.
2026-07-10 12:18:07 -05:00
beardthelion ba82ae785e feat(node,git): cap concurrent served git ops with a 503 load-shed (#62)
PR3 of the #62 served-git hardening stack (timeout #165 and teardown
wiring #150 are merged). A bounded semaphore caps how many upload-pack /
receive-pack / info-refs operations run at once; past the cap a request
is shed with a clean 503 + Retry-After before spawning another git
subprocess, instead of exhausting the PID/thread table. A permit is
acquired at the top of each of the three handlers and held for the whole
op, releasing on return.

The cap is a portable backstop: the compose pids_limit is absent on Fly,
whose 500-connection cap is a different axis. Size --max-concurrent-git-ops
(GITLAWB_MAX_CONCURRENT_GIT_OPS, default 128) below the process budget.
Range 1..=1_048_576 so 0 (shed everything) and an oversized value that
would panic tokio's Semaphore at boot are clean CLI errors.

Known gap, tracked separately: info/refs and the withheld-blob
(upload_pack_excluding) path are not duration-bounded and do not reap
their git child on client disconnect, so a hung git on those two paths
holds its slot until it exits and live git can briefly exceed the cap.
The main pack path (run_git_service) tears its group down on drop.

Tests: Overloaded maps to 503 + Retry-After; the config knob defaults
and rejects out-of-range; git_permit sheds at capacity and releases; and
each of the three endpoints sheds with 503 when the semaphore is
exhausted (load-bearing: drop the permit line and the endpoint test goes
red).
2026-07-10 12:10:30 -05:00
168 changed files with 107754 additions and 3861 deletions

No files matched your search

+19 -15
View File
@@ -8,24 +8,9 @@
# The audit job stays hard-fail: only the advisories explicitly listed below are
# suppressed, so any NEW advisory still breaks CI. Each entry below has no
# available upstream fix; everything fixable is fixed in Cargo.lock instead.
#
# Tracking: the hickory ignores are reachable accepted risk. A scheduled re-check
# (.github/workflows/audit-schedule.yml) runs without these ignores and fails if
# the lockfile moves off the pinned vulnerable versions while the ignores remain.
[advisories]
ignore = [
# hickory-proto 0.25.2 (pulled by libp2p-dns 0.44 -> hickory-resolver 0.25).
# REACHABLE: the node wraps its QUIC transport in the system DNS resolver
# (crates/gitlawb-node/src/p2p/mod.rs:238), so a malicious DNS response can
# trigger these DoS paths. No fix is available: the latest libp2p-dns on
# crates.io is 0.44 and pins hickory 0.25; the fix requires hickory >=0.26.1.
# REMOVE both once libp2p-dns ships a release using hickory 0.26 (verified
# 2026-06-21: cargo update -p hickory-proto locks 0 packages; still 0.25.2).
# Tracking: https://github.com/Gitlawb/node/issues/76
"RUSTSEC-2026-0118", # hickory NSEC3 closest-encloser unbounded loop
"RUSTSEC-2026-0119", # hickory O(n^2) name-compression CPU exhaustion
# rsa 0.9.10 (Marvin timing sidechannel). Present in Cargo.lock because sqlx
# resolves sqlx-mysql unconditionally, and sqlx-mysql depends on rsa. cargo
# audit scans the lockfile, so it flags rsa even though it is NOT linked into
@@ -35,4 +20,23 @@ ignore = [
# ever built with the `mysql` feature, or any other consumer of rsa enters
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
# crypto-bigint 0.5.5 (secret-dependent branches in operations meant
# to be constant-time). REACHABLE accepted risk, on two paths:
# - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256
# PrivateKeySigner that signs transactions in `gl name`
# (crates/gl/src/name.rs) and the node operator
# (crates/gitlawb-node/src/operator.rs).
# - aws-sigv4 (`sigv4a` feature, crates/gitlawb-node/Cargo.toml) ->
# crypto-bigint 0.5 (directly and via p256 0.13): SigV4a key
# derivation, used only for requests to endpoints that require SigV4a
# (e.g. S3 Multi-Region Access Points).
# Exploiting it needs local timing observation of those signing operations.
# The fix is crypto-bigint >=0.7.6, and no compatible dependent uses it:
# every alloy-signer release (through 2.5.0) requires elliptic-curve ^0.13,
# and aws-sigv4 1.6.4 is the first release on crypto-bigint 0.7 (1.5.x -
# 1.6.3 still require ^0.5.4), but it needs Rust 1.94.1, above our 1.91 MSRV
# (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint
# below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`.
"RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix)
]
+214 -12
View File
@@ -1,12 +1,41 @@
# gitlawb node — environment variables
# Twigpine node — environment variables
# Copy to .env and fill in your values.
# All variables are optional unless marked REQUIRED.
# ── Node identity ─────────────────────────────────────────────────────────
# Path to the node's Ed25519 keypair PEM file.
# Path to the node's Ed25519 keypair PEM file. Must name a key file, not a
# directory: a trailing slash, a final `.` or `..`, and `..` traversal are
# refused so the path cannot retarget a parent. A `~/` value must stay under
# home after expansion. Bare `identity.pem` is still legal.
# Generate with: gl identity new
GITLAWB_KEY=/data/keys/identity.pem
# Path to the node's persistent libp2p identity key file. Must include a
# directory, because the node will not keep its p2p identity key in the working
# directory. A value that cannot name a securable key file (a bare filename,
# `..` traversal, a trailing slash, the filesystem root, or a `~/` path that
# escapes home) is refused before the node binds, so the process exits.
# Anything wrong with the storage itself (a symlinked or non-directory parent,
# an unsafe cwd or ancestor, an unreadable or malformed key) leaves p2p off
# while HTTP keeps serving, logged at error as p2p_identity_key_load_failed and
# counted by the gitlawb_p2p_identity_key_load_failed metric. /health still
# reports healthy in that state, so alert on the event or the metric. Fixing
# the storage and restarting restores the same PeerId.
# On Unix it is created 0600 inside a 0700 directory. A key directory that
# grants access beyond its owner is tightened to 0700 on start; one closed too
# far is refused with a chmod 700 remedy rather than widened. On other
# platforms no permissions are enforced.
# If the node logs that it tightened a key directory that was GROUP OR WORLD
# WRITABLE, treat the key that was sitting there as possibly exposed: delete
# it so a fresh identity is generated on the next start. An ordinary 0755
# directory is tightened too, but others could not read a 0600 key or replace
# its directory entry, so do not rotate for that case. A directory that was
# merely closed too far is refused, not tightened, and no key in it was ever
# exposed.
# Keep it on a persistent volume so the PeerId survives redeploys.
# Default: ~/.gitlawb/p2p.key
#GITLAWB_P2P_KEY=/data/keys/p2p.key
# Publicly reachable URL of this node (used in peer announcements)
GITLAWB_PUBLIC_URL=https://your-node.example.com
@@ -25,7 +54,25 @@ DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=20
GITLAWB_DB_MAX_CONNECTIONS=48
# Maximum connections in the DEDICATED advisory-lock pool, separate from the
# pool above. Every in-flight repo write pins one connection here for its whole
# duration, so this is a hard ceiling on simultaneous writes node-wide: size it
# to expected peak concurrent writers, not small. Keeping it separate is what
# stops a push burst from starving ordinary request handlers. Budget
# (GITLAWB_DB_MAX_CONNECTIONS + this) per node against the server's
# max_connections, times node count, plus admin tooling.
GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS=40
# Upper bound, in seconds, on any object-storage transfer that runs while a
# per-repo write lock is HELD (the archive download inside acquire_write and the
# upload inside release). These were free before the lock's connection was
# pinned to the guard; now an unbounded stall holds a lock-pool slot, and enough
# stalls deny every write on the node. The bound applies PER SPAN and there are
# two (the acquire-side refresh, which covers the existence check and download
# together, and the release-side upload), so worst-case slot occupancy is about
# twice this value plus the git work between them. Read it together with the pool
# size above and with GITLAWB_GIT_SERVICE_TIMEOUT_SECS.
GITLAWB_LOCK_HELD_TRANSFER_TIMEOUT_SECS=300
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
@@ -85,23 +132,32 @@ GITLAWB_P2P_PORT=7546
# GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true for isolated dev/test networks.
GITLAWB_BOOTSTRAP_PEERS=
GITLAWB_BOOTSTRAP_DISABLE_SEEDS=false
# Require signatures on peer announce/sync write routes. Keep false during
# rolling upgrades so existing live nodes can still communicate.
# Require a peer write to prove its DID on every transport: RFC 9421 signatures
# on the announce/sync routes, and an Ed25519 payload signature on gossip
# ref-update events. Keep false during rolling upgrades so existing live nodes
# can still communicate. Upgrade every gossip-publishing peer to a build that
# signs events before setting this true, or their updates are dropped on
# arrival with no error visible on their side.
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Require the authenticated pusher to be the repo owner on git-receive-pack.
# A valid did:key signature is authentication, not authorization: anyone can
# sign as their own DID. When true, pushes from a non-owner DID are rejected.
# Keep false until the repo owner is ready for owner-only writes.
GITLAWB_ENFORCE_OWNER_PUSH=false
# mint a key, derive its DID and sign, so with this off every signed caller may
# push to every repository, private ones included. On by default.
#
# Set to false only for a rolling upgrade whose pushers are not yet the repo
# owner. Note that delegated and CI keys count as non-owners: UCAN git/push is
# verified but not yet honored for authorization, so they cannot push while this
# is on. See docs/RUN-A-NODE.md.
GITLAWB_ENFORCE_OWNER_PUSH=true
# Comma-separated libp2p multiaddrs.
# Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
GITLAWB_P2P_BOOTSTRAP=
# ── Access control ────────────────────────────────────────────────────────
# Reserved for private-read mode. Public/private repo read enforcement is not
# wired in the current live release; do not rely on this for private repositories.
# Reserved and currently inert. Repo reads are gated per repository by
# is_public and path-scoped visibility rules; this flag changes nothing.
GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
@@ -109,15 +165,161 @@ GITLAWB_MAX_PACK_BYTES=2147483648
# Max seconds a served git upload-pack / receive-pack (clone / push) may run
# before it is aborted with a 504. Bounds a hung git that would otherwise pin a
# worker and, on push, the repo write lock. Does NOT cover the info/refs
# advertisement or the withheld-blob path, which remain unbounded. Default 600.
# worker and, on push, the repo write lock. Also bounds both info/refs
# advertisements, the withheld-blob pack build, and the push-side candidate
# discovery (rev-list / cat-file), all reaped via process-group teardown (#174).
# On the path-scoped upload-pack path the withheld-blob classification walk and
# the pack serve share ONE deadline, so this value bounds their COMBINED duration
# rather than giving each stage a full budget: a walk that consumes it leaves the
# serve nothing and the clone gets a 504. Serving large path-scoped repos may
# need a higher value here than when each stage was budgeted separately.
# Must be 1..=3153600000 (100 years): the node derives deadlines from this value,
# and a larger one cannot be represented. Default 600.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
# Max seconds the storage-ACQUISITION phase of a served git op may run before the
# request is shed with a 503, separate from the git-run timeout above. A
# concurrency permit is taken before this phase and GITLAWB_GIT_SERVICE_TIMEOUT_SECS
# only starts once git spawns, so without this a stalled backend (a hung Tigris
# HEAD/GET, or a hung pg advisory-lock iteration on push) pins the permit and drains
# the pool until every later request 503s. On expiry the permit is released
# (fail-closed). Kept separate because acquisition and git execution are distinct
# cost centers. Must be positive; set very large to effectively disable. Default 30.
GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS=30
# Max concurrent git READ ops (upload-pack + the upload-pack info/refs
# advertisement) served at once, a global pool separate from the push pool below.
# The anon receive-pack info/refs advertisement has its OWN pool (see below), not
# this one. Over-cap sheds a clean 503 + Retry-After. Anonymous reads draw from
# here, so pair it with GITLAWB_MAX_CONCURRENT_READS_PER_CALLER (below) so one
# caller cannot monopolize the pool. Default 128.
GITLAWB_MAX_CONCURRENT_GIT_OPS=128
# Max concurrent git-receive-pack (push) POST operations, in a pool separate from
# the read pool (GITLAWB_MAX_CONCURRENT_GIT_OPS) so anonymous reads cannot shed an
# authenticated push at admission. The anon receive-pack info/refs advertisement
# runs in a SEPARATE pool of the same size (disjoint from this one), so an
# advertisement flood cannot shed a push either. Over-cap sheds a 503 +
# Retry-After. Default 32.
GITLAWB_MAX_CONCURRENT_GIT_PUSHES=32
# Max pushes PARKED at once waiting for the per-repo write lease of the SAME repo.
# A parked push holds its already-buffered pack body in memory, so this bounds that
# memory for a hot repo. Past the cap the newest push sheds a clean 503 +
# Retry-After ("repo is busy with another push") instead of joining the queue; a
# push to a DIFFERENT repo is never affected. The lease holder is not counted.
# Raising GITLAWB_MAX_CONCURRENT_GIT_PUSHES does not raise this; set it explicitly.
# Default 8.
GITLAWB_REPO_LEASE_MAX_WAITERS=8
# Max concurrent post-push pin loops (IPFS + Pinata pin_new_objects) across all
# repos. The per-repo encrypt-task set caps the pin-task COUNT; this caps how many
# of those loops RUN CONCURRENTLY, and so how much MB-scale object-id list memory is
# held WHILE BEING PINNED. It does not bound retained memory in general: on the local
# IPFS path the caller builds its list before this pool is entered, so a loop parked
# waiting for a permit still holds its full list, and how many park is capped only
# per repo. Do not size memory from this knob alone. A loop DEFERS (waits) when the
# pool is full, never drops a pin. Default 8.
GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
# Max concurrent read ops (upload-pack + the upload-pack info/refs advertisement)
# a single caller may hold, so one caller cannot monopolize the read pool. Keyed
# on the resolved SOURCE IP, never the DID: a signature does not move a caller off
# this cap. The source-IP key is only as granular as GITLAWB_TRUSTED_PROXY below:
# left unset, a node behind an edge/NAT keys all callers on the edge IP and this
# collapses to one global cap. Set GITLAWB_TRUSTED_PROXY for per-client keying; a
# high-fanout caller (CI behind one NAT) then needs the operator to raise this.
# Default 16.
GITLAWB_MAX_CONCURRENT_READS_PER_CALLER=16
# Two further per-source concurrency caps exist on the PUSH side but have NO
# environment variable of their own: both are DERIVED as
# GITLAWB_MAX_CONCURRENT_GIT_PUSHES / 8, with a floor of 1. They are not settable
# independently, so raising the push pool raises both proportionally, and the
# default push pool of 32 gives each a cap of 4.
# * The anon receive-pack info/refs advertisement cap: one source IP may hold at
# most this many slots in the dedicated advert pool, so saturating that pool
# takes ~8 distinct source IPs.
# * The authenticated receive-pack POST cap: one source IP may hold at most this
# many WRITE-pool slots. This one is load-bearing for push availability, since
# it is acquired before the global write permit: without it, one host minting
# disposable did:key identities could open enough slow pushes to monopolize the
# write pool and 503 every other source (the push rate limiter caps arrival
# rate, not in-flight concurrency, and owner enforcement can be turned off for
# a rolling upgrade).
# Keyed on the resolved source IP, never the DID, so a DID farm does not defeat
# them; keying granularity follows GITLAWB_TRUSTED_PROXY like the read cap above.
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
# Max receive-pack requests (info/refs advertisement + push POST) per client
# IP per hour. 0 disables. Default 600.
GITLAWB_PUSH_RATE_LIMIT=600
# ── /ipfs/{cid} visibility-walk admission (#174) ──────────────────────────
# GET /ipfs/{cid} runs a per-repo full-history git walk in a blocking thread to
# decide whether the caller may read a path-scoped blob. It is publicly reachable,
# so it is bounded to keep a permissionless caller from fanning out unbounded
# concurrent walks and exhausting blocking-pool threads + PIDs.
# Max concurrent /ipfs walks across all callers (a pool of its own, disjoint from
# the served-git pools). Over-cap sheds a 503. Default 32.
GITLAWB_MAX_CONCURRENT_IPFS_WALKS=32
# Max concurrent /ipfs walks a single SOURCE IP may hold (keyed like the git
# per-caller caps via GITLAWB_TRUSTED_PROXY; reject-before-insert bounded map).
# Default 4.
GITLAWB_IPFS_WALK_PER_SOURCE=4
# Max legacy (NULL-provenance) repos probed per single /ipfs request, bounding the
# scan-fallback fan-out (git cat-file per candidate repo) for an anonymous caller. A
# truncated scan sheds a retryable 503, never a false 404. Default 256.
GITLAWB_IPFS_MAX_LEGACY_PROBES=256
# Max repo ROWS one /ipfs request's legacy scan may read from the database. The probe
# ceiling above only counts once a probe runs, and quarantined or private repos are
# denied before that, so without this an all-denying inventory paged the whole repo
# table for one anonymous request. A truncated scan sheds a retryable 503 carrying a
# sealed `continuation` token; echo it as ?scan= to resume, so a holder buried past the
# ceiling is reachable in ceil(repos / ceiling) + 1 requests. Raising this also raises
# the per-caller /ipfs work allowance, since each page is charged to it. Lowering it
# sharpens a coarse oracle: laddering to the end reveals the node's total repo count to
# within one ceiling. Default 2048.
GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS=2048
# Max EXPENSIVE path-scope visibility walks per single /ipfs request (only a
# blob in a path-scoped repo costs a full-history walk). Over-cap repos are
# skipped without a verdict and the scan continues; if the object is then found
# nowhere the request sheds a retryable 503 instead of a false 404. The effective
# cap is the tighter of this value and the node's internal per-request ceiling of
# 17, so values above 17 have no effect; lower values do tighten it. Default 64.
GITLAWB_IPFS_MAX_REPOS_WALKED=64
# Ceiling on repos one /ipfs request may VISIT past the visibility gate. Each
# visit costs a repo acquire — on a Tigris cache miss a full archive download,
# so this is also the worst-case object-store fetch count per request — plus a
# cat-file probe. On exhaustion the scan stops and sheds a retryable 503.
# Default 1024.
GITLAWB_IPFS_MAX_REPO_VISITS=1024
# Absolute wall-clock budget for one admitted /ipfs request's acquire+walk
# lifetime (all stages of the whole scan). Per-stage clamps bound the acquire
# and walk stages to the remaining budget, and no stage starts once it is
# exhausted (the scan then sheds a retryable 503). The object-type probe and
# content-read cat-file subprocesses are budget-checked before starting AND each
# run under their own deadline (the lesser of GITLAWB_GIT_SERVICE_TIMEOUT_SECS and
# the remaining budget), reaped via process-group teardown, so a hung cat-file
# cannot hold the request's walk slot past it. One hang path is still unbounded:
# the probe's object-store readability check is a plain filesystem sweep with
# nothing to reap, so a wedged filesystem can hold the slot past the deadline.
# Must be 1..=3153600000 (100 years): the node derives an Instant deadline from
# this value, and a larger one cannot be represented. Default 600.
GITLAWB_IPFS_REQUEST_BUDGET_SECS=600
# Shorter budget (seconds) for the pre-walk CID resolve: the lookup that maps a
# requested CID to its git oid(s), which runs while the scarce walk admission is
# already held. A well-formed CID with no pin row does no probe and no walk work,
# so without this a stalled lookup could hold a walk slot for the whole request
# budget while nothing walked. The effective deadline is the lesser of this and
# the remaining request budget; walk and probe work stay on the request budget,
# so a slow but progressing scan is never shed by it.
# Must be 1..=3153600000 (100 years). Default 10.
GITLAWB_IPFS_RESOLVE_BUDGET_SECS=10
# Max /ipfs/{cid} requests per client IP per hour (route flood brake, distinct
# from the concurrency caps above). 0 disables. Default 600.
GITLAWB_IPFS_RATE_LIMIT=600
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID
+1 -1
View File
@@ -1,5 +1,5 @@
name: Bug report
description: Report a defect in a Gitlawb crate or node behavior
description: Report a defect in a Twigpine crate or node behavior
labels: ["bug"]
body:
- type: markdown
+2 -2
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/Gitlawb/node/blob/main/SECURITY.md
url: https://github.com/Twigpine/node/blob/main/SECURITY.md
about: Please report security issues privately per our security policy, not as a public issue.
- name: Questions & discussion
url: https://github.com/Gitlawb/node/discussions
url: https://github.com/Twigpine/node/discussions
about: For usage questions and open-ended discussion.
+1 -1
View File
@@ -1,5 +1,5 @@
<!--
Thanks for contributing to Gitlawb. Keep PRs focused on one change.
Thanks for contributing to Twigpine. Keep PRs focused on one change.
Protocol-level changes (identity, signatures, wire format) should start
as an issue before code — see CONTRIBUTING.md.
-->
+61
View File
@@ -55,6 +55,35 @@ jobs:
- name: cargo clippy
run: cargo clippy --locked --workspace --all-targets -- -D warnings
release-tag-validation:
name: release tag validation
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Test release tag resolver
run: scripts/test-resolve-release-tag.sh
triage-detector:
name: triage detector matrix
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
# pr-triage.yml runs on pull_request_target and never checks out PR
# code, so its inline detector can only be exercised here, where the
# proposed workflow body IS the checked-out one.
- name: Test inline-test detector
run: node scripts/test-pr-triage-detect.mjs
test:
name: test (${{ matrix.toolchain }})
runs-on: ubuntu-latest
@@ -283,3 +312,35 @@ jobs:
- name: Check gitlawb-core dependency allowlist
run: bash scripts/check-gitlawb-core-deps.sh
# release-please only rewrites a crate's version when the line carries the
# `# x-release-please-version` annotation AND the manifest is listed under
# extra-files in release-please-config.json. Miss either half and the crate
# is skipped silently at release time, freezing at its creation version:
# icaptcha-client was added as a workspace member after the config was last
# written and sat at 0.4.0 through four releases before anyone noticed.
# Exhaustive-by-construction, like the allowlist gate above: the required set
# is derived from `cargo metadata`, so a newly added crate reds CI whether or
# not its author knew this gate existed.
release-versioning:
name: release versioning coverage
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release-versioning
- name: Check every workspace crate is release-managed
run: bash scripts/check-release-versioning.sh
+222 -4
View File
@@ -70,9 +70,227 @@ jobs:
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or (the common Rust
// pattern) an inline #[test] / #[cfg(test)] block added inside the source file.
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
// Async tests use a namespaced attribute (#[tokio::test], #[async_std::test],
// possibly with args like #[tokio::test(flavor = ...)]) rather than the bare
// #[test]. Match only on added lines, with the attribute at the start of the
// line after horizontal whitespace, covering bare, cfg(test), namespaced,
// optional leading ::, and raw-identifier (r#test) forms. No head-file fetch
// or lexer: needs-tests is advisory, not a merge gate.
//
// The final path component may also carry `test` as an underscore-delimited
// segment, so third-party harnesses land without enumerating them one by one
// (#[test_case(1)], #[wasm_bindgen_test]). It is deliberately a segment and
// not a substring: a substring would also match #[contest] / #[latest], and a
// false positive here SUPPRESSES needs-tests rather than adding it. This
// heuristic has to fail loud, because a wrongly labeled PR gets corrected by
// the author while a wrongly cleared one is invisible. That trade costs the
// no-underscore harness names (#[rstest]), which stay unmatched on purpose.
//
// The segment classes are [A-Za-z0-9]+ rather than \w+ on purpose, so that `_`
// is only ever the literal delimiter. \w+ contains `_`, which makes the repeat
// ambiguous and backtracks exponentially: on this workflow's pull_request_target
// trigger f.patch is fork-controlled, so `#[test` followed by a long `_a` run is
// a permissionless stall of the triage job.
// Rust allows whitespace, line breaks, and comments between the
// attribute path and ] or (. A flat regex cannot decide that
// boundary: `//` runs to end of line, and block comments NEST, which
// is beyond regular languages — every enumeration of comment
// spellings so far has either dropped a legal separator (regressing
// real tests into needs-tests) or let unrelated patch records
// complete each other (clearing needs-tests from a testless PR, the
// invisible direction). So the closer is found by a token-aware
// scan instead: it consumes horizontal whitespace, `//`-to-newline,
// and depth-counted `/* */` runs, and accepts only when the first
// real token after the attribute path is `]` or `(`.
//
// Association is positional: the scan starts on the attribute's own
// added line and may continue ONLY through the immediately following
// added lines (a context line, hunk header, or removal breaks the
// chain). Unrelated `]`/`(` lines elsewhere in the patch can never
// complete a path they do not adjoin. The continuation is bounded;
// past the bound the scan gives up and reports NO inline test, which
// fails toward the loud label rather than the silent clearance.
//
// The detector body is fenced by the TRIAGE_DETECTOR markers so
// scripts/test-pr-triage-detect.mjs can extract and run it against
// the committed case matrix. Keep the fenced block self-contained.
// TRIAGE_DETECTOR_BEGIN
const TEST_ATTR_PATH =
"(?:cfg[ \\t]*\\([ \\t]*test[ \\t]*\\)|(?:::[ \\t]*)?(?:[\\w-]+[ \\t]*::[ \\t]*)*(?:r#)?(?:[A-Za-z0-9]+_)*test(?:_[A-Za-z0-9]+)*)";
// Anchored to the start of one added line; group 1 is whatever
// follows the path on that line, handed to the separator scan.
const TEST_ATTR_LINE = new RegExp(
"^[ \\t]*#\\[[ \\t]*" + TEST_ATTR_PATH + "([\\s\\S]*)$"
);
// An attribute split across more added lines than this is not a
// spelling anyone writes; refusing to scan further keeps the walk
// linear in the patch and fails toward the loud label.
const MAX_ATTR_CONTINUATION_LINES = 16;
// segs[0] is the remainder of the attribute's own line; each later
// entry is the text of one immediately following added line. True
// only when the first non-separator token across them is ] or (.
// Single forward pass, no backtracking: fork-controlled input.
function attrCloserFollows(segs) {
let depth = 0; // open block-comment nesting carried across lines
for (const seg of segs) {
let i = 0;
while (i < seg.length) {
if (depth > 0) {
if (seg.startsWith("/*", i)) { depth += 1; i += 2; }
else if (seg.startsWith("*/", i)) { depth -= 1; i += 2; }
else { i += 1; }
continue;
}
const c = seg[i];
if (c === " " || c === "\t") { i += 1; continue; }
if (seg.startsWith("//", i)) break; // comment to end of line
if (seg.startsWith("/*", i)) { depth = 1; i += 2; continue; }
return c === "]" || c === "(";
}
// Line exhausted inside separators; the newline is itself a
// separator, so continue on the next added line.
}
return false; // out of adjoined lines (or bound hit): stay loud
}
// Before the candidate scan, a pre-pass walks added patch lines
// tracking Rust lexical state: raw strings (r#*", br#*", cr#*"),
// block comments (/* */ with nesting), regular strings ("...",
// which can span newlines), and line comments (//). An added line
// that starts inside any of these is skipped as a candidate,
// because a #[test_case] appearing there is lexical non-code, not
// a real test attribute. State is only tracked across consecutive
// added lines and reset at any break (context line, removal,
// hunk header): context lines carry a partial view of the file,
// so a " or r#" or /* there might be inside a string that opened
// before the hunk's visible window, and treating it as an opener
// would mark every later added line as non-code (a false negative
// on a real #[test]). The tradeoff is that a non-code region
// opened by a context line is invisible: an added #[test] inside
// an existing raw string or block comment is scanned as code and
// may match (a false positive, silent direction). This is accepted
// because (1) the old detector had the same behavior, (2) the
// pattern is rare, and (3) needs-tests is advisory, not a merge
// gate. Scanning context lines to close this gap would re-introduce
// the false negative, which affects the far more common case of a
// context line carrying a stray " from a string that opened before
// the visible window. Single forward pass, O(n) in patch length.
function patchAddsInlineTest(patch) {
const lines = patch.split("\n");
const nonCodeStart = new Set();
let rawHashes = -1; // -1 = not in raw string; >=0 = # count
let blockDepth = 0;
let inString = false;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (!line) continue;
const prefix = line[0];
// Only scan added lines. Context and removal lines carry
// a partial view of the file: a " or r#" or /* on a context
// line might be inside a string that opened before the
// hunk's visible window, and treating it as an opener would
// mark every later added line as non-code (a false negative
// on a real #[test]). So lexical state is only tracked across
// consecutive added lines and reset at any break. A non-code
// region opened by a context line is invisible; the candidate
// is still scanned, which fails toward the loud label.
if (prefix !== "+") {
rawHashes = -1; blockDepth = 0; inString = false;
continue;
}
const text = line.slice(1);
if (rawHashes >= 0 || blockDepth > 0 || inString)
nonCodeStart.add(i);
let j = 0;
while (j < text.length) {
if (rawHashes >= 0) {
if (text[j] === '"') {
let h = 1;
while (h <= rawHashes && text[j + h] === "#") h++;
if (h > rawHashes) { rawHashes = -1; j += h; continue; }
}
j += 1; continue;
}
if (blockDepth > 0) {
if (text.startsWith("/*", j)) { blockDepth += 1; j += 2; }
else if (text.startsWith("*/", j)) { blockDepth -= 1; j += 2; }
else { j += 1; }
continue;
}
if (inString) {
if (text[j] === "\\") { j += 2; continue; }
if (text[j] === '"') { inString = false; j += 1; continue; }
j += 1; continue;
}
// Recognize Rust char and byte-char literals ('X', '\X',
// '\u{...}', b'X') so their contents never open string or
// comment state. A char literal has a closing ' within a
// bounded distance; a lifetime or label ('a, 'static,
// 'label:) does not, so it falls through as a regular char.
// This is not a skip-to-next-' rule: the lookahead is 2-4
// chars, or bounded to 20 for \u escapes (at most 6 hex
// digits), never unbounded.
if (text[j] === "'") {
let k = j + 1;
if (text[k] === "\\") {
k += 1;
if (text[k] === "u" && text[k + 1] === "{") {
const uBound = k + 20;
while (k < uBound && text[k] && text[k] !== "}") k++;
if (text[k] === "}") k++;
} else {
k += 1;
}
} else {
k += 1;
}
if (text[k] === "'") { j = k + 1; continue; }
// lifetime or label: just skip the apostrophe
}
if (text[j] === '"') { inString = true; j += 1; continue; }
if (text.startsWith("//", j)) break;
if (text.startsWith("/*", j)) { blockDepth = 1; j += 2; continue; }
const prev = j > 0 ? text[j - 1] : "";
const isIdent = /[A-Za-z0-9_]/.test(prev);
if (!isIdent && text[j] === "r") {
let k = j + 1, h = 0;
while (text[k + h] === "#") h++;
if (text[k + h] === '"') {
rawHashes = h; j = k + h + 1; continue;
}
}
if (!isIdent && (text[j] === "b" || text[j] === "c") && text[j + 1] === "r") {
let k = j + 2, h = 0;
while (text[k + h] === "#") h++;
if (text[k + h] === '"') {
rawHashes = h; j = k + h + 1; continue;
}
}
j += 1;
}
}
for (let i = 0; i < lines.length; i++) {
if (lines[i][0] !== "+") continue;
if (nonCodeStart.has(i)) continue;
const m = TEST_ATTR_LINE.exec(lines[i].slice(1));
if (m === null) continue;
const segs = [m[1]];
for (
let j = i + 1;
j < lines.length &&
lines[j][0] === "+" &&
segs.length <= MAX_ATTR_CONTINUATION_LINES;
j++
) {
segs.push(lines[j].slice(1));
}
if (attrCloserFollows(segs)) return true;
}
return false;
}
// TRIAGE_DETECTOR_END
const addsInlineTest = files.some(
f => f.filename.endsWith(".rs") && !!f.patch && patchAddsInlineTest(f.patch)
);
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
@@ -118,7 +336,7 @@ jobs:
"",
lines,
"",
"See [CONTRIBUTING.md](https://github.com/Gitlawb/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
"See [CONTRIBUTING.md](https://github.com/Twigpine/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
].join("\n");
await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: comment });
+264 -69
View File
@@ -20,7 +20,15 @@ on:
type: string
concurrency:
group: release-${{ github.ref }}
# Not keyed on the ref: two runs racing `imagetools create` on the same
# moving tags could interleave, so runs of this workflow serialize outright.
# GitHub keeps at most one running and one pending run per group; a newer
# queued run replaces the pending one, and cancel-in-progress: false
# protects only the run already executing. So a backfill dispatched while a
# release is running, followed by a push to main, is replaced silently: the
# remedy is to re-dispatch it once the running release completes, which the
# push-only gate on the moving-tag step makes safe in any ordering.
group: release
cancel-in-progress: false
# Least privilege by default; every job opts in to exactly what it needs.
@@ -31,12 +39,13 @@ jobs:
# Skipped on manual dispatch: a backfill must never create or advance a
# release PR as a side effect (npm-publish tolerates the skipped need via
# !cancelled(), and every other job gates on release_created == 'true').
if: ${{ github.repository == 'Gitlawb/node' && github.event_name != 'workflow_dispatch' }}
if: ${{ github.repository == 'Twigpine/node' && github.event_name != 'workflow_dispatch' }}
name: Release Please
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
environment: release
outputs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
@@ -64,7 +73,7 @@ jobs:
sync-release-lock:
name: Sync Cargo.lock on the release PR
needs: release-please
if: ${{ !cancelled() && github.repository == 'Gitlawb/node' && github.event_name != 'workflow_dispatch' }}
if: ${{ !cancelled() && github.repository == 'Twigpine/node' && github.event_name != 'workflow_dispatch' }}
runs-on: ubuntu-latest
permissions:
contents: write
@@ -78,6 +87,7 @@ jobs:
# dispatch the release PR's head carries no check runs and the --locked
# gate never validates it.
actions: write
environment: release
steps:
# Prefer the fresh `pr` output when this run just created/updated the
# release PR; otherwise fall back to querying for an open release-please
@@ -215,21 +225,22 @@ jobs:
permissions:
contents: read
packages: write
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
TAG="${DISPATCH_TAG:-$RELEASE_TAG}"
case "$TAG" in
v[0-9]*) ;;
*) echo "::error::'$TAG' does not look like a release tag (vX.Y.Z)"; exit 1 ;;
esac
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing — a mixed-case owner
# makes the digest push fail with "invalid reference format".
@@ -238,7 +249,9 @@ jobs:
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.rel.outputs.tag }}
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
persist-credentials: false
- name: Set up Docker Buildx
@@ -311,22 +324,28 @@ jobs:
if: ${{ !cancelled() && needs.docker.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
TAG="${DISPATCH_TAG:-$RELEASE_TAG}"
case "$TAG" in
v[0-9]*) ;;
*) echo "::error::'$TAG' does not look like a release tag (vX.Y.Z)"; exit 1 ;;
esac
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing, so a mixed-case owner
# makes the digest push fail with "invalid reference format".
echo "image=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
- name: Download digests
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
@@ -346,24 +365,80 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create and push multi-arch manifest
id: manifest
env:
VERSION: ${{ steps.rel.outputs.version }}
IMAGE: ${{ steps.rel.outputs.image }}
run: |
set -euo pipefail
# ghcr requires a lowercase repository path.
IMAGE="ghcr.io/${GITHUB_REPOSITORY,,}"
MAJOR_MINOR="${VERSION%.*}"
# The docker matrix pushes one digest per arch leg; anything else is
# a broken set, not a smaller multi-arch image.
count="$(find /tmp/digests -maxdepth 1 -type f | wc -l)"
if [ "$count" -ne 2 ]; then
echo "::error::expected 2 arch digests in /tmp/digests, found $count"
exit 1
fi
digests=""
for f in /tmp/digests/*; do
digests="$digests $IMAGE@sha256:$(basename "$f")"
d="$(basename "$f")"
case "$d" in
*[!0-9a-f]*)
echo "::error::digest filename is not lowercase sha256 hex: $d"
exit 1
;;
esac
if [ "${#d}" -ne 64 ]; then
echo "::error::digest is not 64 hex chars: $d"
exit 1
fi
digests="$digests $IMAGE@sha256:$d"
done
# The immutable tag always publishes. The moving tags are applied by
# the gated step below, which never runs on workflow_dispatch.
# shellcheck disable=SC2086
docker buildx imagetools create \
-t "$IMAGE:$VERSION" \
-t "$IMAGE:$MAJOR_MINOR" \
-t "$IMAGE:latest" \
$digests
docker buildx imagetools create -t "$IMAGE:$VERSION" $digests
docker buildx imagetools inspect "$IMAGE:$VERSION"
echo "digests=${digests# }" >> "$GITHUB_OUTPUT"
- name: Move floating tags
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
IMAGE: ${{ steps.rel.outputs.image }}
DIGESTS: ${{ steps.manifest.outputs.digests }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
MAJOR_MINOR="${VERSION%.*}"
# :latest and :X.Y track the newest release. Only a push to main
# carries a release-please-cut version, and release-please versions
# are monotonic, so on this path the tags only ever advance. A
# workflow_dispatch backfill republishes an existing tag and must
# never move a pointer, so this step skips on dispatch. After a
# backfill that is genuinely newest in its minor line, advance :X.Y
# by hand: docker buildx imagetools create -t "$IMAGE:X.Y" <digests>.
# The event gate alone does not bound a stale run: re-running an
# older push run replays its stored needs outputs under the
# original event and ref, and would move the pointers backward to
# that run's version. Floor against live state instead: this run's
# release already exists by now, so VERSION must be the newest
# published release or the run is stale and the tags stay put. The
# releases list is ordered by creation, not semver, so read every
# page and take the semver maximum, not the first page's.
tags="$(gh api --paginate "repos/$GITHUB_REPOSITORY/releases?per_page=100" \
-q '.[] | select(.draft == false) | .tag_name')" || {
echo "::error::could not list releases to bound the moving tags"
exit 1
}
floor="$(printf '%s\n' "$tags" \
| { grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' || true; } \
| sed 's/^v//' | sort -V | tail -n 1)"
if [ -n "$floor" ] && [ "$floor" != "$VERSION" ]; then
echo "::error::release v$floor is newer than this run's v$VERSION; refusing to move the moving tags backward (stale re-run?)"
exit 1
fi
# shellcheck disable=SC2086
docker buildx imagetools create -t "$IMAGE:latest" -t "$IMAGE:$MAJOR_MINOR" $DIGESTS
- name: Release summary
env:
@@ -387,6 +462,7 @@ jobs:
continue-on-error: ${{ startsWith(matrix.target, 'x86_64-pc-windows') }}
permissions:
contents: write
environment: release
strategy:
fail-fast: false
matrix:
@@ -406,10 +482,27 @@ jobs:
- target: x86_64-pc-windows-msvc
os: windows-latest
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash # matrix includes windows-latest, where run: defaults to pwsh
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
persist-credentials: false
- name: Set up Rust toolchain
@@ -455,7 +548,7 @@ jobs:
shell: bash
run: |
set -euo pipefail
VERSION="${{ needs.release-please.outputs.version }}"
VERSION="${{ steps.rel.outputs.version }}"
BIN_DIR="target/${{ matrix.target }}/release"
EXE=""
case "${{ matrix.target }}" in *windows*) EXE=".exe" ;; esac
@@ -475,7 +568,7 @@ jobs:
run: |
set -euo pipefail
TARGET="${{ matrix.target }}"
NAME="gitlawb-node-${{ needs.release-please.outputs.version }}-${TARGET}"
NAME="gitlawb-node-${{ steps.rel.outputs.version }}-${TARGET}"
BIN_DIR="target/${TARGET}/release"
EXE=""
case "$TARGET" in *windows*) EXE=".exe" ;; esac
@@ -516,7 +609,7 @@ jobs:
- name: Attach to release
uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8 # v2.3.2
with:
tag_name: ${{ needs.release-please.outputs.tag_name }}
tag_name: ${{ steps.rel.outputs.tag }}
files: |
dist/*.tar.gz
dist/*.zip
@@ -529,7 +622,9 @@ jobs:
# (release-binaries is skipped on dispatch, hence !cancelled()). Publishing
# auth is npm Trusted Publishing (GitHub OIDC) — no NPM_TOKEN. Each
# @gitlawb package must have this repo + workflow (release.yml) configured
# as its trusted publisher on npmjs.com, or publish fails loudly here —
# as its trusted publisher on npmjs.com, scoped to the release environment
# so a dispatched copy of this file on another ref cannot mint a token, or
# publish fails loudly here —
# deliberately loud: the silent secret-guard skip is how 0.4.x–0.6.0
# never reached npm.
if: >-
@@ -541,26 +636,29 @@ jobs:
permissions:
contents: read
id-token: write # npm trusted publishing (OIDC) + provenance
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.npm_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
TAG="${DISPATCH_TAG:-$RELEASE_TAG}"
case "$TAG" in
v[0-9]*) ;;
*) echo "::error::'$TAG' does not look like a release tag (vX.Y.Z)"; exit 1 ;;
esac
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.rel.outputs.tag }}
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
persist-credentials: false
# npm >= 11.5.1 performs the OIDC token exchange automatically when the
@@ -573,14 +671,23 @@ jobs:
- name: Ensure npm supports trusted publishing
run: |
set -euo pipefail
npm install -g npm@^11.5.1
npm --version
npm install -g npm@11.19.1
# Assert the pin took, so an accidental unpin or a drift off 11.19.1
# fails loudly here instead of silently running a different npm on
# the OIDC path.
installed="$(npm --version)"
echo "npm installed: $installed"
if [ "$installed" != "11.19.1" ]; then
echo "::error::npm is not the pinned 11.19.1 (got: $installed)"
exit 1
fi
- name: Lay in release binaries
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
ASSETS: ${{ steps.rel.outputs.assets }}
run: |
set -euo pipefail
# npm platform package -> Rust target triple (unix only; Windows is not
@@ -597,8 +704,18 @@ jobs:
target="${entry#*:}"
archive="gitlawb-node-${VERSION}-${target}.tar.gz"
echo "==> $pkg <- $archive"
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern "$archive" --dir _dl --clobber
# Download by the asset id the resolver captured and uploader-
# checked, not by name: release assets are mutable, and an id can
# only ever point at the exact blob captured at resolve time.
# The map keys are base64-encoded asset names.
want="$(printf '%s' "$archive" | base64 -w0)"
asset_id="$(printf '%s\n' "$ASSETS" | awk -v n="$want" '$1 == n {print $2; exit}')"
if [ -z "$asset_id" ]; then
echo "::error::release $TAG has no asset $archive captured at resolve time"
exit 1
fi
gh api "repos/$GITHUB_REPOSITORY/releases/assets/$asset_id" \
-H 'Accept: application/octet-stream' > "_dl/$archive"
tar -xzf "_dl/$archive" -C _dl
src="_dl/gitlawb-node-${VERSION}-${target}"
cp "$src/gl" "npm/packages/$pkg/gl"
@@ -637,14 +754,39 @@ jobs:
# Platform packages first, then the wrapper (so its optionalDependencies resolve).
# Skip versions already on the registry so a rerun after a partial publish
# is idempotent instead of erroring on the first existing package.
# npm publish moves the latest dist-tag to whatever it publishes, so a
# backfilled older version would hand :latest to stale code. Decide
# the dist-tag per package from that package's own registry latest:
# a partial publish can leave the platform packages ahead of the
# wrapper. E404 means the package has never been published, so this
# release gets latest; any other lookup failure must fail closed
# rather than guess at the dist-tag and risk moving latest backward.
# stderr goes to a file so a warning on a successful lookup cannot
# leak into registry_latest; it is read only when the lookup fails.
npm_err="$(mktemp)"
trap 'rm -f -- "$npm_err"' EXIT
for pkg in gl-darwin-arm64 gl-darwin-x64 gl-linux-arm64 gl-linux-x64 gl; do
name="@gitlawb/$pkg"
if npm view "$name@$VERSION" version >/dev/null 2>&1; then
echo "==> $name@$VERSION already published, skipping"
continue
fi
echo "==> npm publish $name@$VERSION"
npm publish "npm/packages/$pkg" --provenance --access public
registry_latest="$(npm view "$name" dist-tags.latest 2>"$npm_err")" || {
if ! grep -q E404 "$npm_err"; then
cat "$npm_err" >&2
echo "::error::npm dist-tags lookup failed for $name; not guessing the dist-tag"
exit 1
fi
registry_latest=""
}
dist_tag="latest"
if [ -n "$registry_latest" ] && \
[ "$VERSION" != "$(printf '%s\n%s\n' "$registry_latest" "$VERSION" | sort -V | tail -1)" ]; then
dist_tag="backfill"
echo "::notice::$VERSION is older than $name@$registry_latest; publishing under dist-tag backfill"
fi
echo "==> npm publish $name@$VERSION (dist-tag $dist_tag)"
npm publish "npm/packages/$pkg" --provenance --access public --tag "$dist_tag"
done
homebrew-bump:
@@ -654,6 +796,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN downloads release assets; the tap push uses HOMEBREW_TAP_PAT
environment: release
steps:
- name: Guard on secret
id: guard
@@ -667,11 +810,27 @@ jobs:
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out workflow scripts
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout tap repo
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: Gitlawb/homebrew-tap
repository: Twigpine/homebrew-tap
token: ${{ secrets.HOMEBREW_TAP_PAT }}
path: tap
persist-credentials: false
@@ -680,15 +839,32 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
ASSETS: ${{ steps.rel.outputs.assets }}
run: |
set -euo pipefail
base="https://github.com/${GITHUB_REPOSITORY}/releases/download/${TAG}"
mkdir -p _sums
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern '*.tar.gz.sha256' --dir _sums --clobber
sha() { awk '{print $1}' "_sums/gitlawb-node-${VERSION}-$1.tar.gz.sha256"; }
# Hash the tarballs themselves, fetched by the asset id the resolver
# captured and uploader-checked. The *.sha256 sidecars are mutable
# release assets too; trusting them would trust the same surface.
sha() {
archive="gitlawb-node-${VERSION}-$1.tar.gz"
want="$(printf '%s' "$archive" | base64 -w0)"
asset_id="$(printf '%s\n' "$ASSETS" | awk -v n="$want" '$1 == n {print $2; exit}')"
if [ -z "$asset_id" ]; then
echo "::error::release $TAG has no asset $archive captured at resolve time" >&2
exit 1
fi
# errexit is off inside the $(...) this runs under, so a failed
# download must exit the function itself instead of hashing a
# truncated file.
gh api "repos/$GITHUB_REPOSITORY/releases/assets/$asset_id" \
-H 'Accept: application/octet-stream' > "_sums/$archive" \
|| exit 1
sha256sum "_sums/$archive" | awk '{print $1}'
}
SHA_MAC_ARM="$(sha aarch64-apple-darwin)"
SHA_MAC_X64="$(sha x86_64-apple-darwin)"
SHA_LNX_ARM="$(sha aarch64-unknown-linux-musl)"
@@ -697,8 +873,8 @@ jobs:
mkdir -p tap/Formula
cat > tap/Formula/gl.rb <<EOF
class Gl < Formula
desc "Gitlawb CLI — decentralized git for AI agents and developers"
homepage "https://gitlawb.com"
desc "Twigpine CLI — decentralized git for AI agents and developers"
homepage "https://twigpine.com"
version "${VERSION}"
license "MIT OR Apache-2.0"
@@ -749,7 +925,7 @@ jobs:
working-directory: tap
env:
HOMEBREW_TAP_PAT: ${{ secrets.HOMEBREW_TAP_PAT }}
VERSION: ${{ needs.release-please.outputs.version }}
VERSION: ${{ steps.rel.outputs.version }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
@@ -762,7 +938,7 @@ jobs:
fi
git commit -m "gl ${VERSION}"
# Credentials are not persisted in .git/config; supply the token only for the push.
git push "https://x-access-token:${HOMEBREW_TAP_PAT}@github.com/Gitlawb/homebrew-tap.git" HEAD:main
git push "https://x-access-token:${HOMEBREW_TAP_PAT}@github.com/Twigpine/homebrew-tap.git" HEAD:main
web-sync:
name: Sync web (install scripts + version)
@@ -771,6 +947,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN checks out node; the web PR uses WEB_SYNC_PAT
environment: release
steps:
- name: Guard on secret
id: guard
@@ -784,11 +961,29 @@ jobs:
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out workflow scripts
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout node (release tag)
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
path: node
persist-credentials: false
@@ -796,7 +991,7 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: Gitlawb/web
repository: Twigpine/web
token: ${{ secrets.WEB_SYNC_PAT }}
path: web
persist-credentials: false
@@ -808,8 +1003,8 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
run: |
set -euo pipefail
gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets \
@@ -827,8 +1022,8 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.WEB_SYNC_PAT }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
run: |
set -euo pipefail
# Canonical install scripts (source of truth lives in the node repo).
@@ -839,7 +1034,7 @@ jobs:
# One-time, idempotent drift fixes: drop orphaned binaries, fix stale link.
rm -rf web/public/bin
if [ -f web/public/skill.md ]; then
sed -i 's#github.com/gitlawb/releases#github.com/Gitlawb/node/releases#g' web/public/skill.md
sed -i 's#github.com/gitlawb/releases#github.com/Twigpine/node/releases#g' web/public/skill.md
fi
cd web
@@ -853,16 +1048,16 @@ jobs:
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "$branch"
git commit -m "chore: sync gitlawb ${TAG} (install scripts + version)"
git commit -m "chore: sync Twigpine ${TAG} (install scripts + version)"
# Credentials are not persisted in .git/config; supply the token only for the push.
git push --force "https://x-access-token:${GH_TOKEN}@github.com/Gitlawb/web.git" "HEAD:$branch"
gh pr create --repo Gitlawb/web --head "$branch" \
--title "Sync gitlawb ${TAG}" \
--body "Automated sync from Gitlawb/node ${TAG}: install.sh, install.ps1, public/version.json, and one-time drift fixes (removed orphaned public/bin, fixed skill.md releases link). Review and merge to deploy via Vercel." \
git push --force "https://x-access-token:${GH_TOKEN}@github.com/Twigpine/web.git" "HEAD:$branch"
gh pr create --repo Twigpine/web --head "$branch" \
--title "Sync Twigpine ${TAG}" \
--body "Automated sync from Twigpine/node ${TAG}: install.sh, install.ps1, public/version.json, and one-time drift fixes (removed orphaned public/bin, fixed skill.md releases link). Review and merge to deploy via Vercel." \
|| {
# Only swallow the "PR already exists" case; surface auth/API failures
# (a failed gh pr list must not be misread as "PR exists").
pr_count="$(gh pr list --repo Gitlawb/web --head "$branch" --state open --json number --jq 'length')" || exit 1
pr_count="$(gh pr list --repo Twigpine/web --head "$branch" --state open --json number --jq 'length')" || exit 1
if [ "$pr_count" != "0" ]; then
echo "PR for $branch already exists; branch was updated."
else
+44
View File
@@ -0,0 +1,44 @@
# Contributor guide for coding agents
This file carries the rules a patch author cannot recover from the code alone, for humans and coding agents alike. For build, test, and PR basics, read `CONTRIBUTING.md` first.
## Authorization invariants (the rules PRs most often break)
Every repo-scoped endpoint must bind the caller to an authorization decision before serving or mutating anything. The `known_ungated` allowlist in `authz_guard::every_repo_scoped_handler_is_gated` is empty on main: `list_webhooks`, `list_replicas`, and `list_protected_branches` are gated (`list_webhooks` with read-visibility then owner; the other two with `authorize_repo_read`). Do not re-open those surfaces as exceptions.
- Owner-only mutations (visibility, webhooks, protected branches, merges) are gated against the repo owner. Three gate forms are in use and all are recognized: webhooks and merges call `require_repo_owner` (`crates/gitlawb-node/src/api/mod.rs`), visibility uses its module-local `require_owner`, and protected branches compare inline with `did_matches`. Use the form the surrounding module already uses.
- Not every write is owner-only, and that is by design: `star_repo`/`unstar_repo` bind to the signer (they still require repo read access), `register_replica`/`unregister_replica` bind to the replica's own DID, the bounty actions (claim, submit, approve, cancel, dispute) have their own multi-party rules, and closing a PR or issue is owner-or-author. Do not add owner gates to these.
- Content-serving reads (blobs, trees, raw files) gate through `authorize_repo_read` with the specific path being served, so a withheld subtree is denied even on an otherwise-public repo; repo-level reads (listings) pass `"/"` by the helper's own contract. The source scans below cannot check this argument's value, so review will.
- A route that reaches repo data through a global id, with no repo in its path, or through an extractor shape the scans do not recognize still needs the same gate; the scans cannot see those.
- Read-surface denials must not reveal existence: a caller who may not read a repo (or a withheld subtree) gets the same 404 as a missing repo, never a 403. (Owner-gated mutations currently return 403 after the repo lookup; do not widen that shape to read paths.)
- Two tests in module `authz_guard` (`crates/gitlawb-node/src/api/mod.rs`) scan handler sources and fail on a repo-scoped handler without a recognized gate: `every_in_scope_mutation_has_its_gate` and `every_repo_scoped_handler_is_gated`. If they fail on your handler, add the gate. The second test's `known_ungated` allowlist is empty; never add an entry to it without an open issue and review.
## Adding or changing routes
- A new gated handler needs tests proving its route-specific gate fires: exercise an unauthorized authenticated caller and an anonymous caller where applicable, assert the exact denial status each way, and assert the body leaks nothing. The non-owner tests in `crates/gitlawb-node/src/test_support.rs` show the shape for owner-gated routes.
- A new mutation handler also gets a row in `every_in_scope_mutation_has_its_gate` naming its gate type; the per-handler row is the point of that test.
- A table-driven deny suite covering every deny-bearing route is in review (#194, #195). If it is present in your checkout, add a probe row there for any new deny-bearing route.
## Verifying signatures and certificates
- Derive the verifying key from something outside the artifact you are checking: the node's own DID, a pinned or configured value, a peers lookup. A key read from a field of the object being verified (`Did::from_str(&cert.node_did)`) only proves the artifact is self-consistent, and identities here are permissionless, so anyone can mint a keypair, name it in the artifact, self-sign, and be told it is valid. A mismatch against that anchor has to fail the result, not log a warning.
- A handler whose correct answer is "valid" needs a positive case driven by a trusted, independently anchored artifact, and a rejection case for a well-formed, fully populated forged artifact. Invalid-only suites miss an implementation that rejects everything; a forged artifact that still returns valid is the regression that matters.
- The fields a signature covers are a versioned format. Adding one invalidates every artifact already signed, so give the payload a version, keep a path that verifies the older form, and add a test that verifies an artifact signed before your change.
## Removing a serving path
When a fix removes a path that used to serve content, the same PR must add a test asserting the removed path now denies. The deletion is what turns that test green; without it the path can quietly come back.
## Database migrations
Schema changes are code-defined in `MIGRATIONS` (`crates/gitlawb-node/src/db/mod.rs`). Always append a new versioned entry; never edit an entry that has merged (operators deploy from main). Test runs build the schema from scratch, so editing an applied version stays green in CI while breaking every existing deployment.
## Client behavior (`gl`, `git-remote-gitlawb`)
When a node denies a request, surface the denial to the user. Never render a denial as an empty list or a silent success; that turns an authorization boundary into invisible data loss.
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
+9 -6
View File
@@ -1,12 +1,12 @@
# Contributing to gitlawb
# Contributing to Twigpine
Thanks for your interest in contributing. gitlawb is an open protocol — agents and humans welcome.
Thanks for your interest in contributing. Twigpine is an open protocol — agents and humans welcome.
## Quick start for contributors
```sh
# Clone and build
git clone https://github.com/Gitlawb/node
git clone https://github.com/Twigpine/node
cd node
# Build everything
@@ -31,12 +31,14 @@ crates/
├── gitlawb-core/ crypto primitives (DID, CID, HTTP sigs, UCAN, ref certs)
├── gitlawb-node/ axum HTTP server, git smart HTTP, P2P, GraphQL
├── gl/ CLI — identity, repos, MCP server, Base L2 names
└── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── gitlawb-attest/ provenance attestations for ref-update certificates
└── icaptcha-client/ client for the iCaptcha proof-of-intelligence service
docs/ Operator guides
scripts/ Build helpers
```
Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://github.com/Gitlawb/contracts).
Smart contracts live in a separate repo: [github.com/Twigpine/contracts](https://github.com/Twigpine/contracts).
## Ways to contribute
@@ -54,6 +56,7 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
3. **No breaking changes without discussion.** Open an issue first for protocol-level changes.
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
6. **Security-surface rules.** [`AGENTS.md`](AGENTS.md) carries the authorization invariants PRs most often break; it is written for coding agents and humans alike, so read it before touching endpoints, visibility, or migrations.
## What gets merged, what gets closed
@@ -105,7 +108,7 @@ cargo test -p gitlawb-node
- **TypeScript SDK** (`@gitlawb/sdk`) — client library for the HTTP API
- **Python SDK** (`gitlawb`) — for ML/agent pipeline integration
- **UCAN chain validation** — complete the auth middleware
- **UCAN authorization** — add trusted issuer anchoring, capability checks, and authorization-aware revocation
- **Filecoin storage tier** — wire up cold storage deals
- **Documentation** — guides, tutorials, API examples
- **Node operators** — run a public node and report issues
Generated
+929 -544
View File
File diff suppressed because it is too large. Load diff
+5 -2
View File
@@ -20,8 +20,8 @@ members = [
edition = "2021"
rust-version = "1.91"
license = "MIT OR Apache-2.0"
authors = ["gitlawb contributors"]
repository = "https://github.com/gitlawb/node"
authors = ["Twigpine contributors"]
repository = "https://github.com/Twigpine/node"
[workspace.dependencies]
# async
@@ -53,6 +53,9 @@ chrono = { version = "0.4", features = ["serde"] }
uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core)
url = "2"
# HMAC
hmac = "0.12"
+1
View File
@@ -75,6 +75,7 @@ WORKDIR /data
ENV GITLAWB_REPOS_DIR=/data/repos \
GITLAWB_KEY=/data/keys/identity.pem \
GITLAWB_P2P_KEY=/data/keys/p2p.key \
GITLAWB_HOST=0.0.0.0 \
GITLAWB_PORT=7545 \
GITLAWB_P2P_PORT=7546
+1 -1
View File
@@ -169,7 +169,7 @@
END OF TERMS AND CONDITIONS
Copyright 2026 gitlawb contributors
Copyright 2026 Twigpine contributors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2026 gitlawb contributors
Copyright (c) 2026 Twigpine contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+139 -40
View File
@@ -1,10 +1,10 @@
# Gitlawb Node
# Twigpine Node
**Decentralized git infrastructure for developers, AI agents, and app delivery.**
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Twigpine Node is the open-source node software behind the Twigpine network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Gitlawb is not trying to be only “another git host.” The long-term direction is:
Twigpine is not trying to be only "another git host." The long-term direction is:
```txt
Decentralized GitHub
@@ -19,33 +19,35 @@ The mission is simple: once code is pushed to the network, it should not disappe
## What is in this repository?
This is a Rust workspace with four crates:
This is a Rust workspace with six crates:
| Crate | Purpose |
|---|---|
| `gitlawb-node` | The node daemon: Axum HTTP server, git smart-HTTP, Postgres metadata, libp2p gossip, optional S3/Tigris/IPFS/Arweave/Base PoS hooks. |
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `gl` | The Twigpine CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Twigpine nodes. |
| `gitlawb-core` | Shared primitives: Ed25519 identities, `did:key`, CIDs, RFC 9421 HTTP signatures, certificates, and UCAN tokens. |
| `gitlawb-attest` | Pluggable external provenance attestations for ref-update certificates. |
| `icaptcha-client` | Client for the iCaptcha proof-of-intelligence flow used to protect spam-prone writes. |
---
## Why Gitlawb Nodes?
## Why Twigpine Nodes?
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
Most git hosting today depends on a small number of centralized platforms. Twigpine Nodes are designed for a different model:
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
- **Own your identity**: every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default**: write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport**: repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows**: the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery**: nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction**: the network can evolve from decentralized code storage into code + asset + app delivery.
---
## Current status
Gitlawb Node is live early infrastructure. It is useful today, but some security and reliability features are intentionally staged for compatibility with existing nodes.
Twigpine Node is live early infrastructure. It is useful today, but some security and reliability features are intentionally staged for compatibility with existing nodes.
Good today:
@@ -54,6 +56,7 @@ Good today:
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- Repository and path-scoped visibility enforcement for repository and Git content reads, with 404-shaped repository denials.
- DID identities.
- `gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
@@ -63,11 +66,13 @@ Good today:
Known limitations:
- Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
- Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
- Repository write authorization is secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `true`, so a valid HTTP Signature identifies a pusher and the node additionally requires that pusher to be the repository owner. Set `GITLAWB_ENFORCE_OWNER_PUSH=false` only for the duration of a rolling upgrade whose pushers are not yet the repo owner. Authentication is not authorization on its own (anyone can mint a `did:key` and sign), so owner enforcement is the only thing that prevents every signed caller from pushing to every repository, private ones included.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access.
- Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
- GraphQL mutations require a signed identity (a verified signer bound to the acting DID), are limited to the agent-task queue, and are covered by a CI completeness fence that requires signer acquisition and actor binding on every registered mutation; repo writes are not exposed over GraphQL and remain REST-only.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
See:
@@ -82,7 +87,7 @@ See:
The fastest path is Docker Compose. It starts a node and Postgres.
```bash
git clone https://github.com/Gitlawb/node.git
git clone https://github.com/Twigpine/node.git
cd node
cp .env.example .env
docker compose up -d
@@ -127,10 +132,10 @@ npm install -g @gitlawb/gl
brew install gitlawb/tap/gl
# curl (macOS / Linux)
curl -fsSL https://gitlawb.com/install.sh | sh
curl -fsSL https://twigpine.com/install.sh | sh
# PowerShell (Windows)
irm https://gitlawb.com/install.ps1 | iex
irm https://twigpine.com/install.ps1 | iex
```
Or build from source:
@@ -173,7 +178,7 @@ gl register --node http://localhost:7545
Create a repo:
```bash
gl repo create my-repo --description "My first Gitlawb repo" --node http://localhost:7545
gl repo create my-repo --description "My first Twigpine repo" --node http://localhost:7545
```
Use the git remote helper:
@@ -187,21 +192,21 @@ For public-network use, make sure `GITLAWB_NODE` points to the node you want. Th
### Full lifecycle against an iCaptcha-enforcing node
Public nodes (e.g. `node.gitlawb.com`) require two things on writes:
Public nodes (e.g. `node.twigpine.com`) require two things on writes:
1. **RFC 9421 HTTP Signatures** — every write is signed by your identity key. `gl`
1. **RFC 9421 HTTP Signatures**: every write is signed by your identity key. `gl`
and the `git-remote-gitlawb` helper do this automatically. An old/unsigned CLI
fails with `401 not_an_agent`; `gl` will tell you to upgrade and register.
2. **An iCaptcha proof** on the spam-gated writes (**repo create, fork, register**).
`gl` solves this for you: on the node's `403 icaptcha_proof_required` it reads the
`x-icaptcha-url` / `x-icaptcha-level` hints, requests a challenge, solves it
locally (arithmetic / algebra / sequence), and **retries the same signed request**
with the `x-icaptcha-proof` header — no manual steps, no env vars.
with the `x-icaptcha-proof` header. No manual steps, no env vars.
```bash
gl identity new # create did:key identity
gl register --node https://node.gitlawb.com # signed + auto-solves iCaptcha
gl repo create memlawb --node https://node.gitlawb.com # signed + auto-solves iCaptcha
gl register --node https://node.twigpine.com # signed + auto-solves iCaptcha
gl repo create memlawb --node https://node.twigpine.com # signed + auto-solves iCaptcha
git push origin2 main # origin2 = gitlawb://<your-did>/memlawb (signed)
git clone gitlawb://<your-did>/memlawb # public read, no proof needed
gl doctor # preflight: identity, node, version, iCaptcha
@@ -215,14 +220,61 @@ Notes:
- **Proofs are short-lived (~5 min TTL) and single-use.** If one expires between
solving and use, the client transparently solves a fresh one and retries.
- **What needs what:** create / fork / register are signed **and** iCaptcha-gated;
`git push` is **signed-only** (owner signature is the gate — no per-push challenge);
`git push` is **signed-only** (owner signature is the gate, no per-push challenge);
reads (clone / fetch / `repo info`) need no proof. A non-existent repo returns a
clear `404`, never a placeholder.
- **API-key iCaptcha deployments:** set `GITLAWB_ICAPTCHA_URL` to your iCaptcha
origin and `GITLAWB_ICAPTCHA_API_KEY` to its key. The client only talks to an
`https` origin whose host is allowlisted (that URL or the public default), and
sends the bearer token **only** to your configured origin — never to a URL a
node advertises — so a hostile node can't capture the key or redirect the solve.
sends the bearer token **only** to your configured origin, never to a URL a
node advertises, so a hostile node can't capture the key or redirect the solve.
### Fetching an object by CID
```bash
gl ipfs list # CIDs this node has pinned
gl ipfs get bafkrei... > object.bin # object bytes on stdout
```
Objects that were pinned before the node started recording which repo they came
from are found by scanning its repo inventory, and that scan stops at the
per-request ceilings in the [Configuration](#configuration) table. A stopped scan
answers 503 with a resume token instead of a false "not found", and `gl ipfs get`
follows the token automatically: up to 8 resumes after the first request, so at
most 9 calls to the node, waiting between attempts for as long as the node's
`Retry-After` asks and never longer than 5 seconds.
The whole ladder runs under a 60 second wall-clock deadline. The deadline bounds
the search, not the download: each attempt gets the time left on it to produce
response headers, and once an object is found its bytes stream outside that
deadline. They are not unbounded, though. The client's own 30 second HTTP timeout
is a total request timeout, running from the start of a request until its body
has finished, so a transfer still going 30 seconds after its request began is cut
off. Waits between attempts never run past the deadline either, so a single run
spends at most around 90 seconds on the network: the deadline plus the 30 second
timeout covering the last attempt. Writing the object out sits outside both
bounds, so piping into a reader that stops reading can hold the command open
longer than that.
Two node-side brakes end a ladder early and are reported rather than retried
around. A 429 is terminal, because the node's rate-limit window is an hour and
that wait cannot be honored inside one invocation; a transient overload (a 503
carrying no incomplete-scan code) is retried on the token already held, under the
same cap, clamp and deadline. The per-IP fanout brake can also stop a ladder well
short of the 9 calls, so automatic resumption is not a guarantee of reaching the
object.
When a bound stops the ladder with a usable token in hand, the command prints the
token and the invocation that continues from it before exiting nonzero:
```txt
resume from where this stopped: gl ipfs get bafkrei... --scan <token>
```
Run that to carry on from where the scan stopped. Re-running without `--scan`
restarts at the first row, reproduces the same truncation and spends the node's
per-IP budget again, so the token is the only thing that makes progress. Tokens
are valid for an hour.
---
@@ -309,7 +361,7 @@ POST /api/v1/peers/announce
POST /api/v1/sync/notify
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Staged rollout relaxes who may announce, not who owns a peer row. An unsigned announce may register a previously unseen peer, and it may refresh an existing row whose `http_url` is unchanged, but changing an existing peer's `http_url` requires an RFC 9421 signature from that peer's own DID and is refused with 403 otherwise. An unsigned announce can only register a `did:key`, since that is the only method whose verifying key can be resolved and therefore the only kind of row anyone could ever correct through the signed path; any other DID is refused with 400. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
@@ -317,6 +369,12 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
`POST /api/v1/sync/trigger` is not part of the staged rollout: it always requires a signature in both config modes and returns 401 without one, because each call drives an O(peers) outbound fan-out.
The flag is not HTTP-only. It also gates inbound gossip ref-update events on the libp2p mesh, which carry an Ed25519 payload signature rather than RFC 9421 headers because there are no HTTP headers to sign. Set to `true`, an unsigned gossip event is dropped. Set to `false`, an unsigned event is accepted with a warning during the rolling-upgrade window, and the write budget it consumes is charged to the forwarding peer rather than to the DID it claims, since an unsigned event's claimed DID is asserted and not proven. A present-but-invalid signature is refused in both modes; that is forgery, not an un-upgraded peer.
Because the flag now spans both transports, the rollout order matters in one direction: **upgrade every gossip-publishing peer to a build that signs events before you set this to `true`.** Enabling it while an old publisher is still live drops that publisher's ref-updates on arrival, and the publisher sees no error, because gossip has no response to carry one. Upgrading the HTTP peers alone is not sufficient.
There is a second precondition on gossip ingest that the flag does not control. A publisher's `node_did` must already exist in the receiving node's peers table, or the event is dropped as an unknown peer DID. This holds in both modes, signed and unsigned alike: a valid signature proves key possession, not membership. Rows reach that table over HTTP, either from `GITLAWB_BOOTSTRAP_PEERS` when this node contacts a bootstrap peer and records the DID it reports, or from a prior `POST /api/v1/peers/announce`. So a peer that only ever joined the libp2p mesh, with no HTTP announce and no bootstrap contact in either direction, will have its ref-updates dropped even with a good signature and the flag off. If gossip is silently not landing from a peer you can see on the mesh, check that its DID is in `GET /api/v1/peers` on the receiving side first.
---
## Configuration
@@ -339,22 +397,63 @@ Important node settings:
| `GITLAWB_P2P_PORT` | libp2p QUIC/UDP port. Use `0` to disable. |
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_P2P_KEY` | Path to the node's persistent libp2p identity key file, which fixes the PeerId across restarts. Must name a key file inside a directory, such as `/data/keys/p2p.key` or `./keys/p2p.key`. Two kinds of problem are handled differently. A value that cannot name a securable key file at all (a bare filename, `..` traversal, a trailing `/`, the filesystem root, or a `~/` path that escapes home) is refused before the node binds, so the process exits. Anything wrong with the storage itself (a symlinked or non-directory parent, an unsafe ancestor or working directory, an unreadable or malformed key) leaves p2p off while HTTP keeps serving; the node logs it at error level as `p2p_identity_key_load_failed` and sets the `gitlawb_p2p_identity_key_load_failed` metric to 1. Note that `/health` still reports healthy in that state, so alert on the log event or the metric rather than on the health check. Correcting the storage and restarting restores the same PeerId. On Unix a new key is created `0600` inside a `0700` directory. A key directory that grants access beyond its owner is tightened to `0700` on start; one that is closed too far is refused with a `chmod 700` remedy rather than widened, since a directory locked down on purpose is an operator decision. Rotate the key only when the node logs that the directory was group- or world-writable (or the key file itself was group- or world-readable). An ordinary `0755` directory is tightened too, but others could not read a `0600` key or replace its directory entry. An existing key file is never modified: one with group or other bits set is rejected (`chmod 600`), and so is one this node cannot read. On other platforms no permissions are enforced. Default `~/.gitlawb/p2p.key`; point it at a persistent volume when running in a container. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require peer writes to prove their DID on every transport: RFC 9421 signatures on the announce/sync routes, and an Ed25519 payload signature on gossip ref-update events. Defaults to `false` during the staged rollout below. Upgrade every gossip publisher before enabling, or their updates are dropped with no error on their side. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization (anyone can mint a key and sign), so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack/receive-pack may run before it is aborted (504). Default 600. Does not bound `info/refs` or the withheld-blob path. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
| `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. |
| `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. |
| `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. |
| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Default 8. |
| `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. |
| `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. |
| `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. |
| `GITLAWB_IPFS_MAX_LEGACY_PROBES` | Max legacy (NULL-provenance) repos probed per `/ipfs/{cid}` request, bounding the scan-fallback fan-out. A truncated scan returns a retryable 503, not a false 404. Default 256. |
| `GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS` | Max repo rows one `/ipfs/{cid}` request's legacy scan may read from the database. The probe ceiling above only starts counting once a probe runs, and quarantined or private repos are denied before that, so this is what bounds a scan over an inventory that denies the caller everywhere. A truncated scan sheds a retryable 503 carrying an opaque `continuation` token; echoing it as `?scan=` resumes the scan where it stopped. Every per-request ceiling on this path (rows, probes, visits, retained rule bytes) mints one, so each request advances the ladder by at least the rows it read and a holder buried past a ceiling is reached in a bounded number of requests, `ceil(repos / ceiling) + 1` when this row ceiling is the one that binds. No ceiling ever produces a 404. Every page is charged to the caller's `/ipfs` work allowance, so raising this raises that allowance too. Lowering it sharpens an oracle: because a truncation emits a token and a completed wrap does not, laddering to the end reveals the node's total repo count (private and quarantined included) to within one ceiling. Default 2048. |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max expensive path-scope visibility walks a `/ipfs/{cid}` request may run per phase; over-cap repos are skipped and the scan continues, shedding a retryable 503 (not a false 404) if the object is then found nowhere. The effective cap is the tighter of this knob and the node's internal history-walk ceiling of 17 (`MAX_PIN_SOURCES + 1`), so a value above 17 has no effect while a value below it does tighten the cap. It is charged per phase: the provenance lookup and the legacy-scan fallback get separate equal budgets, so one request can run up to twice the cap in total. Default 64. |
| `GITLAWB_IPFS_MAX_REPO_VISITS` | Ceiling on repos one `/ipfs/{cid}` request may visit (acquire + probe) past the visibility gate. Also the worst-case per-request Tigris fetch count. On exhaustion the scan stops with a retryable 503. Default 1024. |
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
Production note: change the default Postgres password before exposing a node publicly.
Legacy-pin window: releases before the CID-resolver work stored the provider CID (Kubo dag-pb / Pinata) as a pinned object's resolver key. The `/ipfs/{cid}` resolver now recomputes the raw-content CID from the object bytes and refuses to serve a key that does not match, so `GET /api/v1/ipfs/pins` can still advertise an unrepaired legacy CID that 404s. Such a row is repaired opportunistically the next time a push carries the object again (its key is rewritten to the raw CID, the old value kept in `legacy_provider_cid`), but git negotiation omits objects the node already has, so most legacy rows never re-enter a push delta. A deferred one-shot startup sweep, not this opportunistic path, is what fully retires the advertise-then-404 window. Rows whose object bytes are gone stay withheld.
### Upgrading: the PeerId rotates once
This node's libp2p identity is now a keypair generated on first start and kept
at `GITLAWB_P2P_KEY`, rather than one derived from the node DID. Every node
therefore gets a new PeerId once, on the first start after upgrading, and keeps
it from then on as long as that key file survives (put it on a persistent volume
in a container).
Two things to check before upgrading:
- Any `GITLAWB_P2P_BOOTSTRAP` multiaddr that pins a peer's old PeerId with a
`/p2p/<PeerId>` suffix stops matching once that peer upgrades. Update the
suffix, or drop it and let identify supply the current one. Addresses without
the suffix keep working untouched.
- `GITLAWB_P2P_KEY` must name a key file inside a directory, such as
`/data/keys/p2p.key` or `./keys/p2p.key`. A bare filename is refused at
startup, since the node will not keep its identity key in the working
directory.
Peers found over `GITLAWB_BOOTSTRAP_PEERS` and the embedded seed list are
unaffected, since those are HTTP URLs and carry no PeerId.
---
## Optional node staking
Gitlawb Node includes optional Base L2 node-operator hooks. Operators can register a node DID, stake `$GITLAWB`, and post heartbeats.
Twigpine Node includes optional Base L2 node-operator hooks. Operators can register a node DID, stake `$GITLAWB`, and post heartbeats.
PoS is disabled unless these are configured:
@@ -424,8 +523,8 @@ Build:
Output:
```txt
dist/Gitlawb Node.app
dist/Gitlawb Node.dmg
dist/Twigpine Node.app
dist/GitlawbNode-macOS.dmg
```
Features:
@@ -439,7 +538,7 @@ Features:
Unsigned local build:
```bash
xattr -cr "dist/Gitlawb Node.app"
xattr -cr "dist/Twigpine Node.app"
```
---
@@ -454,8 +553,8 @@ Short-term priorities:
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Implement repo write authorization: owner checks, protected branches, and UCAN capability checks.
6. Implement private-read enforcement or remove private repo affordances until it exists.
5. Wire trusted UCAN delegation into repository write authorization.
6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
Product direction:
+36 -26
View File
@@ -2,9 +2,9 @@
## Reporting a Vulnerability
If you discover a security vulnerability in gitlawb, please **do not open a public issue**.
If you discover a security vulnerability in Twigpine, please **do not open a public issue**.
Report it privately by emailing **security@gitlawb.com** with:
Report it privately by emailing **security@twigpine.com** with:
- A description of the vulnerability
- Steps to reproduce
- Potential impact assessment
@@ -28,10 +28,11 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
- Every git object is content-addressed via CIDv1 (SHA-256)
- Tamper-evident by construction — a modified object changes its CID
**UCAN capability tokens**
- Bootstrap UCAN tokens issued at registration
- Capability-scoped: `git:push`, `git:fetch`, `issue:create`, `pr:open`
- JWT-format tokens with expiry
**UCAN token validation**
- Bootstrap UCAN tokens are issued at registration.
- A supplied token's signature, audience, expiry, and proof-chain attenuation are validated.
- Tokens use a signed JSON wire format with expiry.
- Capability grants are not yet consulted by repository write authorization; see the limitations below.
**Smart contracts (Base Sepolia testnet)**
- `GitlawbDIDRegistry` — on-chain DID → document registry
@@ -48,21 +49,20 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
---
## Known Limitations (Planned for v0.2)
## Known Limitations
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
### UCAN chain validation
- The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### Repository write authorization defaults
- `git-receive-pack` verifies HTTP Signatures and, with the default configuration, also requires the pusher DID to be the repository owner: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `true`.
- **Impact:** With the default setting, a valid signature authenticates the pusher *and* the pusher must be the repository owner. Authentication is not authorization on its own — anyone can mint a `did:key` and sign — so owner enforcement is the only thing that prevents every signed caller from pushing to every repository, including private ones.
- **Mitigation:** `GITLAWB_ENFORCE_OWNER_PUSH=false` is the explicit rolling-upgrade compatibility override. Disable it only for the duration of an upgrade whose pushers are not yet the repo owner, then turn it back on. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on.
### UCAN revocation
- Issued UCAN tokens cannot be revoked before expiry.
- **Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### UCAN delegation and revocation
- The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
- Agent lifecycle revocation is not checked by HTTP Signature authorization. Removing or revoking an agent record does not itself block a compromised DID from authenticating.
- **Impact:** Do not use UCANs for collaborator permissions or rely on agent-record revocation as a key-compromise response.
- **Mitigation:** Keep sensitive deployments behind operational network controls and enable owner-only push enforcement where it fits the deployment until trusted delegation and authorization-aware revocation are implemented.
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
@@ -71,10 +71,20 @@ These are **documented, accepted limitations** for the current live release and
- **Fix target:** v0.2
### Private repository reads
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
- **Impact:** Do not store private repositories or secrets on public nodes.
- **Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
- **Fix target:** v0.2
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
- Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
- `GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata.
- Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
### Pull-request review enforcement
- Pull-request review comments are not yet threaded or line-anchored, and merges do not enforce required approvals.
- **Impact:** Teams must use their own review policy or external controls for merge approval requirements.
### GraphQL mutation coverage
- Existing GraphQL mutations require an authenticated signer, but a mutation-specific source-level guardrail has not yet been added for future mutations.
- **Impact:** A new mutation could accidentally omit its signer check without an explicit test fence.
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
@@ -101,20 +111,20 @@ These are **documented, accepted limitations** for the current live release and
| Key storage | PKCS#8 PEM, 0600 permissions |
| Content hashing | SHA-256 via CIDv1 |
| HTTP Signatures | RFC 9421 (Ed25519 + SHA-256 Content-Digest) |
| UCAN tokens | JWT (Ed25519 signatures) |
| UCAN tokens | Signed JSON object (Ed25519 signature) |
| On-chain | ECDSA secp256k1 (Base L2 / Ethereum) |
---
## Threat Model
gitlawb is designed to be secure against:
- **Unauthorized writes** — HTTP Signature auth on all write endpoints
Twigpine is designed to be secure against:
- **Unauthenticated writes** — HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
- **Tampered git objects** — CIDv1 content addressing detects modification
- **Identity spoofing** — DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown** — no single point of control; data on IPFS + Arweave
gitlawb is **not yet** designed to defend against:
Twigpine is **not yet** designed to defend against:
- A compromised node operator (node operators are trusted for their own node)
- Sybil attacks on the DHT (trust score system mitigates, not eliminates)
- Timing attacks on signature verification (not constant-time compared in v0.1)
+5 -5
View File
@@ -1,11 +1,11 @@
{
"$comment": "Canonical seed list for the Gitlawb network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"$comment": "Canonical seed list for the Twigpine network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1,
"updated": "2026-06-30",
"peers": [
{
"name": "gitlawb",
"operator": "Gitlawb",
"operator": "Twigpine",
"did": "did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr",
"http_url": "https://node.gitlawb.com",
"p2p_multiaddr": null,
@@ -13,7 +13,7 @@
},
{
"name": "gitlawb-node2",
"operator": "Gitlawb",
"operator": "Twigpine",
"did": null,
"http_url": "https://node2.gitlawb.com",
"p2p_multiaddr": null,
@@ -21,7 +21,7 @@
},
{
"name": "gitlawb-node3",
"operator": "Gitlawb",
"operator": "Twigpine",
"did": null,
"http_url": "https://node3.gitlawb.com",
"p2p_multiaddr": null,
@@ -29,7 +29,7 @@
},
{
"name": "gitlawb-manila",
"operator": "Gitlawb",
"operator": "Twigpine",
"did": null,
"http_url": "https://manila.gitlawb.com",
"p2p_multiaddr": null,
+18 -1
View File
@@ -11,7 +11,7 @@ name = "git-remote-gitlawb"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
anyhow = { workspace = true }
reqwest = { workspace = true }
tracing = { workspace = true }
@@ -19,3 +19,20 @@ tracing-subscriber = { workspace = true }
[dev-dependencies]
mockito = "1"
tempfile = "3"
[target.'cfg(unix)'.dev-dependencies]
libc = "0.2"
[target.'cfg(windows)'.dev-dependencies]
# Job Object bindings for the real_git_fetch harness: the Windows analog of the
# unix process-group teardown, so the whole `git fetch` tree can be killed
# together on both the timeout and clean-exit paths (INV-22). Win32_Security is
# required by CreateJobObjectW, and Win32_System_Threading by
# JOBOBJECT_EXTENDED_LIMIT_INFORMATION (it embeds IO_COUNTERS).
windows-sys = { version = "0.61", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_System_JobObjects",
"Win32_System_Threading",
] }
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
description = "External Attestation v1: pluggable provenance attachments for Twigpine ref-update certs"
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
+211 -6
View File
@@ -16,7 +16,7 @@
//! by exact match.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64U, Engine};
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use crate::error::{Error, Result};
@@ -105,16 +105,30 @@ impl Attestation {
let bytes = canonical_signing_bytes(&self.type_, &self.payload, &self.cert_hash)?;
let vk = verifying_key_from_did_key(&self.signer)?;
self.verify_sig_with_key(&vk, &bytes)?;
Ok(vk)
}
/// The signature check itself, against a key the caller supplies.
///
/// Split out so the STRICTNESS of the check stays provable on its own.
/// `verifying_key_from_did_key` now refuses a small-order key, so a weak
/// signer can no longer reach this line through `verify_signature`. Without
/// this seam there would be nothing left that fails when `verify_strict` is
/// downgraded to the non-strict `verify`, and that regression would land
/// silently. Check order in `verify_signature` is unchanged.
fn verify_sig_with_key(&self, vk: &VerifyingKey, bytes: &[u8]) -> Result<()> {
let sig_bytes: [u8; 64] = B64U
.decode(&self.sig)
.map_err(|e| Error::Signature(format!("base64url: {e}")))?
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify(&bytes, &sig)
vk.verify_strict(bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(vk)
Ok(())
}
/// Reparse `payload` as `P`. Errors if the type discriminator does not
@@ -181,7 +195,7 @@ fn canonical_signing_bytes(
Ok(out)
}
fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
pub(crate) fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&key.to_bytes());
@@ -202,6 +216,19 @@ fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
"did:key must use base58btc (z-prefix): {did}"
)));
}
// Refuse an oversized id before decoding. base58 decoding is quadratic in
// its input and `signer` comes off an attacker-supplied attestation, so a
// short request could otherwise buy a large decode. An ed25519 did:key
// method-id is a fixed 48 characters, so this bound is slack rather than a
// behavior change, and it only helps ahead of the decode. Mirrors the same
// cap in gitlawb-core's Did::to_verifying_key, which this crate cannot call
// (gitlawb-core is a dev-dependency here).
const MAX_METHOD_ID_LEN: usize = 64;
if method_id.len() > MAX_METHOD_ID_LEN {
return Err(Error::Did(
"did:key method-specific id too long".to_string(),
));
}
let (base, bytes) =
multibase::decode(method_id).map_err(|e| Error::Did(format!("multibase: {e}")))?;
if base != multibase::Base::Base58Btc {
@@ -217,8 +244,20 @@ fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
let key_bytes: [u8; 32] = bytes[ED25519_MULTICODEC.len()..]
.try_into()
.expect("length checked above");
VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))
let key = VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))?;
// `from_bytes` only decompresses, so it accepts a small-order point. This
// crate parses did:key itself rather than going through gitlawb-core's
// `Did`, and it cannot go through it (gitlawb-core is a dev-dependency
// here), so the rejection is mirrored rather than inherited. Signature
// verification is already strict, so this is defense in depth against a
// future consumer of this parser that does not verify strictly.
if key.is_weak() {
return Err(Error::Did("small-order ed25519 key".to_string()));
}
Ok(key)
}
#[cfg(test)]
@@ -465,6 +504,23 @@ mod tests {
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_oversized_method_id_before_decoding() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
// A well-formed prefix on an absurdly long id must fail the length
// bound, not the decoder: base58 decode cost is quadratic in input
// length, so reaching it at all is the defect.
att.signer = format!("did:key:z{}", "1".repeat(256));
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
// Match the bound's own message: any other Did error (a decode or
// length mismatch downstream) means the oversized input was decoded.
match err {
Error::Did(m) => assert!(m.contains("too long"), "unexpected: {m}"),
e => panic!("expected Error::Did, got {e:?}"),
}
}
#[test]
fn verify_rejects_bad_signature_base64() {
let sk = fresh();
@@ -554,4 +610,153 @@ mod tests {
let err = tampered.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
/// Regression guard for strict verification: a signature forged under a
/// weak (small-order) public key satisfies the verification equation
/// but must be rejected. The identity point is such a key: with R the
/// identity and S = 0, [S]B - [k]A is the identity for any message.
/// gitlawb-attest parses did:key itself and never routes through
/// gitlawb-core's `Did`, which it cannot: gitlawb-core is a dev-dependency
/// here. So the choke-point rejection has to be mirrored in this parser or
/// this crate keeps handing out keys the rest of the system rejects.
#[test]
fn verifying_key_from_did_key_rejects_a_small_order_key() {
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak);
let did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err =
verifying_key_from_did_key(&did).expect_err("a small-order did:key must not resolve");
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"rejection must name the small-order key, got: {msg}"
),
other => panic!("expected Error::Did, got {other:?}"),
}
}
/// base58 decoding is quadratic in its input and `signer` is attacker
/// controlled, so an oversized method-id must be refused BEFORE the decode
/// or a hostile attestation buys a large decode for a short request. The
/// assertion is on the length error specifically: a multibase error here
/// would mean the cap ran too late to matter.
#[test]
fn verifying_key_from_did_key_refuses_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "1".repeat(65_536));
let err = verifying_key_from_did_key(&oversized)
.expect_err("an oversized method-id must be refused");
match err {
Error::Did(msg) => assert!(
msg.contains("too long"),
"must fail on the length cap, not after decoding: {msg}"
),
other => panic!("expected Error::Did, got {other:?}"),
}
}
/// Strictness regression guard. The parser now refuses a small-order key,
/// so `verify_signature` can no longer carry a weak signer down to the
/// signature check; this drives that check directly to prove it is still
/// STRICT. Downgrading `verify_strict` to `verify` makes this go red,
/// which nothing else in the suite would catch.
#[test]
fn verify_sig_with_key_is_strict_about_a_weak_key() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak());
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
// R = identity, S = 0: satisfies the non-strict verification equation
// for any message under a small-order key.
let mut forged = [0u8; 64];
forged[0] = 1;
att.sig = B64U.encode(forged);
let bytes = canonical_signing_bytes(&att.type_, &att.payload, &att.cert_hash).expect("jcs");
let err = att
.verify_sig_with_key(&weak_vk, &bytes)
.expect_err("strict verification must reject a small-order key");
assert!(
matches!(err, Error::Signature(_)),
"expected a signature error, got {err:?}"
);
}
/// Accept control for the seam: a genuine signature still verifies through
/// the same helper, so the guard above is not simply rejecting everything.
#[test]
fn verify_sig_with_key_accepts_a_genuine_signature() {
let cert_hash = sample_cert_hash();
let sk = SigningKey::generate(&mut OsRng);
let att = dummy_attestation(&sk, cert_hash);
let bytes = canonical_signing_bytes(&att.type_, &att.payload, &att.cert_hash).expect("jcs");
att.verify_sig_with_key(&sk.verifying_key(), &bytes)
.expect("a real signature must verify");
}
/// Control for the guard above: a real did:key must still resolve.
#[test]
fn verifying_key_from_did_key_still_accepts_a_real_key() {
let sk = SigningKey::from_bytes(&[3u8; 32]);
let vk = sk.verifying_key();
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&vk.to_bytes());
let did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let got = verifying_key_from_did_key(&did).expect("a real did:key must resolve");
assert_eq!(got.to_bytes(), vk.to_bytes());
}
#[test]
fn verify_rejects_weak_key_signature() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let cert_hash = sample_cert_hash();
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let forged_sig = B64U.encode(forged);
// Build an attestation with the weak key as signer and the forged
// signature. Rejection now happens at DID RESOLUTION rather than at
// verify time: `verifying_key_from_did_key` refuses a small-order key
// before `verify_strict` is reached, so the observed error is
// `Error::Did`, not `Error::Signature`. `verify_strict` remains the
// second layer for malleability cases that do not involve a weak
// public key (a small-order R under an honest key), which this fixture
// cannot construct and therefore no longer covers.
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak_key_bytes);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
att.sig = forged_sig;
let err = att.verify_signature(cert_hash).unwrap_err();
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"weak signer must be refused as a small-order key, got: {msg}"
),
other => panic!("expected Error::Did for a small-order signer, got {other:?}"),
}
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! Pluggable provenance attachments for gitlawb ref-update certs.
//! Pluggable provenance attachments for Twigpine ref-update certs.
//!
//! A `gitlawb/ref-update/v1` cert proves who pushed and where a ref moved to.
//! It does not say how the commit was produced. This crate adds an optional
+183 -14
View File
@@ -121,7 +121,12 @@ impl Registry {
attestation: &Attestation,
expected_cert_hash: [u8; 32],
) -> Result<VerifiedAttestation> {
attestation.verify_signature(expected_cert_hash)?;
// The key that actually verified is the trust anchor for the signer a
// consumer sees. `verify_signature` recovers it from the artifact's
// `signer` field and rejects a field that does not match the signing
// key, so this is the canonical DID of the key that signed — never a
// raw artifact string that could disagree with it.
let verified_key = attestation.verify_signature(expected_cert_hash)?;
let fully = match self.by_type.get(attestation.type_.as_str()) {
Some(v) => {
@@ -138,30 +143,55 @@ impl Registry {
Ok(VerifiedAttestation {
type_: attestation.type_.clone(),
signer: attestation.signer.clone(),
signer: crate::attestation::did_key_from_verifying_key(&verified_key),
cert_hash: attestation.cert_hash.clone(),
fully_verified: fully,
})
}
/// Verify a batch, then enforce `RequireAll`.
///
/// Under [`Policy::RequireAll`] this never short-circuits: an attestation
/// that fails to verify — bad signature, cert-hash mismatch, or malformed
/// payload, the actual `Err` paths of [`Registry::verify`] — is dropped
/// from the result rather than aborting the batch. A valid attestation of
/// a type with NO registered verifier is not one of those: under
/// `RequireAll` it verifies to `Ok` with `fully_verified: false` and stays
/// in the result, preserving the distinction between *untrusted* and
/// *invalid* (see `require_all_is_lenient_on_unknown_types_in_the_batch`).
/// Any third party can attach an attestation, so a hard error here would
/// let an unrelated `attacker/spam/v1` deny-of-service the whole cert (the
/// DoS vector the module docstring warns about). The `required_types`
/// check below is the real gate, and it only counts `fully_verified`
/// entries — so a required type represented solely by a malformed
/// attestation still fails with [`Error::RequiredMissing`].
///
/// Under `AcceptKnown` and `RejectUnknown` the batch stays strict: the
/// first verification error is surfaced to the caller.
pub fn verify_all(
&self,
attestations: &[Attestation],
expected_cert_hash: [u8; 32],
) -> Result<Vec<VerifiedAttestation>> {
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
if self.policy != Policy::RequireAll {
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
return Ok(verified);
}
if self.policy == Policy::RequireAll {
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
}
// RequireAll: drop entries that fail to verify instead of aborting.
let verified: Vec<VerifiedAttestation> = attestations
.iter()
.filter_map(|a| self.verify(a, expected_cert_hash).ok())
.collect();
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
}
}
Ok(verified)
@@ -171,7 +201,7 @@ impl Registry {
#[cfg(test)]
mod tests {
use super::*;
use crate::attestation::{Attestation, AttestationPayload};
use crate::attestation::{did_key_from_verifying_key, Attestation, AttestationPayload};
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
@@ -319,6 +349,38 @@ mod tests {
.any(|v| v.type_ == "other/v1" && !v.fully_verified));
}
/// The strict policies must surface a batch member's verification error
/// through `verify_all`, not launder it into a partial result. This binds
/// the POLICY BOUNDARY itself: only the `RequireAll` arm may drop failing
/// entries, and an apparently harmless cleanup that applies its
/// `filter_map(Result::ok)` shape to the strict arms would silently turn
/// the fail-closed contracts into fail-open — with every per-attestation
/// `verify` test still green, because none of them go through the batch
/// path. The failing entry is a wrong-cert-hash attestation (signed over a
/// different hash than the batch expects), a genuine `Err` under every
/// policy.
#[test]
fn strict_policies_propagate_a_batch_members_error() {
let sk = fresh();
let cert_hash = sample_hash();
let mut wrong_hash = sample_hash();
wrong_hash[0] ^= 0xff;
for policy in [Policy::AcceptKnown, Policy::RejectUnknown] {
let mut reg = Registry::new().with_policy(policy);
reg.register(DemoVerifier);
let good = signed_demo(&sk, cert_hash, "ok");
let bad = signed_demo(&sk, wrong_hash, "ok");
let err = reg
.verify_all(&[good, bad], cert_hash)
.expect_err("a strict policy must fail the whole batch");
assert!(
matches!(err, Error::Signature(_) | Error::CertHashMismatch),
"expected the underlying verification error, got {err:?}"
);
}
}
/// A required type that is present but unverified (no verifier registered
/// even though the type appears in `required_types`) must fail
/// `verify_all` with `RequiredMissing`. Catches a misconfiguration where
@@ -336,6 +398,65 @@ mod tests {
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
/// A cert-hash that does not match `sample_hash()`, so an attestation
/// signed against it fails `verify_signature` when the batch is checked
/// against `sample_hash()`.
fn other_hash() -> [u8; 32] {
let mut h = sample_hash();
h[0] ^= 0xff;
h
}
/// Under `RequireAll`, a malformed attestation (here: signed against the
/// wrong cert hash, so it fails signature verification) attached alongside a
/// valid required one must not abort the batch. The junk is dropped and the
/// cert still verifies. Before the fix, `verify_all` propagated the error
/// via `?` and the whole batch failed — the DoS vector the docstring warns
/// about.
#[test]
fn require_all_drops_a_malformed_extra_and_still_accepts() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// Signed against `other_hash()`, so it will not verify under `cert_hash`.
let malformed = signed_demo(&sk, other_hash(), "boom");
let real = signed_demo(&sk, cert_hash, "ok");
let verified = reg
.verify_all(&[malformed, real], cert_hash)
.expect("a malformed extra must not abort the batch under RequireAll");
assert_eq!(verified.len(), 1, "the malformed entry should be dropped");
assert!(verified
.iter()
.any(|v| v.type_ == "demo/v1" && v.fully_verified));
}
/// Leniency must not open a hole: if the *only* attestation of a required
/// type is malformed, dropping it leaves the required type absent and
/// `verify_all` must still fail with `RequiredMissing`. Guards against a fix
/// that skips errors so eagerly it lets an unverified required type pass.
#[test]
fn require_all_rejects_when_only_a_malformed_required_type_is_present() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// The sole demo/v1 is bound to the wrong cert hash → fails verification.
let malformed_required = signed_demo(&sk, other_hash(), "ok");
let err = reg
.verify_all(&[malformed_required], cert_hash)
.unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
#[test]
fn payload_check_failure_rejects() {
let sk = fresh();
@@ -374,4 +495,52 @@ mod tests {
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// The signer a consumer sees must be the canonical DID of the key that
/// actually verified, never a raw artifact string that could disagree.
/// `verify_signature` derives the key from the artifact's `signer` field
/// and the parser is canonical (base58btc, exact length, ed25519
/// multicodec), so a forged field fails verification outright: the
/// must-not here is that no `VerifiedAttestation` ever carries a signer
/// that did not verify.
#[test]
fn verified_signer_is_anchored_to_the_key_that_verified() {
let signer = fresh();
let other = fresh();
assert_ne!(
signer.verifying_key().to_bytes(),
other.verifying_key().to_bytes(),
"fixture precondition: distinct signing keys"
);
let cert_hash = sample_hash();
let reg = Registry::new();
// Control: an honest artifact reports the signer's canonical DID, and
// it agrees with the artifact field.
let att = signed_demo(&signer, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
let expected = did_key_from_verifying_key(&signer.verifying_key());
assert_eq!(v.signer, expected, "signer must be the verified key's DID");
assert_eq!(v.signer, att.signer, "honest artifact: field and key agree");
// Must-not, both directions: a signer field naming a DIFFERENT key
// than the one that signed must fail verification, so the artifact
// string can never redirect the reported signer.
let mut forged = signed_demo(&signer, cert_hash, "ok");
forged.signer = did_key_from_verifying_key(&other.verifying_key());
let err = reg.verify(&forged, cert_hash).unwrap_err();
assert!(
matches!(err, Error::Signature(_)),
"a signer field that does not match the signing key must fail, got {err:?}"
);
let mut forged_rev = signed_demo(&other, cert_hash, "ok");
forged_rev.signer = did_key_from_verifying_key(&signer.verifying_key());
let err_rev = reg.verify(&forged_rev, cert_hash).unwrap_err();
assert!(
matches!(err_rev, Error::Signature(_)),
"mismatched signer field must fail verification, got {err_rev:?}"
);
}
}
+8 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
description = "Core cryptographic primitives for the Twigpine network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
@@ -22,6 +22,7 @@ multihash-codetable = { workspace = true }
cid = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true, optional = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
@@ -30,3 +31,9 @@ chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
# Not optional here: the redirect matrix must run under a bare
# `cargo test -p gitlawb-core`, with no feature selected.
url = { workspace = true }
[features]
redirect = ["dep:url"]
+192 -6
View File
@@ -5,6 +5,30 @@
//!
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
//!
//! ## Replay
//!
//! A certificate proves *who* asserted a ref update and *what* they asserted.
//! It does not prove the assertion is fresh. [`RefUpdateBody::seq`] and
//! [`RefUpdateBody::nonce`] are inside the signed bytes (see
//! [`RefUpdateBody::to_signing_bytes`]), so a verifier can trust the values,
//! but deciding whether a nonce is being reused or a `seq` is going backwards
//! needs state.
//!
//! This crate is deliberately stateless — it is embedded in both nodes and
//! clients — so replay is **not** enforced here and cannot be. Ownership of
//! the check sits with the verifier that accepts and persists certificates,
//! which must:
//!
//! - reject a `nonce` it has already accepted for the same `(repo, ref_name)`;
//! - reject a `seq` that does not advance past the last accepted `seq` for
//! that `(repo, ref_name)`.
//!
//! Until such a verifier exists, replay of ref-update certificates is not
//! enforced anywhere in this workspace. This contract exists so the first one
//! inherits the requirement instead of re-deriving it.
use std::collections::HashSet;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
@@ -39,11 +63,19 @@ pub struct RefUpdateBody {
pub from: String,
/// The new commit hash (SHA-256 hex).
pub to: String,
/// Monotonically increasing sequence number. Prevents replay.
/// Monotonically increasing sequence number, scoped to `(repo, ref_name)`.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a `seq` that does not
/// advance for the same repo and ref. See the module docs on replay.
pub seq: u64,
/// RFC 3339 timestamp of this update.
pub timestamp: DateTime<Utc>,
/// Random nonce for deduplication.
/// Random nonce, unique per certificate.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a nonce it has already
/// accepted for the same repo and ref. See the module docs on replay.
pub nonce: String,
}
@@ -135,10 +167,23 @@ impl RefUpdateCert {
/// Check if this certificate satisfies a threshold of valid signatures
/// from the provided set of authorized maintainer DIDs.
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once.
///
/// `threshold` must be at least 1. Zero is rejected instead of being
/// treated as "no signatures required": a caller that derives the
/// threshold from configuration and gets 0 would otherwise accept any
/// certificate, whether its signatures verify or not.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
if threshold == 0 {
return Err(Error::RefCert("threshold must be at least 1".to_string()));
}
let valid = self.verify_all()?;
let count = valid.iter().filter(|d| maintainers.contains(d)).count();
Ok(count >= threshold)
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
Ok(distinct_signers.len() >= threshold)
}
/// Validate the certificate structure (not signatures).
@@ -345,7 +390,34 @@ mod tests {
}
#[test]
fn threshold_zero_is_always_satisfied() {
fn satisfies_threshold_rejects_duplicated_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let kp3 = Keypair::generate();
let repo_did = kp1.did();
let mut cert = RefUpdateCert::new(
repo_did,
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Copy-paste the only real signature onto the certificate a second
// time. Same signer, same valid signature, still one real signer.
let dup = cert.signatures[0].clone();
cert.signatures.push(dup);
let maintainers = vec![kp1.did(), kp2.did(), kp3.did()];
// Two signature entries but a single distinct signer must not
// satisfy a 2-of-3 threshold.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_rejected() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
@@ -356,6 +428,120 @@ mod tests {
&kp,
)
.unwrap();
assert!(cert.satisfies_threshold(&[], 0).unwrap());
// Zero must never read as "no signatures required", even with an empty
// maintainer set.
let err = cert.satisfies_threshold(&[], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"zero must be refused as an invalid threshold"
);
}
/// The threshold is checked before any signature work, so a zero is
/// reported as a threshold error even when the certificate's signatures
/// are also invalid. Otherwise a broken certificate would let a caller
/// mistake the failure for a legitimate verification result.
#[test]
fn threshold_zero_is_rejected_before_signature_verification() {
let kp = Keypair::generate();
let other = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures[0].sig = other.sign_b64(b"unrelated bytes");
let err = cert.satisfies_threshold(&[kp.did()], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"the invalid threshold must win over the invalid signature"
);
}
/// `seq` and `nonce` are the verifier's replay inputs, so they have to be
/// inside the signed body; tapping either one must invalidate the
/// certificate rather than leave it verifying with different values.
#[test]
fn replay_fields_are_covered_by_the_signature() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
7,
&kp,
)
.unwrap();
cert.verify_all().expect("baseline must verify");
let mut bumped = cert.clone();
bumped.body.seq = 8;
assert!(
matches!(bumped.verify_all(), Err(Error::SignatureInvalid)),
"changing seq must invalidate the signature"
);
let mut renamed = cert.clone();
renamed.body.nonce = "different-nonce".to_string();
assert!(
matches!(renamed.verify_all(), Err(Error::SignatureInvalid)),
"changing nonce must invalidate the signature"
);
}
/// A signature minted by a real key must not verify when the signer is
/// spelled in another multibase alphabet. Otherwise the same identity
/// would satisfy a maintainer allowlist under several DID strings.
///
/// Base64url is deliberate: its 47-character method-id fits under the
/// pre-existing length bound, so this test reaches the canonical-prefix
/// rule and fails on the pre-fix code instead of being refused for length.
#[test]
fn verify_all_rejects_a_non_canonical_did_spelling() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let mut prefixed = vec![0xed, 0x01];
prefixed.extend_from_slice(&kp.verifying_key().to_bytes());
cert.signatures[0].signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base64Url, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = cert
.verify_all()
.expect_err("a non-canonical did:key spelling must not resolve");
assert!(
err.to_string().contains("base58btc"),
"rejection must come from the canonical-prefix rule, not the length guard"
);
assert!(
!matches!(
cert.satisfies_threshold(&[cert.signatures[0].signer.clone()], 1),
Ok(true)
),
"the alternate spelling must not satisfy a threshold"
);
// The canonical spelling of the same key is unaffected.
cert.signatures[0].signer = kp.did();
assert!(cert.satisfies_threshold(&[kp.did()], 1).unwrap());
}
}
+45
View File
@@ -64,6 +64,19 @@ impl Cid {
}
}
/// True when `s` parses as a CIDv1 with the raw codec — the exact shape
/// [`Cid::from_git_object_bytes`] produces and the `/ipfs` resolver looks up.
/// A legacy provider CID (Kubo dag-pb, Pinata CIDv0) parses to a different
/// version or codec and returns `false`, marking it an opportunistic-repair
/// candidate. Decidable from the string alone (no object bytes), so the pin path
/// can gate the byte-read/recompute cost on it and leave non-legacy rows at the
/// existing DB-only skip cost. An unparseable string is non-canonical (`false`).
pub fn is_raw_cidv1(s: &str) -> bool {
s.parse::<CidGeneric<64>>()
.map(|c| c.version() == cid::Version::V1 && c.codec() == RAW)
.unwrap_or(false)
}
impl fmt::Display for Cid {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
@@ -177,6 +190,38 @@ mod tests {
assert!(result.is_err());
}
#[test]
fn is_raw_cidv1_classifies_codec_from_string() {
// The canonical resolver key: CIDv1 + raw codec → not a repair candidate.
let raw = Cid::from_git_object_bytes(b"blob 5\0hello");
assert!(
is_raw_cidv1(raw.as_str()),
"from_git_object_bytes output is CIDv1/raw"
);
// A CIDv0 (Pinata dag-pb legacy shape) → repair candidate.
assert!(
!is_raw_cidv1("QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG"),
"a CIDv0 dag-pb value is a legacy-repair candidate"
);
// A CIDv1 with the dag-pb codec (the Kubo above-block-size root) over the
// same multihash → still a repair candidate (codec, not just version).
let parsed = raw.as_str().parse::<CidGeneric<64>>().unwrap();
const DAG_PB: u64 = 0x70;
let dagpb = CidGeneric::<64>::new_v1(DAG_PB, *parsed.hash()).to_string();
assert!(
!is_raw_cidv1(&dagpb),
"a CIDv1 dag-pb value is a legacy-repair candidate"
);
// Garbage is non-canonical.
assert!(
!is_raw_cidv1("not-a-cid"),
"an unparseable string is non-canonical"
);
}
#[test]
fn sha256_hex_of_empty_input_is_well_known() {
// SHA-256("") is a fixed constant; verifies the hasher is wired correctly.
+273 -6
View File
@@ -1,11 +1,11 @@
//! Decentralized Identifier (DID) types for the gitlawb network.
//! Decentralized Identifier (DID) types for the Twigpine network.
//!
//! Supported methods:
//! - `did:key` — ephemeral, derived directly from an Ed25519 public key
//! - `did:web` — anchored to a domain
//! - `did:gitlawb` — native, anchored to the libp2p DHT
//!
//! The canonical DID for a gitlawb actor is `did:key` during bootstrap
//! The canonical DID for a Twigpine actor is `did:key` during bootstrap
//! and migrates to `did:gitlawb` once DHT anchoring is live.
use ed25519_dalek::VerifyingKey;
@@ -20,6 +20,19 @@ use crate::{Error, Result};
/// Value: 0xed01 encoded as a varint.
const ED25519_MULTICODEC: &[u8] = &[0xed, 0x01];
/// The only multibase prefix a `did:key` method-specific id may carry.
///
/// The DID:key spec encodes Ed25519 keys as base58btc, which is multibase
/// `z`. `multibase::decode` happily accepts every other alphabet, and each
/// spelling of the same key bytes would produce a distinct DID string while
/// gates, allowlists, and database rows compare DIDs as strings — one
/// identity would read as several. Rejecting the alternatives here keeps the
/// mapping from `did:key` to key single-valued for every caller of
/// [`Did::to_verifying_key`] (HTTP signatures, UCAN, ref certs, peer
/// announces). `gitlawb-attest` enforces the same rule for attestation
/// signers.
const DID_KEY_MULTIBASE_PREFIX: char = 'z';
/// A Decentralized Identifier.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)]
@@ -71,7 +84,14 @@ impl Did {
/// Resolve the Ed25519 verifying key from a `did:key`.
///
/// Returns `Err` if the DID is not a `did:key` or the key bytes are invalid.
/// Only the canonical base58btc (`z`) spelling is accepted. Other
/// multibase encodings decode to the same key but spell a different DID
/// string, so accepting them would let one identity appear under several
/// identifiers to string-comparing gates and allowlists.
///
/// Returns `Err` if the DID is not a `did:key`, uses a non-canonical
/// multibase prefix, the key bytes are invalid, or the key is a
/// small-order (weak) point.
pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() {
return Err(Error::InvalidDid(format!(
@@ -80,9 +100,40 @@ impl Did {
)));
}
let (_, bytes) =
// Refuse an oversized id before decoding. base58 decoding is quadratic
// in its input, and both callers of this function take the string from
// an untrusted request: the unauthenticated peer-announce gate, and the
// HTTP-signature keyid. An ed25519 did:key method-id is a fixed 48
// characters, so this bound is slack rather than a behavior change, and
// it has to sit ahead of the decode to be worth anything.
const MAX_METHOD_ID_LEN: usize = 64;
if self.method_id().len() > MAX_METHOD_ID_LEN {
return Err(Error::InvalidDid(
"did:key method-specific id too long".to_string(),
));
}
// Cheap prefix gate before the decode, then the base check on what
// multibase actually parsed. The first keeps a clearly non-canonical id
// off the decoder. The second is belt-and-braces — multibase maps a
// leading `z` only to `Base58Btc` — but it states the requirement
// directly instead of relying on that mapping, and `gitlawb-attest`
// makes the same pair of checks.
if !self.method_id().starts_with(DID_KEY_MULTIBASE_PREFIX) {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
let (base, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
if base != Base::Base58Btc {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
if !bytes.starts_with(ED25519_MULTICODEC) {
return Err(Error::InvalidDid(
"not an ed25519 multicodec key".to_string(),
@@ -93,7 +144,25 @@ impl Did {
.try_into()
.map_err(|_| Error::InvalidDid("ed25519 key must be 32 bytes".to_string()))?;
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))
let key =
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))?;
// `from_bytes` only decompresses, so it accepts a small-order point.
// Such a key satisfies the verification equation for any message, and
// every X25519 shared secret derived from it is the all-zero key: the
// order-1 and order-2 points convert to the all-zero Montgomery u
// directly, and the order-4 and order-8 points are annihilated by the
// scalar clamping instead. `is_weak` covers the whole torsion set.
// Mixed-order points are correctly NOT rejected, since clamping clears
// the cofactor component. Rejecting
// at resolution is what makes this the choke point: every consumer that
// resolves a DID through here inherits the rejection, and a recipient
// that cannot resolve already fails closed downstream.
if key.is_weak() {
return Err(Error::InvalidDid("small-order ed25519 key".to_string()));
}
Ok(key)
}
/// Return the full DID string as a `&str`.
@@ -139,7 +208,7 @@ impl From<Did> for String {
}
}
/// A DID Document for a gitlawb actor.
/// A DID Document for a Twigpine actor.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DidDocument {
#[serde(rename = "@context")]
@@ -308,4 +377,202 @@ mod tests {
assert_eq!(Did::web("example.com").method_id(), "example.com");
assert_eq!(Did::gitlawb("z6MkSomeKey").method_id(), "z6MkSomeKey");
}
/// An oversized method-specific id is refused on its length, before the
/// base58 decode runs.
///
/// base58 decoding is quadratic in the input, and `to_verifying_key` is
/// reached from two places that take an attacker-supplied string: the
/// unauthenticated peer-announce gate, and the HTTP-signature keyid. A
/// 64k-character DID measured 8s through the router without this guard
/// against 4.6ms with it, on a tokio worker with no spawn_blocking, so a
/// single caller could buy CPU-seconds by the thousand.
///
/// Asserting the MESSAGE is what makes this load-bearing: an oversized id
/// errors either way, so a test that only checked `is_err()` would pass
/// with the guard removed, having merely paid the cost it exists to avoid.
#[test]
fn to_verifying_key_rejects_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "z".repeat(64_000));
let did: Did = oversized.parse().expect("method is still `key`");
let start = std::time::Instant::now();
let err = did.to_verifying_key().expect_err("must be refused");
let elapsed = start.elapsed();
assert!(
err.to_string().contains("too long"),
"must be refused on length, not after the decode: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(1),
"refusal must not pay the quadratic decode: took {elapsed:?}"
);
}
/// The bound must not reject a real key. Every `did:key` this project
/// mints is a fixed 48 characters, so the limit is slack, not a behavior
/// change.
#[test]
fn to_verifying_key_still_accepts_a_real_did_key() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[7u8; 32]);
let did = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
did.method_id().len(),
48,
"did:key method-id is fixed-width"
);
did.to_verifying_key().expect("a real did:key must resolve");
}
/// `multibase::decode` accepts every alphabet, so the same Ed25519 key can
/// be spelled as several `did:key` strings. Only base58btc is canonical;
/// the rest must be refused so one identity cannot appear as several to
/// gates and allowlists that compare DID strings.
#[test]
fn to_verifying_key_rejects_non_base58btc_multibase() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[9u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let canonical = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
canonical.to_verifying_key().unwrap().to_bytes(),
signing.verifying_key().to_bytes(),
"base58btc is the accepted spelling"
);
// These spellings fit inside the method-id bound, so they reach the
// canonical-prefix check and must be refused there.
for base in [Base::Base64Url, Base::Base32Lower] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert_ne!(
did.as_str(),
canonical.as_str(),
"the same key must not compare equal under another spelling"
);
let err = did
.to_verifying_key()
.expect_err("non-canonical multibase must be refused");
assert!(
matches!(err, Error::InvalidDid(_)),
"must be an InvalidDid, got {err:?}"
);
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
// A base16 spelling of the same key is longer than the method-id
// bound, so it is refused by the length guard before the prefix check.
// Either way it must not resolve.
for base in [Base::Base16Lower, Base::Base16Upper] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert!(
matches!(did.to_verifying_key(), Err(Error::InvalidDid(_))),
"a base16 spelling of a real key must not resolve"
);
}
}
/// base58flickr (`Z`) is the non-bitcoin sibling alphabet. Its spelling of
/// a real key fits under the length bound, so it must be refused for the
/// prefix, not merely for its size.
#[test]
fn to_verifying_key_rejects_a_base58flickr_spelling() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[11u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!(
"did:key:{}",
multibase::encode(Base::Base58Flickr, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("base58flickr must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// An empty method-specific id must be refused by the prefix rule without
/// panicking on a missing first character.
#[test]
fn to_verifying_key_rejects_an_empty_method_id() {
let did: Did = "did:key:".parse().expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("an empty method id must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// A canonical base58btc spelling is not enough on its own: the bytes must
/// also carry the ed25519-pub multicodec. This pins the check that stops a
/// key of another type from resolving as an Ed25519 key.
#[test]
fn to_verifying_key_rejects_base58btc_with_a_wrong_multicodec() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[12u8; 32]);
let mut buf = vec![0x12, 0x00]; // sha2-256, not ed25519-pub
buf.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!("did:key:{}", multibase::encode(Base::Base58Btc, &buf))
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("a non-ed25519 multicodec must be refused");
assert!(
err.to_string().contains("ed25519"),
"the error must name the expected multicodec: {err}"
);
}
/// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key
/// satisfies the verification equation for any message and, converted to
/// Montgomery form for X25519, yields the all-zero shared secret, so an
/// envelope sealed to it is decryptable by anyone. Resolution is the choke
/// point every `Did`-based consumer shares, so it fails closed here.
#[test]
fn to_verifying_key_rejects_a_small_order_key() {
// The compressed identity point, the canonical small-order key.
let mut weak = [0u8; 32];
weak[0] = 1;
let mut prefixed = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
prefixed.extend_from_slice(ED25519_MULTICODEC);
prefixed.extend_from_slice(&weak);
let did: Did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &prefixed)
)
.parse()
.expect("a small-order did:key is still well-formed");
let err = did
.to_verifying_key()
.expect_err("a small-order did:key must not resolve");
match err {
Error::InvalidDid(msg) => assert!(
msg.contains("small-order"),
"rejection must name the small-order key, got: {msg}"
),
other => panic!("expected Error::InvalidDid, got {other:?}"),
}
}
}
+501 -2
View File
@@ -11,6 +11,22 @@ use zeroize::Zeroizing;
/// X25519 public key (Montgomery u) for an Ed25519 verifying key.
fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
use curve25519_dalek::edwards::CompressedEdwardsY;
// Every X25519 shared secret derived from a small-order point is the
// all-zero key: order-1 and order-2 convert to the all-zero Montgomery u
// directly, order-4 and order-8 are annihilated by the scalar clamping in
// x25519_secret_from_seed. Either way the per-recipient wrap could be
// rebuilt by anyone. Resolution already refuses such a key (see
// Did::to_verifying_key); this guard covers the SEAL side on its own terms
// for a caller that obtained the key some other way. The open side
// (open_blob's attacker-supplied `eph`) skips any entry whose exchange with
// this reader yields the all-zero shared secret
// (`yields_all_zero_shared_secret`), regardless of whether the bytes decode
// to an Edwards point.
if vk.is_weak() {
return Err(anyhow::anyhow!("verifying key is a small-order point"));
}
let edwards = CompressedEdwardsY::from_slice(vk.as_bytes())
.ok()
.and_then(|c| c.decompress())
@@ -18,6 +34,26 @@ fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
Ok(edwards.to_montgomery().to_bytes())
}
/// True when the X25519 exchange between an attacker-supplied `u` and this
/// reader's scalar yields the all-zero shared secret.
///
/// Asked of the RESULT, not of the encoding, and that distinction is the whole
/// guard. The previous version decompressed `u` to an Edwards point and asked
/// `is_small_order`, which reports "safe" for any low-order encoding that does
/// not decompress at all. Measured across the seven standard low-order
/// encodings, six were caught and `u = p - 1` was not: it fails to decompress,
/// so the check returned false, and the exchange is still the all-zero secret.
/// An entry built on it unwraps for every reader.
///
/// Enumerating encodings cannot be exhaustive, because the attack depends on the
/// shared secret rather than on the bytes that produced it, and twist and
/// non-canonical inputs keep arriving. Asking the result needs no list.
fn yields_all_zero_shared_secret(u: &[u8; 32], scalar: &[u8; 32]) -> bool {
use curve25519_dalek::montgomery::MontgomeryPoint;
MontgomeryPoint(*u).mul_clamped(*scalar).to_bytes() == [0u8; 32]
}
/// X25519 secret scalar for an Ed25519 seed (SHA-512 of seed, lower 32, clamped).
/// Returns the scalar wrapped in `Zeroizing`, and scrubs the intermediate
/// SHA-512 digest, so no copy of this secret material lingers in freed memory.
@@ -49,6 +85,13 @@ use serde::{Deserialize, Serialize};
const MAGIC: &[u8] = b"GLENC";
const VERSION: u8 = 2;
/// Upper bound on the blinded recipient list. The envelope author controls
/// this count and `open_blob` pays one X25519 exchange per entry before it can
/// answer "not a recipient", so the count has to be capped rather than
/// optimized (#419). Legitimate envelopes carry a handful of authorized
/// readers, so 256 is far above real use.
const MAX_RECIPIENTS: usize = 256;
#[derive(Serialize, Deserialize)]
struct Recipient {
eph: String, // base64 ephemeral x25519 pubkey (32B)
@@ -68,6 +111,14 @@ pub fn seal_blob(plaintext: &[u8], recipients: &[VerifyingKey]) -> Result<Vec<u8
if recipients.is_empty() {
return Err(anyhow::anyhow!("seal_blob: no recipients"));
}
// Same bound open_blob enforces: an envelope over the cap could never be
// opened, so refuse to seal it (#419).
if recipients.len() > MAX_RECIPIENTS {
return Err(anyhow::anyhow!(
"seal_blob: {} recipients, over the {MAX_RECIPIENTS} cap",
recipients.len()
));
}
let mut content_key = [0u8; 32];
OsRng.fill_bytes(&mut content_key);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
@@ -126,9 +177,20 @@ pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
let header: Header =
serde_json::from_slice(envelope.get(p..p + hlen).context("truncated header")?)
.context("decode header")?;
// Reject before the recipient scan: every entry below is an X25519
// exchange, and a non-recipient would run the whole list (#419).
if header.recipients.len() > MAX_RECIPIENTS {
return Err(anyhow::anyhow!(
"envelope carries {} recipients, over the {MAX_RECIPIENTS} cap",
header.recipients.len()
));
}
let body = &envelope[p + hlen..];
let my_x = XSecret::from(*x25519_secret_from_seed(&keypair.to_seed()));
// The raw scalar is kept alongside the box secret so the exchange can be
// tested BEFORE a box is built from an attacker-supplied ephemeral.
let my_x_scalar = x25519_secret_from_seed(&keypair.to_seed());
let my_x = XSecret::from(*my_x_scalar);
// Identities are blinded: no entry says which recipient it belongs to, so
// try each one. The ChaChaBox AEAD tag authenticates, so exactly the
@@ -140,7 +202,12 @@ pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
.ok()
.and_then(|b| <[u8; 32]>::try_from(b.as_slice()).ok())
{
Some(b) => XPublic::from(b),
// An ephemeral whose exchange with this reader is the all-zero
// shared secret would unwrap for anyone, so the entry is skipped.
// Tested on the exchange itself: an encoding-shaped check misses
// every low-order input that does not decompress to Edwards.
Some(b) if !yields_all_zero_shared_secret(&b, &my_x_scalar) => XPublic::from(b),
Some(_) => continue,
None => continue,
};
// from_slice panics on a wrong length, and the envelope is attacker
@@ -293,4 +360,436 @@ mod tests {
header["nonce"] = bad_nonce;
assert!(open_blob(&reframe(&header), &reader).is_err());
}
#[test]
fn over_cap_recipient_list_is_rejected_before_the_scan() {
// #419: the envelope author controls the recipient count and each entry
// costs an X25519 exchange, so the list is capped. Reframe a valid
// envelope with MAX_RECIPIENTS + 1 junk entries.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
let junk = serde_json::json!({
"eph": B64.encode([0u8; 32]),
"nonce": B64.encode([0u8; 24]),
"wrap": B64.encode([0u8; 48]),
});
header["recipients"] = serde_json::Value::Array(vec![junk.clone(); MAX_RECIPIENTS + 1]);
let hj = serde_json::to_vec(&header).unwrap();
let mut fat = Vec::new();
fat.extend_from_slice(MAGIC);
fat.push(VERSION);
fat.extend_from_slice(&(hj.len() as u32).to_le_bytes());
fat.extend_from_slice(&hj);
fat.extend_from_slice(body);
let err = open_blob(&fat, &reader).unwrap_err();
assert!(
err.to_string().contains("recipients"),
"expected the recipient-cap error, got: {err}"
);
// Exactly at the cap the envelope is still processed end to end.
header["recipients"] = serde_json::Value::Array(vec![junk; MAX_RECIPIENTS]);
let hj = serde_json::to_vec(&header).unwrap();
let mut at_cap = Vec::new();
at_cap.extend_from_slice(MAGIC);
at_cap.push(VERSION);
at_cap.extend_from_slice(&(hj.len() as u32).to_le_bytes());
at_cap.extend_from_slice(&hj);
at_cap.extend_from_slice(body);
let err = open_blob(&at_cap, &reader).unwrap_err();
assert!(
!err.to_string().contains("over the"),
"at-cap envelope must reach the scan, got: {err}"
);
}
#[test]
fn seal_refuses_a_recipient_list_over_the_cap() {
// An envelope over the cap can never be opened, so seal must reject it
// instead of minting one.
let keys: Vec<VerifyingKey> = (0..MAX_RECIPIENTS + 1)
.map(|_| Keypair::generate().verifying_key())
.collect();
let err = seal_blob(b"blob", &keys).unwrap_err();
assert!(
err.to_string().contains("recipients"),
"expected the recipient-cap error, got: {err}"
);
}
/// Every standard low-order X25519 encoding must be detected by
/// `yields_all_zero_shared_secret`, including encodings that do not
/// decompress to an Edwards point (notably `u = p - 1`). The open side
/// rejects on the exchange result, not on decode shape, because an
/// encoding-shaped check misses non-decompressing low-order inputs while
/// still yielding the all-zero shared secret.
#[test]
fn all_zero_shared_secret_is_detected_for_every_standard_low_order_encoding() {
// The seven standard X25519 low-order encodings, driven as a set rather
// than as the two that happen to decompress. The encoding-shaped check
// this replaced reported `p - 1` as safe, and its exchange is all-zero.
let mut one = [0u8; 32];
one[0] = 1;
let hex = |h: &str| -> [u8; 32] {
let mut out = [0u8; 32];
for i in 0..32 {
out[i] = u8::from_str_radix(&h[i * 2..i * 2 + 2], 16).unwrap();
}
out
};
let vectors = [
("u=0", [0u8; 32]),
("u=1", one),
(
"order8-a",
hex("e0eb7a7c3b41b8ae1656e3faf19fc46ada098deb9c32b1fd866205165f49b800"),
),
(
"order8-b",
hex("5f9c95bca3508c24b1d0b1559c83ef5b04445cc4581c8e86d8224eddd09f1157"),
),
(
"p-1",
hex("ecffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
(
"p",
hex("edffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
(
"p+1",
hex("eeffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
];
let kp = Keypair::generate();
let scalar = x25519_secret_from_seed(&kp.to_seed());
for (name, u) in vectors {
assert!(
yields_all_zero_shared_secret(&u, &scalar),
"{name} is a low-order encoding and must be detected"
);
}
// And the other direction, or a guard that returned true always would
// pass everything above while making the envelope unopenable.
let real_u = x25519_public(&kp.verifying_key()).unwrap();
assert!(
!yields_all_zero_shared_secret(&real_u, &scalar),
"a real key must not be treated as low-order"
);
}
/// The open side must skip an entry whose header-supplied `eph` is a
/// low-order point: such an entry forces the all-zero shared secret and
/// would unwrap for any reader. A poisoned entry among honest ones must
/// not DoS the envelope — the honest entry still opens.
#[test]
fn open_blob_skips_a_low_order_ephemeral_and_still_opens_honest_entry() {
use curve25519_dalek::montgomery::MontgomeryPoint;
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Split the envelope framing into header JSON and body.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
// A low-order Montgomery u (u = 0) encoded as the entry's eph.
let low_order_u = MontgomeryPoint([0u8; 32]).to_bytes();
let poisoned = serde_json::json!({
"eph": B64.encode(low_order_u),
"nonce": B64.encode([0u8; 24]),
"wrap": B64.encode([0u8; 32]),
});
let reframe = |header: &serde_json::Value| -> Vec<u8> {
let hj = serde_json::to_vec(header).unwrap();
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(hj.len() as u32).to_le_bytes());
out.extend_from_slice(&hj);
out.extend_from_slice(body);
out
};
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
// Prepend the poisoned entry. The reader's honest entry must still
// unwrap, so the envelope opens despite the low-order entry.
let recipients = header["recipients"].as_array_mut().unwrap();
recipients.insert(0, poisoned);
assert_eq!(
open_blob(&reframe(&header), &reader).unwrap(),
b"private blob contents",
"the honest entry must still open despite a low-order eph entry"
);
}
/// Must-not for the open-side guard: an envelope whose ONLY entry is built
/// on a low-order ephemeral must not open for any reader. Without the
/// guard the all-zero shared secret unwraps for everyone, so the attacker
/// controls the plaintext and the envelope's confidentiality is void.
#[test]
fn open_blob_rejects_a_poisoned_only_envelope() {
use chacha20poly1305::{aead::Aead, KeyInit, XChaCha20Poly1305, XNonce};
use crypto_box::aead::AeadCore;
use curve25519_dalek::montgomery::MontgomeryPoint;
// Build a poisoned envelope from scratch: one recipient entry whose
// eph is the low-order u = 0. The wrap is the encryption of a content
// key under the all-zero shared secret, so without the guard any
// reader's ChaChaBox against u = 0 decrypts it.
let mut content_key: [u8; 32] = Default::default();
OsRng.fill_bytes(&mut content_key);
let attacker_x = XSecret::generate(&mut OsRng);
let zero_box = ChaChaBox::new(&XPublic::from(<[u8; 32]>::default()), &attacker_x);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let wrap = zero_box.encrypt(&nonce, &content_key[..]).unwrap();
let mut body_nonce: [u8; 24] = Default::default();
OsRng.fill_bytes(&mut body_nonce);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let body = body_cipher
.encrypt(
XNonce::from_slice(&body_nonce),
b"attacker-chosen plaintext".as_slice(),
)
.unwrap();
let header = serde_json::json!({
"alg": "xchacha20poly1305",
"nonce": B64.encode(body_nonce),
"recipients": [{
"eph": B64.encode(MontgomeryPoint([0u8; 32]).to_bytes()),
"nonce": B64.encode(nonce),
"wrap": B64.encode(wrap),
}],
});
let header_json = serde_json::to_vec(&header).unwrap();
let mut env = Vec::new();
env.extend_from_slice(MAGIC);
env.push(VERSION);
env.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
env.extend_from_slice(&header_json);
env.extend_from_slice(&body);
let reader = Keypair::generate();
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("not a recipient"),
"a poisoned-only envelope must not open for anyone, got: {err}"
);
}
/// The same must-not, driven with the low-order encoding that does NOT map
/// to an Edwards point.
///
/// This is the case an encoding-shaped guard cannot see. `u = p - 1` is a
/// legal X25519 input whose exchange is the all-zero secret for any scalar,
/// and `MontgomeryPoint::to_edwards` returns None for it, so a check written
/// as "decompress, then ask is_small_order" reports it as safe. Measured
/// across the seven standard low-order encodings: six are caught that way
/// and this one is not.
///
/// Found by a cross-family review of the head that first added the guard,
/// and the comment beside that guard asserted this case was already handled
/// by a decode path that does not exist. That is why the fix rejects on the
/// RESULT of the exchange rather than on a list of encodings: the result is
/// what the attack actually depends on, and it needs no enumeration to be
/// exhaustive.
#[test]
fn open_blob_rejects_a_non_edwards_low_order_ephemeral() {
// No blanket `use ...::Aead` here. Two aead versions are reachable, and a
// blanket import silently resolved the wrap through the one open_blob does
// NOT decrypt with, producing a ciphertext that could never open. The test
// then passed with the guard, without it, and with the original pre-fix
// guard alike, proving nothing. Every call below names its trait.
use chacha20poly1305::{KeyInit, XChaCha20Poly1305, XNonce};
use crypto_box::aead::AeadCore;
// u = p - 1 = 2^255 - 20, little endian.
let mut eph = [0xffu8; 32];
eph[0] = 0xec;
eph[31] = 0x7f;
let mut content_key: [u8; 32] = Default::default();
OsRng.fill_bytes(&mut content_key);
let attacker_x = XSecret::generate(&mut OsRng);
let zero_box = ChaChaBox::new(&XPublic::from(<[u8; 32]>::default()), &attacker_x);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let wrap = crypto_box::aead::Aead::encrypt(&zero_box, &nonce, &content_key[..]).unwrap();
let mut body_nonce: [u8; 24] = Default::default();
OsRng.fill_bytes(&mut body_nonce);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let body = chacha20poly1305::aead::Aead::encrypt(
&body_cipher,
XNonce::from_slice(&body_nonce),
b"attacker-chosen plaintext".as_slice(),
)
.unwrap();
let header = serde_json::json!({
"alg": "xchacha20poly1305",
"nonce": B64.encode(body_nonce),
"recipients": [{
"eph": B64.encode(eph),
"nonce": B64.encode(nonce),
"wrap": B64.encode(wrap),
}],
});
let header_json = serde_json::to_vec(&header).unwrap();
let mut env = Vec::new();
env.extend_from_slice(MAGIC);
env.push(VERSION);
env.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
env.extend_from_slice(&header_json);
env.extend_from_slice(&body);
let reader = Keypair::generate();
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("not a recipient"),
"an envelope whose only entry uses a non-Edwards low-order eph must not \
open for anyone, got: {err}"
);
}
/// The compressed identity point: a well-formed Ed25519 encoding that is
/// small-order, so every shared secret derived from it is all-zero.
fn weak_verifying_key() -> VerifyingKey {
let mut weak = [0u8; 32];
weak[0] = 1;
let vk = VerifyingKey::from_bytes(&weak).expect("identity point decompresses");
assert!(vk.is_weak(), "fixture precondition: key is small-order");
vk
}
/// A small-order recipient key converts to Montgomery u = 0, and X25519
/// against u = 0 is the all-zero shared secret for ANY scalar, so the
/// wrapping box is reconstructable with no secret at all. Reject at the
/// primitive so the seal side is safe regardless of how the caller
/// obtained the key.
#[test]
fn x25519_public_rejects_a_small_order_key() {
let weak_vk = weak_verifying_key();
assert!(
x25519_public(&weak_vk).is_err(),
"a small-order key must not yield an x25519 public key"
);
}
/// Control: a legitimate key still converts, and to a non-zero u.
#[test]
fn x25519_public_still_accepts_a_real_key() {
let kp = Keypair::generate();
let u = x25519_public(&kp.verifying_key()).expect("a real key must convert");
assert_ne!(u, [0u8; 32], "a real key must not map to the zero point");
}
/// The guard has to propagate: no envelope may be produced for a weak
/// recipient even when the DID choke point is bypassed entirely.
#[test]
fn seal_blob_refuses_a_small_order_recipient() {
let weak_vk = weak_verifying_key();
assert!(
seal_blob(b"withheld", &[weak_vk]).is_err(),
"sealing to a weak recipient must fail"
);
// And a mixed set must fail too: one weak recipient exposes the shared
// content key, so a partial envelope is not an acceptable outcome.
let honest = Keypair::generate();
assert!(
seal_blob(b"withheld", &[honest.verifying_key(), weak_vk]).is_err(),
"a mixed honest+weak recipient set must fail closed, not seal partially"
);
}
/// Control for the two above: the ordinary seal/open round trip is intact.
#[test]
fn legit_seal_open_round_trip_still_works() {
let reader = Keypair::generate();
let env = seal_blob(b"withheld blob", &[reader.verifying_key()]).expect("seal");
assert_eq!(open_blob(&env, &reader).expect("open"), b"withheld blob");
}
/// The defect this fix exists to close, kept as an executable regression
/// rather than an assertion about absence. It runs the real attack: craft a
/// small-order did:key, get it into a recipient set alongside an honest
/// reader, then rebuild the wrapping box from the all-zero shared secret
/// that a small-order recipient forces. If any guard regresses, this does
/// not merely fail, it fails printing the plaintext it recovered.
#[test]
fn attacker_cannot_recover_plaintext_via_a_weak_recipient() {
use crate::did::Did;
use std::str::FromStr;
let weak_vk = weak_verifying_key();
// Layer 1: the attacker's did:key string must not resolve at all.
let weak_did = Did::from_verifying_key(&weak_vk).to_string();
assert!(
Did::from_str(&weak_did)
.expect("still a well-formed did:key")
.to_verifying_key()
.is_err(),
"a small-order did:key must not resolve"
);
// Layer 2: even handed the key directly, sealing must refuse. One weak
// recipient would expose the single shared content key, so the honest
// reader's blob would be readable by anyone.
let honest = Keypair::generate();
let secret_plaintext = b"WITHHELD BLOB CONTENTS";
let envelope = match seal_blob(secret_plaintext, &[honest.verifying_key(), weak_vk]) {
Err(_) => return, // no envelope exists, nothing to attack
Ok(env) => env,
};
// Only reachable if a guard regressed. Run the attack and report what
// it got, so the failure names the actual exposure.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(envelope[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header: serde_json::Value = serde_json::from_slice(&envelope[p..p + hlen]).unwrap();
let body = &envelope[p + hlen..];
let body_nonce = B64.decode(header["nonce"].as_str().unwrap()).unwrap();
// X25519 against u = 0 is the all-zero shared secret for any scalar, so
// the sealer's box is reconstructable with a key the attacker picks.
let zero_box = ChaChaBox::new(
&XPublic::from(<[u8; 32]>::default()),
&XSecret::generate(&mut OsRng),
);
for r in header["recipients"].as_array().unwrap() {
let n = B64.decode(r["nonce"].as_str().unwrap()).unwrap();
let w = B64.decode(r["wrap"].as_str().unwrap()).unwrap();
if let Ok(content_key) = zero_box.decrypt(n.as_slice().into(), w.as_slice()) {
let cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let recovered = cipher
.decrypt(XNonce::from_slice(&body_nonce), body)
.expect("body decrypts once the content key is out");
panic!(
"REGRESSION: attacker recovered plaintext with no private key: {:?}",
String::from_utf8_lossy(&recovered)
);
}
}
panic!("an envelope was sealed to a small-order recipient; the seal guard regressed");
}
}
+255 -12
View File
@@ -1,6 +1,6 @@
//! HTTP Signatures (RFC 9421) for gitlawb.
//!
//! Every write request to a gitlawb node must be signed by the actor's
//! Every write request to a Twigpine node must be signed by the actor's
//! Ed25519 private key. The signature covers:
//! - `@method` — HTTP method (uppercase)
//! - `@path` — request path and query
@@ -14,13 +14,13 @@
use base64::{engine::general_purpose::STANDARD, Engine};
use chrono::Utc;
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use crate::did::Did;
use crate::identity::Keypair;
use crate::{Error, Result};
/// The component identifiers covered by every gitlawb signature.
/// The component identifiers covered by every Twigpine signature.
pub const COVERED_COMPONENTS: &[&str] = &["@method", "@path", "content-digest"];
/// The three headers produced by RFC 9421 signing.
@@ -67,11 +67,20 @@ impl HttpSignature {
let components_str = &rest[open + 1..close];
let params_str = &rest[close + 1..]; // starts with ';'
// "\"@method\" \"@path\" \"content-digest\"" → ["@method", "@path", "content-digest"]
let components: Vec<String> = components_str
.split_whitespace()
.map(|s| s.trim_matches('"').to_string())
.collect();
// RFC 9421 §2.1: a component identifier appears at most once. Gitlawb
// only supports bare names; parameters are rejected so reordered
// equivalent forms cannot bypass duplicate detection as distinct strings.
let mut components = Vec::new();
let mut seen = HashSet::new();
for token in components_str.split_whitespace() {
let name = parse_covered_component(token)?;
if !seen.insert(name.clone()) {
return Err(Error::HttpSignature(format!(
"duplicate covered component '{name}' in Signature-Input"
)));
}
components.push(name);
}
let params = parse_params(params_str)?;
@@ -113,15 +122,48 @@ impl HttpSignature {
})
}
/// Reject if the `created` timestamp is more than 5 minutes from now.
/// Reject a `created` timestamp outside the acceptance window, which is
/// ASYMMETRIC: up to 5 minutes into the past, but only 60 seconds into the
/// future.
///
/// The two directions are two comparisons rather than one distance, because
/// they are not the same event. A signature that arrives late is ordinary:
/// clocks run slow, requests queue, and networks are not instant, so the
/// past side keeps the full 300-second allowance. A signature stamped ahead
/// of this node's clock is not ordinary, and folding both sides into an
/// `abs()` let a signer extend its own validity: signed at T with
/// `created = T + 300`, it stayed acceptable until roughly T + 600, so the
/// effective window was double the one this function advertises. The
/// forward tolerance exists only to absorb honest drift between two hosts,
/// which is what 60 seconds buys; it is the same allowance the gossip
/// path's freshness check uses for the same reason.
pub fn check_created(&self) -> Result<()> {
const MAX_AGE_SECS: i64 = 300;
const MAX_FUTURE_SECS: i64 = 60;
let now = Utc::now().timestamp();
let skew = (now - self.created).abs();
if skew > 300 {
// Saturating, not plain subtraction. `created` is parsed as an
// unrestricted i64 straight from the Signature-Input header, so a
// sender picks it: at i64::MIN the past-side subtraction overflows,
// which panics a debug build and wraps a release one into an absurd
// value that can read as acceptable. Saturating gives the answer the
// check wants at both extremes, since a timestamp that far away is
// refused by whichever side it saturates toward.
let age = now.saturating_sub(self.created);
if age > MAX_AGE_SECS {
return Err(Error::HttpSignature(format!(
"clock skew too large: {skew}s (max 300s)"
"clock skew too large: created {age}s in the past (max {MAX_AGE_SECS}s)"
)));
}
let ahead = self.created.saturating_sub(now);
if ahead > MAX_FUTURE_SECS {
return Err(Error::HttpSignature(format!(
"clock skew too large: created {ahead}s in the future (max {MAX_FUTURE_SECS}s)"
)));
}
Ok(())
}
@@ -203,6 +245,33 @@ pub fn compute_content_digest(body: &[u8]) -> String {
format!("sha-256=:{}:", STANDARD.encode(hasher.finalize()))
}
/// Parse one covered-component token from Signature-Input's parenthesized list.
///
/// Gitlawb only supports bare component names (`"@method"`, `"@path"`,
/// `"content-digest"`). RFC 9421 permits parameters on identifiers; they are
/// rejected rather than accepted in a form `missing_components()` and
/// `build_signing_string()` cannot handle consistently.
fn parse_covered_component(token: &str) -> Result<String> {
let token = token.trim();
let open = token
.strip_prefix('"')
.ok_or_else(|| Error::HttpSignature("component must be a quoted string".into()))?;
let close = open
.find('"')
.ok_or_else(|| Error::HttpSignature("unterminated component string".into()))?;
let name = &open[..close];
if name.is_empty() {
return Err(Error::HttpSignature("empty component name".into()));
}
let trailing = open[close + 1..].trim();
if !trailing.is_empty() {
return Err(Error::HttpSignature(format!(
"component parameters are not supported on '{name}'"
)));
}
Ok(name.to_string())
}
/// Parse `;key="value";key2=value` parameter string into a map.
fn parse_params(s: &str) -> Result<HashMap<String, String>> {
let mut map = HashMap::new();
@@ -272,6 +341,91 @@ mod tests {
assert!(missing.contains(&"content-digest"));
}
/// RFC 9421 §2.1: a component identifier must not appear twice in the
/// covered list. Repeating one is not a way to say anything, but it does
/// repeat a line in the signing string, so the size of what a verifier
/// builds (and what a node persists alongside a claim) is set by how many
/// times the caller chose to write the same name.
#[test]
fn parse_rejects_component_parameters() {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest";sf);keyid="{did}";alg="ed25519";created=1000"#
);
let err = HttpSignature::parse(
&sig_input,
"sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:",
)
.expect_err("parameterized components must be refused");
assert!(
err.to_string().contains("component parameters"),
"got: {err}"
);
}
#[test]
fn parse_rejects_reordered_equivalent_component_parameters() {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("content-digest";sf;tr "content-digest";tr;sf "@method" "@path");keyid="{did}";alg="ed25519";created=1000"#
);
let err = HttpSignature::parse(
&sig_input,
"sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:",
)
.expect_err("reordered equivalent parameters must not bypass as distinct components");
assert!(
err.to_string().contains("component parameters"),
"got: {err}"
);
}
#[test]
fn parse_many_distinct_components_completes_quickly() {
use std::time::{Duration, Instant};
let kp = Keypair::generate();
let did = kp.did();
let names: Vec<String> = (0..20_000).map(|i| format!("\"c{i}\"")).collect();
let inner = names.join(" ");
let sig_input = format!(r#"sig1=({inner});keyid="{did}";alg="ed25519";created=1000"#);
let start = Instant::now();
let result = HttpSignature::parse(
&sig_input,
"sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:",
);
let elapsed = start.elapsed();
assert!(
elapsed < Duration::from_secs(2),
"duplicate tracking must stay linear, took {elapsed:?}"
);
assert!(result.is_ok(), "distinct names should parse: {:?}", result);
}
#[test]
fn parse_rejects_duplicate_components() {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("@method" "@path" "@path" "content-digest");keyid="{did}";alg="ed25519";created=1000"#
);
let err = HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:")
.expect_err("a repeated component must be refused");
assert!(
err.to_string().contains("duplicate"),
"the error must name the duplication, got: {err}"
);
// The control: the same list without the repeat still parses.
let ok = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created=1000"#
);
HttpSignature::parse(&ok, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:")
.expect("a distinct component list must still parse");
}
#[test]
fn verify_signature_end_to_end() {
use crate::identity::verify;
@@ -335,6 +489,75 @@ mod tests {
assert!(sig.check_created().is_err());
}
/// Build a parseable signature whose `created` is `offset` seconds from now
/// (negative for the past), so a test can pin one side of the window.
fn sig_created_at_offset(offset: i64) -> HttpSignature {
let kp = Keypair::generate();
let did = kp.did();
let created = Utc::now().timestamp() + offset;
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created={created}"#
);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
}
/// Build a signature at an ABSOLUTE `created`, so a test can drive the
/// extremes an offset from now cannot reach.
fn sig_created_absolute(created: i64) -> HttpSignature {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created={created}"#
);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
}
/// `created` is parsed as an unrestricted i64 from a header the sender
/// writes, so the window check has to survive both ends of the type. Before
/// the saturating form this panicked with "attempt to subtract with
/// overflow" on the past side, which is a debug-build crash any client
/// could trigger by sending one header value.
#[test]
fn extreme_created_values_are_refused_rather_than_overflowing() {
for created in [i64::MIN, i64::MIN + 1, i64::MAX - 1, i64::MAX] {
let err = sig_created_absolute(created)
.check_created()
.expect_err("an extreme created must be refused, not accepted");
let msg = err.to_string();
assert!(
msg.contains("past") || msg.contains("future"),
"created={created} must be refused by a named direction, got: {msg}"
);
}
}
#[test]
fn created_301s_in_past_rejected() {
let err = sig_created_at_offset(-301).check_created().unwrap_err();
assert!(err.to_string().contains("past"), "{err}");
}
#[test]
fn created_299s_in_past_accepted() {
assert!(sig_created_at_offset(-299).check_created().is_ok());
}
#[test]
fn created_61s_in_future_rejected() {
let err = sig_created_at_offset(61).check_created().unwrap_err();
assert!(err.to_string().contains("future"), "{err}");
}
#[test]
fn created_30s_in_future_accepted() {
assert!(sig_created_at_offset(30).check_created().is_ok());
}
#[test]
fn created_exactly_now_accepted() {
assert!(sig_created_at_offset(0).check_created().is_ok());
}
#[test]
fn fresh_signature_passes_clock_skew() {
let kp = Keypair::generate();
@@ -343,6 +566,26 @@ mod tests {
assert!(sig.check_created().is_ok());
}
#[test]
fn extreme_created_timestamps_reject_without_wrapping() {
let kp = Keypair::generate();
let did = kp.did();
let parse = |created: i64| {
let sig_input =
format!(r#"sig1=("@method");keyid="{did}";alg="ed25519";created={created}"#);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
};
// i64::MIN used to panic in debug (subtraction overflow) and wrap to a
// passing skew in release; now - 2^63 wrapped to i64::MIN, whose abs()
// wrapped back to itself and read as fresh. Both must reject.
for created in [i64::MIN, i64::MAX, Utc::now().timestamp() - i64::MAX] {
assert!(
parse(created).check_created().is_err(),
"created={created} must not pass the freshness gate"
);
}
}
#[test]
fn parse_error_missing_sig1_prefix() {
let err = HttpSignature::parse(
+25 -4
View File
@@ -1,6 +1,6 @@
//! Keypair generation, storage, and signing.
//!
//! A gitlawb identity is an Ed25519 keypair. The public key encodes into a
//! A Twigpine identity is an Ed25519 keypair. The public key encodes into a
//! `did:key` DID. The private key is stored as PKCS#8 PEM on disk.
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
@@ -11,7 +11,7 @@ use zeroize::Zeroizing;
use crate::did::Did;
use crate::{Error, Result};
/// An Ed25519 keypair that is the root identity for a gitlawb actor.
/// An Ed25519 keypair that is the root identity for a Twigpine actor.
#[derive(Clone)]
pub struct Keypair {
signing_key: SigningKey,
@@ -78,10 +78,9 @@ impl Keypair {
/// Verify an Ed25519 signature.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
use ed25519_dalek::Verifier;
let sig = Signature::from_bytes(sig_bytes);
verifying_key
.verify(msg, &sig)
.verify_strict(msg, &sig)
.map_err(|_| Error::SignatureInvalid)
}
@@ -209,6 +208,28 @@ mod tests {
);
}
#[test]
fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a
// weak (small-order) public key satisfies the verification equation
// but must be rejected. The identity point is such a key: with R the
// identity and S = 0, [S]B - [k]A is the identity for any message.
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let msg = b"forged message";
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let result = verify(&weak_vk, msg, &forged);
assert!(
result.is_err(),
"signature under a weak (small-order) public key must be rejected"
);
}
#[test]
fn signed_payload_round_trip() {
let kp = Keypair::generate();
+9
View File
@@ -5,7 +5,16 @@ pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
// `url` is the one dependency here that drags a tail (idna, then the icu
// crates), and gitlawb-core is allowlisted to stay embeddable. Every client that
// needs this predicate already parses URLs, so they opt in and nothing else
// pays. `test` is in the cfg so `cargo test -p gitlawb-core` still compiles and
// runs the matrix below with no feature selected; without it the tests would
// silently not run, which is the failure this module exists to prevent.
#[cfg(any(feature = "redirect", test))]
pub mod redirect;
pub mod sanitize;
pub mod scan_token;
pub mod ucan;
pub use error::Error;
+241
View File
@@ -0,0 +1,241 @@
//! Redirect policy shared by every Twigpine HTTP client that signs its requests.
//!
//! Both `gl` (async) and `git-remote-gitlawb` (blocking) attach RFC 9421
//! `Signature` and `Signature-Input` headers, and reqwest strips only
//! `Authorization`, `Cookie`, `Proxy-Authorization` and `WWW-Authenticate` when a
//! redirect crosses hosts. A signature would survive that hop, and it binds
//! `@method`, `@path` and `content-digest` with no authority component, so a node
//! answering 302 could hand a working credential to a host of its choosing and read
//! as the caller anywhere until the clock-skew window closes. On a 307/308 the
//! request body goes along with it, which for the remote helper is the pack.
//!
//! That same missing authority component is why the predicate also pins the
//! request-target: `@path` is signed as the client sent it and verified as the node
//! received it, so a same-origin hop that rewrites the path or the query (a
//! trailing-slash or query normalization) makes the signature cover a target the
//! node never saw and the read 401s. Only a hop that re-issues the identical target,
//! an http-to-https upgrade being the one that matters in practice, is followed.
//!
//! The decision lives here rather than in either client because the two used to
//! disagree: `gl` was scoped to the origin while the remote helper, the binary that
//! actually runs `git clone gitlawb://`, still ran reqwest's default and followed
//! anywhere. One predicate is what keeps a future third client from repeating that.
//!
//! The type is `url::Url`, which is what `reqwest::Url` re-exports, so both clients
//! pass their attempt URLs straight in.
/// Longest redirect chain followed. `reqwest::redirect::Policy::custom` replaces
/// reqwest's built-in limit, so the bound has to be restated by every client that
/// installs a custom policy; the value is reqwest's own default.
///
/// This counts FOLLOWS, matching `Policy::limited`: reqwest pushes the redirecting
/// URL onto `previous` before consulting the policy, and `Limit(max)` refuses once
/// `previous.len() > max`, so a caller comparing against this constant must use `>`
/// too or it permits one hop fewer than it says.
pub const MAX_REDIRECTS: usize = 10;
/// Follow a redirect only when it stays on the origin that issued it AND re-issues
/// the identical request-target.
///
/// `Policy::none()` would have been the simpler answer, but one same-origin redirect
/// shape is legitimate here: a node fronted by a proxy that upgrades http to https,
/// or otherwise re-issues the same path and query, so the policy is scoped rather
/// than switched off.
///
/// Path and query must match exactly, and that is the request-target clause rather
/// than an origin one. `@path` is signed as the client sent it and verified as the
/// node received it, so a hop that rewrites either half leaves a signature covering
/// a target nobody asked for and the node answers 401. Refusing the hop turns a
/// confusing 401 into the 3xx that names what actually happened. One policy covers
/// signed and unsigned callers alike, for the same reason the predicate is shared:
/// two rules would drift.
///
/// The clause pins `@path`, and only `@path`. A Twigpine signature also covers
/// `@method` and `content-digest`, and both of those are still open on a followed
/// hop: on a 301, 302 or 303, reqwest 0.12.28 delegates to tower-http's
/// `FollowRedirect`, which rewrites a POST to a GET and empties the body
/// (tower-http-0.6.8 `src/follow_redirect/mod.rs:273-285`), while its
/// `drop_payload_headers` removes only `Content-Type`, `Content-Length`,
/// `Content-Encoding` and `Transfer-Encoding`. So `Signature`, `Signature-Input` and
/// `Content-Digest` ride along on a request that no longer has the method or the body
/// they were computed over. Only 307 and 308 preserve both. This predicate returning
/// true therefore makes a GET-shaped hop safe to replay against the node's verifier
/// and says nothing about a bodied one: a signed write must not rely on it alone.
///
/// `Url::query` is `None` for `/a` and `Some("")` for `/a?`, and those are two
/// different request-targets on the node side too, so the comparison is strict and
/// needs no special case.
///
/// Host and port must match exactly. Port is compared as `Url::port`, which is
/// `None` for a scheme's default port, so http -> https on the same host compares
/// equal while http -> http on a different port does not. A downgrade from https to
/// http is refused as well: the target is the same host, but the signature would go
/// out in cleartext, which is the same credential leak by a slower route.
///
/// Host comparison rides on `url`'s parse-time normalization (lowercasing and IDN
/// -> punycode), so the spellings an attacker reaches for do not open a gap. That
/// is a property of the parsed `Url`, not of this function, which is why the test
/// matrix pins it: a move to raw string comparison would silently lose it.
pub fn may_follow(previous: &url::Url, next: &url::Url) -> bool {
let same_origin = next.host_str() == previous.host_str() && next.port() == previous.port();
let same_target = next.path() == previous.path() && next.query() == previous.query();
let downgraded = previous.scheme() == "https" && next.scheme() != "https";
same_origin && same_target && !downgraded
}
#[cfg(test)]
mod tests {
use super::*;
/// Every branch of the decision, both ways, plus the spellings that would slip
/// past a comparison less careful than `Url`'s own normalization.
#[test]
fn may_follow_covers_each_origin_branch() {
let url = |s: &str| url::Url::parse(s).unwrap();
let cases: &[(&str, &str, bool, &str)] = &[
(
"http://node.example/a",
"http://node.example/b",
false,
"same origin but the path changed: @path is signed as sent and verified as received",
),
(
"http://node.example/a",
"https://node.example/a",
true,
"http to https on one host with an identical request-target: both ports are \
the scheme default, so the proxy upgrade is still followed",
),
(
"https://node.example/a",
"https://node.example/a/",
false,
"trailing-slash normalization changes the request-target, so it is refused",
),
(
"https://node.example:8443/a",
"https://node.example:8443/a",
true,
"same explicit port",
),
(
"http://node.example/a",
"http://attacker.example/a",
false,
"different host",
),
(
"http://node.example/a",
"http://node.example:8080/a",
false,
"same host, different port",
),
(
"https://node.example/a",
"http://node.example/a",
false,
"https downgraded to cleartext on the same host",
),
(
"https://node.example/a",
"http://node.example:443/a",
false,
"a downgrade dressed up as the https port",
),
// The rows below pass today because `Url::parse` normalizes the host, not
// because anything here compares case-insensitively or decodes IDN. They
// are the variants an attacker reaches for, so they are pinned: swapping
// this predicate for a raw string comparison must break the suite.
// Each of these pairs an IDENTICAL path on both sides, deliberately. The
// request-target clause below would make every one of them false on the
// path alone, and a row that is false for two reasons has stopped pinning
// either. With the paths equal, the host or port comparison is the only
// thing left that can decide them.
(
"https://node.example/a",
"https://NODE.EXAMPLE/a",
true,
"same host in a different case: parse lowercases it",
),
(
"https://node.example/a",
"https://node.example./a",
false,
"a trailing dot is a different host to url, so the redirect is refused",
),
(
"https://exämple.test/a",
"https://xn--exmple-cua.test/a",
true,
"unicode host and its punycode spelling are one host after parse",
),
(
"https://node.example/a",
"https://user:[email protected]/a",
true,
"userinfo is not part of the origin: same host, still followed",
),
(
"https://node.example/a",
"https://[email protected]/a",
false,
"the node's name smuggled into userinfo: the host is the attacker's",
),
(
"https://node.example/a",
"https://attacker.example/a#node.example",
false,
"the node's name pushed into the fragment: the host is the attacker's",
),
// The request-target clause, both directions. `@path` is the only thing a
// Twigpine signature binds the request to, so a hop that rewrites it hands
// the node a signature over a target it never received.
(
"https://node.example/a?x=1",
"https://node.example/a?x=1",
true,
"identical request-target: the same-origin hop that is still followed",
),
(
"https://node.example/a?x=1",
"https://node.example/a?x=2",
false,
"same path but a different query: the request-target covers the query too",
),
(
"https://node.example/a",
"https://node.example/a?x=1",
false,
"a query added where there was none",
),
(
"http://node.example/a",
"http://node.example/a?",
false,
"an empty query added where there was none: a missing query and an empty \
one are different request-targets",
),
(
"https://node.example/a?x=1",
"https://node.example/a",
false,
"the query dropped where there was one: the comparison is symmetric, and \
nothing else in the matrix pins that direction",
),
(
"https://node.example/a#x",
"https://node.example/a#y",
true,
"a fragment-only difference is still followed: a fragment never reaches \
the wire, so it is no part of the request-target the node verifies",
),
];
for (previous, next, expected, why) in cases {
assert_eq!(
may_follow(&url(previous), &url(next)),
*expected,
"{previous} -> {next} ({why})"
);
}
}
}
+499
View File
@@ -0,0 +1,499 @@
//! Sealed continuation token for the node's bounded legacy CID scan (INV-13).
//!
//! The `/ipfs/{cid}` resolver's legacy scan stops at a row ceiling and sheds a
//! retryable 503. To let a holder buried past that ceiling still be reached, the
//! shed carries the scan position so the caller can echo it back and resume. The
//! whole point of the design is that the node keeps NO server-side scan state: the
//! position rides in the caller's token.
//!
//! That makes the token an EMITTED continuation derived from a FETCHED row, and on
//! a scan that served nothing every fetched row is by construction a private or
//! quarantined repo the caller may not read. The row's `created_at` leaks its
//! creation time and its `id` carries the owner's DID, so both halves are withheld
//! fields and the token must be CONFIDENTIAL, not merely tamper-evident:
//!
//! * AEAD-sealed (XChaCha20-Poly1305), never base64-of-plaintext and never
//! signed plaintext. Integrity is not confidentiality.
//! * A fresh `OsRng` nonce on EVERY seal. Under a stream cipher a repeated nonce
//! means repeated keystream, and an attacker who can force the node to seal a
//! position whose plaintext they know XORs two tokens and recovers a withheld
//! row's fields in full, strictly worse than emitting plaintext.
//! * FIXED-WIDTH plaintext. AEAD ciphertext is plaintext-length plus the tag, and
//! every field of a scan position varies in length, so a variable encoding would
//! make token LENGTH a side channel for the sealed row (a short name under a
//! short owner vs a long one) and for the candidate oid's width (40 hex on a
//! sha1 repo, 64 on a sha256 one). Every token this module mints is byte-identical
//! in length. Each field is padded to its own separate width, which is a per-field
//! constant and so still leaks nothing about a given row or candidate.
//! * The canonical CID as associated data, so a token minted while scanning for
//! one CID does not authenticate when replayed against another.
//!
//! Every failure to open (wrong key, tampered bytes, wrong CID, expired, malformed)
//! returns the same `None`. The caller treats that as "no token" and starts at the
//! front, so no failure class is distinguishable and the token is no oracle.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64URL, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit, OsRng, Payload},
XChaCha20Poly1305, XNonce,
};
use rand::RngCore;
/// Keyset position of the last row a truncated scan fetched.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ScanPosition {
/// The row's raw stored `created_at` text, the first half of the keyset cursor.
pub created_at_key: String,
/// The row's `id`, the tiebreaking half of the keyset cursor.
pub id: String,
/// The oid hex of the CANDIDATE this position resumes. One CID can map to several
/// git oids, so a bare row cursor names a row without naming whose walk it belongs
/// to. The candidate is named by IDENTITY rather than by position in the candidate
/// list: that list is ordered by hex and mutates between rungs, so an index would
/// name a different candidate the moment anything is pinned or unpinned.
pub sha256_hex: String,
}
/// Plaintext version byte, so a future layout change is a clean open-failure
/// (treated as absent) rather than a misparse.
///
/// Bumped to 2 when the two halves stopped sharing one width (see [`ID_WIDTH`]), and
/// to 3 when the position gained the candidate oid it resumes (see [`OID_WIDTH`]).
/// A token minted under an earlier layout is a different length and a different
/// framing, so it opens to `None` and the caller restarts at the front, which is the
/// safe direction: a misparse would resume at a fabricated row and skip coverage.
const VERSION: u8 = 3;
/// Byte width the `created_at` half is padded to. Every value stored here is a
/// serialized timestamp, about 30 bytes, so 64 is roomy for the field's whole domain.
/// It is deliberately NOT widened to match [`ID_WIDTH`]: padding both halves to the id
/// width would nearly double every token for a field that can never use the space.
const CREATED_WIDTH: usize = 64;
/// Byte width the `id` half is padded to.
///
/// The bound is set by the WRITERS, not by what a typical id happens to look like.
/// `upsert_mirror_repo` builds `repos.id` as `{owner}/{name}`, and the slug validators
/// in the node's `repo_store` admit an owner of up to 255 bytes and a name of up to
/// 100, so 356 bytes is reachable through the ordinary write path and repo names are
/// peer-controllable. 384 clears that with margin.
///
/// Under-sizing this is not a cosmetic bug. A row at a truncation boundary whose id
/// exceeds the width fails the seal, the handler sheds a 503 with no continuation, and
/// a tokenless shed is byte-identical to the wrapped-scan response whose contract is
/// "the absence of a token means the ladder is over". The boundary row is deterministic
/// for a stable inventory, so every retry reproduces it and every row past it becomes
/// permanently unreachable. Anything past the width still fails loudly rather than
/// silently truncating a cursor into one that resumes at the wrong row.
const ID_WIDTH: usize = 384;
/// Byte width the candidate oid half is padded to.
///
/// Git mints exactly two oid widths and this field carries BOTH. A production repo is
/// created by `store::init_bare` with `git init --bare --object-format=sha1`, so its
/// oids are 40 hex; only the sha256 test fixtures mint 64. The field is therefore
/// length-prefixed like the two row halves rather than a bare fixed 64: a 64-only
/// field would fail every seal on a real deployment, and a failed seal sheds a
/// tokenless 503 that is byte-identical to "your ladder is over".
///
/// The padding to 64 is what keeps the WIDTH off the wire. Without it a 40-hex token
/// is 24 bytes shorter than a 64-hex one, and token length would say which object
/// format the holder's repo uses.
const OID_WIDTH: usize = 64;
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
const PLAINTEXT_LEN: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 2 + OID_WIDTH + 8;
/// Nonce width for XChaCha20-Poly1305.
const NONCE_LEN: usize = 24;
/// A fresh random 32-byte sealing key from the OS CSPRNG.
pub fn new_key() -> [u8; 32] {
let mut key = [0u8; 32];
OsRng.fill_bytes(&mut key);
key
}
// The two halves below are deliberately separate and adjacent. The framing pair owns
// "every token is the same length"; the AEAD pair owns "the contents are confidential
// and CID-bound". Keeping them apart is what lets each property be exercised (and
// broken) without disturbing the other.
/// Encode a position into the FIXED-WIDTH plaintext:
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
///
/// The padding is the point. AEAD ciphertext is plaintext-length plus the tag, and every
/// field of a scan position varies in length, so a length-prefixed encoding with no
/// padding would make token LENGTH a side channel for the sealed row and for the
/// candidate's object format. Each field is padded to its OWN fixed width, which keeps
/// every minted token the same length while letting the id half carry the range the
/// write path actually admits.
fn encode_position(pos: &ScanPosition, expires_at_unix: i64) -> anyhow::Result<Vec<u8>> {
let mut out = vec![0u8; PLAINTEXT_LEN];
out[0] = VERSION;
let mut at = 1;
for (field, width) in [
(pos.created_at_key.as_bytes(), CREATED_WIDTH),
(pos.id.as_bytes(), ID_WIDTH),
(pos.sha256_hex.as_bytes(), OID_WIDTH),
] {
if field.len() > width {
// Loud rather than truncating: a clipped cursor resumes at the wrong row and
// silently skips coverage, which is the availability half of the bug this
// token exists to fix.
anyhow::bail!(
"scan token field is {} bytes, over the {width}-byte fixed width",
field.len()
);
}
out[at..at + 2].copy_from_slice(&(field.len() as u16).to_le_bytes());
at += 2;
out[at..at + field.len()].copy_from_slice(field);
at += width;
}
out[at..at + 8].copy_from_slice(&expires_at_unix.to_le_bytes());
Ok(out)
}
/// Decode what [`encode_position`] wrote. `None` on any structural mismatch.
fn decode_position(bytes: &[u8]) -> Option<(ScanPosition, i64)> {
if bytes.len() != PLAINTEXT_LEN || bytes[0] != VERSION {
return None;
}
let mut at = 1;
let mut fields = [const { String::new() }; 3];
for (slot, width) in fields.iter_mut().zip([CREATED_WIDTH, ID_WIDTH, OID_WIDTH]) {
let len = u16::from_le_bytes([bytes[at], bytes[at + 1]]) as usize;
at += 2;
if len > width {
return None;
}
*slot = String::from_utf8(bytes[at..at + len].to_vec()).ok()?;
at += width;
}
let expires_at = i64::from_le_bytes(bytes[at..at + 8].try_into().ok()?);
let [created_at_key, id, sha256_hex] = fields;
// A zero-length candidate is a third state the encoder never mints. The front-of-table
// sentinel is empty ROW halves with a real oid, so an empty oid would hand the resume
// path a candidate that names nothing; refuse it like every other malformed frame.
if sha256_hex.is_empty() {
return None;
}
Some((
ScanPosition {
created_at_key,
id,
sha256_hex,
},
expires_at,
))
}
/// AEAD-seal `plaintext` under `key`, bound to `cid`, framed as `nonce || ciphertext`.
fn seal_bytes(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> anyhow::Result<Vec<u8>> {
let cipher = XChaCha20Poly1305::new_from_slice(key)
.map_err(|e| anyhow::anyhow!("scan token key: {e}"))?;
// A FRESH nonce per seal, from the OS CSPRNG. Under a stream cipher a repeated nonce
// repeats the keystream, and two tokens sealed under one nonce XOR to the difference
// of their plaintexts, which recovers a withheld row in full when the attacker can
// force one of the two positions. This draw is the property the whole confidentiality
// claim rests on.
let mut nonce = [0u8; NONCE_LEN];
OsRng.fill_bytes(&mut nonce);
let sealed = cipher
.encrypt(
XNonce::from_slice(&nonce),
Payload {
msg: plaintext,
// The canonical CID as associated data: a token minted while scanning for
// one CID does not authenticate against another, so it cannot be replayed
// to seed a different scan.
aad: cid.as_bytes(),
},
)
.map_err(|e| anyhow::anyhow!("scan token seal: {e}"))?;
let mut out = Vec::with_capacity(NONCE_LEN + sealed.len());
out.extend_from_slice(&nonce);
out.extend_from_slice(&sealed);
Ok(out)
}
/// Open what [`seal_bytes`] framed. `None` on any failure, including a wrong `cid`.
fn open_bytes(key: &[u8; 32], cid: &str, raw: &[u8]) -> Option<Vec<u8>> {
if raw.len() <= NONCE_LEN + 16 {
return None;
}
let (nonce, sealed) = raw.split_at(NONCE_LEN);
let cipher = XChaCha20Poly1305::new_from_slice(key).ok()?;
cipher
.decrypt(
XNonce::from_slice(nonce),
Payload {
msg: sealed,
aad: cid.as_bytes(),
},
)
.ok()
}
/// Seal `pos` under `key`, bound to `cid`, expiring at `expires_at_unix`.
///
/// Returns the base64url (no pad) token. Errors only when a field exceeds
/// its own fixed width ([`CREATED_WIDTH`], [`ID_WIDTH`], [`OID_WIDTH`]) or the AEAD
/// itself fails, never silently truncates.
pub fn seal_scan_token(
key: &[u8; 32],
cid: &str,
pos: &ScanPosition,
expires_at_unix: i64,
) -> anyhow::Result<String> {
let plaintext = encode_position(pos, expires_at_unix)?;
Ok(B64URL.encode(seal_bytes(key, cid, &plaintext)?))
}
/// Open a token minted by [`seal_scan_token`] under the same key and CID.
///
/// `None` for every failure class alike (wrong key, tampered, foreign CID, expired,
/// malformed, wrong version), so the caller can treat all of them as "absent" without
/// leaking which one occurred.
pub fn open_scan_token(
key: &[u8; 32],
cid: &str,
token: &str,
now_unix: i64,
) -> Option<ScanPosition> {
let raw = B64URL.decode(token).ok()?;
let plaintext = open_bytes(key, cid, &raw)?;
let (pos, expires_at) = decode_position(&plaintext)?;
if now_unix >= expires_at {
return None;
}
Some(pos)
}
#[cfg(test)]
mod tests {
use super::*;
/// A production-shaped candidate: `git init --bare --object-format=sha1`, so 40 hex.
const OID_40: &str = "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678";
/// A test-fixture-shaped candidate: the sha256 repos the suite creates mint 64 hex.
const OID_64: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
fn pos(created: &str, id: &str) -> ScanPosition {
pos_for(created, id, OID_40)
}
fn pos_for(created: &str, id: &str, sha256_hex: &str) -> ScanPosition {
ScanPosition {
created_at_key: created.to_string(),
id: id.to_string(),
sha256_hex: sha256_hex.to_string(),
}
}
/// Seal a hand-built plaintext through the AEAD half, so a test can frame bytes the
/// encoder would never mint (an old version byte, a zero-length oid) and still exercise
/// the real open path.
fn seal_raw(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> String {
B64URL.encode(seal_bytes(key, cid, plaintext).unwrap())
}
/// Byte offset of the oid length prefix inside the plaintext, derived from the widths
/// rather than hardcoded so a width change moves it with the layout.
const OID_LEN_AT: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH;
/// Scenario 1: both of git's oid widths round trip. The 40-hex case is the PRODUCTION
/// shape (`git init --bare --object-format=sha1`), and an all-64 fixture suite would
/// never exercise it, which is exactly how a fixed-64 field would ship broken.
#[test]
fn round_trips_at_both_oid_widths() {
let key = new_key();
for hex in [OID_40, OID_64] {
let p = pos_for("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo", hex);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("both oid widths must seal");
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 0),
Some(p),
"a {}-hex candidate must open to the identical position",
hex.len()
);
}
}
/// Scenario 2: a token framed under the OLD version opens to `None`, never a misparse.
/// Both legs matter: the version-2 plaintext was a different LENGTH, and a future
/// same-length layout would only be caught by the version byte itself.
#[test]
fn a_prior_layout_version_opens_to_none() {
let key = new_key();
// The version-2 layout verbatim: no oid field, so 461 bytes.
let mut old = vec![0u8; 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 8];
old[0] = 2;
assert_eq!(old.len(), 461, "the version-2 plaintext was 461 bytes");
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &old), 0),
None,
"a version-2 token must open to None so the caller restarts at the front"
);
// Same length, stale version byte: only the version check can refuse this one.
let mut stamped = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
stamped[0] = VERSION - 1;
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &stamped), 0),
None,
"a stale version byte must open to None even at the current width"
);
}
/// Scenario 3: token length is invariant across candidate VALUE and candidate WIDTH.
/// The padding is what keeps the oid width off the wire: without it a 40-hex token is
/// 24 bytes shorter than a 64-hex one and the length says which repo format the
/// holder uses.
#[test]
fn token_length_is_invariant_across_oid_widths() {
let key = new_key();
let created = "2020-01-01T00:00:00+00:00";
let id = "z6MkOwner/private-repo";
let short =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_40), 1 << 40).unwrap();
let long =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_64), 1 << 40).unwrap();
assert_eq!(
short.len(),
long.len(),
"a 40-hex and a 64-hex candidate must mint tokens of identical length, or the \
oid width is a side channel"
);
// The absolute width, pinned by execution rather than by arithmetic on paper. The
// gl client's mock fixtures hardcode this number (`TOKEN_LEN` in
// crates/gl/src/ipfs_cmd.rs), and nothing in that crate seals a real token, so this
// assertion is the only executable check that the two agree.
assert_eq!(
short.len(),
756,
"24 nonce + {PLAINTEXT_LEN} plaintext + 16 tag, base64url no pad"
);
}
/// Scenario 4: the front-of-table sentinel. Empty row halves with a REAL candidate
/// round trip, which is what lets a seal say "this candidate, no row cursor yet".
#[test]
fn empty_row_fields_round_trip_with_a_real_candidate() {
let key = new_key();
let p = pos_for("", "", OID_40);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("the front sentinel must seal");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
/// Scenario 6, third leg: a zero-length oid is a distinguishable third state that the
/// encoder never mints, and accepting it would hand the sentinel machinery a candidate
/// naming nothing. The decode path refuses it.
#[test]
fn a_zero_length_candidate_opens_to_none() {
let key = new_key();
let mut plaintext = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
plaintext[OID_LEN_AT..OID_LEN_AT + 2].copy_from_slice(&0u16.to_le_bytes());
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &plaintext), 0),
None,
"a zero-length candidate must open to None, not to a position naming nothing"
);
}
/// Scenario 6, second leg: an oid past the 64-byte width fails the seal loudly rather
/// than being clipped into a hex that names a different candidate.
#[test]
fn an_oid_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos_for(
"2020-01-01T00:00:03+00:00",
"z6MkOwner/private-repo",
&"a".repeat(OID_WIDTH + 1),
);
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide candidate must fail the seal, never be truncated into a hex that \
resumes the wrong candidate"
);
}
#[test]
fn round_trips_under_the_same_key_and_cid() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn every_failure_class_opens_to_none() {
let key = new_key();
let other = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&other, "bafkcid", &t, 0), None, "wrong key");
assert_eq!(
open_scan_token(&key, "bafkOTHER", &t, 0),
None,
"foreign CID"
);
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 1 << 41),
None,
"expired"
);
assert_eq!(open_scan_token(&key, "bafkcid", "!!not b64", 0), None);
assert_eq!(open_scan_token(&key, "bafkcid", "", 0), None);
let mut flipped: Vec<u8> = t.bytes().collect();
let last = flipped.len() - 1;
flipped[last] = if flipped[last] == b'A' { b'B' } else { b'A' };
assert_eq!(
open_scan_token(&key, "bafkcid", &String::from_utf8(flipped).unwrap(), 0),
None,
"tampered"
);
}
/// The id half must clear the LARGEST repo id the node's own write path admits,
/// not merely a typical one. `upsert_mirror_repo` builds `repos.id` as
/// `{owner}/{name}`, and the slug validators in `repo_store` admit 255 bytes of
/// owner and 100 of name, so 356 is reachable. A width under that turns the
/// boundary row into a seal failure, which sheds a tokenless 503 that is
/// byte-identical to "your ladder is over" and strands every row past it forever.
#[test]
fn round_trips_a_repo_id_at_the_write_paths_maximum() {
let key = new_key();
let id = format!("{}/{}", "o".repeat(255), "n".repeat(100));
assert_eq!(id.len(), 356, "255 owner + '/' + 100 name");
let p = pos("2020-01-01T00:00:03+00:00", &id);
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40)
.expect("a repo id the write path admits must seal, never fail the width");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn a_field_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", &"x".repeat(ID_WIDTH + 1));
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide field must fail the seal, never be truncated into a cursor \
that resumes at the wrong row"
);
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
//! UCAN (User Controlled Authorization Networks) — capability token types.
//!
//! UCANs let a DID delegate specific capabilities to another DID,
//! with optional expiry and revocation. gitlawb uses UCANs for:
//! with optional expiry and revocation. Twigpine uses UCANs for:
//! - Delegating push access to a branch to a CI agent
//! - Granting a reviewer the ability to approve PRs
//! - Bootstrap tokens issued at registration
@@ -57,7 +57,7 @@ impl Capability {
}
}
/// Well-known gitlawb capability strings.
/// Well-known Twigpine capability strings.
pub mod caps {
pub const GIT_PUSH: &str = "git/push";
pub const GIT_FETCH: &str = "git/fetch";
+14 -10
View File
@@ -1,6 +1,6 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
description = "The Twigpine node daemon — git hosting over HTTP with DID auth"
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
@@ -33,6 +33,7 @@ sqlx = { version = "0.8", features = ["postgres", "runtime-tokio-rustls", "chron
clap = { version = "4", features = ["derive", "env"] }
bytes = "1"
libc = "0.2"
zeroize = "1"
cid = { workspace = true }
hex = { workspace = true }
sha2 = { workspace = true }
@@ -44,14 +45,14 @@ dirs-next = "2"
# withholds a canonically-equivalent path regardless of NFC/NFD form (#101).
unicode-normalization = "0.1"
reqwest = { workspace = true }
libp2p-core = "0.43"
libp2p-gossipsub = "0.49.4"
libp2p-identify = "0.47"
libp2p-identity = { version = "0.2", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.48"
libp2p-quic = { version = "0.13", features = ["tokio"] }
libp2p-swarm = { version = "0.47", features = ["macros", "tokio"] }
multiaddr = "0.18"
libp2p-core = "0.44"
libp2p-gossipsub = "0.50.1"
libp2p-identify = "0.48"
libp2p-identity = { version = "0.3", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.49"
libp2p-quic = { version = "0.14", features = ["tokio"] }
libp2p-swarm = { version = "0.48", features = ["macros", "tokio"] }
multiaddr = "0.19"
futures = "0.3"
aws-sdk-s3 = { version = "1", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
aws-config = { version = "1", features = ["behavior-version-latest"] }
@@ -72,8 +73,11 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
libp2p-dns = { version = "0.44.0", features = ["tokio"] }
libp2p-dns = { version = "0.45.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
tempfile = "3"
# `test-util` for `#[tokio::test(start_paused = true)]`: lets the deadline tests assert
# on which timer fired instead of on wall clock.
tokio = { workspace = true, features = ["test-util"] }
File diff suppressed because it is too large. Load diff
+102 -8
View File
@@ -1,12 +1,13 @@
//! GET /api/v1/arweave/anchors — list Arweave ref-update anchors.
use axum::{
extract::{Query, State},
extract::{Extension, Query, State},
Json,
};
use serde::Deserialize;
use crate::error::Result;
use crate::db::normalize_owner_key;
use crate::error::{AppError, Result};
use crate::state::AppState;
#[derive(Debug, Deserialize)]
@@ -21,19 +22,112 @@ fn default_limit() -> i64 {
}
/// GET /api/v1/arweave/anchors
///
/// `?repo=<owner>/<name>` returns anchors for one repository and binds the
/// request to the canonical read-authorization path used by every other
/// repo-scoped read: the same `authorize_repo_read` helper, with the caller
/// identity attached. Missing repositories, quarantined mirrors, and signed
/// non-readers all collapse to the standard `404` (no existence oracle). The
/// unscoped listing is auth-only — the #121 contract — and does not admit
/// visibility filtering, which is the #136 stale-index class and explicitly
/// out of scope for this auth slice.
pub async fn list_anchors(
State(state): State<AppState>,
Query(q): Query<ListAnchorsQuery>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let limit = q.limit.min(200);
let anchors = state
.db
.list_arweave_anchors(q.repo.as_deref(), limit)
.await
.map_err(crate::error::AppError::Internal)?;
// The route's `optional_signature` layer is permissive (it admits unsigned
// legacy callers) so this in-handler check is the actual gate. A signed
// non-reader still passes this check and proceeds to the authz step below.
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
if caller.is_none() {
return Err(AppError::Unauthorized(
"authentication required for anchor listing".into(),
));
}
// Clamp before the SQL query. A negative or non-numeric limit must not
// reach Postgres as `LIMIT -1` (which it rejects as a db error / 500) — the
// contract is the same default ceiling as the listing page itself.
let limit = q.limit.clamp(0, 200);
let anchors = if let Some(repo_slug) = q.repo.as_deref() {
// Parse the user-supplied "owner/name" through the same slug validator
// the sync path uses, so a malformed query is a 400, not a 500.
let (owner, name) = match crate::git::repo_store::validate_repo_slug(repo_slug) {
Ok(parts) => parts,
Err(e) => return Err(AppError::BadRequest(format!("invalid ?repo: {e}"))),
};
// Read gate. Returns `RepoNotFound` (→ 404) indistinguishably for
// missing repos, quarantined mirrors, and signed non-readers.
let (record, _rules) =
crate::api::authorize_repo_read(&state, owner, name, caller, "/").await?;
// Build the SQL filter from the canonical stored slug, NOT from the
// user-supplied `?repo=` string. Anchor rows are written as
// `{normalize_owner_key(owner_did)}/{name}` (the short form), so a
// request that passes authz with the full `did:key:…/name` form would
// match zero rows and return a false empty page. The gate already
// verified the repo is readable; `record.owner_did` is the canonical
// identity and `record.name` is the stored name.
let stored_slug = format!("{}/{}", normalize_owner_key(&record.owner_did), record.name);
state
.db
.list_arweave_anchors(Some(&stored_slug), limit)
.await?
} else {
// No scope → no repo-read decision. Auth-only.
state.db.list_arweave_anchors(None, limit).await?
};
Ok(Json(serde_json::json!({
"anchors": anchors,
"count": anchors.len(),
})))
}
#[cfg(test)]
mod closed_pool_tests {
use super::*;
use axum::http::{Request, StatusCode};
use axum::Router;
use serde_json::Value;
use sqlx::PgPool;
use tower::ServiceExt;
/// #251: a closed pool on /api/v1/arweave/anchors must be 503 db_unavailable.
#[sqlx::test]
async fn list_anchors_closed_pool_returns_503_db_unavailable(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
pool.close().await;
let resp = Router::new()
.route("/api/v1/arweave/anchors", axum::routing::get(list_anchors))
.with_state(state)
.oneshot(
Request::builder()
.uri("/api/v1/arweave/anchors")
.extension(crate::auth::AuthenticatedDid("did:key:test".into()))
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"closed-pool outage must be retryable 503, not 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})
);
}
}
+123 -29
View File
@@ -285,11 +285,22 @@ pub async fn submit_bounty(
Path(id): Path<String>,
Json(req): Json<SubmitBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "claimed" {
return Err(AppError::BadRequest(format!(
@@ -325,11 +336,22 @@ pub async fn approve_bounty(
Path(id): Path<String>,
Json(req): Json<ApproveBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "submitted" {
return Err(AppError::BadRequest(format!(
@@ -372,11 +394,22 @@ pub async fn cancel_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "open" {
return Err(AppError::BadRequest(format!(
@@ -407,11 +440,22 @@ pub async fn dispute_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "claimed" && bounty.status != "submitted" {
return Err(AppError::BadRequest(format!(
@@ -456,21 +500,68 @@ pub async fn dispute_bounty(
Ok(Json(updated))
}
/// Resolve the set of `(owner_did, name)` pairs for repos an anonymous caller
/// can list. Mirrors the `stats()` pattern in `server.rs` (#104): batch-load
/// all deduped repos, batch-load their visibility rules, keep only those that
/// pass `listable_at_root`. Pure I/O after the two DB round-trips — no
/// per-repo authorization queries.
///
/// Both bounty stats handlers mount behind `optional_signature` (see
/// `bounty_read_routes` in server.rs), so the caller is always `None` for
/// anonymous access. Every caller, signed or not, currently receives the
/// anonymous-scoped aggregates (matching the #104 pattern); per-caller
/// visibility can be threaded through here when that work lands.
///
/// A database error is returned to the caller, never collapsed to an empty
/// set: an empty set would render as HTTP 200 with zero counts, which a client
/// reads as confirmed absence of activity.
async fn visible_repo_pairs(state: &AppState) -> Result<Vec<(String, String)>> {
let rows = state.db.list_all_repos_deduped().await?;
let ids: Vec<String> = rows.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = state.db.list_visibility_rules_for_repos(&ids).await?;
let pairs = rows
.iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
// mirror-rows-handled: a mirror with a canonical twin is deduped to the
// canonical row and its rules; a mirror-only repo counts as public, the
// same answer authorize_repo_read and /stats give. Pinned by
// bounty_stats_mirror_rows_follow_canonical_twin_or_count_when_alone.
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, None)
})
.map(|r| {
(
crate::db::normalize_owner_key(&r.owner_did).to_string(),
r.name.clone(),
)
})
.collect();
Ok(pairs)
}
/// GET /api/v1/bounties/stats
///
/// Aggregates are restricted to anonymously-listable repos so private-repo
/// bounty activity is not exposed (#477). The visible-repo set is resolved
/// once per request via `visible_repo_pairs` (two SQL round-trips), then
/// passed into the filtered aggregate queries.
pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountyStatsResponse>> {
let open = state.db.count_bounties_by_status("open").await.unwrap_or(0);
let visible = visible_repo_pairs(&state).await?;
let open = state
.db
.count_bounties_by_status_visible("open", &visible)
.await?;
let claimed = state
.db
.count_bounties_by_status("claimed")
.await
.unwrap_or(0);
.count_bounties_by_status_visible("claimed", &visible)
.await?;
let completed = state
.db
.count_bounties_by_status("completed")
.await
.unwrap_or(0);
.count_bounties_by_status_visible("completed", &visible)
.await?;
let leaders = state.db.bounty_leaderboard(10).await.unwrap_or_default();
let leaders = state.db.bounty_leaderboard_visible(10, &visible).await?;
let leaderboard = leaders
.into_iter()
.map(|(did, cnt, total)| AgentBountyEntry {
@@ -489,11 +580,14 @@ pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountySt
}
/// GET /api/v1/agents/{did}/bounties
///
/// Per-agent earnings restricted to anonymously-listable repos (#477).
pub async fn agent_bounty_stats(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<Json<serde_json::Value>> {
let (count, total) = state.db.agent_bounty_stats(&did).await.unwrap_or((0, 0));
let visible = visible_repo_pairs(&state).await?;
let (count, total) = state.db.agent_bounty_stats_visible(&did, &visible).await?;
Ok(Json(serde_json::json!({
"did": did,
"completed_bounties": count,
+14 -1
View File
@@ -1440,7 +1440,8 @@ mod ref_updates_feed_tests {
// A DB error in the gate fails closed as 500, not swallowed into an empty 200 (the
// regression the old get_repo().ok().flatten() allowed). Inject by dropping a
// column get_repo selects so its query errors.
// column get_repo selects so its query errors. Also pin the anonymous body to
// the exact opaque object (#226).
#[sqlx::test]
async fn repo_events_db_error_fails_closed_500(pool: PgPool) {
let state = test_state(pool.clone()).await;
@@ -1463,6 +1464,18 @@ mod ref_updates_feed_tests {
StatusCode::INTERNAL_SERVER_ERROR,
"a DB error must fail closed (500), never serve an empty 200"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let body = String::from_utf8_lossy(&body);
let v: serde_json::Value = serde_json::from_str(&body).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})
);
}
// Symmetric to the gate DB-error test: a DB error in the CERT fetch (after the gate
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+24 -9
View File
@@ -14,14 +14,13 @@ pub struct LabelRequest {
pub label: String,
}
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = req.label.trim().to_lowercase();
/// Canonicalize and validate a label string. `add_label` and `remove_label`
/// must agree on the form stored in the DB: `add_label` already trims and
/// lowercases before storing, so `remove_label` has to do the same to find
/// the row. Validation is shared so neither handler can drift on charset or
/// length (#344).
fn canonicalize_label(raw: &str) -> std::result::Result<String, AppError> {
let label = raw.trim().to_lowercase();
if label.is_empty() || label.len() > 50 {
return Err(AppError::BadRequest("label must be 1–50 characters".into()));
}
@@ -33,6 +32,17 @@ pub async fn add_label(
"label must contain only alphanumeric characters, hyphens, and colons".into(),
));
}
Ok(label)
}
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = canonicalize_label(&req.label)?;
let record = state
.db
@@ -59,6 +69,8 @@ pub async fn remove_label(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, label)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let label = canonicalize_label(&label)?;
let record = state
.db
.get_repo(&owner, &name)
@@ -66,7 +78,10 @@ pub async fn remove_label(
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
state.db.remove_label(&record.id, &label).await?;
let removed = state.db.remove_label(&record.id, &label).await?;
if !removed {
return Err(AppError::NotFound(format!("label '{label}'")));
}
Ok(Json(serde_json::json!({ "label": label, "removed": true })))
}
+628 -8
View File
@@ -80,15 +80,39 @@ pub(crate) fn did_matches(a: &str, b: &str) -> bool {
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// Client-facing denial shared by every repo-owner gate copy. `require_repo_owner`
/// renders it as `AppError::Forbidden`; a handler that emits its own
/// `(StatusCode, Json)` envelope instead of `AppError` (e.g.
/// `api::tasks::create_task`) must use this constant too, so two copies of one
/// gate cannot drift in observable output.
pub(crate) const REPO_OWNER_ONLY_MESSAGE: &str = "only the repo owner can perform this action";
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
/// and the author check (close policy) share one normalization.
pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> {
if did_matches(caller, &record.owner_did) {
Ok(())
} else {
Err(AppError::Forbidden(
"only the repo owner can perform this action".into(),
))
Err(AppError::Forbidden(REPO_OWNER_ONLY_MESSAGE.into()))
}
}
/// Validate a branch name using the configured git binary (storage boundaries).
pub(crate) fn validate_git_ref_with_git(
git_bin: &str,
name: &str,
) -> std::result::Result<(), crate::git::store::GitRefValidationError> {
crate::git::store::validate_git_ref_with_git(git_bin, name)
}
/// Map ref-validation failures to the correct HTTP surface: malformed caller
/// input stays 400; an inability to run git stays on the git/server-error path.
pub(crate) fn map_git_ref_validation_error(
err: crate::git::store::GitRefValidationError,
) -> AppError {
match err {
crate::git::store::GitRefValidationError::Invalid(msg) => AppError::BadRequest(msg),
crate::git::store::GitRefValidationError::GitUnavailable(msg) => AppError::Git(msg),
}
}
@@ -208,17 +232,26 @@ mod authz_guard {
(issues, "create_issue", "authorize_repo_read("),
(bounties, "create_bounty", "authorize_repo_read("),
(repos, "fork_repo", "authorize_repo_read("),
// get_by_cid gates each iterated repo row directly via visibility_check
// (KTD2a: it must NOT route through authorize_repo_read's fuzzy re-resolve).
(ipfs, "get_by_cid", "visibility_check("),
// #94 sibling read surfaces: gate private-repo metadata on read
// visibility (public repos stay anonymous; private repos 404).
// get_by_cid resolves each candidate (provenance path + legacy scan) through
// the shared `gate_and_serve`; the gate markers themselves are asserted
// below. This row proves the delegation is real, so the gate is actually
// reached rather than dead code. The delegated gate still calls
// `visibility_check` directly and never `authorize_repo_read`, so it keeps
// the property the pre-merge marker enforced: no fuzzy re-resolve.
(ipfs, "get_by_cid", "gate_and_serve("),
// Sibling read surfaces: gate private-repo metadata on read visibility
// (public repos stay anonymous; private repos 404).
(replicas, "list_replicas", "authorize_repo_read("),
(protect, "list_protected_branches", "authorize_repo_read("),
(labels, "list_labels", "authorize_repo_read("),
(events, "list_repo_events", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
// #496: create_task is signer-self AND owner-when-repo-scoped — a
// repo_id naming a hosted repo admits tasks only from its owner.
// Both halves are pinned: the did_matches row guards the signer
// binding, this one the repo-ownership gate.
(tasks, "create_task", "require_repo_owner("),
(tasks, "claim_task", "did_matches("),
(tasks, "complete_task", "did_matches("),
(tasks, "fail_task", "did_matches("),
@@ -250,6 +283,22 @@ mod authz_guard {
"visibility::require_owner must use did_matches for DID-safe owner matching"
);
// The CID read surface (#173) enforces its gate inside the shared
// `gate_and_serve`, which BOTH the provenance path and the legacy scan call, so
// the markers must live there (the get_by_cid row above only proves delegation).
// The repo's own "/" visibility check (KTD2a — never authorize_repo_read's fuzzy
// re-resolve) and the quarantine hard-drop BEFORE visibility (INV-11) are both
// load-bearing: removing either re-opens a leak on the provenance path.
let gate_body = fn_body(ipfs, "gate_and_serve");
assert!(
gate_body.contains("visibility_check("),
"gate_and_serve must gate the CID read surface via visibility_check (KTD2a)"
);
assert!(
gate_body.contains("if quarantined"),
"gate_and_serve must hard-drop a quarantined repo before the visibility gate (INV-11)"
);
for (src, func, marker) in rows {
let body = fn_body(src, func);
assert!(
@@ -259,6 +308,577 @@ mod authz_guard {
}
}
/// Completeness fence over the GraphQL mutation surface: the mirror of
/// `every_in_scope_mutation_has_its_gate` for `graphql/mutation.rs`. Kept a
/// SEPARATE guard with its own table (never folded into the REST rows above):
/// a guard over a heterogeneous set must classify per member, and these
/// same-named task rows belong to the GraphQL surface, not `api/tasks.rs`.
///
/// COMPLETENESS comes from the engine's own SDL (`schema.sdl()`), parsed as a
/// structured AST (#219), NOT introspection and NOT a text parse of source.
/// This matters: async-graphql OMITS `#[graphql(visible = false)]` fields from
/// introspection while leaving them executable, so an introspection-sourced fence
/// could be defeated by a hidden mutation. SDL includes those fields, so no
/// visibility flag (and, via the AST parse, no brace-in-description quirk) can hide
/// a live mutation. The SDL field set must equal the registered set: a new field with
/// no row fails (a repo-write cannot slip in unlisted); a registered name absent from
/// the schema fails (renamed or removed). The set is also cross-checked against
/// introspection, so any field present in SDL but hidden from introspection fails
/// named as a forbidden hidden mutation.
///
/// The MARKER check is the only source-scraped part, and it is not the
/// completeness guarantee. Three honest limits, so a green check is not misread
/// as full authorization coverage:
/// 1. A present marker is not load-bearingness (the gate may not redden on a
/// hostile probe: the #203 present-but-vacuous case). The adversarial tests
/// in `graphql/mutation.rs` are the load-bearing layer.
/// 2. A present marker is not gate-correctness (it may be the wrong bucket for
/// the operation's resource: a repo-write registered as signer-self would
/// pass, so INV-1 review remains the defense).
/// 3. The marker check scrapes the comment-masked, brace-bounded resolver body;
/// a `{`/`}` or a `did_matches(` inside a STRING literal is not masked, so it
/// could mis-slice that body or satisfy the marker vacuously (the same
/// vacuous-marker class as limit 1). Completeness is unaffected: it comes
/// from the SDL AST, not from this scrape.
#[test]
fn every_graphql_mutation_has_its_gate() {
// (Rust snake name, schema field in camelCase, expected gate markers). The
// four are Bucket C signer-self: `/graphql` uses optional_signature, so each
// resolver must call `require_signer(ctx)?` and bind the acting DID with
// `did_matches(`. A repo-write would be Bucket A (`require_repo_owner(`).
// Same-named REST rows live in the guard above against `api/tasks.rs`.
let registered: &[(&str, &str, &[&str])] = &[
(
"create_task",
"createTask",
&[
"require_signer(ctx)?",
"did_matches(",
"require_repo_owner(",
],
),
(
"claim_task",
"claimTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"complete_task",
"completeTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"fail_task",
"failTask",
&["require_signer(ctx)?", "did_matches("],
),
];
// Completeness + hidden-mutation cross-check against the REAL schema (#219),
// delegated to the generic `assert_mutation_fence` so the SDL sourcing and the
// cross-check are ALSO exercised by a synthetic hidden schema in
// `fence_rejects_a_hidden_mutation` (INV-21). Because the real MutationRoot has no
// hidden field today, that synthetic test — not a hypothetical production canary —
// is what turns reverting the SDL source or deleting the cross-check RED.
{
use crate::graphql::mutation::MutationRoot;
use crate::graphql::query::QueryRoot;
use crate::graphql::subscription::SubscriptionRoot;
let schema =
async_graphql::Schema::build(QueryRoot, MutationRoot, SubscriptionRoot).finish();
let registered_camel: Vec<&str> = registered.iter().map(|(_, c, _)| *c).collect();
assert_mutation_fence(&schema, &registered_camel);
}
// Marker present (source scrape; the vacuity limit, backstopped by the
// adversarial tests): each resolver body carries every required gate marker.
let masked = mask_comments(include_str!("../graphql/mutation.rs"));
for (snake, _, markers) in registered {
let body = resolver_body(&masked, snake);
for marker in *markers {
assert!(
body.contains(marker),
"GraphQL mutation `{snake}` is missing its gate marker `{marker}`: gate removed"
);
}
}
}
/// Version-safety self-check (#219, INV-21): the fence's completeness guarantee rests
/// on `schema.sdl()` INCLUDING `visible = false` fields while introspection omits
/// them. That is observed async-graphql behavior, not a documented contract, so pin
/// it here: a synthetic schema with a hidden mutation must have that field in its
/// SDL-derived set and NOT in its introspection-derived set. If a future
/// async-graphql upgrade ever filters SDL by visibility, the first assertion flips
/// RED — the assumption is load-bearing, not trusted.
#[test]
fn sdl_source_sees_hidden_mutations_that_introspection_omits() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(visible = false)]
async fn hidden_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// SDL source (the fix): includes the hidden field.
let sdl_fields = mutation_field_names_from_sdl(&schema.sdl());
assert!(
sdl_fields.iter().any(|f| f == "hiddenMut"),
"SDL-sourced set MUST include the visible=false mutation; if this is RED, \
async-graphql started filtering SDL by visibility and the fence's \
completeness guarantee is broken: {sdl_fields:?}"
);
// Introspection (the fence's own helper): OMITS the hidden field. This is the
// exact gap the fix closes — an introspection-sourced fence never sees `hiddenMut`.
let introspected = introspection_mutation_fields(&schema);
assert!(
!introspected.iter().any(|f| f == "hiddenMut"),
"introspection is expected to OMIT the visible=false mutation (that is the \
gap the SDL source closes); got {introspected:?}"
);
// The exact diff the fence computes: hidden ∈ (sdl \ introspection).
let hidden: Vec<&String> = sdl_fields
.iter()
.filter(|f| !introspected.iter().any(|i| i == *f))
.collect();
assert_eq!(
hidden,
vec!["hiddenMut"],
"the SDL-vs-introspection diff must surface exactly the hidden mutation"
);
}
/// INV-21 wiring guard (#219): drives the generic `assert_mutation_fence` — the SAME
/// helper the real fence calls — against a synthetic schema whose mutation is
/// `visible = false` and IS registered, so completeness passes and ONLY the
/// hidden-from-introspection cross-check can fail. This is what makes the fence's SDL
/// sourcing load-bearing without a production canary: reverting the source to
/// introspection (the hidden field then absent from the SDL set) or deleting the
/// cross-check removes this panic (or changes its message), turning the
/// `should_panic` test RED. The real MutationRoot has no hidden field, so nothing on
/// the real schema could catch that regression.
#[test]
#[should_panic(expected = "hidden from introspection")]
fn fence_rejects_a_hidden_mutation() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(visible = false)]
async fn hidden_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// Both fields registered, so completeness passes and only the cross-check fires.
assert_mutation_fence(&schema, &["visibleMut", "hiddenMut"]);
}
/// Deprecated-mutation guard (#219): a `#[graphql(deprecation = "...")]` mutation is
/// executable and present in SDL, but GraphQL introspection omits it by default. The
/// fence's introspection helper uses `includeDeprecated: true`, so a deprecated (but
/// visible) mutation is NOT mistaken for a hidden one. With it registered, the fence
/// must NOT panic. Remove `includeDeprecated: true` and this goes RED — the deprecated
/// field would look hidden and false-red as a forbidden visible=false mutation.
#[test]
fn fence_admits_a_deprecated_mutation() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(deprecation = "use visibleMut")]
async fn deprecated_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// Must not panic: the deprecated field is visible (present in introspection with
// includeDeprecated) and registered, so it is neither hidden nor unclassified.
assert_mutation_fence(&schema, &["visibleMut", "deprecatedMut"]);
}
/// Completeness + hidden-mutation cross-check for a BUILT schema (#219). Generic over
/// the schema type so the real fence AND the synthetic `fence_rejects_a_hidden_mutation`
/// / `fence_admits_a_deprecated_mutation` tests drive the SAME sourcing and cross-check.
/// Completeness is sourced from SDL (which includes `visible = false` fields); the
/// cross-check fails any SDL field hidden from introspection. This genericity is the
/// INV-21 fix: because the real MutationRoot has no hidden field, only a synthetic
/// hidden schema through THIS helper makes reverting the SDL source or removing the
/// cross-check RED, rather than leaving the class latent behind a production canary.
fn assert_mutation_fence<Q, M, S>(
schema: &async_graphql::Schema<Q, M, S>,
registered_camel: &[&str],
) where
Q: async_graphql::ObjectType + 'static,
M: async_graphql::ObjectType + 'static,
S: async_graphql::SubscriptionType + 'static,
{
let sdl_fields = mutation_field_names_from_sdl(&schema.sdl());
let introspected = introspection_mutation_fields(schema);
// No SDL field may be hidden from introspection (a visible=false mutation).
for f in &sdl_fields {
assert!(
introspected.iter().any(|i| i == f),
"GraphQL mutation `{f}` is in the schema but hidden from introspection \
(a `#[graphql(visible = false)]` field): hidden mutations are forbidden \
by this fence — make it visible so its classification is enforceable, or \
remove it"
);
}
// Completeness both directions against the registered (camelCase) set.
for f in &sdl_fields {
assert!(
registered_camel.contains(&f.as_str()),
"unclassified GraphQL mutation `{f}`: add it to \
every_graphql_mutation_has_its_gate with a deliberate gate \
(`require_repo_owner(` for a repo-write, `did_matches(` for a task op)"
);
}
for c in registered_camel {
assert!(
sdl_fields.iter().any(|f| f == c),
"registered GraphQL mutation `{c}` is not in the schema \
(renamed or removed); update the table"
);
}
}
/// The mutation field names via the engine's introspection, run against a passed
/// schema. STRUCTURED (names arrive as data). Uses `fields(includeDeprecated: true)`:
/// GraphQL's default omits `@deprecated` fields, but a `#[graphql(deprecation = "...")]`
/// mutation is still executable and present in SDL, so without this the cross-check
/// above would false-red a deprecated mutation as a hidden (visible=false) one. Only a
/// real visibility filter should trip that arm; deprecation is a lifecycle marker.
fn introspection_mutation_fields<Q, M, S>(
schema: &async_graphql::Schema<Q, M, S>,
) -> Vec<String>
where
Q: async_graphql::ObjectType + 'static,
M: async_graphql::ObjectType + 'static,
S: async_graphql::SubscriptionType + 'static,
{
use async_graphql::Value;
let resp = tokio::runtime::Runtime::new()
.expect("tokio runtime")
.block_on(schema.execute(
"{ __schema { mutationType { fields(includeDeprecated: true) { name } } } }",
));
assert!(
resp.errors.is_empty(),
"schema introspection failed: {:?}",
resp.errors
);
let mut names = Vec::new();
if let Value::Object(root) = &resp.data {
if let Some(Value::Object(sc)) = root.get("__schema") {
if let Some(Value::Object(mt)) = sc.get("mutationType") {
if let Some(Value::List(field_list)) = mt.get("fields") {
for field in field_list {
if let Value::Object(fo) = field {
if let Some(Value::String(n)) = fo.get("name") {
names.push(n.clone());
}
}
}
}
}
}
}
names
}
/// The mutation field names sourced from the built schema's SDL — the completeness
/// source (#219). Builds the same DB-free schema as the introspection helper and runs
/// its `.sdl()` (sync, no runtime needed) through [`mutation_field_names_from_sdl`].
/// SDL includes `#[graphql(visible = false)]` fields, so this source cannot be
/// defeated by hiding a mutation from introspection.
fn graphql_mutation_fields_from_sdl() -> Vec<String> {
use crate::graphql::mutation::MutationRoot;
use crate::graphql::query::QueryRoot;
use crate::graphql::subscription::SubscriptionRoot;
let schema =
async_graphql::Schema::build(QueryRoot, MutationRoot, SubscriptionRoot).finish();
mutation_field_names_from_sdl(&schema.sdl())
}
/// The mutation type's field names extracted from an async-graphql SDL string via
/// the engine's OWN parser AST (`async_graphql::parser::parse_schema`), NOT
/// introspection. This is the completeness source (#219): unlike introspection,
/// which async-graphql filters by `visible` (a `#[graphql(visible = false)]`
/// mutation is executable but ABSENT from introspection), `schema.sdl()` includes
/// every registered field, so a hidden mutation still appears here and must carry a
/// classified row. Parsing the AST (not a brace/line text scan) means a `}` inside a
/// field description cannot mis-slice the type block. camelCase, as async-graphql
/// emits. An empty result on a navigation miss is safe: the set-equality in
/// `every_graphql_mutation_has_its_gate` turns "no fields" into a loud failure for
/// every registered name, never a vacuous pass.
fn mutation_field_names_from_sdl(sdl: &str) -> Vec<String> {
use async_graphql::parser::parse_schema;
use async_graphql::parser::types::{TypeKind, TypeSystemDefinition};
let doc = parse_schema(sdl).expect("async-graphql SDL must parse");
// The mutation root type name: from the `schema { mutation: X }` block if
// present, else the GraphQL default "Mutation".
let mutation_type = doc
.definitions
.iter()
.find_map(|def| match def {
TypeSystemDefinition::Schema(s) => s
.node
.mutation
.as_ref()
.map(|m| m.node.as_str().to_string()),
_ => None,
})
.unwrap_or_else(|| "Mutation".to_string());
// The fields of that object type.
doc.definitions
.iter()
.find_map(|def| match def {
TypeSystemDefinition::Type(t) if t.node.name.node.as_str() == mutation_type => {
match &t.node.kind {
TypeKind::Object(obj) => Some(
obj.fields
.iter()
.map(|f| f.node.name.node.as_str().to_string())
.collect::<Vec<_>>(),
),
_ => None,
}
}
_ => None,
})
.unwrap_or_default()
}
#[test]
fn sdl_extractor_includes_a_hidden_visible_false_field() {
// A field that would be hidden from introspection must still be listed from SDL.
let sdl = "schema { query: QueryRoot mutation: MutationRoot }\n\
type QueryRoot { ping: Int! }\n\
type MutationRoot { createTask: Int! hiddenMut: Int! }\n";
let fields = mutation_field_names_from_sdl(sdl);
assert!(
fields.iter().any(|f| f == "createTask"),
"visible field must be listed: {fields:?}"
);
assert!(
fields.iter().any(|f| f == "hiddenMut"),
"a visible=false field (present in SDL, hidden from introspection) MUST be \
listed by the completeness source: {fields:?}"
);
}
#[test]
fn sdl_extractor_is_not_fooled_by_a_brace_in_a_field_description() {
// A `}` inside a field description must not truncate the type block (the AST
// parse is immune to the text-parse quirk the old design feared).
let sdl = "schema { query: QueryRoot mutation: MutationRoot }\n\
type QueryRoot { ping: Int! }\n\
type MutationRoot {\n\
\"\"\"a description with a } brace and { another\"\"\"\n\
createTask: Int!\n\
claimTask: Int!\n\
}\n";
let fields = mutation_field_names_from_sdl(sdl);
assert!(
fields.iter().any(|f| f == "createTask") && fields.iter().any(|f| f == "claimTask"),
"both fields must be extracted despite a brace in a description: {fields:?}"
);
}
#[test]
fn sdl_extractor_returns_empty_for_a_query_only_schema() {
let sdl = "schema { query: QueryRoot }\ntype QueryRoot { ping: Int! }\n";
assert!(
mutation_field_names_from_sdl(sdl).is_empty(),
"a schema with no mutation root yields no mutation fields (no panic)"
);
}
/// The comment-masked body of `async fn <snake>` in `graphql/mutation.rs`, bounded
/// to the resolver's OWN braces (its `{ .. }`), for the marker check. Brace-matching
/// on the comment-masked source means a `}` in a trailing helper cannot extend the
/// body and a comment marker does not count. A `{`/`}` inside a STRING literal is
/// not masked, a documented marker-check-only residual (limit 3); completeness does
/// not use this.
fn resolver_body(masked_src: &str, snake: &str) -> String {
let needle = format!("async fn {snake}(");
let start = masked_src.find(&needle).unwrap_or_else(|| {
panic!("resolver `async fn {snake}(` not found in graphql/mutation.rs")
});
let open = start
+ masked_src[start..]
.find('{')
.expect("resolver body opening brace");
let mut depth = 0i32;
let mut end = masked_src.len();
for (i, c) in masked_src[open..].char_indices() {
match c {
'{' => depth += 1,
'}' => {
depth -= 1;
if depth == 0 {
end = open + i + 1;
break;
}
}
_ => {}
}
}
masked_src[open..end].to_string()
}
/// `src` with `//` line comments and `/* */` block comments (nested) blanked to
/// spaces, WITHOUT opening a comment inside a `"..."` string (so a `"https://"` does
/// not erase the rest of its line). String CONTENTS are left intact, so a brace or
/// a gate marker inside a string is not masked; for the marker check that is a
/// documented vacuity residual (limit 3), and completeness does not use this at
/// all. Char literals and raw/byte strings are not tracked (absent in resolver
/// bodies); a `'"'` char or a raw string is the remaining narrow residual.
fn mask_comments(src: &str) -> String {
let chars: Vec<char> = src.chars().collect();
let mut out = String::with_capacity(src.len());
let mut i = 0;
let mut block_depth = 0usize;
let mut in_string = false;
while i < chars.len() {
let c = chars[i];
let next = chars.get(i + 1).copied();
if block_depth > 0 {
if c == '/' && next == Some('*') {
block_depth += 1;
out.push_str(" ");
i += 2;
} else if c == '*' && next == Some('/') {
block_depth -= 1;
out.push_str(" ");
i += 2;
} else {
out.push(if c == '\n' { '\n' } else { ' ' });
i += 1;
}
} else if in_string {
out.push(c);
if c == '\\' {
if let Some(n) = next {
out.push(n);
i += 2;
} else {
i += 1;
}
} else {
if c == '"' {
in_string = false;
}
i += 1;
}
} else if c == '"' {
in_string = true;
out.push('"');
i += 1;
} else if c == '/' && next == Some('/') {
while i < chars.len() && chars[i] != '\n' {
out.push(' ');
i += 1;
}
} else if c == '/' && next == Some('*') {
block_depth = 1;
out.push_str(" ");
i += 2;
} else {
out.push(c);
i += 1;
}
}
out
}
/// The SDL field set (the completeness source of truth, #219) is exactly the
/// registered mutations. If async-graphql adds or renames a field, this reddens
/// and forces the `every_graphql_mutation_has_its_gate` table to be updated. Uses
/// SDL, not introspection: introspection omits `visible = false` fields, so an
/// introspection-based pin here would go green while a hidden mutation existed.
#[test]
fn schema_lists_exactly_the_registered_mutations() {
let mut fields = graphql_mutation_fields_from_sdl();
fields.sort();
assert_eq!(
fields,
vec!["claimTask", "completeTask", "createTask", "failTask"],
"the SDL mutation field set is the authoritative discovered set"
);
}
/// The marker check is bounded to a resolver's OWN body and is comment- and
/// string-aware: a `//` inside a `"https://"` string does not erase a real gate on
/// that line (no false-red), and a trailing module helper's `did_matches(` does not
/// rescue a resolver whose own gate was removed (no false-green).
#[test]
fn resolver_body_is_bounded_and_comment_and_string_aware() {
let src = "\
impl MutationRoot {
async fn ok(&self) -> Result<u32> {
let _ = (\"see https://x/y\", did_matches(a, b)); // URL string then gate, one line
Ok(0)
}
async fn last(&self) -> Result<u32> {
// did_matches( only in a comment
Ok(0)
}
}
fn helper() { let _ = did_matches(z, z); }
#[cfg(test)]
mod tests { async fn t() { let _ = did_matches(x, y); } }
";
let masked = mask_comments(src);
assert!(
resolver_body(&masked, "ok").contains("did_matches("),
"a `//` inside a string must not erase the real gate on that line"
);
assert!(
!resolver_body(&masked, "last").contains("did_matches("),
"the marker check must be bounded to the resolver's own body (a trailing \
helper's marker must not rescue a resolver whose gate was removed)"
);
}
/// Proves the comment-stripping that GUARD-1 added: a marker that appears only
/// in a full-line comment (the real `replicas.rs` false-pass shape) must NOT
/// satisfy a row.
File diff suppressed because it is too large. Load diff
+118 -2
View File
@@ -50,6 +50,17 @@ pub async fn create_pr(
let target_branch = req
.target_branch
.unwrap_or_else(|| record.default_branch.clone());
// Validate both refs before they are stored, since both are later
// interpolated into git argv (git diff / worktree add / merge). Validate the
// RESOLVED target, not just a caller-supplied one: create_repo also gates
// default_branch, but validating here as well means a PR can never feed the
// git sink an unchecked ref even if a default was poisoned by an older row
// or a future path that skips create_repo's gate.
crate::api::validate_git_ref_with_git(&state.git_bin, &req.source_branch)
.map_err(crate::api::map_git_ref_validation_error)?;
crate::api::validate_git_ref_with_git(&state.git_bin, &target_branch)
.map_err(crate::api::map_git_ref_validation_error)?;
let number = state.db.next_pr_number(&record.id).await?;
let now = Utc::now().to_rfc3339();
@@ -213,7 +224,7 @@ pub async fn merge_pr(
.repo_store
.acquire_write(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
.map_err(|e| crate::api::repos::acquire_write_app_error(&e, &name))?;
let disk_path = guard.path().to_path_buf();
let merger_did = auth.0;
let merge_result = store::merge_branch(
@@ -225,7 +236,10 @@ pub async fn merge_pr(
);
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(merge_result.is_ok()).await;
// Short-circuit on a refused publish before the PR is marked merged and
// before the webhook fires. Both are irreversible announcements of a merge
// commit that only exists on this node's disk.
guard.release(merge_result.is_ok()).await.into_result()?;
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
@@ -424,3 +438,105 @@ pub async fn list_comments(
let comments = state.db.list_pr_comments(&pr.id).await?;
Ok(Json(serde_json::json!({ "comments": comments })))
}
/// #173 F1 follow-up: merge holds no admission permit, so an exhausted write-lock
/// pool must shed 503 + Retry-After.
#[cfg(test)]
mod lock_pool_shed_tests {
use super::*;
use axum::response::IntoResponse;
use sqlx::PgPool;
fn seed_repo(owner_did: &str, name: &str) -> crate::db::RepoRecord {
let now = Utc::now();
crate::db::RepoRecord {
id: Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
}
}
#[sqlx::test]
async fn merge_pr_lock_pool_exhaustion_sheds_503_not_500(pool: PgPool) {
let owner = "did:key:zMERGELOCKPOOLOWNERAAAAAAAAAAAAAAAAAAAAA";
let mut state = crate::test_support::test_state(pool.clone()).await;
state.repo_store = crate::git::repo_store::RepoStore::new(
std::path::PathBuf::from("/tmp/gitlawb-pulls-lockpool"),
None,
crate::git::repo_store::build_lock_pool(&pool, 1, std::time::Duration::from_secs(1)),
std::time::Duration::from_secs(300),
);
let repo = seed_repo(owner, "lp-merge");
let repo_id = repo.id.clone();
state.db.create_repo(&repo).await.expect("seed repo");
let now = Utc::now().to_rfc3339();
state
.db
.create_pr(&PullRequest {
id: Uuid::new_v4().to_string(),
repo_id: repo_id.clone(),
number: 1,
title: "lp".to_string(),
body: None,
author_did: owner.to_string(),
source_branch: "feature".to_string(),
target_branch: "main".to_string(),
status: "open".to_string(),
merged_by_did: None,
merged_at: None,
created_at: now.clone(),
updated_at: now,
})
.await
.expect("seed open PR");
let held = state
.repo_store
.acquire_write(owner, "other-repo")
.await
.expect("the first write takes the only lock-pool connection");
let shed = merge_pr(
State(state.clone()),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), "lp-merge".to_string(), 1)),
)
.await;
let err = shed.expect_err("an exhausted lock pool must fail the call");
let resp = err.into_response();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"merge_pr: an exhausted write-lock pool must shed 503, not a 500 git error"
);
assert_eq!(
resp.headers()
.get("retry-after")
.map(|v| v.to_str().unwrap()),
Some("1"),
"merge_pr: a capacity shed must tell the client when to retry"
);
let _ = held.release(false).await;
let admitted = merge_pr(
State(state.clone()),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), "lp-merge".to_string(), 1)),
)
.await;
assert!(
!matches!(admitted, Err(AppError::Overloaded(_))),
"with the lock pool free, merge_pr must not be shed as capacity; got {:?}",
admitted.err()
);
}
}
+9 -10
View File
@@ -39,11 +39,11 @@ pub async fn register_replica(
) -> Result<(StatusCode, Json<serde_json::Value>)> {
validate_replica_url(&req.url)?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// The mutation response carries repo metadata (name, replica count), so
// registration applies the same read-visibility decision as the listing:
// a caller who may not read the repo gets the same 404 as a missing one.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let replica_did = &auth.0;
@@ -93,11 +93,10 @@ pub async fn unregister_replica(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Same gate as register_replica: removal mutates repo-scoped metadata and
// its response discloses the replica count.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let replica_did = &auth.0;
state.db.unregister_replica(&record.id, replica_did).await?;
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -1,4 +1,4 @@
//! DID resolution — look up a node by its gitlawb DID.
//! DID resolution — look up a node by its Twigpine DID.
//!
//! First checks the local peers table (HTTP-layer peer discovery),
//! then queries the Kademlia DHT via the libp2p swarm.
@@ -14,7 +14,7 @@ use crate::state::AppState;
/// GET /api/v1/resolve/{did}
///
/// Resolves a gitlawb DID to an HTTP URL and p2p multiaddr.
/// Resolves a Twigpine DID to an HTTP URL and p2p multiaddr.
/// Checks local peer cache first, then falls back to Kademlia DHT lookup.
pub async fn resolve_did(
State(state): State<AppState>,
+92 -6
View File
@@ -30,6 +30,33 @@ fn forbidden(msg: &str) -> (StatusCode, Json<Value>) {
)
}
/// This module's error shape with an opaque body: the real error is logged
/// server-side and never serialized (#250), matching the envelope
/// `AppError::Db` renders. Also mirrors its status split: pool-class failures
/// are a retryable 503 `db_unavailable`, everything else the opaque 500
/// `db_error`, so a transient outage is not reported as permanent.
fn db_error(e: anyhow::Error) -> (StatusCode, Json<Value>) {
tracing::error!(error = %format!("{e:#}"), "task database error");
if let Some(sql) = e.chain().find_map(|c| c.downcast_ref::<sqlx::Error>()) {
if crate::error::db_unavailable(sql) {
return (
StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})),
);
}
}
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})),
)
}
// ── Request / response types ──────────────────────────────────────────────────
#[derive(Deserialize)]
@@ -101,6 +128,37 @@ pub async fn create_task(
if !crate::api::did_matches(&auth.0, &body.delegator_did) {
return Err(forbidden("delegator_did must be the authenticated signer"));
}
// #496: a caller-supplied repo_id must name a repo the caller owns. Without
// this, any signed caller can plant a task — payload and UCAN included —
// under a foreign repo id, where the named assignee can claim it. Resolve
// the id against hosted repos: a hosted, non-quarantined repo admits tasks
// only from its owner (403 otherwise). An id naming no hosted repo
// (unknown, or quarantined — which is hidden as if it did not exist, so
// 403ing it would confirm a real id) is stored verbatim as an opaque
// label: this is a write gate and does not bind an unresolved id to a
// repo. Task reads are caller-independent on this base (the list endpoints
// are not repo-scoped), before and after this change. The 403-vs-201 split
// between a hosted repo id and an unknown one is itself a one-bit oracle —
// a stranger holding a private repo's id learns it is hosted here — but
// denying foreign repos requires drawing exactly that line, and repo ids
// already surface via the task list on this base. Repo-less tasks are
// unaffected.
if let Some(repo_id) = body.repo_id.as_deref() {
let record = state.db.get_repo_by_id(repo_id).await.map_err(db_error)?;
if let Some(record) = record {
let quarantined = state
.db
.is_repo_quarantined(&record.id)
.await
.map_err(db_error)?;
if !quarantined && crate::api::require_repo_owner(&record, &auth.0).is_err() {
// Same denial text the GraphQL copy relays from
// `require_repo_owner`, so the two copies of this gate do not
// disagree in observable output.
return Err(forbidden(crate::api::REPO_OWNER_ONLY_MESSAGE));
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -117,12 +175,7 @@ pub async fn create_task(
updated_at: now,
deadline: body.deadline,
};
state.db.create_task(&task).await.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "error": e.to_string() })),
)
})?;
state.db.create_task(&task).await.map_err(db_error)?;
Ok((StatusCode::CREATED, Json(task_to_json(&task))))
}
@@ -297,3 +350,36 @@ pub async fn fail_task(
});
Ok(Json(task_to_json(&task)))
}
#[cfg(test)]
mod tests {
use super::*;
/// The gate's DB helper must mirror `AppError::Db`: a pool-level failure is
/// the retryable 503 `db_unavailable`, a query fault the opaque 500.
#[test]
fn db_error_matches_app_error_status_split() {
let unavailable = db_error(anyhow::Error::from(sqlx::Error::PoolTimedOut));
assert_eq!(unavailable.0, StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(unavailable.1 .0["error"], crate::error::DB_UNAVAILABLE_CODE);
assert_eq!(
unavailable.1 .0["message"],
crate::error::DB_UNAVAILABLE_MESSAGE
);
let query = db_error(anyhow::Error::from(sqlx::Error::Protocol(
"column \"machine_id\" does not exist".into(),
)));
assert_eq!(query.0, StatusCode::INTERNAL_SERVER_ERROR);
assert_eq!(query.1 .0["error"], "db_error");
assert_eq!(query.1 .0["message"], crate::error::DB_ERROR_MESSAGE);
assert!(!query.1 .0.to_string().contains("machine_id"));
}
/// `.context()` layers must not hide the sqlx error from the status split.
#[test]
fn db_error_sees_sqlx_through_context() {
let wrapped = anyhow::Error::from(sqlx::Error::PoolClosed).context("repo lookup failed");
assert_eq!(db_error(wrapped).0, StatusCode::SERVICE_UNAVAILABLE);
}
}
+67 -4
View File
@@ -46,9 +46,10 @@ pub async fn create_webhook(
// endpoints (SSRF). Delivery runs on the shared no-redirect client
// (main.rs), which closes the 3xx-to-internal bounce.
if !crate::api::peers::is_public_http_url(&req.url) {
return Err(AppError::BadRequest(
"webhook URL must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(),
));
return Err(AppError::BadRequest(format!(
"webhook URL {}",
crate::api::peers::PUBLIC_HTTP_URL_REQUIREMENT
)));
}
let events = req.events.unwrap_or_else(|| vec!["*".into()]);
@@ -141,7 +142,19 @@ pub async fn delete_webhook(
#[cfg(test)]
mod tests {
use crate::api::peers::is_public_http_url;
use axum::extract::{Extension, Path, State};
use axum::Json;
use chrono::Utc;
use gitlawb_core::identity::Keypair;
use sqlx::PgPool;
use uuid::Uuid;
use super::{create_webhook, CreateWebhookRequest};
use crate::api::peers::{is_public_http_url, PUBLIC_HTTP_URL_REQUIREMENT};
use crate::auth::AuthenticatedDid;
use crate::db::RepoRecord;
use crate::error::AppError;
use crate::test_support::test_state;
// create_webhook gates req.url through is_public_http_url. Pin the exact
// SSRF targets from issue #81 so the webhook path can never regress to the
@@ -152,6 +165,7 @@ mod tests {
"http://127.0.0.1:5432/",
"http://169.254.169.254/latest/meta-data/",
"http://localhost/",
"http://node.localhost/",
"http://10.0.0.5/",
"http://[::1]/",
// IPv6 transition encodings smuggling loopback v4 (6to4 / NAT64).
@@ -169,4 +183,53 @@ mod tests {
assert!(is_public_http_url("https://hooks.example.com/gitlawb"));
assert!(is_public_http_url("http://203.0.113.10:7545/"));
}
#[sqlx::test]
async fn webhook_localhost_rejection_uses_shared_public_url_contract(pool: PgPool) {
let state = test_state(pool).await;
let owner = Keypair::generate().did().to_string();
let repo_id = Uuid::new_v4().to_string();
state
.db
.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "webhook-contract".to_string(),
owner_did: owner.clone(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/webhook-contract.git".to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
let result = create_webhook(
State(state.clone()),
Extension(AuthenticatedDid(owner.clone())),
Path((owner, "webhook-contract".to_string())),
Json(CreateWebhookRequest {
url: "https://node.localhost/hook".to_string(),
secret: None,
events: None,
}),
)
.await;
match result {
Err(AppError::BadRequest(message)) => assert_eq!(
message,
format!("webhook URL {PUBLIC_HTTP_URL_REQUIREMENT}")
),
Err(error) => panic!("unexpected webhook rejection: {error}"),
Ok(_) => panic!("a .localhost webhook must be rejected"),
}
assert!(
state.db.list_webhooks(&repo_id).await.unwrap().is_empty(),
"a rejected .localhost webhook must leave no row behind"
);
}
}
+62
View File
@@ -0,0 +1,62 @@
//! #26 Split PR 2 — three-outcome recovery policy types.
//!
//! This module owns the recovery classification policy the reviewer
//! demanded: only a trustworthy, protocol-defined absence authorizes
//! a paid re-upload. Everything else stays non-terminal.
//!
//! Deliberately narrow: the ANS-104 codec lives in `ans104`, and the
//! gateway probe, the `verify_anchor` path, and the public verify
//! endpoint are deferred until the uploader, retrieval proof, and
//! recovery consumer can be proven as one vertical slice (reviewer 2,
//! round 5). A probe against a real provider contract — same-network
//! upload/read pair, redirect-safe client, envelope-supplying read
//! API, provider-documented absence evidence — belongs in that slice,
//! not here. What ships here is the policy type plus the rule that
//! only `DefinitivelyAbsent` permits spending another upload.
/// Outcome of a gateway probe for a persisted `item_id`.
// Policy surface for the recovery slice: no construction on this
// head (the probe arrives with the provider contract); the unit
// test pins the rule meanwhile.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ProbeOutcome {
/// The persisted item was served back with a verifiable envelope
/// bound to the requested id. No re-upload allowed.
Present,
/// Trustworthy, protocol-defined proof the item was never
/// accepted. Authorizes re-upload. The exact evidence that
/// establishes this is defined by the provider contract in the
/// vertical slice that introduces the probe — never by a
/// mock-only body shape.
DefinitivelyAbsent,
/// Anything else: transport failure, oversized body, bad
/// signature, id mismatch, ambiguous gateway response. The
/// outbox stays non-terminal.
Indeterminate,
}
impl ProbeOutcome {
/// True iff the recovery code is allowed to spend another paid
/// upload request. Only `DefinitivelyAbsent` qualifies.
#[allow(dead_code)] // the recovery consumer lives in a later slice
pub fn permits_reupload(self) -> bool {
matches!(self, ProbeOutcome::DefinitivelyAbsent)
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Only `DefinitivelyAbsent` authorizes a paid re-upload. This is
/// the policy the reviewer named: collapsing any ambiguous outcome
/// to absent spends a second immutable upload for the same
/// transition.
#[test]
fn only_definitively_absent_authorizes_reupload() {
assert!(!ProbeOutcome::Present.permits_reupload());
assert!(ProbeOutcome::DefinitivelyAbsent.permits_reupload());
assert!(!ProbeOutcome::Indeterminate.permits_reupload());
}
}
+275 -36
View File
@@ -34,6 +34,38 @@ use gitlawb_core::http_sig::{
};
use gitlawb_core::identity::verify;
/// Per-route-group ceiling on how much request body [`require_signature`] will
/// buffer BEFORE it has authenticated anything.
///
/// This is middleware state rather than a constant because a single number cannot
/// serve both a 2 GB git push and a 2 MB profile update: the git write group
/// passes `GITLAWB_MAX_PACK_BYTES`, every other signed group passes
/// [`crate::server::SIGNED_BODY_LIMIT`]. Making it a required piece of state means
/// a new signed route group cannot be wired up without choosing a limit.
#[derive(Clone, Copy, Debug)]
pub struct SignatureBodyLimit(pub usize);
/// The hint for an unresolvable DID, chosen by method. The hint is only true
/// for a method we have no resolver for: a did:key that parsed and then failed
/// on its key material (a small-order key, wrong multicodec, wrong length)
/// would be sent looking for the wrong problem by the alpha-support wording.
fn unresolvable_did_hint(key_id: &Did) -> &'static str {
if key_id.is_did_key() {
"the DID is a did:key whose key material did not resolve"
} else {
"only did:key is supported in alpha"
}
}
/// Bound the echoed keyid in the unresolvable-DID message. `key_id` is
/// unauthenticated request input, so reflecting it back at whatever length the
/// caller chose would amplify attacker input into the response body. The bound
/// is slack (a did:key method-id is a fixed 48 characters) and only defends
/// the echo, not the resolution.
fn bounded_did_echo(key_id: &str) -> String {
key_id.chars().take(96).collect()
}
/// Axum middleware that enforces HTTP Signature authentication (RFC 9421).
///
/// Every write request must carry:
@@ -48,20 +80,52 @@ use gitlawb_core::identity::verify;
/// 4. Resolves the did:key to an Ed25519 VerifyingKey
/// 5. Rebuilds the signing string and verifies the Ed25519 signature
/// 6. Verifies Content-Digest matches the request body
pub async fn require_signature(request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream
pub async fn require_signature(
State(SignatureBodyLimit(limit)): State<SignatureBodyLimit>,
request: Request,
next: Next,
) -> Response {
verify_signature(limit, request, next).await
}
/// The signature check itself, shared by [`require_signature`] and the optional
/// variant. `limit` bounds the pre-auth buffer; see [`SignatureBodyLimit`].
async fn verify_signature(limit: usize, request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream.
//
// The buffer is bounded, because this runs BEFORE any identity check: an
// unauthenticated caller with no DID, no signature and no repo reaches this
// line. Nothing upstream supplies the bound. On the git write routes
// `RequestBodyLimitLayer` sits on the inner router, and a later `.layer()` is
// the OUTER service, so this middleware runs first and sees the raw body. On
// every other signed group axum's 2 MB `DefaultBodyLimit` is enforced by the
// `Bytes`/`Json` extractors through a request extension, which collecting the
// body here bypasses. `Limited` stops reading at the limit rather than reading
// everything and then measuring it, which is the property that matters.
let (parts, body) = request.into_parts();
let body_bytes =
match body.collect().await {
Ok(collected) => collected.to_bytes(),
Err(_) => return (
let body_bytes = match http_body_util::Limited::new(body, limit).collect().await {
Ok(collected) => collected.to_bytes(),
// Over the limit: a distinct status from the 401 an unsigned request gets
// and the 400 below, so a client can tell "too big" from "not authenticated"
// and from "your body did not arrive".
Err(e) if e.is::<http_body_util::LengthLimitError>() => return (
StatusCode::PAYLOAD_TOO_LARGE,
Json(json!({
"error": "body_too_large",
"message": format!("request body exceeds the {limit}-byte limit for this route"),
})),
)
.into_response(),
Err(_) => {
return (
StatusCode::BAD_REQUEST,
Json(
json!({ "error": "unreadable_body", "message": "could not read request body" }),
),
)
.into_response(),
};
.into_response()
}
};
let sig_input = parts
.headers
@@ -120,7 +184,7 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
COVERED_COMPONENTS.join(", "),
missing.join(", ")
),
"hint": "See https://gitlawb.com/agents#authentication",
"hint": "See https://twigpine.com/agents#authentication",
})),
)
.into_response();
@@ -145,11 +209,14 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
StatusCode::BAD_REQUEST,
Json(json!({
"error": "unresolvable_did",
"message": format!("cannot resolve DID '{}': {e}", sig.key_id),
"hint": "only did:key is supported in alpha",
"message": format!(
"cannot resolve DID '{}': {e}",
bounded_did_echo(&sig.key_id.to_string())
),
"hint": unresolvable_did_hint(&sig.key_id),
})),
)
.into_response()
.into_response();
}
};
@@ -162,17 +229,34 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
.unwrap_or("/")
.to_string();
let content_digest = parts
// Required, never defaulted. The signing string is rebuilt from the request,
// so an absent header would make the covered `content-digest` empty on both
// sides: verification would pass and the digest-versus-body comparison below
// would have nothing to compare, leaving a signed request free to carry a
// body its signature never covered. `sign_request` always emits the header,
// so refusing here costs no conforming client anything.
let content_digest = match parts
.headers
.get("content-digest")
.and_then(|v| v.to_str().ok())
.unwrap_or("")
.to_string();
{
Some(v) => v.to_string(),
None => {
return (
StatusCode::BAD_REQUEST,
Json(json!({
"error": "missing_content_digest",
"message": "Content-Digest header is required on a signed request",
})),
)
.into_response()
}
};
let mut request_values: HashMap<String, String> = HashMap::new();
request_values.insert("@method".to_string(), method);
request_values.insert("@path".to_string(), path_and_query);
request_values.insert("content-digest".to_string(), content_digest);
request_values.insert("content-digest".to_string(), content_digest.clone());
// The @signature-params value is the part of Signature-Input after "sig1="
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(&sig_input);
@@ -217,23 +301,18 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
.into_response();
}
// Verify Content-Digest matches the actual request body
if let Some(claimed) = parts
.headers
.get("content-digest")
.and_then(|v| v.to_str().ok())
{
let actual = compute_content_digest(&body_bytes);
if claimed != actual {
return (
StatusCode::BAD_REQUEST,
Json(json!({
"error": "content_digest_mismatch",
"message": "Content-Digest does not match request body",
})),
)
.into_response();
}
// Verify Content-Digest matches the actual request body. Unconditional: the
// header's presence was established above, so there is no branch here that
// skips the comparison.
if content_digest != compute_content_digest(&body_bytes) {
return (
StatusCode::BAD_REQUEST,
Json(json!({
"error": "content_digest_mismatch",
"message": "Content-Digest does not match request body",
})),
)
.into_response();
}
tracing::info!(did = %sig.key_id, "✓ authenticated request");
@@ -248,11 +327,17 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
/// Optional variant for rolling upgrades: verify and inject `AuthenticatedDid` when
/// RFC 9421 signature headers are present, but allow legacy unsigned requests to
/// continue when no signature attempt was made.
pub async fn optional_signature(request: Request, next: Next) -> Response {
pub async fn optional_signature(
State(SignatureBodyLimit(limit)): State<SignatureBodyLimit>,
request: Request,
next: Next,
) -> Response {
let has_signature_headers = request.headers().contains_key("signature-input")
|| request.headers().contains_key("signature");
if has_signature_headers {
return require_signature(request, next).await;
// Same pre-auth buffer, same bound: presenting signature headers is not
// authentication, so this path is as reachable as the required one.
return verify_signature(limit, request, next).await;
}
next.run(request).await
}
@@ -380,7 +465,7 @@ fn human_detected(message: &str) -> impl IntoResponse {
"error": "not_an_agent",
"message": message,
"hint": "gl identity new && gl register",
"docs": "https://gitlawb.com/agents",
"docs": "https://twigpine.com/agents",
})),
)
}
@@ -423,6 +508,58 @@ mod tests {
assert!(validate_ucan_chain(&token, &node_did, &agent_did).is_ok());
}
/// The two-branch hint selection is the point of the unresolvable-DID
/// change: a did:key that failed on its key material must be told the key
/// material did not resolve, and any other method must be told only did:key
/// is supported. A regression that drops the is_did_key() check would send
/// a did:key caller after the wrong problem, so both branches are pinned.
#[test]
fn unresolvable_did_hint_distinguishes_key_material_from_method() {
// A real did:key whose key material fails to resolve (small-order):
// key-material hint.
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let weak_vk = ed25519_dalek::VerifyingKey::from_bytes(&weak).unwrap();
let weak_did = Did::from_verifying_key(&weak_vk);
assert!(
weak_did.to_verifying_key().is_err(),
"fixture precondition: small-order did:key must not resolve"
);
assert_eq!(
unresolvable_did_hint(&weak_did),
"the DID is a did:key whose key material did not resolve"
);
// A method with no resolver: alpha-support hint.
let web = Did::web("example.com");
assert!(
web.to_verifying_key().is_err(),
"fixture precondition: did:web must not resolve locally"
);
assert_eq!(
unresolvable_did_hint(&web),
"only did:key is supported in alpha"
);
}
/// The unresolvable-DID message echoes the keyid, which is unauthenticated
/// request input. The echoed value must be bounded so a hostile caller
/// cannot amplify arbitrary-length input into the response body.
#[test]
fn unresolvable_did_message_bounds_the_echoed_keyid() {
let long_keyid = format!("did:key:z{}", "A".repeat(2000));
let shown = bounded_did_echo(&long_keyid);
assert!(
shown.len() <= 96,
"echoed keyid must be bounded, got {} chars",
shown.len()
);
assert!(
!shown.contains(&"A".repeat(1000)),
"the full keyid must not survive into the message"
);
}
#[test]
fn validate_ucan_chain_wrong_issuer() {
let node = Keypair::generate();
@@ -490,6 +627,7 @@ mod tests {
use clap::Parser;
let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
@@ -516,10 +654,37 @@ mod tests {
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
close_issue_rate_limiter: RateLimiter::new(120, Duration::from_secs(3600)),
ipfs_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_work_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
ipfs_max_legacy_probes: crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST,
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
ipfs_max_legacy_scan_rows: crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
ipfs_max_legacy_scan_rule_bytes:
crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new_bounded(12, Duration::from_secs(60), 10_000),
shutdown_tx: tokio::sync::watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_push_advert_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
encrypt_inflight: crate::state::EncryptInflight::new(),
repo_write_leases: crate::state::RepoWriteLeases::new(8),
git_read_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_push_advert_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
git_write_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_ipfs_walk_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_bin: "git".to_string(),
}
}
@@ -610,4 +775,78 @@ mod tests {
let body_json: serde_json::Value = serde_json::from_slice(&body_bytes).unwrap();
assert_eq!(body_json["error"], "invalid_ucan");
}
/// A request that signs `content-digest` as the EMPTY string and then sends
/// no Content-Digest header at all must be refused before any handler runs.
///
/// This is the whole point of requiring the header rather than defaulting it:
/// the middleware rebuilds the signing string from the request, so an absent
/// header makes the covered digest empty on both sides. The Ed25519 check
/// passes, the body-versus-digest comparison is skipped for want of a header,
/// and a signed request carries a body the signature never covered.
#[tokio::test]
async fn signed_request_without_content_digest_is_rejected() {
use base64::{engine::general_purpose::STANDARD, Engine};
use std::sync::atomic::{AtomicBool, Ordering};
static REACHED: AtomicBool = AtomicBool::new(false);
let kp = Keypair::generate();
let did = kp.did();
let created = chrono::Utc::now().timestamp();
let signature_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created={created}"#
);
let sig_params_value = &signature_input["sig1=".len()..];
let mut values: HashMap<String, String> = HashMap::new();
values.insert("@method".to_string(), "POST".to_string());
values.insert("@path".to_string(), "/x".to_string());
// Exactly what the middleware derives when the header is absent.
values.insert("content-digest".to_string(), String::new());
let signing_string =
build_signing_string(COVERED_COMPONENTS, sig_params_value, &values).unwrap();
let signature = format!(
"sig1=:{}:",
STANDARD.encode(kp.sign(signing_string.as_bytes()).to_bytes())
);
let app = Router::new()
.route(
"/x",
axum::routing::post(|| async {
REACHED.store(true, Ordering::SeqCst);
StatusCode::OK
}),
)
.layer(middleware::from_fn_with_state(
SignatureBodyLimit(crate::server::SIGNED_BODY_LIMIT),
require_signature,
));
let req = Request::builder()
.method("POST")
.uri("/x")
.header("signature-input", signature_input)
.header("signature", signature)
.body(Body::from("a body no signature covers"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::BAD_REQUEST,
"a signed request with no Content-Digest header must be refused"
);
let bytes = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(
body["error"], "missing_content_digest",
"the refusal must name the absent Content-Digest header"
);
assert!(
!REACHED.load(Ordering::SeqCst),
"the handler must never see a request whose body no signature covers"
);
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! Embedded seed list of public Gitlawb network nodes.
//! Embedded seed list of public Twigpine network nodes.
//!
//! This module parses `bootstrap-peers.json` (embedded at compile time) and
//! merges its contents into the runtime config so a fresh `docker compose up`
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+123 -7
View File
@@ -69,6 +69,12 @@ fn resolve_all_recipients(dids: &BTreeSet<String>) -> Result<Vec<VerifyingKey>,
enum SealPlan {
/// An existing envelope already covers exactly this recipient set; nothing to do.
SkipUnchanged,
/// The stored tag matches, but the stored reader set contains a DID that
/// must not resolve (e.g. a small-order did:key sealed before the choke
/// point landed). The old envelope is already public and cannot be
/// withdrawn, but it must be surfaced for operator action rather than
/// silently skipped as unchanged.
SkipUnresolvableStored(Vec<String>),
/// No recipient DID resolved to a key, so there is nothing to seal to.
SkipNoRecipients,
/// At least one recipient DID is unresolvable. Fail closed: never seal to a
@@ -90,15 +96,30 @@ enum SealPlan {
/// recover the blob; reader removal is not retroactive (the old envelope is
/// already public). The comparison is on the opaque node-keyed tag, never the
/// DID list.
///
/// A matching stored tag is normally a skip, but the set is still resolved
/// before skipping: a reader that used to resolve and now must not (a
/// small-order did:key sealed before the choke point landed) is surfaced as
/// [`SealPlan::SkipUnresolvableStored`] instead of being silently treated as
/// unchanged. The old envelope is already public and cannot be withdrawn, but
/// the operator must be able to see that a persisted envelope was sealed to a
/// weak recipient.
fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&str>) -> SealPlan {
let tag = recipients_tag(node_seed, dids);
if stored_tag == Some(tag.as_str()) {
return SealPlan::SkipUnchanged;
}
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
// The stored envelope already covers exactly this set, so there is
// nothing to re-seal. Resolve anyway so a now-unresolvable reader in an
// unchanged set is surfaced rather than hidden by the tag match.
match resolve_all_recipients(dids) {
Ok(_) => SealPlan::SkipUnchanged,
Err(unresolved) => SealPlan::SkipUnresolvableStored(unresolved),
}
} else {
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
}
}
}
@@ -131,6 +152,21 @@ pub async fn encrypt_and_pin(
// covered (which would permanently lock out the dropped readers, #47).
let (keys, tag) = match plan_seal(node_seed, dids, stored_tag.as_deref()) {
SealPlan::SkipUnchanged => continue,
SealPlan::SkipUnresolvableStored(unresolved) => {
// A stored envelope already covers this exact reader set, and
// the set now contains a DID that must not resolve (a
// small-order did:key sealed before the choke point landed).
// The old envelope is already public and cannot be withdrawn
// or re-sealed; surface it so the operator can act.
let sample: Vec<&String> = unresolved.iter().take(3).collect();
tracing::warn!(
oid = %oid,
unresolved_count = unresolved.len(),
unresolved_sample = ?sample,
"stored envelope was sealed to a recipient that must not resolve; operator action may be required"
);
continue;
}
SealPlan::SkipNoRecipients => {
tracing::warn!(oid = %oid, "no resolvable recipient keys; skipping encrypted pin");
continue;
@@ -170,7 +206,7 @@ pub async fn encrypt_and_pin(
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
tracing::warn!(oid = %oid, "pin_git_object returned no cid; skipping");
@@ -206,6 +242,49 @@ mod tests {
use super::*;
use ed25519_dalek::SigningKey;
/// A did:key encoding a small-order point. Well-formed, indistinguishable
/// from a real one by eye, and refused at resolution since the choke-point
/// guard landed.
fn weak_did_key() -> String {
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let vk = ed25519_dalek::VerifyingKey::from_bytes(&weak).expect("decompresses");
Did::from_verifying_key(&vk).to_string()
}
/// The #47 fail-closed arm already refuses to seal to a partial recipient
/// set. Because a small-order did:key no longer RESOLVES, it lands in the
/// unresolved set and that existing arm catches it: nothing is sealed, and
/// no new code path was added to make that happen.
#[test]
fn weak_recipient_falls_into_the_existing_fail_closed_arm() {
let weak = weak_did_key();
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(weak.clone());
match plan_seal(&[0u8; 32], &dids, None) {
SealPlan::SkipUnresolvable(unresolved) => assert!(
unresolved.contains(&weak),
"the weak DID must be named in the unresolved set, got {unresolved:?}"
),
other => panic!("expected SkipUnresolvable, got {other:?}"),
}
}
/// Must-not-over-reject control: an all-legitimate recipient set still seals.
#[test]
fn legitimate_recipient_set_still_seals() {
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(did_key(11));
match plan_seal(&[0u8; 32], &dids, None) {
SealPlan::Seal { keys, .. } => assert_eq!(keys.len(), 2, "both recipients resolve"),
other => panic!("expected Seal, got {other:?}"),
}
}
fn did_key(seed: u8) -> String {
let vk = SigningKey::from_bytes(&[seed; 32]).verifying_key();
Did::from_verifying_key(&vk).to_string()
@@ -348,6 +427,43 @@ mod tests {
));
}
/// P1 remediation (review): an unchanged reader set is normally a skip, but
/// the skip must not hide a recipient that now fails to resolve. A blob
/// sealed to a small-order did:key before the choke-point guard landed keeps
/// its old CID and stays decryptable-without-a-private-key unless the
/// matching-tag path surfaces it. This pins that the stored-tag match still
/// resolves the set instead of short-circuiting.
#[test]
fn plan_seal_surfaces_unresolvable_recipient_in_unchanged_reader_set() {
let weak = weak_did_key();
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(weak.clone());
let stored = recipients_tag(&SEED, &dids);
match plan_seal(&SEED, &dids, Some(&stored)) {
SealPlan::SkipUnresolvableStored(unresolved) => assert!(
unresolved.contains(&weak),
"the weak DID must be named in the unresolved set, got {unresolved:?}"
),
other => panic!("expected SkipUnresolvableStored, got {other:?}"),
}
}
/// Degenerate control: an empty set with a matching stored tag stays a
/// silent skip. The state is unreachable in practice (a seal is never
/// recorded for an empty reader set), so this pins only that the
/// resolve-on-match path must not turn it into a re-seal or a warning.
#[test]
fn plan_seal_unchanged_empty_set_still_skips() {
let empty = BTreeSet::new();
let empty_tag = recipients_tag(&SEED, &empty);
assert!(matches!(
plan_seal(&SEED, &empty, Some(&empty_tag)),
SealPlan::SkipUnchanged
));
}
#[test]
fn plan_seal_reseals_when_recipient_set_changed() {
// A stored tag for a DIFFERENT set is a miss: a newly added reader must
+278 -11
View File
@@ -35,18 +35,49 @@ pub enum AppError {
#[error("invalid request: {0}")]
BadRequest(String),
/// A DID was well-formed enough to carry to a resolver but no verifying key
/// could be derived from it. Its own code rather than plain `bad_request`
/// because the auth middleware already answers `unresolvable_did` for the
/// same failure on a request's keyid, and a client should not have to
/// substring-match a message to tell this apart from the other validation
/// failures on the same route.
#[error("unresolvable did: {0}")]
UnresolvableDid(String),
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
/// A bounded search that could not complete. `continuation`, when present, is the
/// sealed scan position the caller echoes as `?scan=` to resume where the search
/// stopped (#173 round 13, F2). It is AEAD-sealed at the mint site, never plaintext:
/// the row it names is by construction one the caller was denied (INV-13).
#[error("search incomplete: {message}")]
SearchIncomplete {
message: String,
continuation: Option<String>,
},
#[error("git error: {0}")]
Git(String),
#[error("git service timed out: {0}")]
Timeout(String),
#[error("server overloaded: {0}")]
Overloaded(String),
#[error("repository is busy")]
RepoBusy,
#[error("repository is temporarily unavailable")]
RepoUnavailable,
#[error("repository write was fenced by a concurrent publish")]
RepoWriteFenced,
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
@@ -60,9 +91,17 @@ pub enum AppError {
pub const DB_UNAVAILABLE_CODE: &str = "db_unavailable";
pub const DB_UNAVAILABLE_MESSAGE: &str = "database is temporarily unavailable";
/// Generic client-facing message for `AppError::Internal`. The real error is
/// logged server-side; never put sqlx/anyhow detail in the HTTP body (#226).
pub const INTERNAL_ERROR_MESSAGE: &str = "an internal error occurred";
/// Generic client-facing message for non-unavailable `AppError::Db`. Query /
/// schema errors stay in logs; the HTTP body must not leak them (#226).
pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
pub(crate) fn db_unavailable(e: &sqlx::Error) -> bool {
matches!(
e,
sqlx::Error::PoolTimedOut
@@ -80,7 +119,31 @@ impl From<anyhow::Error> for AppError {
fn from(err: anyhow::Error) -> Self {
match err.downcast::<sqlx::Error>() {
Ok(sql) => AppError::Db(sql),
Err(err) => AppError::Internal(err),
// Lock contention is transient and ordinary, so it must not land as a
// 500. The internal message names the owner slug and repo, so the
// variant carries nothing: the detail stays in the log at the raise
// site and the client gets a fixed retryable body.
Err(err) => match err.downcast::<crate::git::repo_store::LockPoolBusy>() {
Ok(_) => AppError::Overloaded("git write locks at capacity, retry shortly".into()),
Err(err) => match err.downcast::<crate::git::repo_store::RepoBusy>() {
Ok(_) => AppError::RepoBusy,
// Same reasoning one rung down: a refused under-lock refresh is a
// transient storage condition, and its internal message names the
// owner slug and repo, so the variant carries nothing.
Err(err) => match err.downcast::<crate::git::repo_store::RepoUnavailable>() {
Ok(_) => AppError::RepoUnavailable,
// And one more rung: a publish the store refused twice is
// transient in the same way, and the retry is the client's
// to make. The variant carries nothing for the same reason
// as the two above.
Err(err) => match err.downcast::<crate::git::repo_store::RepoWriteFenced>()
{
Ok(_) => AppError::RepoWriteFenced,
Err(err) => AppError::Internal(err),
},
},
},
},
}
}
}
@@ -132,35 +195,123 @@ impl IntoResponse for AppError {
// IcaptchaProofRequired is handled above (it carries extra headers/fields).
AppError::IcaptchaProofRequired { .. } => unreachable!("handled before this match"),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::UnresolvableDid(msg) => {
(StatusCode::BAD_REQUEST, "unresolvable_did", msg.clone())
}
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
AppError::Incomplete(msg) => {
(StatusCode::UNPROCESSABLE_ENTITY, "incomplete", msg.clone())
}
// A bounded search that could not complete (the CID resolver hit its
// legacy-probe or walk ceiling), distinct from the 404 that asserts a
// definitive not-found: absence was NOT proven, so the caller should
// retry rather than treat it as gone (#173, F2). 503, retryable.
AppError::SearchIncomplete { message, .. } => (
StatusCode::SERVICE_UNAVAILABLE,
"search_incomplete",
message.clone(),
),
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
// 504, distinct from the 500 git_error and from the read-gate's 404 /
// the auth 401, so the client can tell a deadline from a failure.
AppError::Timeout(msg) => (StatusCode::GATEWAY_TIMEOUT, "git_timeout", msg.clone()),
// 503 with a FIXED body: the caller should retry, and must not be told
// which repo is contended or for how long.
AppError::RepoBusy => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_busy",
"repository is busy — retry".into(),
),
// 503 with a FIXED body for the same reason: the caller should retry, and
// must not be told which repo could not be refreshed or why.
AppError::RepoUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_unavailable",
"repository is temporarily unavailable, retry".into(),
),
// 503 with a FIXED body again, and its own code: the caller should
// retry, but the condition is not contention, so a client that
// distinguishes them should be able to. The body must not say which
// repo lost its publish or to whom.
AppError::RepoWriteFenced => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_write_fenced",
"repository changed underneath this write, retry".into(),
),
AppError::Db(e) if db_unavailable(e) => (
StatusCode::SERVICE_UNAVAILABLE,
DB_UNAVAILABLE_CODE,
DB_UNAVAILABLE_MESSAGE.into(),
),
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
e.to_string(),
),
// 503 with a Retry-After (attached after this match — the shared tail
// can't carry per-variant headers). This is the single place Overloaded
// becomes a response, so it can never ship a 503 without the retry hint.
AppError::Overloaded(msg) => {
(StatusCode::SERVICE_UNAVAILABLE, "overloaded", msg.clone())
}
// Opaque body + server log: bare `?` on sqlx paths becomes `AppError::Db`
// via `From`, so this arm (not `Internal`) is the common leak for open
// routes like GET /api/v1/repos and GET /api/v1/peers (#226).
AppError::Db(e) => {
tracing::error!(error = %e, "database error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"db_error",
DB_ERROR_MESSAGE.into(),
)
}
// Opaque body: handlers that map with `.map_err(AppError::Internal)`
// (e.g. GET /ipfs/{cid}) land here; other DB failures usually hit `Db`.
// Log `{e:#}` so context-wrapped anyhow chains keep the leaf cause
// (Display alone is only the outermost layer; see api/repos.rs).
AppError::Internal(e) => {
tracing::error!(error = %format!("{e:#}"), "internal error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
INTERNAL_ERROR_MESSAGE.into(),
)
}
};
let body = Json(json!({
let mut body = json!({
"error": code,
"message": message,
}));
});
// A truncated CID search may carry the sealed position the caller echoes as
// `?scan=` to resume. Rendered as a third body field, present only when the
// shed actually left something to resume: a wrapped scan and a throttled
// request both omit it, and its ABSENCE is what tells a caller the ladder is
// over. It is opaque ciphertext; see `gitlawb_core::scan_token`.
if let AppError::SearchIncomplete {
continuation: Some(token),
..
} = &self
{
body["continuation"] = json!(token);
}
(status, body).into_response()
let mut resp = (status, Json(body)).into_response();
// Both retryable 503s advertise when to retry: Overloaded (capacity shed) and
// SearchIncomplete (a bounded CID search cut short by a cap — retry may complete
// it). They ride the shared tail above for body/status, so the header is attached
// here rather than in bespoke early returns, keeping each variant handled once.
if matches!(
self,
AppError::Overloaded(_)
| AppError::SearchIncomplete { .. }
| AppError::RepoBusy
| AppError::RepoUnavailable
| AppError::RepoWriteFenced
) {
resp.headers_mut().insert(
axum::http::header::RETRY_AFTER,
axum::http::HeaderValue::from_static("1"),
);
}
resp
}
}
@@ -182,4 +333,120 @@ mod tests {
StatusCode::INTERNAL_SERVER_ERROR
);
}
#[test]
fn overloaded_maps_to_503_with_retry_after() {
let resp = AppError::Overloaded("x".into()).into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
/// #226: raw sqlx/DB detail must never appear in the Internal 500 body.
#[tokio::test]
async fn internal_error_body_is_opaque() {
use serde_json::{json, Value};
let leak = "error returned from database: relation \"repos\" does not exist";
let resp = AppError::Internal(anyhow::anyhow!("{leak}")).into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
// Exact object: a new `detail` field with different sensitive text must
// also fail, not only a repeat of the original error string.
assert_eq!(
v,
json!({
"error": "internal_error",
"message": INTERNAL_ERROR_MESSAGE,
})
);
}
/// #226: `AppError::Db` query errors (the common `?` path) must also be opaque.
#[tokio::test]
async fn db_error_body_is_opaque() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::Protocol(
"error returned from database: column \"is_public\" does not exist".into(),
))
.into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": "db_error",
"message": DB_ERROR_MESSAGE,
})
);
}
/// Connection-level failures must stay 503 `db_unavailable`, not collapse
/// into the opaque 500 `db_error` arm if `db_unavailable` loses a variant.
#[tokio::test]
async fn db_pool_timeout_stays_503_unavailable() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::PoolTimedOut).into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": DB_UNAVAILABLE_CODE,
"message": DB_UNAVAILABLE_MESSAGE,
})
);
}
/// #251: bare `?` on anyhow-wrapped sqlx relies on this downcast so a
/// closed pool becomes 503 `db_unavailable`, not 500 `internal_error`.
#[test]
fn pool_closed_via_anyhow_from_is_503_db_unavailable() {
let err: AppError = anyhow::Error::from(sqlx::Error::PoolClosed).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
}
#[test]
fn lock_pool_busy_via_anyhow_from_is_503_overloaded() {
let err: AppError = anyhow::Error::new(crate::git::repo_store::LockPoolBusy).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
#[test]
fn repo_busy_unavailable_and_fenced_advertise_retry_after() {
for err in [
AppError::RepoBusy,
AppError::RepoUnavailable,
AppError::RepoWriteFenced,
] {
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
}
}
+158 -28
View File
@@ -61,6 +61,27 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
Ok(())
}
/// Remove a single issue ref while the write guard still holds the advisory lock,
/// after a definite publish refusal. Best-effort: a failed delete leaves a
/// local-only orphan, which is still better than telling the client to retry into a
/// duplicate id. Not used on `UploadUnknowable`, where the PUT may still land.
pub fn delete_issue_ref(
git_bin: &str,
repo_path: &Path,
issue_id: &str,
deadline: std::time::Instant,
) -> Result<()> {
let ref_name = format!("refs/gitlawb/issues/{issue_id}");
crate::git::visibility_pack::run_bounded_git(
git_bin,
&["update-ref", "-d", &ref_name],
repo_path,
b"",
deadline,
)?;
Ok(())
}
/// List all issue refs and return their JSON content.
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
@@ -108,19 +129,22 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
/// Returns Ok(Some(full_id)) on unique match, Ok(None) if not found,
/// Err if the prefix is ambiguous (matches more than one issue).
pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<String>> {
// Try exact match first — fast path for callers passing the full UUID.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let check = Command::new("git")
.args(["cat-file", "-e", &exact_ref])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
if check.status.success() {
return Ok(Some(id_or_prefix.to_string()));
// The id comes from the request path and is spliced into a git revision and a
// for-each-ref pattern below. Issue ids are UUIDs, so accept only ASCII
// alphanumerics and '-': anything else (`^{blob}`, `@{0}`, `*`, `/`, `..`) is
// revision or glob syntax, never an issue id, and is reported as not found.
if id_or_prefix.is_empty()
|| !id_or_prefix
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'-')
{
return Ok(None);
}
// Prefix search: list all refs that start with the given string.
let prefix_glob = format!("refs/gitlawb/issues/{id_or_prefix}*");
// List all refs that start with the given string; the glob also matches the exact
// ref. The id handed back is always git's own refname, never the request input.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let prefix_glob = format!("{exact_ref}*");
let list = Command::new("git")
.args(["for-each-ref", "--format=%(refname)", &prefix_glob])
.current_dir(repo_path)
@@ -132,25 +156,44 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
}
let output = String::from_utf8_lossy(&list.stdout);
let matches: Vec<&str> = output.lines().filter(|l| !l.trim().is_empty()).collect();
let matches: Vec<&str> = output
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
match matches.len() {
0 => Ok(None),
1 => {
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = matches[0]
.trim()
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(matches[0].trim())
.to_string();
Ok(Some(full_id))
// An exact id wins over longer ids that share it as a prefix, but only if its
// object exists: for-each-ref also lists a ref whose blob is missing, so a
// dangling exact ref falls through to the prefix rules below.
let exact = match matches.iter().find(|m| **m == exact_ref) {
Some(m) => {
let check = Command::new("git")
.args(["cat-file", "-e", m])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
check.status.success().then_some(*m)
}
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
}
None => None,
};
let found = match exact {
Some(m) => m,
None => match matches.as_slice() {
[] => return Ok(None),
[m] => *m,
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
},
};
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = found
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(found)
.to_string();
Ok(Some(full_id))
}
/// Close an issue by updating its status to "closed" in the git ref.
@@ -234,6 +277,43 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string()));
}
#[test]
fn test_resolve_exact_id_wins_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open"}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open"}"#).unwrap();
let resolved = resolve_issue_id(dir.path(), "abc12345").unwrap();
assert_eq!(resolved, Some("abc12345".to_string()));
}
#[test]
fn test_resolve_dangling_exact_id_does_not_win_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open","n":1}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open","n":2}"#).unwrap();
// Dangle the exact ref: the ref stays listed by for-each-ref, but its blob is gone.
let out = Command::new("git")
.args(["rev-parse", "refs/gitlawb/issues/abc12345"])
.current_dir(dir.path())
.output()
.unwrap();
let hash = String::from_utf8(out.stdout).unwrap();
let hash = hash.trim();
std::fs::remove_file(
dir.path()
.join(".git/objects")
.join(&hash[..2])
.join(&hash[2..]),
)
.unwrap();
let result = resolve_issue_id(dir.path(), "abc12345");
assert!(result.unwrap_err().to_string().contains("ambiguous"));
assert!(close_issue(dir.path(), "abc12345").is_err());
}
#[test]
fn test_resolve_prefix_matches_unique() {
let dir = TempDir::new().unwrap();
@@ -249,6 +329,35 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string()));
}
#[test]
fn test_resolve_rejects_revision_syntax() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "abc12345-0000-0000-0000-000000000000";
create_issue(
dir.path(),
full_id,
r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
// Each of these resolves to (or globs over) the issue blob in git, but none
// is an issue id, so the lookup must not accept it.
for crafted in [
"abc12345-0000-0000-0000-000000000000^{blob}",
"abc12345-0000-0000-0000-000000000000^{}",
"abc12345-0000-0000-0000-000000000000~0",
"*",
"",
] {
assert_eq!(
resolve_issue_id(dir.path(), crafted).unwrap(),
None,
"{crafted:?}"
);
assert_eq!(get_issue(dir.path(), crafted).unwrap(), None, "{crafted:?}");
}
}
#[test]
fn test_resolve_prefix_not_found() {
let dir = TempDir::new().unwrap();
@@ -278,6 +387,27 @@ mod tests {
assert!(result.unwrap_err().to_string().contains("ambiguous"));
}
#[test]
fn delete_issue_ref_removes_a_created_ref() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "eee88888-0000-0000-0000-000000000000";
create_issue(
dir.path(),
full_id,
r#"{"id":"eee88888-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
delete_issue_ref(
"git",
dir.path(),
full_id,
std::time::Instant::now() + std::time::Duration::from_secs(30),
)
.unwrap();
assert_eq!(resolve_issue_id(dir.path(), full_id).unwrap(), None);
}
#[test]
fn test_close_issue_via_prefix() {
let dir = TempDir::new().unwrap();
+1
View File
@@ -1,4 +1,5 @@
pub mod issues;
pub mod publish;
pub mod push_delta;
pub mod repo_store;
pub mod smart_http;
+358
View File
@@ -0,0 +1,358 @@
//! The publication boundary: WHO wrote, and HOW FAR that write got.
//!
//! Four lifecycle defects on this branch shared one root cause. A resource was
//! tracked by a coarse state — "release started", "the path exists", "a row with
//! this name exists" — when the safety question was about the IDENTITY and the
//! PUBLICATION STAGE of one specific write attempt. A tail replicated refs from
//! an attempt that had not dispatched a PUT; a read served a tree whose
//! generation nobody had confirmed; a fork claimed a concurrent attempt's row and
//! deleted a successor's object; a response-loss failure was compensated as if it
//! proved non-publication.
//!
//! This module is the vocabulary that makes those questions answerable, and it is
//! deliberately free of any object-storage type:
//!
//! - [`PublishAttemptId`] — minted before the request is built, carried with the
//! bytes as user metadata, read back off the store to decide whether what is
//! published is THIS attempt's work.
//! - [`PublishStage`] / [`PublishStageCell`] — how far one attempt got, observable
//! from outside the future that is performing it, so a CANCELLED attempt can
//! still be classified.
//! - [`UploadError`] — what the client KNOWS about dispatch and commit, not just
//! which HTTP status came back. Destructive compensation is licensed only by
//! [`UploadError::proves_not_published`].
//!
//! # Porting note (#79)
//!
//! PR #79 (`feat/storage-abstraction`) deletes `git/tigris.rs` and replaces it
//! with a `BlobStore`/`RepoArchive` layer. Everything in this file is
//! backend-agnostic on purpose and is meant to survive that swap unchanged: a new
//! backend supplies its own error classifier (the one in `tigris.rs` is a single
//! private function) and keeps these types as the contract its callers read.
use std::sync::Mutex;
/// A durable identity for ONE publish attempt.
///
/// The point is reconciliation. A conditional PUT whose response was lost leaves
/// the client unable to say whether its bytes committed; an attempt id stored
/// alongside those bytes turns that into a question the store can answer, because
/// "is the published object mine?" is decidable where "did my request succeed?"
/// is not.
///
/// Fork creation passes the DB row id it is about to insert, so the object, the
/// on-disk clone and the database row are all stamped with the same attempt
/// identity and every cleanup can be made conditional on still owning all three.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PublishAttemptId(String);
impl PublishAttemptId {
/// A fresh identity for an attempt that has nothing else to be named after.
pub fn new() -> Self {
Self(uuid::Uuid::new_v4().to_string())
}
/// Name the attempt after a caller-owned identity — fork creation uses the
/// `record.id` it is about to insert, which is what ties the object back to
/// the exact row rather than to the logical owner/name.
pub fn from_owned(id: impl Into<String>) -> Self {
Self(id.into())
}
pub fn as_str(&self) -> &str {
&self.0
}
}
impl Default for PublishAttemptId {
fn default() -> Self {
Self::new()
}
}
impl std::fmt::Display for PublishAttemptId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.0)
}
}
/// The object user-metadata key an attempt id travels in.
///
/// S3 lowercases user-metadata keys, so this must already be lowercase or the
/// read back would never match what was written.
pub const ATTEMPT_METADATA_KEY: &str = "gitlawb-attempt";
/// The precondition an upload is fenced on.
///
/// Object storage is the only place a fence can hold. Dropping the future of an
/// in-flight PUT does not cancel the request the server is already processing,
/// so no amount of local locking stops an abandoned writer's bytes from landing
/// after a successor has published. A conditional PUT the store itself refuses
/// is what actually stops it.
#[derive(Clone, Debug)]
pub enum UploadPrecondition {
/// Publish only if the stored object is still the generation we observed.
IfMatch(String),
/// Publish only if nothing is stored under the key yet.
IfAbsent,
/// No fence. Last writer wins.
Unconditional,
}
/// What the store holds under a key right now.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct StoredGeneration {
/// The generation a later conditional upload can fence itself on.
pub etag: Option<String>,
/// The attempt that published it, when the object carries our metadata.
/// `None` for anything written by something that does not stamp attempts
/// (an operator upload, a pre-attempt-id archive).
pub attempt: Option<String>,
}
impl StoredGeneration {
/// Is what the store holds the work of `attempt`?
///
/// This is the whole reconciliation primitive. It is deliberately a
/// three-way question collapsed to a boolean only at the point of use: an
/// object with no attempt metadata answers `false`, because "somebody else's
/// bytes" and "bytes nobody stamped" license exactly the same caution.
pub fn belongs_to(&self, attempt: &PublishAttemptId) -> bool {
self.attempt.as_deref() == Some(attempt.as_str())
}
}
/// The receipt of a publish the store ACKNOWLEDGED.
///
/// Only produced on a response the client actually read, so holding one is proof
/// of publication in a way that "the upload future returned" is not.
#[derive(Clone, Debug)]
pub struct UploadReceipt {
pub attempt: PublishAttemptId,
pub etag: Option<String>,
}
/// How far ONE publish attempt got.
///
/// Observable through [`PublishStageCell`] from outside the future doing the
/// work, which is the property the whole design turns on: a handler future that
/// is DROPPED never returns an outcome, so the only way to classify a cancelled
/// attempt is to read the stage it had reached when it was abandoned.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum PublishStage {
/// Nothing has been attempted.
Idle,
/// No object-storage backend is configured, so there is no publication to
/// confirm and the local write IS the durable copy. Distinct from `Idle`,
/// which means an attempt was possible and had not started: a cancellation
/// under `NoBackend` loses nothing, while a cancellation under `Idle` means
/// a publish that could have happened never did.
NoBackend,
/// The archive is being built. The request has not been constructed, let
/// alone sent. A cancellation here is a DEFINITE "no publication was
/// attempted" — not an ambiguous in-flight PUT.
PreparingArchive,
/// The request is on the wire. Its fate is unknown until the store answers,
/// and a cancellation here does NOT stop the server processing it.
PutDispatched { attempt: PublishAttemptId },
/// The store acknowledged the write. This is the ONLY stage that licenses
/// replication, a 2xx, or treating the local tree as durable.
Published {
attempt: PublishAttemptId,
etag: Option<String>,
},
/// The store definitively did not publish this attempt (a refused
/// precondition, or a failure that proves the bytes never committed).
Refused,
/// The attempt may or may not have committed and has not been reconciled.
/// Destructive compensation is NEVER licensed from here.
Ambiguous { attempt: PublishAttemptId },
}
impl PublishStage {
/// Did this attempt reach a state where the store may hold its bytes?
///
/// True from dispatch onward. `false` is what makes deleting the attempt's
/// object or local tree safe.
pub fn may_have_published(&self) -> bool {
matches!(
self,
PublishStage::PutDispatched { .. }
| PublishStage::Published { .. }
| PublishStage::Ambiguous { .. }
)
}
/// The attempt whose fate is unresolved, when there is one. This is what a
/// reconciliation HEAD is compared against.
pub fn unresolved_attempt(&self) -> Option<&PublishAttemptId> {
match self {
PublishStage::PutDispatched { attempt } | PublishStage::Ambiguous { attempt } => {
Some(attempt)
}
_ => None,
}
}
}
/// A [`PublishStage`] an in-flight upload writes and an outside observer reads.
///
/// `std::sync::Mutex` rather than an async lock on purpose: every write is a
/// field assignment with no await inside, and the reader that matters most runs
/// inside a `Drop` impl, where an async lock cannot be awaited at all.
#[derive(Debug)]
pub struct PublishStageCell(Mutex<PublishStage>);
impl PublishStageCell {
pub fn new() -> Self {
Self::seeded(PublishStage::Idle)
}
pub fn seeded(stage: PublishStage) -> Self {
Self(Mutex::new(stage))
}
pub fn set(&self, stage: PublishStage) {
*self.0.lock().expect("publish stage mutex poisoned") = stage;
}
pub fn get(&self) -> PublishStage {
self.0.lock().expect("publish stage mutex poisoned").clone()
}
}
impl Default for PublishStageCell {
fn default() -> Self {
Self::new()
}
}
/// What the client knows about whether a request it could not complete ever
/// reached the wire.
///
/// The classifier that produces this is the ONLY backend-aware code in the
/// publish path; a pluggable store (#79) supplies its own and every caller below
/// keeps working off [`UploadError`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum DispatchKnowledge {
/// The request could not be built or sent. The store cannot hold these bytes.
NeverSent,
/// The request may have been sent, and may have committed.
MaybeSent,
}
/// Why an upload did not publish, split by what it leaves the caller ENTITLED TO
/// DO — not by which HTTP status came back.
///
/// The split exists because the previous shape ("precondition lost" vs "some
/// other error") made every caller treat a lost response as proof of failure.
/// Smithy timeout, dispatch and response errors all allow that the request was
/// sent and committed: a server can accept a complete conditional PUT and lose
/// or corrupt the response before the client observes success. Compensating that
/// as a definite failure deletes state that IS published.
#[derive(Debug, thiserror::Error)]
pub enum UploadError {
/// The store explicitly refused the fence (412, or 409 under create-only).
/// DEFINITE: this attempt did not publish, and a successor did.
#[error("upload precondition lost (HTTP {status})")]
PreconditionLost { status: u16 },
/// The attempt provably never committed: it failed before the request could
/// be dispatched, or the store answered a definite client-side refusal.
/// Destructive compensation is safe here and ONLY here.
#[error("upload did not reach the store: {0:#}")]
NotPublished(#[source] anyhow::Error),
/// The request may have been dispatched and may have committed; the client
/// never learned which. The attempt id is carried so the caller can ask the
/// store rather than guess.
#[error("upload outcome is unknowable (attempt {attempt}): {source:#}")]
Ambiguous {
attempt: PublishAttemptId,
#[source]
source: anyhow::Error,
},
}
impl UploadError {
/// May this caller destroy state that would be needed if the write HAD
/// landed — delete the object, drop the only local clone, invalidate the
/// cache?
///
/// Only a proven non-publication says yes. Every new variant must default to
/// `false`, which is why this is a match on the safe arms rather than a
/// negation of the unsafe one.
pub fn proves_not_published(&self) -> bool {
matches!(
self,
UploadError::PreconditionLost { .. } | UploadError::NotPublished(_)
)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn only_definite_outcomes_license_destructive_compensation() {
assert!(UploadError::PreconditionLost { status: 412 }.proves_not_published());
assert!(
UploadError::NotPublished(anyhow::anyhow!("no route to host")).proves_not_published()
);
assert!(
!UploadError::Ambiguous {
attempt: PublishAttemptId::new(),
source: anyhow::anyhow!("response body truncated"),
}
.proves_not_published(),
"a response-loss failure must never license deleting state the write may own"
);
}
#[test]
fn a_stage_may_have_published_only_from_dispatch_onward() {
let attempt = PublishAttemptId::new();
assert!(!PublishStage::Idle.may_have_published());
assert!(!PublishStage::NoBackend.may_have_published());
assert!(
!PublishStage::PreparingArchive.may_have_published(),
"compression has not constructed a request, let alone sent one"
);
assert!(!PublishStage::Refused.may_have_published());
assert!(PublishStage::PutDispatched {
attempt: attempt.clone()
}
.may_have_published());
assert!(PublishStage::Ambiguous {
attempt: attempt.clone()
}
.may_have_published());
assert!(PublishStage::Published {
attempt,
etag: None
}
.may_have_published());
}
#[test]
fn an_unstamped_object_never_belongs_to_an_attempt() {
let attempt = PublishAttemptId::new();
assert!(StoredGeneration {
etag: Some("\"e\"".into()),
attempt: Some(attempt.as_str().to_string()),
}
.belongs_to(&attempt));
assert!(
!StoredGeneration {
etag: Some("\"e\"".into()),
attempt: None,
}
.belongs_to(&attempt),
"bytes nobody stamped must not be claimed by this attempt"
);
assert!(!StoredGeneration {
etag: Some("\"e\"".into()),
attempt: Some("someone-else".into()),
}
.belongs_to(&attempt));
}
}
+493 -106
View File
@@ -19,7 +19,7 @@
//! objects the reachable withheld set never classified, so subtracting that set
//! is not enough — a dangling private blob would slip through (#99). The caller
//! signals a full scan via [`PinCandidateSet::full_scan`] and must then apply
//! the fail-closed blob filter (`visibility_pack::replicable_objects_fail_closed`)
//! the fail-closed blob-and-tree filter (`visibility_pack::replicable_objects_fail_closed`)
//! instead of the plain reachable-only subtraction.
//!
//! Because the pin candidate set needs only the OID *set* (never the per-path
@@ -29,9 +29,9 @@
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::process::Command;
use std::time::{Duration, Instant};
use anyhow::{bail, Context, Result};
use anyhow::Result;
/// Env var that forces the push path to always full-scan, bypassing the delta
/// optimization (KTD7 kill-switch). Reuses the already-tested fallback branch
@@ -64,11 +64,21 @@ pub enum PinCandidates {
/// Fail-closed: any condition where the introduced set cannot be safely
/// determined returns [`PinCandidates::FullScanRequired`] rather than a partial
/// set, so the caller full-scans instead of silently under-pinning.
pub fn resolve_push_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> PinCandidates {
// KTD7 kill-switch: force the (already-tested) full-scan fallback. The env
// read is split out from the pure logic so the resolver stays unit-testable
// without touching process-global state.
resolve_push_delta_inner(repo_path, new_tips, old_tips, force_full_scan())
pub fn resolve_push_delta(
repo_path: &Path,
new_tips: &[&str],
old_tips: &[&str],
git_bin: &str,
deadline: Instant,
) -> PinCandidates {
resolve_push_delta_inner(
repo_path,
new_tips,
old_tips,
force_full_scan(),
git_bin,
deadline,
)
}
/// Whether the force-full-scan kill-switch env var is set.
@@ -83,6 +93,8 @@ fn resolve_push_delta_inner(
new_tips: &[&str],
old_tips: &[&str],
force_full_scan: bool,
git_bin: &str,
deadline: Instant,
) -> PinCandidates {
if force_full_scan {
tracing::debug!("{FORCE_FULL_SCAN_ENV} set — forcing full scan");
@@ -102,7 +114,7 @@ fn resolve_push_delta_inner(
// commit). Bare `cat-file -t` returns `tag` for an annotated tag, and
// `for-each-ref %(*objecttype)` peels only one level — neither is correct.
for tip in new_tips {
match peeled_object_type(repo_path, tip) {
match peeled_object_type(repo_path, tip, git_bin, deadline) {
Some(t) if t == "commit" => {}
other => {
tracing::debug!(
@@ -115,7 +127,7 @@ fn resolve_push_delta_inner(
}
}
match rev_list_delta(repo_path, new_tips, old_tips) {
match rev_list_delta(repo_path, new_tips, old_tips, git_bin, deadline) {
Ok(oids) => PinCandidates::Delta(oids),
Err(e) => {
tracing::debug!(err = %e, "push-delta rev-list failed — forcing full scan");
@@ -126,41 +138,43 @@ fn resolve_push_delta_inner(
/// Return the fully-peeled object type of `sha` (e.g. `commit`, `tree`,
/// `blob`), or `None` if the object is missing/unpeelable or git errored.
fn peeled_object_type(repo_path: &Path, sha: &str) -> Option<String> {
let output = Command::new("git")
.args(["cat-file", "-t", &format!("{sha}^{{}}")])
.current_dir(repo_path)
.output()
.ok()?;
if !output.status.success() {
return None;
}
Some(String::from_utf8_lossy(&output.stdout).trim().to_string())
fn peeled_object_type(
repo_path: &Path,
sha: &str,
git_bin: &str,
deadline: Instant,
) -> Option<String> {
let peel = format!("{sha}^{{}}");
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
&["cat-file", "-t", &peel],
repo_path,
b"",
deadline,
)
.ok()?;
Some(String::from_utf8_lossy(&out).trim().to_string())
}
/// Run `git rev-list --objects --no-object-names <new> --not <old>` and return
/// the bare OID set. Decides on `status.success()` *before* parsing stdout, so
/// a walk that prints a valid prefix then errors mid-walk is discarded.
fn rev_list_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> Result<Vec<String>> {
fn rev_list_delta(
repo_path: &Path,
new_tips: &[&str],
old_tips: &[&str],
git_bin: &str,
deadline: Instant,
) -> Result<Vec<String>> {
let mut args: Vec<&str> = vec!["rev-list", "--objects", "--no-object-names"];
args.extend_from_slice(new_tips);
if !old_tips.is_empty() {
args.push("--not");
args.extend_from_slice(old_tips);
}
let output = Command::new("git")
.args(&args)
.current_dir(repo_path)
.output()
.context("failed to run git rev-list for push delta")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git rev-list failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
let out =
crate::git::visibility_pack::run_bounded_git(git_bin, &args, repo_path, b"", deadline)?;
let stdout = String::from_utf8_lossy(&out);
Ok(stdout
.lines()
.map(|l| l.trim().to_string())
@@ -175,23 +189,19 @@ fn rev_list_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> Res
/// reconciliation sweep relies on. It returns *all* objects (including
/// unreachable/dangling ones), which is what the sweep needs to catch
/// stragglers — do not swap it for a reachability walk.
pub fn list_all_objects(repo_path: &Path) -> Result<Vec<String>> {
let output = Command::new("git")
.args([
pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<Vec<String>> {
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
&[
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname)",
])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git cat-file failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
],
repo_path,
b"",
deadline,
)?;
let stdout = String::from_utf8_lossy(&out);
Ok(stdout
.lines()
.map(|l| l.trim().to_string())
@@ -201,25 +211,26 @@ pub fn list_all_objects(repo_path: &Path) -> Result<Vec<String>> {
/// Like [`list_all_objects`] but pairs each OID with its object type, via
/// `--batch-check='%(objectname) %(objecttype)'`. The pin path's fail-closed
/// filter needs to tell blobs (content, withholdable) from commits/trees
/// (structural, never withheld) without typing the candidate list itself.
pub fn list_all_objects_with_type(repo_path: &Path) -> Result<Vec<(String, String)>> {
let output = Command::new("git")
.args([
/// filter needs to tell blobs (content, withholdable) and trees (content,
/// withholdable since #135/#173) from commits/tags (structural, never withheld)
/// without typing the candidate list itself.
pub fn list_all_objects_with_type(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<Vec<(String, String)>> {
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
&[
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname) %(objecttype)",
])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git cat-file failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
],
repo_path,
b"",
deadline,
)?;
let stdout = String::from_utf8_lossy(&out);
Ok(stdout
.lines()
.filter_map(|l| {
@@ -234,20 +245,48 @@ pub fn list_all_objects_with_type(repo_path: &Path) -> Result<Vec<(String, Strin
/// Every blob OID in the repo, including unreachable/dangling ones. The
/// fail-closed pin filter drops any candidate blob absent from the reachable,
/// visibility-allowed set; a dangling private blob is in this set but not the
/// allowed set, so it never replicates (#99).
pub fn all_blob_oids(repo_path: &Path) -> Result<HashSet<String>> {
Ok(list_all_objects_with_type(repo_path)?
.into_iter()
.filter(|(_, ty)| ty == "blob")
.map(|(oid, _)| oid)
.collect())
/// allowed set, so it never replicates (#99). Production uses
/// [`all_blob_and_tree_oids`] (one typed pass); this wrapper stays for tests.
#[cfg(test)]
pub fn all_blob_oids(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<HashSet<String>> {
Ok(all_blob_and_tree_oids(repo_path, git_bin, deadline)?.0)
}
/// Every blob OID and every tree OID in the repo, including unreachable/dangling
/// ones, from one typed enumeration. The fail-closed pin filter drops any
/// candidate blob or tree absent from the matching reachable visibility-allowed
/// set; a dangling or withheld object is in this universe but not the allowed
/// set, so it never replicates (#99, #172).
pub fn all_blob_and_tree_oids(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<(HashSet<String>, HashSet<String>)> {
let mut blobs = HashSet::new();
let mut trees = HashSet::new();
for (oid, ty) in list_all_objects_with_type(repo_path, git_bin, deadline)? {
match ty.as_str() {
"blob" => {
blobs.insert(oid);
}
"tree" => {
trees.insert(oid);
}
_ => {}
}
}
Ok((blobs, trees))
}
/// The pin candidate OIDs for a push plus whether they came from a full-repo
/// scan. `full_scan` is true when the delta could not be used and the whole
/// object DB (including dangling objects) was enumerated — the caller must then
/// apply the fail-closed blob filter, because the reachable-only withheld set
/// cannot classify dangling blobs (#99).
/// apply the fail-closed blob-and-tree filter, because the reachable-only withheld set
/// cannot classify dangling blobs or trees (#99, #172).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PinCandidateSet {
pub candidates: Vec<String>,
@@ -268,22 +307,68 @@ pub struct PinCandidateSet {
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
/// rev-list, and the full-scan fallback — run under ONE permit held for the
/// whole blocking scan. The deletion-only fast path (no new tips) spawns no git
/// and never parks.
///
/// `force_full_scan` forces the full-scan fallback regardless of the tips — the
/// coalesced-drain `PendingWork::FullScan` marker (#174 F5). It composes with
/// the env kill-switch (either forces), and it disqualifies the empty-tips fast
/// path: a forced-scan call with no tips must still enumerate the repo, never
/// silently resolve to an empty delta (which would pin nothing and lose the
/// drained pushes' work — the F5 bug shape).
pub async fn resolve_candidates_for_push(
scan_sem: std::sync::Arc<tokio::sync::Semaphore>,
repo_path: PathBuf,
new_tips: Vec<String>,
old_tips: Vec<String>,
git_bin: String,
timeout: Duration,
force_full_scan: bool,
) -> PinCandidateSet {
let force = force_full_scan || self::force_full_scan();
// No-git fast path: a deletion-only push resolves to an empty delta without
// spawning any git child, so it must not park on the scan pool. A forced
// full scan (caller flag or kill-switch) disqualifies it — the scan spawns git.
if new_tips.is_empty() && !force {
tracing::info!(delta = 0usize, repo = %repo_path.display(), "pin candidate set from push delta");
return PinCandidateSet {
candidates: Vec::new(),
full_scan: false,
};
}
// Scan admission (#174 F4): DEFER, never shed — a dropped scan would
// silently under-pin this push. The permit moves into the blocking closure
// so a started scan always completes holding it. Residuals at
// `acquire_scan_permit`.
let permit =
crate::state::acquire_scan_permit(scan_sem, &repo_path, "pin-candidate scan").await;
tokio::task::spawn_blocking(move || {
let _permit = permit;
// ONE shared deadline for the whole scan, per jatmn ("the same deadline").
let deadline = Instant::now() + timeout;
let new_refs: Vec<&str> = new_tips.iter().map(String::as_str).collect();
let old_refs: Vec<&str> = old_tips.iter().map(String::as_str).collect();
match resolve_push_delta(&repo_path, &new_refs, &old_refs) {
// `force` already folds in the env kill-switch, so the forced arm skips the
// delta machinery outright; the unforced arm goes through the normal
// resolver (whose own env read is false here by construction).
let resolved = if force {
tracing::debug!("full scan forced (coalesced-drain marker or kill-switch)");
PinCandidates::FullScanRequired
} else {
resolve_push_delta(&repo_path, &new_refs, &old_refs, &git_bin, deadline)
};
match resolved {
PinCandidates::Delta(objs) => {
tracing::info!(delta = objs.len(), repo = %repo_path.display(), "pin candidate set from push delta");
PinCandidateSet { candidates: objs, full_scan: false }
}
PinCandidates::FullScanRequired => {
tracing::warn!(repo = %repo_path.display(), "pin delta unavailable (non-commit tip, git error, or force-full-scan) — full-scan fallback");
match list_all_objects(&repo_path) {
match list_all_objects(&repo_path, &git_bin, deadline) {
Ok(objs) => PinCandidateSet { candidates: objs, full_scan: true },
Err(e) => {
tracing::warn!(repo = %repo_path.display(), err = %e, "full-scan fallback failed; pinning nothing this push (reconciliation sweep backstops)");
@@ -304,8 +389,13 @@ pub async fn resolve_candidates_for_push(
mod tests {
use super::*;
use std::collections::HashSet;
use std::process::Command;
use tempfile::TempDir;
fn td() -> std::time::Instant {
std::time::Instant::now() + std::time::Duration::from_secs(600)
}
/// Minimal git helper for building test repos.
struct Repo {
_td: TempDir,
@@ -363,9 +453,10 @@ mod tests {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c2], &[&c1]))
.into_iter()
.collect();
let got: HashSet<String> =
delta(resolve_push_delta(&repo.path, &[&c2], &[&c1], "git", td()))
.into_iter()
.collect();
// The new blob b.txt and commit c2 are in the delta; the old blob a.txt
// and commit c1 are not.
let new_blob = repo.rev("HEAD:b.txt");
@@ -383,7 +474,7 @@ mod tests {
// genuinely new objects, never fewer.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c1], &[]))
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c1], &[], "git", td()))
.into_iter()
.collect();
assert!(got.contains(&c1));
@@ -399,9 +490,15 @@ mod tests {
// Rewrite history: reset to base, commit a different file.
repo.git(&["reset", "-q", "--hard", &base]);
let new_tip = repo.commit_file("c.txt", "three\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&new_tip], &[&old_tip]))
.into_iter()
.collect();
let got: HashSet<String> = delta(resolve_push_delta(
&repo.path,
&[&new_tip],
&[&old_tip],
"git",
td(),
))
.into_iter()
.collect();
assert!(got.contains(&new_tip), "new tip in delta");
assert!(got.contains(&repo.rev(&format!("{new_tip}:c.txt"))));
// No error; force-push computes new-minus-old cleanly.
@@ -413,7 +510,7 @@ mod tests {
repo.commit_file("a.txt", "one\n");
// All updates were deletions => new_tips empty after the caller strips zeros.
assert_eq!(
resolve_push_delta(&repo.path, &[], &[ZERO]),
resolve_push_delta(&repo.path, &[], &[ZERO], "git", td()),
PinCandidates::Delta(Vec::new())
);
}
@@ -424,7 +521,7 @@ mod tests {
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
assert_eq!(
resolve_push_delta(&repo.path, &[&blob], &[]),
resolve_push_delta(&repo.path, &[&blob], &[], "git", td()),
PinCandidates::FullScanRequired,
"a blob tip must force full scan (rev-list would exit 0 and walk it)"
);
@@ -436,7 +533,7 @@ mod tests {
repo.commit_file("a.txt", "one\n");
let tree = repo.rev("HEAD^{tree}");
assert_eq!(
resolve_push_delta(&repo.path, &[&tree], &[]),
resolve_push_delta(&repo.path, &[&tree], &[], "git", td()),
PinCandidates::FullScanRequired
);
}
@@ -449,7 +546,7 @@ mod tests {
repo.git(&["tag", "-a", "treetag", "-m", "x", &tree]);
let tag = repo.rev("treetag");
assert_eq!(
resolve_push_delta(&repo.path, &[&tag], &[]),
resolve_push_delta(&repo.path, &[&tag], &[], "git", td()),
PinCandidates::FullScanRequired,
"annotated tag peeling to a tree must force full scan"
);
@@ -465,7 +562,7 @@ mod tests {
repo.git(&["tag", "-a", "t2", "-m", "x", &t1]);
let t2 = repo.rev("t2");
assert_eq!(
resolve_push_delta(&repo.path, &[&t2], &[]),
resolve_push_delta(&repo.path, &[&t2], &[], "git", td()),
PinCandidates::FullScanRequired
);
}
@@ -480,7 +577,7 @@ mod tests {
let c1 = repo.commit_file("a.txt", "one\n");
repo.git(&["tag", "-a", "rel", "-m", "release", &c1]);
let tag = repo.rev("rel");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&tag], &[]))
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&tag], &[], "git", td()))
.into_iter()
.collect();
assert!(
@@ -501,7 +598,7 @@ mod tests {
let t1 = repo.rev("t1");
repo.git(&["tag", "-a", "t2", "-m", "x", &t1]);
let t2 = repo.rev("t2");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&t2], &[]))
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&t2], &[], "git", td()))
.into_iter()
.collect();
assert!(
@@ -516,9 +613,16 @@ mod tests {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let set =
resolve_candidates_for_push(repo.path.clone(), vec![c2.clone()], vec![c1.clone()])
.await;
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![c2.clone()],
vec![c1.clone()],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!set.full_scan, "happy-path delta is not a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
let new_blob = repo.rev("HEAD:b.txt");
@@ -536,20 +640,206 @@ mod tests {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
let all: HashSet<String> = list_all_objects(&repo.path).unwrap().into_iter().collect();
let set = resolve_candidates_for_push(repo.path.clone(), vec![blob], vec![]).await;
let all: HashSet<String> = list_all_objects(&repo.path, "git", td())
.unwrap()
.into_iter()
.collect();
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![blob],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(set.full_scan, "non-commit tip is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(got, all, "non-commit tip falls back to full repo scan");
}
/// F4 defer proof 2: `resolve_candidates_for_push`'s git stages (the per-tip
/// cat-file type probe + delta rev-list, and the full-scan fallback) run under a
/// scan-admission permit: with a zero-permit pool the call parks and spawns no
/// git; once a permit is available the SAME call runs (defer, not shed). On
/// ungated code the git runs regardless of the pool (RED).
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_defers_when_scan_pool_exhausted() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
// Fake git records ANY invocation; cat-file reports a commit tip so the
// delta stage proceeds, rev-list yields an empty delta.
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!(
"#!/bin/sh\necho ran >> \"{}\"\ncase \"$1\" in\n cat-file) echo commit ;;\n *) : ;;\nesac\nexit 0\n",
marker.display()
),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let tip = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string();
let sem: Arc<Semaphore> = Arc::new(Semaphore::new(0));
let blocked = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
sem.clone(),
dir.path().to_path_buf(),
vec![tip.clone()],
vec![],
git_bin.clone(),
Duration::from_secs(5),
false,
),
)
.await;
assert!(
blocked.is_err(),
"the pin-candidate scan must defer (park on admission) when the pool is exhausted"
);
assert!(
!marker.exists(),
"the scan's git must not spawn while its admission permit is unavailable (F4)"
);
// Release admission: the SAME scan now runs (defer, not shed).
sem.add_permits(1);
let set = resolve_candidates_for_push(
sem,
dir.path().to_path_buf(),
vec![tip],
vec![],
git_bin,
Duration::from_secs(5),
false,
)
.await;
assert!(
marker.exists(),
"once admission is available the deferred scan runs its git"
);
assert!(!set.full_scan, "commit tip + empty rev-list is a delta");
assert!(set.candidates.is_empty());
}
/// F4 fast-path negative arm: a deletion-only push (no new tips, kill-switch
/// off) computes its empty delta without spawning ANY git child, so it must
/// complete without acquiring from the scan pool — even at zero permits. The
/// per-tip cat-file probe means every push with a non-empty new tip DOES spawn
/// git, so this is the only genuinely git-free stage.
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_no_git_fast_path_skips_admission() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!("#!/bin/sh\necho ran >> \"{}\"\nexit 0\n", marker.display()),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let set = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
Arc::new(Semaphore::new(0)),
dir.path().to_path_buf(),
vec![],
vec!["deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string()],
fake.to_str().unwrap().to_string(),
Duration::from_secs(5),
false,
),
)
.await
.expect("the no-new-tips fast path must not park on the scan pool");
assert_eq!(
set,
PinCandidateSet {
candidates: Vec::new(),
full_scan: false
}
);
assert!(!marker.exists(), "the fast path must spawn no git at all");
}
/// #174 F5: the coalesced-drain FullScan marker is signalled via the explicit
/// `force_full_scan` flag, and a flagged call with NO tips must still enumerate
/// the whole repo (full_scan=true, non-empty candidates). The RED arm of the
/// encoding question: if the marker were encoded as a plain empty-tips call
/// (flag off — the fast path), the drain would resolve to an empty delta and
/// pin nothing, silently losing the coalesced pushes' work.
#[tokio::test]
async fn forced_full_scan_with_no_tips_enumerates_the_repo() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let all: HashSet<String> = list_all_objects(&repo.path, "git", td())
.unwrap()
.into_iter()
.collect();
assert!(!all.is_empty(), "fixture repo has objects");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
true,
)
.await;
assert!(set.full_scan, "the forced call is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(
got, all,
"a forced full scan with no tips enumerates the repo — never an empty delta"
);
// The discriminator: the SAME empty-tips call without the flag is the
// deletion-only fast path (empty delta, pin nothing). The two must differ,
// or the marker encoding has collapsed into the silent-loss shape.
let unforced = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!unforced.full_scan);
assert!(unforced.candidates.is_empty());
}
#[test]
fn missing_oid_tip_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let bogus = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
assert_eq!(
resolve_push_delta(&repo.path, &[bogus], &[]),
resolve_push_delta(&repo.path, &[bogus], &[], "git", td()),
PinCandidates::FullScanRequired,
"a missing/corrupt tip OID must force full scan"
);
@@ -564,7 +854,7 @@ mod tests {
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let old_tree = repo.rev(&format!("{c1}^{{tree}}"));
let result = resolve_push_delta(&repo.path, &[&c2], &[&old_tree]);
let result = resolve_push_delta(&repo.path, &[&c2], &[&old_tree], "git", td());
match result {
PinCandidates::FullScanRequired => {} // safe: caller full-scans
PinCandidates::Delta(objs) => {
@@ -583,10 +873,15 @@ mod tests {
// branch2 from base advances independently
repo.git(&["checkout", "-q", "-b", "branch2", &base]);
let b2 = repo.commit_file("c.txt", "three\n");
let got: HashSet<String> =
delta(resolve_push_delta(&repo.path, &[&b1, &b2], &[&base, &base]))
.into_iter()
.collect();
let got: HashSet<String> = delta(resolve_push_delta(
&repo.path,
&[&b1, &b2],
&[&base, &base],
"git",
td(),
))
.into_iter()
.collect();
assert!(got.contains(&b1), "branch1 new commit");
assert!(got.contains(&b2), "branch2 new commit");
}
@@ -595,7 +890,7 @@ mod tests {
fn empty_repo_no_tips_yields_empty_delta() {
let repo = Repo::new();
assert_eq!(
resolve_push_delta(&repo.path, &[], &[]),
resolve_push_delta(&repo.path, &[], &[], "git", td()),
PinCandidates::Delta(Vec::new())
);
}
@@ -609,12 +904,12 @@ mod tests {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
assert_eq!(
resolve_push_delta_inner(&repo.path, &[&c1], &[], true),
resolve_push_delta_inner(&repo.path, &[&c1], &[], true, "git", td()),
PinCandidates::FullScanRequired
);
// And with the flag off, the same push yields a Delta.
assert!(matches!(
resolve_push_delta_inner(&repo.path, &[&c1], &[], false),
resolve_push_delta_inner(&repo.path, &[&c1], &[], false, "git", td()),
PinCandidates::Delta(_)
));
}
@@ -624,7 +919,7 @@ mod tests {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
repo.commit_file("b.txt", "two\n");
let all = list_all_objects(&repo.path).unwrap();
let all = list_all_objects(&repo.path, "git", td()).unwrap();
// 2 commits + 2 trees + 2 blobs = 6 objects.
assert_eq!(all.len(), 6, "got: {all:?}");
}
@@ -642,7 +937,7 @@ mod tests {
std::fs::write(repo.path.join("orphan.bin"), b"dangling\n").unwrap();
let dangling = repo.git(&["hash-object", "-w", "orphan.bin"]);
let blobs = all_blob_oids(&repo.path).unwrap();
let blobs = all_blob_oids(&repo.path, "git", td()).unwrap();
assert!(blobs.contains(&reachable_blob), "reachable blob present");
assert!(
blobs.contains(&dangling),
@@ -652,7 +947,7 @@ mod tests {
assert!(!blobs.contains(&tree), "tree is not a blob");
// The typed lister tags each object; spot-check the dangling blob's type.
let typed = list_all_objects_with_type(&repo.path).unwrap();
let typed = list_all_objects_with_type(&repo.path, "git", td()).unwrap();
assert!(
typed
.iter()
@@ -660,4 +955,96 @@ mod tests {
"dangling object is typed as a blob"
);
}
#[cfg(unix)]
#[test]
fn list_all_objects_times_out_on_a_hung_git_instead_of_blocking() {
use std::os::unix::fs::PermissionsExt;
use std::time::{Duration, Instant};
let dir = tempfile::TempDir::new().unwrap();
// Fake git: hang on cat-file (bounded to 30s so a broken test can't wedge).
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
"#!/bin/sh\ncase \"$1\" in\n cat-file) i=0; while [ $i -lt 30 ]; do sleep 1; i=$((i+1)); done ;;\n *) : ;;\nesac\n",
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let (tx, rx) = std::sync::mpsc::channel();
let path = dir.path().to_path_buf();
std::thread::spawn(move || {
let _ = tx.send(list_all_objects(
&path,
&git_bin,
Instant::now() + Duration::from_millis(150),
));
});
let res = rx.recv_timeout(Duration::from_secs(10)).expect(
"list_all_objects must return within the watchdog budget, not block on a hung git",
);
assert!(
res.is_err(),
"a hung git must make list_all_objects error out, not hang"
);
}
/// #174 finding 2: the DELTA-path children — `peeled_object_type` (cat-file),
/// `rev_list_delta` (rev-list) — and `list_all_objects_with_type` (cat-file) are
/// the remaining candidate-discovery execs (the common push path). Each must
/// return within the watchdog budget on a hung git rather than block and pin the
/// write permit. Revert any one to a bare `Command::output()` and its arm blocks
/// past the recv budget (RED).
#[cfg(unix)]
#[test]
fn delta_path_exec_fns_time_out_on_a_hung_git() {
use std::os::unix::fs::PermissionsExt;
use std::time::{Duration, Instant};
let dir = tempfile::TempDir::new().unwrap();
// Fake git hangs on BOTH cat-file and rev-list (bounded 30s so a broken
// test can't leak a permanent orphan or wedge the suite).
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
"#!/bin/sh\ncase \"$1\" in\n cat-file|rev-list) i=0; while [ $i -lt 30 ]; do sleep 1; i=$((i+1)); done ;;\n *) : ;;\nesac\n",
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let path = dir.path().to_path_buf();
// Run `f` on a thread and require it to RETURN (not block) within 10s.
fn returns_within<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static) -> T {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let _ = tx.send(f());
});
rx.recv_timeout(Duration::from_secs(10)).expect(
"a delta-path exec fn must return within the watchdog budget, not block on a hung git",
)
}
let dl = || Instant::now() + Duration::from_millis(150);
let sha = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
let (p, g, d) = (path.clone(), git_bin.clone(), dl());
assert!(
returns_within(move || peeled_object_type(&p, sha, &g, d)).is_none(),
"peeled_object_type must time out to None on a hung cat-file"
);
let (p, g, d) = (path.clone(), git_bin.clone(), dl());
assert!(
returns_within(move || rev_list_delta(&p, &[sha], &[], &g, d)).is_err(),
"rev_list_delta must error out on a hung rev-list"
);
let (p, g, d) = (path.clone(), git_bin.clone(), dl());
assert!(
returns_within(move || list_all_objects_with_type(&p, &g, d)).is_err(),
"list_all_objects_with_type must error out on a hung cat-file"
);
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+775 -56
View File
@@ -5,17 +5,121 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::atomic::AtomicBool;
use std::sync::{Arc, Mutex, OnceLock};
use anyhow::{Context, Result};
use aws_sdk_s3::Client as S3Client;
use tracing::{debug, info};
use tracing::{debug, info, warn};
// The publication vocabulary lives in `git::publish`, which carries no
// object-storage types, and is re-exported here so existing `git::tigris::…`
// imports keep resolving. #79 deletes this file; `git::publish` is what survives
// the swap, and the only backend-aware code below is `classify_dispatch`.
pub use super::publish::{
DispatchKnowledge, PublishAttemptId, PublishStage, PublishStageCell, StoredGeneration,
UploadError, UploadPrecondition, UploadReceipt, ATTEMPT_METADATA_KEY,
};
/// What happened to a conditional, attempt-guarded delete.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AttemptDelete {
/// The object was this attempt's and is gone.
Deleted,
/// Nothing is stored under the key.
Absent,
/// Something is stored, and it is NOT this attempt's work. Left alone.
NotOurs,
}
/// THE ONLY BACKEND-AWARE CLASSIFIER in the publish path.
///
/// It answers one question: does this failure PROVE the request never committed?
/// Anything short of proof is [`DispatchKnowledge::MaybeSent`], because the
/// caller's next move on a definite failure is destructive (delete the object,
/// drop the only local clone, invalidate the cache) and being wrong there
/// destroys published state.
///
/// - An HTTP status means the server answered. A 4xx is a refusal it took
/// BEFORE storing anything, so it proves non-publication. A 5xx does not: the
/// store can commit and then fail to report it.
/// - No status at all means no response was read. Only a construction failure
/// proves the request never left; Smithy's timeout, dispatch and response
/// errors all allow that the bytes arrived and committed while the answer was
/// lost, corrupted, or never parsed.
/// - `SdkError` is `#[non_exhaustive]`, so the fallback arm must be the CAUTIOUS
/// one. A future variant defaulting to "definitely failed" would silently
/// re-open exactly the class this closes.
///
/// #79 replaces `tigris.rs` with a `BlobStore` layer; this function is the piece
/// each backend reimplements, and nothing above it changes.
fn classify_put_failure<E, R>(
err: &aws_sdk_s3::error::SdkError<E, R>,
status: Option<u16>,
) -> DispatchKnowledge {
if let Some(status) = status {
return if (400..500).contains(&status) {
DispatchKnowledge::NeverSent
} else {
DispatchKnowledge::MaybeSent
};
}
match err {
aws_sdk_s3::error::SdkError::ConstructionFailure(_) => DispatchKnowledge::NeverSent,
_ => DispatchKnowledge::MaybeSent,
}
}
/// Wrapper around the S3 client with the configured bucket.
#[derive(Clone)]
pub struct TigrisClient {
s3: S3Client,
bucket: String,
/// Test-only seam: when set, the blocking compression inside `upload_tracked`
/// parks on this barrier. That is the ONLY window in which a publish attempt
/// can be cancelled with `PublishStage::PreparingArchive` still holding, and
/// it is unreachable from outside without a seam because compression of a
/// test-sized repo finishes in microseconds.
#[cfg(test)]
compress_gate: Option<Arc<BlockingGate>>,
}
/// A gate a BLOCKING thread parks on until a test opens it.
///
/// A condvar rather than a held `MutexGuard`: the test's assertions run while the
/// gate is shut, and holding a `std::sync::MutexGuard` across those awaits is
/// both a lint violation and a real hazard. Here the test owns no guard at all —
/// it flips a flag and notifies.
#[cfg(test)]
#[derive(Default)]
pub struct BlockingGate {
open: Mutex<bool>,
opened: std::sync::Condvar,
}
#[cfg(test)]
impl BlockingGate {
/// A gate that starts SHUT.
pub fn shut() -> Self {
Self::default()
}
fn wait(&self) {
let mut open = self.open.lock().expect("compression gate poisoned");
while !*open {
open = self
.opened
.wait(open)
.expect("compression gate poisoned while waiting");
}
}
/// Let every parked compression through. Call this at teardown so the
/// blocking thread is not stranded for the life of the process.
pub fn open(&self) {
*self.open.lock().expect("compression gate poisoned") = true;
self.opened.notify_all();
}
}
impl TigrisClient {
@@ -28,9 +132,48 @@ impl TigrisClient {
Ok(Self {
s3,
bucket: bucket.to_string(),
#[cfg(test)]
compress_gate: None,
})
}
/// Build a client pointed at an arbitrary endpoint, for tests.
///
/// The production constructor reads the endpoint and credentials from the
/// environment, which a test cannot steer without mutating process-global
/// state. This takes both explicitly so a test can aim the client at a
/// closed port and get a prompt transport error out of `exists()`.
///
/// `RetryConfig::disabled()` is load-bearing, not tidiness: the SDK's default
/// policy retries a connection refusal with backoff, which turns each failing
/// call into seconds of waiting.
#[cfg(test)]
pub fn for_testing_with_endpoint(bucket: &str, endpoint: &str) -> Self {
use aws_sdk_s3::config::{retry::RetryConfig, Credentials, Region};
let config = aws_sdk_s3::config::Config::builder()
.endpoint_url(endpoint)
.credentials_provider(Credentials::new("test", "test", None, None, "test"))
.region(Region::new("auto"))
.retry_config(RetryConfig::disabled())
.behavior_version_latest()
.build();
Self {
s3: S3Client::from_conf(config),
bucket: bucket.to_string(),
compress_gate: None,
}
}
/// Test-only: park this client's blocking compression on `gate` until the
/// holder releases it, so a cancellation can be aimed at
/// [`PublishStage::PreparingArchive`] rather than at the PUT.
#[cfg(test)]
pub fn with_compress_gate(mut self, gate: Arc<BlockingGate>) -> Self {
self.compress_gate = Some(gate);
self
}
/// S3 key for a given repo: `repos/v1/{owner_slug}/{repo_name}.tar.zst`
fn repo_key(owner_slug: &str, repo_name: &str) -> String {
format!("repos/v1/{owner_slug}/{repo_name}.tar.zst")
@@ -58,34 +201,325 @@ impl TigrisClient {
}
}
/// Upload a local bare repo directory to Tigris as a tar.zst archive.
pub async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), "uploading repo to tigris");
/// Read the ETag of a repo archive, or `None` when nothing is stored under
/// the key. The ETag identifies the generation a later conditional upload
/// can fence itself on.
///
/// Separate from `exists` rather than folded into it: `exists` has callers
/// that only want the boolean, and widening its return type would churn
/// every one of them for no benefit.
pub async fn head_etag(&self, owner_slug: &str, repo_name: &str) -> Result<Option<String>> {
Ok(self
.head_generation(owner_slug, repo_name)
.await?
.map(|g| g.etag)
.unwrap_or(None))
}
// Create tar.zst in memory
let archive_bytes = tokio::task::spawn_blocking({
let local_path = local_path.to_path_buf();
move || compress_repo(&local_path)
})
/// The full stored generation: the ETag AND the attempt that published it.
///
/// The attempt half is what makes a lost response recoverable. An ETag alone
/// answers "has the generation moved", which every writer sees the same way;
/// the attempt id answers "are the published bytes MINE", which is the
/// question a client whose PUT response vanished actually needs answered.
pub async fn head_generation(
&self,
owner_slug: &str,
repo_name: &str,
) -> Result<Option<StoredGeneration>> {
let key = Self::repo_key(owner_slug, repo_name);
match self
.s3
.head_object()
.bucket(&self.bucket)
.key(&key)
.send()
.await
{
Ok(out) => {
let etag = out
.e_tag()
.context(format!("tigris HEAD {key}: hit carried no ETag"))?
.to_string();
let attempt = out
.metadata()
.and_then(|m| m.get(ATTEMPT_METADATA_KEY))
.cloned();
Ok(Some(StoredGeneration {
etag: Some(etag),
attempt,
}))
}
Err(e) => {
if e.as_service_error().is_some_and(|e| e.is_not_found()) {
Ok(None)
} else {
Err(anyhow::anyhow!("tigris HEAD {key}: {e}"))
}
}
}
}
/// Did `attempt`'s bytes land? The reconciliation an ambiguous dispatch owes
/// before anything downstream may treat it as confirmation.
///
/// `Ok(false)` is deliberately NOT "the PUT failed": an abandoned request may
/// still be in flight, so a negative answer licenses refusing and retrying,
/// never deleting. Only `Ok(true)` upgrades an unresolved attempt to
/// published.
pub async fn attempt_landed(
&self,
owner_slug: &str,
repo_name: &str,
attempt: &PublishAttemptId,
) -> Result<bool> {
Ok(self
.head_generation(owner_slug, repo_name)
.await?
.is_some_and(|g| g.belongs_to(attempt)))
}
/// Upload a local bare repo directory to Tigris as a tar.zst archive,
/// fenced by `precondition`, under a freshly minted attempt identity.
pub async fn upload(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
precondition: UploadPrecondition,
) -> std::result::Result<UploadReceipt, UploadError> {
self.upload_tracked(
owner_slug,
repo_name,
local_path,
precondition,
PublishAttemptId::new(),
None,
)
.await
.context("tar task panicked")?
.context("compressing repo")?;
}
/// The full form: a caller-chosen attempt identity, and a stage cell the
/// upload reports its progress into.
///
/// The stage cell is the answer to "a dropped future never returns an
/// outcome". Compression runs inside `spawn_blocking` and the conditional PUT
/// is not even constructed until it finishes, so a handler cancelled during
/// compression definitely never attempted publication — but nothing could
/// observe that, because the only report was the return value of a future
/// that no longer exists. Marking [`PublishStage::PreparingArchive`] before
/// the blocking call and [`PublishStage::PutDispatched`] immediately before
/// `send()` makes that boundary readable from outside.
pub async fn upload_tracked(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
precondition: UploadPrecondition,
attempt: PublishAttemptId,
stage: Option<&PublishStageCell>,
) -> std::result::Result<UploadReceipt, UploadError> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), attempt = %attempt, "uploading repo to tigris");
if let Some(stage) = stage {
stage.set(PublishStage::PreparingArchive);
}
// Create tar.zst in memory. Nothing has been sent at this point and
// nothing can be: the request below is not constructed until this
// returns. A failure or a cancellation here is a definite
// non-publication.
let compressed = {
let local_path = local_path.to_path_buf();
#[cfg(test)]
let gate = self.compress_gate.clone();
tokio::task::spawn_blocking(move || {
// Test-only seam: park INSIDE the blocking compression, which is
// the window a handler can be cancelled in while
// `PublishStage::PreparingArchive` still holds.
#[cfg(test)]
if let Some(gate) = gate {
// Blocks until the test opens the gate.
gate.wait();
}
compress_repo(&local_path)
})
.await
};
let archive_bytes = match compressed {
Ok(Ok(bytes)) => bytes,
Ok(Err(e)) => {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
return Err(UploadError::NotPublished(e.context("compressing repo")));
}
Err(e) => {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
return Err(UploadError::NotPublished(
anyhow::Error::new(e).context("tar task panicked"),
));
}
};
let body = aws_sdk_s3::primitives::ByteStream::from(archive_bytes);
self.s3
let mut req = self
.s3
.put_object()
.bucket(&self.bucket)
.key(&key)
// The attempt identity travels WITH the bytes. That is what makes a
// lost response recoverable: the client can HEAD the key afterwards
// and ask whether the published object is its own work, which is
// decidable, instead of asking whether its request succeeded, which
// is not.
.metadata(ATTEMPT_METADATA_KEY, attempt.as_str())
.body(body)
.content_type("application/zstd")
.send()
.await
.context(format!("tigris PUT {key}"))?;
.content_type("application/zstd");
match &precondition {
UploadPrecondition::IfMatch(etag) => req = req.if_match(etag),
UploadPrecondition::IfAbsent => req = req.if_none_match("*"),
UploadPrecondition::Unconditional => {}
}
info!(key = %key, "uploaded repo to tigris");
Ok(())
// From HERE the bytes may reach the store. Everything downstream that
// could destroy state has to treat this stage as "maybe published".
if let Some(stage) = stage {
stage.set(PublishStage::PutDispatched {
attempt: attempt.clone(),
});
}
let sent = req.send().await;
let out = match sent {
Ok(out) => out,
Err(e) => {
// `PutObjectError` models no PreconditionFailed variant (its arms are
// EncryptionTypeMismatch, InvalidRequest, InvalidWriteOffset,
// TooManyParts, Unhandled), so a refused precondition arrives as
// `Unhandled` and matching the enum would classify it as a generic
// failure. The raw HTTP status off the response is the only place the
// answer actually lives.
//
// Read it via `raw_response()`, not a `ServiceError`-only match: the
// SDK exposes the raw response for BOTH `ServiceError` and
// `ResponseError`, and a refused conditional PUT whose error body the
// SDK cannot parse (malformed XML, premature close) surfaces as
// `ResponseError`. Matching only `ServiceError` would classify that
// unparsable 409/412 as a generic failure, and `RepoWriteGuard::release`
// would log-and-succeed instead of taking the supersede retry,
// acknowledging a write that was definitively not published.
let status = e.raw_response().map(|raw| raw.status().as_u16());
// 412 is always a lost precondition. 409 is one only when we asked
// for create-only, which is how S3-compatible stores report "the key
// already exists". Everything else, 404 included, is a real failure:
// archive keys are never deleted by the write path, so a 404 here
// means something permanent like a missing bucket or a misrouted
// endpoint, and reporting that as a lost precondition would tell a
// caller to expect a successor that does not exist.
let lost = match status {
Some(412) => true,
Some(409) => matches!(precondition, UploadPrecondition::IfAbsent),
_ => false,
};
if lost {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
return Err(UploadError::PreconditionLost {
status: status.expect("a lost precondition came from a status"),
});
}
let knowledge = classify_put_failure(&e, status);
let ctx = anyhow::Error::new(e).context(format!("tigris PUT {key}"));
return Err(match knowledge {
DispatchKnowledge::NeverSent => {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
UploadError::NotPublished(ctx)
}
DispatchKnowledge::MaybeSent => {
warn!(
key = %key,
attempt = %attempt,
status = ?status,
"tigris PUT failed WITHOUT proving it did not commit — the outcome \
is ambiguous and must be reconciled, not compensated"
);
if let Some(stage) = stage {
stage.set(PublishStage::Ambiguous {
attempt: attempt.clone(),
});
}
UploadError::Ambiguous {
attempt: attempt.clone(),
source: ctx,
}
}
});
}
};
let etag = out.e_tag().map(str::to_string);
if let Some(stage) = stage {
stage.set(PublishStage::Published {
attempt: attempt.clone(),
etag: etag.clone(),
});
}
info!(key = %key, attempt = %attempt, "uploaded repo to tigris");
Ok(UploadReceipt { attempt, etag })
}
/// Delete a repo archive ONLY while it is still `attempt`'s work.
///
/// The unconditional delete this replaces used the logical owner/name as its
/// cleanup authority, so a failed attempt compensating late could erase the
/// object a SUCCESSOR had already published under the same name and already
/// returned 201 for. A second name lookup before the delete only narrows that
/// window; reading the attempt id off the object and fencing the delete on
/// the generation it came from closes it.
pub async fn delete_if_attempt_matches(
&self,
owner_slug: &str,
repo_name: &str,
attempt: &PublishAttemptId,
) -> Result<AttemptDelete> {
let Some(generation) = self.head_generation(owner_slug, repo_name).await? else {
return Ok(AttemptDelete::Absent);
};
if !generation.belongs_to(attempt) {
return Ok(AttemptDelete::NotOurs);
}
let key = Self::repo_key(owner_slug, repo_name);
let mut req = self.s3.delete_object().bucket(&self.bucket).key(&key);
// The If-Match guard is what makes this atomic rather than merely
// narrowed: between the HEAD above and this call a successor can publish,
// and the store refusing on the moved generation is the only thing that
// stops the delete landing on their object.
if let Some(etag) = generation.etag.as_deref() {
req = req.if_match(etag);
}
match req.send().await {
Ok(_) => Ok(AttemptDelete::Deleted),
Err(e) => {
// A refused conditional delete means the generation moved: the
// object is no longer ours, which is a successful outcome for a
// guard whose whole job is not to touch somebody else's bytes.
if e.raw_response()
.map(|raw| raw.status().as_u16())
.is_some_and(|s| s == 412 || s == 409)
{
return Ok(AttemptDelete::NotOurs);
}
Err(anyhow::anyhow!("tigris conditional DELETE {key}: {e}"))
}
}
}
/// Download a repo archive from Tigris and extract to local disk.
@@ -93,10 +527,32 @@ impl TigrisClient {
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
local_path: &super::repo_store::ValidatedRepoDiskPath,
swap_authority: Option<Arc<AtomicBool>>,
) -> Result<()> {
self.download_to(owner_slug, repo_name, local_path, true, swap_authority)
.await
.map(|_| ())
}
/// Download a repo archive from Tigris and extract it, returning the
/// directory that was populated.
///
/// `publish` controls whether the extract is swapped into `target` in place
/// (the live-path mutation used by writes; returns `target`) or unpacked
/// into a fresh temp directory under `target`'s parent (a non-mutating
/// snapshot read; returns the temp dir, which the caller owns and cleans
/// up). The snapshot form never touches the live repo path.
pub async fn download_to(
&self,
owner_slug: &str,
repo_name: &str,
target: &super::repo_store::ValidatedRepoDiskPath,
publish: bool,
swap_authority: Option<Arc<AtomicBool>>,
) -> Result<DownloadExtract> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), "downloading repo from tigris");
debug!(key = %key, path = %target.as_path().display(), "downloading repo from tigris");
let resp = self
.s3
@@ -114,31 +570,59 @@ impl TigrisClient {
.context("reading tigris response body")?
.into_bytes();
// Extract tar.zst to local path
tokio::task::spawn_blocking({
let local_path = local_path.to_path_buf();
move || decompress_repo(&data, &local_path)
// The snapshot temp dir is decided HERE, in the async layer, before the
// extraction runs. Cleanup ownership moves into the blocking task so a
// cancelled async future cannot drop it while extraction is still running.
let snapshot_tmp = if publish {
None
} else {
let parent = target.parent().context("snapshot path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = target
.file_name()
.context("snapshot path has no file name")?
.to_string_lossy();
Some(parent.join(format!(
".{file_name}.tmp-snapshot.{}",
uuid::Uuid::new_v4()
)))
};
// Extract tar.zst to a directory.
let extracted = tokio::task::spawn_blocking({
let target = target.clone();
let snapshot_tmp = snapshot_tmp.clone();
move || -> Result<DownloadExtract> {
let result = (|| -> Result<DownloadExtract> {
if publish {
decompress_repo(&data, &target, swap_authority.as_ref())?;
return Ok(DownloadExtract::Published(()));
}
// Non-mutating snapshot: unpack into the temp dir decided above.
// The live repo path is never touched.
let tmp_dir = snapshot_tmp.expect("snapshot path was decided above");
std::fs::create_dir_all(&tmp_dir).context("creating temp extract dir")?;
let unpack = (|| -> Result<()> {
let decoder = zstd::stream::Decoder::new(&data[..])?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(&tmp_dir).context("unpacking tar.zst")?;
Ok(())
})();
if let Err(e) = unpack {
let _ = std::fs::remove_dir_all(&tmp_dir);
return Err(e);
}
Ok(DownloadExtract::Snapshot(TempSnapshotDir { path: tmp_dir }))
})();
result
}
})
.await
.context("extract task panicked")?
.context("extracting repo")?;
info!(key = %key, path = %local_path.display(), "downloaded repo from tigris");
Ok(())
}
/// Delete a repo archive from Tigris.
#[allow(dead_code)]
pub async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
self.s3
.delete_object()
.bucket(&self.bucket)
.key(&key)
.send()
.await
.context(format!("tigris DELETE {key}"))?;
Ok(())
info!(key = %key, path = %target.as_path().display(), "downloaded repo from tigris");
Ok(extracted)
}
}
@@ -161,7 +645,7 @@ fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
/// `decompress_repo`. Concurrent extractions unpack into isolated temp dirs in
/// parallel, but the final `remove_dir_all` + `rename` must not interleave for
/// the same `local_path`, or they race to a nondeterministic overwrite/failure.
fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
// KNOWN LIMITATION: this map is never evicted — one (PathBuf, Arc<Mutex>)
// entry accrues per distinct repo path for the process lifetime. Bounded by
// the number of repos a node hosts, so it's negligible for normal use, but
@@ -174,6 +658,35 @@ fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
.clone()
}
/// Owns a snapshot temp dir from extraction through handoff to [`RepoSnapshot`].
/// Dropped when the `spawn_blocking` join result is abandoned, so cancellation
/// before the outer future resumes still removes the directory.
pub(crate) struct TempSnapshotDir {
path: PathBuf,
}
impl TempSnapshotDir {
pub(crate) fn into_repo_snapshot(self) -> super::repo_store::RepoSnapshot {
let path = self.path.clone();
std::mem::forget(self);
super::repo_store::RepoSnapshot::from_owned_path(path)
}
}
impl Drop for TempSnapshotDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.path);
}
}
/// What [`TigrisClient::download_to`] produced.
pub enum DownloadExtract {
/// Published into the validated live repo path.
Published(()),
/// Unpacked into a throwaway temp dir that cleans up on drop until adopted.
Snapshot(TempSnapshotDir),
}
/// Decompress a tar.zst byte vector into a local directory.
///
/// Extraction is atomic with respect to `local_path`: the archive is unpacked
@@ -181,11 +694,16 @@ fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
/// fully succeeds. A corrupt or truncated archive therefore can never clobber a
/// good existing copy at `local_path` — on failure we discard the temp dir and
/// leave `local_path` exactly as it was.
fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
let parent = local_path.parent().context("repo path has no parent")?;
fn decompress_repo(
data: &[u8],
local_path: &super::repo_store::ValidatedRepoDiskPath,
swap_authority: Option<&Arc<AtomicBool>>,
) -> Result<()> {
let live = local_path.as_path();
let parent = live.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = local_path
let file_name = live
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
@@ -210,17 +728,218 @@ fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
return Err(e);
}
// Swap the freshly-extracted repo into place. rename within the same parent
// is effectively atomic, but most platforms refuse to rename onto a
// non-empty dir, so remove the old copy first. Serialize this per repo path:
// concurrent extractions unpack into isolated temp dirs, but their swaps
// must not interleave or they race to a nondeterministic overwrite/failure.
let lock = publish_lock(local_path);
let _publish = lock.lock().expect("publish lock poisoned");
if local_path.exists() {
std::fs::remove_dir_all(local_path).context("removing stale repo dir")?;
}
std::fs::rename(&tmp_dir, local_path).context("swapping extracted repo into place")?;
// Swap through the validated-path helper so CodeQL sees the barrier before the
// remove/rename sink (`rust/path-injection`).
super::repo_store::swap_extracted_into_validated_repo(local_path, &tmp_dir, swap_authority)?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use aws_sdk_s3::primitives::ByteStream;
use futures::FutureExt;
/// The envs the probe needs, all of them, or it does not run.
///
/// `AWS_ENDPOINT_URL_S3` is included on purpose: without it the SDK resolves
/// to real AWS S3, and a probe that passed there would say nothing about
/// Tigris.
fn probe_env() -> Option<String> {
if std::env::var("GITLAWB_TIGRIS_PROBE").ok().as_deref() != Some("1") {
return None;
}
for name in [
"AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY",
"AWS_ENDPOINT_URL_S3",
] {
if std::env::var(name).is_err() {
eprintln!("tigris conditional-write probe: {name} is unset, skipping");
return None;
}
}
match std::env::var("GITLAWB_TIGRIS_BUCKET") {
Ok(b) if !b.is_empty() => Some(b),
_ => {
eprintln!(
"tigris conditional-write probe: GITLAWB_TIGRIS_BUCKET is unset, skipping"
);
None
}
}
}
/// One conditional PUT, reported as the status that REFUSED it, or `None`
/// when the store accepted the write.
///
/// Accepted is the interesting answer here, not an error: it means the
/// endpoint ignored the header we fenced on.
async fn conditional_put(
s3: &S3Client,
bucket: &str,
key: &str,
body: &'static [u8],
if_match: Option<&str>,
if_none_match: Option<&str>,
) -> Result<Option<u16>, String> {
let mut req = s3
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(body));
if let Some(v) = if_match {
req = req.if_match(v);
}
if let Some(v) = if_none_match {
req = req.if_none_match(v);
}
match req.send().await {
Ok(_) => Ok(None),
Err(e) => match e.raw_response() {
// Same raw-response rule as `upload`: a refused conditional PUT
// whose body the SDK cannot parse surfaces as `ResponseError`, and
// the status has to come off the raw response for both variants.
// Without it, an unparsable 409/412 would report "no HTTP
// response" here and skip the supersede retry.
Some(raw) => Ok(Some(raw.status().as_u16())),
None => Err(format!("conditional PUT {key}: no HTTP response: {e}")),
},
}
}
/// The probe body, written to RETURN its failures rather than panic on
/// them, so the caller's cleanup is reached on every arm.
async fn conditional_write_probe(s3: &S3Client, bucket: &str, key: &str) -> Result<(), String> {
// 1. A plain PUT under a throwaway key.
let seeded = s3
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(b"probe-one"))
.send()
.await
.map_err(|e| format!("seeding PUT {key}: {e}"))?;
// 2. Its ETag, which is the generation the next arm fences against.
let etag = seeded
.e_tag()
.ok_or_else(|| format!("seeding PUT {key} returned no ETag"))?
.to_string();
// 3. A deliberately wrong If-Match. A store honoring it answers 412.
let wrong = format!("\"{}\"", "0".repeat(32));
if etag.trim_matches('"') == wrong.trim_matches('"') {
return Err(format!(
"the seeded ETag {etag} collides with the deliberately wrong one, \
so this arm would prove nothing"
));
}
match conditional_put(s3, bucket, key, b"probe-two", Some(&wrong), None).await? {
Some(412) => {}
Some(status) => {
return Err(format!(
"a stale If-Match must be refused with 412, the endpoint answered {status}"
))
}
None => {
return Err(
"a stale If-Match was ACCEPTED: this endpoint does not honor If-Match, so \
the release fence cannot hold here"
.to_string(),
)
}
}
// 4. If-None-Match `*` over the object that now exists. This arm matters
// MORE than the one above. An ignored If-Match eventually surfaces as
// odd behavior, because a stale writer overwrites and someone notices
// the lost tree. An ignored If-None-Match just returns 200, so a publish
// that should have been fenced lands with no error anywhere: the silent
// no-op the bucket-type caveat on this test describes.
match conditional_put(s3, bucket, key, b"probe-three", None, Some("*")).await? {
// Either status is a pass, and the asymmetry with the If-Match arm
// above mirrors `upload`'s classifier exactly: 412 is always a lost
// precondition, and 409 is one too when we asked for create-only.
// AWS documents 409 for a create-only conflict racing a delete, so a
// store answering it is enforcing the precondition and we already
// handle it. Pinning 412 alone here would fail the probe against a
// backend that is behaving correctly, which sends whoever runs it
// chasing a fault that is not there.
Some(412) | Some(409) => {}
Some(status) => {
return Err(format!(
"create-only over an existing object must be refused with 412 or 409, \
the endpoint answered {status}"
))
}
None => {
return Err(
"If-None-Match * was ACCEPTED over an existing object: this endpoint does \
not honor create-only, so a fenced publish lands silently"
.to_string(),
)
}
}
Ok(())
}
/// Probe the REAL Tigris endpoint for the conditional-write semantics the
/// release fence depends on.
///
/// UNTIL THIS IS RUN AGAINST REAL CREDENTIALS, the fence is verified against
/// vendor documentation and an in-process mock, not against the backend it
/// runs on. The mock implements the semantics we believe Tigris has; it
/// cannot tell us whether Tigris actually has them.
///
/// The bucket matters, not just the endpoint. Tigris documents conditional
/// operations as supported on Single-region and Multi-region buckets only.
/// Global and Dual-region buckets are eventually consistent, and a
/// conditional PUT evaluated against a stale replica would make the fence a
/// silent no-op rather than an error. So point `GITLAWB_TIGRIS_BUCKET` at a
/// throwaway bucket of the SAME type production uses.
///
/// Ignored by default and additionally gated on `GITLAWB_TIGRIS_PROBE=1`,
/// because it writes to a real bucket and costs real requests. Run with:
/// `GITLAWB_TIGRIS_PROBE=1 cargo test -p gitlawb-node --bin gitlawb-node
/// tigris_honors_conditional_writes -- --ignored --nocapture`
#[tokio::test]
#[ignore = "writes to a real Tigris bucket; needs GITLAWB_TIGRIS_PROBE=1 plus credentials"]
async fn tigris_honors_conditional_writes() {
let Some(bucket) = probe_env() else {
eprintln!(
"tigris conditional-write probe: skipped. Set GITLAWB_TIGRIS_PROBE=1, \
GITLAWB_TIGRIS_BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and \
AWS_ENDPOINT_URL_S3 to run it."
);
return;
};
let config = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await;
let s3 = S3Client::new(&config);
// A fresh key per run, so a probe that somehow orphaned an object on an
// earlier run cannot change what this one observes.
let key = format!("probe/conditional-write-{}.bin", uuid::Uuid::new_v4());
// CLEANUP MUST RUN ON EVERY ARM, and a failing assertion is precisely
// the case this probe exists to catch, so the delete cannot sit after
// the checks. The body returns its failures rather than panicking, and
// `catch_unwind` covers the panic an SDK call could still raise; either
// way the delete below is reached before the verdict is re-raised.
let outcome = std::panic::AssertUnwindSafe(conditional_write_probe(&s3, &bucket, &key))
.catch_unwind()
.await;
if let Err(e) = s3.delete_object().bucket(&bucket).key(&key).send().await {
eprintln!("tigris conditional-write probe: cleanup of {key} failed: {e}");
}
match outcome {
Ok(Ok(())) => {}
Ok(Err(msg)) => panic!("tigris conditional-write probe: {msg}"),
Err(payload) => std::panic::resume_unwind(payload),
}
}
}
File diff suppressed because it is too large. Load diff
+164
View File
@@ -14,6 +14,68 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
///
/// Kept as its own constant on this PR's base (main still renders
/// `AppError::Db` with `e.to_string()`). If/when #247's `DB_ERROR_MESSAGE`
/// lands, fold this into that shared constant.
pub const GRAPHQL_DB_ERROR_MESSAGE: &str = "a database error occurred";
fn anyhow_has_sqlx(e: &anyhow::Error) -> bool {
e.chain().any(|c| c.downcast_ref::<sqlx::Error>().is_some())
}
/// Map an `anyhow` failure from the db layer to a GraphQL error.
///
/// - Real DB faults (`sqlx::Error` anywhere in the chain) → opaque client
/// message + `error!` log with the full `{e:#}` cause chain.
/// - Application/business errors (e.g. claim race, not-in-claimed-state) →
/// keep the actionable message; log at `warn!` so they are not mistaken for
/// infrastructure failures (#250 review).
pub(crate) fn graphql_db_err(e: anyhow::Error) -> async_graphql::Error {
if anyhow_has_sqlx(&e) {
tracing::error!(error = %format!("{e:#}"), "graphql database error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
} else {
tracing::warn!(error = %format!("{e:#}"), "graphql application error");
async_graphql::Error::new(e.to_string())
}
}
/// Map an `AppError` from a shared collector (e.g. ref-update feed) to a
/// GraphQL error.
///
/// Fail closed: only explicitly curated variants surface their `Display`
/// text. Unnamed variants (including `Git`, which may embed on-disk paths)
/// render opaque so a future addition cannot leak by default (#255 review).
pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error {
match e {
crate::error::AppError::Db(sql) => graphql_db_err(sql.into()),
crate::error::AppError::Internal(err) => {
tracing::error!(error = %format!("{err:#}"), "graphql internal error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
// Curated client-safe variants — `Display` is intentional API text.
safe @ (crate::error::AppError::RepoNotFound(_)
| crate::error::AppError::RepoExists(_)
| crate::error::AppError::NotFound(_)
| crate::error::AppError::Unauthorized(_)
| crate::error::AppError::Forbidden(_)
| crate::error::AppError::BadRequest(_)
| crate::error::AppError::TooManyRequests(_)
| crate::error::AppError::Incomplete(_)) => {
tracing::warn!(error = %safe, "graphql application error");
async_graphql::Error::new(safe.to_string())
}
other => {
tracing::error!(error = %other, "graphql unclassified AppError (opaque)");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
}
}
pub fn build_schema(
db: Arc<Db>,
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
@@ -25,3 +87,105 @@ pub fn build_schema(
.data(task_event_tx)
.finish()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
let leak = "error returned from database: column \"is_public\" does not exist";
// Context layer must not hide sqlx from the chain walk (db helpers
// wrap with `.context(...)` in several places).
let err = graphql_db_err(
anyhow::Error::from(sqlx::Error::Protocol(leak.into())).context("loading repos"),
);
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains("is_public"));
assert!(!err.message.contains(leak));
assert!(!err.message.contains("loading repos"));
}
#[test]
fn graphql_db_err_keeps_business_message() {
let msg = "task not claimable: not found or already claimed";
let err = graphql_db_err(anyhow::anyhow!("{msg}"));
assert_eq!(err.message, msg);
}
#[test]
fn graphql_app_err_opaques_db_and_internal() {
let leak = "column \"is_public\" does not exist";
let db_err = graphql_app_err(crate::error::AppError::Db(sqlx::Error::Protocol(
leak.into(),
)));
assert_eq!(db_err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!db_err.message.contains("is_public"));
let internal = graphql_app_err(crate::error::AppError::Internal(anyhow::anyhow!(
"loading repo: {leak}"
)));
assert_eq!(internal.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!internal.message.contains("is_public"));
}
#[test]
fn graphql_app_err_keeps_safe_variant_messages() {
let err = graphql_app_err(crate::error::AppError::NotFound("widget".into()));
assert!(
err.message.contains("widget"),
"safe NotFound message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
let err = graphql_app_err(crate::error::AppError::BadRequest("bad cid".into()));
assert!(
err.message.contains("bad cid"),
"safe BadRequest message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
}
#[test]
fn graphql_app_err_opaques_unclassified_variants() {
// `Git` may embed on-disk paths from libgit2; fail closed.
let path = "/var/lib/gitlawb/repos/owner/secret.git";
let err = graphql_app_err(crate::error::AppError::Git(format!(
"failed to open '{path}'"
)));
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains(path));
assert!(!err.message.contains("failed to open"));
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
#[test]
fn every_graphql_map_err_uses_opaque_helpers() {
for (file, src) in [
("query.rs", include_str!("query.rs")),
("mutation.rs", include_str!("mutation.rs")),
] {
for (lineno, line) in src.lines().enumerate() {
let code = line.split("//").next().unwrap_or(line);
let Some(idx) = code.find(".map_err(") else {
continue;
};
let after = code[idx + ".map_err(".len()..].trim_start();
let ok = after.starts_with("crate::graphql::graphql_db_err")
|| after.starts_with("crate::graphql::graphql_app_err")
|| after.starts_with("|_|")
|| after.starts_with("|_ ");
assert!(
ok,
"{file}:{}: `.map_err(` must use graphql_db_err / graphql_app_err \
or discard (`|_|`): {line}",
lineno + 1
);
}
}
}
}
+321 -38
View File
@@ -36,6 +36,33 @@ impl MutationRoot {
}
let delegator_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
// #496: same write gate as the REST handler — a repo_id naming a
// hosted, non-quarantined repo admits tasks only from its owner, so a
// stranger cannot file under a foreign repo id. An id naming no hosted
// repo (unknown, or quarantined — hidden as if it did not exist) is
// stored verbatim as an opaque label; the gate does not bind an
// unresolved id to a repo. Task reads are caller-independent on this
// base (the tasks resolver is not repo-scoped), before and after this
// change. As in tasks.rs, the success-vs-denial split between a hosted
// repo id and an unknown one is itself a one-bit oracle — a stranger
// holding a private repo's id learns it is hosted here — but denying
// foreign repos requires drawing exactly that line.
if let Some(repo_id) = input.repo_id.as_deref() {
let record = db
.get_repo_by_id(repo_id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if let Some(record) = record {
let quarantined = db
.is_repo_quarantined(&record.id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if !quarantined {
crate::api::require_repo_owner(&record, caller)
.map_err(crate::graphql::graphql_app_err)?;
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -54,7 +81,7 @@ impl MutationRoot {
};
db.create_task(&task)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
Ok(AgentTaskType::from(task))
}
@@ -76,7 +103,7 @@ impl MutationRoot {
let task = db
.claim_task(&id, &assignee_did)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
@@ -108,7 +135,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -118,7 +145,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "completed", input.result.as_deref())
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -149,7 +176,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -160,7 +187,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -186,47 +213,126 @@ mod tests {
.join("; ")
}
/// N2: GraphQL mutations require a verified signer and bind the acting DID to
/// it. Unsigned is rejected; a signer other than the claimed actor is
/// The `id` scalar returned by a successful `createTask` mutation.
fn created_id(resp: &Response) -> String {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::Object(task) = obj.get("createTask").expect("createTask key")
else {
panic!("createTask not an object");
};
let async_graphql::Value::String(id) = task.get("id").expect("id key") else {
panic!("id not a string");
};
id.clone()
}
/// N2: every GraphQL mutation requires a verified signer and binds the acting
/// DID to it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate.
#[sqlx::test]
async fn mutation_requires_and_binds_signer(pool: PgPool) {
async fn each_mutation_requires_and_binds_signer(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let schema = state.graphql_schema.as_ref();
let assignee = "did:key:zASSIGNEEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let actor = "did:key:zGQLACTORAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let other = "did:key:zGQLOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let cases: Vec<(&str, String)> = vec![
(
"createTask",
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write" }}) {{ id }} }}"#
),
),
(
"claimTask",
format!(
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{actor}") {{ id }} }}"#
),
),
(
"completeTask",
format!(
r#"mutation {{ completeTask(id: "no-such-task", byDid: "{actor}", input: {{}}) {{ id }} }}"#
),
),
(
"failTask",
format!(
r#"mutation {{ failTask(id: "no-such-task", byDid: "{actor}", input: {{}}) {{ id }} }}"#
),
),
];
for (name, q) in &cases {
let resp = schema.execute(Request::new(q.as_str())).await;
assert!(
errors(&resp).contains("authentication required"),
"{name}: unsigned mutation must be rejected: {}",
errors(&resp)
);
let resp = schema
.execute(Request::new(q.as_str()).data(AuthenticatedDid(other.into())))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"{name}: DID mismatch must be rejected: {}",
errors(&resp)
);
let resp = schema
.execute(Request::new(q.as_str()).data(AuthenticatedDid(actor.into())))
.await;
let errs = errors(&resp);
assert!(
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"{name}: matching signer must pass the auth gate: {errs}"
);
if *name == "claimTask" {
assert!(
errs.contains("task not claimable"),
"claim race / missing task must keep its business message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"business error must not be rewritten as opaque DB error: {errs}"
);
}
}
}
/// #250: mutation DB faults must be opaque; create_task hits agent_tasks.
#[sqlx::test]
async fn create_task_db_error_message_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
sqlx::query("ALTER TABLE agent_tasks DROP COLUMN status")
.execute(&pool)
.await
.unwrap();
let delegator = "did:key:zGQLDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAA";
let q = format!(
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{assignee}") {{ id }} }}"#
r#"mutation {{
createTask(
delegatorDid: "{delegator}",
input: {{ kind: "build", capability: "repo:write" }}
) {{ id }}
}}"#
);
// 1. Unsigned → rejected before any DB work.
let resp = schema.execute(Request::new(&q)).await;
assert!(
errors(&resp).contains("authentication required"),
"unsigned mutation must be rejected: {}",
errors(&resp)
);
// 2. Signed as someone other than the claimed assignee → rejected.
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(
"did:key:zOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB".into(),
)))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"DID mismatch must be rejected: {}",
errors(&resp)
);
// 3. Signed as the claimed assignee → passes the auth gate (any remaining
// error is the missing task, not an auth error).
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(assignee.into())))
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(delegator.into())))
.await;
let errs = errors(&resp);
assert!(
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"matching signer must pass the auth gate: {errs}"
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("column") && !errs.contains("status"),
"schema text leaked: {errs}"
);
}
@@ -290,4 +396,181 @@ mod tests {
errors(&resp)
);
}
/// #496 (GraphQL): createTask applies the same repo-ownership gate as the
/// REST handler — a stranger naming a hosted repo is rejected without
/// persisting anything, while the owner files. The quarantined and
/// unknown-id arms are pinned too, including that each accepted write
/// stores the supplied id verbatim, so this copy of the gate cannot regress
/// silently.
#[sqlx::test]
async fn create_task_rejects_foreign_repo_id(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let owner = "did:key:zGQLTASKOWNERAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zGQLTASKSTRANGERBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
)
};
// Stranger signs as themselves (so the signer binding passes) but
// names the victim's repo → rejected by the ownership gate, leaking
// nothing about the repo.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::api::REPO_OWNER_ONLY_MESSAGE),
"a non-owner must not file under a foreign repo id (shared denial text): {errs}"
);
// The denial must not persist: no task row may carry the victim's
// repo id afterwards.
let stored = state
.db
.list_tasks(None, None, 50)
.await
.expect("list tasks");
assert!(
stored
.iter()
.all(|t| t.repo_id.as_deref() != Some(repo.id.as_str())),
"a denied write must not leave a task under the foreign repo id"
);
// The owner files under their own repo id.
let resp = schema
.execute(Request::new(q(owner)).data(AuthenticatedDid(owner.into())))
.await;
assert!(
errors(&resp).is_empty(),
"the owner should file against their repo: {}",
errors(&resp)
);
// Unknown repo ids stay accepted as opaque labels.
let q_unknown = format!(
r#"mutation {{ createTask(delegatorDid: "{stranger}", input: {{ kind: "build", capability: "repo:write", repoId: "no-such-repo-id" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(q_unknown).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"an id naming no hosted repo stays an opaque label: {}",
errors(&resp)
);
// Read back the row: the unresolved id must be stored verbatim, not
// dropped or rewritten to a repo it never named.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some("no-such-repo-id"),
"an unresolved id must be stored verbatim as a label"
);
// A quarantined repo id is treated as unknown (accepted, never a 403
// that would confirm the id names a real repo). This pins the
// `!quarantined` arm on this copy of the gate.
state
.db
.set_repo_quarantine(&repo.id, true)
.await
.expect("quarantine");
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"a quarantined repo id must not 403 (existence oracle): {}",
errors(&resp)
);
// Read back the row: the quarantined id is stored verbatim as a label
// too, so the accepted write is not silently dropped.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some(repo.id.as_str()),
"a quarantined id must be stored verbatim as a label"
);
}
/// #496 (GraphQL): a repo-lookup fault in createTask's gate must stay
/// opaque. Dropping a column `get_repo_by_id` selects faults that lookup;
/// the resolver must return the fixed DB message and leak no schema text
/// (#250), as `create_task_db_error_message_is_opaque` pins for the
/// agent_tasks fault.
#[sqlx::test]
async fn create_task_gate_repo_lookup_fault_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner = "did:key:zGQLTASKFAULTOWNERAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-fault-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-fault-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
sqlx::query("ALTER TABLE repos DROP COLUMN machine_id")
.execute(&pool)
.await
.expect("drop machine_id");
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{owner}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(owner.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"gate sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("machine_id") && !errs.contains("column"),
"schema text leaked through the gate: {errs}"
);
}
}
+102 -7
View File
@@ -14,7 +14,7 @@ impl QueryRoot {
let repos = db
.list_all_repos_deduped()
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
@@ -27,7 +27,7 @@ impl QueryRoot {
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
Ok(repos
.into_iter()
@@ -71,7 +71,7 @@ impl QueryRoot {
let updates =
crate::api::events::collect_visible_ref_updates(db, repo.as_deref(), limit, caller)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_app_err)?;
// Resolve the trusted display owner_did per row, identical to the REST
// feed: the stored wire value is untrusted, so it is echoed only when it
@@ -86,7 +86,7 @@ impl QueryRoot {
let owner_dids = db
.resolve_ref_update_owner_dids(&pairs)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
let resolved: Vec<RefUpdateType> = updates
.into_iter()
@@ -110,13 +110,21 @@ impl QueryRoot {
ctx: &Context<'_>,
status: Option<String>,
assignee_did: Option<String>,
#[graphql(default = 50)] limit: i64,
#[graphql(
default = 50,
desc = "Max 200; larger requests are clamped to 200 (no error). Negative values clamp to 0."
)]
limit: i64,
) -> Result<Vec<AgentTaskType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
// Clamp before SQL: a negative LIMIT is a client fault that Postgres
// rejects with 2201W, which would otherwise trip the opaque DB path
// and write an error-level log on every probe (#250 review).
let limit = limit.clamp(0, 200);
let tasks = db
.list_tasks(status.as_deref(), assignee_did.as_deref(), limit)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
Ok(tasks.into_iter().map(AgentTaskType::from).collect())
}
@@ -125,7 +133,7 @@ impl QueryRoot {
let t = db
.get_task(&id)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
.map_err(crate::graphql::graphql_db_err)?;
Ok(t.map(AgentTaskType::from))
}
}
@@ -419,4 +427,91 @@ mod tests {
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&authed(&schema, q, "did:key:z6MkQuar").await), 0);
}
/// #250: anonymous GraphQL query DB failures must not leak sqlx/schema text.
#[sqlx::test]
async fn repos_query_db_error_message_is_opaque(pool: PgPool) {
let db = db(pool.clone()).await;
db.create_repo(&repo("r1", OWNER, "widget", true))
.await
.unwrap();
sqlx::query("ALTER TABLE repos DROP COLUMN is_public")
.execute(&pool)
.await
.unwrap();
let schema = schema(db);
let resp = anon(&schema, "{ repos { name ownerDid } }").await;
assert!(
!resp.errors.is_empty(),
"DB failure must surface as a GraphQL error"
);
for err in &resp.errors {
assert_eq!(
err.message,
crate::graphql::GRAPHQL_DB_ERROR_MESSAGE,
"raw DB detail leaked into GraphQL error: {}",
err.message
);
assert!(
!err.message.contains("is_public") && !err.message.contains("column"),
"schema text leaked: {}",
err.message
);
}
}
/// #250: negative tasks(limit) must not hit Postgres (and must not 500-log).
#[sqlx::test]
async fn tasks_negative_limit_clamped(pool: PgPool) {
let db = db(pool).await;
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: -1) { id } }").await;
assert!(
resp.errors.is_empty(),
"negative limit must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(count_tasks(&resp), 0);
}
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises Max 200).
#[sqlx::test]
async fn tasks_limit_ceiling_clamped_to_200(pool: PgPool) {
let db = db(pool).await;
let now = Utc::now().to_rfc3339();
for i in 0..201 {
db.create_task(&crate::db::AgentTask {
id: format!("task-ceil-{i}"),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: OWNER.into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: 5000) { id } }").await;
assert_eq!(count_tasks(&resp), 200, "limit above 200 must clamp to 200");
}
fn count_tasks(resp: &async_graphql::Response) -> usize {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::List(rows) = obj.get("tasks").expect("tasks key") else {
panic!("tasks not a list");
};
rows.len()
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+172 -12
View File
@@ -1,4 +1,4 @@
//! Prometheus metrics for the gitlawb node.
//! Prometheus metrics for the Twigpine node.
//!
//! Exposes a small, deliberately conservative set of counters and histograms
//! that cover the questions an operator actually asks of a live node:
@@ -11,6 +11,11 @@
//! `gitlawb_sync_queue_processed_total{status}`
//! * are webhooks reaching their endpoints? —
//! `gitlawb_webhook_deliveries_total{result}`
//! * is inbound gossip being admitted or shed, and for which reason?
//! `gitlawb_gossip_ingest_events_total{outcome}`
//! * is the gossip replay seen-set full, and therefore admitting events it
//! cannot deduplicate?
//! `gitlawb_gossip_replay_guard_saturated_total`
//! * how big are the packs we're sending and receiving? —
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
@@ -33,8 +38,8 @@
use std::sync::OnceLock;
use prometheus::{
Encoder, Histogram, HistogramOpts, IntCounterVec, IntGauge, IntGaugeVec, Opts, Registry,
TextEncoder,
Encoder, Histogram, HistogramOpts, IntCounter, IntCounterVec, IntGauge, IntGaugeVec, Opts,
Registry, TextEncoder,
};
/// The single, process-wide metrics registry. Initialized by [`init`].
@@ -49,18 +54,30 @@ static AUTH_SUCCESSES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_FAILURES: OnceLock<IntCounterVec> = OnceLock::new();
static SYNC_PROCESSED: OnceLock<IntCounterVec> = OnceLock::new();
static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
/// 1 when the node started without its p2p identity because the key storage
/// could not be used. The node keeps serving HTTP in that state and /health
/// stays green, so without this the outage is visible only to whoever reads
/// the startup log.
static P2P_KEY_LOAD_FAILED: OnceLock<IntGauge> = OnceLock::new();
/// One-time initializer. Builds the registry, registers every metric,
/// and sets the constant `gitlawb_info` gauge. Idempotent — calling
/// more than once is a silent no-op. MUST be called from `main()` after
/// the node DID is known.
pub fn init(version: &str, node_did: &str) {
if REGISTRY.get().is_some() {
return;
}
// Guard with Once so two concurrent callers cannot both pass an unsynchronized
// `REGISTRY.get()` check and then race on `OnceLock::set(...).expect(...)`,
// panicking the loser (#192 F4). Once runs the body exactly once, so the
// `.expect("set X once")` calls below can never observe an already-set slot.
static INIT: std::sync::Once = std::sync::Once::new();
INIT.call_once(|| init_inner(version, node_did));
}
fn init_inner(version: &str, node_did: &str) {
let registry = Registry::new();
let info = IntGaugeVec::new(
@@ -163,6 +180,33 @@ pub fn init(version: &str, node_did: &str) {
.set(webhook_deliveries)
.expect("set WEBHOOK_DELIVERIES once");
let gossip_ingest = IntCounterVec::new(
Opts::new(
"gitlawb_gossip_ingest_events_total",
"Total inbound gossip ref-update events and what the ingest path decided about each",
),
&["outcome"],
)
.expect("gitlawb_gossip_ingest_events_total definition");
registry
.register(Box::new(gossip_ingest.clone()))
.expect("register gitlawb_gossip_ingest_events_total");
GOSSIP_INGEST
.set(gossip_ingest)
.expect("set GOSSIP_INGEST once");
let gossip_replay_guard_saturated = IntCounter::with_opts(Opts::new(
"gitlawb_gossip_replay_guard_saturated_total",
"Inbound gossip ref-updates admitted without being recorded because the replay seen-set was at capacity",
))
.expect("gitlawb_gossip_replay_guard_saturated_total definition");
registry
.register(Box::new(gossip_replay_guard_saturated.clone()))
.expect("register gitlawb_gossip_replay_guard_saturated_total");
GOSSIP_REPLAY_GUARD_SATURATED
.set(gossip_replay_guard_saturated)
.expect("set GOSSIP_REPLAY_GUARD_SATURATED once");
let pack_size = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_pack_size_bytes",
@@ -197,6 +241,18 @@ pub fn init(version: &str, node_did: &str) {
.set(peers_connected)
.expect("set PEERS_CONNECTED once");
let p2p_key_load_failed = IntGauge::with_opts(Opts::new(
"gitlawb_p2p_identity_key_load_failed",
"1 if the node is running without p2p because its identity key could not be loaded",
))
.expect("gitlawb_p2p_identity_key_load_failed definition");
registry
.register(Box::new(p2p_key_load_failed.clone()))
.expect("register gitlawb_p2p_identity_key_load_failed");
P2P_KEY_LOAD_FAILED
.set(p2p_key_load_failed)
.expect("set P2P_KEY_LOAD_FAILED once");
REGISTRY
.set(registry)
.expect("set REGISTRY once (init must be called exactly once)");
@@ -218,6 +274,17 @@ pub fn record_fetch(repo: &str) {
}
}
/// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry
/// is not initialized). Lets api-layer tests assert the completed-fetch count with
/// a unique label instead of scraping the encoded text.
#[cfg(test)]
pub fn fetch_count_for_test(repo: &str) -> u64 {
FETCHES
.get()
.map(|c| c.with_label_values(&[repo]).get())
.unwrap_or(0)
}
/// Record one HTTP signature check that passed.
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_success(route: &str) {
@@ -247,13 +314,75 @@ pub fn record_sync_processed(status: &str) {
}
/// Record one webhook delivery attempt outcome. `result` ∈
/// {ok, http_error, network_error, skipped}.
/// {ok, http_error, network_error, skipped, invalid_url}.
/// `invalid_url` is a policy skip: the stored hook URL failed the public-URL
/// check at delivery time and the POST was never sent.
pub fn record_webhook_delivery(result: &str) {
if let Some(c) = WEBHOOK_DELIVERIES.get() {
c.with_label_values(&[result]).inc();
}
}
/// Record what the gossip ingest path decided about one inbound ref-update.
/// `outcome` ∈ {accepted, unsigned_admitted, write_failed, rejected,
/// source_rate_limited, author_rate_limited, unsigned_source_rate_limited,
/// replayed, stale_timestamp}.
///
/// `accepted` is reserved for signature-verified events. An unsigned event that
/// survives the rolling-upgrade window is `unsigned_admitted`, so the
/// authenticated-admission rate is not silently padded by legacy compatibility
/// traffic; an operator can tell whether the fleet still relies on the
/// compatibility allowance.
///
/// The three shed reasons stay separate labels rather than one `rate_limited`
/// because they answer different operator questions: `source_rate_limited` is a
/// forwarding peer over the pre-parse brake, `unsigned_source_rate_limited` is
/// that same forwarder relaying unproven traffic, and `author_rate_limited`
/// names a principal a signature actually proved. Collapsing them would make a
/// flood of unauthenticated garbage indistinguishable from one registered peer
/// pushing too hard, which is exactly the distinction an alert needs.
///
/// The label is the variant, never the free-form reason string that
/// `Rejected`/`WriteFailed` carry: those are attacker-influenced and would blow
/// up the label cardinality of a process-wide registry.
pub fn record_gossip_ingest(outcome: &str) {
if let Some(c) = GOSSIP_INGEST.get() {
c.with_label_values(&[outcome]).inc();
}
}
/// Record one inbound gossip ref-update that the replay seen-set could not
/// record because it was at capacity.
///
/// Label-less on purpose: there is one degradation mode and nothing about it
/// varies per event, so a label would only add cardinality.
///
/// This is an alert-worthy signal, not a debug counter, and it is the ONLY way
/// the degraded state is visible. At capacity the guard fails open, which means
/// the event is admitted and `gitlawb_gossip_ingest_events_total{outcome=
/// "accepted"}` counts it exactly as it counts a healthy admission. Fail-open is
/// the right policy (a saturated set that dropped all fresh gossip would convert
/// a loud resource attack into quiet mesh-wide censorship), but it is not a free
/// one: while this counter is moving, a captured signature can again be replayed
/// against the freshness window's full 10 minutes, spending a `peer_exists`
/// round trip and a debit from the victim author's budget on every replay.
pub fn record_gossip_replay_guard_saturated() {
if let Some(c) = GOSSIP_REPLAY_GUARD_SATURATED.get() {
c.inc();
}
}
/// Test-only: current `gitlawb_gossip_replay_guard_saturated_total` value (0 if
/// the registry is not initialized). Lets the seen-set tests assert the
/// saturation signal fires without scraping the encoded text.
#[cfg(test)]
pub fn replay_guard_saturated_count_for_test() -> u64 {
GOSSIP_REPLAY_GUARD_SATURATED
.get()
.map(|c| c.get())
.unwrap_or(0)
}
/// Record a pack body size observation (bytes).
pub fn observe_pack_size(bytes: f64) {
if let Some(h) = PACK_SIZE.get() {
@@ -268,6 +397,13 @@ pub fn set_peers_connected(count: i64) {
}
}
/// Record whether the node is running without its p2p identity.
pub fn set_p2p_key_load_failed(failed: bool) {
if let Some(g) = P2P_KEY_LOAD_FAILED.get() {
g.set(i64::from(failed));
}
}
/// Encode the registry as the standard Prometheus text exposition format.
/// Returns an error if `init` was never called.
pub fn encode() -> Result<String, prometheus::Error> {
@@ -286,11 +422,11 @@ pub fn encode() -> Result<String, prometheus::Error> {
mod tests {
use super::*;
// Note: these tests are not run in parallel by default (cargo runs
// tests in a binary in parallel via threads but the OnceLock guard
// means only the first init() call succeeds; subsequent ones are
// no-ops). The encode test below is structured so it's safe to run
// alongside other tests in the same binary.
// Note: cargo runs tests in a binary in parallel via threads, and several
// tests here call init(). init() is guarded by std::sync::Once, so concurrent
// callers run the body exactly once and never race on the OnceLock setters
// (#192 F4). The encode test below is structured so it's safe to run alongside
// other tests in the same binary regardless of ordering.
#[test]
fn encode_after_init_returns_prometheus_text() {
@@ -321,6 +457,24 @@ mod tests {
);
}
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
/// hit concurrent callers (both passing the old unsynchronized `REGISTRY.get()`
/// check, then racing on `OnceLock::set(...).expect(...)`) is fixed by the
/// `Once` guard. That race is inherently non-deterministic to assert in the
/// shared-binary suite — a 32-thread barrier stress-test reproduces it only in
/// isolation (verified: RED before the `Once` fix, GREEN after). This guard
/// covers the deterministic half of the contract: a repeat call is a no-op,
/// never a panic, and the registry stays usable.
#[test]
fn init_is_idempotent_no_panic_on_repeat() {
init("0.0.0-a", "did:key:a");
init("0.0.0-b", "did:key:b");
assert!(
encode().is_ok(),
"registry must stay usable after a repeated init"
);
}
#[test]
fn record_helpers_are_noops_before_init() {
// These don't panic. They also don't show up in encode() output
@@ -331,6 +485,12 @@ mod tests {
record_auth_failure("test/route", "test_reason");
record_sync_processed("done");
record_webhook_delivery("ok");
// `record_gossip_replay_guard_saturated` is deliberately NOT called
// here. It is a label-less process-wide counter and the seen-set test
// that asserts it asserts a before/after delta, so an increment from a
// second test running concurrently in this binary would be a flake with
// no diagnostic value.
record_gossip_ingest("accepted");
observe_pack_size(1024.0);
set_peers_connected(0);
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+208
View File
@@ -121,6 +121,45 @@ impl RateLimiter {
true
}
/// Non-consuming check: is this key ALREADY at its limit for the current window?
/// Unlike [`check`], it records nothing and never inserts a new key — used to shed
/// expensive preparatory work (e.g. the `/ipfs/{cid}` legacy scan's O(repos) DB
/// preload) BEFORE it runs, without perturbing the per-unit budget the consuming
/// `check` maintains (#173, F3). An unknown key or a disabled limiter is not
/// throttled. Prunes the key's expired timestamps as a side effect (keeps state
/// tidy) but adds none, so it cannot itself fill or grow the map.
pub(crate) async fn is_throttled(&self, key: &str) -> bool {
if self.max_requests == 0 {
return false;
}
let now = Instant::now();
let mut state = self.state.lock().await;
if let Some(window) = state.get_mut(key) {
window
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
return window.timestamps.len() >= self.max_requests;
}
false
}
/// The configured key ceiling. Tests use it to assert that a limiter a
/// production factory builds is actually bounded, which flooding the real
/// ceiling in distinct keys would be far too slow to show.
#[cfg(test)]
pub fn max_keys(&self) -> usize {
self.max_keys
}
/// The configured per-key request cap. Tests use it to assert that a
/// limiter a production factory builds carries the cap it is documented
/// with; re-asserting the constant on its own cannot tell whether the
/// factory passed that constant or a different one.
#[cfg(test)]
pub fn max_requests(&self) -> usize {
self.max_requests
}
pub async fn cleanup(&self) {
let now = Instant::now();
let mut state = self.state.lock().await;
@@ -130,6 +169,118 @@ impl RateLimiter {
!w.timestamps.is_empty()
});
}
/// Number of distinct keys currently tracked. Test-only introspection so a
/// cross-module test can assert that a sweep actually evicted expired entries
/// and observe what it reclaimed. There is no production reader.
#[cfg(test)]
pub(crate) async fn tracked_keys(&self) -> usize {
self.state.lock().await.len()
}
}
/// Per-source concurrency cap derived from the write-pool size: one resolved client
/// key (see [`client_key`]) may hold at most an eighth of the pool, so saturating it
/// takes ~8 distinct keys. Real for an IPv4 or single-address caller; a caller with a
/// routed IPv6 /64 has 2^64 keys, because `client_key` returns the full address with
/// no prefix folding. Narrowing that keying is a deferred decision, so this cap is a
/// bound per key, not a bound per operator.
///
/// Floored at 1 because the value feeds [`PerCallerConcurrency`], where a cap of 0
/// would shed EVERY receive-pack advertisement and break all pushes. The floor is
/// load-bearing at the minimum write-pool size (1), which integer-divides to 0.
pub(crate) fn per_source_push_cap(max_concurrent_git_pushes: usize) -> usize {
(max_concurrent_git_pushes / 8).max(1)
}
/// A bounded per-caller CONCURRENCY limiter — distinct from [`RateLimiter`], which
/// caps request RATE. Each caller key may hold at most `per_caller` in-flight
/// permits at once; beyond that [`try_acquire`](Self::try_acquire) returns `None`
/// and the caller sheds. Used to stop one caller (a single anonymous source-IP or
/// DID) monopolizing the served-git read pool (#174).
///
/// The key map is self-bounding: a key is removed the instant its in-flight count
/// reaches zero, so it never holds more keys than there are concurrently-active
/// callers (itself bounded by the read semaphore). A `max_keys` reject-before-insert
/// backstop guarantees a key farm can never grow the map even if that invariant
/// weakened — a NEW key at the cap is rejected WITHOUT allocating an entry (INV-15).
///
/// Uses a `std::sync::Mutex` (not the file's `tokio::sync::Mutex`) because the
/// permit's `Drop` must release synchronously; the critical section holds no await.
#[derive(Clone)]
pub struct PerCallerConcurrency {
state: Arc<std::sync::Mutex<HashMap<String, usize>>>,
per_caller: usize,
max_keys: usize,
}
/// RAII permit from [`PerCallerConcurrency::try_acquire`]. On drop it decrements
/// the caller's in-flight count and removes the key when it reaches zero.
pub struct PerCallerPermit {
state: Arc<std::sync::Mutex<HashMap<String, usize>>>,
key: String,
}
impl PerCallerConcurrency {
pub fn new(per_caller: usize, max_keys: usize) -> Self {
Self {
state: Arc::new(std::sync::Mutex::new(HashMap::new())),
per_caller: per_caller.max(1),
max_keys: max_keys.max(1),
}
}
/// Convenience constructor with the default key bound.
pub fn with_default_max_keys(per_caller: usize) -> Self {
Self::new(per_caller, DEFAULT_MAX_KEYS)
}
/// `Some(permit)` when the caller is under its cap and the map has room;
/// `None` (shed) otherwise. Reject-before-insert: a new key at `max_keys` is
/// rejected without allocating.
pub fn try_acquire(&self, key: &str) -> Option<PerCallerPermit> {
// Recover from a poisoned lock rather than panicking: the critical section
// is pure counter arithmetic and cannot itself panic, so a poisoned mutex
// would only ever come from an unrelated abort, and a slightly-off count
// self-heals as permits drop. A panic here would instead brick the limiter
// for every caller (each subsequent lock re-panics).
let mut map = self.state.lock().unwrap_or_else(|e| e.into_inner());
match map.get_mut(key) {
Some(count) => {
if *count >= self.per_caller {
return None;
}
*count += 1;
}
None => {
if map.len() >= self.max_keys {
return None;
}
map.insert(key.to_string(), 1);
}
}
Some(PerCallerPermit {
state: self.state.clone(),
key: key.to_string(),
})
}
#[cfg(test)]
pub fn tracked_keys(&self) -> usize {
self.state.lock().unwrap().len()
}
}
impl Drop for PerCallerPermit {
fn drop(&mut self) {
let mut map = self.state.lock().unwrap_or_else(|e| e.into_inner());
if let Some(count) = map.get_mut(&self.key) {
*count -= 1;
if *count == 0 {
map.remove(&self.key);
}
}
}
}
pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
@@ -288,6 +439,63 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
mod tests {
use super::*;
#[test]
fn per_caller_concurrency_caps_one_caller_and_frees_on_drop() {
let lim = PerCallerConcurrency::new(2, 100);
let p1 = lim.try_acquire("did:key:zA").expect("first under cap");
let p2 = lim.try_acquire("did:key:zA").expect("second under cap");
assert!(
lim.try_acquire("did:key:zA").is_none(),
"a third in-flight op for the same caller sheds (over the per-caller cap)"
);
// A DIFFERENT caller is unaffected — the cap is per-caller, not global.
let _other = lim
.try_acquire("did:key:zB")
.expect("a different caller has its own budget");
drop(p1);
assert!(
lim.try_acquire("did:key:zA").is_some(),
"freeing one in-flight slot lets the same caller back in"
);
drop(p2);
}
#[test]
fn per_caller_concurrency_map_is_self_bounding_and_reject_before_insert() {
// Self-bounding: acquire+drop many distinct keys — the map never grows
// because a key is removed the instant its in-flight count hits zero.
let lim = PerCallerConcurrency::new(4, 3);
for i in 0..50 {
let _p = lim.try_acquire(&format!("k{i}"));
}
assert_eq!(
lim.tracked_keys(),
0,
"keys with zero in-flight ops are removed, so an acquire+drop flood leaves the map empty"
);
// Reject-before-insert: HOLD max_keys distinct keys, then a new key sheds
// WITHOUT growing the map past the cap (INV-15 — a rejected request never
// allocates an entry).
let held: Vec<_> = (0..3)
.map(|i| lim.try_acquire(&format!("h{i}")).unwrap())
.collect();
assert_eq!(
lim.tracked_keys(),
3,
"three distinct callers held concurrently"
);
assert!(
lim.try_acquire("h3").is_none(),
"a new key at max_keys is rejected"
);
assert_eq!(
lim.tracked_keys(),
3,
"the rejected new key did not allocate an entry (reject-before-insert)"
);
drop(held);
}
#[tokio::test]
async fn allows_within_limit() {
let limiter = RateLimiter::new(3, Duration::from_secs(60));
+780 -17
View File
@@ -42,17 +42,37 @@ async fn graphql_playground() -> impl IntoResponse {
))
}
/// Pre-auth body-buffer ceiling for every signed route group except the git write
/// routes. Deliberately equal to axum's `DefaultBodyLimit` (2 MB), which is what
/// these groups' `Bytes`/`Json` extractors already enforce: that default works
/// through a request extension the extractors consult, and `require_signature`
/// collects the raw body itself, so it never saw the cap. Matching the number
/// keeps this change behavior-preserving on the response path (nothing that used
/// to be accepted starts failing) and purely a memory fix.
pub const SIGNED_BODY_LIMIT: usize = 2 * 1024 * 1024;
/// Applies the standard auth middleware pair to a router: HTTP Signature verification
/// followed by UCAN chain validation. The two layers run in this order for every
/// matched request: `require_signature` first (sets `AuthenticatedDid`), then
/// `require_ucan_chain` (reads it).
fn add_auth_layers(router: Router<AppState>, state: AppState) -> Router<AppState> {
///
/// `body_limit` bounds the request body `require_signature` buffers before it has
/// authenticated anything (see [`auth::SignatureBodyLimit`]): `max_pack_bytes` for
/// the git write group, [`SIGNED_BODY_LIMIT`] for every other group.
fn add_auth_layers(
router: Router<AppState>,
state: AppState,
body_limit: usize,
) -> Router<AppState> {
router
.layer(middleware::from_fn_with_state(
state,
auth::require_ucan_chain,
))
.layer(middleware::from_fn(auth::require_signature))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(body_limit),
auth::require_signature,
))
}
pub fn build_router(state: AppState) -> Router {
@@ -63,7 +83,10 @@ pub fn build_router(state: AppState) -> Router {
// Attach the verified DID to /graphql when a signature is present. The
// layer covers only routes added before it, so /graphql/ws (added after,
// read-only subscriptions) stays open.
.layer(middleware::from_fn(auth::optional_signature))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
))
.route_service("/graphql/ws", GraphQLSubscription::new(schema));
// ── Task routes (write — require HTTP Signature) ───────────────────────
@@ -74,6 +97,7 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task)),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Task routes (read — open) ──────────────────────────────────────────
@@ -107,6 +131,7 @@ pub fn build_router(state: AppState) -> Router {
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(limiter)),
state.clone(),
SIGNED_BODY_LIMIT,
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(create_ip_limiter));
@@ -181,6 +206,7 @@ pub fn build_router(state: AppState) -> Router {
axum::routing::delete(agents::deregister_agent),
),
state.clone(),
SIGNED_BODY_LIMIT,
);
// Body limit is raised to GITLAWB_MAX_PACK_BYTES (default 2 GB) for git
@@ -205,22 +231,68 @@ pub fn build_router(state: AppState) -> Router {
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit)),
state.clone(),
pack_limit,
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(push_limiter));
// ── IPFS content-addressed retrieval and pin listing ──────────────────
// `/ipfs/{cid}` carries `optional_signature` so `get_by_cid` sees the caller
// identity and can apply per-repo visibility (#110); anonymous callers stay
// anonymous and still read genuinely public content. `/api/v1/ipfs/pins`
// stays unsigned — gating the pin index is tracked separately (#121).
let ipfs_routes = Router::new()
// Two independent sub-routers, then merged. They share a URL prefix family
// but have separate rate-limit policies and must not share a bucket.
//
// `/ipfs/{cid}` (CID resolver): carries `optional_signature` so `get_by_cid`
// sees the caller identity and can apply per-repo visibility (#110); anon
// callers stay anonymous and still read genuinely public content. The
// per-IP flood brake is layered on because the resolver is anon-reachable
// and each request can drive a full-history git walk — the brake is the
// outermost layer (rejects a flood before the walk-admission work), mirroring
// the push/create routers. The `IpRateLimiter` extension MUST be attached
// or `rate_limit_by_ip` is a silent no-op.
//
// `/api/v1/ipfs/pins` (pin listing): now carries `optional_signature` only
// (#121). The handler rejects requests without a verified `AuthenticatedDid`
// with 401. It does NOT carry the CID flood brake — `list_pins` is a single
// `list_pinned_cids()` call, no walk, and routing pins through the resolver's
// bucket would let `/ipfs/{cid}` traffic exhaust the bucket and 429 the
// pins endpoint, or let signed pin polling exhaust the bucket for legitimate
// CID reads. The two surfaces have separate availability contracts.
//
// Both sub-routers are built first with their own layer sets, then merged.
// This is the structure the prior routing guidance called for and the
// earlier 429 test for `/ipfs/{cid}` (test_support.rs) assumes.
let ipfs_limiter = rate_limit::IpRateLimiter {
limiter: state.ipfs_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let ipfs_cid_routes = Router::new()
.route("/ipfs/{cid}", get(ipfs::get_by_cid))
.layer(middleware::from_fn(auth::optional_signature))
.merge(Router::new().route("/api/v1/ipfs/pins", get(ipfs::list_pins)));
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
))
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(ipfs_limiter));
let ipfs_pins_routes = Router::new()
.route("/api/v1/ipfs/pins", get(ipfs::list_pins))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
let ipfs_routes = ipfs_cid_routes.merge(ipfs_pins_routes);
// ── Arweave permanent anchors ──────────────────────────────────────────
let arweave_routes = Router::new().route("/api/v1/arweave/anchors", get(arweave::list_anchors));
// `list_anchors` rejects callers without a verified `AuthenticatedDid`, so
// unsigned enumeration is denied. The same `optional_signature` layer used
// on the other read surfaces is applied here, closing the anonymous
// anchor-listing gap alongside `/api/v1/ipfs/pins`. The verify endpoint
// and its probe are deferred to the vertical slice with the uploader and
// a real provider contract.
let arweave_routes = Router::new()
.route("/api/v1/arweave/anchors", get(arweave::list_anchors))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
// ── Bounty routes (write — require HTTP Signature) ─────────────────
let bounty_write_routes = add_auth_layers(
@@ -247,6 +319,7 @@ pub fn build_router(state: AppState) -> Router {
post(bounties::dispute_bounty),
),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Bounty routes (read — open) ──────────────────────────────────────
@@ -262,12 +335,16 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/agents/{did}/bounties",
get(bounties::agent_bounty_stats),
)
.layer(middleware::from_fn(auth::optional_signature));
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
// ── Profile routes (write — require HTTP Signature) ─────────────────
let profile_write_routes = add_auth_layers(
Router::new().route("/api/v1/profile", axum::routing::put(profiles::set_profile)),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Issue routes (write — require HTTP Signature, no rate limit) ─────
@@ -282,6 +359,7 @@ pub fn build_router(state: AppState) -> Router {
post(issues::create_issue_comment),
),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Peer discovery routes ─────────────────────────────────────────────
@@ -300,6 +378,7 @@ pub fn build_router(state: AppState) -> Router {
let sync_trigger_routes = add_auth_layers(
Router::new().route("/api/v1/sync/trigger", post(peers::trigger_sync)),
state.clone(),
SIGNED_BODY_LIMIT,
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
@@ -316,9 +395,12 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/peers/announce", post(peers::announce))
.route("/api/v1/sync/notify", post(peers::notify_sync));
peer_write_routes = if state.config.require_signed_peer_writes {
add_auth_layers(peer_write_routes, state.clone())
add_auth_layers(peer_write_routes, state.clone(), SIGNED_BODY_LIMIT)
} else {
peer_write_routes.layer(middleware::from_fn(auth::optional_signature))
peer_write_routes.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
))
};
let peer_write_routes = peer_write_routes
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
@@ -330,7 +412,15 @@ pub fn build_router(state: AppState) -> Router {
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
.route("/api/v1/repos", get(repos::list_repos))
.route("/api/v1/repos/federated", get(repos::list_federated_repos))
.route(
"/api/v1/repos/federated",
get(repos::list_federated_repos)
.route_layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.route_layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.federated_rate_limiter.clone(),
trust: state.push_limiter_trust,
})),
)
.route("/api/v1/repos/{owner}/{repo}", get(repos::get_repo))
.route(
"/api/v1/repos/{owner}/{repo}/commits",
@@ -417,7 +507,10 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/repos/{owner}/{repo}/replicas",
get(replicas::list_replicas),
)
.layer(middleware::from_fn(auth::optional_signature));
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
// git-upload-pack (clone/fetch) — same raised body limit as receive-pack so
// large pack responses from the server don't get truncated on the client side.
@@ -449,7 +542,13 @@ pub fn build_router(state: AppState) -> Router {
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit))
.layer(middleware::from_fn(auth::optional_signature));
// `pack_limit`, not SIGNED_BODY_LIMIT: git-upload-pack POSTs live here and
// the group has deliberately raised its ceiling, so the pre-auth buffer for
// a signed fetch negotiation must match rather than narrow it.
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(pack_limit),
auth::optional_signature,
));
// ── Meta ──────────────────────────────────────────────────────────────
let meta_routes = Router::new()
@@ -608,3 +707,667 @@ async fn p2p_info(State(state): State<AppState>) -> Json<serde_json::Value> {
None => Json(json!({ "enabled": false })),
}
}
#[cfg(test)]
mod tests {
use super::*;
use axum::body::Body;
use axum::http::{Request, StatusCode};
use sqlx::PgPool;
use tower::ServiceExt;
use crate::db::RepoRecord;
use crate::test_support::test_state;
/// Regression: anonymous callers must not see the pin/anchor index (#121, #134).
#[sqlx::test]
async fn unsigned_get_pins_and_anchors_is_401_through_build_router(pool: PgPool) {
let state = test_state(pool).await;
// Seed identifiable rows so the denial oracle proves the 401 body omits
// them: a status-only check would stay green if a later change served
// the listing before the auth layer rejected.
state
.db
.record_pinned_cid("seeded-pin-oid", "bafyPIN-SECRET-DO-NOT-LEAK", None)
.await
.expect("seed pin");
let owner_did = "did:key:zSEEDOWNERAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let repo = seed_repo_inline(owner_did, "seeded-repo");
state.db.create_repo(&repo).await.expect("seed repo");
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: "zSEEDOWNERAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/seeded-repo",
owner_did,
ref_name: "refs/heads/secret",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-secret-tx-DO-NOT-LEAK",
arweave_url: "https://arweave.net/secret-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let router = build_router(state);
let pins = Request::builder()
.method("GET")
.uri("/api/v1/ipfs/pins?limit=50")
.body(Body::empty())
.unwrap();
let pins_resp = router.clone().oneshot(pins).await.unwrap();
assert_eq!(
pins_resp.status(),
StatusCode::UNAUTHORIZED,
"anonymous pin listing must be rejected"
);
let bytes = axum::body::to_bytes(pins_resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("bafyPIN-SECRET-DO-NOT-LEAK"),
"the 401 body must not leak the seeded pin cid; body was: {body}"
);
let anchors = Request::builder()
.method("GET")
.uri("/api/v1/arweave/anchors?limit=50")
.body(Body::empty())
.unwrap();
let anchors_resp = router.oneshot(anchors).await.unwrap();
assert_eq!(
anchors_resp.status(),
StatusCode::UNAUTHORIZED,
"anonymous anchors listing must be rejected"
);
let bytes = axum::body::to_bytes(anchors_resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("irys-secret-tx-DO-NOT-LEAK"),
"the 401 body must not leak the anchor's irys tx id; body was: {body}"
);
}
/// Regression: a real RFC-9421 signature produced exactly as `gl` does — built
/// with `gitlawb_core::http_sig::sign_request` over a GET, headers attached,
/// and sent through the actual `build_router` — is verified by the
/// `optional_signature` layer that wraps the pin/anchor routes, and the
/// handler returns 200. Pairs with the anonymous-denial test above; one
/// proves headers are required, the other proves a valid header is honored.
/// Without this test the unsigned-denial test would stay green even if the
/// `optional_signature` layer were never wired onto these routes, because
/// signed and unsigned requests would fail identically (#134 review).
#[sqlx::test]
async fn signed_get_pins_and_anchors_succeeds_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let state = test_state(pool).await;
let router = build_router(state);
let kp = Keypair::generate();
let pins_path = "/api/v1/ipfs/pins";
let signed = sign_request(&kp, "GET", pins_path, b"");
let pins = Request::builder()
.method("GET")
.uri(pins_path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let pins_resp = router.clone().oneshot(pins).await.unwrap();
assert_eq!(
pins_resp.status(),
StatusCode::OK,
"a valid signature on /api/v1/ipfs/pins must be honored through build_router"
);
let anchors_path = "/api/v1/arweave/anchors";
let signed = sign_request(&kp, "GET", anchors_path, b"");
let anchors = Request::builder()
.method("GET")
.uri(anchors_path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let anchors_resp = router.oneshot(anchors).await.unwrap();
assert_eq!(
anchors_resp.status(),
StatusCode::OK,
"a valid signature on /api/v1/arweave/anchors must be honored through build_router"
);
}
/// Companion to the two regressions above: a request that *carries* signature
/// headers but whose signature does not verify (here, garbled) must be denied
/// with 401, not silently treated as anonymous and re-checked by the handler.
/// This pins the failure mode of the `optional_signature` layer: when the
/// caller claims to be signed, the layer must commit to verifying — there is
/// no fall-through path that lets a bad signature bypass auth.
#[sqlx::test]
async fn malformed_signature_on_pins_is_401_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let state = test_state(pool).await;
let router = build_router(state);
let kp = Keypair::generate();
let path = "/api/v1/ipfs/pins";
let mut signed = sign_request(&kp, "GET", path, b"");
// Flip a character well inside the signature value (base64) so the header
// still parses but does not verify against the key.
let mut tampered = signed.signature.clone();
let mid = tampered.len() / 2;
let flipped = if tampered.as_bytes()[mid] == b'A' {
'B'
} else {
'A'
};
tampered.replace_range(mid..mid + 1, &flipped.to_string());
signed.signature = tampered;
let req = Request::builder()
.method("GET")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"a malformed signature must be rejected by the auth layer, not silently accepted"
);
}
/// The same malformed-signature rejection must hold on the anchors route,
/// which carries its own `optional_signature` layer beside pins: a later
/// edit that dropped the layer here (but not on pins) would otherwise stay
/// green.
#[sqlx::test]
async fn malformed_signature_on_anchors_is_401_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let state = test_state(pool).await;
let router = build_router(state);
let kp = Keypair::generate();
let path = "/api/v1/arweave/anchors";
let mut signed = sign_request(&kp, "GET", path, b"");
let mut tampered = signed.signature.clone();
let mid = tampered.len() / 2;
let flipped = if tampered.as_bytes()[mid] == b'A' {
'B'
} else {
'A'
};
tampered.replace_range(mid..mid + 1, &flipped.to_string());
signed.signature = tampered;
let req = Request::builder()
.method("GET")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"a malformed signature must be rejected by the auth layer, not silently accepted"
);
}
// ── Scoped anchor contract (P1 follow-up: ?repo= must authorize_repo_read) ──
//
// The follow-up review (after the auth-layer wiring landed) found that a
// signed but unauthorized caller could still obtain scoped anchor metadata
// because the `?repo=` branch handed the user-supplied string straight to
// the SQL filter. These tests exercise the full production contract end to
// end: real RFC-9421 signature → `optional_signature` layer →
// `authorize_repo_read` → SQL. They fail closed if the authz call is moved
// after the query or removed entirely.
/// Inline repo seed for the scoped-anchor tests. Mirrors the shape in
/// `test_support::tests::seed_repo` without taking a cross-module private
/// helper as a dependency.
fn seed_repo_inline(owner_did: &str, name: &str) -> RepoRecord {
let now = chrono::Utc::now();
RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
}
}
/// Unsigned scoped anchor request → 401, identical to the global anchor
/// 401 test. Proves the auth layer fires before any scope decision (no
/// existence oracle via the `?repo=` path either).
#[sqlx::test]
async fn unsigned_scoped_anchors_is_401_through_build_router(pool: PgPool) {
let state = test_state(pool).await;
// Seed an anchor under a repo the request claims to scope to, so the
// denial oracle proves the 401 body omits it rather than merely
// returning the right status over an empty table.
let owner_did = "did:key:zSCOPEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let repo = seed_repo_inline(owner_did, "priv");
state.db.create_repo(&repo).await.expect("seed repo");
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: "zSCOPEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/priv",
owner_did,
ref_name: "refs/heads/secret",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-scoped-secret-DO-NOT-LEAK",
arweave_url: "https://arweave.net/scoped-secret",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let router = build_router(state);
let resp = router
.oneshot(
Request::builder()
.method("GET")
.uri("/api/v1/arweave/anchors?repo=did:key:zSCOPED%2Fpriv")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"unsigned scoped anchor request must be 401, before any scope lookup"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("irys-scoped-secret-DO-NOT-LEAK"),
"the 401 body must not leak the anchor's irys tx id; body was: {body}"
);
}
/// Signed owner on a private repo → 200, anchor metadata returned. Mirrors
/// the existing `list_webhooks_accepts_a_real_gl_signature_e2e` shape: real
/// signature, real middleware, real `authorize_repo_read`.
#[sqlx::test]
async fn signed_scoped_anchors_owner_succeeds_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let owner_did = kp.did().to_string();
let state = test_state(pool).await;
let mut repo = seed_repo_inline(&owner_did, "scoped-priv");
repo.is_public = false;
state.db.create_repo(&repo).await.expect("seed repo");
let short = owner_did.split(':').next_back().unwrap().to_string();
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!("{short}/scoped-priv"),
owner_did: &owner_did,
ref_name: "refs/heads/main",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-owner-tx",
arweave_url: "https://arweave.net/owner-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let path = format!("/api/v1/arweave/anchors?repo={short}/scoped-priv");
let signed = sign_request(&kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"the owner of a private repo must see their scoped anchors"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
body.contains("irys-owner-tx"),
"owner must see the anchor's irys tx id; body was: {body}"
);
// Owner must see the ref name + new SHA so this is not just a count oracle.
assert!(
body.contains("refs/heads/main"),
"owner must see the ref name; body was: {body}"
);
// Sanity: a second test would need a different repo name to avoid
// colliding on `did:key:zSCOPED/scoped-priv` in the anchors table.
// Distinct repo names per test keep the rows queryable in isolation.
let _ = repo;
}
/// #121 R3: the SQL filter must use the canonical stored slug, not the
/// user-supplied `?repo=` string. Anchor rows are stored as
/// `{normalize_owner_key(owner_did)}/{name}` (the short form). Authz
/// passes for the full-DID form, but a literal `WHERE repo=$1` against
/// the full DID would match zero rows and return a false empty page,
/// breaking callers that legitimately use the full `did:key:…/name`
/// form in `?repo=`. RED before the fix: handler built the filter from
/// the raw `?repo=` string, so this query returned `anchors: []`.
#[sqlx::test]
async fn signed_scoped_anchors_full_did_form_matches_stored_slug_through_build_router(
pool: PgPool,
) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let owner_did = kp.did().to_string();
let state = test_state(pool).await;
let mut repo = seed_repo_inline(&owner_did, "scoped-did-form");
repo.is_public = false;
state.db.create_repo(&repo).await.expect("seed repo");
// Use the same slug-construction the writer uses
// (`db::normalize_owner_key`) so the test reader and the production
// writer can't drift if a future change moves the slug off the
// short-form. P3 (reviewer-3): the previous `split(':').next_back()`
// hand-rolled a third convention that agreed with the writer for
// `did:key` owners and diverged for every other method.
let short = crate::db::normalize_owner_key(&owner_did).to_string();
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!("{short}/scoped-did-form"),
owner_did: &owner_did,
ref_name: "refs/heads/main",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-did-form-tx",
arweave_url: "https://arweave.net/did-form-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
// P2 (reviewer-3): the previous suite only proved the authorized
// row is present. A correct filter must ALSO prove that anchors
// for OTHER repos do not leak into the response — the failure
// mode of the original bug was a cross-repo leak, not a
// fail-safe empty page. Seed a second repo (also under the
// same owner, so the authz test stays scoped) with its own
// irys_tx_id and assert it is absent from the body.
let other_owner = {
let kp2 = Keypair::generate();
kp2.did().to_string()
};
let mut other_repo = seed_repo_inline(&other_owner, "other-repo");
other_repo.is_public = true;
state
.db
.create_repo(&other_repo)
.await
.expect("seed other repo");
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!(
"{}/other-repo",
crate::db::normalize_owner_key(&other_owner)
),
owner_did: &other_owner,
ref_name: "refs/heads/main",
old_sha: "0".repeat(64).as_str(),
new_sha: "2".repeat(64).as_str(),
cid: Some("bafyother"),
irys_tx_id: "irys-other-repo-tx",
arweave_url: "https://arweave.net/other-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed other anchor");
// The caller passes the FULL DID form in ?repo=. validate_repo_slug
// accepts it, authorize_repo_read resolves it, and the handler MUST
// then translate to the canonical stored slug (short form) before
// running the SQL — otherwise this returns 200 with anchors: [].
let path = format!("/api/v1/arweave/anchors?repo={owner_did}/scoped-did-form");
let signed = sign_request(&kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"authz passed via the full DID form, so the response must be 200"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
body.contains("irys-did-form-tx"),
"the SQL filter must use the stored short slug, not the full DID — \
a false-empty page would break callers that pass ?repo=did:key:…/name. \
body was: {body}"
);
assert!(
!body.contains("irys-other-repo-tx"),
"the scoped anchor filter must exclude other repos' anchors — \
a cross-repo leak would expose metadata of repos the caller \
did not authorize. body was: {body}"
);
let _ = repo;
}
/// Signed non-reader on a private repo → 404, anchor metadata MUST NOT
/// leak. This is the test that catches the "auth-but-no-authz" bug if
/// the gate is moved after the SQL query or removed entirely. The 404
/// is the standard `repo_not_found` shape — indistinguishable from the
/// missing-repo case below.
#[sqlx::test]
async fn signed_scoped_anchors_non_reader_is_404_no_leak_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let owner_kp = Keypair::generate();
let stranger_kp = Keypair::generate();
let owner_did = owner_kp.did().to_string();
let stranger_did = stranger_kp.did().to_string();
let state = test_state(pool).await;
let mut repo = seed_repo_inline(&owner_did, "scoped-priv-nr");
repo.is_public = false;
state.db.create_repo(&repo).await.expect("seed repo");
let short_owner = owner_did.split(':').next_back().unwrap().to_string();
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!("{short_owner}/scoped-priv-nr"),
owner_did: &owner_did,
ref_name: "refs/heads/secret",
old_sha: "2".repeat(64).as_str(),
new_sha: "3".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-secret-tx-DO-NOT-LEAK",
arweave_url: "https://arweave.net/secret-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let _ = stranger_did; // signature carries the DID; this is just documentation
let path = format!("/api/v1/arweave/anchors?repo={short_owner}/scoped-priv-nr");
let signed = sign_request(&stranger_kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::NOT_FOUND,
"a signed non-reader must be 404 on a private repo's anchors"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("irys-secret-tx-DO-NOT-LEAK"),
"the 404 body must not leak the anchor's irys tx id; body was: {body}"
);
assert!(
!body.contains("refs/heads/secret"),
"the 404 body must not leak the ref name; body was: {body}"
);
let v: serde_json::Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v["error"], "repo_not_found",
"non-reader denial must carry the same error code as a missing repo"
);
}
/// Signed caller on a missing repo → 404, same shape as the non-reader
/// case. This is the indistinguishability half: a probe cannot tell
/// "private" from "absent" from the response.
#[sqlx::test]
async fn signed_scoped_anchors_missing_repo_is_404_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let state = test_state(pool).await;
let short = kp
.did()
.to_string()
.split(':')
.next_back()
.unwrap()
.to_string();
let path = format!("/api/v1/arweave/anchors?repo={short}/does-not-exist");
let signed = sign_request(&kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::NOT_FOUND,
"a missing repo must 404 indistinguishably from a non-readable private repo"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let v: serde_json::Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v["error"], "repo_not_found",
"the 404 body must carry the standard repo_not_found error code"
);
}
/// `?limit=-1` must not crash as `LIMIT -1` (Postgres 500). It clamps to
/// zero and returns 200 with an empty list — the same shape as a valid
/// listing that happens to have no rows in the configured range.
#[sqlx::test]
async fn signed_anchors_negative_limit_clamps_to_zero_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let state = test_state(pool).await;
let path = "/api/v1/arweave/anchors?limit=-1";
let signed = sign_request(&kp, "GET", path, b"");
let req = Request::builder()
.method("GET")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"?limit=-1 must clamp, not 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let v: serde_json::Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(v["count"], 0, "clamped limit yields an empty list");
assert_eq!(v["anchors"].as_array().map(|a| a.len()), Some(0));
}
}
File diff suppressed because it is too large. Load diff
+82 -2
View File
@@ -605,7 +605,7 @@ async fn replicate_encrypted_blobs(
continue;
}
};
match crate::ipfs_pin::pin_git_object(ipfs_api, &blob.oid, &envelope).await {
match crate::ipfs_pin::pin_git_object(ipfs_api, &blob.oid, &envelope, None).await {
Ok(cid) if !cid.is_empty() => {
if cid != blob.cid {
warn!(oid = %blob.oid, expected = %blob.cid, got = %cid, "replicated envelope CID mismatch; skipping record");
@@ -701,6 +701,7 @@ async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> an
if mode == MirrorMode::Promisor {
args.push("--filter=blob:limit=10g");
}
args.push("--");
args.push(remote_url);
args.push(local_str);
@@ -729,7 +730,10 @@ async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> an
async fn fetch_repo(local_path: &Path, remote_url: &str, mode: MirrorMode) -> anyhow::Result<()> {
let local_str = local_path.to_str().unwrap_or(".");
git_run(&["-C", local_str, "remote", "set-url", "origin", remote_url]).await?;
git_run(&[
"-C", local_str, "remote", "set-url", "origin", "--", remote_url,
])
.await?;
match mode {
MirrorMode::Promisor => {
@@ -1006,6 +1010,82 @@ mod tests {
assert_eq!(probe, PromisorProbe::NotPromisor);
}
/// #374: `clone_repo` interpolates the peer's origin URL into a `git
/// clone --mirror` argument list. Without a `--` delimiter git parses a
/// remote beginning with `-` as an option instead of a repository.
///
/// The delimiter is what forces the same string to be a repository, and
/// git says so in its own words: it reports the whole argument as a
/// repository it cannot find. Undelimited, git consumes it as an option
/// and fails against the *destination* instead (`fatal: repository '<dest>' does not exist`),
/// never naming the injected string — which is why this assertion is RED
/// before the fix and GREEN after.
///
/// The payload is deliberately free of `:` and spaces: a colon would make
/// git read the argument as an ssh-style `host:path` URL and report a
/// hostname rather than a repository, which is a different code path.
#[tokio::test]
async fn clone_never_parses_a_remote_as_a_git_option() {
let td = TempDir::new().unwrap();
let dest = td.path().join("mirror.git");
let injected = "--upload-pack=false";
let err = clone_repo(injected, &dest, MirrorMode::Plain)
.await
.expect_err("a remote that is not a repository must fail the clone")
.to_string();
assert!(
err.contains(injected),
"git must report the argument as a repository, not consume it as an option: {err}"
);
}
/// #374: `fetch_repo` interpolates the peer's origin URL into
/// `git remote set-url`, a second argv construction with the same contract as
/// the clone. It needs its own proof: the clone test cannot reach this path.
///
/// The assertion is on stored state rather than on the error string, and that
/// is deliberate. `git_run` formats its failure as `git {args:?} failed:
/// {stderr}`, so the injected value is in the message via the debug-printed
/// argv whether or not the delimiter is present — asserting on containment
/// would pass with `--` deleted. Delimited, `set-url` succeeds and stores the
/// value verbatim; undelimited, git exits 129 with `unknown option` and leaves
/// the previous URL in place, so this goes red without the fix.
///
/// The fetch that follows the rewrite fails, because the stored URL is not a
/// repository. That is expected and ignored: the property under test is what
/// `set-url` wrote, not whether the subsequent fetch could succeed.
#[tokio::test]
async fn fetch_never_parses_a_remote_as_a_git_option() {
let td = TempDir::new().unwrap();
let local = td.path().join("mirror.git");
let local_str = local.to_str().unwrap();
let placeholder = "http://example.invalid/x.git";
assert!(Command::new("git")
.args(["init", "-q", "--bare", local_str])
.status()
.unwrap()
.success());
assert!(Command::new("git")
.args(["-C", local_str, "remote", "add", "origin", placeholder])
.status()
.unwrap()
.success());
let injected = "--upload-pack=false";
let _ = fetch_repo(&local, injected, MirrorMode::Plain).await;
assert_eq!(
git_config_get(local_str, "remote.origin.url")
.await
.as_deref(),
Some(injected),
"set-url must store an option-shaped remote verbatim, not reject it as an option"
);
}
#[tokio::test]
async fn probe_reports_unknown_on_git_error() {
// A path git cannot resolve as a repo at all (exit 128) is an indeterminate
File diff suppressed because it is too large. Load diff
+25
View File
@@ -437,6 +437,31 @@ mod tests {
);
}
// #135 T1: a Mode-B rule on `/secret/**` must DENY the withheld directory's
// OWN path `/secret` (the `path == prefix` arm), not just strict descendants —
// otherwise get_by_cid's tree gate would serve the /secret tree object and leak
// its children. Pins parity with get_tree, which denies the /secret path.
#[test]
fn subtree_rule_denies_the_withheld_directory_itself() {
let reader = "did:key:z6MkReader";
let rules = [rule("/secret/**", VisibilityMode::B, &[reader])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/secret"),
Decision::Deny,
"anon denied at the withheld directory's OWN path /secret"
);
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/public"),
Decision::Allow,
"anon allowed at a sibling path outside the withheld subtree"
);
assert_eq!(
visibility_check(&rules, true, OWNER, Some(reader), "/secret"),
Decision::Allow,
"listed reader allowed at the withheld directory (caller-aware)"
);
}
// #153 regression: cross-method DID must still be denied even when the
// trailing segment collides with a bare owner key.
#[test]
+13
View File
@@ -79,6 +79,19 @@ async fn fire_event_async(
let delivery_id = uuid::Uuid::new_v4().to_string();
tokio::spawn(async move {
// Policy skip, not a delivery failure: the stored URL predates the
// create-time gate. The "invalid_url" label is documented on
// record_webhook_delivery in metrics.rs.
if !crate::api::peers::is_public_http_url(&hook.url) {
crate::metrics::record_webhook_delivery("invalid_url");
tracing::warn!(
url = %hook.url,
event = %event_name,
"webhook delivery skipped: stored URL is not a public http(s) target"
);
return;
}
let signature = hook.secret.as_deref().map(|s| sign_payload(s, &bytes));
let mut req = client
+574
View File
@@ -0,0 +1,574 @@
//! #174 U6 — INV-22 completeness guard (rung-raising).
//!
//! INV-22: a permit held per op recovers only if every path that holds it is also
//! duration-bounded and reaps the process group before releasing admission, and every
//! detached git/blocking task carries its own admission. PR #174 fixed five paths
//! (U1-U5) that violated this. Each fix has a per-unit RED/GREEN regression; together
//! those form the five-revert matrix. This guard adds the missing piece: a source-scan
//! tripwire that fails when a NEW site reintroduces the class, or when one of the five
//! gates is removed.
//!
//! It lives in `tests/` (a separate crate) on purpose: a guard that scanned the same
//! file it lives in would match its own identifier literals and pass vacuously. Here
//! the scanned `src/` files never contain this file's literals, so each check is
//! load-bearing — reverting the named gate turns the assertion red.
//!
//! These are deliberately coarse structural checks, not a parser. They cannot prove a
//! gate is *correct* (the per-unit tests do that); they prove a gate is *present and
//! not bypassed*, which is what stops the class from silently regressing.
use std::path::Path;
fn src(rel: &str) -> String {
let p = Path::new(env!("CARGO_MANIFEST_DIR")).join("src").join(rel);
std::fs::read_to_string(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()))
}
#[test]
fn inv22_concurrency_gates_present_and_not_bypassed() {
let repos = src("api/repos.rs");
let smart_http = src("git/smart_http.rs");
let vis = src("git/visibility_pack.rs");
let ipfs = src("api/ipfs.rs");
// U1 / P1-a: run_bounded_git stands the watchdog down only after confirming the
// child actually terminated (WNOWAIT), not on the raw stdout-drain EOF — otherwise
// a child that closes stdout then hangs pins the permit past the deadline. The
// probe is defined and called, so >= 2 occurrences; reverting the fix removes both.
assert!(
vis.matches("child_terminated_without_reaping").count() >= 2,
"U1/P1-a gate missing: run_bounded_git must confirm child exit via \
child_terminated_without_reaping before signalling the watchdog"
);
// U2 / P1-c: on client disconnect KillGroupOnDrop must launch a detached reaper
// that runs the full TERM/grace/KILL/reap, not a lone SIGTERM. The reaper is spawned
// via a runtime handle in Drop; `Handle::try_current` is unique to that launch (the
// timeout path already has an async context and never calls it).
assert!(
smart_http.contains("Handle::try_current"),
"U2/P1-c gate missing: KillGroupOnDrop::drop must launch the full \
TERM/grace/KILL reaper on disconnect (Handle::try_current), not a lone SIGTERM"
);
// U4 / P1-d: git_receive_pack must acquire the per-source write sub-cap before the
// global write permit. The acquire reads `state.git_write_per_caller`; comments name
// the field without the `state.` prefix, so this targets the real acquire site.
assert!(
repos.contains("state.git_write_per_caller"),
"U4/P1-d gate missing: git_receive_pack must acquire the per-source write cap \
(state.git_write_per_caller) before the global write permit"
);
// U5 / P1-e: the detached post-push encryption walk must run through the
// admission-gated helper, which is wired to the shared encrypt pool.
assert!(
repos.contains("fn withheld_recipients_gated")
&& repos.contains("state.git_encrypt_semaphore"),
"U5/P1-e gate missing: the encryption walk must run through \
withheld_recipients_gated, which acquires git_encrypt_semaphore"
);
// U1 / R2 (#173 round-10): the path-scoped filtered-pack serve must thread the
// caller's AdmissionGuard through BOTH git stages so read + per-caller admission is
// held until the pack-objects group is reaped on disconnect, closing the cap bypass
// the plain upload_pack path already fixed. Two load-bearing markers: rev-list hands
// the disarmed guard back (its tuple return type), and build_filtered_pack forwards
// that guard into the pack-objects stage (the `admission` arg after the
// "pack-objects" label). Reverting either — dropping the guard between stages, or
// passing `None` to pack-objects — trips this.
assert!(
smart_http.contains("Result<(Vec<String>, Option<AdmissionGuard>)>")
&& smart_http.contains("\"pack-objects\",\n admission,"),
"U1/R2 gate missing: build_filtered_pack must thread the AdmissionGuard through \
rev-list -> pack-objects so the permits are held until the pack-objects group \
is reaped on disconnect (the path-scoped half of #174 P1-a)"
);
// U2 / R1: a cancelled or timed-out `GET /ipfs/{cid}` must release admission only
// after the blocking work it admitted has finished, not the instant the handler
// future drops (the /ipfs half of #174 P1-a). The mechanism is the shared
// `Arc<WalkAdmission>`: both walk permits are moved into it once per request and a
// clone rides every `spawn_blocking`, so the permits release when the LAST holder
// drops. Reverting to handler-local permits trips this; the per-site clones are
// bound separately by `inv22_ipfs_walk_admission_reaches_every_blocking_site`.
//
// This previously required the permits to be owned by a detached `tokio::spawn`
// running the whole pipeline. That closed the same bypass but kept an abandoned
// request's full legacy scan running against a held slot, so the merge of #173 and
// #174 kept the Arc and dropped the detached task.
assert!(
ipfs.contains("struct WalkAdmission")
&& ipfs.contains("let admission = std::sync::Arc::new(WalkAdmission {"),
"U2/R1 gate missing: get_by_cid must move both /ipfs admission permits into a \
shared Arc<WalkAdmission> whose clones ride the blocking work, so admission is \
released only once that work completes (the /ipfs half of #174 P1-a)"
);
// U2 / KTD2 (#173 round-10): the probe/read children on the /ipfs path must be the
// duration-bounded twins (process-group teardown via run_bounded_git), not the bare
// `store::object_type` / `read_object_content` (or an unbounded `cat-file -s`) a tokio
// timeout cannot cancel — otherwise a wedged cat-file lingers and pins the held
// admission past the deadline. Reverting any twin call site back to a bare read trips
// this.
assert!(
ipfs.contains("object_type_bounded(")
&& ipfs.contains("object_size_bounded(")
&& ipfs.contains("read_object_content_bounded("),
"U2/KTD2 gate missing: the /ipfs probe+read must call the run_bounded_git-backed \
*_bounded twins so a wedged cat-file is reaped at the deadline, not left to pin \
the held /ipfs walk admission"
);
// P1-e non-bypass tripwire: the bounded recipients walk is spawn_blocking'd nowhere
// but inside withheld_recipients_gated. A second call site (count > 1) is a new
// detached git walk that skips the admission gate — exactly the class U5 closed.
assert_eq!(
repos.matches("withheld_blob_recipients_bounded").count(),
1,
"P1-e bypass: the bounded recipients walk must be invoked only inside \
withheld_recipients_gated; a new call site bypasses the encrypt-walk admission cap"
);
// U4 / P2-2: the detached post-push encryption task must be gated by the per-repo
// coalescing set (`encrypt_inflight.try_begin`) so the OUTSTANDING parked-task set is
// bounded to <=1 per repo. Removing the gate lets N rapid pushes spawn N parked
// waiters (the unbounded set U4 closed); the semaphore only caps active walks.
assert!(
repos.contains("state.encrypt_inflight.try_begin"),
"U4/P2-2 gate missing: the detached post-push encryption spawn must consult \
encrypt_inflight.try_begin to coalesce per repo (bound the outstanding-task set)"
);
// F5: coalescing must REQUEUE, not shed. The in-flight task pins only its own
// pre-spawn snapshot, so a coalesced push's tip pairs are recorded on its key and
// the task must loop-drain them (`finish_or_take_pending`) before releasing it.
// Removing the drain reverts to the silent loss: a coalesced push's pins and
// recovery copies are absent until an unrelated later push. Scan only the
// production half of the file — the u5 tests in its `mod tests` also name the
// drain call, and matching them would make this check vacuous.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
repos_production.contains("finish_or_take_pending"),
"F5 gate missing: the post-push encryption task must loop-drain coalesced \
pushes via finish_or_take_pending before releasing its repo key"
);
// F4: every post-receive scan helper admits itself to the shared scan pool via
// `crate::state::acquire_scan_permit` BEFORE its spawn_blocking git walk, so a
// push burst cannot accumulate unbounded concurrent scans once the write permit
// is released. Two halves, both load-bearing: the helper body must actually
// acquire the pool (state.rs sits the helper at the end of the file, so the
// definition tail contains no other `acquire_owned` to match vacuously), and
// within each scan helper the first qualified call precedes the first
// `spawn_blocking`. Severing a call site pushes the next occurrence past the
// helper's own `spawn_blocking` (or off the end of the file), turning the
// assertion red; comments name the helper without the `crate::state::` prefix,
// so this targets the real call sites.
let state_rs = src("state.rs");
let helper_def = state_rs
.find("fn acquire_scan_permit")
.expect("F4 gate missing: state.rs no longer defines acquire_scan_permit");
assert!(
state_rs[helper_def..].contains("acquire_owned"),
"F4 gate gutted: acquire_scan_permit must acquire the scan pool via acquire_owned"
);
let push_delta = src("git/push_delta.rs");
for (file_src, file, helper) in [
(&repos, "api/repos.rs", "fn replication_withheld_set"),
(&repos, "api/repos.rs", "fn fail_closed_full_scan_objects"),
(
&push_delta,
"git/push_delta.rs",
"fn resolve_candidates_for_push",
),
] {
let start = file_src
.find(helper)
.unwrap_or_else(|| panic!("{file}: `{helper}` not found"));
let tail = &file_src[start..];
let acquire = tail
.find("crate::state::acquire_scan_permit(")
.unwrap_or_else(|| {
panic!("F4 gate missing: {file} `{helper}` no longer acquires a scan permit")
});
let spawn = tail.find("spawn_blocking").unwrap_or_else(|| {
panic!("{file}: `{helper}` lost its spawn_blocking walk — update this guard")
});
assert!(
acquire < spawn,
"F4 gate bypassed: {file} `{helper}` must acquire its git_encrypt_semaphore \
permit BEFORE dispatching the blocking git scan"
);
}
}
/// F4 (repo_store advisory-unlock cancellation safety): `RepoWriteGuard::release`
/// must await `pg_advisory_unlock` while `self` still owns the pooled connection,
/// and must not mark itself `released` until that await resolves. Either shape,
/// reintroduced, re-opens the mid-unlock cancellation leak: taking the connection
/// early leaves `Drop` with `conn == None`, and setting `released = true` early
/// leaves the `Drop` backstop inert — both strand the session lock on cancellation.
///
/// Scoped to the `release` fn body: the `Drop` impl legitimately takes the
/// connection and unlocks, so a whole-file scan would match it and read as a false
/// pass. Reverting either ordering turns this red (proven load-bearing).
#[test]
fn f4_release_keeps_conn_owned_until_unlock_resolves() {
let repo_store = src("git/repo_store.rs");
let rel_start = repo_store
.find("pub async fn release(self")
.or_else(|| repo_store.find("pub async fn release(mut self"))
.expect("F4 gate: repo_store.rs no longer defines RepoWriteGuard::release");
let rel_end = repo_store[rel_start..]
.find("impl Drop for RepoWriteGuard")
.map(|off| rel_start + off)
.expect("F4 gate: release fn / Drop impl markers moved — update this guard");
let release_body = &repo_store[rel_start..rel_end];
let unlock = release_body
.find("pg_advisory_unlock")
.expect("F4 gate: release must still issue pg_advisory_unlock");
let before_unlock = &release_body[..unlock];
// (a) the connection must still be owned by `self` at the unlock await.
assert!(
!before_unlock.contains("self.conn.take()"),
"F4 regression: RepoWriteGuard::release takes self.conn BEFORE awaiting \
pg_advisory_unlock. A cancellation during the unlock await then strands the \
session advisory lock (Drop sees conn == None and skips its backstop). \
Unlock through the still-owned connection instead."
);
// (b) `released` must not be set before the unlock await — the other
// reintroduction shape a single-reorder check on (a) alone is blind to.
assert!(
!before_unlock.contains("released = true"),
"F4 regression: RepoWriteGuard::release sets `released = true` BEFORE awaiting \
pg_advisory_unlock. A cancellation during the await then leaves the Drop \
backstop inert (it early-returns on released). Set released only AFTER the \
unlock await resolves."
);
}
/// F6/KTD-5 (IPFS metadata queries deadline-wrapped): `get_by_cid` acquires
/// the scarce walk permits (RAII, held for the whole request) BEFORE its metadata
/// queries, and the per-repo loop's first budget gate runs only later. So
/// both `list_repos_page_for_scan` and `list_visibility_rules_for_repos` must be clamped to the
/// remaining request budget — otherwise a query blocked in Postgres pins the walk slot
/// for the whole stall, past the budget. This scans the PRODUCTION half of `api/ipfs.rs`
/// (the `mod tests` half names the same calls in its own harness and would make the
/// check vacuous) and asserts each query call is immediately preceded by a
/// `tokio::time::timeout(` wrapper. Removing either wrapper turns this red (proven
/// load-bearing).
#[test]
fn f6_ipfs_metadata_queries_are_deadline_wrapped() {
let ipfs = src("api/ipfs.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`: the handler carries
// in-line `#[cfg(test)]` query counters, so splitting on the attribute cut the
// production half off above the calls this guard exists to check and the scan went
// vacuously quiet (found 0 of each rather than failing on a missing wrapper).
let production = ipfs
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// EVERY occurrence must be wrapped, not just the first. The handler now reaches
// these queries from two places (the provenance fast path, one repo at a time, and
// the legacy scan's preload), and an exact-count check would have to be relaxed
// every time a call site is added, which is how a guard quietly stops covering the
// site that matters. Requiring all of them scales with the code instead.
for call in [
".list_repos_page_for_scan(",
".list_visibility_rules_for_repos(",
".get_repo_by_id(",
".is_repo_quarantined(",
] {
let occurrences = production.matches(call).count();
assert!(
occurrences >= 1,
"F6 guard stale: api/ipfs.rs no longer calls `{call}` — update this guard"
);
for (n, (idx, _)) in production.match_indices(call).enumerate() {
// The wrapper opens a few lines above the call (match tokio::time::timeout(
// ... remaining budget ..., <query> )); a 240-char lookback covers it
// without reaching the previous statement.
let window = &production[idx.saturating_sub(240)..idx];
assert!(
window.contains("tokio::time::timeout("),
"F6 gate missing: occurrence {n} of `{call}` (of {occurrences}) is not \
wrapped in tokio::time::timeout(...) clamped to the remaining request \
budget. An unwrapped await pins the held walk permit for the whole DB \
stall, past GITLAWB_IPFS_REQUEST_BUDGET_SECS."
);
}
}
}
/// #174 F2 / KTD-3: the detached post-receive Pinata replication task must enqueue
/// only the push's `(ref, old, new)` tuples and RE-DERIVE its object set inside the
/// worker once a pin slot frees — it must NOT move the full per-push object list into
/// the closure and hold it across the `pin_semaphore` acquire. Retaining the list makes
/// every parked task (under a slow Pinata backend) hold an MB-scale OID list, so
/// outstanding memory grows O(pushes x object-list). Coalescing/shedding the task is
/// forbidden (its per-ref effects are non-idempotent), so the fix bounds the retained
/// data, not the task count.
///
/// Two load-bearing checks, both against the PRODUCTION half of `api/repos.rs` (the
/// `mod tests` half names the same identifiers in its own harness and would make the
/// scan vacuous): (a) the closure-local `object_list_pinata` binding — the retain form —
/// must be gone; reintroducing `let object_list_pinata = object_list;` turns this red.
/// (b) the re-derivation (`pinata_object_list_for_refs`) must appear AFTER the Pinata
/// pin permit is acquired, so the object list is materialized only inside the pin-bounded
/// section and a parked task holds O(ref tuples).
#[test]
fn f2_pinata_enqueues_refs_not_retained_object_lists() {
let repos = src("api/repos.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (a) the retained-list form must be gone from production.
assert!(
!production.contains("object_list_pinata"),
"F2/KTD-3 regression: the Pinata task retains a full per-push object list \
(`object_list_pinata`) across the pin-permit acquire. Enqueue only the ref \
tuples and re-derive the object set inside the worker (pinata_object_list_for_refs)."
);
// (b) the re-derivation runs AFTER the pin permit is acquired. Anchor on the
// Pinata pin-admission clone so the window is the Pinata task, not the sibling
// IPFS/encrypt spawn (which shares `pin_semaphore` but never re-derives).
let anchor = production
.find("let pin_sem_pinata")
.expect("F2 gate stale: the Pinata pin-admission clone (pin_sem_pinata) moved");
let tail = &production[anchor..];
let acquire = tail
.find(".acquire_owned()")
.expect("F2 gate: the Pinata task no longer acquires a pin permit (acquire_owned)");
let rederive = tail.find("pinata_object_list_for_refs(").expect(
"F2 gate missing: the Pinata task must re-derive its object set via \
pinata_object_list_for_refs; it can no longer move a pre-resolved list into the closure",
);
assert!(
acquire < rederive,
"F2 gate bypassed: the Pinata object-set re-derivation must run AFTER the pin \
permit is acquired, so a parked task never holds the MB-scale object list"
);
// The re-derivation is driven by the push's ref tuples (the enqueued unit), not a
// retained object list — tie "enqueue ref_updates" to the call explicitly.
assert!(
tail[rederive..].starts_with("pinata_object_list_for_refs(")
&& tail[rederive..]
.get(..400)
.map(|w| w.contains("ref_updates_clone"))
.unwrap_or(false),
"F2 gate: pinata_object_list_for_refs must re-derive from the push's ref tuples \
(ref_updates_clone), the small unit the parked task retains"
);
}
/// #174 U2 / F3 (second same-repo writer serialized until a disconnected first push's
/// git group is reaped): `git_receive_pack` must take the per-repo in-process write
/// lease and CARRY it on the write-path `AdmissionGuard` (via `.with_lease`), so the
/// lease rides `KillGroupOnDrop`'s detached reaper and frees only after the group is
/// reaped — supplementing the pg advisory lock, which `RepoWriteGuard::Drop` releases at
/// the disconnect instant. Tying the lease to `RepoWriteGuard` instead (or dropping the
/// `.with_lease` carry) reopens the race; the behavioral RED/GREEN test
/// (`f3_second_push_serialized_until_disconnected_group_reaped`) is the real bar, this
/// is the completeness tripwire. Scanned against the PRODUCTION half of `api/repos.rs`
/// (the `mod tests` half names these identifiers in its own harness).
#[test]
fn f3_second_writer_leased_until_reap() {
let repos = src("api/repos.rs");
let smart_http = src("git/smart_http.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// The lease is acquired, then carried by the AdmissionGuard (.with_lease), before
// receive_pack runs the write. Severing any of the three turns this red.
let lease_acquire = repos_production.find("repo_write_leases").expect(
"F3 gate missing: git_receive_pack no longer takes the per-repo write lease \
(state.repo_write_leases)",
);
let with_lease = repos_production.find(".with_lease(").expect(
"F3 gate missing: the write-path AdmissionGuard no longer carries the lease \
(.with_lease). The lease must ride the disconnect reaper via the AdmissionGuard, \
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
);
let receive = repos_production
.find("smart_http::receive_pack(")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack");
assert!(
lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \
AdmissionGuard (.with_lease), BEFORE receive_pack runs the write"
);
// The AdmissionGuard must actually hold the lease (Option<RepoWriteLease>) via a
// with_lease setter, so it travels into the detached reaper. Removing the field or
// method turns this red.
assert!(
smart_http.contains("_lease: Option<crate::state::RepoWriteLease>")
&& smart_http.contains("pub fn with_lease("),
"F3 gate missing: AdmissionGuard must hold an Option<RepoWriteLease> set via \
with_lease, so the lease rides the guard into KillGroupOnDrop's reaper"
);
}
/// #174 U1 — every blocking walk in the `/ipfs` scan carries the request's walk
/// admission.
///
/// The admission is an `Arc<WalkAdmission>` cloned into each `spawn_blocking`
/// closure, so the global + per-source permits release only when the last holder
/// drops: a client disconnect leaves the abandoned closure holding the slot, and a
/// panicking closure leaves the handler holding it.
///
/// Only the walk site runs under `state.git_bin`, so only it can be pinned by the
/// fake-git harness and mutation-verified dynamically (see
/// `get_by_cid_walk_permit_held_through_blocking_walk`). The probe and the content
/// read deliberately shell to the real `git`, which is why this structural check
/// exists: it binds all three sites, and any blocking site added to this loop
/// later, without reversing that deliberate independence.
///
/// MUTATION (RED): delete any one `Arc::clone(ctx.admission)` binding, or drop the
/// clone from inside its closure, and the count falls below three.
#[test]
fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
let ipfs = src("api/ipfs.rs");
// The shared owner must exist and be built once per request.
assert!(
ipfs.contains("struct WalkAdmission") && ipfs.contains("Arc::new(WalkAdmission {"),
"U1 gate missing: the /ipfs walk admission must be a shared WalkAdmission, \
not a handler-local permit pair"
);
// Every blocking site in the scan takes its own clone...
// `Arc::clone(ctx.admission)` in the gate, `Arc::clone(&admission)` if a site is
// ever added back in the handler body itself; count both spellings.
let clones = ipfs.matches("Arc::clone(ctx.admission)").count()
+ ipfs.matches("Arc::clone(&admission)").count();
assert!(
clones >= 3,
"U1 gate bypassed: expected an admission clone for each of the three \
/ipfs spawn_blocking sites (probe, walk, read); found {clones}. A blocking \
walk that does not hold the admission lets a disconnect or a panic free the \
slot while its git child is still running."
);
// ...and each clone is actually moved INTO the blocking closure, not merely
// created in the async frame (which would hold nothing across the join).
let held = ipfs.matches("let _admission = ").count();
assert!(
held >= 3,
"U1 gate bypassed: each admission clone must be bound inside its \
spawn_blocking closure so the blocking work owns it; found {held} of 3."
);
// The count of blocking sites is itself the thing being covered: if a fourth
// appears, it needs an admission clone too and this gate must be revisited.
let sites = ipfs.matches("spawn_blocking(move ||").count();
assert_eq!(
sites, 3,
"the /ipfs scan grew or lost a spawn_blocking site ({sites} found, expected 3); \
give any new blocking walk its own Arc::clone(&admission) and update this gate"
);
}
/// #174 U5: the post-receive replication tail is spawned at the DURABILITY BOUNDARY,
/// which is the moment receive-pack returns success, not the end of the handler and
/// not after `guard.release()`.
///
/// The tail owes this push its pins, recovery copy, and announcements. Everything
/// below the spawn stays in the cancellable request future, so anything the tail is
/// spawned after is a window where a client disconnect drops that work while the pack
/// is already durable on disk. `guard.release()` is such a window: on success it
/// awaits the Tigris upload and then the advisory unlock.
///
/// The lower bound matters just as much as the upper one: `release` runs on failure
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so
/// the two cannot drift apart.
///
/// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
/// drives the actual disconnect through a parked `release`. Same instrument the F3
/// gate above uses.
///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
/// `guard.release(` and the ordering assertion fails; take it out of the
/// `if push_succeeded` block and the failed-push assertion fails.
#[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs");
// Production half only — the tests below name these identifiers too.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
let success_flag = production
.find("let push_succeeded = receive_result.is_ok();")
.expect(
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()",
);
let gate_open = production
.find("if push_succeeded {")
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded");
let spawn = production
.find("tokio::spawn(post_receive_replication_tail(")
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production
.find(".release(push_succeeded)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate");
let touch = production
.find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
let webhook = production
.find("webhooks::fire_event(")
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks");
assert!(
success_flag < gate_open && gate_open < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
rejected push now spawns a tail"
);
assert!(
spawn < release && spawn < touch && spawn < webhook,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements"
);
}
@@ -0,0 +1,177 @@
//! #26 Split PR 4 — `GITLAWB_ENFORCE_OWNER_PUSH` default is consistent across surfaces.
//!
//! Reviewer 2 (2026-08-28) closed PR #224 with a P2 finding: SECURITY.md and
//! README.md both stated the default was `false`, while `Config::enforce_owner_push`
//! declared `default_value_t = true` and the runtime, .env.example, and the env
//! var table on the README all agreed with the code. Two operator-facing
//! surfaces lied about the authorization policy. PR #330 flipped the code
//! default and updated .env.example / README env table / docs/RUN-A-NODE.md,
//! but missed these two prose mentions.
//!
//! This guard pins the documentation and example surfaces together. Each one
//! is read off the working tree at test time, not a copy in the binary, so a
//! revert of the doc edit turns it red:
//!
//! 1. `SECURITY.md` says the default is `true`.
//! 2. `README.md` says the default is `true` in the known-limitations prose.
//! 3. `README.md` says the default is `true` in the env-var table row.
//! 4. `.env.example` sets `GITLAWB_ENFORCE_OWNER_PUSH=true`.
//!
//! The runtime declared default (`Config::enforce_owner_push`'s
//! `default_value_t = true`) is protected by the
//! `enforce_owner_push_is_declared_true_independent_of_the_environment`
//! unit test in `config.rs`, which reads the parser directly and is
//! env-independent. The two checks together cover the surfaces that
//! produced the original doc-vs-code drift; this guard cannot reach into
//! the binary crate's `Config` type (it is private to the bin target,
//! which has no `lib.rs`) without a larger refactor, so the contract is
//! stated as the four documentation/example surfaces above.
//!
//! The four are checked independently. A check that just verified "at least
//! one surface says true" would pass while a stale SECURITY.md lied; a check
//! that only verified the binary would not catch the doc drift that
//! produced this finding in the first place.
use std::path::Path;
fn read(rel: &str) -> String {
let p = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join(rel);
std::fs::read_to_string(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()))
}
/// Pin the prose mentions of `GITLAWB_ENFORCE_OWNER_PUSH` so a future doc
/// edit cannot reintroduce the "defaults to false" wording the reviewer
/// flagged. The wording in the test is what the docs MUST say about the
/// default; absence of any prose mention is also a fail because operators
/// rely on the docs to learn the default.
#[test]
fn security_md_states_owner_push_default_is_true() {
let security = read("SECURITY.md");
let section = section_after(&security, "### Repository write authorization defaults")
.unwrap_or_else(|| panic!("SECURITY.md is missing the owner-push section"));
assert!(
section.contains("defaults to `true`"),
"SECURITY.md owner-push section must state the default is true. \
Found:\n{section}"
);
assert!(
!section_claims_default_is_false(&section),
"SECURITY.md owner-push section must not claim the default is false. \
Found:\n{section}"
);
}
#[test]
fn readme_known_limitations_states_owner_push_default_is_true() {
let readme = read("README.md");
let limitations = section_after(&readme, "Known limitations:")
.unwrap_or_else(|| panic!("README.md is missing the Known limitations: section"));
// The owner-push line is one bullet in the limitations list. Pull the
// single line that mentions the env var so the assertion is local to
// the claim, not a substring of unrelated text.
let line = limitations
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| {
panic!(
"README.md Known limitations must mention GITLAWB_ENFORCE_OWNER_PUSH.\n\
Got:\n{limitations}"
)
});
assert!(
line.contains("defaults to `true`") || line.contains("defaults to true"),
"README Known-limitations owner-push bullet must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README Known-limitations owner-push bullet must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn readme_env_var_table_states_owner_push_default_is_true() {
let readme = read("README.md");
let line = readme
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH") && l.contains("|"))
.unwrap_or_else(|| {
panic!("README.md is missing the env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH")
});
assert!(
line.contains("Defaults to `true`") || line.contains("defaults to `true`"),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn env_example_sets_owner_push_true() {
let env = read(".env.example");
let line = env
.lines()
.find(|l| l.trim_start().starts_with("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| panic!(".env.example is missing GITLAWB_ENFORCE_OWNER_PUSH"));
assert!(
line.contains("=true"),
".env.example GITLAWB_ENFORCE_OWNER_PUSH must be set to true. Found:\n{line}"
);
}
/// A claim that the default is false can hide in many phrasings. Match the
/// forms that have appeared in the wild on this branch: "defaults to `false`",
/// "defaults to false", "default is false", "is `false` for compatibility",
/// "is false for compatibility". The match is intentionally narrow — false
/// positives would suppress a real doc edit.
fn section_claims_default_is_false(section: &str) -> bool {
let lower = section.to_ascii_lowercase();
lower.contains("defaults to `false`")
|| lower.contains("defaults to false")
|| lower.contains("default is `false`")
|| lower.contains("default is false")
|| (lower.contains("`false`") && lower.contains("for compatibility"))
}
/// Return the prose after a marker up to the next heading boundary. The
/// marker is normally a markdown heading (e.g. `### Section`); if it has
/// no `#` prefix it is treated as a plain-text marker and the slice is
/// bounded by the next heading of any level. This matters because a
/// plain-text marker would otherwise produce `header_level = 0`, making
/// the same-or-higher-level stop condition unsatisfiable and letting the
/// slice run to EOF.
fn section_after(doc: &str, heading: &str) -> Option<String> {
let start = doc.find(heading)?;
let after = &doc[start..];
let header_level = heading.chars().take_while(|c| *c == '#').count();
// Skip past the heading line itself.
let body_start = after.find('\n')? + 1;
let body = &after[body_start..];
// Stop at the next heading of the same or higher level, or at the next
// heading of any level when the marker had no `#` prefix.
let mut end = body.len();
for line in body.lines() {
let level = line.chars().take_while(|c| *c == '#').count();
if level > 0 && (header_level == 0 || level <= header_level) {
end = line.as_ptr() as usize - body.as_ptr() as usize;
break;
}
}
Some(body[..end].to_string())
}
@@ -0,0 +1,637 @@
//! Which p2p key-storage failures stop the node, and which leave HTTP up.
//!
//! The two domains are decided by different code at different phases, so the
//! only honest proof is at the process boundary: spawn the real binary and
//! watch what it does. A helper's returned error says nothing about whether
//! `main` treated it as fatal.
//!
//! Every row also asserts the key tree is byte-identical afterwards. A refusal
//! that mutates storage on its way out is its own defect, and a returned error
//! cannot show that.
//!
//! No database: `DATABASE_URL` points at a closed port on purpose, so a row
//! that reaches the degraded server proves the outcome is observable without
//! Postgres. That is what makes the port-zero no-I/O guarantee testable at all.
#![cfg(unix)]
use std::io::{BufRead, BufReader, Read, Write};
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
use std::process::{Child, Command, Stdio};
use std::time::{Duration, Instant};
/// Wall clock, never an iteration count: a fixed loop count starves on a slow
/// or loaded machine and turns a real pass into a flake.
const DEADLINE: Duration = Duration::from_secs(15);
/// Kills the child on every exit path, including a panicking assertion.
///
/// Without this a failing row leaves a node behind that retries its database
/// connection forever, holding the row's tempdir open. The first run of this
/// file did exactly that, and the leak is invisible until something else in
/// the suite gets slow.
struct ChildGuard(Child);
impl ChildGuard {
/// The tracing subscriber writes to STDOUT; only anyhow's final error
/// print goes to stderr. A row that watches one stream sees half the
/// story, so both are read.
fn take_stdout(&mut self) -> std::process::ChildStdout {
self.0.stdout.take().expect("stdout piped")
}
fn take_stderr(&mut self) -> std::process::ChildStderr {
self.0.stderr.take().expect("stderr piped")
}
}
impl Drop for ChildGuard {
fn drop(&mut self) {
let _ = self.0.kill();
let _ = self.0.wait();
}
}
/// `(path, is_dir, is_symlink, mode, len)` for everything under `root`, sorted.
/// Taken before and after each row so a refusal that creates, chmods, or
/// deletes anything fails the row.
fn snapshot(root: &Path) -> Vec<String> {
fn walk(dir: &Path, out: &mut Vec<String>) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
for e in entries.flatten() {
let p = e.path();
let Ok(md) = std::fs::symlink_metadata(&p) else {
continue;
};
out.push(format!(
"{} dir={} link={} mode={:04o} len={}",
p.display(),
md.is_dir(),
md.is_symlink(),
md.permissions().mode() & 0o7777,
if md.is_file() { md.len() } else { 0 }
));
if md.is_dir() && !md.is_symlink() {
walk(&p, out);
}
}
}
let mut out = Vec::new();
walk(root, &mut out);
out.sort();
out
}
struct Row {
/// Sandbox the row owns: HOME, repos, cwd and the key tree all live here.
home: tempfile::TempDir,
}
impl Row {
/// The only directory the no-mutation assertions watch.
fn tree(&self) -> PathBuf {
self.home.path().join("keytree")
}
}
impl Row {
fn new() -> Row {
let home = tempfile::tempdir().expect("tempdir");
std::fs::set_permissions(home.path(), std::fs::Permissions::from_mode(0o700))
.expect("chmod home");
// Created here, before any row takes its `before` snapshot, so the
// no-mutation assertions measure what the NODE did and not what this
// harness set up.
std::fs::create_dir_all(home.path().join("repos")).expect("repos dir");
// The p2p key tree lives in its own subdirectory so the no-mutation
// assertions can watch exactly it. HOME also holds `.gitlawb`, which
// the node creates for its NODE identity on every start; measuring the
// whole home would read that legitimate write as a p2p storage
// mutation.
let tree = home.path().join("keytree");
std::fs::create_dir(&tree).expect("key tree");
std::fs::set_permissions(&tree, std::fs::Permissions::from_mode(0o700))
.expect("chmod key tree");
Row { home }
}
fn spawn(&self, p2p_key: &str, p2p_port: &str, cwd: &Path) -> ChildGuard {
self.spawn_env(p2p_key, p2p_port, cwd, &[])
}
fn spawn_env(
&self,
p2p_key: &str,
p2p_port: &str,
cwd: &Path,
extra: &[(&str, &str)],
) -> ChildGuard {
let repos = self.home.path().join("repos");
let mut cmd = Command::new(env!("CARGO_BIN_EXE_gitlawb-node"));
cmd.env_clear()
.env("PATH", std::env::var("PATH").unwrap_or_default())
// The subscriber is built from the default env filter, so with a
// cleared environment the node logs nothing at all and every
// assertion here reads as a timeout with empty output. The rows
// are about which log lines appear, so the filter is part of the
// fixture, not incidental setup.
.env("RUST_LOG", "info")
.env("HOME", self.home.path())
.env("GITLAWB_REPOS_DIR", &repos)
.env("GITLAWB_HOST", "127.0.0.1")
// Ephemeral: the real port is read back from the ready log.
.env("GITLAWB_PORT", "0")
// A closed port, so the node stays on the degraded path for the
// whole row instead of ever reaching a database.
.env("DATABASE_URL", "postgres://127.0.0.1:1/nonexistent")
.env("GITLAWB_P2P_PORT", p2p_port)
.env("GITLAWB_P2P_KEY", p2p_key)
.env("GITLAWB_METRICS_ADDR", "")
.current_dir(cwd)
.stdout(Stdio::piped())
.stderr(Stdio::piped());
for (k, v) in extra {
cmd.env(k, v);
}
cmd.spawn().map(ChildGuard).expect("spawn gitlawb-node")
}
}
/// Read the child's stderr until `needle` appears, the stream ends, or the
/// deadline passes. Returns everything read.
///
/// The read runs on its own thread feeding a channel, and the deadline is
/// enforced with `recv_timeout`. Checking elapsed time between blocking
/// `read_line` calls does NOT work: a child that goes quiet blocks the read
/// forever and the check never runs, so the deadline looks present and cannot
/// fire. The first version of this file did that and hung for minutes on a row
/// that was supposed to fail in fifteen seconds.
fn read_until(child: &mut ChildGuard, needle: &str) -> (bool, String) {
read_until_all(child, &[needle])
}
/// Read until EVERY needle has appeared, the stream ends, or the deadline
/// passes.
///
/// Waiting on a single line is a race here: the degraded server logs "ready"
/// from a spawned task while the p2p gate logs its verdict from the main task,
/// so whichever the reader stops at first can leave the other uncaptured. That
/// made this file pass or fail depending on scheduling.
fn read_until_all(child: &mut ChildGuard, needles: &[&str]) -> (bool, String) {
let stdout = child.take_stdout();
let (tx, rx) = std::sync::mpsc::channel::<String>();
std::thread::spawn(move || {
let mut reader = BufReader::new(stdout);
loop {
let mut line = String::new();
match reader.read_line(&mut line) {
Ok(0) | Err(_) => break,
Ok(_) => {
if tx.send(line).is_err() {
break;
}
}
}
}
});
let mut acc = String::new();
let start = Instant::now();
loop {
if needles.iter().all(|n| acc.contains(n)) {
return (true, acc);
}
let left = match DEADLINE.checked_sub(start.elapsed()) {
Some(d) if !d.is_zero() => d,
_ => return (false, acc),
};
match rx.recv_timeout(left) {
Ok(line) => acc.push_str(&line),
// Sender gone: the stream ended. One last check, then give up.
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => {
return (needles.iter().all(|n| acc.contains(n)), acc)
}
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => return (false, acc),
}
}
}
fn wait_for_exit(child: &mut ChildGuard) -> (std::process::ExitStatus, String) {
// Both streams: the refusal reason is anyhow's stderr print, while
// "binding HTTP listener" is a tracing line on stdout, and the fatal rows
// assert on one of each.
let mut out = String::new();
let mut o = child.take_stdout();
let mut e = child.take_stderr();
let reader = std::thread::spawn(move || {
let mut buf = String::new();
let _ = o.read_to_string(&mut buf);
buf
});
let _ = e.read_to_string(&mut out);
let status = child.0.wait().expect("wait");
out.push_str(&reader.join().unwrap_or_default());
(status, out)
}
/// Strip ANSI escapes: the subscriber colourises even when piped, which would
/// otherwise sit between `addr=` and the value.
fn strip_ansi(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut chars = s.chars();
while let Some(c) = chars.next() {
if c == '\u{1b}' {
for c2 in chars.by_ref() {
if c2.is_ascii_alphabetic() {
break;
}
}
} else {
out.push(c);
}
}
out
}
/// A lexically invalid value must stop the process before the listener binds.
fn assert_fatal_before_bind(row: &Row, key: &str, cwd: &Path) {
let before = snapshot(&row.tree());
let mut child = row.spawn(key, "7546", cwd);
let (status, err) = wait_for_exit(&mut child);
assert!(
!status.success(),
"key={key:?} must exit non-zero\n--- stderr ---\n{err}"
);
assert!(
err.contains("invalid configuration"),
"key={key:?} must fail as invalid configuration\n--- stderr ---\n{err}"
);
assert!(
!err.contains("binding HTTP listener"),
"key={key:?} must be refused BEFORE the listener binds\n--- stderr ---\n{err}"
);
assert_eq!(
snapshot(&row.tree()),
before,
"key={key:?}: a configuration refusal must not touch the key tree"
);
}
/// A live storage fault must leave HTTP serving with p2p off, and must not
/// mutate the tree on its way to that verdict.
fn assert_degrades_with_http_up(row: &Row, key: &str, cwd: &Path, label: &str) {
let before = snapshot(&row.tree());
let mut child = row.spawn(key, "7546", cwd);
let (found, log) = read_until_all(
&mut child,
&[
"degraded HTTP server ready",
"failed to load p2p identity key",
],
);
assert!(
found,
"{label}: the node must bind and serve while p2p is off\n--- stderr ---\n{log}"
);
// The operator docs promise a specific signal, so pin it here: the level,
// the stable event name, and the message. A degrade that logs at warn or
// renames the event silently breaks the alert those docs tell operators to
// build, and asserting only the prose would not notice.
let clean_log = strip_ansi(&log);
let signal = clean_log
.lines()
.find(|l| l.contains("failed to load p2p identity key"))
.unwrap_or_else(|| panic!("{label}: the p2p failure must be logged\n{clean_log}"));
assert!(
signal.contains("ERROR"),
"{label}: the p2p failure must be logged at error level, got: {signal}"
);
assert!(
signal.contains("p2p_identity_key_load_failed"),
"{label}: the p2p failure must carry the documented event name, got: {signal}"
);
let clean = strip_ansi(&log);
let addr = clean
.lines()
.find(|l| l.contains("degraded HTTP server ready"))
.and_then(|l| l.rsplit("addr=").next())
.map(|a| {
a.trim()
.trim_start_matches("Some(")
.trim_matches(|c| c == '"' || c == ')')
})
.and_then(|a| a.parse::<std::net::SocketAddr>().ok())
.unwrap_or_else(|| panic!("{label}: could not parse the bound address from {clean}"));
// A served response, any status: the degraded server answers 503, which is
// still proof the port is up and answering.
let mut sock = std::net::TcpStream::connect(addr).expect("connect to the degraded server");
sock.set_read_timeout(Some(DEADLINE)).unwrap();
sock.write_all(b"GET /health HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n")
.expect("send request");
let mut resp = String::new();
let _ = sock.read_to_string(&mut resp);
assert!(
resp.starts_with("HTTP/1.1 "),
"{label}: the degraded server must answer, got {resp:?}"
);
drop(child);
assert_eq!(
snapshot(&row.tree()),
before,
"{label}: a storage refusal must not mutate the key tree"
);
}
#[test]
fn lexically_invalid_key_paths_stop_the_node_before_it_binds() {
for key in [
"p2p.key",
"./p2p.key",
"a/../p2p.key",
"../p2p.key",
"/p2p.key",
"keys/",
"~/",
"~//etc/p2p.key",
"~/../x/p2p.key",
] {
let row = Row::new();
let cwd = row.home.path().to_path_buf();
assert_fatal_before_bind(&row, key, &cwd);
}
}
#[test]
fn live_storage_faults_leave_http_up_with_p2p_off() {
// Symlinked final parent.
{
let row = Row::new();
let real = row.tree().join("real");
std::fs::create_dir(&real).unwrap();
std::fs::set_permissions(&real, std::fs::Permissions::from_mode(0o700)).unwrap();
let link = row.tree().join("keys");
std::os::unix::fs::symlink(&real, &link).unwrap();
let key = link.join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "symlinked parent");
}
// Regular file where the key directory should be.
{
let row = Row::new();
let file = row.tree().join("keys");
std::fs::write(&file, b"not a directory").unwrap();
let key = file.join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "regular-file parent");
}
// Unreadable parent: an inspection error, the purest instance of a live
// fault that used to be reported as invalid configuration.
{
let row = Row::new();
let parent = row.tree().join("locked");
std::fs::create_dir(&parent).unwrap();
std::fs::set_permissions(&parent, std::fs::Permissions::from_mode(0o000)).unwrap();
let key = parent.join("keys").join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "unreadable parent");
// Restore so the tempdir can be cleaned up.
let _ = std::fs::set_permissions(&parent, std::fs::Permissions::from_mode(0o700));
}
// World-writable non-sticky ancestor: already the degrade class today, kept
// so the row set covers both sides of the policy.
{
let row = Row::new();
let anc = row.tree().join("open");
std::fs::create_dir(&anc).unwrap();
std::fs::set_permissions(&anc, std::fs::Permissions::from_mode(0o777)).unwrap();
let key = anc.join("keys").join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "world-writable ancestor");
}
}
#[test]
fn port_zero_does_no_key_storage_io_at_all() {
// A lexically invalid path that would be fatal with p2p enabled.
{
let row = Row::new();
let before = snapshot(&row.tree());
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn("p2p.key", "0", &cwd);
let (found, log) = read_until(&mut child, "p2p disabled");
assert!(
found,
"port zero must bypass key handling entirely\n--- stderr ---\n{log}"
);
drop(child);
assert_eq!(
snapshot(&row.tree()),
before,
"port zero must not touch the key tree"
);
}
// A hostile tree that would degrade with p2p enabled.
{
let row = Row::new();
let real = row.tree().join("real");
std::fs::create_dir(&real).unwrap();
let link = row.tree().join("keys");
std::os::unix::fs::symlink(&real, &link).unwrap();
let before = snapshot(&row.tree());
let cwd = row.home.path().to_path_buf();
let key: PathBuf = link.join("p2p.key");
let mut child = row.spawn(key.to_str().unwrap(), "0", &cwd);
let (found, log) = read_until(&mut child, "p2p disabled");
assert!(
found,
"port zero must bypass a hostile tree too\n--- stderr ---\n{log}"
);
drop(child);
assert_eq!(
snapshot(&row.tree()),
before,
"port zero must not touch a hostile key tree"
);
}
}
/// A terminal `.` is a directory-valued spelling. Path drops it, so the load
/// path would otherwise chmod the parent and publish a file named `keys`.
#[test]
fn terminal_dot_key_path_is_fatal_and_does_not_mutate() {
// Absent leaf: `/data/keys/.` must not create `keys` as a file or chmod `/data`.
{
let row = Row::new();
let data = row.tree().join("data");
std::fs::create_dir(&data).unwrap();
std::fs::set_permissions(&data, std::fs::Permissions::from_mode(0o755)).unwrap();
let key = format!("{}/keys/.", data.display());
let cwd = row.home.path().to_path_buf();
assert_fatal_before_bind(&row, &key, &cwd);
let mode = std::fs::metadata(&data).unwrap().permissions().mode() & 0o7777;
assert_eq!(mode, 0o755, "rejection must not chmod the parent");
assert!(
!data.join("keys").exists(),
"rejection must not create a file named keys"
);
}
// Existing directory at the would-be leaf: still boot-fatal, directory untouched.
{
let row = Row::new();
let data = row.tree().join("data");
let keys = data.join("keys");
std::fs::create_dir_all(&keys).unwrap();
std::fs::set_permissions(&data, std::fs::Permissions::from_mode(0o755)).unwrap();
std::fs::set_permissions(&keys, std::fs::Permissions::from_mode(0o700)).unwrap();
let key = format!("{}/.", keys.display());
let cwd = row.home.path().to_path_buf();
assert_fatal_before_bind(&row, &key, &cwd);
assert!(
keys.is_dir(),
"the existing keys directory must remain a directory"
);
let data_mode = std::fs::metadata(&data).unwrap().permissions().mode() & 0o7777;
let keys_mode = std::fs::metadata(&keys).unwrap().permissions().mode() & 0o7777;
assert_eq!(data_mode, 0o755, "rejection must not chmod the parent");
assert_eq!(
keys_mode, 0o700,
"rejection must not chmod the existing keys dir"
);
}
}
/// GITLAWB_KEY has the same Path-retarget hole as GITLAWB_P2P_KEY: a terminal
/// `.` would publish a file named `keys` under `data`. Fatal before bind, and
/// the parent tree is unchanged. p2p is off so it cannot chmod a shared dir.
#[test]
fn identity_terminal_dot_is_fatal_and_does_not_mutate() {
let row = Row::new();
let data = row.home.path().join("data");
std::fs::create_dir(&data).unwrap();
std::fs::set_permissions(&data, std::fs::Permissions::from_mode(0o755)).unwrap();
let key = format!("{}/keys/.", data.display());
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn_env("p2p.key", "0", &cwd, &[("GITLAWB_KEY", key.as_str())]);
let (status, err) = wait_for_exit(&mut child);
assert!(
!status.success(),
"GITLAWB_KEY={key:?} must exit non-zero\n--- stderr ---\n{err}"
);
assert!(
err.contains("invalid configuration"),
"GITLAWB_KEY={key:?} must fail as invalid configuration\n--- stderr ---\n{err}"
);
assert!(
!err.contains("binding HTTP listener"),
"GITLAWB_KEY={key:?} must be refused BEFORE the listener binds\n--- stderr ---\n{err}"
);
let mode = std::fs::metadata(&data).unwrap().permissions().mode() & 0o7777;
assert_eq!(mode, 0o755, "rejection must not chmod the parent");
assert!(
!data.join("keys").exists(),
"rejection must not create a file named keys"
);
}
/// An existing 0755 identity parent is usable and must not be chmodded. p2p is
/// off so a default `~/.gitlawb/p2p.key` cannot tighten the same directory.
#[test]
fn identity_existing_0755_parent_is_not_chmodded_on_boot() {
let row = Row::new();
let iddir = row.home.path().join("idparent");
std::fs::create_dir(&iddir).unwrap();
std::fs::set_permissions(&iddir, std::fs::Permissions::from_mode(0o755)).unwrap();
let key = iddir.join("identity.pem");
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn_env(
"p2p.key",
"0",
&cwd,
&[("GITLAWB_KEY", key.to_str().unwrap())],
);
let (found, log) = read_until(&mut child, "degraded HTTP server ready");
assert!(
found,
"a 0755 identity parent must still boot\n--- stderr ---\n{log}"
);
drop(child);
let mode = std::fs::metadata(&iddir).unwrap().permissions().mode() & 0o7777;
assert_eq!(
mode, 0o755,
"boot must not chmod the existing identity parent"
);
assert_eq!(
std::fs::metadata(&key).unwrap().permissions().mode() & 0o7777,
0o600,
"the identity key must be owner-only"
);
}
/// Creating into a world-writable identity parent is refused before bind, and
/// the directory is left untouched.
#[test]
fn identity_writable_parent_is_fatal_and_unchanged() {
let row = Row::new();
let iddir = row.home.path().join("idparent");
std::fs::create_dir(&iddir).unwrap();
std::fs::set_permissions(&iddir, std::fs::Permissions::from_mode(0o777)).unwrap();
let key = iddir.join("identity.pem");
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn_env(
"p2p.key",
"0",
&cwd,
&[("GITLAWB_KEY", key.to_str().unwrap())],
);
let (status, err) = wait_for_exit(&mut child);
assert!(
!status.success(),
"a writable identity parent must exit non-zero\n--- stderr ---\n{err}"
);
assert!(
!err.contains("binding HTTP listener"),
"a writable identity parent must be refused BEFORE the listener binds\n--- stderr ---\n{err}"
);
let mode = std::fs::metadata(&iddir).unwrap().permissions().mode() & 0o7777;
assert_eq!(mode, 0o777, "refusal must not chmod the parent");
assert!(!key.exists(), "refusal must not publish the identity key");
}
/// A 2700 (setgid, owner rwx) p2p key directory is repairable: boot, tighten
/// to 0700, HTTP still comes up.
#[test]
fn p2p_setgid_key_directory_is_tightened_and_http_comes_up() {
let row = Row::new();
let keys = row.tree().join("keys");
std::fs::create_dir(&keys).unwrap();
std::fs::set_permissions(&keys, std::fs::Permissions::from_mode(0o2700)).unwrap();
let key = keys.join("p2p.key");
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn(key.to_str().unwrap(), "7546", &cwd);
let (found, log) = read_until_all(
&mut child,
&["degraded HTTP server ready", "generated new p2p identity"],
);
assert!(
found,
"a 2700 key directory must boot and generate a key\n--- stderr ---\n{log}"
);
drop(child);
let mode = std::fs::metadata(&keys).unwrap().permissions().mode() & 0o7777;
assert_eq!(
mode, 0o700,
"2700 must be normalized to 0700, found {mode:04o}"
);
}
+3 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
description = "The Twigpine CLI — identity management, node control, MCP server"
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
@@ -11,7 +11,7 @@ name = "gl"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
icaptcha-client = { path = "../icaptcha-client" }
base64 = { workspace = true }
tokio = { workspace = true }
@@ -42,3 +42,4 @@ alloy = { version = "1", default-features = false, features = [
mockito = "1"
tempfile = "3"
tokio = { workspace = true }
ed25519-dalek = { workspace = true }
+27 -28
View File
@@ -2,7 +2,6 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -17,7 +16,11 @@ pub struct AgentArgs {
pub enum AgentCmd {
/// List agents registered on a node
List {
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
/// Filter by capability (e.g. git:push)
#[arg(long)]
@@ -29,7 +32,11 @@ pub enum AgentCmd {
Show {
/// Agent DID or short key segment
did: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
},
}
@@ -57,24 +64,16 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
.get(&path)
.await
.context("failed to connect to node")?;
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
// Keep the bespoke 404 hint (older node without the agents API) ahead of
// read_json; every other status routes through it (capped + sanitized error).
if resp.status() == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!(
"this node does not yet support the agents API (v0.3+)\n\
upgrade the node or check GITLAWB_NODE is pointing to the right server"
);
}
let body: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("list agents failed ({status}): {msg}");
}
let body = crate::http::read_json(resp, "list agents").await?;
let agents = body["agents"].as_array().cloned().unwrap_or_default();
@@ -89,7 +88,7 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
let short = did
.split(':')
.next_back()
.map(|s| &s[..s.len().min(16)])
.map(|s| crate::text::truncate(s, 16))
.unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
let caps = agent["capabilities"]
@@ -117,24 +116,15 @@ async fn cmd_show(did: String, node: String) -> Result<()> {
.get(&format!("/api/v1/agents/{did}"))
.await
.context("failed to connect to node")?;
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
// Keep the bespoke 404 hint ahead of read_json; other statuses route through it.
if resp.status() == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!(
"agent not found, or this node does not yet support the agents API (v0.3+)\n\
upgrade the node or check GITLAWB_NODE is pointing to the right server"
);
}
let agent: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = agent["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("agent not found ({status}): {msg}");
}
let agent = crate::http::read_json(resp, "agent").await?;
let full_did = agent["did"].as_str().unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
@@ -215,11 +205,14 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not found"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_list(server.url(), None).await.unwrap_err();
assert!(err.to_string().contains("agents API"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -230,11 +223,14 @@ mod tests {
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"internal error"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_list(server.url(), None).await.unwrap_err();
assert!(err.to_string().contains("list agents failed"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -261,6 +257,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"agent not found"}"#)
.expect(1)
.create_async()
.await;
@@ -268,6 +265,8 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("agents API") || err.to_string().contains("not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
+146 -60
View File
@@ -2,7 +2,6 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -39,7 +38,11 @@ pub enum BountyCmd {
/// Deadline in seconds (default 7 days)
#[arg(long)]
deadline: Option<i64>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -51,7 +54,11 @@ pub enum BountyCmd {
/// Filter by status: open, claimed, submitted, completed
#[arg(long)]
status: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -60,7 +67,11 @@ pub enum BountyCmd {
Show {
/// Bounty ID
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -72,7 +83,11 @@ pub enum BountyCmd {
/// Your wallet address for payout
#[arg(long)]
wallet: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -84,7 +99,11 @@ pub enum BountyCmd {
/// Pull request ID
#[arg(long)]
pr: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -96,7 +115,11 @@ pub enum BountyCmd {
/// On-chain payout tx hash
#[arg(long)]
tx_hash: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -105,14 +128,22 @@ pub enum BountyCmd {
Cancel {
/// Bounty ID
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show bounty stats and leaderboard
Stats {
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
},
}
@@ -189,13 +220,7 @@ async fn cmd_create(
.await
.context("failed to connect to node")?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("create bounty failed ({status}): {msg}");
}
let body = crate::http::read_json(resp, "create bounty").await?;
let id = body["id"].as_str().unwrap_or("?");
println!("Bounty created: {id}");
@@ -231,11 +256,14 @@ async fn cmd_list(
u
};
let resp = client
.get_authed(&url)
.await
.context("failed to connect to node")?;
let body: Value = resp.json().await.unwrap_or_default();
let body = crate::http::read_json(
client
.get_authed(&url)
.await
.context("failed to connect to node")?,
"bounties",
)
.await?;
let bounties = body["bounties"].as_array();
if let Some(arr) = bounties {
@@ -247,7 +275,7 @@ async fn cmd_list(
let title = b["title"].as_str().unwrap_or("?");
let amount = b["amount"].as_i64().unwrap_or(0);
let st = b["status"].as_str().unwrap_or("?");
let short_id = &id[..8.min(id.len())];
let short_id = crate::text::truncate(id, 8);
println!("{short_id} {st:<10} {amount:>12} $GITLAWB {title}");
}
}
@@ -259,13 +287,7 @@ async fn cmd_show(id: String, node: String, dir: Option<PathBuf>) -> Result<()>
let client = signed_client(&node, dir.as_deref());
let resp = client.get_authed(&format!("/api/v1/bounties/{id}")).await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("not found");
anyhow::bail!("bounty not found: {msg}");
}
let body = crate::http::read_json(resp, "show bounty").await?;
println!("{}", serde_json::to_string_pretty(&body)?);
Ok(())
@@ -289,13 +311,7 @@ async fn cmd_claim(
)
.await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("claim failed ({status}): {msg}");
}
let _ = crate::http::read_json(resp, "claim").await?;
println!("Bounty {id} claimed. Deadline starts now.");
Ok(())
@@ -314,13 +330,7 @@ async fn cmd_submit(id: String, pr: String, node: String, dir: Option<PathBuf>)
)
.await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("submit failed ({status}): {msg}");
}
let _ = crate::http::read_json(resp, "submit").await?;
println!("Bounty {id} submitted with PR {pr}. Awaiting creator approval.");
Ok(())
@@ -344,13 +354,7 @@ async fn cmd_approve(
)
.await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("approve failed ({status}): {msg}");
}
let _ = crate::http::read_json(resp, "approve").await?;
println!("Bounty {id} approved! Payout released to agent.");
Ok(())
@@ -369,13 +373,7 @@ async fn cmd_cancel(id: String, node: String, dir: Option<PathBuf>) -> Result<()
)
.await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("cancel failed ({status}): {msg}");
}
let _ = crate::http::read_json(resp, "cancel").await?;
println!("Bounty {id} cancelled. Tokens refunded.");
Ok(())
@@ -383,8 +381,11 @@ async fn cmd_cancel(id: String, node: String, dir: Option<PathBuf>) -> Result<()
async fn cmd_stats(node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
let resp = client.get("/api/v1/bounties/stats").await?;
let body: Value = resp.json().await.unwrap_or_default();
// Route through read_json so a node denial (403/404/5xx, incl. a non-JSON
// body) surfaces as an error instead of a fake `Bounty Stats` with every
// counter zeroed (#186, the denial-as-success class #123 fixes elsewhere).
let body =
crate::http::read_json(client.get("/api/v1/bounties/stats").await?, "bounty stats").await?;
let open = body["open"].as_i64().unwrap_or(0);
let claimed = body["claimed"].as_i64().unwrap_or(0);
@@ -402,7 +403,7 @@ async fn cmd_stats(node: String) -> Result<()> {
let did = entry["did"].as_str().unwrap_or("?");
let count = entry["completed"].as_i64().unwrap_or(0);
let earned = entry["total_earned"].as_i64().unwrap_or(0);
let short = &did[did.len().saturating_sub(8)..];
let short = crate::text::truncate_suffix(did, 8);
println!(
" {}. ...{short} {count} bounties {earned} $GITLAWB",
i + 1
@@ -521,6 +522,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not found"}"#)
.expect(1)
.create_async()
.await;
@@ -528,6 +530,8 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -544,4 +548,86 @@ mod tests {
cmd_stats(server.url()).await.unwrap();
}
/// #186 (F1): a node denial on the stats endpoint must surface as an error,
/// not print a fake `Bounty Stats` with every counter zeroed. RED before the
/// fix (unwrap_or_default + no status check -> Ok with zeros), GREEN after
/// (read_json bails on the non-2xx).
#[tokio::test]
async fn cmd_stats_surfaces_denial_not_fake_result() {
let mut server = mockito::Server::new_async().await;
let m = server
.mock("GET", mockito::Matcher::Regex(r"/stats$".to_string()))
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"forbidden"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_stats(server.url())
.await
.expect_err("a 403 must be an error, not a zeroed fake result");
assert!(err.to_string().contains("403"), "err={err}");
// Pin the mock hit explicitly. The status assertion above already fails
// when the route does not match (mockito answers an unmatched route with
// 501, so "403" is absent), but that proof is incidental to mockito's
// fallback status; assert the hit directly so it survives a reworded
// assertion.
m.assert_async().await;
}
/// #186 (F1): the same denial-as-error behavior across the non-2xx response
/// classes, not just 403 — a 404 and a 500 (incl. a non-JSON body) must each
/// surface as an error with the status, never a zeroed fake result.
#[tokio::test]
async fn cmd_stats_surfaces_denial_across_status_classes() {
for (status, body, json) in [
(404, r#"{"message":"not found"}"#, true),
(500, "internal boom", false),
] {
let mut server = mockito::Server::new_async().await;
let mut m = server
.mock("GET", mockito::Matcher::Regex(r"/stats$".to_string()))
.with_status(status)
.with_body(body);
if json {
m = m.with_header("content-type", "application/json");
}
let m = m.expect(1).create_async().await;
let err = cmd_stats(server.url()).await.unwrap_err().to_string();
assert!(
err.contains(&status.to_string()),
"status {status} must surface as an error, got: {err}"
);
m.assert_async().await;
}
}
#[tokio::test]
async fn cmd_list_repo_scoped_surfaces_denial_not_empty() {
// `gl bounty list --repo owner/name` hits the gated /repos/{o}/{n}/bounties;
// a 404 must Err, not silently print nothing.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"/repos/alice/secret/bounties".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list(Some("alice/secret".to_string()), None, server.url(), None).await;
assert!(
result.is_err(),
"bounty list --repo must Err on a gated 404"
);
// Prove the gated repo-scoped path was actually requested: without this,
// an unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
}
+627 -51
View File
@@ -4,7 +4,6 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -26,7 +25,11 @@ pub enum CertCmd {
List {
/// Repository in <owner>/<repo> or <repo> format
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -37,7 +40,11 @@ pub enum CertCmd {
repo: String,
/// Certificate ID
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -82,12 +89,7 @@ async fn resolve_repo(
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = signed_client(node, dir);
let info: Value = client
.get_authed("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let info = crate::http::read_json(client.get_authed("/").await?, "node info").await?;
let did = info["did"].as_str().context("node info missing 'did'")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -100,12 +102,7 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/certs");
let resp: Value = client
.get_authed(&path)
.await?
.json()
.await
.context("failed to list certificates")?;
let resp = crate::http::read_json(client.get_authed(&path).await?, "certificates").await?;
let certs = resp["certificates"].as_array().cloned().unwrap_or_default();
@@ -120,7 +117,10 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let id = cert["id"].as_str().unwrap_or("?");
let ref_name = cert["ref_name"].as_str().unwrap_or("?");
let new_sha = cert["new_sha"].as_str().unwrap_or("?");
let issued_at = cert["issued_at"].as_str().map(|s| &s[..19]).unwrap_or("?");
let issued_at = cert["issued_at"]
.as_str()
.map(|s| crate::text::truncate(s, 19))
.unwrap_or("?");
println!(" {id:.8} {issued_at} {ref_name} {new_sha:.12}");
}
Ok(())
@@ -139,14 +139,10 @@ async fn cmd_show(
let client = signed_client(&node, dir.as_deref());
let id = resolve_cert_id(&client, &owner, &name, &id).await?;
// Fetch the certificate
// Fetch the certificate. read_json checks status first and surfaces the node's
// capped+sanitized message on a non-2xx (a bounded error read, not the whole body).
let path = format!("/api/v1/repos/{owner}/{name}/certs/{id}");
let resp = client
.get_authed(&path)
.await?
.error_for_status()
.context("certificate not found")?;
let cert: Value = resp.json().await.context("certificate not found")?;
let cert = crate::http::read_json(client.get_authed(&path).await?, "certificate").await?;
let cert_id = cert["id"].as_str().unwrap_or("?");
let ref_name = cert["ref_name"].as_str().unwrap_or("?");
@@ -191,27 +187,20 @@ async fn cmd_show(
// Contextual only — the verdict above stands on its own, so a node-info
// hiccup here must not turn a successfully displayed certificate into an
// error exit.
let current_node_did = match client.get("/").await {
Ok(resp) => resp
.json::<Value>()
.await
.ok()
.and_then(|info| info["did"].as_str().map(str::to_string)),
Err(_) => None,
// error exit. Route the lookup through read_json so a denial or a capped
// error body yields a reportable reason instead of an opaque None, and keep
// "carried no DID" distinct from "the lookup failed": an empty DID must not
// reach the comparison and fabricate a mismatch warning.
let current: std::result::Result<String, String> = match client.get("/").await {
Ok(resp) => match crate::http::read_json(resp, "node info").await {
Ok(info) => did_from_node_info(&info),
Err(e) => Err(e.to_string()),
},
Err(e) => Err(e.to_string()),
};
match current_node_did.as_deref() {
Some(current) if current == node_did => {
println!(" Issuing node DID matches the node being queried.");
}
Some(current) => {
println!(" WARNING: Certificate node DID ({node_did}) does not match");
println!(" current node DID ({current}).");
println!(" This certificate was issued by a different node.");
}
None => {
println!(" NOTE: could not fetch current node info — skipping node-DID comparison.");
}
let current_node_did = current.as_ref().ok().cloned();
for line in did_check_report(&current, node_did) {
println!("{line}");
}
if require_valid {
@@ -239,6 +228,46 @@ async fn cmd_show(
Ok(())
}
/// Pull the node's DID out of a `GET /` body for the comparison in `cmd_show`.
///
/// A missing OR empty `did` is a failure, not a value: letting `""` through
/// would reach the comparison and print a mismatch WARNING against a DID the
/// node never claimed, which reads as "issued by a different node" when the
/// truth is that the lookup told us nothing.
fn did_from_node_info(info: &serde_json::Value) -> std::result::Result<String, String> {
match info["did"].as_str() {
Some(did) if !did.is_empty() => Ok(did.to_string()),
_ => Err("node info response carried no DID".to_string()),
}
}
/// Select the report lines for the node-DID comparison in `cmd_show`.
///
/// `current` is the current node's DID, or the reason it could not be
/// determined. A comparison verdict (match or WARNING) is only produced when a
/// real DID was obtained; otherwise the report degrades to a could-not-compare
/// hint naming the reason, plus the offline-verification guidance. The signature
/// verdict printed above stands on its own either way — this block only answers
/// *which* node issued the certificate.
fn did_check_report(current: &std::result::Result<String, String>, node_did: &str) -> Vec<String> {
match current {
Ok(current) if current == node_did => {
vec![" Issuing node DID matches the node being queried.".to_string()]
}
Ok(current) => vec![
format!(" WARNING: Certificate node DID ({node_did}) does not match"),
format!(" current node DID ({current})."),
" This certificate was issued by a different node.".to_string(),
],
Err(reason) => vec![
format!(" Could not fetch the current node's DID ({reason}), so the comparison"),
" with the certificate's node DID is unavailable.".to_string(),
" To verify offline, use the node's Ed25519 public key derived from:".to_string(),
format!(" did:key → {node_did}"),
],
}
}
/// Rebuild the node's canonical signing payload (field order must match
/// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the
/// certificate's Ed25519 signature against the key embedded in `node_did`.
@@ -291,14 +320,7 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
}
let path = format!("/api/v1/repos/{owner}/{name}/certs?prefix={id}");
let resp: Value = client
.get_authed(&path)
.await?
.error_for_status()
.context("failed to list certificates")?
.json()
.await
.context("failed to list certificates")?;
let resp = crate::http::read_json(client.get_authed(&path).await?, "certificates").await?;
let certs = resp["certificates"].as_array().cloned().unwrap_or_default();
let matches: Vec<String> = certs
@@ -397,4 +419,558 @@ mod tests {
);
assert!(garbage.is_err(), "malformed signature must not verify");
}
#[tokio::test]
async fn cmd_list_surfaces_denial_not_empty() {
// A gated 404 on the repo-scoped certs read must Err, not print "No certificates".
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/alice/secret/certs$".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list("alice/secret".to_string(), server.url(), None).await;
assert!(result.is_err(), "cert list must Err on a gated 404");
// Prove the gated certs path was actually requested: without this, an
// unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
#[tokio::test]
async fn resolve_repo_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the GET / node-info
// fetch. A gated 404 there must Err (surfacing the status), proving the
// read_json conversion is load-bearing rather than silently ignored.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = resolve_repo("noslash", &server.url(), Some(dir.path()))
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
// The `GET /` node-info lookup after the cert loads is a fail-soft diagnostic:
// a response-level failure degrades to a could-not-compare hint and the command
// completes Ok, never a fabricated mismatch warning and never a fatal Err. The
// cert fetch itself stays fail-closed. A >=36-char id skips resolve_cert_id so
// only two mocks are needed.
#[tokio::test]
async fn cmd_show_completes_with_degraded_hint_when_node_info_denied() {
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must complete despite a denied node-info lookup"
);
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_degrades_on_malformed_node_info() {
// A 2xx node-info body that fails to parse degrades the same way a denial
// does: hint printed, command completes Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_body("not json")
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must complete despite malformed node info"
);
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_degrades_when_node_info_lacks_did() {
// A 2xx node-info body with no `did` routes to the degraded hint, not a
// fabricated empty-DID mismatch warning; the command completes Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body("{}")
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must complete when node info lacks a DID"
);
_cert.assert_async().await;
_root.assert_async().await;
}
// Must-not case: the certificate fetch itself stays fail-closed. A gated 404
// aborts the command with the status surfaced, and the node-info lookup is
// never reached (the expect(0) assert proves it never ran).
#[tokio::test]
async fn cmd_show_surfaces_denied_certificate() {
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository not found"}"#)
.expect(1)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"n"}"#)
.expect(0)
.create_async()
.await;
let err = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_reports_matching_node_did() {
// Pins the unchanged success path: node info fetched, DIDs compared, Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"n"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(result.is_ok(), "cert show must report a matching node DID");
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_warns_on_mismatching_node_did() {
// A real, differing node DID drives the WARNING branch end to end; the
// command still completes Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:other"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must warn, not fail, on a mismatching node DID"
);
_cert.assert_async().await;
_root.assert_async().await;
}
// ── --verify issuer anchoring ────────────────────────────────────────────
//
// Every other cmd_show test passes require_valid=false, so the whole
// `if require_valid` block went unexecuted. It is the security-bearing half of
// the command: a valid signature only proves the certificate is internally
// consistent (a hostile node can mint a keypair, name it in node_did, and
// self-sign), so --verify must additionally anchor the issuer to a DID the
// caller trusts. These drive all four outcomes plus the must-not case.
//
// The certificate must carry a REAL signature over the canonical payload:
// with a bogus one, --verify bails on the signature and never reaches the
// anchoring, which would make every assertion below vacuous.
fn signed_cert(node_kp: &gitlawb_core::identity::Keypair) -> (String, String) {
let node_did = node_kp.did().as_str().to_string();
let id = "a".repeat(36); // >= 36 chars skips resolve_cert_id's prefix lookup
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "b".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = node_kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let body = serde_json::json!({
"id": id,
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "b".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
})
.to_string();
(id, body)
}
/// Mount the cert fetch, plus a `GET /` answering with `node_info` (a JSON
/// body) or a denial when `None`.
async fn cert_server(
server: &mut mockito::Server,
id: &str,
cert_body: &str,
node_info: Option<&str>,
) -> (mockito::Mock, mockito::Mock) {
let cert = server
.mock("GET", format!("/api/v1/repos/alice/r/certs/{id}").as_str())
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_body)
.expect(1)
.create_async()
.await;
let root = match node_info {
Some(b) => {
server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(b)
.create_async()
.await
}
None => {
server
.mock("GET", "/")
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.create_async()
.await
}
};
(cert, root)
}
#[tokio::test]
async fn verify_ok_when_issuing_node_is_the_queried_node() {
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let info = format!(r#"{{"did":"{}"}}"#, kp.did().as_str());
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let got = cmd_show("alice/r".to_string(), id, server.url(), None, true, None).await;
assert!(got.is_ok(), "valid sig + matching issuer must pass");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_bails_when_issuer_is_a_different_node() {
// The self-signing-hostile-node case: the signature verifies against the
// key the cert names, but that key is not the node we queried.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let other = gitlawb_core::identity::Keypair::generate();
let info = format!(r#"{{"did":"{}"}}"#, other.did().as_str());
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("expected issuer"), "got: {err}");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_bails_when_node_info_denied_and_no_expect_node() {
// Fail closed: with no trusted DID to anchor against, --verify must NOT
// fall through to a pass just because the signature checked out.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, None).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("cannot anchor the issuer"), "got: {err}");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_ok_when_expect_node_anchors_a_denied_lookup() {
// --expect-node supplies the trust anchor the denied lookup could not, so
// the same denial that fails the case above now passes.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, None).await;
let got = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(kp.did().as_str().to_string()),
)
.await;
assert!(got.is_ok(), "explicit anchor must pass");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_bails_on_a_bad_signature_before_anchoring() {
// The must-not: a forged certificate naming the queried node must fail on
// the signature, even though its issuer would otherwise anchor cleanly.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let forged = body.replace(&"b".repeat(40), &"c".repeat(40));
let info = format!(r#"{{"did":"{}"}}"#, kp.did().as_str());
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &forged, Some(&info)).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("signature did not verify"), "got: {err}");
cert.assert_async().await;
}
// The must-not case for the DID extraction: an empty or missing `did` has to
// become a could-not-compare reason. If it leaks through as a value, the
// comparison below fabricates a mismatch WARNING against a DID the node
// never claimed.
#[test]
fn did_from_node_info_rejects_empty_and_missing() {
assert!(did_from_node_info(&serde_json::json!({"did": "n"})).is_ok());
for body in [
serde_json::json!({"did": ""}),
serde_json::json!({}),
serde_json::json!({"did": 7}),
] {
let got = did_from_node_info(&body);
assert!(
got.is_err(),
"must not yield a comparable DID: {body} -> {got:?}"
);
}
// And the reason it produces must route to the degraded hint, never a verdict.
let report =
did_check_report(&did_from_node_info(&serde_json::json!({"did": ""})), "n").join("\n");
assert!(
!report.contains("WARNING"),
"fabricated a mismatch: {report}"
);
assert!(report.contains("Could not fetch"), "got: {report}");
}
// did_check_report is the three-way selector between the match text, the
// mismatch WARNING, and the degraded could-not-compare hint. Substring
// asserts (not full-line equality) so cosmetic wording edits don't break them.
#[test]
fn did_check_report_match() {
let report = did_check_report(&Ok("n".to_string()), "n").join("\n");
assert!(
report.contains("matches the node being queried"),
"got: {report}"
);
assert!(!report.contains("WARNING"), "got: {report}");
assert!(!report.contains("Could not fetch"), "got: {report}");
}
#[test]
fn did_check_report_mismatch() {
let report = did_check_report(&Ok("did:key:other".to_string()), "n").join("\n");
assert!(report.contains("WARNING"), "got: {report}");
assert!(report.contains("does not match"), "got: {report}");
assert!(report.contains("did:key:other"), "got: {report}");
assert!(!report.contains("Could not fetch"), "got: {report}");
}
#[test]
fn did_check_report_missing_did_reason() {
let report =
did_check_report(&Err("node info response carried no DID".to_string()), "n").join("\n");
assert!(report.contains("Could not fetch"), "got: {report}");
assert!(
report.contains("node info response carried no DID"),
"got: {report}"
);
assert!(report.contains("verify offline"), "got: {report}");
// The must-not case: no real DID was obtained, so no comparison verdict
// may be claimed in either direction.
assert!(!report.contains("WARNING"), "got: {report}");
assert!(
!report.contains("matches the node being queried"),
"got: {report}"
);
}
#[test]
fn did_check_report_lookup_error_reason() {
let report =
did_check_report(&Err("node info failed (403): denied".to_string()), "n").join("\n");
assert!(report.contains("Could not fetch"), "got: {report}");
assert!(report.contains("403"), "got: {report}");
assert!(report.contains("verify offline"), "got: {report}");
assert!(!report.contains("WARNING"), "got: {report}");
}
}
+40 -18
View File
@@ -2,7 +2,6 @@
use anyhow::{Context, Result};
use clap::Args;
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -15,7 +14,11 @@ pub struct ChangelogArgs {
/// Maximum number of events to show
#[arg(long, default_value = "20")]
pub limit: usize,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
pub node: String,
#[arg(long)]
pub dir: Option<PathBuf>,
@@ -31,7 +34,7 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
};
if repo.is_empty() {
anyhow::bail!("no repo specified — pass <repo> or run from inside a gitlawb repo");
anyhow::bail!("no repo specified — pass <repo> or run from inside a Twigpine repo");
}
let (owner, name) = if let Some((o, n)) = repo.split_once('/') {
@@ -42,12 +45,7 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(&args.node, None);
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let info = crate::http::read_json(client.get("/").await?, "node info").await?;
let did = info["did"].as_str().context("node missing DID")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -64,13 +62,7 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
.await
.context("failed to connect to node")?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("changelog failed ({status}): {msg}");
}
let body = crate::http::read_json(resp, "changelog").await?;
let events = body["events"].as_array().cloned().unwrap_or_default();
@@ -83,14 +75,14 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
for event in &events {
let ts = event["timestamp"].as_str().unwrap_or("?");
let date = &ts[..ts.len().min(10)];
let date = crate::text::truncate(ts, 10);
match event["type"].as_str().unwrap_or("") {
"commit" => {
let sha = event["sha"].as_str().unwrap_or("?");
let msg = event["message"].as_str().unwrap_or("?");
let first_line = msg.lines().next().unwrap_or(msg);
let short_sha = &sha[..sha.len().min(8)];
let short_sha = crate::text::truncate(sha, 8);
println!(" {date} commit {short_sha} {first_line}");
}
"pr_merged" => {
@@ -212,6 +204,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.expect(1)
.create_async()
.await;
@@ -223,6 +216,8 @@ mod tests {
};
let err = run(args).await.unwrap_err();
assert!(err.to_string().contains("changelog failed"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -270,4 +265,31 @@ mod tests {
let err = rt.block_on(run(args)).unwrap_err();
assert!(err.to_string().contains("no repo specified"));
}
#[tokio::test]
async fn resolve_via_run_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the inline GET /
// node-info fetch during resolution. A gated 404 there must Err (surfacing
// the status), proving the read_json conversion is load-bearing.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = run(ChangelogArgs {
repo: Some("noslash".to_string()),
limit: 20,
node: server.url(),
dir: Some(dir.path().to_path_buf()),
})
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
}
+355 -16
View File
@@ -1,4 +1,4 @@
//! `gl clone`: clean partial clone of a gitlawb repo with private subtrees.
//! `gl clone`: clean partial clone of a Twigpine repo with private subtrees.
//!
//! A repo may withhold blob content under some path globs from the caller
//! (Phase 3). The resulting pack is not closed under reachability, so a stock
@@ -12,9 +12,58 @@ use serde::Deserialize;
use std::path::Path;
use std::process::Command;
use crate::http::NodeClient;
use crate::http::{sanitize_node_msg, NodeClient};
use crate::identity::load_keypair_from_dir;
/// Report why one candidate blob could not be recovered, in the register of the
/// recovery loop's other warnings.
///
/// Both halves are gateway-supplied text on their way to a terminal: the oid comes
/// out of a manifest fetched from a caller-chosen Arweave gateway, and a transport
/// error carries the URL the same manifest chose. So the whole line is defanged and
/// length-capped, like every other node/gateway string this crate prints.
///
/// Under `cfg(test)` the line is also mirrored into a per-thread buffer, so a test
/// can assert the loop actually says what it skipped rather than only that it
/// returned nothing.
fn warn_skip(oid: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch encrypted blob {oid}: {why}; skipping"
));
}
/// The same report for a manifest rather than a blob. The manifest is one step
/// earlier in the same recovery: losing it silently means every blob it would have
/// named is missing with no reason given anywhere.
fn warn_skip_manifest(id: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch blob manifest {id}: {why}; skipping"
));
}
/// Sanitize a warning and write it, once.
///
/// The write goes to [`warn_sink`], which is stderr in a normal build and the tests'
/// per-thread mirror under `cfg(test)`. That indirection is the point: the mirror
/// used to sit BESIDE an `eprintln!`, so deleting the user-visible write left every
/// assertion on the mirror green and the shipped behaviour could be removed with the
/// tests unchanged. There is one write now, and the tests observe it.
fn emit_warning(line: &str) {
use std::io::Write;
let line = sanitize_node_msg(line);
let _ = writeln!(warn_sink(), "{line}");
}
#[cfg(not(test))]
fn warn_sink() -> impl std::io::Write {
std::io::stderr()
}
#[cfg(test)]
fn warn_sink() -> impl std::io::Write {
tests::WarnMirror
}
#[derive(Args)]
pub struct CloneArgs {
/// Repo to clone: gitlawb://<owner_did>/<name> or <owner>/<name>.
@@ -27,7 +76,11 @@ pub struct CloneArgs {
#[arg(long)]
pub branch: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
pub node: String,
/// Arweave gateway for B3 manifest discovery/fetch when a node cannot supply
@@ -110,8 +163,8 @@ pub fn setup_partial_clone(
if withheld_globs.is_empty() {
match branch {
Some(b) => git_global(&["clone", "-q", "--branch", b, remote_url, dest_str])?,
None => git_global(&["clone", "-q", remote_url, dest_str])?,
Some(b) => git_global(&["clone", "-q", "--branch", b, "--", remote_url, dest_str])?,
None => git_global(&["clone", "-q", "--", remote_url, dest_str])?,
}
return Ok(());
}
@@ -121,6 +174,7 @@ pub fn setup_partial_clone(
"-q",
"--filter=blob:none",
"--no-checkout",
"--",
remote_url,
dest_str,
])?;
@@ -312,15 +366,30 @@ async fn recover_encrypted_blobs(
.await
{
Ok(r) if r.status().is_success() => r,
_ => continue,
// The node path has the same silent exit the gateway path had: a 403, a
// 404, or a dead connection all reached the caller as "blob not
// recoverable" with no reason attached. One unreachable blob still must
// not end the recovery of the rest, so it warns and moves on.
Ok(r) => {
warn_skip(oid, &format!("node returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(oid, &format!("node request failed: {e}"));
continue;
}
};
let Ok(envelope) = env_resp.bytes().await else {
continue;
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(oid, &format!("reading the envelope failed: {e}"));
continue;
}
};
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
Err(e) => {
eprintln!("warning: could not decrypt {oid}: {e}");
emit_warning(&format!("warning: could not decrypt {oid}: {e}"));
continue;
}
};
@@ -338,7 +407,9 @@ async fn recover_encrypted_blobs(
recovered.push(p.clone());
}
} else {
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
}
}
Ok(recovered)
@@ -510,7 +581,7 @@ fn merge_manifests(
.collect()
}
/// Option B3 fallback recovery, with no dependency on a gitlawb node API. Query
/// Option B3 fallback recovery, with no dependency on a Twigpine node API. Query
/// the Arweave gateway for this repo's encrypted manifests, merge them, and for
/// each blob still missing locally that the caller can decrypt, pull the envelope
/// from a public IPFS gateway, decrypt, and install it as a loose object. Returns
@@ -642,7 +713,14 @@ async fn recover_from_arweave(
for r in refs {
let m = match client.get(format!("{ag}/{}", r.id)).send().await {
Ok(resp) if resp.status().is_success() => resp,
_ => continue,
Ok(resp) => {
warn_skip_manifest(&r.id, &format!("gateway returned {}", resp.status()));
continue;
}
Err(e) => {
warn_skip_manifest(&r.id, &format!("gateway request failed: {e}"));
continue;
}
};
if let Ok(parsed) = m.json::<Manifest>().await {
manifests.push((parsed, r.height));
@@ -684,10 +762,31 @@ async fn recover_from_arweave(
}
let env_resp = match client.get(format!("{ig}/ipfs/{cid}")).send().await {
Ok(r) if r.status().is_success() => r,
_ => continue,
// Every other outcome used to leave through a bare `continue`, so a
// gateway that answered 503, 404, or nothing at all was reported to the
// caller as "blob not recoverable" with no reason attached. This is the
// second in-repo client of GET /ipfs/{cid} and it has no resume ladder,
// so saying what happened is all the recourse there is. It still skips to
// the next candidate either way: one unreachable blob must not end the
// recovery of the rest.
Ok(r) => {
warn_skip(&oid, &format!("gateway returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(&oid, &format!("gateway request failed: {e}"));
continue;
}
};
let Ok(envelope) = env_resp.bytes().await else {
continue;
// A body that dies part-way through is the same mid-read failure the capped
// read exists to stop rendering as silence, and it sits INSIDE the loop whose
// status arms were already fixed.
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(&oid, &format!("reading the envelope failed: {e}"));
continue;
}
};
// open_blob succeeds only if this caller is a recipient: this is the
// authorization gate (no node, no DID check needed).
@@ -708,7 +807,9 @@ async fn recover_from_arweave(
recovered.push(p.clone());
}
} else {
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
}
}
Ok(recovered)
@@ -801,9 +902,42 @@ pub async fn run(args: CloneArgs) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use std::cell::RefCell;
use std::process::Command;
use tempfile::TempDir;
thread_local! {
/// Test-visible copy of the stderr warnings `warn_skip` emits.
/// `#[tokio::test]` runs the future on the test's own thread, so a
/// thread-local is enough.
static WARNINGS: RefCell<String> = const { RefCell::new(String::new()) };
}
/// The `cfg(test)` warning sink. `emit_warning` writes here instead of to
/// stderr, so the assertions below observe the shipped write rather than a copy
/// made beside it: delete that write and they go red.
pub(super) struct WarnMirror;
impl std::io::Write for WarnMirror {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
let text = String::from_utf8_lossy(buf).into_owned();
WARNINGS.with(|w| w.borrow_mut().push_str(&text));
Ok(buf.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
fn reset_warnings() {
WARNINGS.with(|w| w.borrow_mut().clear());
}
fn warnings() -> String {
WARNINGS.with(|w| w.borrow().clone())
}
fn g(args: &[&str], dir: &Path) {
assert!(Command::new("git")
.args(args)
@@ -961,6 +1095,85 @@ mod tests {
);
}
#[test]
fn setup_partial_clone_plain_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], None).unwrap();
assert!(dest.join("file.txt").exists());
}
#[test]
fn setup_partial_clone_with_branch_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let branch_out = Command::new("git")
.args([
"-C",
td.path().join("origin").to_str().unwrap(),
"branch",
"--show-current",
])
.output()
.unwrap();
let branch = String::from_utf8(branch_out.stdout)
.unwrap()
.trim()
.to_string();
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], Some(&branch)).unwrap();
assert!(dest.join("file.txt").exists());
}
/// #374: every `setup_partial_clone` shape interpolates `remote_url` into a
/// `git clone` argv. The `--` delimiter is what forces an option-shaped
/// remote to be read as a repository rather than parsed as an option.
///
/// The assertion is on git's own words — `repository '<value>'`, single-quoted
/// — and not merely on the value appearing somewhere in the error. `git_global`
/// formats its failure as `git {args:?} failed: {stderr}`, so the injected
/// string is already present in the debug-printed argv whether or not the
/// delimiter is there; asserting on containment alone would pass with `--`
/// deleted. Undelimited, git consumes the value as an option, leaving the
/// destination as the only positional, and names *that* instead
/// (`repository '<dest>' does not exist`) — so this goes red without the fix.
///
/// The payload carries no `:` or space: a colon would make git read the
/// argument as an ssh-style `host:path` URL and report a hostname, which is a
/// different parse.
fn assert_option_shaped_remote_is_a_repository(withheld: &[String], branch: Option<&str>) {
let td = TempDir::new().unwrap();
let dest = td.path().join("dest");
let injected = "--upload-pack=false";
let err = setup_partial_clone(&dest, injected, withheld, &[], branch)
.expect_err("an option-shaped remote is not a repository and must fail")
.to_string();
assert!(
err.contains(&format!("repository '{injected}'")),
"git must name the argument as the repository, not consume it as an option: {err}"
);
}
#[test]
fn plain_clone_never_parses_a_remote_as_a_git_option() {
assert_option_shaped_remote_is_a_repository(&[], None);
}
#[test]
fn branch_clone_never_parses_a_remote_as_a_git_option() {
// Pins that `--branch <b>` stays ahead of the delimiter: if `--` were moved
// before `--branch`, git would read the branch name as a positional instead.
assert_option_shaped_remote_is_a_repository(&[], Some("main"));
}
#[test]
fn sparse_clone_never_parses_a_remote_as_a_git_option() {
// The withheld-globs arm is a separate argv construction
// (`--filter=blob:none --no-checkout`) and needs its own proof.
assert_option_shaped_remote_is_a_repository(&["/secret/**".to_string()], None);
}
#[test]
fn sparse_patterns_subtree_and_exact() {
assert_eq!(sparse_patterns("/secret/**"), vec!["/secret/".to_string()]);
@@ -1255,6 +1468,132 @@ mod tests {
assert_eq!(a.get("o1").map(String::as_str), Some("cidTS"));
}
/// A gateway that refuses one blob must SAY so, and must not take the rest of the
/// recovery down with it.
///
/// The IPFS fetch used to leave through a bare `_ => continue` on every
/// non-success, so a 503 (the truncation status a tuned node makes more likely),
/// a 404, and a dead connection all reached the caller as the same silent "blob
/// not recoverable". Two withheld blobs here: the gateway answers 503 for the
/// first and serves the second. The recovered path proves the loop continued; the
/// warning proves the skip was reported and names both the object and the status.
#[tokio::test]
async fn recover_from_arweave_reports_a_refusing_gateway_and_continues() {
use gitlawb_core::encrypt::seal_blob;
use gitlawb_core::identity::Keypair;
reset_warnings();
let (td, url) = bare_remote(&[
("public/a.txt", b"pub\n"),
("secret/b.txt", b"SECRET B\n"),
("secret/c.txt", b"SECRET C\n"),
]);
let dest = td.path().join("dest");
let bare = url.strip_prefix("file://").unwrap();
assert!(Command::new("git")
.args(["-C", bare, "config", "uploadpack.allowFilter", "true"])
.status()
.unwrap()
.success());
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
let oid_of = |path: &str| {
let out = Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
&format!("HEAD:{path}"),
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let refused_oid = oid_of("secret/b.txt");
let served_oid = oid_of("secret/c.txt");
// Origin death, so recovery has to go through the gateways.
std::fs::remove_dir_all(bare).unwrap();
let reader = Keypair::generate();
let envelope = seal_blob(b"SECRET C\n", &[reader.verifying_key()]).unwrap();
let refused_cid = "cidrefused";
let served_cid = "cidserved";
let mut server = mockito::Server::new_async().await;
let _gql = server
.mock("POST", "/graphql")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}}]}}}"#)
.create_async()
.await;
let manifest_body = serde_json::json!({
"timestamp": "2026-06-11T00:00:00Z",
"blobs": [
{ "oid": refused_oid, "cid": refused_cid },
{ "oid": served_oid, "cid": served_cid },
],
})
.to_string();
let _tx = server
.mock("GET", "/TX1")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(manifest_body)
.create_async()
.await;
let refusal = server
.mock("GET", format!("/ipfs/{refused_cid}").as_str())
.with_status(503)
.with_header("content-type", "application/json")
.with_body(r#"{"error":"search_incomplete","message":"scan truncated"}"#)
.expect(1)
.create_async()
.await;
let ok = server
.mock("GET", format!("/ipfs/{served_cid}").as_str())
.with_status(200)
.with_body(envelope)
.expect(1)
.create_async()
.await;
let paths = recover_from_arweave(
&server.url(),
&server.url(),
"alice",
"myrepo",
&dest,
&reader,
)
.await
.unwrap();
refusal.assert_async().await;
ok.assert_async().await;
assert_eq!(
paths,
vec!["secret/c.txt".to_string()],
"a refused candidate must not stop the loop reaching the next one"
);
let warnings = warnings();
assert!(
warnings.contains(&refused_oid),
"the skip must name the object it could not fetch, got: {warnings}"
);
assert!(
warnings.contains("503"),
"the skip must name the gateway's status rather than failing silently, \
got: {warnings}"
);
assert!(
!warnings.contains(&served_oid),
"the blob that was served must not be reported as skipped, got: {warnings}"
);
}
/// Read-path end-to-end over a mocked Arweave + IPFS gateway: discover the
/// manifest via GraphQL, fetch it, fetch the envelope, decrypt with the
/// caller's key, and install the previously-withheld blob.
+190 -49
View File
@@ -1,10 +1,10 @@
//! `gl doctor` — check your gitlawb installation and connectivity.
//! `gl doctor` — check your Twigpine installation and connectivity.
//!
//! Checks:
//! 1. Identity — ~/.gitlawb/identity.pem exists, DID parseable
//! 2. Registration — ~/.gitlawb/ucan.json exists (registered with a node)
//! 3. GITLAWB_NODE — env var is set to a non-localhost URL
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.gitlawb.com)
//! 3. GITLAWB_NODE — env var points at a public node (unset is advisory)
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.twigpine.com)
//! 5. git helper — git-remote-gitlawb is in PATH
//! 6. git — git is installed
//! 7. version — gl is up to date with latest GitHub release
@@ -15,7 +15,7 @@ use std::path::PathBuf;
use crate::http::NodeClient;
const PUBLIC_NODE: &str = "https://node.gitlawb.com";
const PUBLIC_NODE: &str = "https://node.twigpine.com";
const GITHUB_API_BASE: &str = "https://api.github.com";
#[derive(Args)]
@@ -70,7 +70,7 @@ impl Check {
}
pub async fn run(args: DoctorArgs) -> Result<()> {
println!("gl doctor — checking your gitlawb setup");
println!("gl doctor — checking your Twigpine setup");
println!();
let dir = args.dir.unwrap_or_else(|| {
@@ -80,7 +80,6 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
});
let mut checks = Vec::new();
let mut all_ok = true;
// ── 1. Identity ───────────────────────────────────────────────────────
let pem_path = dir.join("identity.pem");
@@ -141,32 +140,16 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
match std::env::var("GITLAWB_NODE") {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Ok(v) if is_loopback_url(&v) => {
checks.push(Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
));
}
Ok(v) if !v.is_empty() => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
_ => {
checks.push(Check::fail(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
"export GITLAWB_NODE=https://node.gitlawb.com",
));
}
}
checks.push(gitlawb_node_env_check(
std::env::var("GITLAWB_NODE").ok().as_deref(),
));
// ── 4. Node connectivity ──────────────────────────────────────────────
let client = NodeClient::new(&args.node, None);
// GITLAWB_NODE="" binds args.node to "" through clap, and check 3 groups
// empty with unset. The probe must apply the same normalization or it
// would dial "" and Fail while the env row only Warns.
let node_url = effective_node_url(&args.node);
let client = NodeClient::new(node_url, None);
match client.get("/").await {
Ok(resp) if resp.status().is_success() => {
let info = resp.json::<serde_json::Value>().await.unwrap_or_default();
@@ -175,7 +158,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
let short_did = did.chars().take(30).collect::<String>();
checks.push(Check::pass(
"node",
format!("{} — v{version} ({short_did}…)", args.node),
format!("{} — v{version} ({short_did}…)", node_url),
));
// Capability drift: a node newer than this CLI may require features
// (RFC 9421 signing, iCaptcha) the CLI doesn't speak.
@@ -187,21 +170,21 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
"node is v{version} but gl is v{gl_ver} — your CLI may be missing \
features (signing / iCaptcha) this node requires"
),
"upgrade gl: curl -sSf https://gitlawb.com/install.sh | sh",
"upgrade gl: curl -sSf https://twigpine.com/install.sh | sh",
));
}
}
Ok(resp) => {
checks.push(Check::fail(
"node",
format!("{} returned HTTP {}", args.node, resp.status()),
format!("{} returned HTTP {}", node_url, resp.status()),
"check GITLAWB_NODE env var or try gl register --node <url>",
));
}
Err(e) => {
checks.push(Check::fail(
"node",
format!("{} unreachable: {e}", args.node),
format!("{} unreachable: {e}", node_url),
"check your internet connection or set GITLAWB_NODE",
));
}
@@ -241,7 +224,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
checks.push(Check::fail(
"git-remote-gitlawb",
"not found in PATH — gitlawb:// clone/push will not work",
"curl -sSf https://gitlawb.com/install.sh | sh",
"curl -sSf https://twigpine.com/install.sh | sh",
));
}
@@ -285,23 +268,21 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
CheckState::Fail => "✗",
};
println!(" {icon} {:<24} {}", check.label, check.detail);
if matches!(check.state, CheckState::Fail) {
all_ok = false;
}
}
println!();
let has_failures_now = has_failures(&checks);
let has_issues = checks
.iter()
.any(|c| matches!(c.state, CheckState::Fail | CheckState::Warn));
if !has_issues {
println!("Everything looks good. Run `gl quickstart` to create your first repo.");
} else {
if all_ok {
println!("Setup looks good with some warnings:");
} else {
if has_failures_now {
println!("Some checks failed. Suggested fixes:");
} else {
println!("Setup looks good with some warnings:");
}
for check in &checks {
if matches!(check.state, CheckState::Fail | CheckState::Warn) {
@@ -311,14 +292,75 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
}
println!();
if !all_ok {
if has_failures_now {
println!("Run `gl quickstart` for a guided setup.");
}
}
// #357: a non-zero exit is the only signal scripting and CI have. Warn-only
// rows stay exit 0 (iCaptcha offline, version drift, shell-alias shadowing
// are all non-fatal); Fail-class rows (identity, registration, node
// reachable, git-remote-gitlawb present, git present) flip to exit 1.
// Direct `process::exit` rather than returning Err: the prose above is the
// user-facing summary, and an anyhow Error frame would just duplicate it.
if has_failures(&checks) {
std::process::exit(1);
}
Ok(())
}
/// Classify the `GITLAWB_NODE` env var for `gl doctor`. Extracted so unit
/// tests can pin the Warn tier directly (#357): an unset env is advisory, not
/// a failure, because `--node` defaults to `https://node.twigpine.com` and the
/// CLI works without it. Flipping this back to `Check::fail` must break a
/// test, not just change prose.
fn gitlawb_node_env_check(env_val: Option<&str>) -> Check {
match env_val {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Some(v) if is_loopback_url(v) => Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
),
Some(v) if !v.is_empty() => Check::pass("GITLAWB_NODE", v.to_string()),
// `--node` defaults to `https://node.twigpine.com` and the CLI works
// fine without the variable, so an unset env is advisory, not a
// failure. Warn-class keeps the exit code 0 for a stock working
// install; this used to be Fail, which the obvious "return non-zero
// on any failure" change would have flipped to exit 1 on every
// default-config install (#357).
_ => Check::warn(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
format!("export GITLAWB_NODE={PUBLIC_NODE}"),
),
}
}
/// The URL the connectivity probe should dial. `GITLAWB_NODE` set but empty
/// (or whitespace-only) binds `args.node` to `""` through clap rather than
/// to the `default_value`, and check 3 already groups empty with unset, so
/// the probe applies the same normalization: an empty effective URL means
/// the public node, same as unset.
fn effective_node_url(node: &str) -> &str {
if node.trim().is_empty() {
PUBLIC_NODE
} else {
node
}
}
/// True when at least one check is Fail-class. Warn-only runs (iCaptcha offline,
/// version drift, shell-alias shadowing, GITLAWB_NODE unset) keep exit 0;
/// this is the gating predicate for #357's exit-status fix.
fn has_failures(checks: &[Check]) -> bool {
checks.iter().any(|c| matches!(c.state, CheckState::Fail))
}
/// Check if a binary name exists anywhere on PATH.
/// True when the rc file contains a real `unalias` command naming `gl` —
/// not a comment, and not a longer word like `unalias global`. Ordering
@@ -408,7 +450,7 @@ fn which_in_path(name: &str) -> bool {
.unwrap_or(false)
}
/// Fetch the latest release tag from Gitlawb/node (the actual release repo — see install.sh's
/// Fetch the latest release tag from Twigpine/node (the actual release repo — see install.sh's
/// `GITLAWB_RELEASE_REPO` default) and compare to current version.
async fn check_version(current: &'static str, github_api_base: &str) -> Check {
let client = match reqwest::Client::builder()
@@ -422,7 +464,7 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
let resp = match client
.get(format!(
"{github_api_base}/repos/Gitlawb/node/releases/latest"
"{github_api_base}/repos/Twigpine/node/releases/latest"
))
.send()
.await
@@ -457,7 +499,7 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
Check::warn(
"version",
format!("v{current} → v{tag} available"),
"curl -sSf https://gitlawb.com/install.sh | sh",
"curl -sSf https://twigpine.com/install.sh | sh",
)
} else {
Check::pass("version", format!("v{current} (up to date)"))
@@ -550,7 +592,7 @@ mod tests {
assert!(is_loopback_url("http://localhost:7545"));
assert!(is_loopback_url("http://[::1]:7545"));
assert!(!is_loopback_url("https://localhost.example"));
assert!(!is_loopback_url("https://node.gitlawb.com/localhost"));
assert!(!is_loopback_url("https://node.twigpine.com/localhost"));
assert!(!is_loopback_url("not a url"));
}
@@ -603,7 +645,7 @@ mod tests {
async fn test_check_version_queries_gitlawb_node_releases() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Gitlawb/node/releases/latest")
.mock("GET", "/repos/Twigpine/node/releases/latest")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"tag_name":"v9.9.9"}"#)
@@ -619,7 +661,7 @@ mod tests {
async fn test_check_version_up_to_date() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Gitlawb/node/releases/latest")
.mock("GET", "/repos/Twigpine/node/releases/latest")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"tag_name":"v0.1.0"}"#)
@@ -635,7 +677,7 @@ mod tests {
async fn test_check_version_http_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Gitlawb/node/releases/latest")
.mock("GET", "/repos/Twigpine/node/releases/latest")
.with_status(403)
.create_async()
.await;
@@ -644,4 +686,103 @@ mod tests {
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("GitHub API returned HTTP 403"));
}
/// #357: an all-Ok run does not flip the exit code to non-zero — a healthy
/// install must keep returning 0 so scripts like `gl doctor && gl push`
/// keep working.
#[test]
fn exit_predicate_is_false_for_all_ok() {
let checks = vec![
Check::pass("identity", "ok"),
Check::pass("registration", "ok"),
Check::pass("git", "ok"),
];
assert!(!has_failures(&checks));
}
/// #357: Warn-class rows (iCaptcha offline, version drift, shell-alias
/// shadowing, GITLAWB_NODE unset on a default-config install) are NOT
/// exit-status failures — they are still printed to the user but the
/// process exits 0. This is the rule that the previous "any
/// Fail-or-Warn row → exit 1" change would have broken for stock
/// installs that simply have not set the env var.
#[test]
fn exit_predicate_is_false_for_warn_only() {
let checks = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export GITLAWB_NODE=..."),
Check::warn("iCaptcha", "offline", "check connectivity"),
Check::warn("version", "drift", "upgrade"),
Check::warn("shell alias", "omz git plugin", "unalias gl"),
];
assert!(!has_failures(&checks));
}
/// #357: a single Fail-class row trips the exit code. The set of Fail
/// rows is whatever `run` writes through `Check::fail`: identity missing
/// or unparseable, registration missing/malformed, node unreachable,
/// git-remote-gitlawb absent, git absent.
#[test]
fn exit_predicate_is_true_when_a_fail_row_is_present() {
let checks = vec![
Check::pass("registration", "ok"),
Check::fail("identity", "missing", "gl identity new"),
Check::pass("git", "ok"),
];
assert!(has_failures(&checks));
let mixed_warns = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export"),
Check::fail("node", "unreachable", "check network"),
Check::warn("iCaptcha", "offline", "check"),
];
assert!(has_failures(&mixed_warns));
}
/// #357: `GITLAWB_NODE` unset must be Warn, not Fail. This pins the
/// re-tier so flipping it back to `Check::fail` breaks the suite — the
/// previous `has_failures`-only tests built their own `Check::warn` rows
/// and could not see the re-tier at all.
#[test]
fn gitlawb_node_env_unset_is_warn() {
let check = gitlawb_node_env_check(None);
assert!(matches!(check.state, CheckState::Warn));
assert!(check.detail.contains("not set"));
let empty = gitlawb_node_env_check(Some(""));
assert!(matches!(empty.state, CheckState::Warn));
}
/// `GITLAWB_NODE=""` binds `args.node` to `""` through clap, and check 3
/// groups empty with unset. The probe must dial the same URL unset
/// would — removing the normalization makes check 4 probe `""` and Fail
/// while the env row only Warns, so an empty env install exits 1.
#[test]
fn effective_node_url_treats_empty_and_blank_as_unset() {
assert_eq!(effective_node_url(""), PUBLIC_NODE);
assert_eq!(effective_node_url(" "), PUBLIC_NODE);
assert_eq!(effective_node_url("\t\n"), PUBLIC_NODE);
assert_eq!(
effective_node_url("http://127.0.0.1:7545"),
"http://127.0.0.1:7545"
);
}
#[test]
fn gitlawb_node_env_set_is_pass() {
let check = gitlawb_node_env_check(Some("https://node.twigpine.com"));
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("https://node.twigpine.com"));
}
#[test]
fn gitlawb_node_env_loopback_is_pass() {
for url in ["http://127.0.0.1:7545", "http://localhost:7545"] {
let check = gitlawb_node_env_check(Some(url));
assert!(
matches!(check.state, CheckState::Ok),
"loopback {url} must be Ok, not Warn/Fail"
);
}
}
}
+951 -2
View File
@@ -1,4 +1,4 @@
//! Signed HTTP client for gitlawb API calls (async).
//! Signed HTTP client for Twigpine API calls (async).
//!
//! Writes are signed with RFC 9421 HTTP Signatures. When the node gates a write
//! behind iCaptcha (HTTP 403 `icaptcha_proof_required`, advertised via the
@@ -10,11 +10,53 @@ use anyhow::{Context, Result};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use icaptcha_client::IcaptchaCfg;
use serde_json::Value;
/// Max times we'll fetch a fresh proof and retry a 403-iCaptcha response
/// (absorbs proof expiry / first-seen replay).
const MAX_ICAPTCHA_RETRIES: usize = 2;
/// Total request timeout: from the start of connecting through the end of the
/// response body, so it bounds a slow download and not just a slow handshake.
const TOTAL_REQUEST_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
/// Follow a redirect only when it stays on the origin that issued it AND re-issues the
/// identical request-target, and only for as long as the chain bound allows.
///
/// The decision itself is [`gitlawb_core::redirect::may_follow`], shared with
/// `git-remote-gitlawb` so the two signing clients cannot drift apart on it. Refusal
/// is `stop`, not `error`: the 3xx comes back as an ordinary response and each caller
/// reports it through the status path it already has.
///
/// `Policy::custom` replaces reqwest's built-in limit, so the chain bound is restated
/// here. Same-origin redirects can cycle, and this is what stops a node answering 302
/// to itself from being followed indefinitely. It is not what makes the request
/// finite: `.timeout(...)` on the same builder is a TOTAL request timeout covering the
/// whole chain, so without this bound the worst case is a 30 second spin, not an
/// endless one. The bound is what keeps that spin from costing the node a request per
/// round trip for the full 30 seconds.
///
/// `>` and not `>=`: reqwest pushes the redirecting URL onto `previous` before
/// consulting the policy, so on the first redirect `previous.len()` is already 1, and
/// `>=` would permit `MAX_REDIRECTS - 1` follows. `Policy::limited(max)` refuses at
/// `previous.len() > max`, and matching it is the point of reusing its value.
fn same_origin_redirect(attempt: reqwest::redirect::Attempt<'_>) -> reqwest::redirect::Action {
let Some(previous) = attempt.previous().last() else {
// No previous URL to compare against. Unreachable through reqwest, which
// pushes the redirecting URL before consulting the policy, but the safe
// reading of "cannot prove same-origin" is to refuse.
return attempt.stop();
};
if attempt.previous().len() > gitlawb_core::redirect::MAX_REDIRECTS {
return attempt.stop();
}
if gitlawb_core::redirect::may_follow(previous, attempt.url()) {
attempt.follow()
} else {
attempt.stop()
}
}
pub struct NodeClient {
inner: reqwest::Client,
pub node_url: String,
@@ -23,8 +65,20 @@ pub struct NodeClient {
impl NodeClient {
pub fn new(node_url: impl Into<String>, keypair: Option<Keypair>) -> Self {
Self::with_timeout(node_url, keypair, TOTAL_REQUEST_TIMEOUT)
}
/// `new` with the total request timeout as a parameter, so a test can drive the
/// timeout's behaviour without waiting out the shipped value. `new` supplies the
/// shipped one, which is what makes the scaled-down test cover the real client.
fn with_timeout(
node_url: impl Into<String>,
keypair: Option<Keypair>,
timeout: std::time::Duration,
) -> Self {
let inner = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.timeout(timeout)
.redirect(reqwest::redirect::Policy::custom(same_origin_redirect))
.user_agent(format!("gl/{} gitlawb-cli", env!("CARGO_PKG_VERSION")))
.build()
.expect("failed to build HTTP client");
@@ -203,6 +257,133 @@ async fn obtain_proof(cfg: IcaptchaCfg) -> Result<String> {
.context("iCaptcha solver task panicked")?
}
/// What a capped body read produced. `text` is the bytes that arrived; the two flags
/// say why the read stopped where it did, which a caller that CLASSIFIES on the body
/// cannot work out from the text alone.
pub(crate) struct CappedBody {
/// The bytes read, lossily decoded.
pub(crate) text: String,
/// The cap cut the body short.
pub(crate) truncated: bool,
/// A chunk read FAILED part-way through, so the body is not merely short, it is
/// unfinished and the node may have had more to say.
pub(crate) read_failed: bool,
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
///
/// `truncated` reports whether the cap cut the body short. A caller that CLASSIFIES
/// on the body needs it: a cut body fails JSON parse, and a parse failure is
/// indistinguishable from a node that sent no code at all, so without this flag an
/// oversized body silently picks a different arm.
///
/// `read_failed` reports the other way a body can end early. A mid-body read error
/// used to leave through the same exit as a clean end of stream, so a 500 whose body
/// died in transit surfaced as an empty message and the caller was told
/// `node returned 500: ` with nothing after the colon. That is a report of what the
/// node said, and the node never got to say it.
pub(crate) async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> CappedBody {
let mut buf: Vec<u8> = Vec::new();
let mut truncated = false;
let mut read_failed = false;
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
truncated = true;
break; // hit the cap mid-chunk
}
}
Ok(None) => break, // clean end of body
Err(_) => {
read_failed = true;
break;
}
}
}
// A body that lands exactly on the cap may or may not have more behind it;
// report it as cut, since the classification that follows cannot tell either.
if buf.len() >= cap {
truncated = true;
}
CappedBody {
text: String::from_utf8_lossy(&buf).into_owned(),
truncated,
read_failed,
}
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim. We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
pub(crate) fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !gitlawb_core::sanitize::is_bidi_format(*c))
.take(200)
.collect()
}
/// Read a JSON response, surfacing a node denial/error instead of parsing it as
/// the requested resource. On a non-2xx status it returns an `Err` carrying the
/// node's sanitized `message` plus the status; on success it parses the
/// body and propagates a parse error, so a truncated/garbage 2xx body is an error
/// rather than a silently-empty success (the denial-as-success bug #123 fixes).
/// `what` names the resource for the error text (e.g. "repo", "commits").
///
/// Callers must route gated reads through this rather than `resp.json().await?`:
/// the bare parse renders a gated 404/5xx body back as the resource.
/// Cap on the error body `read_json` reads before parsing a node's `message`.
/// Matches the capped-read bound used on the sync error path; large enough for any
/// legitimate error, small enough that a hostile node cannot exhaust memory.
const ERROR_BODY_CAP: usize = 8 * 1024;
pub(crate) async fn read_json(resp: reqwest::Response, what: &str) -> Result<Value> {
let status = resp.status();
if !status.is_success() {
// Bound the error read: `resp.json()` would buffer and parse the WHOLE
// body, so a hostile node could stream an arbitrarily large valid JSON
// error and exhaust this process's memory (#186). Read a small capped body
// first, then best-effort extract `message`. The error body may also be
// non-JSON (503 degraded, 413 body-limit from middleware), so a parse miss
// falls back to the capped raw text; an empty body to the status alone.
let raw = read_body_capped(resp, ERROR_BODY_CAP).await.text;
// Preserve the prior contract: surface the JSON `message` when present in
// the capped body, else the status alone ("request failed") — a non-JSON
// or message-less body (503 degraded, 413 body-limit) never surfaces raw
// node bytes. A body whose `message` sits past the cap parses as truncated
// garbage here and correctly falls back rather than being buffered whole.
//
// Fall back to `error` before giving up: the node's shared AppError
// envelope carries both keys, but the task API answers with `error` alone
// (`{"error":"task not found"}` in gitlawb-node/src/api/tasks.rs), so
// reading only `message` blanked every `gl task` denial to the generic
// "request failed". `sync.rs`'s hand-rolled reader already reads both; this
// matches it.
let msg = serde_json::from_str::<Value>(&raw)
.ok()
.and_then(|b| {
b.get("message")
.or_else(|| b.get("error"))
.and_then(|m| m.as_str())
.map(str::to_owned)
})
.unwrap_or_else(|| "request failed".to_owned());
anyhow::bail!("{what} failed ({status}): {}", sanitize_node_msg(&msg));
}
// Success: a truncated/garbage 2xx body must be an `Err`, not `Ok(Null)` that
// a caller then renders as an empty success.
resp.json()
.await
.with_context(|| format!("invalid JSON in {what} response"))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -587,4 +768,772 @@ mod tests {
ic.challenge.assert();
ic.answer.assert();
}
// ── redirect policy ─────────────────────────────────────────────────
/// The signed headers must not survive a redirect off the node's origin.
///
/// reqwest strips only `Authorization`, `Cookie`, `Proxy-Authorization` and
/// `WWW-Authenticate` across hosts, so `Signature` and `Signature-Input` used to
/// ride a 302 straight to whatever origin the node named. The signature binds
/// `@method`, `@path` and `content-digest` and nothing about the authority, so the
/// receiving host holds a credential that reads path-scoped objects as the caller
/// at any node until the clock-skew window closes.
///
/// Two mockito servers are two ports on one host, which is exactly the boundary
/// this policy draws (and the one reqwest's own header stripping draws). The
/// second server answers everything and expects nothing: a followed redirect
/// fails the expectation whether or not the signature came with it. MUTATION
/// (RED): drop the `.redirect(...)` line and the second server is hit.
#[tokio::test]
async fn cross_origin_redirect_is_not_followed() {
let mut elsewhere = Server::new_async().await;
let never = elsewhere
.mock("GET", mockito::Matcher::Any)
.with_status(200)
.with_body("bytes from the redirect target")
.expect(0)
.create_async()
.await;
let signature_seen = elsewhere
.mock("GET", mockito::Matcher::Any)
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.expect(0)
.create_async()
.await;
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(302)
.with_header("location", &format!("{}/api/v1/thing", elsewhere.url()))
.expect(1)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(test_keypair()));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
assert_eq!(
resp.status(),
302,
"a refused redirect stops rather than errors, so the caller sees the 3xx \
and reports it through the status path it already has"
);
let body = resp.text().await.unwrap();
assert!(
!body.contains("bytes from the redirect target"),
"the redirect target's bytes must never reach the caller, got: {body}"
);
bounce.assert_async().await;
never.assert_async().await;
signature_seen.assert_async().await;
}
/// A node redirecting to itself is same-origin, so the origin predicate follows it
/// every time and only the chain bound ends the loop. Deleting the bound left the
/// whole suite green, because nothing here had ever built a cycle.
///
/// It has a second job now. A redirect back to the identical path and query is the
/// only same-origin hop the predicate still follows, so the eleven hits below are
/// also this crate's executed proof that such a hop IS followed. The old positive
/// fixture drove a trailing-slash rewrite, which the request-target rule refuses,
/// and an http-to-https upgrade cannot be mocked over mockito's plain http.
///
/// The route answers 301 pointing back at itself. Bounded, the handler is hit
/// once for the original request plus `MAX_REDIRECTS` follows and the call returns
/// the 301 (a refused redirect stops rather than errors). Unbounded, it runs until
/// the client's total request timeout cuts it off, which is a 30 second spin at the
/// shipped value and a request per round trip for the node.
///
/// MUTATION (RED): delete the `previous().len()` check.
#[tokio::test]
async fn a_self_redirect_stops_at_the_chain_bound() {
let mut node = Server::new_async().await;
let hits = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let h = hits.clone();
let loop_route = node
.mock("GET", "/api/v1/loop")
.with_status(301)
.with_header("location", "/api/v1/loop")
.with_body_from_request(move |_req| {
h.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Vec::new()
})
.expect_at_least(1)
.create_async()
.await;
let client = NodeClient::with_timeout(
node.url(),
Some(test_keypair()),
std::time::Duration::from_secs(5),
);
let resp = client
.get_signed("/api/v1/loop")
.await
.expect("the bound must end the chain, not the timeout");
assert_eq!(
resp.status(),
301,
"the chain ends by refusing the next hop, so the last 3xx is what comes back"
);
assert_eq!(
hits.load(std::sync::atomic::Ordering::SeqCst),
gitlawb_core::redirect::MAX_REDIRECTS + 1,
"one original request plus MAX_REDIRECTS follows, matching what \
Policy::limited(MAX_REDIRECTS) would have permitted"
);
loop_route.assert_async().await;
}
/// A same-origin hop that REWRITES the request-target is refused, even though the
/// origin never changes.
///
/// The signature binds `@path` as the literal path-and-query the client signed, and
/// the node rebuilds it from the URI it received. A `/api/v1/thing` to
/// `/api/v1/thing/` bounce therefore presents a signature over a target the node
/// never saw, and every signed read behind such a proxy 401s. Refusing the hop
/// hands the caller the 3xx that names what happened instead.
///
/// The target mock expects zero hits and is asserted: mockito only checks
/// `.expect(N)` when `.assert()` runs, so an unbound or unasserted mock passes
/// vacuously.
#[tokio::test]
async fn same_origin_path_changing_redirect_is_refused() {
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(301)
.with_header("location", "/api/v1/thing/")
.expect(1)
.create_async()
.await;
let target = node
.mock("GET", "/api/v1/thing/")
.with_status(200)
.with_body("bytes from the rewritten target")
.expect(0)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(test_keypair()));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
assert_eq!(
resp.status(),
301,
"a refused redirect stops rather than errors, so the caller sees the 3xx \
and reports it through the status path it already has"
);
let body = resp.text().await.unwrap();
assert!(
!body.contains("bytes from the rewritten target"),
"the rewritten target's bytes must never reach the caller, got: {body}"
);
bounce.assert_async().await;
target.assert_async().await;
}
// ── the node's own verification, run over what the client actually sent ──
/// What the verifying mock made of a request that reached it.
///
/// The empty slot (`None`) is its own state and means the mock was never hit at
/// all, which is what the refusal test asserts. It must stay distinguishable from
/// [`Verdict::WrongIdentity`], because "nobody verified anything" and "something
/// verified against the wrong key" are opposite findings.
///
/// The payloads are read through `Debug` in the assertion messages and nowhere
/// else, which the dead-code pass does not count; they carry the detail that makes
/// a failure legible, so they stay.
#[derive(Debug)]
#[allow(dead_code)]
enum Verdict {
/// The chain accepted the signature AND the key it resolved is the test's DID.
Accepted,
/// The chain refused it. Carries the error so a failure reads as the actual
/// rejection rather than a bare hit count.
Rejected(String),
/// The chain accepted a signature made by somebody else. A key resolved from
/// the parsed `key_id` is read out of the artifact under verification, so an
/// accept on it alone proves consistency, never authenticity.
WrongIdentity { expected: String, got: String },
}
/// The node's `require_signature` verification, over a request this crate did not
/// build: parse the headers, recompute the content-digest from the body, rebuild
/// the signing string over `@method`/`@path`/`content-digest`, Ed25519-verify.
/// Returns the DID the signature resolved to, so a caller can pin the identity.
///
/// A hand-copy of its twin in `crates/git-remote-gitlawb/src/main.rs`, which gl
/// cannot import (`git-remote-gitlawb` is a binary crate and this is its test
/// module). Keep the two textually identical apart from the mockito seam around
/// them, so an edit to one is visibly an edit to both.
///
/// The production verifier both copies mirror is `crate::auth::require_signature` in
/// `crates/gitlawb-node/src/auth/mod.rs`. This is a re-implementation, not a call, so
/// an edit to that middleware has to land here too: otherwise the copies drift and
/// this test keeps passing against a rule the node has stopped applying.
///
/// It asserts internally, which is deliberate but constrains its callers: inside
/// `with_body_from_request` those assertions fire on the server thread and reach
/// the client as a transport error, not as a recorded verdict. So the identity
/// check lives in the caller as a [`Verdict`] variant, never as an assert in here.
fn node_verifies(
method: &str,
path_and_query: &str,
body: &[u8],
sig_input: &str,
sig_header: &str,
content_digest: &str,
) -> anyhow::Result<String> {
use gitlawb_core::http_sig::{build_signing_string, compute_content_digest, HttpSignature};
use gitlawb_core::identity::verify;
use std::collections::HashMap;
let sig = HttpSignature::parse(sig_input, sig_header)?;
sig.check_created()?;
assert!(
sig.missing_components().is_empty(),
"signature must cover all required components"
);
assert_eq!(sig.alg, "ed25519");
assert_eq!(
content_digest,
compute_content_digest(body),
"content-digest must match the body"
);
let vk = sig.key_id.to_verifying_key()?;
let mut values = HashMap::new();
values.insert("@method".to_string(), method.to_uppercase());
values.insert("@path".to_string(), path_and_query.to_string());
values.insert("content-digest".to_string(), content_digest.to_string());
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(sig_input);
let components: Vec<&str> = sig.components.iter().map(String::as_str).collect();
let signing_string = build_signing_string(&components, sig_params_value, &values)?;
let sig_array: [u8; 64] = sig.signature_bytes.as_slice().try_into()?;
verify(&vk, signing_string.as_bytes(), &sig_array)?;
Ok(sig.key_id.to_string())
}
/// Pull a header value off a received mockito request, or explain which one the
/// client failed to send.
fn received_header(req: &mockito::Request, name: &str) -> String {
req.header(name)
.first()
.unwrap_or_else(|| panic!("the client sent no {name} header"))
.to_str()
.unwrap()
.to_string()
}
/// Run [`node_verifies`] over a GET that arrived at the mock and record what the
/// node would have made of it, pinned to `expected_did`.
fn record_get_verdict(
req: &mockito::Request,
expected_did: &str,
slot: &std::sync::Arc<std::sync::Mutex<Option<Verdict>>>,
) {
let verdict = match node_verifies(
"GET",
req.path_and_query(),
b"",
&received_header(req, "signature-input"),
&received_header(req, "signature"),
&received_header(req, "content-digest"),
) {
Ok(did) if did == expected_did => Verdict::Accepted,
Ok(did) => Verdict::WrongIdentity {
expected: expected_did.to_string(),
got: did,
},
Err(e) => Verdict::Rejected(e.to_string()),
};
*slot.lock().unwrap() = Some(verdict);
}
/// The finding's repro, now a guard: a rewritten same-origin target must never
/// receive the signature, and the proof is the node's own verification, not a hit
/// count.
///
/// The target mock runs the full `require_signature` chain over the request it
/// receives. Post-fix the hop is refused, so the slot stays empty. Pre-fix the hop
/// is followed and the slot records the Ed25519 rejection of a signature made over
/// `/api/v1/thing` and presented at `/api/v1/thing/`, which is the 401 an operator
/// behind such a proxy actually sees. The verdict is asserted first, so a failure
/// speaks about verification rather than about reachability.
///
/// Its paired positive control is
/// `a_direct_signed_get_verifies_under_the_node_verifier`: without it, an empty
/// slot would be satisfied just as well by a harness that can never record
/// anything.
#[tokio::test]
async fn a_rewritten_target_never_receives_the_signature() {
let kp = test_keypair();
let expected_did = kp.did().to_string();
let slot = std::sync::Arc::new(std::sync::Mutex::new(None::<Verdict>));
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(301)
.with_header("location", "/api/v1/thing/")
.expect(1)
.create_async()
.await;
let recorder = slot.clone();
let did_for_target = expected_did.clone();
let target = node
.mock("GET", "/api/v1/thing/")
.with_status(200)
.with_body_from_request(move |req| {
record_get_verdict(req, &did_for_target, &recorder);
Vec::new()
})
.expect(0)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(kp));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
let verdict = slot.lock().unwrap().take();
assert!(
verdict.is_none(),
"the node's own verifier must never see this request: the signature covers \
/api/v1/thing and the rewritten target is /api/v1/thing/, so what arrives \
there is a stale request-target; recorded verdict: {verdict:?}"
);
assert_eq!(
resp.status(),
301,
"the caller sees the 3xx, not the rewritten target's answer"
);
bounce.assert_async().await;
target.assert_async().await;
}
/// The positive control for the test above, and the proof that the client signs
/// the query it sends.
///
/// A direct signed GET, no redirect anywhere, through the same verifying mock. The
/// verdict must be `Accepted`, which is what makes the refusal test's empty slot
/// attributable to the refusal rather than to a harness that cannot record. The
/// path carries a query, so a client that signed the bare path would land here as
/// `Rejected`.
#[tokio::test]
async fn a_direct_signed_get_verifies_under_the_node_verifier() {
let kp = test_keypair();
let expected_did = kp.did().to_string();
let slot = std::sync::Arc::new(std::sync::Mutex::new(None::<Verdict>));
let mut node = Server::new_async().await;
let recorder = slot.clone();
let did_for_target = expected_did.clone();
let route = node
.mock("GET", "/api/v1/thing?x=1")
.with_status(200)
.with_body_from_request(move |req| {
record_get_verdict(req, &did_for_target, &recorder);
Vec::new()
})
.expect(1)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(kp));
let resp = client.get_signed("/api/v1/thing?x=1").await.unwrap();
assert_eq!(resp.status(), 200);
let verdict = slot.lock().unwrap().take();
assert!(
matches!(verdict, Some(Verdict::Accepted)),
"a direct signed GET must verify under the node's own chain and resolve to \
{expected_did}, or the refusal test's empty slot proves nothing; recorded \
verdict: {verdict:?}"
);
route.assert_async().await;
}
// ── read_body_capped ────────────────────────────────────────────────
/// A body whose read FAILS mid-stream must be distinguishable from a body that
/// ended. Both used to leave through the same `_ => break`, so a 500 whose body
/// died in transit produced an empty message and the caller was told
/// `node returned 500: ` with nothing after the colon.
///
/// The fixture is a raw listener that promises 64 bytes in `Content-Length`,
/// writes 5, and closes. mockito cannot express that: it always completes the
/// response it advertises. MUTATION (RED): restore the single `_ => break` arm
/// (or hard-code `read_failed: false`) and the flag reads false.
#[tokio::test]
async fn read_body_capped_flags_a_mid_body_read_failure() {
let addr = spawn_short_body_listener().await;
let resp = reqwest::get(format!("http://{addr}/truncated"))
.await
.expect("headers arrive before the body is cut");
let body = read_body_capped(resp, 8192).await;
assert!(
body.read_failed,
"a body cut off mid-stream must be reported as a failed read, not as a \
body that ended: got {:?}",
body.text
);
assert!(
!body.truncated,
"the cap did not cut this one; 5 bytes are nowhere near 8 KiB"
);
}
/// The must-not half: a body that ends cleanly must NOT be flagged, or the flag
/// means nothing and every terminal starts claiming the node went quiet.
#[tokio::test]
async fn read_body_capped_does_not_flag_a_clean_body() {
let mut server = Server::new_async().await;
let _m = server
.mock("GET", "/ok")
.with_status(500)
.with_body("node said this")
.create_async()
.await;
let resp = reqwest::get(format!("{}/ok", server.url())).await.unwrap();
let body = read_body_capped(resp, 8192).await;
assert_eq!(body.text, "node said this");
assert!(!body.read_failed, "a complete body is not a failed read");
assert!(!body.truncated, "a complete body is not a truncated one");
}
/// Answer one request with headers promising more body than gets written, then
/// close the connection. Returns the listener's address.
async fn spawn_short_body_listener() -> std::net::SocketAddr {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.unwrap();
let mut scratch = [0u8; 1024];
let _ = sock.read(&mut scratch).await;
let _ = sock
.write_all(b"HTTP/1.1 500 Internal Server Error\r\nContent-Length: 64\r\n\r\nshort")
.await;
let _ = sock.flush().await;
// Drop closes the socket with 59 of the promised bytes never sent.
});
addr
}
// ── total request timeout ───────────────────────────────────────────
/// The client's timeout is a TOTAL request timeout, so it bounds a download and
/// not just the handshake. `gl ipfs get`'s documentation leans on exactly that:
/// the wall-clock deadline covers the search and deliberately stops at the
/// response headers, and this timeout is the only thing left bounding the body.
///
/// Driven at 250ms through `with_timeout`, the seam `new` itself calls with
/// `TOTAL_REQUEST_TIMEOUT`, because a test at the shipped 30s has no place in this
/// suite. What that costs is the value; what it proves is the SHAPE, which is the
/// part in doubt: that the deadline keeps running once the headers have landed. A
/// timeout that covered only the handshake would let this request hang until the
/// listener gives up.
#[tokio::test]
async fn total_timeout_cuts_off_a_body_that_outruns_it() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.unwrap();
let mut scratch = [0u8; 1024];
let _ = sock.read(&mut scratch).await;
// Headers land immediately, then the body stalls indefinitely.
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 1024\r\n\r\nfirst")
.await;
let _ = sock.flush().await;
tokio::time::sleep(std::time::Duration::from_secs(30)).await;
});
let client = NodeClient::with_timeout(
format!("http://{addr}"),
None,
std::time::Duration::from_millis(250),
);
let started = std::time::Instant::now();
let resp = client.get("/slow").await.expect("headers arrive promptly");
assert_eq!(
resp.status(),
200,
"the stall is in the body, not the status"
);
let err = resp
.bytes()
.await
.expect_err("a body still arriving past the total timeout must be cut off");
let elapsed = started.elapsed();
assert!(
err.is_timeout(),
"the body read must end in a timeout, got: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(5),
"the timeout must fire on its own schedule, not wait out the listener; \
took {elapsed:?}"
);
}
#[test]
fn shipped_client_uses_the_documented_total_timeout() {
// The scaled-down test above proves the shape at 250ms. This pins the value
// `new` actually ships, so the two together cover the documented behaviour.
assert_eq!(TOTAL_REQUEST_TIMEOUT, std::time::Duration::from_secs(30));
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[test]
fn sanitize_preserves_legitimate_and_rtl_text() {
// Must not over-strip: a plain word, a genuine RTL SCRIPT letter (Arabic
// U+0627, category Lo — NOT a format char), and ZWJ (U+200D, a legitimate
// Cf char, e.g. emoji sequences) all survive. Guards the shared predicate
// against being widened into a blanket Cf stripper.
let out = sanitize_node_msg("ok \u{0627}\u{200D}b");
assert_eq!(out, "ok \u{0627}\u{200D}b");
}
// ── read_json (status-checked, bounded, sanitized read; #123) ────────
/// Drive a real `reqwest::Response` off a mockito mock so `read_json` sees an
/// actual HTTP status + body, the same shape the gated read arms produce.
async fn response_for(
server: &mut Server,
status: usize,
body: &str,
json: bool,
) -> reqwest::Response {
let mut m = server.mock("GET", "/x").with_status(status).with_body(body);
if json {
m = m.with_header("content-type", "application/json");
}
let _m = m.create_async().await;
NodeClient::new(server.url(), None).get("/x").await.unwrap()
}
#[tokio::test]
async fn read_json_returns_body_on_2xx() {
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
200,
r#"{"name":"r","owner_did":"did:gitlawb:z"}"#,
true,
)
.await;
let v = read_json(resp, "repo").await.unwrap();
assert_eq!(v["name"], "r");
}
#[tokio::test]
async fn read_json_errs_on_404_surfacing_message_and_status() {
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
404,
r#"{"message":"repository 'o/r' not found"}"#,
true,
)
.await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("404"), "err={err}");
assert!(err.contains("not found"), "err={err}");
}
#[tokio::test]
async fn read_json_errs_on_500_surfacing_message() {
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 500, r#"{"message":"internal boom"}"#, true).await;
let err = read_json(resp, "commits").await.unwrap_err().to_string();
assert!(err.contains("500"), "err={err}");
assert!(err.contains("internal boom"), "err={err}");
}
#[tokio::test]
async fn read_json_errs_on_non_json_error_body_with_fallback() {
// 503 with a plain-text (middleware) body: no `message` field to surface.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 503, "service unavailable", false).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("503"), "err={err}");
assert!(err.contains("request failed"), "err={err}");
}
#[tokio::test]
async fn read_json_sanitizes_control_and_bidi_in_message() {
// A hostile node embeds ESC, BEL, and a right-to-left override in
// the error message; none may reach the terminal verbatim.
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
404,
r#"{"message":"a\u001b[31mb\u0007c\u202ed"}"#,
true,
)
.await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(!err.contains('\u{1b}'), "ESC leaked: {err:?}");
assert!(!err.contains('\u{7}'), "BEL leaked: {err:?}");
assert!(!err.contains('\u{202e}'), "bidi override leaked: {err:?}");
}
#[tokio::test]
async fn read_json_errs_on_garbage_2xx_body() {
// The #123 correctness point: a 200 with a non-JSON/truncated body must be
// an `Err`, NOT `Ok(Null)` that a caller renders as an empty success.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 200, "this is not json", false).await;
assert!(read_json(resp, "repo").await.is_err());
}
#[tokio::test]
async fn read_json_errs_on_empty_2xx_body() {
// A zero-byte 200 body must be an `Err`, not a silent `Ok(Null)`.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 200, "", false).await;
assert!(read_json(resp, "repo").await.is_err());
}
#[tokio::test]
async fn read_json_errs_on_non_2xx_json_without_message_or_error() {
// A non-2xx JSON body carrying NEITHER key is the real fallback case.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 403, r#"{"detail":"forbidden"}"#, true).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("403"), "err={err}");
assert!(err.contains("request failed"), "err={err}");
// The body's own text must not leak when no recognised key carries it.
assert!(!err.contains("forbidden"), "raw body leaked: {err}");
}
#[tokio::test]
async fn read_json_surfaces_error_key_when_message_absent() {
// The task API answers with `error` alone — see
// gitlawb-node/src/api/tasks.rs `{"error":"task not found"}`. Reading only
// `message` blanked those denials to the generic "request failed", so the
// user was told nothing about why the call failed.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 404, r#"{"error":"task not found"}"#, true).await;
let err = read_json(resp, "task show").await.unwrap_err().to_string();
assert!(err.contains("404"), "err={err}");
assert!(err.contains("task not found"), "reason not surfaced: {err}");
}
#[tokio::test]
async fn read_json_prefers_message_over_error_key() {
// The shared AppError envelope carries both; `message` is the human text
// and must win over the machine code in `error`.
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
500,
r#"{"error":"db_error","message":"database unavailable"}"#,
true,
)
.await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("database unavailable"), "err={err}");
assert!(
!err.contains("db_error"),
"machine code won over message: {err}"
);
}
#[tokio::test]
async fn read_json_sanitizes_error_key_text() {
// The `error` fallback goes through the same sanitizer as `message`; a
// hostile node must not reach the terminal through the new path. The
// controls are JSON \u escapes so the body stays VALID JSON — embedding
// them raw would fail the parse and pass this test vacuously.
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
404,
r#"{"error":"a\u001b[31mb\u0007c\u202ed"}"#,
true,
)
.await;
let err = read_json(resp, "task show").await.unwrap_err().to_string();
// Proves the parse succeeded and the `error` key was read, not the
// "request failed" fallback (which would make the asserts below vacuous).
assert!(err.contains("task show failed (404"), "err={err}");
assert!(
!err.contains("request failed"),
"fell back, sanitizer untested: {err}"
);
assert!(
err.contains('a') && err.contains('d'),
"text dropped: {err}"
);
assert!(!err.contains('\u{1b}'), "ESC leaked: {err:?}");
assert!(!err.contains('\u{7}'), "BEL leaked: {err:?}");
assert!(!err.contains('\u{202e}'), "bidi override leaked: {err:?}");
}
/// #186 (F2): the non-2xx error path must read a CAPPED body, not buffer and
/// parse the whole thing — a hostile node can stream an arbitrarily large valid
/// JSON error. Place the `message` field AFTER more than the cap of padding: the
/// bounded read stops before it, so it must be ABSENT from the surfaced error.
/// RED before the fix (`resp.json()` parses the full body → the far message
/// appears), GREEN after (capped read truncates → fallback → far message gone).
/// The status is still surfaced, so a legitimate small error is unaffected.
#[tokio::test]
async fn read_json_bounds_the_error_body_read() {
let mut server = Server::new_async().await;
// >8 KiB of padding, then the marker message at the very end of the body.
let big = format!(
r#"{{"padding":"{}","message":"FARMARKER_BEYOND_CAP"}}"#,
"A".repeat(16 * 1024)
);
let resp = response_for(&mut server, 500, &big, true).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("500"), "bounded read still errors: {err}");
assert!(
!err.contains("FARMARKER_BEYOND_CAP"),
"the error body read must be capped: a message beyond the cap must not be surfaced, err={err}"
);
}
/// #186 (F2): an EMPTY non-2xx body is its own input class — the capped read
/// yields "", the JSON parse fails, and it falls back to the status alone
/// ("request failed"), never a panic or a spurious success.
#[tokio::test]
async fn read_json_errs_on_empty_non_2xx_body() {
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 502, "", false).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("502"), "err={err}");
assert!(err.contains("request failed"), "err={err}");
}
}
+1 -1
View File
@@ -154,7 +154,7 @@ async fn cmd_new_with_reader(
println!(" DID: {did}");
println!(" Key: {}", path.display());
println!();
println!(" Your DID is your identity on the gitlawb network.");
println!(" Your DID is your identity on the Twigpine network.");
println!(" Keep your key file safe — it cannot be recovered if lost.");
Ok(())
+7 -3
View File
@@ -1,7 +1,7 @@
//! `gl init` — zero-to-push in one command.
//!
//! Detects or initializes a git repo, ensures an identity exists,
//! registers with the node, creates a remote repo, adds the gitlawb
//! registers with the node, creates a remote repo, adds the Twigpine
//! remote, and pushes.
use anyhow::{Context, Result};
@@ -19,7 +19,11 @@ pub struct InitArgs {
pub name: Option<String>,
/// Node URL to register with
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
pub node: String,
/// Identity directory (default: ~/.gitlawb)
@@ -151,7 +155,7 @@ pub async fn run(args: InitArgs) -> Result<()> {
println!(" Repository created.");
}
// 5. Add gitlawb remote
// 5. Add Twigpine remote
let did_short = did.to_string();
let did_short = did_short.split(':').next_back().unwrap_or(&did_short);
let remote_url = format!("gitlawb://{did_short}/{repo_name}");
+2123 -30
View File
File diff suppressed because it is too large. Load diff
+136 -58
View File
@@ -3,7 +3,7 @@
use anyhow::{Context, Result};
use chrono::Utc;
use clap::{Args, Subcommand};
use serde_json::{json, Value};
use serde_json::json;
use std::path::PathBuf;
use uuid::Uuid;
@@ -32,7 +32,11 @@ pub enum IssueCmd {
/// Issue body (optional)
#[arg(long, short)]
body: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -41,7 +45,11 @@ pub enum IssueCmd {
List {
/// Repository in <owner>/<repo> or <repo> format
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -52,7 +60,11 @@ pub enum IssueCmd {
repo: String,
/// Issue ID
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -63,7 +75,11 @@ pub enum IssueCmd {
repo: String,
/// Issue ID
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -74,7 +90,11 @@ pub enum IssueCmd {
id: String,
#[arg(long, short)]
body: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -83,7 +103,11 @@ pub enum IssueCmd {
Comments {
repo: String,
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -142,12 +166,7 @@ async fn resolve_repo(
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(node, None);
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let info = crate::http::read_json(client.get("/").await?, "node info").await?;
let did = info["did"].as_str().context("node info missing 'did'")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -198,13 +217,7 @@ async fn cmd_create(
.post(&path, &request_body)
.await
.context("failed to connect to node")?;
let status = resp.status();
let result: Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("create issue failed ({status}): {msg}");
}
let result = crate::http::read_json(resp, "create issue").await?;
let id = result["id"].as_str().unwrap_or("?");
println!("✓ Created issue #{id}");
@@ -221,12 +234,7 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/issues");
let resp: Value = client
.get_authed(&path)
.await?
.json()
.await
.context("failed to list issues")?;
let resp = crate::http::read_json(client.get_authed(&path).await?, "issues").await?;
let issues = resp["issues"].as_array().cloned().unwrap_or_default();
@@ -243,7 +251,7 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let status = issue["status"].as_str().unwrap_or("?");
let created = issue["created_at"]
.as_str()
.map(|s| &s[..10])
.map(|s| crate::text::truncate(s, 10))
.unwrap_or("?");
let icon = match status {
"open" => "○",
@@ -264,13 +272,7 @@ async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>)
.get_authed(&path)
.await
.context("failed to connect to node")?;
let status = resp.status();
let issue: Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = issue["message"].as_str().unwrap_or("issue not found");
anyhow::bail!("show failed ({status}): {msg}");
}
let issue = crate::http::read_json(resp, "show").await?;
let title = issue["title"].as_str().unwrap_or("(no title)");
let status = issue["status"].as_str().unwrap_or("?");
@@ -301,13 +303,7 @@ async fn cmd_close(repo: String, id: String, node: String, dir: Option<PathBuf>)
.post(&format!("{path}/close"), &body)
.await
.context("failed to connect to node")?;
let status = resp.status();
let result: Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("close issue failed ({status}): {msg}");
}
let _ = crate::http::read_json(resp, "close issue").await?;
println!("✗ Closed issue {id}");
Ok(())
@@ -332,13 +328,7 @@ async fn cmd_issue_comment(
)
.await
.context("failed to connect to node")?;
let code = resp.status();
let result: Value = resp.json().await.context("invalid JSON")?;
if !code.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("comment failed ({code}): {msg}");
}
let _ = crate::http::read_json(resp, "comment").await?;
println!("· Comment posted on issue {id}");
Ok(())
@@ -353,14 +343,15 @@ async fn cmd_issue_comments(
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = signed_client(&node, dir.as_deref());
let resp: Value = client
.get_authed(&format!(
"/api/v1/repos/{owner}/{name}/issues/{id}/comments"
))
.await?
.json()
.await
.context("invalid JSON")?;
let resp = crate::http::read_json(
client
.get_authed(&format!(
"/api/v1/repos/{owner}/{name}/issues/{id}/comments"
))
.await?,
"issue comments",
)
.await?;
let comments = resp["comments"].as_array().cloned().unwrap_or_default();
if comments.is_empty() {
@@ -374,10 +365,13 @@ async fn cmd_issue_comments(
let author_short = author
.split(':')
.next_back()
.map(|s| &s[..s.len().min(8)])
.map(|s| crate::text::truncate(s, 8))
.unwrap_or("?");
let cbody = c["body"].as_str().unwrap_or("");
let created = c["created_at"].as_str().map(|s| &s[..10]).unwrap_or("?");
let created = c["created_at"]
.as_str()
.map(|s| crate::text::truncate(s, 10))
.unwrap_or("?");
println!(" · {author_short} ({created})");
println!(" {cbody}");
println!();
@@ -448,6 +442,34 @@ mod tests {
.unwrap();
}
#[tokio::test]
async fn test_cmd_list_survives_malformed_timestamps() {
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
// A short timestamp used to panic on `&s[..10]`; a timestamp whose
// byte 10 is mid-char panics even with a len() guard.
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/issues$".to_string()),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"issues":[{"id":"a","title":"short ts","status":"open","created_at":"2026"},{"id":"b","title":"mb ts","status":"open","created_at":"2026-08-1é5T00:00:00Z"},{"id":"c","title":"empty ts","status":"open","created_at":""}]}"#)
.create_async()
.await;
cmd_list(
"myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_create_success() {
let dir = TempDir::new().unwrap();
@@ -490,6 +512,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.expect(1)
.create_async()
.await;
@@ -504,6 +527,8 @@ mod tests {
assert!(result.is_err());
assert!(result.unwrap_err().to_string().contains("repo not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy is_err() vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -551,6 +576,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"issue not found"}"#)
.expect(1)
.create_async()
.await;
@@ -564,6 +590,8 @@ mod tests {
.unwrap_err();
assert!(err.to_string().contains("show failed"), "got: {err}");
assert!(err.to_string().contains("issue not found"), "got: {err}");
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -612,6 +640,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"issue not found"}"#)
.expect(1)
.create_async()
.await;
@@ -624,6 +653,8 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("close issue failed"), "got: {err}");
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -670,6 +701,7 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"issue not found"}"#)
.expect(1)
.create_async()
.await;
@@ -683,6 +715,8 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("issue not found"), "got: {err}");
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -740,4 +774,48 @@ mod tests {
.await
.unwrap();
}
#[tokio::test]
async fn cmd_list_surfaces_denial_not_empty() {
// A gated 404 on the issue-list read must Err, not print "No issues".
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/alice/secret/issues$".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list("alice/secret".to_string(), server.url(), None).await;
assert!(result.is_err(), "issue list must Err on a gated 404");
// Prove the gated issues path was actually requested: without this, an
// unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
#[tokio::test]
async fn resolve_repo_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the GET / node-info
// fetch. A gated 404 there must Err (surfacing the status), proving the
// read_json conversion is load-bearing rather than silently ignored.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = resolve_repo("noslash", &server.url(), Some(dir.path()))
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
}
+8 -7
View File
@@ -30,6 +30,7 @@ mod star;
mod status;
mod sync;
mod task;
mod text;
mod ucan_cmd;
mod visibility;
mod webhook;
@@ -38,7 +39,7 @@ mod whoami;
#[derive(Parser)]
#[command(
name = "gl",
about = "gitlawb CLI — identity, repos, MCP server",
about = "Twigpine CLI — identity, repos, MCP server",
version,
propagate_version = true
)]
@@ -49,16 +50,16 @@ struct Cli {
#[derive(Subcommand)]
enum Commands {
/// Manage your gitlawb identity (DID + keypair)
/// Manage your Twigpine identity (DID + keypair)
Identity {
#[command(subcommand)]
cmd: identity::IdentityCmd,
},
/// Register this agent with a gitlawb node
/// Register this agent with a Twigpine node
Register(register::RegisterArgs),
/// Clone a gitlawb repo, handling private subtrees cleanly
/// Clone a Twigpine repo, handling private subtrees cleanly
Clone(clone::CloneArgs),
/// Manage repositories
@@ -85,10 +86,10 @@ enum Commands {
/// Manage webhooks for a repository
Webhook(webhook::WebhookArgs),
/// Mirror a public GitHub/GitLab/any-git repo into gitlawb
/// Mirror a public GitHub/GitLab/any-git repo into Twigpine
Mirror(mirror::MirrorArgs),
/// MCP server — expose gitlawb tools to LLM agents
/// MCP server — expose Twigpine tools to LLM agents
Mcp(mcp::McpArgs),
/// Sync repos from peer nodes (HTTP fallback for p2p gossip)
@@ -100,7 +101,7 @@ enum Commands {
/// Register and resolve names on Base L2
Name(name::NameArgs),
/// Check your gitlawb installation and connectivity
/// Check your Twigpine installation and connectivity
Doctor(doctor::DoctorArgs),
/// Zero-to-push in one command — init repo, identity, register, remote
+927 -169
View File
File diff suppressed because it is too large. Load diff
Loaded 100 of 168 files, more files were not shown because too many files have changed in this diff. Show more