Compare commits

..
Author SHA1 Message Date
beardthelion f577142a35 ci: trigger the PR checks on the retargeted base 2026-11-04 16:48:32 -06:00
beardthelion 5d182be9c8 test(node): pass a libp2p identity to p2p::start in the gossip delivery tests 2026-11-04 16:24:34 -06:00
beardthelion 955023e328 fix(node): dedupe the ref-update row on the replay key, not on a random id
The reservation releases on a failed write so a republish can repair the
half that never landed. But each ingest minted a fresh row id, so a retry
after a partial failure (row stored, queue write failed) stored the same
signed content a second time. A replay in a degraded state could repeat a
landed side effect until a budget refused it.

The row id for a verified event is now derived from the replay key, so the
retry's insert dedupes on the existing ON CONFLICT instead of duplicating.
The queue entry still retries, which is the repair the release exists for.
Unsigned events keep a random id: they carry no replay key by design, and
deduplicating unauthenticated content would let an attacker pre-send a
victim's expected event and have the genuine one dropped.

Two existing tests pinned the duplicate write and now assert the deduped
shape: the partial-failure republish repairs the queue entry while the
row stays at one, and the restart readmission's residual effect is a
duplicate queue entry (a refetch), not a second record. The new test
asserts both the retry reaches the writes (WriteFailed, not Replayed) and
the row count stays at one.
2026-11-04 16:18:21 -06:00
beardthelion c65bd616d5 fix(node): harden the gossip rejection path against unauthenticated bytes
Two findings on the pre-auth reject path:

- An unsigned event under require_signed was resolved and slug-validated
  before the flag refused it, so a hostile flood bought DID resolution and
  a rejection reason built from attacker-chosen fields under only the
  loose pre-parse brake. The unsigned refusal now runs immediately after
  the version gate, before any field is read.

- The swarm loop logged IngestOutcome::Rejected and WriteFailed reasons
  verbatim, and several producers embed wire fields (the claimed node_did
  in resolve failures, for one). A signed event carrying a control-bearing
  node_did reached the operator log with escape sequences intact and no
  length bound. Both warn sites now render through sanitize_for_log, so
  every present and future Rejected producer is bounded and control-free
  at the one sink.

The live-mesh test now also publishes a signed event with a 4096-char
control-bearing node_did and asserts the warn stream shows the refusal
with no escape byte and no run past the sanitizer ceiling. A unit test
pins the new ordering by asserting the exact fixed refusal for an
unsigned event with a hostile node_did.
2026-11-04 16:18:21 -06:00
beardthelion 8dcb4934ff test(node): drive the live-mesh publish and refusal path end to end
Two real swarms on QUIC loopback with real signatures and real Postgres:
a signed ref-update from one node is admitted and stored by the other, a
re-delivery of the same signed content under a malleated wire encoding is
refused by the replay guard, and an unsigned event is refused under
require_signed. Both refusals are asserted in the warn stream, and a
control event landing after them proves the mesh still delivers.

This closes the seams ref_update_publish_args names as needing a live
swarm: the PublishRefUpdate select! arm dispatching to sign+publish, and
require_signed reaching the receive path. It needs a test-only command
variant, PublishRawRefUpdate, since the production publish path signs and
serializes itself and can never put a malleated or unsigned event on the
wire.

Still uncovered and named in the test's doc comment: the run_ingest_sweep
select! arm on its 300-second interval, and the main.rs half of the flag
wiring above p2p::start.
2026-11-04 16:18:21 -06:00
beardthelion c7e8293aeb test(node): pass the replay guard to the post-fork shed tests
The two ingest tests added on the target branch after this PR forked
(an_over_budget_author_is_shed_without_a_peer_lookup and
the_early_author_check_does_not_track_an_unseen_did) still call
ingest_ref_update with the pre-guard signature. Thread a fresh
ReplayGuard through each call, matching every other test site.
2026-11-04 16:18:21 -06:00
Kevin Codex 904e49f3bb fix(node): render rejected gossip timestamps bounded before logging
DateTime::parse_from_rfc3339 accepts arbitrarily many fractional-second
digits, so a signed timestamp can parse successfully and still carry
megabytes of attacker-chosen wire bytes into the TooOld / TooFarFuture
refusal details, which the swarm loop logs verbatim. Only the
parser-error branch sanitized. The root cause was treating successful
parsing as proof the wire representation is safe to expose.

The TooOld and TooFarFuture variants now carry the PARSED instant and
the detail renders its canonical to_rfc3339() form - bounded by this
build's formatter and control-free by construction - while the
unparseable arm keeps sanitize_for_log on the wire string. All three
arms go through one freshness_refusal_detail helper so the invariant
has a single home, plus a regression test driving an overlong but
parseable fractional stamp through both directions.
2026-11-04 16:18:21 -06:00
beardthelion 16adbd8ade fix(review): reconcile the replay guard with the unsigned-admission split
Integration only, from rebasing this branch onto #325's current head. #325 now
returns IngestOutcome::UnsignedAdmitted where it used to return Accepted for an
unsigned rolling-upgrade admission, and this branch predates that split.

Three sites. The seen-set bypass test asserted Accepted on both unsigned
deliveries and on the stale-timestamp case; both now expect UnsignedAdmitted,
and the properties under test are unchanged, that unsigned bytes are admitted
twice rather than deduplicated and that the freshness window does not reach
them. The warn-only-on-admission test's budget-spent case gained the
ReplayGuard argument the signature now takes, with a fresh guard because that
case drives an unsigned event the replay block skips.

The ingest match also grows an arm rather than changing one: `None if unsigned`
returns UnsignedAdmitted without settling a reservation, since the replay block
is gated on `verified` and an unsigned admission never holds one.
2026-11-04 16:18:21 -06:00
beardthelion 70cefe3b0c fix(core): stop an attacker-supplied created timestamp from overflowing the window check
created is parsed as an unrestricted i64 straight from the Signature-Input
header, so the sender picks it. At i64::MIN the past-side subtraction overflows,
which panics a debug build and wraps a release one into a value that can read as
inside the window. Confirmed by execution before the fix: 'attempt to subtract
with overflow'.

Saturating subtraction answers correctly at both ends, since a timestamp that
far out is refused by whichever side it saturates toward. A test drives all four
extremes of the type and asserts each is refused by a named direction; reverting
to plain subtraction reddens it on the overflow.

The symmetric abs() form this replaced had the same hazard, so splitting the
window into two comparisons did not introduce it, but it did not remove it
either. Found by a cross-family review pass after six same-family reviewers and
I had all read the line.
2026-11-04 16:18:21 -06:00
beardthelion ccebc3fa54 test(node): close the replay guard's remaining proof gaps, and fix the sibling window
Seven tests for the six gaps a review found. The reservation's release path was
reasoned rather than executed: the drop guard was only ever proven through an
early refusal, while the case its own doc comment names, a transient write
failure burning the event's slot, was never driven. Both write directions now
are. An expired entry at capacity must be replaced in place rather than answer
Saturated, confirm on an already-swept entry is pinned as a deliberate no-op,
and the single-critical-section shape is now driven concurrently rather than
asserted sequentially. Restart behavior was documented but untested; a fresh
guard readmits a seen event and the freshness window still bounds it, which is
the composition that makes the restart exposure finite.

check_created in gitlawb-core used a symmetric abs() window, so a request
stamped 299 seconds ahead was accepted and a signer could roughly double a
signature's effective validity by stamping forward. It is now two comparisons
like the gossip path, 300s late and 60s early, with the error naming the
direction so a fast peer and a slow one need different operator action. There
was no future-direction test at all; there are now five covering both. Every
caller was checked and none depended on the symmetry.

Adding a freshness window obliges auditing the siblings, which is how this one
surfaced: the repo argued both ways in two files for a week.
2026-11-04 16:18:21 -06:00
beardthelion 04ba7a6792 fix(review): bound the saturated guard's rescan and the log it writes
At capacity the inline sweep ran on every event, so a full O(capacity) retain
happened under the lock once per message and reclaimed nothing when nothing had
expired. The guard that bounds replay became a CPU amplifier in exactly the
state an attacker drives toward. The sweep is now rate limited to once a second;
the periodic sweep still reclaims on its own cadence, so only the redundant
rescans go away.

An unparseable timestamp was echoed into the refusal detail verbatim. That value
is attacker-controlled and arbitrary length, so it reached a warn! as both a
log-injection and an unbounded-size sink. Only that arm needs sanitizing; the
other two ran through the parser first.

The capacity rationale cited a count of registered DIDs, which this same file
says elsewhere an attacker mints freely through the announce path, so it was not
a bound at all. It now cites the bound that is real: reaching saturation costs a
hundred thousand durable rows and as many sync enqueues inside one retention
horizon, which the database makes loud.

ingest_now was read twice per ingest while its own doc comment claimed the two
layers share one reading, which is the invariant the retention derivation rests
on. Now read once and passed to both.

Also: the Unparseable outcome is driven through ingest rather than only as a
pure function, the periodic sweep is observable without a live swarm, the
restart exposure is written down where the rest of the tradeoffs already are,
and the saturation-counter lock covers every test that can reach Saturated
rather than the two I first found.
2026-11-04 16:18:21 -06:00
beardthelion 46474bcd23 fix(node): bound replay of authenticated gossip ref-update events
Wires the freshness check and the replay guard into ingest, immediately after
signature verification and above the author debit. The lower placement, just
above the writes, stops the duplicate row and the duplicate sync but still lets
every replay drain the victim author's budget, which is the harm the defect
names: a captured signature replayed 500 times empties the victim's window and
their next genuine push is refused.

The guard runs on the verified path only. Unsigned event bytes are predictable,
so applying it there would let an attacker pre-send a victim's expected event
and have the genuine one dropped as a replay, which is a censorship primitive
rather than a defense.

A replay flood still costs a parse and one Ed25519 verify, because the guard
has to sit below verification for the reason above. What it removes is the
peer_exists round trip, the victim's author debit, the ref-update row and the
sync enqueue.

The existing author-budget test signed one event and ingested the same bytes
five hundred times, so it had to re-sign per iteration to keep exercising the
budget under a shared guard. Giving it a fresh guard per call would have kept
it green while gutting the property it exists to prove. Its over-budget probe
needed the same treatment, since the replay gate sits above the author gate and
would have refused the burst's last bytes before the budget assertion ran.

Both saturation tests now take a shared lock so each keeps an exact assertion
on a process-wide counter; a lower bound would stay green if the fail-open
branch ever double-counted.
2026-11-04 16:18:21 -06:00
beardthelion 9f4efcc90b feat(node): add the freshness check and replay guard, not yet wired
Builds the two layers that bound replay of an authenticated ref-update, with
nothing calling them yet; the ingest path is wired in the next commit.

The freshness check is deliberately two-directional rather than an absolute
delta: an abs() window admits an event stamped up to the window ahead and pins
its seen-set slot until the clock catches up. Producers were enumerated before
settling the unparseable arm; the sole production publish site emits RFC-3339,
so an unparseable timestamp is refused rather than admitted.

The replay guard keys on SHA-256 of the canonical signing bytes, not the raw
wire bytes, because one signature verifies against many encodings and only the
canonical form collapses them to a single key. A golden digest is frozen for
the pre-version artifact, and the same constant is asserted for its v-injected
twin, so the collapse is pinned rather than described. Reservations settle
through a drop guard so only a confirmed entry outlives the ingest call, which
keeps a transient write failure from permanently burning an event's slot.

Replayed and StaleTimestamp are separate outcomes because they diagnose
different conditions, a mesh replay against a broken clock or a healing
partition, and folding them would be the same observability lie the unsigned
shed variant already exists to avoid.
2026-11-04 16:18:20 -06:00
142 changed files with 3133 additions and 73699 deletions

No files matched your search

-31
View File
@@ -21,25 +21,6 @@ ignore = [
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
# lru 0.16.3 (use-after-free in pop()). Reachable via alloy. The
# lockfile holds only 0.16.3 — the aws-sdk-s3 path that once pulled
# 0.12.5 no longer does. The RUSTSEC-2026-0253 advisory covers this
# range, so the ignore stays load-bearing until a fixed 0.16.x lands;
# REMOVE when the locked version is fixed or lru leaves the tree.
# Retirement owner: node maintainers, enforced by the weekly
# audit-schedule drift guard ("Fail if lru moved while its ignore
# remains"), which fails if either locked version moves or lru
# leaves the tree. No separate tracking issue exists; file one
# against the lru upgrade if this waiver needs discussion outside
# the schedule job.
"RUSTSEC-2026-0253",
#
# Round-3 P2 (reviewer 2): the previous version of this file
# ignored RUSTSEC-2026-0258 for h2 0.4.13, but the lockfile
# already has h2 0.4.18 (verified 2026-08-31: cargo tree -p h2
# shows 0.4.18) — the fix is in. The ignore was stale and would
# have silently re-accepted the advisory if h2 dropped back to
# 0.4.13. Removed.
# crypto-bigint 0.5.5 (secret-dependent branches in operations meant
# to be constant-time). REACHABLE accepted risk, on two paths:
# - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256
@@ -58,16 +39,4 @@ ignore = [
# (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint
# below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`.
"RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix)
# libsqlite3-sys 0.30.1 (bundled SQLite WAL uninitialized-memory disclosure,
# CVE-2026-119375). Present in Cargo.lock because sqlx resolves sqlx-sqlite
# unconditionally; sqlx is built postgres-only
# (crates/gitlawb-node/Cargo.toml), so sqlx-sqlite and its bundled SQLite C
# source are never compiled or linked (`cargo tree -i sqlx-sqlite` prints
# nothing) and there is no reachable WAL open to attack. The fix requires
# libsqlite3-sys >= 0.39, but sqlx-sqlite 0.8.6 pins ^0.30.1; a compatible
# bump needs a sqlx upgrade, not a lockfile refresh. REMOVE this ignore if
# sqlx is ever built with the `sqlite` feature or any other libsqlite3-sys
# consumer enters the build.
"RUSTSEC-2026-0459", # libsqlite3-sys bundled SQLite CVE (not linked in our build)
]
+6 -71
View File
@@ -6,16 +6,8 @@
# Path to the node's Ed25519 keypair PEM file. Must name a key file, not a
# directory: a trailing slash, a final `.` or `..`, and `..` traversal are
# refused so the path cannot retarget a parent. A `~/` value must stay under
# home after expansion, and the path must be absolute: its parent is also
# the delegation store that `gl ucan import` writes and `git-remote-gitlawb`
# reads, and the two do not share a working directory, so a relative path
# sends them to different places. Bare `identity.pem` under the default
# config dir is still legal.
# home after expansion. Bare `identity.pem` is still legal.
# Generate with: gl identity new
# Leaving it unset uses ~/.gitlawb/identity.pem, which is the shared default
# for both `gl` and `git-remote-gitlawb`. The bare-filename form is for the
# node's own service config; client tooling should inherit the default or use
# an absolute path.
GITLAWB_KEY=/data/keys/identity.pem
# Path to the node's persistent libp2p identity key file. Must include a
@@ -47,12 +39,6 @@ GITLAWB_KEY=/data/keys/identity.pem
# Publicly reachable URL of this node (used in peer announcements)
GITLAWB_PUBLIC_URL=https://your-node.example.com
# Base URL for the web view of repos on this node (used by `gl` to print a
# working View: link after repo creation). Omit for nodes with no web front-end.
# Distinct from GITLAWB_PUBLIC_URL: that is API reachability for peers;
# this is browser reachability for humans.
# GITLAWB_WEB_URL=https://gitlawb.com
# ── Server ────────────────────────────────────────────────────────────────
GITLAWB_HOST=0.0.0.0
GITLAWB_PORT=7545
@@ -160,10 +146,9 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# push to every repository, private ones included. On by default.
#
# Set to false only for a rolling upgrade whose pushers are not yet the repo
# owner. A holder of an owner-rooted, time-bounded git/push UCAN naming that
# repository can push while this is on — see docs/RUN-A-NODE.md for issuing and
# importing one. A CI key with no such delegation counts as a non-owner and
# cannot push.
# owner. Note that delegated and CI keys count as non-owners: UCAN git/push is
# verified but not yet honored for authorization, so they cannot push while this
# is on. See docs/RUN-A-NODE.md.
GITLAWB_ENFORCE_OWNER_PUSH=true
# Comma-separated libp2p multiaddrs.
@@ -171,8 +156,8 @@ GITLAWB_ENFORCE_OWNER_PUSH=true
GITLAWB_P2P_BOOTSTRAP=
# ── Access control ────────────────────────────────────────────────────────
# Reserved and currently inert. Repo reads are gated per repository by
# is_public and path-scoped visibility rules; this flag changes nothing.
# Reserved for private-read mode. Public/private repo read enforcement is not
# wired in the current live release; do not rely on this for private repositories.
GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
@@ -335,19 +320,6 @@ GITLAWB_IPFS_RESOLVE_BUDGET_SECS=10
# from the concurrency caps above). 0 disables. Default 600.
GITLAWB_IPFS_RATE_LIMIT=600
# Per-client-IP rate limit for the anonymous task read routes
# (GET /api/v1/tasks, GET /api/v1/tasks/{id}), in requests per hour.
# Both are publicly reachable (optional_signature). GET /api/v1/tasks runs
# collect_visible_tasks and returns a visibility-filtered page;
# GET /api/v1/tasks/{id} runs get_visible_task and returns an opaque 404 when
# the task is hidden or missing. These reads can require task, repository,
# and visibility-rule queries even when no task is returned, so the brake
# bounds the cost of anonymous probes. Keyed on the resolved client IP via
# GITLAWB_TRUSTED_PROXY. 0 disables. Default: 1200 (a list page
# followed by per-task reads is a normal client pattern, so this sits above
# the /ipfs budget). The GraphQL/WS task-read brake shares this budget.
GITLAWB_TASK_READ_RATE_LIMIT=1200
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID
@@ -356,14 +328,6 @@ GITLAWB_TASK_READ_RATE_LIMIT=1200
# brake caps a single-source flood. Uses GITLAWB_TRUSTED_PROXY below to resolve
# the client IP. 0 disables. Default 120.
GITLAWB_CREATE_RATE_LIMIT=120
# Per-DID creation requests allowed inside the window below (repo create,
# issue create, PR create, register, fork). 0 is rejected at startup: it would
# silently disable the limiter. Default 10.
GITLAWB_RATE_LIMIT_MAX=10
# Window the per-DID creation count above applies over, in seconds. 0 is
# rejected at startup: every timestamp would prune immediately, disabling the
# limiter. Default 3600.
GITLAWB_RATE_LIMIT_WINDOW_SECS=3600
# ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─
# /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from
@@ -389,35 +353,6 @@ GITLAWB_TRUSTED_PROXY=
# Enable automatic background sync from known peers
GITLAWB_AUTO_SYNC=false
# ── External upstream mirroring ───────────────────────────────────────────
# Default-off scheduled fetch worker for repositories configured as INBOUND.
# Enabling requires GITLAWB_ENFORCE_OWNER_PUSH=true; the node refuses to start
# otherwise. The worker updates branches and tags atomically while preserving
# Gitlawb-owned review/internal refs. Enabling also requires Git >= 2.37 on
# the host for http.curloptResolve DNS pinning; the node refuses to start
# when the installed Git is older.
GITLAWB_UPSTREAM_MIRROR_ENABLED=false
# Pause after each complete scan. Must be 5..=86400 seconds.
GITLAWB_UPSTREAM_MIRROR_INTERVAL_SECS=300
# Deadline for the external fetch child only; the preflight probes run on
# their own short bound. Must be 1..=3153600000 seconds.
GITLAWB_UPSTREAM_MIRROR_FETCH_TIMEOUT_SECS=600
# Keyset page size; every page is processed before the next scan delay.
GITLAWB_UPSTREAM_MIRROR_PAGE_SIZE=100
# ── Reconciliation sweep ─────────────────────────────────────────────────
# Periodic durability sweep: re-derives the public pin set and the withheld-blob
# recovery set each hour and fills gaps so a dropped replication job never means
# data loss. Defaults to true; set to false to disable the sweep even when a pin
# backend (IPFS/Pinata) is configured.
#
# Phase-capability matrix:
# - Public pin repair: IPFS-only, Pinata-only, or both (requires the
# respective backend to be configured).
# - Encrypted recovery repair: requires local IPFS (GITLAWB_IPFS_API).
# Pinata-only nodes reconcile public pins only; encrypted recovery
# reconciliation is not performed.
GITLAWB_RECONCILIATION_SWEEP=true
# ── iCaptcha proof-of-intelligence gate ───────────────────────────────────
# Optional gate on create_repo + register: require callers to present an
# iCaptcha proof (X-ICaptcha-Proof header) earned at icaptcha.gitlawb.com.
-24
View File
@@ -86,27 +86,3 @@ jobs:
exit 1
fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version."
# Drift guard: the RUSTSEC-2026-0253 (lru) ignore is only valid while
# the lockfile holds exactly the version it was written for (0.16.3
# via alloy). Any move — an upgrade that may carry the upstream fix,
# or removal of the dependency — must revisit the ignore. Retirement
# owner: node maintainers, via this weekly job (no separate tracking
# issue; see the waiver comment in .cargo/audit.toml).
- name: Fail if lru moved while its ignore remains
run: |
set -euo pipefail
versions="$(grep -A1 'name = "lru"' Cargo.lock | grep '^version' | cut -d'"' -f2 | sort -u | tr '\n' ' ')"
echo "lru versions in Cargo.lock: ${versions:-not present}"
lru_ignore_present=false
if grep -q 'RUSTSEC-2026-0253' .cargo/audit.toml; then
lru_ignore_present=true
fi
# Drift = the lockfile no longer matches what the ignore assumes:
# the version moved (possible fix) or lru left the tree
# (dead ignore). Both must fail for explicit re-triage.
if [ "$lru_ignore_present" = true ] && [ "${versions}" != "0.16.3 " ]; then
echo "::error::lru versions are [${versions:-absent from Cargo.lock}] but the RUSTSEC-2026-0253 ignore is still in .cargo/audit.toml. Re-triage the waiver against the advisory's fixed range and either drop the ignore or update the pinned set."
exit 1
fi
echo "No drift: lru ignore is consistent with the pinned versions."
-100
View File
@@ -1,100 +0,0 @@
name: CodeQL
# Advanced-setup replacement for CodeQL default setup. Default setup analyzes
# every detected language on every PR, including Swift (macos-app/), which
# autobuilds on a macOS runner for ~13 minutes per PR and has hung for 55 on
# main; that single job was the longest check on most PRs. Here a PR analyzes
# only the languages it touches (scripts/ci-surfaces.sh), while push to main
# and the weekly schedule analyze every language, so main's code-scanning
# baseline stays complete.
#
# Requires CodeQL default setup to be DISABLED (Settings -> Code security ->
# Code scanning): GitHub rejects advanced-setup uploads while it is enabled.
#
# A PR that skips a language may show "configuration not found" on the CodeQL
# check for that language; that is informational and expected.
on:
pull_request:
branches: [main]
push:
branches: [main]
schedule:
- cron: "17 4 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: codeql-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
changes:
name: changed languages
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
codeql: ${{ steps.surfaces.outputs.codeql }}
matrix: ${{ steps.surfaces.outputs.codeql_matrix }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
fetch-depth: 2
- name: Map changed paths to languages
id: surfaces
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "pull_request" ]; then
# -z: literal paths; plain --name-only C-quotes unusual names.
git diff -z --name-only --no-renames HEAD^1 HEAD > changed-paths
tr '\0' '\n' < changed-paths >&2
scripts/ci-surfaces.sh < changed-paths > surfaces.out
else
scripts/ci-surfaces.sh --all > surfaces.out
fi
cat surfaces.out
grep -E '^(codeql|codeql_matrix)=' surfaces.out >> "$GITHUB_OUTPUT"
analyze:
name: Analyze (${{ matrix.language }})
needs: changes
if: needs.changes.outputs.codeql == 'true'
runs-on: ${{ matrix.runner }}
# Normal runs take <=14 min (Swift autobuild is the longest); cap a hung
# autobuild instead of letting it hold a runner for an hour.
timeout-minutes: 30
permissions:
contents: read
security-events: write
actions: read
packages: read
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.changes.outputs.matrix) }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Initialize CodeQL
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- name: Autobuild
if: matrix.build-mode == 'autobuild'
uses: github/codeql-action/autobuild@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
category: "/language:${{ matrix.language }}"
+3 -114
View File
@@ -30,62 +30,8 @@ env:
RUST_BACKTRACE: 1
jobs:
# Decide which checks this event needs. On pull_request, each job below runs
# only when the PR touches the surface it validates, and the cargo test /
# release-build jobs are scoped to the touched crates plus their reverse
# dependencies (scripts/ci-surfaces.sh). Every other event (push to main,
# merge_group, the release flow's workflow_dispatch) runs the full suite, so
# main and the merge queue keep whole-workspace coverage and main-red-alert
# still sees a complete run. A skipped job reports as "skipped", not green.
changes:
name: changed surfaces
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
rust: ${{ steps.surfaces.outputs.rust }}
rust_packages: ${{ steps.surfaces.outputs.rust_packages }}
windows: ${{ steps.surfaces.outputs.windows }}
windows_packages: ${{ steps.surfaces.outputs.windows_packages }}
docker: ${{ steps.surfaces.outputs.docker }}
audit: ${{ steps.surfaces.outputs.audit }}
core_deps: ${{ steps.surfaces.outputs.core_deps }}
release_versioning: ${{ steps.surfaces.outputs.release_versioning }}
release_tag: ${{ steps.surfaces.outputs.release_tag }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
# The pull_request checkout is the test-merge commit; depth 2 brings
# its first parent (the base tip) so the diff is exactly this PR.
fetch-depth: 2
# Every job's gate comes from the mapper, so pin its mapping first; a
# failure here fails this job and the run is red, not silently narrowed.
- name: Test the surface mapper
run: scripts/test-ci-surfaces.sh
- name: Map changed paths to surfaces
id: surfaces
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "pull_request" ]; then
# -z: literal paths; plain --name-only C-quotes unusual names.
git diff -z --name-only --no-renames HEAD^1 HEAD > changed-paths
tr '\0' '\n' < changed-paths >&2
scripts/ci-surfaces.sh < changed-paths > surfaces.out
else
scripts/ci-surfaces.sh --all > surfaces.out
fi
cat surfaces.out
cat surfaces.out >> "$GITHUB_OUTPUT"
fmt-clippy:
name: fmt + clippy
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -111,8 +57,6 @@ jobs:
release-tag-validation:
name: release tag validation
needs: changes
if: needs.changes.outputs.release_tag == 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
@@ -124,26 +68,8 @@ jobs:
- name: Test release tag resolver
run: scripts/test-resolve-release-tag.sh
triage-detector:
name: triage detector matrix
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
# pr-triage.yml runs on pull_request_target and never checks out PR
# code, so its inline detector can only be exercised here, where the
# proposed workflow body IS the checked-out one.
- name: Test inline-test detector
run: node scripts/test-pr-triage-detect.mjs
test:
name: test (${{ matrix.toolchain }})
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
@@ -188,17 +114,10 @@ jobs:
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
# `--workspace`, or `-p <crate>...` for the crates a PR touches plus
# every crate that depends on them.
PACKAGES: ${{ needs.changes.outputs.rust_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo test --locked $PACKAGES
run: cargo test --locked --workspace
build-release:
name: build --release
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
@@ -218,20 +137,10 @@ jobs:
key: release
- name: cargo build --release
env:
PACKAGES: ${{ needs.changes.outputs.rust_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo build --locked --release $PACKAGES
run: cargo build --locked --release --workspace
audit:
name: cargo audit
# cargo audit reads only Cargo.lock (+ .cargo/audit.toml), so a PR that
# leaves both unchanged cannot change its verdict; a newly published
# advisory against main is caught by the push-to-main run and the weekly
# Scheduled Audit instead of turning every unrelated PR red.
needs: changes
if: needs.changes.outputs.audit == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -291,8 +200,6 @@ jobs:
msrv:
name: MSRV (Rust 1.91)
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -316,12 +223,6 @@ jobs:
docker-build:
name: Docker build smoke test
# Runs when the image's own inputs change: the Dockerfile, the manifests
# its dependency layer copies, the lockfile, bootstrap-peers.json. A
# source-only Rust change is already compiled by `build --release` (stable)
# and `MSRV` (1.91, the image's toolchain); push to main still runs this.
needs: changes
if: needs.changes.outputs.docker == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -338,8 +239,6 @@ jobs:
test-windows:
name: test (windows, non-blocking)
needs: changes
if: needs.changes.outputs.windows == 'true'
runs-on: windows-latest
timeout-minutes: 45
# Non-blocking, mirroring the informational `beta` lane above. We ship `gl`
@@ -367,13 +266,7 @@ jobs:
key: windows
- name: cargo test (shipped Windows crates)
shell: bash
env:
# `-p gl -p git-remote-gitlawb`, or whichever of the two a PR affects.
PACKAGES: ${{ needs.changes.outputs.windows_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo test --locked $PACKAGES
run: cargo test --locked -p gl -p git-remote-gitlawb
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
@@ -383,8 +276,6 @@ jobs:
# cannot do. Regen instructions live in the allowlist header.
core-deps-purity:
name: gitlawb-core dependency purity
needs: changes
if: needs.changes.outputs.core_deps == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
@@ -417,8 +308,6 @@ jobs:
# not its author knew this gate existed.
release-versioning:
name: release versioning coverage
needs: changes
if: needs.changes.outputs.release_versioning == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
+3 -215
View File
@@ -76,221 +76,9 @@ jobs:
// line after horizontal whitespace, covering bare, cfg(test), namespaced,
// optional leading ::, and raw-identifier (r#test) forms. No head-file fetch
// or lexer: needs-tests is advisory, not a merge gate.
//
// The final path component may also carry `test` as an underscore-delimited
// segment, so third-party harnesses land without enumerating them one by one
// (#[test_case(1)], #[wasm_bindgen_test]). It is deliberately a segment and
// not a substring: a substring would also match #[contest] / #[latest], and a
// false positive here SUPPRESSES needs-tests rather than adding it. This
// heuristic has to fail loud, because a wrongly labeled PR gets corrected by
// the author while a wrongly cleared one is invisible. That trade costs the
// no-underscore harness names (#[rstest]), which stay unmatched on purpose.
//
// The segment classes are [A-Za-z0-9]+ rather than \w+ on purpose, so that `_`
// is only ever the literal delimiter. \w+ contains `_`, which makes the repeat
// ambiguous and backtracks exponentially: on this workflow's pull_request_target
// trigger f.patch is fork-controlled, so `#[test` followed by a long `_a` run is
// a permissionless stall of the triage job.
// Rust allows whitespace, line breaks, and comments between the
// attribute path and ] or (. A flat regex cannot decide that
// boundary: `//` runs to end of line, and block comments NEST, which
// is beyond regular languages — every enumeration of comment
// spellings so far has either dropped a legal separator (regressing
// real tests into needs-tests) or let unrelated patch records
// complete each other (clearing needs-tests from a testless PR, the
// invisible direction). So the closer is found by a token-aware
// scan instead: it consumes horizontal whitespace, `//`-to-newline,
// and depth-counted `/* */` runs, and accepts only when the first
// real token after the attribute path is `]` or `(`.
//
// Association is positional: the scan starts on the attribute's own
// added line and may continue ONLY through the immediately following
// added lines (a context line, hunk header, or removal breaks the
// chain). Unrelated `]`/`(` lines elsewhere in the patch can never
// complete a path they do not adjoin. The continuation is bounded;
// past the bound the scan gives up and reports NO inline test, which
// fails toward the loud label rather than the silent clearance.
//
// The detector body is fenced by the TRIAGE_DETECTOR markers so
// scripts/test-pr-triage-detect.mjs can extract and run it against
// the committed case matrix. Keep the fenced block self-contained.
// TRIAGE_DETECTOR_BEGIN
const TEST_ATTR_PATH =
"(?:cfg[ \\t]*\\([ \\t]*test[ \\t]*\\)|(?:::[ \\t]*)?(?:[\\w-]+[ \\t]*::[ \\t]*)*(?:r#)?(?:[A-Za-z0-9]+_)*test(?:_[A-Za-z0-9]+)*)";
// Anchored to the start of one added line; group 1 is whatever
// follows the path on that line, handed to the separator scan.
const TEST_ATTR_LINE = new RegExp(
"^[ \\t]*#\\[[ \\t]*" + TEST_ATTR_PATH + "([\\s\\S]*)$"
);
// An attribute split across more added lines than this is not a
// spelling anyone writes; refusing to scan further keeps the walk
// linear in the patch and fails toward the loud label.
const MAX_ATTR_CONTINUATION_LINES = 16;
// segs[0] is the remainder of the attribute's own line; each later
// entry is the text of one immediately following added line. True
// only when the first non-separator token across them is ] or (.
// Single forward pass, no backtracking: fork-controlled input.
function attrCloserFollows(segs) {
let depth = 0; // open block-comment nesting carried across lines
for (const seg of segs) {
let i = 0;
while (i < seg.length) {
if (depth > 0) {
if (seg.startsWith("/*", i)) { depth += 1; i += 2; }
else if (seg.startsWith("*/", i)) { depth -= 1; i += 2; }
else { i += 1; }
continue;
}
const c = seg[i];
if (c === " " || c === "\t") { i += 1; continue; }
if (seg.startsWith("//", i)) break; // comment to end of line
if (seg.startsWith("/*", i)) { depth = 1; i += 2; continue; }
return c === "]" || c === "(";
}
// Line exhausted inside separators; the newline is itself a
// separator, so continue on the next added line.
}
return false; // out of adjoined lines (or bound hit): stay loud
}
// Before the candidate scan, a pre-pass walks added patch lines
// tracking Rust lexical state: raw strings (r#*", br#*", cr#*"),
// block comments (/* */ with nesting), regular strings ("...",
// which can span newlines), and line comments (//). An added line
// that starts inside any of these is skipped as a candidate,
// because a #[test_case] appearing there is lexical non-code, not
// a real test attribute. State is only tracked across consecutive
// added lines and reset at any break (context line, removal,
// hunk header): context lines carry a partial view of the file,
// so a " or r#" or /* there might be inside a string that opened
// before the hunk's visible window, and treating it as an opener
// would mark every later added line as non-code (a false negative
// on a real #[test]). The tradeoff is that a non-code region
// opened by a context line is invisible: an added #[test] inside
// an existing raw string or block comment is scanned as code and
// may match (a false positive, silent direction). This is accepted
// because (1) the old detector had the same behavior, (2) the
// pattern is rare, and (3) needs-tests is advisory, not a merge
// gate. Scanning context lines to close this gap would re-introduce
// the false negative, which affects the far more common case of a
// context line carrying a stray " from a string that opened before
// the visible window. Single forward pass, O(n) in patch length.
function patchAddsInlineTest(patch) {
const lines = patch.split("\n");
const nonCodeStart = new Set();
let rawHashes = -1; // -1 = not in raw string; >=0 = # count
let blockDepth = 0;
let inString = false;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (!line) continue;
const prefix = line[0];
// Only scan added lines. Context and removal lines carry
// a partial view of the file: a " or r#" or /* on a context
// line might be inside a string that opened before the
// hunk's visible window, and treating it as an opener would
// mark every later added line as non-code (a false negative
// on a real #[test]). So lexical state is only tracked across
// consecutive added lines and reset at any break. A non-code
// region opened by a context line is invisible; the candidate
// is still scanned, which fails toward the loud label.
if (prefix !== "+") {
rawHashes = -1; blockDepth = 0; inString = false;
continue;
}
const text = line.slice(1);
if (rawHashes >= 0 || blockDepth > 0 || inString)
nonCodeStart.add(i);
let j = 0;
while (j < text.length) {
if (rawHashes >= 0) {
if (text[j] === '"') {
let h = 1;
while (h <= rawHashes && text[j + h] === "#") h++;
if (h > rawHashes) { rawHashes = -1; j += h; continue; }
}
j += 1; continue;
}
if (blockDepth > 0) {
if (text.startsWith("/*", j)) { blockDepth += 1; j += 2; }
else if (text.startsWith("*/", j)) { blockDepth -= 1; j += 2; }
else { j += 1; }
continue;
}
if (inString) {
if (text[j] === "\\") { j += 2; continue; }
if (text[j] === '"') { inString = false; j += 1; continue; }
j += 1; continue;
}
// Recognize Rust char and byte-char literals ('X', '\X',
// '\u{...}', b'X') so their contents never open string or
// comment state. A char literal has a closing ' within a
// bounded distance; a lifetime or label ('a, 'static,
// 'label:) does not, so it falls through as a regular char.
// This is not a skip-to-next-' rule: the lookahead is 2-4
// chars, or bounded to 20 for \u escapes (at most 6 hex
// digits), never unbounded.
if (text[j] === "'") {
let k = j + 1;
if (text[k] === "\\") {
k += 1;
if (text[k] === "u" && text[k + 1] === "{") {
const uBound = k + 20;
while (k < uBound && text[k] && text[k] !== "}") k++;
if (text[k] === "}") k++;
} else {
k += 1;
}
} else {
k += 1;
}
if (text[k] === "'") { j = k + 1; continue; }
// lifetime or label: just skip the apostrophe
}
if (text[j] === '"') { inString = true; j += 1; continue; }
if (text.startsWith("//", j)) break;
if (text.startsWith("/*", j)) { blockDepth = 1; j += 2; continue; }
const prev = j > 0 ? text[j - 1] : "";
const isIdent = /[A-Za-z0-9_]/.test(prev);
if (!isIdent && text[j] === "r") {
let k = j + 1, h = 0;
while (text[k + h] === "#") h++;
if (text[k + h] === '"') {
rawHashes = h; j = k + h + 1; continue;
}
}
if (!isIdent && (text[j] === "b" || text[j] === "c") && text[j + 1] === "r") {
let k = j + 2, h = 0;
while (text[k + h] === "#") h++;
if (text[k + h] === '"') {
rawHashes = h; j = k + h + 1; continue;
}
}
j += 1;
}
}
for (let i = 0; i < lines.length; i++) {
if (lines[i][0] !== "+") continue;
if (nonCodeStart.has(i)) continue;
const m = TEST_ATTR_LINE.exec(lines[i].slice(1));
if (m === null) continue;
const segs = [m[1]];
for (
let j = i + 1;
j < lines.length &&
lines[j][0] === "+" &&
segs.length <= MAX_ATTR_CONTINUATION_LINES;
j++
) {
segs.push(lines[j].slice(1));
}
if (attrCloserFollows(segs)) return true;
}
return false;
}
// TRIAGE_DETECTOR_END
const addsInlineTest = files.some(
f => f.filename.endsWith(".rs") && !!f.patch && patchAddsInlineTest(f.patch)
);
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+[ \t]*#\[[ \t]*(?:cfg[ \t]*\([ \t]*test[ \t]*\)|(?:::[ \t]*)?(?:[\w-]+[ \t]*::[ \t]*)*(?:r#)?test\b)/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
+34 -231
View File
@@ -20,15 +20,7 @@ on:
type: string
concurrency:
# Not keyed on the ref: two runs racing `imagetools create` on the same
# moving tags could interleave, so runs of this workflow serialize outright.
# GitHub keeps at most one running and one pending run per group; a newer
# queued run replaces the pending one, and cancel-in-progress: false
# protects only the run already executing. So a backfill dispatched while a
# release is running, followed by a push to main, is replaced silently: the
# remedy is to re-dispatch it once the running release completes, which the
# push-only gate on the moving-tag step makes safe in any ordering.
group: release
group: release-${{ github.ref }}
cancel-in-progress: false
# Least privilege by default; every job opts in to exactly what it needs.
@@ -45,7 +37,6 @@ jobs:
permissions:
contents: write
pull-requests: write
environment: release
outputs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
@@ -87,7 +78,6 @@ jobs:
# dispatch the release PR's head carries no check runs and the --locked
# gate never validates it.
actions: write
environment: release
steps:
# Prefer the fresh `pr` output when this run just created/updated the
# release PR; otherwise fall back to querying for an open release-please
@@ -225,7 +215,6 @@ jobs:
permissions:
contents: read
packages: write
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -237,7 +226,6 @@ jobs:
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
@@ -249,9 +237,7 @@ jobs:
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ steps.rel.outputs.tag }}
persist-credentials: false
- name: Set up Docker Buildx
@@ -326,7 +312,6 @@ jobs:
permissions:
contents: read
packages: write
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -338,14 +323,9 @@ jobs:
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing, so a mixed-case owner
# makes the digest push fail with "invalid reference format".
echo "image=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
- name: Download digests
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
@@ -365,80 +345,24 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create and push multi-arch manifest
id: manifest
env:
VERSION: ${{ steps.rel.outputs.version }}
IMAGE: ${{ steps.rel.outputs.image }}
run: |
set -euo pipefail
# The docker matrix pushes one digest per arch leg; anything else is
# a broken set, not a smaller multi-arch image.
count="$(find /tmp/digests -maxdepth 1 -type f | wc -l)"
if [ "$count" -ne 2 ]; then
echo "::error::expected 2 arch digests in /tmp/digests, found $count"
exit 1
fi
# ghcr requires a lowercase repository path.
IMAGE="ghcr.io/${GITHUB_REPOSITORY,,}"
MAJOR_MINOR="${VERSION%.*}"
digests=""
for f in /tmp/digests/*; do
d="$(basename "$f")"
case "$d" in
*[!0-9a-f]*)
echo "::error::digest filename is not lowercase sha256 hex: $d"
exit 1
;;
esac
if [ "${#d}" -ne 64 ]; then
echo "::error::digest is not 64 hex chars: $d"
exit 1
fi
digests="$digests $IMAGE@sha256:$d"
digests="$digests $IMAGE@sha256:$(basename "$f")"
done
# The immutable tag always publishes. The moving tags are applied by
# the gated step below, which never runs on workflow_dispatch.
# shellcheck disable=SC2086
docker buildx imagetools create -t "$IMAGE:$VERSION" $digests
docker buildx imagetools create \
-t "$IMAGE:$VERSION" \
-t "$IMAGE:$MAJOR_MINOR" \
-t "$IMAGE:latest" \
$digests
docker buildx imagetools inspect "$IMAGE:$VERSION"
echo "digests=${digests# }" >> "$GITHUB_OUTPUT"
- name: Move floating tags
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
IMAGE: ${{ steps.rel.outputs.image }}
DIGESTS: ${{ steps.manifest.outputs.digests }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
MAJOR_MINOR="${VERSION%.*}"
# :latest and :X.Y track the newest release. Only a push to main
# carries a release-please-cut version, and release-please versions
# are monotonic, so on this path the tags only ever advance. A
# workflow_dispatch backfill republishes an existing tag and must
# never move a pointer, so this step skips on dispatch. After a
# backfill that is genuinely newest in its minor line, advance :X.Y
# by hand: docker buildx imagetools create -t "$IMAGE:X.Y" <digests>.
# The event gate alone does not bound a stale run: re-running an
# older push run replays its stored needs outputs under the
# original event and ref, and would move the pointers backward to
# that run's version. Floor against live state instead: this run's
# release already exists by now, so VERSION must be the newest
# published release or the run is stale and the tags stay put. The
# releases list is ordered by creation, not semver, so read every
# page and take the semver maximum, not the first page's.
tags="$(gh api --paginate "repos/$GITHUB_REPOSITORY/releases?per_page=100" \
-q '.[] | select(.draft == false) | .tag_name')" || {
echo "::error::could not list releases to bound the moving tags"
exit 1
}
floor="$(printf '%s\n' "$tags" \
| { grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' || true; } \
| sed 's/^v//' | sort -V | tail -n 1)"
if [ -n "$floor" ] && [ "$floor" != "$VERSION" ]; then
echo "::error::release v$floor is newer than this run's v$VERSION; refusing to move the moving tags backward (stale re-run?)"
exit 1
fi
# shellcheck disable=SC2086
docker buildx imagetools create -t "$IMAGE:latest" -t "$IMAGE:$MAJOR_MINOR" $DIGESTS
- name: Release summary
env:
@@ -462,7 +386,6 @@ jobs:
continue-on-error: ${{ startsWith(matrix.target, 'x86_64-pc-windows') }}
permissions:
contents: write
environment: release
strategy:
fail-fast: false
matrix:
@@ -482,27 +405,10 @@ jobs:
- target: x86_64-pc-windows-msvc
os: windows-latest
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash # matrix includes windows-latest, where run: defaults to pwsh
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Set up Rust toolchain
@@ -548,7 +454,7 @@ jobs:
shell: bash
run: |
set -euo pipefail
VERSION="${{ steps.rel.outputs.version }}"
VERSION="${{ needs.release-please.outputs.version }}"
BIN_DIR="target/${{ matrix.target }}/release"
EXE=""
case "${{ matrix.target }}" in *windows*) EXE=".exe" ;; esac
@@ -568,7 +474,7 @@ jobs:
run: |
set -euo pipefail
TARGET="${{ matrix.target }}"
NAME="gitlawb-node-${{ steps.rel.outputs.version }}-${TARGET}"
NAME="gitlawb-node-${{ needs.release-please.outputs.version }}-${TARGET}"
BIN_DIR="target/${TARGET}/release"
EXE=""
case "$TARGET" in *windows*) EXE=".exe" ;; esac
@@ -609,7 +515,7 @@ jobs:
- name: Attach to release
uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8 # v2.3.2
with:
tag_name: ${{ steps.rel.outputs.tag }}
tag_name: ${{ needs.release-please.outputs.tag_name }}
files: |
dist/*.tar.gz
dist/*.zip
@@ -622,9 +528,7 @@ jobs:
# (release-binaries is skipped on dispatch, hence !cancelled()). Publishing
# auth is npm Trusted Publishing (GitHub OIDC) — no NPM_TOKEN. Each
# @gitlawb package must have this repo + workflow (release.yml) configured
# as its trusted publisher on npmjs.com, scoped to the release environment
# so a dispatched copy of this file on another ref cannot mint a token, or
# publish fails loudly here —
# as its trusted publisher on npmjs.com, or publish fails loudly here —
# deliberately loud: the silent secret-guard skip is how 0.4.x–0.6.0
# never reached npm.
if: >-
@@ -636,7 +540,6 @@ jobs:
permissions:
contents: read
id-token: write # npm trusted publishing (OIDC) + provenance
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -648,7 +551,6 @@ jobs:
env:
DISPATCH_TAG: ${{ inputs.npm_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
@@ -656,9 +558,7 @@ jobs:
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ steps.rel.outputs.tag }}
persist-credentials: false
# npm >= 11.5.1 performs the OIDC token exchange automatically when the
@@ -671,23 +571,14 @@ jobs:
- name: Ensure npm supports trusted publishing
run: |
set -euo pipefail
npm install -g npm@11.19.1
# Assert the pin took, so an accidental unpin or a drift off 11.19.1
# fails loudly here instead of silently running a different npm on
# the OIDC path.
installed="$(npm --version)"
echo "npm installed: $installed"
if [ "$installed" != "11.19.1" ]; then
echo "::error::npm is not the pinned 11.19.1 (got: $installed)"
exit 1
fi
npm install -g npm@^11.5.1
npm --version
- name: Lay in release binaries
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
ASSETS: ${{ steps.rel.outputs.assets }}
run: |
set -euo pipefail
# npm platform package -> Rust target triple (unix only; Windows is not
@@ -704,18 +595,8 @@ jobs:
target="${entry#*:}"
archive="gitlawb-node-${VERSION}-${target}.tar.gz"
echo "==> $pkg <- $archive"
# Download by the asset id the resolver captured and uploader-
# checked, not by name: release assets are mutable, and an id can
# only ever point at the exact blob captured at resolve time.
# The map keys are base64-encoded asset names.
want="$(printf '%s' "$archive" | base64 -w0)"
asset_id="$(printf '%s\n' "$ASSETS" | awk -v n="$want" '$1 == n {print $2; exit}')"
if [ -z "$asset_id" ]; then
echo "::error::release $TAG has no asset $archive captured at resolve time"
exit 1
fi
gh api "repos/$GITHUB_REPOSITORY/releases/assets/$asset_id" \
-H 'Accept: application/octet-stream' > "_dl/$archive"
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern "$archive" --dir _dl --clobber
tar -xzf "_dl/$archive" -C _dl
src="_dl/gitlawb-node-${VERSION}-${target}"
cp "$src/gl" "npm/packages/$pkg/gl"
@@ -754,39 +635,14 @@ jobs:
# Platform packages first, then the wrapper (so its optionalDependencies resolve).
# Skip versions already on the registry so a rerun after a partial publish
# is idempotent instead of erroring on the first existing package.
# npm publish moves the latest dist-tag to whatever it publishes, so a
# backfilled older version would hand :latest to stale code. Decide
# the dist-tag per package from that package's own registry latest:
# a partial publish can leave the platform packages ahead of the
# wrapper. E404 means the package has never been published, so this
# release gets latest; any other lookup failure must fail closed
# rather than guess at the dist-tag and risk moving latest backward.
# stderr goes to a file so a warning on a successful lookup cannot
# leak into registry_latest; it is read only when the lookup fails.
npm_err="$(mktemp)"
trap 'rm -f -- "$npm_err"' EXIT
for pkg in gl-darwin-arm64 gl-darwin-x64 gl-linux-arm64 gl-linux-x64 gl; do
name="@gitlawb/$pkg"
if npm view "$name@$VERSION" version >/dev/null 2>&1; then
echo "==> $name@$VERSION already published, skipping"
continue
fi
registry_latest="$(npm view "$name" dist-tags.latest 2>"$npm_err")" || {
if ! grep -q E404 "$npm_err"; then
cat "$npm_err" >&2
echo "::error::npm dist-tags lookup failed for $name; not guessing the dist-tag"
exit 1
fi
registry_latest=""
}
dist_tag="latest"
if [ -n "$registry_latest" ] && \
[ "$VERSION" != "$(printf '%s\n%s\n' "$registry_latest" "$VERSION" | sort -V | tail -1)" ]; then
dist_tag="backfill"
echo "::notice::$VERSION is older than $name@$registry_latest; publishing under dist-tag backfill"
fi
echo "==> npm publish $name@$VERSION (dist-tag $dist_tag)"
npm publish "npm/packages/$pkg" --provenance --access public --tag "$dist_tag"
echo "==> npm publish $name@$VERSION"
npm publish "npm/packages/$pkg" --provenance --access public
done
homebrew-bump:
@@ -796,7 +652,6 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN downloads release assets; the tap push uses HOMEBREW_TAP_PAT
environment: release
steps:
- name: Guard on secret
id: guard
@@ -810,22 +665,6 @@ jobs:
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out workflow scripts
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout tap repo
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
@@ -839,32 +678,15 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
ASSETS: ${{ steps.rel.outputs.assets }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
base="https://github.com/${GITHUB_REPOSITORY}/releases/download/${TAG}"
mkdir -p _sums
# Hash the tarballs themselves, fetched by the asset id the resolver
# captured and uploader-checked. The *.sha256 sidecars are mutable
# release assets too; trusting them would trust the same surface.
sha() {
archive="gitlawb-node-${VERSION}-$1.tar.gz"
want="$(printf '%s' "$archive" | base64 -w0)"
asset_id="$(printf '%s\n' "$ASSETS" | awk -v n="$want" '$1 == n {print $2; exit}')"
if [ -z "$asset_id" ]; then
echo "::error::release $TAG has no asset $archive captured at resolve time" >&2
exit 1
fi
# errexit is off inside the $(...) this runs under, so a failed
# download must exit the function itself instead of hashing a
# truncated file.
gh api "repos/$GITHUB_REPOSITORY/releases/assets/$asset_id" \
-H 'Accept: application/octet-stream' > "_sums/$archive" \
|| exit 1
sha256sum "_sums/$archive" | awk '{print $1}'
}
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern '*.tar.gz.sha256' --dir _sums --clobber
sha() { awk '{print $1}' "_sums/gitlawb-node-${VERSION}-$1.tar.gz.sha256"; }
SHA_MAC_ARM="$(sha aarch64-apple-darwin)"
SHA_MAC_X64="$(sha x86_64-apple-darwin)"
SHA_LNX_ARM="$(sha aarch64-unknown-linux-musl)"
@@ -925,7 +747,7 @@ jobs:
working-directory: tap
env:
HOMEBREW_TAP_PAT: ${{ secrets.HOMEBREW_TAP_PAT }}
VERSION: ${{ steps.rel.outputs.version }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
@@ -947,7 +769,6 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN checks out node; the web PR uses WEB_SYNC_PAT
environment: release
steps:
- name: Guard on secret
id: guard
@@ -961,29 +782,11 @@ jobs:
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out workflow scripts
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout node (release tag)
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ needs.release-please.outputs.tag_name }}
path: node
persist-credentials: false
@@ -1003,8 +806,8 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets \
@@ -1022,8 +825,8 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.WEB_SYNC_PAT }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# Canonical install scripts (source of truth lives in the node repo).
+1 -1
View File
@@ -40,5 +40,5 @@ When a node denies a request, surface the denial to the user. Never render a den
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow's jobs are `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. On a PR, `scripts/ci-surfaces.sh` runs each job only when the PR touches its inputs, and scopes `cargo test` and the release build to the touched crates plus every crate that depends on them. Root `Cargo.toml`/`Cargo.lock`/`.cargo/` changes run the Rust jobs (fmt, clippy, tests, release build, MSRV) on the whole workspace; every other job still runs only when its own inputs change. Edits to the CI wiring itself (`pr-checks.yml`, `codeql.yml`, `scripts/ci-surfaces.sh`), push to `main`, and the merge queue run every job on the whole workspace. Edits to any other workflow run none of these jobs on a PR; CodeQL only analyzes them as Actions code. Run `cargo test --workspace` locally regardless. The same script limits CodeQL (`codeql.yml`) to the languages a PR touches. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
+1 -1
View File
@@ -81,7 +81,7 @@ time once updated.
**Requirements:**
- Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs)
- PostgreSQL 16+ — required for the node (it uses `pg_input_is_valid`, which exists only on PostgreSQL 16 and later). Use the bundled `docker-compose.yml` for local dev.
- PostgreSQL — required for the node. Use the bundled `docker-compose.yml` for local dev.
- Docker (optional, for full-stack local testing)
**Environment variables:**
Generated
+5 -16
View File
@@ -2208,7 +2208,7 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -2701,7 +2701,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de"
dependencies = [
"data-encoding",
"syn 1.0.109",
"syn 2.0.117",
]
[[package]]
@@ -3473,13 +3473,10 @@ name = "git-remote-gitlawb"
version = "0.7.1"
dependencies = [
"anyhow",
"chrono",
"dirs",
"gitlawb-core",
"libc",
"mockito",
"reqwest",
"serde_json",
"tempfile",
"tracing",
"tracing-subscriber",
@@ -3545,7 +3542,6 @@ dependencies = [
"axum",
"base64 0.22.1",
"bytes",
"chacha20poly1305",
"chrono",
"cid",
"clap",
@@ -3567,9 +3563,7 @@ dependencies = [
"libp2p-swarm",
"mockito",
"multiaddr",
"percent-encoding",
"prometheus",
"rand 0.8.6",
"reqwest",
"serde",
"serde_json",
@@ -3586,7 +3580,6 @@ dependencies = [
"tracing",
"tracing-subscriber",
"unicode-normalization",
"url",
"uuid",
"zeroize",
"zstd",
@@ -3602,12 +3595,9 @@ dependencies = [
"chrono",
"clap",
"dirs",
"ed25519-dalek 2.2.0",
"gitlawb-core",
"icaptcha-client",
"libc",
"mockito",
"percent-encoding",
"reqwest",
"serde",
"serde_json",
@@ -3616,7 +3606,6 @@ dependencies = [
"tokio",
"tracing",
"tracing-subscriber",
"url",
"urlencoding",
"uuid",
]
@@ -3652,9 +3641,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.19"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
checksum = "839c0e8a181239723652be9062bb56ca5bf5f64011f73b623f6f4fc59086a228"
dependencies = [
"atomic-waker",
"bytes",
@@ -7959,7 +7948,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.61.2",
]
[[package]]
+1 -3
View File
@@ -54,10 +54,8 @@ uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core; also used for the
# absolute-browser-url validation on GITLAWB_WEB_URL in the node and CLI)
# take a parsed URL without pulling reqwest into gitlawb-core)
url = "2"
percent-encoding = "2"
# HMAC
hmac = "0.12"
-106
View File
@@ -1,106 +0,0 @@
# Gitlawb Wire Protocol Specification (v1, alpha)
This document describes the wire behavior a second implementation can interop
with **today**. Anything the node does not ship yet is explicitly marked
*planned*; nothing below is aspirational unless it says so.
## 1. Identity & DIDs
- **Key type:** Ed25519.
- **Method (alpha):** `did:key` only. Resolution is deterministic public-key
extraction from the multicodec prefix (`z6Mk…`, multicodec `0xed01`).
- Signed requests presenting any other DID method are rejected; the node's
auth layer answers with a hint that only `did:key` is supported in alpha.
- *Planned:* `did:web` and `did:gitlawb` resolution. Clients MUST NOT assume
either is accepted anywhere yet.
## 2. Authentication (RFC 9421)
- **Transport:** signed requests are expected over TLS; the node does not
terminate HTTP signatures over cleartext.
- **HTTP signatures:** `Signature-Input` covers `@method`, `@path`, and
`content-digest`:
`sig1=("@method" "@path" "content-digest");keyid="did:key:…";alg="ed25519";created=<unix>`
All three components are mandatory: a `Signature-Input` missing one is
rejected as `incomplete_signature`.
- **`@path` value:** the node binds `@path` to the path *and* query string of
the request, not to the path-only value RFC 9421 defines for `@path`. A
second implementation must sign `GET /r/info/refs?service=git-upload-pack`
including the `?service=…` part.
- **Signer identity:** `keyid` is the actor DID (`did:key` only, see §1).
- **Integrity:** `Content-Digest` is SHA-256 over the request body and is
verified when the header is present. Because `content-digest` is a covered
component, every signed request signs a digest value: on a body-less
request the signed value is the SHA-256 of the empty body, which the node
also accepts when the header is absent entirely.
## 3. Proof of Intelligence (iCaptcha)
- **Mode:** controlled by `ICAPTCHA_MODE` = `off` | `shadow` | `enforce`,
**default `off`**. A node with the gate off never demands proof.
- **Scope:** the gate applies to a small set of write endpoints (agent
registration and repo create/fork), not to all writes.
- **Wire shape:** the client-supplied header is `x-icaptcha-proof` — that is
the only iCaptcha header a node reads from a request. When proof is
required and missing/invalid, the node answers
`403 icaptcha_proof_required` and sets `x-icaptcha-url` and
`x-icaptcha-level` **on the response** (also mirrored as JSON fields) so
the client can go solve a challenge and retry.
## 4. Ref-Update Certificate
*Planned:* the `gitlawb/ref-update/v1` schema below is defined in code, but no
shipped endpoint serves or verifies these certificates today; nothing in
sections 2-3 produces one on the wire. It is documented here so a second
implementation shares the type vocabulary ahead of the wire surface.
The hash fields reserve 64 hex characters for the SHA-256 object format; the
repositories the node serves are sha1 with 40-hex OIDs (see §5), so the field
width is a forward reservation, not what a current push produces.
Two distinct certificate-shaped payloads exist; this section documents the
**core signing schema**, which is what `gitlawb/ref-update/v1` names:
- **Type string:** `gitlawb/ref-update/v1`.
- **Body (canonical JSON, the signed bytes):** `type`, `repo` (the
repository DID), `ref_name`, `from` (previous hash, 64 hex chars,
all-zeros for a new ref), `to` (target hash, 64 hex chars), `seq`
(monotonically increasing, replay prevention), `timestamp` (RFC 3339),
`nonce`.
- **Signatures:** a list of `{signer: did:key, sig: base64url-unpadded
Ed25519}` entries appended outside the signed body; thresholds are counted
over distinct signers that verify.
The node's REST API separately serves **push receipts** — a different,
node-issued JSON payload attesting that a push was processed. Receipts are
not `gitlawb/ref-update/v1` documents and are not interchangeable with them;
they are documented with the node's HTTP API, not here.
## 5. Storage & Git Transport
- **Object format:** bare repositories are created with
`--object-format=sha1`; production OIDs are 40 hex chars. *Planned:*
SHA-256 object format.
- **Smart-HTTP endpoints:**
- `GET /{owner}/{repo}/info/refs?service=git-upload-pack` — fetch
advertisement.
- `GET /{owner}/{repo}/info/refs?service=git-receive-pack` — push
advertisement.
The `?service=` parameter is mandatory and takes exactly those two
values; a missing parameter is a `400`, not an omitted-service default.
- `POST /{owner}/{repo}/git-upload-pack` — clone/fetch. Request and
response bodies are pkt-line framed; the result content type is
`application/x-git-upload-pack-result`.
- `POST /{owner}/{repo}/git-receive-pack` — push. **Authentication is
mandatory**: an unsigned push is refused with `401`. Same pkt-line
framing; the result content type is
`application/x-git-receive-pack-result`.
Signed requests on this path follow §2, including TLS transport.
- **Fetch vs push auth split:** fetch-side requests against a withheld or
private repo answer `404` (existence is not confirmed to unauthorized
callers), while the push path answers `401` for missing/invalid
signatures. A second implementation must not treat the two symmetrically.
- **Content addressing:** the node records IPFS CID metadata per branch and
serves it over REST (the repo `refs` endpoint maps branch heads to CIDs).
*Planned:* IPNS-backed branch pointers; IPNS is not part of the shipped
wire contract.
+8 -12
View File
@@ -53,7 +53,6 @@ Good today:
- Local or Docker node startup.
- Postgres-backed repo metadata.
- Bounded GraphQL queries with [repository pagination](docs/graphql-pagination.md).
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
@@ -67,12 +66,12 @@ Good today:
Known limitations:
- Repository write authorization is one switch: `GITLAWB_ENFORCE_OWNER_PUSH`, which defaults to `true`. With it off — rolling upgrades only — a valid HTTP Signature identifies a pusher but does not authorize one, so every signed caller may push everywhere.
- UCAN capabilities are consulted on the push path only. There, the chain's root issuer is anchored to the repository owner, so an owner-rooted, time-bounded delegation that names the repository grants scoped collaborator access for pushing to it, for the `git/push`, `*`, or `repo/admin` actions. A `*` *resource* is refused: a delegation must name the repository it applies to, so it cannot grow to cover repos created later. Push-only cuts the other way too: a delegation grants no read access, so on a private repository the delegate must also be a reader (`gl visibility set`) — the two gates a push crosses are described in [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). The rest is unchanged: there is no revocation path, `nb` constraints are refused rather than interpreted, and no other route — reads, pull requests, issues, agents — consults capabilities at all.
- Repository write authorization is not secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility, so a valid HTTP Signature identifies a pusher but does not enforce owner-only pushes.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access.
- Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Task list/get reads (REST and GraphQL) are repository/task-gated and omit `ucan_token`. Slash-form `repo_id` rows (mirror-hosted tasks) can appear on list for the delegator or trimmed assignee, but `GET /api/v1/tasks/{id}` and GraphQL `task` fail closed for every caller because there is no local repository backing to serve. Read visibility is not a blanket data-classification boundary: IPFS-pin and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations require a signed identity (a verified signer bound to the acting DID), are limited to the agent-task queue, and are covered by a CI completeness fence that requires signer acquisition and actor binding on every registered mutation; repo writes are not exposed over GraphQL and remain REST-only.
- Current GraphQL mutations require an authenticated signer, but there is no mutation-specific guardrail that prevents a future mutation from omitting that check.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
See:
@@ -312,8 +311,7 @@ metadata local disk / optional S3
| DID | A user, agent, or node identity derived from an Ed25519 public key. |
| HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. |
| Ref certificate | Signed record of a ref update. Useful for audit and replication. |
| Ref certificate versioning | The wire format is versioned; new certificates are currently issued as v1. `gl cert show <repo> <id> --verify` requires `--expect-node <did>`, and the queried node's self-reported DID is **not** accepted as a trust anchor. v2 is reserved for a future payload shape that carries the version inside the signed bytes; this client refuses to verify v2+ rather than guessing. |
| UCAN | Capability token. An owner delegates `git/push` on a repo to another DID; the node honors it when the proof chain roots at that owner. See [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| UCAN | Delegation token for future capability-based workflows. |
| Peer announce | Node-to-node HTTP announcement of DID + public URL. |
| Gossipsub | libp2p topic for ref-update events. |
| Smart HTTP | Standard git protocol over HTTP for clone/fetch/push. |
@@ -401,10 +399,9 @@ Important node settings:
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_P2P_KEY` | Path to the node's persistent libp2p identity key file, which fixes the PeerId across restarts. Must name a key file inside a directory, such as `/data/keys/p2p.key` or `./keys/p2p.key`. Two kinds of problem are handled differently. A value that cannot name a securable key file at all (a bare filename, `..` traversal, a trailing `/`, the filesystem root, or a `~/` path that escapes home) is refused before the node binds, so the process exits. Anything wrong with the storage itself (a symlinked or non-directory parent, an unsafe ancestor or working directory, an unreadable or malformed key) leaves p2p off while HTTP keeps serving; the node logs it at error level as `p2p_identity_key_load_failed` and sets the `gitlawb_p2p_identity_key_load_failed` metric to 1. Note that `/health` still reports healthy in that state, so alert on the log event or the metric rather than on the health check. Correcting the storage and restarting restores the same PeerId. On Unix a new key is created `0600` inside a `0700` directory. A key directory that grants access beyond its owner is tightened to `0700` on start; one that is closed too far is refused with a `chmod 700` remedy rather than widened, since a directory locked down on purpose is an operator decision. Rotate the key only when the node logs that the directory was group- or world-writable (or the key file itself was group- or world-readable). An ordinary `0755` directory is tightened too, but others could not read a `0600` key or replace its directory entry. An existing key file is never modified: one with group or other bits set is rejected (`chmod 600`), and so is one this node cannot read. On other platforms no permissions are enforced. Default `~/.gitlawb/p2p.key`; point it at a persistent volume when running in a container. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. Defaults to `false` during the staged rollout below. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the pusher on `git-receive-pack` to be the repo owner, or to present an owner-rooted, time-bounded UCAN `git/push` capability naming the repository (`X-Ucan`). **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. A CI or delegated key with no such delegation counts as a non-owner and cannot push while this is on. The delegation clears the push gate only; on a private repository the delegate must also be a reader. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require peer writes to prove their DID on every transport: RFC 9421 signatures on the announce/sync routes, and an Ed25519 payload signature on gossip ref-update events. Defaults to `false` during the staged rollout below. Upgrade every gossip publisher before enabling, or their updates are dropped with no error on their side. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_RECONCILIATION_SWEEP` | Enable the hourly durability sweep that re-pins/backstops missing objects (default `true`; disabled when no IPFS/Pinata backend is configured). Public pin repair runs against any configured backend (IPFS, Pinata, or both). Encrypted recovery repair requires local IPFS (`GITLAWB_IPFS_API`); Pinata-only nodes reconcile public pins only. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
@@ -422,7 +419,6 @@ Important node settings:
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
| `GITLAWB_TASK_READ_RATE_LIMIT` | Max anonymous task-read requests (`GET /api/v1/tasks`, `GET /api/v1/tasks/{id}`) per client IP per hour, sharing the budget with the GraphQL/WS task-read brake. Uses `GITLAWB_TRUSTED_PROXY` to resolve the client IP. 0 disables. Default 1200 (above the `/ipfs` budget to allow list pages followed by per-task reads). |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
@@ -557,7 +553,7 @@ Short-term priorities:
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Wire trusted UCAN delegation into repository write authorization.
5. Close default-open write authorization and wire trusted UCAN delegation into repository permissions.
6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
+11 -11
View File
@@ -26,7 +26,7 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
**Content addressing**
- Every git object is content-addressed via CIDv1 (SHA-256)
- Tamper-evident by construction: a modified object changes its CID
- Tamper-evident by construction — a modified object changes its CID
**UCAN token validation**
- Bootstrap UCAN tokens are issued at registration.
@@ -35,8 +35,8 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
- Capability grants are not yet consulted by repository write authorization; see the limitations below.
**Smart contracts (Base Sepolia testnet)**
- `GitlawbDIDRegistry`: on-chain DID → document registry
- `GitlawbNameRegistry`: human name → DID registry
- `GitlawbDIDRegistry` — on-chain DID → document registry
- `GitlawbNameRegistry` — human name → DID registry
- Both auditable on-chain, no admin keys
---
@@ -54,9 +54,9 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
### Repository write authorization defaults
- `git-receive-pack` verifies HTTP Signatures and, with the default configuration, also requires the pusher DID to be the repository owner: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `true`.
- **Impact:** With the default setting, a valid signature authenticates the pusher, and the pusher must be the repository owner. Authentication is not authorization on its own, since anyone can mint a `did:key` and sign, so owner enforcement is the only thing that prevents every signed caller from pushing to every repository, including private ones.
- **Mitigation:** `GITLAWB_ENFORCE_OWNER_PUSH=false` is the explicit rolling-upgrade compatibility override. Disable it only for the duration of an upgrade whose pushers are not yet the repo owner, then turn it back on. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on.
- `git-receive-pack` verifies HTTP Signatures, but `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility during rollout.
- **Impact:** With the default setting, a valid signature authenticates the pusher but does not require that DID to be the repository owner.
- **Mitigation:** Set `GITLAWB_ENFORCE_OWNER_PUSH=true` on nodes where owner-only pushes are required. Confirm that every legitimate pusher uses the owner DID before enabling it.
### UCAN delegation and revocation
- The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
@@ -73,7 +73,7 @@ These are documented limitations of the current live release. They should be pri
### Private repository reads
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
- Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
- Task list/get reads (`GET /api/v1/tasks`, `GET /api/v1/tasks/{id}`, and GraphQL `tasks` / `task` over HTTP and WebSocket) are repository/task-gated through shared visibility collectors and omit `ucan_token`. Slash-form `repo_id` rows (mirror-hosted tasks) can appear on list for the delegator or trimmed assignee, but `GET /api/v1/tasks/{id}` and GraphQL `task` fail closed for every caller because there is no local repository backing to serve. All of those transports share the same per-IP `GITLAWB_TASK_READ_RATE_LIMIT` budget. `/api/v1/ipfs/pins` and `/api/v1/arweave/anchors` are not repository-gated; pin and anchor listings expose object and ref metadata.
- `GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata.
- Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
@@ -119,10 +119,10 @@ These are documented limitations of the current live release. They should be pri
## Threat Model
Twigpine is designed to be secure against:
- **Unauthenticated writes:** HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
- **Tampered git objects:** CIDv1 content addressing detects modification
- **Identity spoofing:** DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown:** no single point of control; data on IPFS + Arweave
- **Unauthenticated writes** — HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
- **Tampered git objects** — CIDv1 content addressing detects modification
- **Identity spoofing** — DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown** — no single point of control; data on IPFS + Arweave
Twigpine is **not yet** designed to defend against:
- A compromised node operator (node operators are trusted for their own node)
+1 -9
View File
@@ -1,7 +1,7 @@
{
"$comment": "Canonical seed list for the Twigpine network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1,
"updated": "2026-10-02",
"updated": "2026-06-30",
"peers": [
{
"name": "gitlawb",
@@ -42,14 +42,6 @@
"http_url": "https://rapybus-gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-05-16"
},
{
"name": "moonwake-ledger",
"operator": "Moonwake Ledger",
"did": "did:key:z6Mkh9AZ46Z3RkGZfFCom5vzX2535f54d6G2o2ZGkCqjwW8K",
"http_url": "https://node.moonwakeledger.com",
"p2p_multiaddr": null,
"added": "2026-10-02"
}
]
}
-9
View File
@@ -14,17 +14,8 @@ path = "src/main.rs"
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
anyhow = { workspace = true }
reqwest = { workspace = true }
# Reading the node's DID from `GET /` so a delegated push can address its
# invocation to the right executor.
serde_json = { workspace = true }
# The invocation carries the delegation's expiry, so an unbounded write capability
# is never minted; converting the stored i64 timestamp needs chrono.
chrono = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
# Home-directory lookup for the shared GITLAWB_KEY resolver in gitlawb-core,
# which takes `home` as an argument so core's dependency allowlist stays lean.
dirs = "5"
[dev-dependencies]
mockito = "1"
File diff suppressed because it is too large. Load diff
@@ -95,36 +95,6 @@ fn upload_pack(repo: &Path, advertise: bool, input: &[u8]) -> Vec<u8> {
out.stdout
}
/// Run `git receive-pack --stateless-rpc [--advertise-refs] <repo>`, optionally
/// feeding `input` on stdin. Returns raw stdout (the wire bytes). The receive
/// half of `upload_pack`, used so the shim can serve a real push.
fn receive_pack(repo: &Path, advertise: bool, input: &[u8]) -> Vec<u8> {
let mut cmd = Command::new("git");
cmd.arg("receive-pack").arg("--stateless-rpc");
if advertise {
cmd.arg("--advertise-refs");
}
cmd.arg(repo)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
let mut child = cmd.spawn().expect("spawn git receive-pack");
let mut stdin = child.stdin.take().unwrap();
let input = input.to_vec();
let writer = std::thread::spawn(move || {
let _ = stdin.write_all(&input);
// drop closes stdin
});
let out = child.wait_with_output().expect("wait receive-pack");
writer.join().ok();
assert!(
out.status.success(),
"git receive-pack failed: {}",
String::from_utf8_lossy(&out.stderr)
);
out.stdout
}
#[derive(Clone, Copy)]
enum ShimMode {
/// Faithful v0 serving: pipe each POST to `git upload-pack --stateless-rpc`.
@@ -137,11 +107,6 @@ enum ShimMode {
struct Shim {
base_url: String,
posts: Arc<AtomicUsize>,
/// Advertisement (GET /info/refs) count. One `git fetch` performs exactly
/// one advertisement, so tests assert this is EXACTLY one: any fetch-level
/// retry that comes back re-advertises and goes red on the spot, which the
/// lower-bound POST assertions alone cannot see (#275 round 6).
gets: Arc<AtomicUsize>,
stop: Arc<AtomicBool>,
handle: Option<std::thread::JoinHandle<()>>,
}
@@ -171,17 +136,15 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
let addr = listener.local_addr().unwrap();
let base_url = format!("http://{addr}");
let posts = Arc::new(AtomicUsize::new(0));
let gets = Arc::new(AtomicUsize::new(0));
let stop = Arc::new(AtomicBool::new(false));
let posts_t = posts.clone();
let gets_t = gets.clone();
let stop_t = stop.clone();
let handle = std::thread::spawn(move || {
while !stop_t.load(Ordering::SeqCst) {
match listener.accept() {
Ok((stream, _)) => {
handle_conn(stream, &repo, mode, &posts_t, &gets_t);
handle_conn(stream, &repo, mode, &posts_t);
}
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(Duration::from_millis(5));
@@ -194,7 +157,6 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
Shim {
base_url,
posts,
gets,
stop,
handle: Some(handle),
}
@@ -220,13 +182,7 @@ fn normalize_accepted_stream(stream: &TcpStream) {
stream.set_read_timeout(Some(Duration::from_secs(30))).ok();
}
fn handle_conn(
stream: TcpStream,
repo: &Path,
mode: ShimMode,
posts: &AtomicUsize,
gets: &AtomicUsize,
) {
fn handle_conn(stream: TcpStream, repo: &Path, mode: ShimMode, posts: &AtomicUsize) {
normalize_accepted_stream(&stream);
let mut reader = BufReader::new(stream);
@@ -262,22 +218,7 @@ fn handle_conn(
reader.read_exact(&mut body).ok();
}
let (content_type, payload) = if method == "GET" && target.contains("service=git-receive-pack")
{
// Receive-pack advertisement, same wrapper shape as upload-pack.
let adv = receive_pack(repo, true, b"");
let mut wrapped = pkt(b"# service=git-receive-pack\n");
wrapped.extend_from_slice(b"0000");
wrapped.extend_from_slice(&adv);
("application/x-git-receive-pack-advertisement", wrapped)
} else if method == "POST" && target.ends_with("/git-receive-pack") {
posts.fetch_add(1, Ordering::SeqCst);
(
"application/x-git-receive-pack-result",
receive_pack(repo, false, &body),
)
} else if method == "GET" && target.contains("/info/refs") {
gets.fetch_add(1, Ordering::SeqCst);
let (content_type, payload) = if method == "GET" && target.contains("/info/refs") {
// v0 advertisement, wrapped exactly as the node's info_refs does.
let adv = upload_pack(repo, true, b"");
let mut wrapped = pkt(b"# service=git-upload-pack\n");
@@ -352,34 +293,6 @@ fn fetch_with_helper(clone: &Path, node_url: &str) -> (bool, std::process::Outpu
run_bounded(cmd, Duration::from_secs(30))
}
/// Run `git push` in `local` through the helper, same wiring and hard timeout as
/// [`fetch_with_helper`]. `args` carries the push spec, e.g.
/// `["--delete", "doomed"]`.
fn push_with_helper(local: &Path, node_url: &str, args: &[&str]) -> (bool, std::process::Output) {
let helper_bin = PathBuf::from(env!("CARGO_BIN_EXE_git-remote-gitlawb"));
let helper_dir = helper_bin.parent().unwrap().to_path_buf();
let path_env = match std::env::var_os("PATH") {
Some(p) => {
let mut dirs = vec![helper_dir.clone()];
dirs.extend(std::env::split_paths(&p));
std::env::join_paths(dirs).unwrap()
}
None => helper_dir.clone().into_os_string(),
};
let mut cmd = Command::new("git");
cmd.arg("-C")
.arg(local)
.arg("push")
.arg("origin")
.args(args)
.env("PATH", path_env)
.env("LC_ALL", "C")
.env("GITLAWB_NODE", node_url)
.env("GITLAWB_KEY", "/nonexistent-key-for-anon-push");
run_bounded(cmd, Duration::from_secs(30))
}
/// A synthetic non-success `ExitStatus` used only on the unix timeout path, where
/// `reap_group` has already consumed the child and the later `wait` returns ECHILD.
/// The timeout branch already reports `completed == false`, so the exact status is
@@ -1022,7 +935,6 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
let server = std::thread::spawn(move || {
let posts = AtomicUsize::new(0);
let gets = AtomicUsize::new(0);
let stream = loop {
match listener.accept() {
Ok((s, _)) => break s,
@@ -1037,7 +949,7 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
// the assertions below run on every platform.
stream.set_nonblocking(true).unwrap();
accepted_tx.send(()).unwrap();
handle_conn(stream, &repo, ShimMode::Normal, &posts, &gets);
handle_conn(stream, &repo, ShimMode::Normal, &posts);
});
let mut client = TcpStream::connect(addr).unwrap();
@@ -1101,17 +1013,6 @@ fn real_git_multi_round_fetch_completes() {
posts >= 2,
"fixture did not force multi-round negotiation (observed {posts} POST(s)); the bridging path was not exercised"
);
// Exactly one advertisement: the POST bound above is a floor, so a
// reintroduced fetch-level retry could satisfy it by running a second full
// fetch instead of multi-round negotiation. Re-advertising is what a
// second fetch cannot avoid, so this is the committed guard on the
// retry's absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
// The fetched tip is present and the clone's object graph is intact.
let server_head = String::from_utf8(git(&server, &["rev-parse", "HEAD"])).unwrap();
@@ -1204,17 +1105,6 @@ fn real_git_withheld_shaped_first_post() {
"helper hung on a withheld-shaped response (should forward-and-terminate, not deadlock). stderr:\n{stderr}"
);
// Exactly one advertisement, pass or fail: the POST assertions below are
// floors, so a reintroduced fetch-level retry could mask the withheld
// shape behind a second full fetch. A retry cannot avoid re-advertising,
// so this is the committed guard on its absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
if out.status.success() {
// Real git accepted the mid-negotiation pack: the withheld multi-round
// path works end to end with no extra handling.
@@ -1352,77 +1242,3 @@ fn run_bounded_bounds_join_when_leader_exits_leaving_a_pipe_holder() {
held pipes do not stall the joins)"
);
}
/// #369 end-to-end: a real `git push --delete` through the helper completes.
/// The receive-pack request for a delete-only push is commands + flush with NO
/// pack, and git holds its write pipe open for the report — so a helper that
/// reads stdin to EOF deadlocks against it. The shim pipes the POST to a real
/// `git receive-pack --stateless-rpc`, so the branch actually has to be deleted
/// server-side for the assertions to pass.
#[test]
fn real_git_push_delete_completes() {
let dir = tempfile::tempdir().unwrap();
let server = dir.path().join("server.git");
let local = dir.path().join("local");
// A bare repo holding the doomed branch, plus a live one.
git(
dir.path(),
&[
"init",
"--bare",
server.to_str().unwrap(),
"--initial-branch=main",
],
);
let work = dir.path().join("work");
git(
dir.path(),
&["init", work.to_str().unwrap(), "--initial-branch=main"],
);
git(&work, &["config", "user.email", "t@t"]);
git(&work, &["config", "user.name", "t"]);
std::fs::write(work.join("f"), "x").unwrap();
git(&work, &["add", "f"]);
git(&work, &["commit", "-m", "c"]);
git(&work, &["branch", "doomed"]);
git(&work, &["push", server.to_str().unwrap(), "main", "doomed"]);
// The pusher: a local repo whose origin is the gitlawb remote pointing at
// the shim. Nothing needs to be fetched first; git takes the old-oid from
// the receive-pack advertisement.
git(
dir.path(),
&["init", local.to_str().unwrap(), "--initial-branch=main"],
);
git(
&local,
&["remote", "add", "origin", "gitlawb://did:key:z6MkTest/repo"],
);
let shim = start_shim(server.clone(), ShimMode::Normal);
let (completed, out) = push_with_helper(&local, &shim.base_url, &["--delete", "doomed"]);
let posts = shim.posts.load(Ordering::SeqCst);
assert!(
completed,
"git push --delete did not complete within the timeout (deadlock signature). stderr:\n{}",
String::from_utf8_lossy(&out.stderr)
);
assert!(
out.status.success(),
"git push --delete failed. stderr:\n{}",
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(posts, 1, "a delete-only push is exactly one POST");
let gone = Command::new("git")
.arg("-C")
.arg(&server)
.args(["rev-parse", "--verify", "--quiet", "refs/heads/doomed"])
.output()
.unwrap();
assert!(
!gone.status.success(),
"refs/heads/doomed must be deleted in the server repo"
);
}
+10 -217
View File
@@ -105,30 +105,16 @@ impl Attestation {
let bytes = canonical_signing_bytes(&self.type_, &self.payload, &self.cert_hash)?;
let vk = verifying_key_from_did_key(&self.signer)?;
self.verify_sig_with_key(&vk, &bytes)?;
Ok(vk)
}
/// The signature check itself, against a key the caller supplies.
///
/// Split out so the STRICTNESS of the check stays provable on its own.
/// `verifying_key_from_did_key` now refuses a small-order key, so a weak
/// signer can no longer reach this line through `verify_signature`. Without
/// this seam there would be nothing left that fails when `verify_strict` is
/// downgraded to the non-strict `verify`, and that regression would land
/// silently. Check order in `verify_signature` is unchanged.
fn verify_sig_with_key(&self, vk: &VerifyingKey, bytes: &[u8]) -> Result<()> {
let sig_bytes: [u8; 64] = B64U
.decode(&self.sig)
.map_err(|e| Error::Signature(format!("base64url: {e}")))?
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify_strict(bytes, &sig)
vk.verify_strict(&bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(())
Ok(vk)
}
/// Reparse `payload` as `P`. Errors if the type discriminator does not
@@ -195,7 +181,7 @@ fn canonical_signing_bytes(
Ok(out)
}
pub(crate) fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&key.to_bytes());
@@ -216,19 +202,6 @@ fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
"did:key must use base58btc (z-prefix): {did}"
)));
}
// Refuse an oversized id before decoding. base58 decoding is quadratic in
// its input and `signer` comes off an attacker-supplied attestation, so a
// short request could otherwise buy a large decode. An ed25519 did:key
// method-id is a fixed 48 characters, so this bound is slack rather than a
// behavior change, and it only helps ahead of the decode. Mirrors the same
// cap in gitlawb-core's Did::to_verifying_key, which this crate cannot call
// (gitlawb-core is a dev-dependency here).
const MAX_METHOD_ID_LEN: usize = 64;
if method_id.len() > MAX_METHOD_ID_LEN {
return Err(Error::Did(
"did:key method-specific id too long".to_string(),
));
}
let (base, bytes) =
multibase::decode(method_id).map_err(|e| Error::Did(format!("multibase: {e}")))?;
if base != multibase::Base::Base58Btc {
@@ -244,20 +217,8 @@ fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
let key_bytes: [u8; 32] = bytes[ED25519_MULTICODEC.len()..]
.try_into()
.expect("length checked above");
let key = VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))?;
// `from_bytes` only decompresses, so it accepts a small-order point. This
// crate parses did:key itself rather than going through gitlawb-core's
// `Did`, and it cannot go through it (gitlawb-core is a dev-dependency
// here), so the rejection is mirrored rather than inherited. Signature
// verification is already strict, so this is defense in depth against a
// future consumer of this parser that does not verify strictly.
if key.is_weak() {
return Err(Error::Did("small-order ed25519 key".to_string()));
}
Ok(key)
VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))
}
#[cfg(test)]
@@ -504,23 +465,6 @@ mod tests {
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_oversized_method_id_before_decoding() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
// A well-formed prefix on an absurdly long id must fail the length
// bound, not the decoder: base58 decode cost is quadratic in input
// length, so reaching it at all is the defect.
att.signer = format!("did:key:z{}", "1".repeat(256));
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
// Match the bound's own message: any other Did error (a decode or
// length mismatch downstream) means the oversized input was decoded.
match err {
Error::Did(m) => assert!(m.contains("too long"), "unexpected: {m}"),
e => panic!("expected Error::Did, got {e:?}"),
}
}
#[test]
fn verify_rejects_bad_signature_base64() {
let sk = fresh();
@@ -539,43 +483,6 @@ mod tests {
assert!(matches!(err, Error::Signature(_)));
}
/// The identity-point forgery must be rejected: the shared attestation
/// verifier is a cert-bound provenance gate, so accepting the weak-key
/// signature would let anyone mint a forged attestation that verifies.
/// Strict verification rejects small-order public keys and R (the identity
/// point here), which ordinary verification does not.
#[test]
fn verify_rejects_identity_point_forgery() {
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&fresh(), cert_hash);
// Public key A = identity point (0,1); signature R = identity, S = 0.
// The equation `[S]B = R + [k]A` then holds for any k and any message.
let identity = [
1u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
let mut buf = Vec::with_capacity(34);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&identity);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let mut sig = [0u8; 64];
sig[..32].copy_from_slice(&identity);
att.sig = B64U.encode(sig);
let err = att.verify_signature(cert_hash).unwrap_err();
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"identity-point signer must be refused as a small-order key, got: {msg}"
),
other => panic!("expected Error::Did for an identity-point signer, got {other:?}"),
}
}
/// A payload that happens to contain a `cert_hash` field of its own does
/// not interfere with the outer binding: the attestation envelope's
/// `cert_hash` is the only field consulted by `verify_signature`, and the
@@ -652,111 +559,6 @@ mod tests {
/// weak (small-order) public key satisfies the verification equation
/// but must be rejected. The identity point is such a key: with R the
/// identity and S = 0, [S]B - [k]A is the identity for any message.
/// gitlawb-attest parses did:key itself and never routes through
/// gitlawb-core's `Did`, which it cannot: gitlawb-core is a dev-dependency
/// here. So the choke-point rejection has to be mirrored in this parser or
/// this crate keeps handing out keys the rest of the system rejects.
#[test]
fn verifying_key_from_did_key_rejects_a_small_order_key() {
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak);
let did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err =
verifying_key_from_did_key(&did).expect_err("a small-order did:key must not resolve");
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"rejection must name the small-order key, got: {msg}"
),
other => panic!("expected Error::Did, got {other:?}"),
}
}
/// base58 decoding is quadratic in its input and `signer` is attacker
/// controlled, so an oversized method-id must be refused BEFORE the decode
/// or a hostile attestation buys a large decode for a short request. The
/// assertion is on the length error specifically: a multibase error here
/// would mean the cap ran too late to matter.
#[test]
fn verifying_key_from_did_key_refuses_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "1".repeat(65_536));
let err = verifying_key_from_did_key(&oversized)
.expect_err("an oversized method-id must be refused");
match err {
Error::Did(msg) => assert!(
msg.contains("too long"),
"must fail on the length cap, not after decoding: {msg}"
),
other => panic!("expected Error::Did, got {other:?}"),
}
}
/// Strictness regression guard. The parser now refuses a small-order key,
/// so `verify_signature` can no longer carry a weak signer down to the
/// signature check; this drives that check directly to prove it is still
/// STRICT. Downgrading `verify_strict` to `verify` makes this go red,
/// which nothing else in the suite would catch.
#[test]
fn verify_sig_with_key_is_strict_about_a_weak_key() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak());
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
// R = identity, S = 0: satisfies the non-strict verification equation
// for any message under a small-order key.
let mut forged = [0u8; 64];
forged[0] = 1;
att.sig = B64U.encode(forged);
let bytes = canonical_signing_bytes(&att.type_, &att.payload, &att.cert_hash).expect("jcs");
let err = att
.verify_sig_with_key(&weak_vk, &bytes)
.expect_err("strict verification must reject a small-order key");
assert!(
matches!(err, Error::Signature(_)),
"expected a signature error, got {err:?}"
);
}
/// Accept control for the seam: a genuine signature still verifies through
/// the same helper, so the guard above is not simply rejecting everything.
#[test]
fn verify_sig_with_key_accepts_a_genuine_signature() {
let cert_hash = sample_cert_hash();
let sk = SigningKey::generate(&mut OsRng);
let att = dummy_attestation(&sk, cert_hash);
let bytes = canonical_signing_bytes(&att.type_, &att.payload, &att.cert_hash).expect("jcs");
att.verify_sig_with_key(&sk.verifying_key(), &bytes)
.expect("a real signature must verify");
}
/// Control for the guard above: a real did:key must still resolve.
#[test]
fn verifying_key_from_did_key_still_accepts_a_real_key() {
let sk = SigningKey::from_bytes(&[3u8; 32]);
let vk = sk.verifying_key();
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&vk.to_bytes());
let did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let got = verifying_key_from_did_key(&did).expect("a real did:key must resolve");
assert_eq!(got.to_bytes(), vk.to_bytes());
}
#[test]
fn verify_rejects_weak_key_signature() {
let mut weak_key_bytes = [0u8; 32];
@@ -770,13 +572,7 @@ mod tests {
let forged_sig = B64U.encode(forged);
// Build an attestation with the weak key as signer and the forged
// signature. Rejection now happens at DID RESOLUTION rather than at
// verify time: `verifying_key_from_did_key` refuses a small-order key
// before `verify_strict` is reached, so the observed error is
// `Error::Did`, not `Error::Signature`. `verify_strict` remains the
// second layer for malleability cases that do not involve a weak
// public key (a small-order R under an honest key), which this fixture
// cannot construct and therefore no longer covers.
// signature. The weak-key check happens at verify time.
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
@@ -788,12 +584,9 @@ mod tests {
att.sig = forged_sig;
let err = att.verify_signature(cert_hash).unwrap_err();
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"weak signer must be refused as a small-order key, got: {msg}"
),
other => panic!("expected Error::Did for a small-order signer, got {other:?}"),
}
assert!(
matches!(err, Error::Signature(_)),
"signature under a weak (small-order) public key must be rejected"
);
}
}
+14 -183
View File
@@ -121,12 +121,7 @@ impl Registry {
attestation: &Attestation,
expected_cert_hash: [u8; 32],
) -> Result<VerifiedAttestation> {
// The key that actually verified is the trust anchor for the signer a
// consumer sees. `verify_signature` recovers it from the artifact's
// `signer` field and rejects a field that does not match the signing
// key, so this is the canonical DID of the key that signed — never a
// raw artifact string that could disagree with it.
let verified_key = attestation.verify_signature(expected_cert_hash)?;
attestation.verify_signature(expected_cert_hash)?;
let fully = match self.by_type.get(attestation.type_.as_str()) {
Some(v) => {
@@ -143,55 +138,30 @@ impl Registry {
Ok(VerifiedAttestation {
type_: attestation.type_.clone(),
signer: crate::attestation::did_key_from_verifying_key(&verified_key),
signer: attestation.signer.clone(),
cert_hash: attestation.cert_hash.clone(),
fully_verified: fully,
})
}
/// Verify a batch, then enforce `RequireAll`.
///
/// Under [`Policy::RequireAll`] this never short-circuits: an attestation
/// that fails to verify — bad signature, cert-hash mismatch, or malformed
/// payload, the actual `Err` paths of [`Registry::verify`] — is dropped
/// from the result rather than aborting the batch. A valid attestation of
/// a type with NO registered verifier is not one of those: under
/// `RequireAll` it verifies to `Ok` with `fully_verified: false` and stays
/// in the result, preserving the distinction between *untrusted* and
/// *invalid* (see `require_all_is_lenient_on_unknown_types_in_the_batch`).
/// Any third party can attach an attestation, so a hard error here would
/// let an unrelated `attacker/spam/v1` deny-of-service the whole cert (the
/// DoS vector the module docstring warns about). The `required_types`
/// check below is the real gate, and it only counts `fully_verified`
/// entries — so a required type represented solely by a malformed
/// attestation still fails with [`Error::RequiredMissing`].
///
/// Under `AcceptKnown` and `RejectUnknown` the batch stays strict: the
/// first verification error is surfaced to the caller.
pub fn verify_all(
&self,
attestations: &[Attestation],
expected_cert_hash: [u8; 32],
) -> Result<Vec<VerifiedAttestation>> {
if self.policy != Policy::RequireAll {
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
return Ok(verified);
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
// RequireAll: drop entries that fail to verify instead of aborting.
let verified: Vec<VerifiedAttestation> = attestations
.iter()
.filter_map(|a| self.verify(a, expected_cert_hash).ok())
.collect();
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
if self.policy == Policy::RequireAll {
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
}
}
}
Ok(verified)
@@ -201,7 +171,7 @@ impl Registry {
#[cfg(test)]
mod tests {
use super::*;
use crate::attestation::{did_key_from_verifying_key, Attestation, AttestationPayload};
use crate::attestation::{Attestation, AttestationPayload};
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
@@ -349,38 +319,6 @@ mod tests {
.any(|v| v.type_ == "other/v1" && !v.fully_verified));
}
/// The strict policies must surface a batch member's verification error
/// through `verify_all`, not launder it into a partial result. This binds
/// the POLICY BOUNDARY itself: only the `RequireAll` arm may drop failing
/// entries, and an apparently harmless cleanup that applies its
/// `filter_map(Result::ok)` shape to the strict arms would silently turn
/// the fail-closed contracts into fail-open — with every per-attestation
/// `verify` test still green, because none of them go through the batch
/// path. The failing entry is a wrong-cert-hash attestation (signed over a
/// different hash than the batch expects), a genuine `Err` under every
/// policy.
#[test]
fn strict_policies_propagate_a_batch_members_error() {
let sk = fresh();
let cert_hash = sample_hash();
let mut wrong_hash = sample_hash();
wrong_hash[0] ^= 0xff;
for policy in [Policy::AcceptKnown, Policy::RejectUnknown] {
let mut reg = Registry::new().with_policy(policy);
reg.register(DemoVerifier);
let good = signed_demo(&sk, cert_hash, "ok");
let bad = signed_demo(&sk, wrong_hash, "ok");
let err = reg
.verify_all(&[good, bad], cert_hash)
.expect_err("a strict policy must fail the whole batch");
assert!(
matches!(err, Error::Signature(_) | Error::CertHashMismatch),
"expected the underlying verification error, got {err:?}"
);
}
}
/// A required type that is present but unverified (no verifier registered
/// even though the type appears in `required_types`) must fail
/// `verify_all` with `RequiredMissing`. Catches a misconfiguration where
@@ -398,65 +336,6 @@ mod tests {
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
/// A cert-hash that does not match `sample_hash()`, so an attestation
/// signed against it fails `verify_signature` when the batch is checked
/// against `sample_hash()`.
fn other_hash() -> [u8; 32] {
let mut h = sample_hash();
h[0] ^= 0xff;
h
}
/// Under `RequireAll`, a malformed attestation (here: signed against the
/// wrong cert hash, so it fails signature verification) attached alongside a
/// valid required one must not abort the batch. The junk is dropped and the
/// cert still verifies. Before the fix, `verify_all` propagated the error
/// via `?` and the whole batch failed — the DoS vector the docstring warns
/// about.
#[test]
fn require_all_drops_a_malformed_extra_and_still_accepts() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// Signed against `other_hash()`, so it will not verify under `cert_hash`.
let malformed = signed_demo(&sk, other_hash(), "boom");
let real = signed_demo(&sk, cert_hash, "ok");
let verified = reg
.verify_all(&[malformed, real], cert_hash)
.expect("a malformed extra must not abort the batch under RequireAll");
assert_eq!(verified.len(), 1, "the malformed entry should be dropped");
assert!(verified
.iter()
.any(|v| v.type_ == "demo/v1" && v.fully_verified));
}
/// Leniency must not open a hole: if the *only* attestation of a required
/// type is malformed, dropping it leaves the required type absent and
/// `verify_all` must still fail with `RequiredMissing`. Guards against a fix
/// that skips errors so eagerly it lets an unverified required type pass.
#[test]
fn require_all_rejects_when_only_a_malformed_required_type_is_present() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// The sole demo/v1 is bound to the wrong cert hash → fails verification.
let malformed_required = signed_demo(&sk, other_hash(), "ok");
let err = reg
.verify_all(&[malformed_required], cert_hash)
.unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
#[test]
fn payload_check_failure_rejects() {
let sk = fresh();
@@ -495,52 +374,4 @@ mod tests {
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// The signer a consumer sees must be the canonical DID of the key that
/// actually verified, never a raw artifact string that could disagree.
/// `verify_signature` derives the key from the artifact's `signer` field
/// and the parser is canonical (base58btc, exact length, ed25519
/// multicodec), so a forged field fails verification outright: the
/// must-not here is that no `VerifiedAttestation` ever carries a signer
/// that did not verify.
#[test]
fn verified_signer_is_anchored_to_the_key_that_verified() {
let signer = fresh();
let other = fresh();
assert_ne!(
signer.verifying_key().to_bytes(),
other.verifying_key().to_bytes(),
"fixture precondition: distinct signing keys"
);
let cert_hash = sample_hash();
let reg = Registry::new();
// Control: an honest artifact reports the signer's canonical DID, and
// it agrees with the artifact field.
let att = signed_demo(&signer, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
let expected = did_key_from_verifying_key(&signer.verifying_key());
assert_eq!(v.signer, expected, "signer must be the verified key's DID");
assert_eq!(v.signer, att.signer, "honest artifact: field and key agree");
// Must-not, both directions: a signer field naming a DIFFERENT key
// than the one that signed must fail verification, so the artifact
// string can never redirect the reported signer.
let mut forged = signed_demo(&signer, cert_hash, "ok");
forged.signer = did_key_from_verifying_key(&other.verifying_key());
let err = reg.verify(&forged, cert_hash).unwrap_err();
assert!(
matches!(err, Error::Signature(_)),
"a signer field that does not match the signing key must fail, got {err:?}"
);
let mut forged_rev = signed_demo(&other, cert_hash, "ok");
forged_rev.signer = did_key_from_verifying_key(&signer.verifying_key());
let err_rev = reg.verify(&forged_rev, cert_hash).unwrap_err();
assert!(
matches!(err_rev, Error::Signature(_)),
"mismatched signer field must fail verification, got {err_rev:?}"
);
}
}
+11 -381
View File
@@ -5,28 +5,6 @@
//!
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
//!
//! ## Replay
//!
//! A certificate proves *who* asserted a ref update and *what* they asserted.
//! It does not prove the assertion is fresh. [`RefUpdateBody::seq`] and
//! [`RefUpdateBody::nonce`] are inside the signed bytes (see
//! [`RefUpdateBody::to_signing_bytes`]), so a verifier can trust the values,
//! but deciding whether a nonce is being reused or a `seq` is going backwards
//! needs state.
//!
//! This crate is deliberately stateless — it is embedded in both nodes and
//! clients — so replay is **not** enforced here and cannot be. Ownership of
//! the check sits with the verifier that accepts and persists certificates,
//! which must:
//!
//! - reject a `nonce` it has already accepted for the same `(repo, ref_name)`;
//! - reject a `seq` that does not advance past the last accepted `seq` for
//! that `(repo, ref_name)`.
//!
//! Until such a verifier exists, replay of ref-update certificates is not
//! enforced anywhere in this workspace. This contract exists so the first one
//! inherits the requirement instead of re-deriving it.
use std::collections::HashSet;
@@ -41,30 +19,6 @@ use crate::{Error, Result};
/// The certificate type discriminant. Always `"gitlawb/ref-update/v1"`.
pub const CERT_TYPE: &str = "gitlawb/ref-update/v1";
/// Exact url-safe unpadded base64 length of a 64-byte Ed25519 signature.
/// The tolerant threshold scan requires this canonical unpadded length before
/// attempting decode: the length gate is a DoS bound that keeps an
/// attacker-sized sig string from costing allocation before the signature is
/// rejected. A longer string is not meaningless (the same 64 bytes can appear
/// in padded or non-url-safe encodings) but no in-tree signer produces one,
/// so the scan skips non-canonical entries rather than decoding them.
const ENCODED_SIG_LEN: usize = 86;
/// Hard ceiling on signature entries a certificate may carry into the
/// tolerant threshold scan. A legitimate certificate holds one entry per
/// maintainer (a re-sign after key rotation may briefly leave a second), so
/// this is far above any real signer set, while turning the scan's worst case
/// from "one curve verification per attacker-appended entry" into a small
/// constant. Rejection is loud (`Err`), not a silent prefix scan: scanning
/// only the first N entries would let junk placed ahead of the real
/// signatures starve them out of the window, which is the #349 denial again
/// in positional form. An attacker positioned to inflate the list past this
/// ceiling can already suppress the certificate outright. The loud
/// rejection does deny a legitimately over-long certificate; that is an
/// explicit, detectable denial above the cap rather than a silent window
/// junk placement could exploit.
const MAX_SIGNATURE_ENTRIES: usize = 64;
/// A signature on a ref-update certificate.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CertSignature {
@@ -87,19 +41,11 @@ pub struct RefUpdateBody {
pub from: String,
/// The new commit hash (SHA-256 hex).
pub to: String,
/// Monotonically increasing sequence number, scoped to `(repo, ref_name)`.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a `seq` that does not
/// advance for the same repo and ref. See the module docs on replay.
/// Monotonically increasing sequence number. Prevents replay.
pub seq: u64,
/// RFC 3339 timestamp of this update.
pub timestamp: DateTime<Utc>,
/// Random nonce, unique per certificate.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a nonce it has already
/// accepted for the same repo and ref. See the module docs on replay.
/// Random nonce for deduplication.
pub nonce: String,
}
@@ -162,15 +108,9 @@ impl RefUpdateCert {
Ok(())
}
/// Verify all signatures on this certificate, failing closed.
/// Verify all signatures on this certificate.
///
/// On success returns the list of DIDs whose signatures are valid. This is
/// strict: it returns an error if *any* signature entry is malformed or
/// invalid. Because signature entries live outside the signed body (anyone
/// can append one without key material), do not use this to make a
/// threshold decision on an untrusted certificate — a single appended junk
/// entry would reject the whole cert. Use [`Self::satisfies_threshold`],
/// which ignores invalid entries and counts only the valid signers.
/// Returns the list of DIDs whose signatures are valid.
pub fn verify_all(&self) -> Result<Vec<Did>> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let signing_bytes = self.body.to_signing_bytes()?;
@@ -199,65 +139,12 @@ impl RefUpdateCert {
/// from the provided set of authorized maintainer DIDs.
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once. Malformed and invalid
/// signature entries are skipped, not fatal — entries live outside the
/// signed body, so any third party can append junk without key material,
/// and short-circuiting on the first bad one (as [`Self::verify_all`]
/// does) would let that junk deny an otherwise valid certificate (#349).
///
/// Tolerance must not become an unbounded verification path (#349 review
/// round 2): entries are attacker-extensible, so the work done for them is
/// bounded BEFORE the loop, not discovered inside it:
///
/// - a list over [`MAX_SIGNATURE_ENTRIES`] is rejected outright (see the
/// constant for why loud rejection beats a silent prefix scan);
/// - an encoded signature is length-gated before base64 work — a 64-byte
/// Ed25519 signature is exactly [`ENCODED_SIG_LEN`] characters in
/// url-safe unpadded base64, so an oversized blob never allocates;
/// - entries whose signer is not in `maintainers`, or is already counted,
/// are skipped on a string compare, before any decode or curve work —
/// so even within the cap, junk naming strangers costs no crypto;
/// - the scan returns as soon as the threshold is met.
/// signature from the same maintainer counts once.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
if threshold == 0 {
return Err(Error::RefCert("threshold must be at least 1".to_string()));
}
if self.signatures.len() > MAX_SIGNATURE_ENTRIES {
return Err(Error::RefCert(format!(
"certificate carries {} signature entries (max {})",
self.signatures.len(),
MAX_SIGNATURE_ENTRIES
)));
}
let signing_bytes = self.body.to_signing_bytes()?;
let mut counted: HashSet<&Did> = HashSet::new();
for cert_sig in &self.signatures {
if counted.contains(&cert_sig.signer) || !maintainers.contains(&cert_sig.signer) {
continue;
}
if cert_sig.sig.len() != ENCODED_SIG_LEN {
continue;
}
let Ok(vk) = cert_sig.signer.to_verifying_key() else {
continue;
};
let Ok(sig_bytes_vec) = URL_SAFE_NO_PAD.decode(&cert_sig.sig) else {
continue;
};
let Ok(sig_bytes) = <[u8; 64]>::try_from(sig_bytes_vec) else {
continue;
};
if verify(&vk, &signing_bytes, &sig_bytes).is_ok() {
counted.insert(&cert_sig.signer);
if counted.len() >= threshold {
return Ok(true);
}
}
}
Ok(counted.len() >= threshold)
let valid = self.verify_all()?;
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
Ok(distinct_signers.len() >= threshold)
}
/// Validate the certificate structure (not signatures).
@@ -490,151 +377,8 @@ mod tests {
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
/// An attacker can append junk signature entries to a cert (they live
/// outside the signed body, so no key material is needed). `satisfies_threshold`
/// must ignore them and still count the real signers — before the fix,
/// `verify_all`'s `?` on the junk made the whole check error out, denying a
/// valid 2-of-2 cert.
#[test]
fn satisfies_threshold_ignores_appended_junk_signatures() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
cert.countersign(&kp2).unwrap();
// Junk sits AHEAD of the real entries so the scan must actually skip
// it: appended last, the threshold-2 early return at the second real
// signature would fire before any junk is examined and the test would
// stay green even if the skip logic never ran.
cert.signatures.insert(
0,
CertSignature {
signer: kp1.did(),
sig: "!!!not-base64!!!".to_string(),
},
);
// Valid base64 but only 3 bytes — fails the 64-byte length check.
cert.signatures.insert(
0,
CertSignature {
signer: kp2.did(),
sig: "AAAA".to_string(),
},
);
// Exactly ENCODED_SIG_LEN of valid base64 decoding to garbage —
// exercises the decode+verify path before the skip, not just the
// length gate.
cert.signatures.insert(
0,
CertSignature {
signer: kp1.did(),
sig: "A".repeat(ENCODED_SIG_LEN),
},
);
let maintainers = vec![kp1.did(), kp2.did()];
assert!(cert.satisfies_threshold(&maintainers, 2).unwrap());
}
/// A junk entry that names a real maintainer DID but carries a signature
/// that does not verify must not be counted — otherwise appending garbage
/// under a maintainer's DID could forge a threshold.
#[test]
fn satisfies_threshold_does_not_count_a_forged_maintainer_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Well-formed 64-byte signature from kp2, but over unrelated bytes, so
// it fails verification against the cert body.
cert.signatures.push(CertSignature {
signer: kp2.did(),
sig: kp2.sign_b64(b"unrelated bytes"),
});
let maintainers = vec![kp1.did(), kp2.did()];
// Only kp1 actually signed the body: 2-of-2 must fail, 1-of-2 holds.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
}
/// The tolerant scan must not be an unbounded verification path: a list
/// past MAX_SIGNATURE_ENTRIES is rejected loudly before any curve work,
/// while a list at the cap still verifies normally.
#[test]
fn satisfies_threshold_rejects_an_oversized_signature_list() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let junk = || CertSignature {
signer: Keypair::generate().did(),
sig: "AAAA".to_string(),
};
while cert.signatures.len() < MAX_SIGNATURE_ENTRIES {
cert.signatures.push(junk());
}
let maintainers = vec![kp.did()];
// At the cap: still fine, the real signature counts.
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
// One past the cap: loud rejection, no tolerant scan.
cert.signatures.push(junk());
let err = cert.satisfies_threshold(&maintainers, 1).unwrap_err();
assert!(err.to_string().contains("signature entries"));
}
/// An entry whose encoded signature is not the exact base64 length of a
/// 64-byte Ed25519 signature is skipped before any base64 decode — an
/// attacker-sized sig string must cost neither allocation nor curve work,
/// and must not affect the verdict.
#[test]
fn satisfies_threshold_skips_an_oversized_encoded_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Names a real maintainer, valid base64 alphabet, but megabytes long:
// the length gate must refuse it without decoding.
cert.signatures.push(CertSignature {
signer: kp2.did(),
sig: "A".repeat(2 * 1024 * 1024),
});
let maintainers = vec![kp1.did(), kp2.did()];
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_rejected() {
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
@@ -645,120 +389,6 @@ mod tests {
&kp,
)
.unwrap();
// Zero must never read as "no signatures required", even with an empty
// maintainer set.
let err = cert.satisfies_threshold(&[], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"zero must be refused as an invalid threshold"
);
}
/// The threshold is checked before any signature work, so a zero is
/// reported as a threshold error even when the certificate's signatures
/// are also invalid. Otherwise a broken certificate would let a caller
/// mistake the failure for a legitimate verification result.
#[test]
fn threshold_zero_is_rejected_before_signature_verification() {
let kp = Keypair::generate();
let other = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures[0].sig = other.sign_b64(b"unrelated bytes");
let err = cert.satisfies_threshold(&[kp.did()], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"the invalid threshold must win over the invalid signature"
);
}
/// `seq` and `nonce` are the verifier's replay inputs, so they have to be
/// inside the signed body; tapping either one must invalidate the
/// certificate rather than leave it verifying with different values.
#[test]
fn replay_fields_are_covered_by_the_signature() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
7,
&kp,
)
.unwrap();
cert.verify_all().expect("baseline must verify");
let mut bumped = cert.clone();
bumped.body.seq = 8;
assert!(
matches!(bumped.verify_all(), Err(Error::SignatureInvalid)),
"changing seq must invalidate the signature"
);
let mut renamed = cert.clone();
renamed.body.nonce = "different-nonce".to_string();
assert!(
matches!(renamed.verify_all(), Err(Error::SignatureInvalid)),
"changing nonce must invalidate the signature"
);
}
/// A signature minted by a real key must not verify when the signer is
/// spelled in another multibase alphabet. Otherwise the same identity
/// would satisfy a maintainer allowlist under several DID strings.
///
/// Base64url is deliberate: its 47-character method-id fits under the
/// pre-existing length bound, so this test reaches the canonical-prefix
/// rule and fails on the pre-fix code instead of being refused for length.
#[test]
fn verify_all_rejects_a_non_canonical_did_spelling() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let mut prefixed = vec![0xed, 0x01];
prefixed.extend_from_slice(&kp.verifying_key().to_bytes());
cert.signatures[0].signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base64Url, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = cert
.verify_all()
.expect_err("a non-canonical did:key spelling must not resolve");
assert!(
err.to_string().contains("base58btc"),
"rejection must come from the canonical-prefix rule, not the length guard"
);
assert!(
!matches!(
cert.satisfies_threshold(&[cert.signatures[0].signer.clone()], 1),
Ok(true)
),
"the alternate spelling must not satisfy a threshold"
);
// The canonical spelling of the same key is unaffected.
cert.signatures[0].signer = kp.did();
assert!(cert.satisfies_threshold(&[kp.did()], 1).unwrap());
assert!(cert.satisfies_threshold(&[], 0).unwrap());
}
}
+3 -209
View File
@@ -20,19 +20,6 @@ use crate::{Error, Result};
/// Value: 0xed01 encoded as a varint.
const ED25519_MULTICODEC: &[u8] = &[0xed, 0x01];
/// The only multibase prefix a `did:key` method-specific id may carry.
///
/// The DID:key spec encodes Ed25519 keys as base58btc, which is multibase
/// `z`. `multibase::decode` happily accepts every other alphabet, and each
/// spelling of the same key bytes would produce a distinct DID string while
/// gates, allowlists, and database rows compare DIDs as strings — one
/// identity would read as several. Rejecting the alternatives here keeps the
/// mapping from `did:key` to key single-valued for every caller of
/// [`Did::to_verifying_key`] (HTTP signatures, UCAN, ref certs, peer
/// announces). `gitlawb-attest` enforces the same rule for attestation
/// signers.
const DID_KEY_MULTIBASE_PREFIX: char = 'z';
/// A Decentralized Identifier.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)]
@@ -84,14 +71,7 @@ impl Did {
/// Resolve the Ed25519 verifying key from a `did:key`.
///
/// Only the canonical base58btc (`z`) spelling is accepted. Other
/// multibase encodings decode to the same key but spell a different DID
/// string, so accepting them would let one identity appear under several
/// identifiers to string-comparing gates and allowlists.
///
/// Returns `Err` if the DID is not a `did:key`, uses a non-canonical
/// multibase prefix, the key bytes are invalid, or the key is a
/// small-order (weak) point.
/// Returns `Err` if the DID is not a `did:key` or the key bytes are invalid.
pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() {
return Err(Error::InvalidDid(format!(
@@ -113,27 +93,9 @@ impl Did {
));
}
// Cheap prefix gate before the decode, then the base check on what
// multibase actually parsed. The first keeps a clearly non-canonical id
// off the decoder. The second is belt-and-braces — multibase maps a
// leading `z` only to `Base58Btc` — but it states the requirement
// directly instead of relying on that mapping, and `gitlawb-attest`
// makes the same pair of checks.
if !self.method_id().starts_with(DID_KEY_MULTIBASE_PREFIX) {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
let (base, bytes) =
let (_, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
if base != Base::Base58Btc {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
if !bytes.starts_with(ED25519_MULTICODEC) {
return Err(Error::InvalidDid(
"not an ed25519 multicodec key".to_string(),
@@ -144,25 +106,7 @@ impl Did {
.try_into()
.map_err(|_| Error::InvalidDid("ed25519 key must be 32 bytes".to_string()))?;
let key =
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))?;
// `from_bytes` only decompresses, so it accepts a small-order point.
// Such a key satisfies the verification equation for any message, and
// every X25519 shared secret derived from it is the all-zero key: the
// order-1 and order-2 points convert to the all-zero Montgomery u
// directly, and the order-4 and order-8 points are annihilated by the
// scalar clamping instead. `is_weak` covers the whole torsion set.
// Mixed-order points are correctly NOT rejected, since clamping clears
// the cofactor component. Rejecting
// at resolution is what makes this the choke point: every consumer that
// resolves a DID through here inherits the rejection, and a recipient
// that cannot resolve already fails closed downstream.
if key.is_weak() {
return Err(Error::InvalidDid("small-order ed25519 key".to_string()));
}
Ok(key)
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))
}
/// Return the full DID string as a `&str`.
@@ -425,154 +369,4 @@ mod tests {
);
did.to_verifying_key().expect("a real did:key must resolve");
}
/// `multibase::decode` accepts every alphabet, so the same Ed25519 key can
/// be spelled as several `did:key` strings. Only base58btc is canonical;
/// the rest must be refused so one identity cannot appear as several to
/// gates and allowlists that compare DID strings.
#[test]
fn to_verifying_key_rejects_non_base58btc_multibase() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[9u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let canonical = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
canonical.to_verifying_key().unwrap().to_bytes(),
signing.verifying_key().to_bytes(),
"base58btc is the accepted spelling"
);
// These spellings fit inside the method-id bound, so they reach the
// canonical-prefix check and must be refused there.
for base in [Base::Base64Url, Base::Base32Lower] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert_ne!(
did.as_str(),
canonical.as_str(),
"the same key must not compare equal under another spelling"
);
let err = did
.to_verifying_key()
.expect_err("non-canonical multibase must be refused");
assert!(
matches!(err, Error::InvalidDid(_)),
"must be an InvalidDid, got {err:?}"
);
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
// A base16 spelling of the same key is longer than the method-id
// bound, so it is refused by the length guard before the prefix check.
// Either way it must not resolve.
for base in [Base::Base16Lower, Base::Base16Upper] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert!(
matches!(did.to_verifying_key(), Err(Error::InvalidDid(_))),
"a base16 spelling of a real key must not resolve"
);
}
}
/// base58flickr (`Z`) is the non-bitcoin sibling alphabet. Its spelling of
/// a real key fits under the length bound, so it must be refused for the
/// prefix, not merely for its size.
#[test]
fn to_verifying_key_rejects_a_base58flickr_spelling() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[11u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!(
"did:key:{}",
multibase::encode(Base::Base58Flickr, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("base58flickr must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// An empty method-specific id must be refused by the prefix rule without
/// panicking on a missing first character.
#[test]
fn to_verifying_key_rejects_an_empty_method_id() {
let did: Did = "did:key:".parse().expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("an empty method id must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// A canonical base58btc spelling is not enough on its own: the bytes must
/// also carry the ed25519-pub multicodec. This pins the check that stops a
/// key of another type from resolving as an Ed25519 key.
#[test]
fn to_verifying_key_rejects_base58btc_with_a_wrong_multicodec() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[12u8; 32]);
let mut buf = vec![0x12, 0x00]; // sha2-256, not ed25519-pub
buf.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!("did:key:{}", multibase::encode(Base::Base58Btc, &buf))
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("a non-ed25519 multicodec must be refused");
assert!(
err.to_string().contains("ed25519"),
"the error must name the expected multicodec: {err}"
);
}
/// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key
/// satisfies the verification equation for any message and, converted to
/// Montgomery form for X25519, yields the all-zero shared secret, so an
/// envelope sealed to it is decryptable by anyone. Resolution is the choke
/// point every `Did`-based consumer shares, so it fails closed here.
#[test]
fn to_verifying_key_rejects_a_small_order_key() {
// The compressed identity point, the canonical small-order key.
let mut weak = [0u8; 32];
weak[0] = 1;
let mut prefixed = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
prefixed.extend_from_slice(ED25519_MULTICODEC);
prefixed.extend_from_slice(&weak);
let did: Did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &prefixed)
)
.parse()
.expect("a small-order did:key is still well-formed");
let err = did
.to_verifying_key()
.expect_err("a small-order did:key must not resolve");
match err {
Error::InvalidDid(msg) => assert!(
msg.contains("small-order"),
"rejection must name the small-order key, got: {msg}"
),
other => panic!("expected Error::InvalidDid, got {other:?}"),
}
}
}
+2 -501
View File
@@ -11,22 +11,6 @@ use zeroize::Zeroizing;
/// X25519 public key (Montgomery u) for an Ed25519 verifying key.
fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
use curve25519_dalek::edwards::CompressedEdwardsY;
// Every X25519 shared secret derived from a small-order point is the
// all-zero key: order-1 and order-2 convert to the all-zero Montgomery u
// directly, order-4 and order-8 are annihilated by the scalar clamping in
// x25519_secret_from_seed. Either way the per-recipient wrap could be
// rebuilt by anyone. Resolution already refuses such a key (see
// Did::to_verifying_key); this guard covers the SEAL side on its own terms
// for a caller that obtained the key some other way. The open side
// (open_blob's attacker-supplied `eph`) skips any entry whose exchange with
// this reader yields the all-zero shared secret
// (`yields_all_zero_shared_secret`), regardless of whether the bytes decode
// to an Edwards point.
if vk.is_weak() {
return Err(anyhow::anyhow!("verifying key is a small-order point"));
}
let edwards = CompressedEdwardsY::from_slice(vk.as_bytes())
.ok()
.and_then(|c| c.decompress())
@@ -34,26 +18,6 @@ fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
Ok(edwards.to_montgomery().to_bytes())
}
/// True when the X25519 exchange between an attacker-supplied `u` and this
/// reader's scalar yields the all-zero shared secret.
///
/// Asked of the RESULT, not of the encoding, and that distinction is the whole
/// guard. The previous version decompressed `u` to an Edwards point and asked
/// `is_small_order`, which reports "safe" for any low-order encoding that does
/// not decompress at all. Measured across the seven standard low-order
/// encodings, six were caught and `u = p - 1` was not: it fails to decompress,
/// so the check returned false, and the exchange is still the all-zero secret.
/// An entry built on it unwraps for every reader.
///
/// Enumerating encodings cannot be exhaustive, because the attack depends on the
/// shared secret rather than on the bytes that produced it, and twist and
/// non-canonical inputs keep arriving. Asking the result needs no list.
fn yields_all_zero_shared_secret(u: &[u8; 32], scalar: &[u8; 32]) -> bool {
use curve25519_dalek::montgomery::MontgomeryPoint;
MontgomeryPoint(*u).mul_clamped(*scalar).to_bytes() == [0u8; 32]
}
/// X25519 secret scalar for an Ed25519 seed (SHA-512 of seed, lower 32, clamped).
/// Returns the scalar wrapped in `Zeroizing`, and scrubs the intermediate
/// SHA-512 digest, so no copy of this secret material lingers in freed memory.
@@ -85,13 +49,6 @@ use serde::{Deserialize, Serialize};
const MAGIC: &[u8] = b"GLENC";
const VERSION: u8 = 2;
/// Upper bound on the blinded recipient list. The envelope author controls
/// this count and `open_blob` pays one X25519 exchange per entry before it can
/// answer "not a recipient", so the count has to be capped rather than
/// optimized (#419). Legitimate envelopes carry a handful of authorized
/// readers, so 256 is far above real use.
const MAX_RECIPIENTS: usize = 256;
#[derive(Serialize, Deserialize)]
struct Recipient {
eph: String, // base64 ephemeral x25519 pubkey (32B)
@@ -111,14 +68,6 @@ pub fn seal_blob(plaintext: &[u8], recipients: &[VerifyingKey]) -> Result<Vec<u8
if recipients.is_empty() {
return Err(anyhow::anyhow!("seal_blob: no recipients"));
}
// Same bound open_blob enforces: an envelope over the cap could never be
// opened, so refuse to seal it (#419).
if recipients.len() > MAX_RECIPIENTS {
return Err(anyhow::anyhow!(
"seal_blob: {} recipients, over the {MAX_RECIPIENTS} cap",
recipients.len()
));
}
let mut content_key = [0u8; 32];
OsRng.fill_bytes(&mut content_key);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
@@ -177,20 +126,9 @@ pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
let header: Header =
serde_json::from_slice(envelope.get(p..p + hlen).context("truncated header")?)
.context("decode header")?;
// Reject before the recipient scan: every entry below is an X25519
// exchange, and a non-recipient would run the whole list (#419).
if header.recipients.len() > MAX_RECIPIENTS {
return Err(anyhow::anyhow!(
"envelope carries {} recipients, over the {MAX_RECIPIENTS} cap",
header.recipients.len()
));
}
let body = &envelope[p + hlen..];
// The raw scalar is kept alongside the box secret so the exchange can be
// tested BEFORE a box is built from an attacker-supplied ephemeral.
let my_x_scalar = x25519_secret_from_seed(&keypair.to_seed());
let my_x = XSecret::from(*my_x_scalar);
let my_x = XSecret::from(*x25519_secret_from_seed(&keypair.to_seed()));
// Identities are blinded: no entry says which recipient it belongs to, so
// try each one. The ChaChaBox AEAD tag authenticates, so exactly the
@@ -202,12 +140,7 @@ pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
.ok()
.and_then(|b| <[u8; 32]>::try_from(b.as_slice()).ok())
{
// An ephemeral whose exchange with this reader is the all-zero
// shared secret would unwrap for anyone, so the entry is skipped.
// Tested on the exchange itself: an encoding-shaped check misses
// every low-order input that does not decompress to Edwards.
Some(b) if !yields_all_zero_shared_secret(&b, &my_x_scalar) => XPublic::from(b),
Some(_) => continue,
Some(b) => XPublic::from(b),
None => continue,
};
// from_slice panics on a wrong length, and the envelope is attacker
@@ -360,436 +293,4 @@ mod tests {
header["nonce"] = bad_nonce;
assert!(open_blob(&reframe(&header), &reader).is_err());
}
#[test]
fn over_cap_recipient_list_is_rejected_before_the_scan() {
// #419: the envelope author controls the recipient count and each entry
// costs an X25519 exchange, so the list is capped. Reframe a valid
// envelope with MAX_RECIPIENTS + 1 junk entries.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
let junk = serde_json::json!({
"eph": B64.encode([0u8; 32]),
"nonce": B64.encode([0u8; 24]),
"wrap": B64.encode([0u8; 48]),
});
header["recipients"] = serde_json::Value::Array(vec![junk.clone(); MAX_RECIPIENTS + 1]);
let hj = serde_json::to_vec(&header).unwrap();
let mut fat = Vec::new();
fat.extend_from_slice(MAGIC);
fat.push(VERSION);
fat.extend_from_slice(&(hj.len() as u32).to_le_bytes());
fat.extend_from_slice(&hj);
fat.extend_from_slice(body);
let err = open_blob(&fat, &reader).unwrap_err();
assert!(
err.to_string().contains("recipients"),
"expected the recipient-cap error, got: {err}"
);
// Exactly at the cap the envelope is still processed end to end.
header["recipients"] = serde_json::Value::Array(vec![junk; MAX_RECIPIENTS]);
let hj = serde_json::to_vec(&header).unwrap();
let mut at_cap = Vec::new();
at_cap.extend_from_slice(MAGIC);
at_cap.push(VERSION);
at_cap.extend_from_slice(&(hj.len() as u32).to_le_bytes());
at_cap.extend_from_slice(&hj);
at_cap.extend_from_slice(body);
let err = open_blob(&at_cap, &reader).unwrap_err();
assert!(
!err.to_string().contains("over the"),
"at-cap envelope must reach the scan, got: {err}"
);
}
#[test]
fn seal_refuses_a_recipient_list_over_the_cap() {
// An envelope over the cap can never be opened, so seal must reject it
// instead of minting one.
let keys: Vec<VerifyingKey> = (0..MAX_RECIPIENTS + 1)
.map(|_| Keypair::generate().verifying_key())
.collect();
let err = seal_blob(b"blob", &keys).unwrap_err();
assert!(
err.to_string().contains("recipients"),
"expected the recipient-cap error, got: {err}"
);
}
/// Every standard low-order X25519 encoding must be detected by
/// `yields_all_zero_shared_secret`, including encodings that do not
/// decompress to an Edwards point (notably `u = p - 1`). The open side
/// rejects on the exchange result, not on decode shape, because an
/// encoding-shaped check misses non-decompressing low-order inputs while
/// still yielding the all-zero shared secret.
#[test]
fn all_zero_shared_secret_is_detected_for_every_standard_low_order_encoding() {
// The seven standard X25519 low-order encodings, driven as a set rather
// than as the two that happen to decompress. The encoding-shaped check
// this replaced reported `p - 1` as safe, and its exchange is all-zero.
let mut one = [0u8; 32];
one[0] = 1;
let hex = |h: &str| -> [u8; 32] {
let mut out = [0u8; 32];
for i in 0..32 {
out[i] = u8::from_str_radix(&h[i * 2..i * 2 + 2], 16).unwrap();
}
out
};
let vectors = [
("u=0", [0u8; 32]),
("u=1", one),
(
"order8-a",
hex("e0eb7a7c3b41b8ae1656e3faf19fc46ada098deb9c32b1fd866205165f49b800"),
),
(
"order8-b",
hex("5f9c95bca3508c24b1d0b1559c83ef5b04445cc4581c8e86d8224eddd09f1157"),
),
(
"p-1",
hex("ecffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
(
"p",
hex("edffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
(
"p+1",
hex("eeffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
];
let kp = Keypair::generate();
let scalar = x25519_secret_from_seed(&kp.to_seed());
for (name, u) in vectors {
assert!(
yields_all_zero_shared_secret(&u, &scalar),
"{name} is a low-order encoding and must be detected"
);
}
// And the other direction, or a guard that returned true always would
// pass everything above while making the envelope unopenable.
let real_u = x25519_public(&kp.verifying_key()).unwrap();
assert!(
!yields_all_zero_shared_secret(&real_u, &scalar),
"a real key must not be treated as low-order"
);
}
/// The open side must skip an entry whose header-supplied `eph` is a
/// low-order point: such an entry forces the all-zero shared secret and
/// would unwrap for any reader. A poisoned entry among honest ones must
/// not DoS the envelope — the honest entry still opens.
#[test]
fn open_blob_skips_a_low_order_ephemeral_and_still_opens_honest_entry() {
use curve25519_dalek::montgomery::MontgomeryPoint;
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Split the envelope framing into header JSON and body.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
// A low-order Montgomery u (u = 0) encoded as the entry's eph.
let low_order_u = MontgomeryPoint([0u8; 32]).to_bytes();
let poisoned = serde_json::json!({
"eph": B64.encode(low_order_u),
"nonce": B64.encode([0u8; 24]),
"wrap": B64.encode([0u8; 32]),
});
let reframe = |header: &serde_json::Value| -> Vec<u8> {
let hj = serde_json::to_vec(header).unwrap();
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(hj.len() as u32).to_le_bytes());
out.extend_from_slice(&hj);
out.extend_from_slice(body);
out
};
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
// Prepend the poisoned entry. The reader's honest entry must still
// unwrap, so the envelope opens despite the low-order entry.
let recipients = header["recipients"].as_array_mut().unwrap();
recipients.insert(0, poisoned);
assert_eq!(
open_blob(&reframe(&header), &reader).unwrap(),
b"private blob contents",
"the honest entry must still open despite a low-order eph entry"
);
}
/// Must-not for the open-side guard: an envelope whose ONLY entry is built
/// on a low-order ephemeral must not open for any reader. Without the
/// guard the all-zero shared secret unwraps for everyone, so the attacker
/// controls the plaintext and the envelope's confidentiality is void.
#[test]
fn open_blob_rejects_a_poisoned_only_envelope() {
use chacha20poly1305::{aead::Aead, KeyInit, XChaCha20Poly1305, XNonce};
use crypto_box::aead::AeadCore;
use curve25519_dalek::montgomery::MontgomeryPoint;
// Build a poisoned envelope from scratch: one recipient entry whose
// eph is the low-order u = 0. The wrap is the encryption of a content
// key under the all-zero shared secret, so without the guard any
// reader's ChaChaBox against u = 0 decrypts it.
let mut content_key: [u8; 32] = Default::default();
OsRng.fill_bytes(&mut content_key);
let attacker_x = XSecret::generate(&mut OsRng);
let zero_box = ChaChaBox::new(&XPublic::from(<[u8; 32]>::default()), &attacker_x);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let wrap = zero_box.encrypt(&nonce, &content_key[..]).unwrap();
let mut body_nonce: [u8; 24] = Default::default();
OsRng.fill_bytes(&mut body_nonce);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let body = body_cipher
.encrypt(
XNonce::from_slice(&body_nonce),
b"attacker-chosen plaintext".as_slice(),
)
.unwrap();
let header = serde_json::json!({
"alg": "xchacha20poly1305",
"nonce": B64.encode(body_nonce),
"recipients": [{
"eph": B64.encode(MontgomeryPoint([0u8; 32]).to_bytes()),
"nonce": B64.encode(nonce),
"wrap": B64.encode(wrap),
}],
});
let header_json = serde_json::to_vec(&header).unwrap();
let mut env = Vec::new();
env.extend_from_slice(MAGIC);
env.push(VERSION);
env.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
env.extend_from_slice(&header_json);
env.extend_from_slice(&body);
let reader = Keypair::generate();
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("not a recipient"),
"a poisoned-only envelope must not open for anyone, got: {err}"
);
}
/// The same must-not, driven with the low-order encoding that does NOT map
/// to an Edwards point.
///
/// This is the case an encoding-shaped guard cannot see. `u = p - 1` is a
/// legal X25519 input whose exchange is the all-zero secret for any scalar,
/// and `MontgomeryPoint::to_edwards` returns None for it, so a check written
/// as "decompress, then ask is_small_order" reports it as safe. Measured
/// across the seven standard low-order encodings: six are caught that way
/// and this one is not.
///
/// Found by a cross-family review of the head that first added the guard,
/// and the comment beside that guard asserted this case was already handled
/// by a decode path that does not exist. That is why the fix rejects on the
/// RESULT of the exchange rather than on a list of encodings: the result is
/// what the attack actually depends on, and it needs no enumeration to be
/// exhaustive.
#[test]
fn open_blob_rejects_a_non_edwards_low_order_ephemeral() {
// No blanket `use ...::Aead` here. Two aead versions are reachable, and a
// blanket import silently resolved the wrap through the one open_blob does
// NOT decrypt with, producing a ciphertext that could never open. The test
// then passed with the guard, without it, and with the original pre-fix
// guard alike, proving nothing. Every call below names its trait.
use chacha20poly1305::{KeyInit, XChaCha20Poly1305, XNonce};
use crypto_box::aead::AeadCore;
// u = p - 1 = 2^255 - 20, little endian.
let mut eph = [0xffu8; 32];
eph[0] = 0xec;
eph[31] = 0x7f;
let mut content_key: [u8; 32] = Default::default();
OsRng.fill_bytes(&mut content_key);
let attacker_x = XSecret::generate(&mut OsRng);
let zero_box = ChaChaBox::new(&XPublic::from(<[u8; 32]>::default()), &attacker_x);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let wrap = crypto_box::aead::Aead::encrypt(&zero_box, &nonce, &content_key[..]).unwrap();
let mut body_nonce: [u8; 24] = Default::default();
OsRng.fill_bytes(&mut body_nonce);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let body = chacha20poly1305::aead::Aead::encrypt(
&body_cipher,
XNonce::from_slice(&body_nonce),
b"attacker-chosen plaintext".as_slice(),
)
.unwrap();
let header = serde_json::json!({
"alg": "xchacha20poly1305",
"nonce": B64.encode(body_nonce),
"recipients": [{
"eph": B64.encode(eph),
"nonce": B64.encode(nonce),
"wrap": B64.encode(wrap),
}],
});
let header_json = serde_json::to_vec(&header).unwrap();
let mut env = Vec::new();
env.extend_from_slice(MAGIC);
env.push(VERSION);
env.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
env.extend_from_slice(&header_json);
env.extend_from_slice(&body);
let reader = Keypair::generate();
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("not a recipient"),
"an envelope whose only entry uses a non-Edwards low-order eph must not \
open for anyone, got: {err}"
);
}
/// The compressed identity point: a well-formed Ed25519 encoding that is
/// small-order, so every shared secret derived from it is all-zero.
fn weak_verifying_key() -> VerifyingKey {
let mut weak = [0u8; 32];
weak[0] = 1;
let vk = VerifyingKey::from_bytes(&weak).expect("identity point decompresses");
assert!(vk.is_weak(), "fixture precondition: key is small-order");
vk
}
/// A small-order recipient key converts to Montgomery u = 0, and X25519
/// against u = 0 is the all-zero shared secret for ANY scalar, so the
/// wrapping box is reconstructable with no secret at all. Reject at the
/// primitive so the seal side is safe regardless of how the caller
/// obtained the key.
#[test]
fn x25519_public_rejects_a_small_order_key() {
let weak_vk = weak_verifying_key();
assert!(
x25519_public(&weak_vk).is_err(),
"a small-order key must not yield an x25519 public key"
);
}
/// Control: a legitimate key still converts, and to a non-zero u.
#[test]
fn x25519_public_still_accepts_a_real_key() {
let kp = Keypair::generate();
let u = x25519_public(&kp.verifying_key()).expect("a real key must convert");
assert_ne!(u, [0u8; 32], "a real key must not map to the zero point");
}
/// The guard has to propagate: no envelope may be produced for a weak
/// recipient even when the DID choke point is bypassed entirely.
#[test]
fn seal_blob_refuses_a_small_order_recipient() {
let weak_vk = weak_verifying_key();
assert!(
seal_blob(b"withheld", &[weak_vk]).is_err(),
"sealing to a weak recipient must fail"
);
// And a mixed set must fail too: one weak recipient exposes the shared
// content key, so a partial envelope is not an acceptable outcome.
let honest = Keypair::generate();
assert!(
seal_blob(b"withheld", &[honest.verifying_key(), weak_vk]).is_err(),
"a mixed honest+weak recipient set must fail closed, not seal partially"
);
}
/// Control for the two above: the ordinary seal/open round trip is intact.
#[test]
fn legit_seal_open_round_trip_still_works() {
let reader = Keypair::generate();
let env = seal_blob(b"withheld blob", &[reader.verifying_key()]).expect("seal");
assert_eq!(open_blob(&env, &reader).expect("open"), b"withheld blob");
}
/// The defect this fix exists to close, kept as an executable regression
/// rather than an assertion about absence. It runs the real attack: craft a
/// small-order did:key, get it into a recipient set alongside an honest
/// reader, then rebuild the wrapping box from the all-zero shared secret
/// that a small-order recipient forces. If any guard regresses, this does
/// not merely fail, it fails printing the plaintext it recovered.
#[test]
fn attacker_cannot_recover_plaintext_via_a_weak_recipient() {
use crate::did::Did;
use std::str::FromStr;
let weak_vk = weak_verifying_key();
// Layer 1: the attacker's did:key string must not resolve at all.
let weak_did = Did::from_verifying_key(&weak_vk).to_string();
assert!(
Did::from_str(&weak_did)
.expect("still a well-formed did:key")
.to_verifying_key()
.is_err(),
"a small-order did:key must not resolve"
);
// Layer 2: even handed the key directly, sealing must refuse. One weak
// recipient would expose the single shared content key, so the honest
// reader's blob would be readable by anyone.
let honest = Keypair::generate();
let secret_plaintext = b"WITHHELD BLOB CONTENTS";
let envelope = match seal_blob(secret_plaintext, &[honest.verifying_key(), weak_vk]) {
Err(_) => return, // no envelope exists, nothing to attack
Ok(env) => env,
};
// Only reachable if a guard regressed. Run the attack and report what
// it got, so the failure names the actual exposure.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(envelope[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header: serde_json::Value = serde_json::from_slice(&envelope[p..p + hlen]).unwrap();
let body = &envelope[p + hlen..];
let body_nonce = B64.decode(header["nonce"].as_str().unwrap()).unwrap();
// X25519 against u = 0 is the all-zero shared secret for any scalar, so
// the sealer's box is reconstructable with a key the attacker picks.
let zero_box = ChaChaBox::new(
&XPublic::from(<[u8; 32]>::default()),
&XSecret::generate(&mut OsRng),
);
for r in header["recipients"].as_array().unwrap() {
let n = B64.decode(r["nonce"].as_str().unwrap()).unwrap();
let w = B64.decode(r["wrap"].as_str().unwrap()).unwrap();
if let Ok(content_key) = zero_box.decrypt(n.as_slice().into(), w.as_slice()) {
let cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let recovered = cipher
.decrypt(XNonce::from_slice(&body_nonce), body)
.expect("body decrypts once the content key is out");
panic!(
"REGRESSION: attacker recovered plaintext with no private key: {:?}",
String::from_utf8_lossy(&recovered)
);
}
}
panic!("an envelope was sealed to a small-order recipient; the seal guard regressed");
}
}
-20
View File
@@ -566,26 +566,6 @@ mod tests {
assert!(sig.check_created().is_ok());
}
#[test]
fn extreme_created_timestamps_reject_without_wrapping() {
let kp = Keypair::generate();
let did = kp.did();
let parse = |created: i64| {
let sig_input =
format!(r#"sig1=("@method");keyid="{did}";alg="ed25519";created={created}"#);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
};
// i64::MIN used to panic in debug (subtraction overflow) and wrap to a
// passing skew in release; now - 2^63 wrapped to i64::MIN, whose abs()
// wrapped back to itself and read as fresh. Both must reject.
for created in [i64::MIN, i64::MAX, Utc::now().timestamp() - i64::MAX] {
assert!(
parse(created).check_created().is_err(),
"created={created} must not pass the freshness gate"
);
}
}
#[test]
fn parse_error_missing_sig1_prefix() {
let err = HttpSignature::parse(
-40
View File
@@ -77,14 +77,6 @@ impl Keypair {
}
/// Verify an Ed25519 signature.
///
/// Strict verification: rejects small-order `R` and small-order public keys
/// (the identity point, and any point of low order). Ordinary `verify` accepts
/// a signature forged with the identity point as the public key plus
/// `R = identity, S = 0`, which verifies for *any* message. `identity::verify`
/// is the shared primitive behind HTTP request authentication, UCANs, and
/// certificates, so weak-key acceptance is an authentication bypass, not a
/// malleability nuance.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
let sig = Signature::from_bytes(sig_bytes);
verifying_key
@@ -216,38 +208,6 @@ mod tests {
);
}
/// The identity-point forgery: with public key A = identity, R = identity,
/// and S = 0, the equation `[S]B = R + [k]A` holds for every message,
/// because `[k]·identity = identity`. Ordinary (non-strict) Ed25519
/// verification accepts it, so the shared `verify` primitive must use
/// strict verification, which rejects small-order R and public keys.
#[test]
fn verify_rejects_identity_point_forgery() {
use ed25519_dalek::Verifier;
// The identity point (0,1) compresses to y = 1 with sign bit 0.
let identity = [
1u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
let vk = VerifyingKey::from_bytes(&identity).expect("identity point is on the curve");
let mut sig_bytes = [0u8; 64];
sig_bytes[..32].copy_from_slice(&identity);
let msg = b"arbitrary message the key owner never signed";
// Prove the forged signature satisfies the ordinary verification
// equation, so the strict check below is what actually defends the
// boundary (not a signature that was already invalid everywhere).
assert!(
vk.verify(msg, &Signature::from_bytes(&sig_bytes)).is_ok(),
"identity-point forgery must satisfy ordinary verification (this is why strict is needed)"
);
assert!(
verify(&vk, msg, &sig_bytes).is_err(),
"strict verification must reject the identity-point forgery"
);
}
#[test]
fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a
-299
View File
@@ -1,299 +0,0 @@
//! Where the identity key and the delegation store live.
//!
//! `gl` and `git-remote-gitlawb` have to agree on this. `gl ucan import` writes a
//! delegation to `<dir>/delegations/`, and the helper reads it back from the same
//! place when it builds the `X-Ucan` header on push. If the two resolve
//! `GITLAWB_KEY` differently the push goes out with no header and the node refuses
//! the delegate, with nothing on either side to say why — so the rules live here
//! once, in the crate both binaries already depend on, rather than being written
//! twice and drifting.
//!
//! The home directory is a parameter rather than something this module looks up.
//! `gitlawb-core` is embedded by every consumer and is held to an explicit
//! dependency allowlist (`ci/gitlawb-core-allowed-deps.txt`); the callers already
//! carry `dirs`, so taking `home` keeps the rules shared without widening core's
//! tree. It also makes every case below testable against a fixed home.
use std::ffi::OsStr;
use std::path::{Component, Path, PathBuf};
use crate::{Error, Result};
/// Environment variable naming the identity PEM.
pub const KEY_ENV: &str = "GITLAWB_KEY";
/// Name of the PEM file inside the identity directory.
pub const KEY_FILE_NAME: &str = "identity.pem";
/// Directory under the home directory used when `GITLAWB_KEY` is unset.
pub const DEFAULT_DIR_NAME: &str = ".gitlawb";
/// Absolute path of the identity PEM: `$GITLAWB_KEY`, else
/// `<home>/.gitlawb/identity.pem`.
///
/// An empty `GITLAWB_KEY` counts as unset. That is what a shell leaves behind for
/// `FOO=` and for an unset variable expanded into a wrapper script, and reading it
/// as a path would resolve against the process working directory instead.
pub fn identity_key_path(home: Option<&Path>) -> Result<PathBuf> {
// `var_os`, not `var`: `var` folds a non-UTF-8 value into the same `Err` as
// unset, so an operator whose key path is not valid UTF-8 would silently get
// the default directory rather than theirs — or an error naming the real
// problem.
match std::env::var_os(KEY_ENV) {
Some(raw) if !raw.is_empty() => resolve_key_value(Path::new(&raw), home),
_ => Ok(require_home(home)?
.join(DEFAULT_DIR_NAME)
.join(KEY_FILE_NAME)),
}
}
/// The directory holding `identity.pem` and `delegations/` — the parent of
/// [`identity_key_path`].
pub fn identity_dir(home: Option<&Path>) -> Result<PathBuf> {
let key = identity_key_path(home)?;
key.parent().map(Path::to_path_buf).ok_or_else(|| {
Error::Key(format!(
"{KEY_ENV} has no parent directory: {}",
key.display()
))
})
}
/// The home directory, demanded only where the value being resolved needs it.
///
/// An absolute `GITLAWB_KEY` never needs home, so requiring it up front would
/// discard a perfectly good key on a host where `dirs::home_dir()` returns `None`
/// — no `HOME` and no passwd entry, which is an ordinary container shape. The
/// helper would then push unsigned, blaming the home directory for a setting the
/// operator had configured correctly.
fn require_home(home: Option<&Path>) -> Result<&Path> {
home.ok_or_else(|| {
Error::Key(format!(
"could not determine the home directory, which is needed to resolve this \
{KEY_ENV} value. Set {KEY_ENV} to an absolute path to avoid needing it."
))
})
}
/// Apply the `GITLAWB_KEY` rules to a raw value.
///
/// Split out from [`identity_key_path`] so the rules can be tested without setting
/// a process-global environment variable, which would make the tests race.
fn resolve_key_value(raw: &Path, home: Option<&Path>) -> Result<PathBuf> {
let path = expand_tilde(raw, home)?;
if !path.is_absolute() {
return Err(Error::Key(format!(
"{KEY_ENV} must be an absolute path (got {}). It also determines where \
delegations are stored, and `gl` and `git-remote-gitlawb` do not share a \
working directory, so a relative path sends them to different stores.",
raw.display()
)));
}
if path.parent().is_none() {
return Err(Error::Key(format!(
"{KEY_ENV} must name the key file, not the filesystem root (got {}). \
Point it at the PEM, e.g. ~/{DEFAULT_DIR_NAME}/{KEY_FILE_NAME}.",
raw.display()
)));
}
Ok(path)
}
/// Expand a leading `~/`, and only that.
///
/// `~user` is shell syntax this does not implement; leaving its `~` in place makes
/// it fail the absolute-path check with a message that names the real problem,
/// which beats resolving it somewhere the operator did not ask for. A bare `~` or
/// `~/` is refused outright: it names a directory where a file is required, and
/// expanding it to the home directory would put the delegation store beside the
/// home directory rather than inside it, since the store is the key's *parent*.
///
/// Matched on the first path component rather than on a string prefix. That is
/// what lets the value stay an `OsStr` end to end: the helper's old
/// `str::strip_prefix("~/")` needed a `String` first, which is why it reached for
/// `env::var` and folded every non-UTF-8 path into "unset".
fn expand_tilde(path: &Path, home: Option<&Path>) -> Result<PathBuf> {
let mut components = path.components();
match components.next() {
Some(Component::Normal(first)) if first == OsStr::new("~") => {
let rest = components.as_path();
if rest.as_os_str().is_empty() {
return Err(Error::Key(format!(
"{KEY_ENV} must name the key file, not a directory (got {}). \
Point it at the PEM, e.g. ~/{DEFAULT_DIR_NAME}/{KEY_FILE_NAME}.",
path.display()
)));
}
Ok(require_home(home)?.join(rest))
}
_ => Ok(path.to_path_buf()),
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A home that is absolute on the host running the tests. `/home/op` is not
/// absolute on Windows — it has a root but no prefix — so a shared literal
/// would make the absolute-path assertions test the wrong thing there.
fn home() -> PathBuf {
if cfg!(windows) {
PathBuf::from(r"C:\Users\op")
} else {
PathBuf::from("/home/op")
}
}
#[test]
fn absolute_value_is_taken_verbatim() {
let raw = home().join("data").join("keys").join(KEY_FILE_NAME);
assert_eq!(resolve_key_value(&raw, Some(&home())).unwrap(), raw);
}
#[test]
fn tilde_slash_expands_to_the_home_directory() {
let resolved = resolve_key_value(Path::new("~/keys/identity.pem"), Some(&home())).unwrap();
assert_eq!(resolved, home().join("keys").join(KEY_FILE_NAME));
}
/// The shell-style spelling of the default resolves to the default. Worth
/// pinning: this is the form an operator gets by copying a path out of their
/// shell, and the two binaries used to reach it by different routes.
#[test]
fn the_tilde_spelling_of_the_default_resolves_to_the_default() {
let resolved =
resolve_key_value(Path::new("~/.gitlawb/identity.pem"), Some(&home())).unwrap();
assert_eq!(resolved, home().join(DEFAULT_DIR_NAME).join(KEY_FILE_NAME));
}
/// A relative value resolves against the working directory, and `gl` and
/// `git-remote-gitlawb` do not share one: the import would land where the helper
/// never looks.
#[test]
fn relative_values_are_refused() {
for raw in ["identity.pem", "keys/identity.pem", "./keys/identity.pem"] {
assert!(
resolve_key_value(Path::new(raw), Some(&home())).is_err(),
"{raw} is relative and must be refused"
);
}
}
/// `~user` is shell syntax, not a path, and a bare `~` names a directory where
/// a file is required. Refused rather than guessed at.
#[test]
fn unsupported_tilde_forms_are_refused() {
for raw in ["~", "~/", "~someone/keys/identity.pem"] {
assert!(
resolve_key_value(Path::new(raw), Some(&home())).is_err(),
"{raw} must be refused rather than resolved"
);
}
}
/// The root has no parent, so the delegation store would have nowhere to go.
#[test]
fn the_filesystem_root_is_refused() {
assert!(resolve_key_value(Path::new("/"), Some(&home())).is_err());
}
/// The whole point of `var_os`: a non-UTF-8 value must reach the rules rather
/// than being folded into "unset" by `var`. Byte 0xFF is not valid UTF-8 in any
/// position, so this value is unreachable through `env::var`.
#[cfg(unix)]
#[test]
fn non_utf8_values_reach_the_rules() {
use std::ffi::OsString;
use std::os::unix::ffi::OsStringExt;
let relative = OsString::from_vec(b"keys/\xFF/identity.pem".to_vec());
assert!(
resolve_key_value(Path::new(&relative), Some(&home())).is_err(),
"a non-UTF-8 relative path must be refused, not silently defaulted"
);
let mut absolute = OsString::from("/data/");
absolute.push(OsString::from_vec(vec![0xFF]));
absolute.push("/identity.pem");
let resolved = resolve_key_value(Path::new(&absolute), Some(&home())).unwrap();
assert_eq!(resolved.as_os_str(), absolute.as_os_str());
}
/// Unset and empty both mean "use the default", and the two accessors must stay
/// consistent: the directory is the parent of the key, never a sibling of it.
/// The process environment is global, so the two cases share one test and one
/// lock rather than racing each other.
#[test]
fn unset_and_empty_both_select_the_default_directory() {
use std::sync::Mutex;
static LOCK: Mutex<()> = Mutex::new(());
let _guard = LOCK.lock().unwrap_or_else(|e| e.into_inner());
let restore = std::env::var_os(KEY_ENV);
std::env::remove_var(KEY_ENV);
let unset = (
identity_key_path(Some(&home())),
identity_dir(Some(&home())),
);
std::env::set_var(KEY_ENV, "");
let empty = (
identity_key_path(Some(&home())),
identity_dir(Some(&home())),
);
match restore {
Some(v) => std::env::set_var(KEY_ENV, v),
None => std::env::remove_var(KEY_ENV),
}
for (label, (key, dir)) in [("unset", unset), ("empty", empty)] {
assert_eq!(
key.unwrap(),
home().join(DEFAULT_DIR_NAME).join(KEY_FILE_NAME),
"{label} key path"
);
assert_eq!(
dir.unwrap(),
home().join(DEFAULT_DIR_NAME),
"{label} directory"
);
}
}
}
#[cfg(test)]
mod no_home_tests {
use super::*;
/// An absolute key needs no home directory. Demanding one up front discarded a
/// correctly-configured `GITLAWB_KEY` on any host where `dirs::home_dir()`
/// returns `None` — no `HOME` and no passwd entry, an ordinary container shape —
/// and the helper then pushed unsigned while blaming the home directory.
#[test]
fn an_absolute_key_resolves_without_a_home_directory() {
let raw = if cfg!(windows) {
r"C:\data\keys\identity.pem"
} else {
"/data/keys/identity.pem"
};
let resolved = resolve_key_value(Path::new(raw), None)
.expect("an absolute key must not need a home directory");
assert_eq!(resolved, PathBuf::from(raw));
assert_eq!(resolved.parent().unwrap(), Path::new(raw).parent().unwrap());
}
/// The forms that genuinely need a home still say so, rather than resolving
/// somewhere arbitrary.
#[test]
fn the_forms_that_need_a_home_report_its_absence() {
let err = resolve_key_value(Path::new("~/keys/identity.pem"), None)
.expect_err("a ~/ path cannot resolve without a home directory");
assert!(
err.to_string().contains("home directory"),
"the error must name the missing home directory, got: {err}"
);
}
}
-1
View File
@@ -5,7 +5,6 @@ pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
pub mod identity_path;
// `url` is the one dependency here that drags a tail (idna, then the icu
// crates), and gitlawb-core is allowlisted to stay embeddable. Every client that
// needs this predicate already parses URLs, so they opt in and nothing else
-20
View File
@@ -30,30 +30,10 @@ pub fn is_bidi_format(c: char) -> bool {
)
}
/// Both halves of INV-6 in one call: drop every `Cc` control (which defangs
/// ANSI and OSC escapes) and every bidi/format control that
/// [`is_bidi_format`] names. What remains is safe to write to a terminal.
///
/// Length is not this function's business: a caller that shows the text
/// decides how much of it to show.
pub fn strip_terminal_controls(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !is_bidi_format(*c))
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn strip_removes_escapes_and_reordering_but_keeps_text() {
let hostile = "ok\u{1b}[31m red\u{1b}]0;title\u{7}\u{202E}dlrow\u{202C} \u{0627}\u{200D}";
let clean = strip_terminal_controls(hostile);
assert_eq!(clean, "ok[31m red]0;titledlrow \u{0627}\u{200D}");
assert!(!clean.chars().any(|c| c.is_control() || is_bidi_format(c)));
}
#[test]
fn strips_every_reordering_code_point() {
// Asserted individually so a partial predicate (e.g. one that covers the
+25 -781
View File
@@ -16,19 +16,6 @@ use crate::did::Did;
use crate::identity::Keypair;
use crate::{Error, Result};
/// Proof chains deeper than this fail closed, in every walk over `prf`:
/// [`Ucan::verify_chain`], [`Ucan::chain_lifetime_is_bounded`] and
/// [`Ucan::chain_grants_push_to`] all stop here. Nothing this codebase mints is
/// longer than owner → agent → node. The bound exists for the hand-built token:
/// the node verifies the chain on the `X-Ucan` header of any signed request, and
/// nothing else says how deep a sender may nest `prf`. In practice not very
/// deep — a proof is a JSON string inside the next link's JSON, so each level
/// re-escapes the one beneath it and the encoding roughly doubles per link
/// (eight links is about 11 KiB, thirteen about 270 KiB) — but a walk should
/// refuse on its own terms rather than count on the encoding running the sender
/// out of room first.
pub const MAX_CHAIN_DEPTH: usize = 8;
/// A UCAN capability: what resource the token grants access to.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Capability {
@@ -62,32 +49,11 @@ impl Capability {
/// action field and `repo/admin` in the parent's action position act as
/// wildcards that cover any delegated value; wildcards on `self` carry no
/// special meaning.
///
/// Constraints (`nb`) participate, and conservatively:
///
/// | parent | child | verdict |
/// |---|---|---|
/// | none | anything | attenuated — adding constraints narrows |
/// | some | identical | attenuated |
/// | some | different | refused — narrowing is unprovable without semantics |
/// | some | none | refused — dropping constraints widens |
///
/// The last row is the one that matters. Ignoring `nb` here let a holder of a
/// constrained capability re-delegate the same resource and action with the
/// constraints removed, and the chain still verified — so a consumer that
/// refuses constrained capabilities at the leaf saw an unconstrained one and
/// granted it. Since `nb` has no interpreted semantics yet, "different" cannot
/// be shown to be narrower and is refused with it.
pub fn is_attenuated_by(&self, parent: &Capability) -> bool {
let resource_ok = parent.with == self.with || parent.with == "*";
let action_ok =
parent.can == self.can || parent.can == "*" || parent.can == caps::REPO_ADMIN;
let constraints_ok = match (&parent.constraints, &self.constraints) {
(None, _) => true,
(Some(p), Some(c)) => p == c,
(Some(_), None) => false,
};
resource_ok && action_ok && constraints_ok
resource_ok && action_ok
}
}
@@ -105,157 +71,6 @@ pub mod caps {
pub const REPO_ADMIN: &str = "repo/admin";
}
/// Rules shared by every boundary that decides whether a UCAN can push.
///
/// Four places answer "can this token push to this repository?": `gl ucan
/// delegate` and the MCP `ucan_delegate` tool when a token is issued, `gl ucan
/// import` when one is stored, `git-remote-gitlawb` when it mints an invocation,
/// and the node when it authorizes the push. For three review rounds each of them
/// carried its own copy of the answer, and every round found a token that one
/// boundary accepted and the next refused: a constrained grant that imported and
/// then authorized nothing; a wildcard proof the helper narrowed and the node
/// rejected; a resource whose owner the leaf named but the chain root did not.
///
/// The rules live here, once, in the crate all four already depend on. A boundary
/// may add checks of its own — the node anchors the chain root to a repository
/// record it holds independently, import binds the audience to the local key —
/// but the definition of a *usable push capability* is not one of them.
pub mod push {
use super::{caps, Capability, Ucan};
/// The `did:key` representation rule: `did:key:z6Mk…` and bare `z6Mk…` are
/// the same identity. Mirror rows store the bare form; canonical rows and
/// every token store the full form.
///
/// Collapses representation only within `did:key`. `did:web` and
/// `did:gitlawb` share the base58 space, so a trailing-segment compare would
/// treat `did:key:X` and `did:gitlawb:X` as equal; after stripping the prefix,
/// a value that still contains `:` is a non-key DID and matches nothing bare.
pub fn did_key_eq(a: &str, b: &str) -> bool {
if a == b {
return true;
}
fn key_id(d: &str) -> &str {
d.strip_prefix("did:key:").unwrap_or(d)
}
let (ka, kb) = (key_id(a), key_id(b));
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// The actions that authorize a push: `git/push` itself, the action wildcard,
/// and `repo/admin`, which covers it.
pub fn is_push_action(can: &str) -> bool {
can == caps::GIT_PUSH || can == "*" || can == caps::REPO_ADMIN
}
/// `gitlawb://repos/<owner>/<repo>` → `(owner, repo)`, or `None` for any other
/// shape. Exactly two non-empty segments: an owner DID never contains `/`, so
/// a third segment, a trailing slash, or an empty half is malformed rather than
/// something to interpret.
pub fn parse_repo_resource(with: &str) -> Option<(&str, &str)> {
let rest = with.strip_prefix("gitlawb://repos/")?;
let (owner, repo) = rest.split_once('/')?;
if owner.is_empty() || repo.is_empty() || repo.contains('/') {
return None;
}
Some((owner, repo))
}
/// The canonical repository name a capability or URL names: at most one
/// trailing `.git` stripped, the same normalization the node's smart-HTTP
/// routes apply (`smart_http_repo_name`). A git remote carries the suffix;
/// `RepoRecord.name` does not, so every boundary that compares a capability
/// to a record must go through the same strip or a `…/myrepo.git` capability
/// names a repository that never exists.
///
/// At-most-once, not repeatedly: a repository literally named `foo.git`
/// (creatable via the peer mirror path, which skips API name validation) is
/// addressed as `foo.git.git`, and stripping twice would send it to `foo`.
pub fn canonical_repo_name(repo: &str) -> &str {
repo.strip_suffix(".git").unwrap_or(repo)
}
/// A push-class capability whose resource is the wildcard. Refused at
/// issuance, at import, by the helper, and by the node: a delegation's scope is
/// fixed when it is issued, and `*` cannot say which repositories it covered
/// at that moment.
pub fn is_push_wildcard(with: &str, can: &str) -> bool {
with == "*" && is_push_action(can)
}
pub use super::MAX_CHAIN_DEPTH;
impl Capability {
/// Push-class, unconstrained, and naming exactly this repository.
///
/// Unconstrained because `nb` has no semantics yet: an owner who wrote
/// constraints meant to restrict, and honouring the capability while
/// ignoring them would grant more than was intended. The owner segment is
/// compared with [`did_key_eq`], so a token issued against the full DID
/// matches a mirror row keyed on the bare form.
///
/// The capability's repo segment goes through [`canonical_repo_name`]
/// while `repo` (a `RepoRecord.name`, already canonical) does not: a
/// delegation written `…/myrepo.git` authorizes `myrepo`, the same
/// mapping the smart-HTTP URL applies.
pub fn grants_push_to(&self, owner: &str, repo: &str) -> bool {
self.constraints.is_none()
&& is_push_action(&self.can)
&& parse_repo_resource(&self.with)
.is_some_and(|(o, r)| did_key_eq(o, owner) && canonical_repo_name(r) == repo)
}
}
impl Ucan {
/// Whether this chain — the leaf **and every proof behind it** — carries a
/// capability that [`Capability::grants_push_to`] this repository.
///
/// Every link, not just the leaf. [`Capability::is_attenuated_by`] accepts
/// a concrete child under a `*` parent, so a leaf that names the repository
/// can sit on a proof that names every repository the root owns — including
/// ones created after the delegation was issued. Checking the leaf alone
/// let exactly that through.
///
/// This establishes scope, not trust. It does not verify signatures,
/// expiry, or audience; call [`Ucan::verify_chain`] first, and anchor the
/// root it returns to something held independently of the token.
///
/// Fails closed on anything it cannot vouch for: a proof that does not
/// decode, more than one proof per link, or a chain deeper than
/// [`MAX_CHAIN_DEPTH`].
pub fn chain_grants_push_to(&self, owner: &str, repo: &str) -> bool {
self.chain_grants_push_to_at(owner, repo, 0)
}
fn chain_grants_push_to_at(&self, owner: &str, repo: &str, depth: usize) -> bool {
if depth >= MAX_CHAIN_DEPTH {
return false;
}
if !self
.payload
.att
.iter()
.any(|c| c.grants_push_to(owner, repo))
{
return false;
}
// `verify_chain` refuses more than one proof per link for the same
// reason: two proofs mean two roots, and nothing says which authorized
// what.
if self.payload.prf.len() > 1 {
return false;
}
match self.payload.prf.first() {
None => true,
Some(token) => match Ucan::decode(token) {
Ok(proof) => proof.chain_grants_push_to_at(owner, repo, depth + 1),
Err(_) => false,
},
}
}
}
}
/// The UCAN payload (what gets signed).
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct UcanPayload {
@@ -331,37 +146,6 @@ impl Ucan {
}
}
/// Whether every link in this chain carries a finite `exp`.
///
/// `exp` is optional in the format, and [`Self::is_expired`] reports `false`
/// when it is absent — so a link without one never expires. With no revocation
/// mechanism, a chain containing such a link is a permanent grant: a leaked
/// token cannot be withdrawn, and the issuer's only remedy is to rotate the
/// identity the resource is keyed on.
///
/// Consumers that turn a UCAN into write authority should require this. It is
/// deliberately not enforced inside [`Self::verify_chain`], because a
/// non-expiring token is well-formed and may be perfectly appropriate for a
/// read-only or advisory capability; whether an unbounded grant is acceptable
/// is the consumer's policy, not the format's.
///
/// Fails closed past [`MAX_CHAIN_DEPTH`], like every other walk over `prf`.
pub fn chain_lifetime_is_bounded(&self) -> bool {
self.chain_lifetime_is_bounded_at(0)
}
fn chain_lifetime_is_bounded_at(&self, depth: usize) -> bool {
if depth >= MAX_CHAIN_DEPTH {
return false;
}
if self.payload.exp.is_none() {
return false;
}
self.payload.prf.iter().all(|token| {
Self::decode(token).is_ok_and(|proof| proof.chain_lifetime_is_bounded_at(depth + 1))
})
}
/// Check if this UCAN's not-before time is in the future (token not yet valid).
pub fn is_before_valid(&self) -> bool {
if let Some(nbf) = self.payload.nbf {
@@ -464,33 +248,8 @@ impl Ucan {
/// 3. Check the proof is not expired
/// 4. Recursively verify the proof's own chain
///
/// A UCAN with no proofs is its own root, so it returns its own issuer.
///
/// **This establishes internal consistency, not trust.** `did:key` is
/// self-certifying, so anyone can mint a keypair and produce a chain that
/// verifies. A caller making an authorization decision MUST compare the
/// returned root against an identity it trusts for some reason outside this
/// token — a repo owner, a configured value, a registry lookup. Discarding
/// the return value is only correct when the caller is checking that a token
/// is well-formed and deliberately does not care who issued it.
///
/// The walk stops at [`MAX_CHAIN_DEPTH`] and fails closed there. The node
/// runs this on the `X-Ucan` header of any signed request before it asks
/// what the chain grants, so this is the first place sender-chosen depth is
/// felt: each link costs a signature check and a decode of everything
/// beneath it. The scope walk was bounded and this one was not, which made
/// that bound decorative — a deep chain met the unbounded walk first.
pub fn verify_chain(&self) -> Result<Did> {
self.verify_chain_at(0)
}
fn verify_chain_at(&self, depth: usize) -> Result<Did> {
if depth >= MAX_CHAIN_DEPTH {
return Err(Error::Ucan(format!(
"proof chain deeper than {MAX_CHAIN_DEPTH} links is not accepted"
)));
}
/// A UCAN with no proofs (root capability) passes trivially.
pub fn verify_chain(&self) -> Result<()> {
// First verify our own signature
self.verify_signature()?;
@@ -502,44 +261,34 @@ impl Ucan {
return Err(Error::Ucan("token is not yet valid".to_string()));
}
if self.payload.prf.len() > 1 {
return Err(Error::Ucan(
"multi-proof chains are not supported: more than one proof means \
more than one root, and which root authorized a given capability \
is ambiguous"
.to_string(),
));
}
for proof_token in &self.payload.prf {
let proof = Self::decode(proof_token)
.map_err(|e| Error::Ucan(format!("failed to decode proof: {e}")))?;
let Some(proof_token) = self.payload.prf.first() else {
// No proofs: this token is its own root.
return Ok(self.payload.iss.clone());
};
let proof = Self::decode(proof_token)
.map_err(|e| Error::Ucan(format!("failed to decode proof: {e}")))?;
// The proof's audience must be this UCAN's issuer
if proof.payload.aud != self.payload.iss {
return Err(Error::Ucan(format!(
"proof chain broken: proof audience {} does not match issuer {}",
proof.payload.aud, self.payload.iss
)));
}
// Every delegated capability must be covered by the proof (attenuation).
for cap in &self.payload.att {
let covered = proof.payload.att.iter().any(|p| cap.is_attenuated_by(p));
if !covered {
// The proof's audience must be this UCAN's issuer
if proof.payload.aud != self.payload.iss {
return Err(Error::Ucan(format!(
"capability attenuation violated: '{}' on '{}' not covered by proof",
cap.can, cap.with
"proof chain broken: proof audience {} does not match issuer {}",
proof.payload.aud, self.payload.iss
)));
}
// Every delegated capability must be covered by the proof (attenuation).
for cap in &self.payload.att {
let covered = proof.payload.att.iter().any(|p| cap.is_attenuated_by(p));
if !covered {
return Err(Error::Ucan(format!(
"capability attenuation violated: '{}' on '{}' not covered by proof",
cap.can, cap.with
)));
}
}
// Verify the proof's signature and chain recursively
proof.verify_chain()?;
}
// Recurse; the root of the proof's chain is the root of ours.
proof.verify_chain_at(depth + 1)
Ok(())
}
}
@@ -915,509 +664,4 @@ mod tests {
delegated.verify_chain().unwrap();
}
#[test]
fn verify_chain_returns_the_root_issuer_of_a_delegated_chain() {
// owner -> agent (delegation), agent -> node (invocation).
// The root is the owner: that is the identity the whole chain rests on,
// and the only one a caller can meaningfully anchor a trust decision to.
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let caps_vec = vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let delegation =
Ucan::issue(&owner, agent.did(), caps_vec.clone(), None).expect("issue delegation");
let invocation = Ucan::delegate(&agent, node.did(), caps_vec, None, &delegation)
.expect("wrap invocation");
assert_eq!(
invocation.verify_chain().expect("chain must verify"),
owner.did(),
"the root issuer is the owner who started the chain, not the agent presenting it"
);
}
/// A chain is only bounded if EVERY link is. An unbounded link anywhere makes
/// the whole grant permanent, because `is_expired` reports false for it and
/// there is no revocation path to withdraw it.
#[test]
fn chain_lifetime_is_bounded_requires_an_expiry_on_every_link() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let cap = || vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let hour = Utc::now() + chrono::Duration::hours(1);
let bounded_root = Ucan::issue(&owner, agent.did(), cap(), Some(hour)).expect("issue");
let unbounded_root = Ucan::issue(&owner, agent.did(), cap(), None).expect("issue");
assert!(
Ucan::delegate(&agent, node.did(), cap(), Some(hour), &bounded_root)
.expect("wrap")
.chain_lifetime_is_bounded(),
"both links finite"
);
assert!(
!Ucan::delegate(&agent, node.did(), cap(), None, &bounded_root)
.expect("wrap")
.chain_lifetime_is_bounded(),
"the leaf has no expiry, so the grant never lapses"
);
assert!(
!Ucan::delegate(&agent, node.did(), cap(), Some(hour), &unbounded_root)
.expect("wrap")
.chain_lifetime_is_bounded(),
"a bounded leaf cannot rescue an unbounded proof: the holder can always \
mint a fresh leaf from it"
);
assert!(
!unbounded_root.chain_lifetime_is_bounded(),
"a self-issued token with no expiry is itself unbounded"
);
}
/// A three-link chain: owner -> lead -> agent, which is the real shape of an
/// org delegating to a team lead who delegates to a CI identity.
///
/// Every other chain here is depth two, where the immediate proof IS the root —
/// so nothing distinguishes recursing to the true root from simply returning the
/// proof's issuer. Both `assert_eq!` and `assert_ne!` below are load-bearing:
/// without the second, returning the middle issuer would still satisfy a test
/// that only checked "not the leaf".
#[test]
fn verify_chain_walks_past_the_immediate_proof_to_the_true_root() {
let owner = Keypair::generate();
let lead = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let cap = || vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let root = Ucan::issue(&owner, lead.did(), cap(), None).expect("owner -> lead");
let mid = Ucan::delegate(&lead, agent.did(), cap(), None, &root).expect("lead -> agent");
let leaf = Ucan::delegate(&agent, node.did(), cap(), None, &mid).expect("agent -> node");
let found = leaf.verify_chain().expect("a three-link chain must verify");
assert_eq!(
found,
owner.did(),
"the root is the owner who started the chain, two hops up"
);
assert_ne!(
found,
lead.did(),
"returning the immediate proof's issuer is not walking to the root"
);
}
#[test]
fn verify_chain_returns_self_as_root_for_a_self_issued_token() {
// A token with no proofs roots at its own issuer. This is what makes a
// self-minted token useless: the caller compares this against the repo
// owner and it will only ever match when the presenter IS the owner.
let agent = Keypair::generate();
let node = Keypair::generate();
let ucan =
Ucan::issue(&agent, node.did(), vec![Capability::new("*", "*")], None).expect("issue");
assert_eq!(
ucan.verify_chain().expect("a root token still verifies"),
agent.did(),
"a self-minted token roots at the minter, however permissive its capabilities"
);
}
/// Stripping `nb` is a widening, and a widening must fail attenuation.
///
/// Without this, a constrained delegation is trivially escalated: the holder
/// re-delegates the same resource and action with the constraints removed,
/// `verify_chain` accepts the chain because attenuation only compared `with`
/// and `can`, and a consumer that refuses constrained capabilities at the leaf
/// (as the node's push gate does) then sees an unconstrained one and grants it.
/// Guarding only the leaf guards the wrong end of the chain.
#[test]
fn verify_chain_rejects_a_child_that_strips_the_parents_constraints() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let constrained = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)
.with_constraints(serde_json::json!({ "refs": ["refs/heads/feat/*"] }));
let delegation = Ucan::issue(&owner, agent.did(), vec![constrained], None)
.expect("issue constrained delegation");
// Same resource, same action, constraints dropped.
let widened = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH);
let forged =
Ucan::delegate(&agent, node.did(), vec![widened], None, &delegation).expect("wrap");
let err = forged
.verify_chain()
.expect_err("dropping the parent's constraints must fail attenuation");
assert!(
err.to_string().contains("attenuation"),
"the failure must name attenuation, got: {err}"
);
}
#[test]
fn verify_chain_accepts_a_child_that_keeps_the_parents_constraints() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let nb = serde_json::json!({ "refs": ["refs/heads/feat/*"] });
let constrained = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)
.with_constraints(nb.clone());
let delegation =
Ucan::issue(&owner, agent.did(), vec![constrained.clone()], None).expect("issue");
let invocation =
Ucan::delegate(&agent, node.did(), vec![constrained], None, &delegation).expect("wrap");
assert_eq!(
invocation
.verify_chain()
.expect("an unchanged constraint must verify"),
owner.did()
);
}
#[test]
fn an_unconstrained_parent_still_allows_a_child_to_add_constraints() {
// Adding `nb` narrows, which is always a legal attenuation.
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let open = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH);
let delegation = Ucan::issue(&owner, agent.did(), vec![open], None).expect("issue");
let narrowed = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)
.with_constraints(serde_json::json!({ "refs": ["refs/heads/main"] }));
let invocation =
Ucan::delegate(&agent, node.did(), vec![narrowed], None, &delegation).expect("wrap");
assert_eq!(
invocation.verify_chain().expect("narrowing must verify"),
owner.did()
);
}
#[test]
fn verify_chain_rejects_a_multi_proof_chain() {
// Two proofs mean two roots, and nothing says which root authorized a
// given capability. Returning either one would be unsound, so refuse.
let owner_a = Keypair::generate();
let owner_b = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let caps_vec = vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let proof_a = Ucan::issue(&owner_a, agent.did(), caps_vec.clone(), None).expect("issue a");
let proof_b = Ucan::issue(&owner_b, agent.did(), caps_vec.clone(), None).expect("issue b");
// `delegate` only ever writes one proof, so build the two-proof payload by hand.
let payload = UcanPayload {
ucan: "1.0.0".to_string(),
iss: agent.did(),
aud: node.did(),
att: caps_vec,
exp: None,
nbf: None,
prf: vec![
proof_a.encode().expect("encode a"),
proof_b.encode().expect("encode b"),
],
};
let signing_bytes = serde_json::to_vec(&payload).expect("serialize payload");
let s = agent.sign_b64(&signing_bytes);
let multi = Ucan { payload, s };
let err = multi
.verify_chain()
.expect_err("a two-proof chain must be refused");
assert!(
err.to_string().contains("multi-proof"),
"the error must name the reason, got: {err}"
);
}
}
#[cfg(test)]
mod push_scope_tests {
use super::push::*;
use super::{caps, Capability, Ucan};
use crate::identity::Keypair;
fn hour() -> chrono::DateTime<chrono::Utc> {
chrono::Utc::now() + chrono::Duration::hours(1)
}
#[test]
fn did_key_eq_collapses_representation_only_within_did_key() {
assert!(did_key_eq("did:key:z6MkAbc", "z6MkAbc"));
assert!(did_key_eq("z6MkAbc", "did:key:z6MkAbc"));
assert!(did_key_eq("did:key:z6MkAbc", "did:key:z6MkAbc"));
assert!(!did_key_eq("did:key:z6MkAbc", "did:key:z6MkXyz"));
// A bare id must never match across methods: `did:gitlawb` shares the
// base58 space with `did:key`.
assert!(!did_key_eq("did:gitlawb:z6MkAbc", "z6MkAbc"));
assert!(!did_key_eq("did:key:z6MkAbc", "did:gitlawb:z6MkAbc"));
assert!(!did_key_eq("did:web:example.com", "example.com"));
}
#[test]
fn parse_repo_resource_requires_exactly_two_segments() {
assert_eq!(
parse_repo_resource("gitlawb://repos/did:key:z6Mk/r"),
Some(("did:key:z6Mk", "r"))
);
for bad in [
"*",
"",
"gitlawb://repos/",
"gitlawb://repos/owner",
"gitlawb://repos/owner/",
"gitlawb://repos//r",
"gitlawb://repos/owner/r/extra",
"https://repos/owner/r",
] {
assert_eq!(parse_repo_resource(bad), None, "{bad:?}");
}
}
#[test]
fn is_push_wildcard_is_the_resource_wildcard_with_a_push_action() {
for can in [caps::GIT_PUSH, "*", caps::REPO_ADMIN] {
assert!(is_push_wildcard("*", can), "{can}");
}
assert!(!is_push_wildcard("*", caps::GIT_FETCH));
assert!(!is_push_wildcard("*", caps::PR_OPEN));
assert!(!is_push_wildcard("gitlawb://repos/o/r", caps::GIT_PUSH));
}
#[test]
fn a_capability_grants_push_only_when_concrete_unconstrained_and_push_class() {
let owner = "did:key:z6MkOwner";
let ok = Capability::new("gitlawb://repos/did:key:z6MkOwner/r", caps::GIT_PUSH);
assert!(ok.grants_push_to(owner, "r"));
// Bare and full owner forms are the same identity.
assert!(ok.grants_push_to("z6MkOwner", "r"));
assert!(!Capability::new("*", caps::GIT_PUSH).grants_push_to(owner, "r"));
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/r", caps::GIT_FETCH)
.grants_push_to(owner, "r")
);
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/other", caps::GIT_PUSH)
.grants_push_to(owner, "r")
);
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOther/r", caps::GIT_PUSH)
.grants_push_to(owner, "r")
);
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/r", caps::GIT_PUSH)
.with_constraints(serde_json::json!({"max_bytes": 1}))
.grants_push_to(owner, "r")
);
for can in ["*", caps::REPO_ADMIN] {
assert!(
Capability::new("gitlawb://repos/did:key:z6MkOwner/r", can)
.grants_push_to(owner, "r"),
"{can} covers git/push"
);
}
}
#[test]
fn a_git_suffixed_capability_names_the_canonical_repo() {
let owner = "did:key:z6MkOwner";
// A git remote carries `.git`; the record does not. The capability
// matches the repository the URL would reach, so a delegation pasted
// from the remote still authorizes.
assert!(Capability::new(
"gitlawb://repos/did:key:z6MkOwner/myrepo.git",
caps::GIT_PUSH
)
.grants_push_to(owner, "myrepo"));
// At most one suffix is stripped: a repo literally named `foo.git`
// (reachable as `foo.git.git` over smart HTTP) is a different
// repository from `foo`, and a single `.git` must not resolve to it.
assert!(Capability::new(
"gitlawb://repos/did:key:z6MkOwner/foo.git.git",
caps::GIT_PUSH
)
.grants_push_to(owner, "foo.git"));
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/foo.git", caps::GIT_PUSH)
.grants_push_to(owner, "foo.git")
);
}
/// The chain walk, at every depth: a wildcard anywhere denies; concrete
/// everywhere grants. The grandparent case is the one a single-level check
/// misses, because the immediate proof already names the repo.
#[test]
fn chain_grants_push_only_when_every_link_names_the_repo() {
let owner = Keypair::generate();
let a = Keypair::generate();
let b = Keypair::generate();
let node = Keypair::generate();
let owner_s = owner.did().to_string();
let res = format!("gitlawb://repos/{owner_s}/r");
let concrete = || vec![Capability::new(&res, caps::GIT_PUSH)];
let wild = || vec![Capability::new("*", caps::GIT_PUSH)];
// Single self-issued link.
assert!(Ucan::issue(&owner, a.did(), concrete(), Some(hour()))
.unwrap()
.chain_grants_push_to(&owner_s, "r"));
assert!(!Ucan::issue(&owner, a.did(), wild(), Some(hour()))
.unwrap()
.chain_grants_push_to(&owner_s, "r"));
// Two links: concrete leaf on a wildcard proof must deny.
let wild_proof = Ucan::issue(&owner, a.did(), wild(), Some(hour())).unwrap();
let leaf = Ucan::delegate(&a, node.did(), concrete(), Some(hour()), &wild_proof).unwrap();
assert!(
leaf.verify_chain().is_ok(),
"attenuation accepts it — that is why we walk"
);
assert!(!leaf.chain_grants_push_to(&owner_s, "r"));
// Three links: wildcard grandparent, concrete parent, concrete leaf.
let gp = Ucan::issue(&owner, a.did(), wild(), Some(hour())).unwrap();
let parent = Ucan::delegate(&a, b.did(), concrete(), Some(hour()), &gp).unwrap();
let leaf = Ucan::delegate(&b, node.did(), concrete(), Some(hour()), &parent).unwrap();
assert!(leaf.verify_chain().is_ok());
assert!(
!leaf.chain_grants_push_to(&owner_s, "r"),
"a wildcard two links up must deny"
);
// Three links, all concrete.
let gp = Ucan::issue(&owner, a.did(), concrete(), Some(hour())).unwrap();
let parent = Ucan::delegate(&a, b.did(), concrete(), Some(hour()), &gp).unwrap();
let leaf = Ucan::delegate(&b, node.did(), concrete(), Some(hour()), &parent).unwrap();
assert!(leaf.chain_grants_push_to(&owner_s, "r"));
// And the bare owner form is the same repo.
assert!(leaf.chain_grants_push_to(owner_s.strip_prefix("did:key:").unwrap(), "r"));
// But not another repo.
assert!(!leaf.chain_grants_push_to(&owner_s, "other"));
}
/// Fail closed on anything the walk cannot vouch for.
#[test]
fn chain_walk_fails_closed_on_undecodable_or_multi_proofs_or_excess_depth() {
let owner = Keypair::generate();
let a = Keypair::generate();
let owner_s = owner.did().to_string();
let res = format!("gitlawb://repos/{owner_s}/r");
let cap = || vec![Capability::new(&res, caps::GIT_PUSH)];
let mut broken = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
broken.payload.prf = vec!["not a ucan".into()];
assert!(!broken.chain_grants_push_to(&owner_s, "r"));
let good = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
let mut two = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
two.payload.prf = vec![good.encode().unwrap(), good.encode().unwrap()];
assert!(!two.chain_grants_push_to(&owner_s, "r"));
// Deeper than MAX_CHAIN_DEPTH, all concrete: refused on depth alone.
let mut cur = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
let mut signer = a;
for _ in 0..MAX_CHAIN_DEPTH {
let next = Keypair::generate();
cur = Ucan::delegate(&signer, next.did(), cap(), Some(hour()), &cur).unwrap();
signer = next;
}
assert!(
!cur.chain_grants_push_to(&owner_s, "r"),
"depth bound must fail closed"
);
}
}
/// Every walk over `prf` stops at the same depth. The round-ten P1: the scope
/// walk was bounded while `verify_chain` and `chain_lifetime_is_bounded` were
/// not, and the node runs `verify_chain` on an attacker-supplied `X-Ucan` header
/// before the bounded walk ever sees it, so the bound protected nothing.
#[cfg(test)]
mod chain_depth_tests {
use super::{caps, Capability, Ucan, MAX_CHAIN_DEPTH};
use crate::identity::Keypair;
fn hour() -> chrono::DateTime<chrono::Utc> {
chrono::Utc::now() + chrono::Duration::hours(1)
}
/// A well-formed, fully signed, expiring chain of exactly `links` links, every
/// one naming the owner's repository, so the only thing any walk can object
/// to is how long it is. Returns the leaf and the owner it roots at.
fn concrete_chain(links: usize) -> (Ucan, Keypair) {
assert!(links >= 1);
let owner = Keypair::generate();
let owner_s = owner.did().to_string();
let res = format!("gitlawb://repos/{owner_s}/r");
let cap = || vec![Capability::new(&res, caps::GIT_PUSH)];
let mut signer = Keypair::generate();
let mut cur = Ucan::issue(&owner, signer.did(), cap(), Some(hour())).unwrap();
for _ in 1..links {
let next = Keypair::generate();
cur = Ucan::delegate(&signer, next.did(), cap(), Some(hour()), &cur).unwrap();
signer = next;
}
(cur, owner)
}
#[test]
fn a_chain_at_the_bound_passes_every_walk() {
let (leaf, owner) = concrete_chain(MAX_CHAIN_DEPTH);
let owner_s = owner.did().to_string();
assert_eq!(
leaf.verify_chain()
.expect("a chain at the bound must verify"),
owner.did()
);
assert!(leaf.chain_lifetime_is_bounded());
assert!(leaf.chain_grants_push_to(&owner_s, "r"));
}
#[test]
fn one_link_past_the_bound_fails_every_walk_closed() {
let (leaf, owner) = concrete_chain(MAX_CHAIN_DEPTH + 1);
let owner_s = owner.did().to_string();
let err = leaf
.verify_chain()
.expect_err("verify_chain must refuse a chain past the bound");
assert!(
err.to_string().contains("deeper than"),
"the refusal must say depth was the reason, not a fake signature failure: {err}"
);
assert!(
!leaf.chain_lifetime_is_bounded(),
"an unwalkable chain cannot be vouched for as bounded"
);
assert!(!leaf.chain_grants_push_to(&owner_s, "r"));
}
/// A chain well past the bound is refused from the top, before any per-link
/// work. Only a few links past it, deliberately: a proof is embedded as a JSON
/// string inside the next link's JSON, so every level re-escapes the one
/// inside it and the encoding roughly doubles per link — eight links is about
/// 10 KiB, sixteen is megabytes, and a chain of "a few hundred levels" cannot
/// physically be encoded, let alone sent as a header. The bound is still
/// what stops the reachable dozen-odd levels from each costing a decode of
/// everything beneath them.
#[test]
fn a_chain_well_past_the_bound_is_refused_not_walked() {
let (leaf, _) = concrete_chain(MAX_CHAIN_DEPTH + 4);
let err = leaf.verify_chain().expect_err("must refuse");
assert!(err.to_string().contains("deeper than"), "{err}");
assert!(!leaf.chain_lifetime_is_bounded());
}
}
-4
View File
@@ -23,8 +23,6 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
axum = { version = "0.8", features = ["http1", "http2", "json", "ws"] }
async-graphql = { version = "7", features = ["chrono", "uuid", "tracing"] }
async-graphql-axum = "7"
@@ -40,7 +38,6 @@ cid = { workspace = true }
hex = { workspace = true }
sha2 = { workspace = true }
hmac = { workspace = true }
chacha20poly1305 = "0.10"
http-body-util = "0.1"
tokio-util = { version = "0.7", features = ["io"] }
dirs-next = "2"
@@ -76,7 +73,6 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
rand = { workspace = true }
libp2p-dns = { version = "0.45.0", features = ["tokio"] }
[dev-dependencies]
File diff suppressed because it is too large. Load diff
+9 -56
View File
@@ -1,13 +1,12 @@
//! GET /api/v1/arweave/anchors — list Arweave ref-update anchors.
use axum::{
extract::{Extension, Query, State},
extract::{Query, State},
Json,
};
use serde::Deserialize;
use crate::db::normalize_owner_key;
use crate::error::{AppError, Result};
use crate::error::Result;
use crate::state::AppState;
#[derive(Debug, Deserialize)]
@@ -22,62 +21,17 @@ fn default_limit() -> i64 {
}
/// GET /api/v1/arweave/anchors
///
/// `?repo=<owner>/<name>` returns anchors for one repository and binds the
/// request to the canonical read-authorization path used by every other
/// repo-scoped read: the same `authorize_repo_read` helper, with the caller
/// identity attached. Missing repositories, quarantined mirrors, and signed
/// non-readers all collapse to the standard `404` (no existence oracle). The
/// unscoped listing is auth-only — the #121 contract — and does not admit
/// visibility filtering, which is the #136 stale-index class and explicitly
/// out of scope for this auth slice.
pub async fn list_anchors(
State(state): State<AppState>,
Query(q): Query<ListAnchorsQuery>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// The route's `optional_signature` layer is permissive (it admits unsigned
// legacy callers) so this in-handler check is the actual gate. A signed
// non-reader still passes this check and proceeds to the authz step below.
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
if caller.is_none() {
return Err(AppError::Unauthorized(
"authentication required for anchor listing".into(),
));
}
// Clamp before the SQL query. A negative or non-numeric limit must not
// reach Postgres as `LIMIT -1` (which it rejects as a db error / 500) — the
// contract is the same default ceiling as the listing page itself.
let limit = q.limit.clamp(0, 200);
let anchors = if let Some(repo_slug) = q.repo.as_deref() {
// Parse the user-supplied "owner/name" through the same slug validator
// the sync path uses, so a malformed query is a 400, not a 500.
let (owner, name) = match crate::git::repo_store::validate_repo_slug(repo_slug) {
Ok(parts) => parts,
Err(e) => return Err(AppError::BadRequest(format!("invalid ?repo: {e}"))),
};
// Read gate. Returns `RepoNotFound` (→ 404) indistinguishably for
// missing repos, quarantined mirrors, and signed non-readers.
let (record, _rules) =
crate::api::authorize_repo_read(&state, owner, name, caller, "/").await?;
// Build the SQL filter from the canonical stored slug, NOT from the
// user-supplied `?repo=` string. Anchor rows are written as
// `{normalize_owner_key(owner_did)}/{name}` (the short form), so a
// request that passes authz with the full `did:key:…/name` form would
// match zero rows and return a false empty page. The gate already
// verified the repo is readable; `record.owner_did` is the canonical
// identity and `record.name` is the stored name.
let stored_slug = format!("{}/{}", normalize_owner_key(&record.owner_did), record.name);
state
.db
.list_arweave_anchors(Some(&stored_slug), limit)
.await?
} else {
// No scope → no repo-read decision. Auth-only.
state.db.list_arweave_anchors(None, limit).await?
};
let limit = q.limit.min(200);
// Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251).
let anchors = state
.db
.list_arweave_anchors(q.repo.as_deref(), limit)
.await?;
Ok(Json(serde_json::json!({
"anchors": anchors,
@@ -106,7 +60,6 @@ mod closed_pool_tests {
.oneshot(
Request::builder()
.uri("/api/v1/arweave/anchors")
.extension(crate::auth::AuthenticatedDid("did:key:test".into()))
.body(axum::body::Body::empty())
.unwrap(),
)
+29 -123
View File
@@ -285,22 +285,11 @@ pub async fn submit_bounty(
Path(id): Path<String>,
Json(req): Json<SubmitBountyRequest>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "claimed" {
return Err(AppError::BadRequest(format!(
@@ -336,22 +325,11 @@ pub async fn approve_bounty(
Path(id): Path<String>,
Json(req): Json<ApproveBountyRequest>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "submitted" {
return Err(AppError::BadRequest(format!(
@@ -394,22 +372,11 @@ pub async fn cancel_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "open" {
return Err(AppError::BadRequest(format!(
@@ -440,22 +407,11 @@ pub async fn dispute_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "claimed" && bounty.status != "submitted" {
return Err(AppError::BadRequest(format!(
@@ -500,68 +456,21 @@ pub async fn dispute_bounty(
Ok(Json(updated))
}
/// Resolve the set of `(owner_did, name)` pairs for repos an anonymous caller
/// can list. Mirrors the `stats()` pattern in `server.rs` (#104): batch-load
/// all deduped repos, batch-load their visibility rules, keep only those that
/// pass `listable_at_root`. Pure I/O after the two DB round-trips — no
/// per-repo authorization queries.
///
/// Both bounty stats handlers mount behind `optional_signature` (see
/// `bounty_read_routes` in server.rs), so the caller is always `None` for
/// anonymous access. Every caller, signed or not, currently receives the
/// anonymous-scoped aggregates (matching the #104 pattern); per-caller
/// visibility can be threaded through here when that work lands.
///
/// A database error is returned to the caller, never collapsed to an empty
/// set: an empty set would render as HTTP 200 with zero counts, which a client
/// reads as confirmed absence of activity.
async fn visible_repo_pairs(state: &AppState) -> Result<Vec<(String, String)>> {
let rows = state.db.list_all_repos_deduped().await?;
let ids: Vec<String> = rows.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = state.db.list_visibility_rules_for_repos(&ids).await?;
let pairs = rows
.iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
// mirror-rows-handled: a mirror with a canonical twin is deduped to the
// canonical row and its rules; a mirror-only repo counts as public, the
// same answer authorize_repo_read and /stats give. Pinned by
// bounty_stats_mirror_rows_follow_canonical_twin_or_count_when_alone.
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, None)
})
.map(|r| {
(
crate::db::normalize_owner_key(&r.owner_did).to_string(),
r.name.clone(),
)
})
.collect();
Ok(pairs)
}
/// GET /api/v1/bounties/stats
///
/// Aggregates are restricted to anonymously-listable repos so private-repo
/// bounty activity is not exposed (#477). The visible-repo set is resolved
/// once per request via `visible_repo_pairs` (two SQL round-trips), then
/// passed into the filtered aggregate queries.
pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountyStatsResponse>> {
let visible = visible_repo_pairs(&state).await?;
let open = state
.db
.count_bounties_by_status_visible("open", &visible)
.await?;
let open = state.db.count_bounties_by_status("open").await.unwrap_or(0);
let claimed = state
.db
.count_bounties_by_status_visible("claimed", &visible)
.await?;
.count_bounties_by_status("claimed")
.await
.unwrap_or(0);
let completed = state
.db
.count_bounties_by_status_visible("completed", &visible)
.await?;
.count_bounties_by_status("completed")
.await
.unwrap_or(0);
let leaders = state.db.bounty_leaderboard_visible(10, &visible).await?;
let leaders = state.db.bounty_leaderboard(10).await.unwrap_or_default();
let leaderboard = leaders
.into_iter()
.map(|(did, cnt, total)| AgentBountyEntry {
@@ -580,14 +489,11 @@ pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountySt
}
/// GET /api/v1/agents/{did}/bounties
///
/// Per-agent earnings restricted to anonymously-listable repos (#477).
pub async fn agent_bounty_stats(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<Json<serde_json::Value>> {
let visible = visible_repo_pairs(&state).await?;
let (count, total) = state.db.agent_bounty_stats_visible(&did, &visible).await?;
let (count, total) = state.db.agent_bounty_stats(&did).await.unwrap_or((0, 0));
Ok(Json(serde_json::json!({
"did": did,
"completed_bounties": count,
-2
View File
@@ -52,7 +52,6 @@ pub async fn list_certs(
"node_did": c.node_did,
"signature": c.signature,
"issued_at": c.issued_at,
"version": c.version,
})
})
.collect();
@@ -93,6 +92,5 @@ pub async fn get_cert(
"node_did": cert.node_did,
"signature": cert.signature,
"issued_at": cert.issued_at,
"version": cert.version,
})))
}
+31 -10
View File
@@ -6,24 +6,29 @@ use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs
/// Returns [{oid, cid}] for every encrypted blob in the repo, to any caller who
/// can read the repo. Not recipient-scoped: recipient identities are not stored,
/// so access control here is repo readability and decryption is gated by the
/// envelope crypto (only a real recipient can open an envelope).
///
/// Quarantined repos are opaque 404 via [`crate::api::authorize_repo_read`] —
/// same as issues/changelogs — so a public-but-quarantined mirror cannot leak
/// its encrypted blob index.
pub async fn list_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
@@ -40,9 +45,17 @@ pub async fn get_encrypted_blob(
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo, oid)): Path<(String, String, String)>,
) -> Result<Vec<u8>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")));
}
let cid = state
.db
.encrypted_blob_cid(&record.id, &oid)
@@ -68,9 +81,17 @@ pub async fn replicate_encrypted_blobs(
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
+1 -3
View File
@@ -11,8 +11,7 @@ use crate::state::AppState;
/// Hard ceiling on rows any ref-update feed returns in one request. Shared by the
/// shared collector's clamp and the per-handler request caps so they can't drift.
/// `pub(crate)` so the GraphQL complexity meter can price against the same bound.
pub(crate) const MAX_VISIBLE_REF_UPDATES: i64 = 200;
const MAX_VISIBLE_REF_UPDATES: i64 = 200;
/// Collect up to `limit` ref-update rows visible to `caller`, newest first,
/// paging past rows the feed gate drops. Filtering after a plain SQL `LIMIT`
@@ -438,7 +437,6 @@ mod ref_updates_feed_tests {
node_did: "did:key:z6MkNode".into(),
signature: "sig".into(),
issued_at: Utc::now().to_rfc3339(),
version: 1,
}
}
+10 -448
View File
@@ -2130,77 +2130,14 @@ async fn gate_and_serve(
/// GET /api/v1/ipfs/pins
///
/// Returns all CIDs that have been pinned from git objects received via push.
/// Each entry includes the git SHA-256 hex and the timestamp when it was
/// pinned. The wire contract is nullable by design:
///
/// - `cid` / `local_cid` — node-local resolver keys: the raw CIDv1 the
/// `GET /ipfs/{cid}` endpoint serves. Non-null ONLY when the stored value
/// parses as a raw CIDv1 AND writer-owned local provenance confirms the
/// bytes went through this node's IPFS daemon. Otherwise null, even when
/// the database holds some other CID-shaped string for the row.
/// - `pinata_cid` — the provider identifier Pinata returned (dag-pb/UnixFS,
/// not a node resolver key). The only usable identifier for remote-only
/// rows; `gl` consumers must read this field, never `cid`, for those.
/// - `local_pinned` — `true` iff the local-IPFS writer path
/// (`record_pinned_cid_with_source`) pushed the bytes into the local
/// daemon. Writer-owned; never inferred from CID shape. A row can carry
/// `local_pinned = true` with a null `cid` when its stored key predates
/// the raw-key contract (legacy provider key awaiting repair).
/// - `pinata_pinned` — `true` iff the row has a non-null `pinata_cid`.
///
/// Rows with neither a local nor a Pinata CID are omitted so the response
/// only contains rows with at least one backend. Remote-only rows stay
/// visible through `pinata_cid`/`pinata_pinned`; nothing is hidden.
///
/// #218 review P2: the response surfaces writer-owned provenance so a
/// `gl` consumer can distinguish local-only, remote-only, and dual rows
/// without re-inferring semantics from nullability.
pub async fn list_pins(
State(state): State<AppState>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
if auth.is_none() {
return Err(crate::error::AppError::Unauthorized(
"authentication required for pin listing".into(),
));
}
/// Returns all CIDs that have been pinned to the local IPFS node from git
/// objects received via push. Each entry includes the git SHA-256 hex, the
/// CIDv1 string, and the timestamp when it was pinned.
pub async fn list_pins(State(state): State<AppState>) -> Result<Json<serde_json::Value>> {
// Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251).
let pins = state.db.list_pinned_cids().await?;
let pins: Vec<serde_json::Value> = pins
.into_iter()
.filter(|p| p.cid.is_some() || p.pinata_cid.is_some())
.map(|p| {
// A stored `cid` is advertised as a node-local resolver key
// only when it passes the raw-CIDv1 contract AND local
// provenance is confirmed. The historical `cid` column has
// held more than one namespace (Kubo dag-pb, Pinata CIDv0),
// and the `/ipfs/{cid}` resolver recomputes the raw CID
// from object bytes and rejects anything else — so a
// legacy provider key in `cid`/`local_cid` would promise a
// local object the node immediately refuses to serve.
// Such rows stay listed (with their provenance flags) but
// carry no local key until repair rewrites them.
let local_key = match &p.cid {
Some(c) if gitlawb_core::cid::is_raw_cidv1(c) && p.local_ipfs_provenance => {
Some(c.clone())
}
_ => None,
};
serde_json::json!({
"sha256_hex": p.sha256_hex,
"cid": local_key.clone(),
"local_cid": local_key,
"pinata_cid": p.pinata_cid,
"local_pinned": p.local_ipfs_provenance,
"pinata_pinned": p.pinata_cid.is_some(),
"pinned_at": p.pinned_at,
})
})
.collect();
Ok(Json(serde_json::json!({
"pins": pins,
"count": pins.len(),
@@ -2489,7 +2426,6 @@ mod closed_pool_tests {
.oneshot(
Request::builder()
.uri("/api/v1/ipfs/pins")
.extension(crate::auth::AuthenticatedDid("did:key:test".into()))
.body(axum::body::Body::empty())
.unwrap(),
)
@@ -2514,369 +2450,6 @@ mod closed_pool_tests {
);
}
/// #218 review P2: the `list_pins` API response surfaces writer-owned
/// provenance (`local_pinned`, `pinata_pinned`) so a `gl` consumer
/// can distinguish local-only, remote-only, and dual rows without
/// re-inferring semantics from nullability. `cid`/`local_cid` carry a
/// value ONLY for rows whose stored key parses as a raw CIDv1 with
/// confirmed local provenance; anything else (Pinata-only rows,
/// legacy provider keys) exposes its usable identifier through
/// `pinata_cid` alone, and every non-null local key must resolve
/// through `GET /ipfs/{cid}`.
#[sqlx::test]
async fn list_pins_reports_writer_owned_provenance_for_all_shapes(pool: sqlx::PgPool) {
use sqlx::Row as _;
let state = crate::test_support::test_state(pool.clone()).await;
let db = &state.db;
// Raw resolver keys must be REAL raw CIDv1 strings: the handler
// gates `cid`/`local_cid` on `is_raw_cidv1`, so placeholder text
// would fail the gate for the wrong reason. Provider CIDs stay
// opaque Qm… strings (never syntax-checked, only echoed).
fn raw_cid_for(content: &[u8]) -> String {
gitlawb_core::cid::Cid::from_git_object_bytes(content).to_string()
}
// Two real blobs on disk in one public repo, so the local and
// dual rows below can be followed through `GET /ipfs/{cid}`.
// The resolver locates repos through `repo_store.acquire`, not
// the row's `disk_path`, so the recipe overwrites the acquired
// bare path with a real clone: workdir commit of both files,
// cloned --bare into the store path, oids via rev-parse, raw
// keys recomputed from the file bytes exactly as the serve
// path does.
fn run_git(args: &[&str], cwd: &std::path::Path) {
let out = std::process::Command::new("git")
.args(args)
.current_dir(cwd)
.output()
.expect("git runs");
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
let content1 = b"pins follow-through one\n";
let content2 = b"pins follow-through two\n";
let tmp = tempfile::TempDir::new().unwrap();
let work = tmp.path().join("work-pins-list");
std::fs::create_dir_all(&work).unwrap();
std::fs::write(work.join("one.txt"), content1).unwrap();
std::fs::write(work.join("two.txt"), content2).unwrap();
run_git(
&["init", "-q", "--object-format=sha256", "-b", "main"],
&work,
);
run_git(&["config", "user.email", "t@t"], &work);
run_git(&["config", "user.name", "t"], &work);
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "seed"], &work);
let oid_of = |spec: &str| {
let out = std::process::Command::new("git")
.args(["rev-parse", spec])
.current_dir(&work)
.output()
.expect("git rev-parse runs");
assert!(out.status.success(), "rev-parse {spec} failed");
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let sha_local = oid_of("HEAD:one.txt");
let sha_dual = oid_of("HEAD:two.txt");
let raw1 = raw_cid_for(content1);
let raw4 = raw_cid_for(content2);
db.upsert_mirror_repo(
"zPinsListOwner",
"pins-list",
"/unused-pins-list",
None,
false,
)
.await
.unwrap();
let rec = db
.get_repo("zPinsListOwner", "pins-list")
.await
.unwrap()
.expect("mirror repo row");
let bare = state
.repo_store
.acquire(&rec.owner_did, &rec.name)
.await
.unwrap();
let _ = std::fs::remove_dir_all(&bare);
std::fs::create_dir_all(&bare).unwrap();
run_git(
&[
"clone",
"--bare",
"-q",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
tmp.path(),
);
// The combination table for what the response must carry:
//
// shape | local_pinned | pinata_pinned | local_cid | pinata_cid
// -------------------+--------------+---------------+-----------+-----------
// local-only | true | false | raw | null
// pinata-only (raw) | false | true | null | provider
// pinata-only (null) | false | true | null | raw(=provider)
// dual | true | true | raw | provider
// legacy provider key| true | false | null | null
//
// The pinata-only (raw) row keeps a genuine raw key in the
// column but must NOT surface it: provenance is unconfirmed,
// so no local field may carry it. The legacy row predates the
// raw-key contract (dag-pb key, provenance claimed) and must
// surface no local key until repair rewrites it — while the
// row itself (and its `local_pinned` flag) stays visible.
//
// Distinct shas (the table is keyed on sha256_hex) keep the
// rows independent.
// (1) local-only: real local pin via the production writer.
db.record_pinned_cid_with_source(&sha_local, &raw1, &rec.id)
.await
.unwrap();
// (2) pinata-only (raw != provider): the post-v27/v30 row shape
// produced by `record_pinata_cid`. The stored raw key is genuine
// but local provenance is unconfirmed, so neither local field
// may carry it; the provider field is the usable identifier.
let sha_pinata_raw = "sha_p2_pinata_only_distinct_cids";
let raw2 = raw_cid_for(b"pins pinata-only bytes\n");
let pinata2 = "QmPinataProviderCidForRawOnlyRow";
assert_ne!(raw2, pinata2);
db.record_pinata_cid(
sha_pinata_raw,
&raw2,
pinata2,
Some("repo-p2-pinata-raw"),
i64::MAX,
)
.await
.unwrap();
// (3) pinata-only (raw == provider): `record_pinata_cid` stores
// `cid = NULL` so the resolver key isn't aliased to a dag-pb
// provider CID that doesn't hash to the raw bytes (#173).
let sha_pinata_null = "sha_p2_pinata_only_null_cid";
let same = "QmPinataOnlyCidRawEqualsProvider";
db.record_pinata_cid(
sha_pinata_null,
same,
same,
Some("repo-p2-pinata-null"),
i64::MAX,
)
.await
.unwrap();
// (4) dual: local first (sets the flag and `cid`), then Pinata
// (sets `pinata_cid` and preserves `cid` and `local_ipfs_provenance`).
let pinata4 = "QmPinataProviderCidForDualRow";
db.record_pinned_cid_with_source(&sha_dual, &raw4, &rec.id)
.await
.unwrap();
db.record_pinata_cid(&sha_dual, &raw4, pinata4, Some(&rec.id), i64::MAX)
.await
.unwrap();
// (5) legacy provider key: a pre-fix row whose `cid` is a dag-pb
// provider identifier with claimed local provenance. Production
// writers can no longer produce this shape, so it is seeded with
// raw SQL — the point is that the projection must not trust column
// shape as proof of a raw local key.
let sha_legacy = "sha_p2_legacy_provider_key";
sqlx::query(
"INSERT INTO pinned_cids (sha256_hex, cid, pinned_at, pinata_cid, repo_id, local_ipfs_provenance)
VALUES ($1, $2, $3, NULL, $4, TRUE)",
)
.bind(sha_legacy)
.bind("QmLegacyProviderKeyPredatesRawContract")
.bind("2026-07-01T00:00:00Z")
.bind(&rec.id)
.execute(&pool)
.await
.unwrap();
// Hit the production handler end-to-end through a one-shot router
// request, so the test exercises the actual response shape
// (not a unit test on PinnedCidRecord fields).
let resp = Router::new()
.route("/api/v1/ipfs/pins", axum::routing::get(list_pins))
.layer(axum::Extension(crate::auth::AuthenticatedDid(
"did:key:zPinListTest".to_string(),
)))
.with_state(state.clone())
.oneshot(
axum::http::Request::builder()
.uri("/api/v1/ipfs/pins")
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(resp.status(), axum::http::StatusCode::OK);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
let pins = v
.get("pins")
.and_then(|p| p.as_array())
.expect("pins array");
assert_eq!(
pins.len(),
5,
"all five shapes must appear in the response (every row has at least one of cid/pinata_cid set)"
);
// Build a map sha -> pin object for ergonomic assertions.
let by_sha: std::collections::HashMap<String, &Value> = pins
.iter()
.map(|p| {
(
p.get("sha256_hex")
.and_then(|s| s.as_str())
.unwrap()
.to_string(),
p,
)
})
.collect();
// Helper: assert a single pin's fields.
let assert_pin = |sha: &str,
local_pinned: bool,
pinata_pinned: bool,
local_cid: Option<&str>,
pinata_cid: Option<&str>| {
let pin = by_sha.get(sha).unwrap_or_else(|| {
panic!(
"pin row for {sha} missing; got shas {:?}",
by_sha.keys().collect::<Vec<_>>()
)
});
assert_eq!(
pin.get("local_pinned").and_then(|v| v.as_bool()),
Some(local_pinned),
"{sha}: local_pinned mismatch",
);
assert_eq!(
pin.get("pinata_pinned").and_then(|v| v.as_bool()),
Some(pinata_pinned),
"{sha}: pinata_pinned mismatch",
);
assert_eq!(
pin.get("local_cid").and_then(|v| v.as_str()),
local_cid,
"{sha}: local_cid mismatch",
);
assert_eq!(
pin.get("pinata_cid").and_then(|v| v.as_str()),
pinata_cid,
"{sha}: pinata_cid mismatch",
);
// Round-3 P2: `cid` is the LOCAL resolver key, full
// stop. It is the raw CID when the row has a local pin
// (local-only or dual shape) and `None` for Pinata-only
// rows. The previous contract aliased the Pinata provider
// CID into `cid` for Pinata-only rows, which made
// `gl ipfs list` advertise a CID the node's own
// `/ipfs/{cid}` resolver cannot serve (404). The
// provenance split is exposed via `local_cid`,
// `pinata_cid`, and the boolean `local_pinned` /
// `pinata_pinned` flags.
let cid_value = pin.get("cid").cloned().unwrap_or(serde_json::Value::Null);
let expected_cid = local_cid
.map(|s| serde_json::Value::String(s.to_string()))
.unwrap_or(serde_json::Value::Null);
assert_eq!(cid_value, expected_cid, "{sha}: cid mismatch");
};
// (1) local-only
assert_pin(&sha_local, true, false, Some(raw1.as_str()), None);
// (2) pinata-only (raw != provider): the stored raw key is
// genuine but provenance is unconfirmed, so NO local field may
// carry it — remote-only state is visible through `pinata_cid`
// alone.
assert_pin(sha_pinata_raw, false, true, None, Some(pinata2));
// (3) pinata-only (raw == provider) — cid NULL, pinata_cid set
assert_pin(sha_pinata_null, false, true, None, Some(same));
// (4) dual
assert_pin(&sha_dual, true, true, Some(raw4.as_str()), Some(pinata4));
// (5) legacy provider key: no local field carries the dag-pb
// key, but the row and its claimed provenance stay visible so
// the repair gap is observable rather than hidden.
assert_pin(sha_legacy, true, false, None, None);
// Sanity: the `local_ipfs_provenance` column itself is what
// powers `local_pinned`, so the response field must agree
// with the database column. Reading directly avoids any
// confusion if the writer path changes.
let rows = sqlx::query("SELECT sha256_hex, local_ipfs_provenance FROM pinned_cids")
.fetch_all(&pool)
.await
.unwrap();
let mut db_provenance: std::collections::HashMap<String, bool> = Default::default();
for r in rows {
let sha: String = r.get("sha256_hex");
let p: bool = r.get("local_ipfs_provenance");
db_provenance.insert(sha, p);
}
for (sha, expected) in [
(sha_local.as_str(), true),
(sha_pinata_raw, false),
(sha_pinata_null, false),
(sha_dual.as_str(), true),
(sha_legacy, true),
] {
assert_eq!(
db_provenance.get(sha).copied(),
Some(expected),
"db column for {sha} does not match expected writer-owned provenance"
);
}
// Follow-through: every non-null local key in the response must
// resolve through `GET /ipfs/{cid}`. The local-only and dual rows
// above name real blobs on disk in a public repo, so both must
// serve their exact bytes; the Pinata-only and legacy rows carry
// no local key and have nothing to follow.
let follow_router = Router::new()
.route("/ipfs/{cid}", axum::routing::get(get_by_cid))
.with_state(state.clone());
for (raw, content) in [(&raw1, &content1[..]), (&raw4, &content2[..])] {
let resp = follow_router
.clone()
.oneshot(
axum::http::Request::builder()
.uri(format!("/ipfs/{raw}"))
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
axum::http::StatusCode::OK,
"advertised local key {raw} must resolve"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
assert_eq!(
body.as_ref(),
content,
"resolved bytes for {raw} must match the pinned blob"
);
}
}
/// #251 / CodeRabbit nit: cover `get_by_cid`'s DB-error conversion path — a
/// valid CID must still yield 503 on a closed pool.
#[sqlx::test]
@@ -2947,10 +2520,7 @@ mod tests {
"/ipfs/{cid}",
axum::routing::get(crate::api::ipfs::get_by_cid),
)
.layer(axum::middleware::from_fn_with_state(
crate::auth::SignatureBodyLimit(crate::server::SIGNED_BODY_LIMIT),
crate::auth::optional_signature,
))
.layer(axum::middleware::from_fn(crate::auth::optional_signature))
.with_state(state)
}
@@ -8785,7 +8355,6 @@ mod tests {
#[tokio::test]
async fn get_by_cid_per_source_cap_sheds_same_source_admits_other() {
let mut state = crate::test_support::test_state_lazy();
state.db.pool().close().await;
// Global pool has room; the per-source cap is 1.
state.git_ipfs_walk_semaphore = Arc::new(Semaphore::new(8));
state.git_ipfs_walk_per_caller = crate::rate_limit::PerCallerConcurrency::new(1, 100);
@@ -8812,23 +8381,16 @@ mod tests {
"a source at its per-source /ipfs walk cap must shed 503 with global capacity free"
);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(body["error"], "overloaded");
// A DIFFERENT source is NOT shed by the per-source cap: it clears admission and
// proceeds to the closed DB, which has a distinct db_unavailable error code.
// proceeds (then errors on the lazy DB, which is not a 503).
let resp = ipfs_router(state)
.oneshot(get_cid(&cid, Some(other)))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(
body["error"],
crate::error::DB_UNAVAILABLE_CODE,
"a different source must clear admission and reach the closed database"
assert_ne!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"a different source must not be shed by the per-source cap"
);
}
+40 -75
View File
@@ -238,32 +238,49 @@ pub async fn close_issue(
headers: axum::http::HeaderMap,
crate::rate_limit::PeerAddr(peer): crate::rate_limit::PeerAddr,
) -> Result<Json<serde_json::Value>> {
// READ-GATE before any snapshot work or rate charging, through the shared
// authorize_repo_read gate so quarantined repos are opaque here too. The
// author fallback below needs the issue blob, which needs the repo tree, so
// authorship cannot be established without a download; but a caller who
// cannot even READ the repo must be stopped here, cheaply, before any
// Tigris transfer or extraction happens. Without this, any signed non-owner
// could issue parallel close requests for arbitrary issue ids and drive
// unbounded downloads and blocking extraction (a disposable-identity DoS),
// because the route has no other pre-authorization.
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// READ-GATE before any snapshot work or rate charging. The author fallback below
// needs the issue blob, which needs the repo tree, so authorship cannot be established
// without a download; but a caller who cannot even READ the repo must be
// stopped here, cheaply, before any Tigris transfer or extraction happens.
// Without this, any signed non-owner could issue parallel close requests for
// arbitrary issue ids and drive unbounded downloads and blocking extraction
// (a disposable-identity DoS), because the route has no other pre-authorization.
//
// Rate limiting runs AFTER this gate so a 429 cannot distinguish a hidden
// repo from a missing one (INV-12 read-denial status contract).
// Rate limiting runs AFTER this gate so a 429 cannot distinguish a hidden repo
// from a missing one (INV-12 read-denial status contract).
//
// mirror-rows-handled: a repo row synced from a peer is stored public and
// carries none of the owner's visibility rules, so for such a row this gate
// can only return allow. That is deliberate rather than overlooked, and it
// is the same verdict every other read gate in this API reaches for one.
// Refusing instead would deny the repo's real owner and the issue's real
// author on any node whose only copy of the repo is a synced one, which is
// an ordinary state here, and it would not buy the protection it appears
// to, because a synced row's recorded owner comes from the peer that sent
// it. The expensive work this check guards is bounded ahead of it by the
// per-IP limiter and the read pool slot taken below, and the authoritative
// owner-or-author decision still runs below and again under the write lock.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
// carries none of the owner's visibility rules, so for such a row this check
// can only return allow. That is deliberate rather than overlooked, and it is
// the same verdict every other read gate in this API reaches for one. Refusing
// instead would deny the repo's real owner and the issue's real author on any
// node whose only copy of the repo is a synced one, which is an ordinary state
// here, and it would not buy the protection it appears to, because a synced
// row's recorded owner comes from the peer that sent it. The expensive work
// this check guards is bounded ahead of it by the per-IP limiter and the read
// pool slot taken below, and the authoritative owner-or-author decision still
// runs below and again under the write lock.
{
// mirror-rows-handled: synced mirror rows carry no owner rules; read gate only.
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.0.as_str();
if crate::visibility::visibility_check(
&rules,
record.is_public,
&record.owner_did,
Some(caller),
"/",
) == crate::visibility::Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
}
// Per-IP flood brake, layered on the same shared limiter and trusted-proxy
// policy as the push advertisement. The pre-lock snapshot downloads the
@@ -485,15 +502,6 @@ mod tests {
.await
.expect("get_repo")
.expect("repo exists");
// upsert_mirror_repo seeds the row quarantined, and the read gate now
// hides quarantined rows as opaque 404 before the owner-or-author check
// this test names. Release it so the stranger is refused by the 403, not
// by the quarantine short-circuit.
state
.db
.set_repo_quarantine(&record.id, false)
.await
.expect("release seed quarantine");
// An independent session holds the repo's write lock for the whole call.
let key = crate::git::repo_store::advisory_lock_key_for_test(
@@ -695,15 +703,6 @@ mod tests {
.await
.expect("get_repo")
.expect("seeded repo exists");
// upsert_mirror_repo seeds the admission flag as quarantined, but the
// gate now hides quarantined rows from every caller class. Release it
// so these tests exercise what they name: the owner-or-author check
// after the read gate, not the quarantine short-circuit.
state
.db
.set_repo_quarantine(&record.id, false)
.await
.expect("release seed quarantine");
let _ = owner_did;
let path = state
.repo_store
@@ -729,40 +728,6 @@ mod tests {
path
}
/// The read gate blocks the owner (and any caller) before the
/// owner-or-author checks: even the owner gets the opaque 404 on a
/// quarantined repo, and the seeded row's quarantined flag is left set
/// deliberately. The released-mirror variants above exercise the
/// owner-or-author path.
#[sqlx::test]
async fn quarantined_repo_close_is_opaque_404(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner_did = "did:key:z6MkT1Owner";
state
.db
.upsert_mirror_repo("z6MkT1Owner", "tquar", "/unused", None, true)
.await
.expect("seed repo row");
let res = close_issue(
axum::extract::State(state.clone()),
axum::Extension(crate::auth::AuthenticatedDid(owner_did.to_string())),
axum::extract::Path((
"z6MkT1Owner".to_string(),
"tquar".to_string(),
"1".to_string(),
)),
axum::http::HeaderMap::new(),
crate::rate_limit::PeerAddr(Some("203.0.113.70:5000".parse().unwrap())),
)
.await;
assert!(
matches!(res, Err(crate::error::AppError::RepoNotFound(_))),
"a quarantined repo must 404 close_issue for every caller: {:?}",
res.ok().map(|_| "unexpected success")
);
}
/// INV-21(c) positive twin 1: the OWNER can still close. The reorder moved the
/// owner check above the lock, so this is the arm most likely to have broken,
/// and the deny test alone could not see it.
+19 -28
View File
@@ -14,13 +14,14 @@ pub struct LabelRequest {
pub label: String,
}
/// Canonicalize and validate a label string. `add_label` and `remove_label`
/// must agree on the form stored in the DB: `add_label` already trims and
/// lowercases before storing, so `remove_label` has to do the same to find
/// the row. Validation is shared so neither handler can drift on charset or
/// length (#344).
fn canonicalize_label(raw: &str) -> std::result::Result<String, AppError> {
let label = raw.trim().to_lowercase();
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = req.label.trim().to_lowercase();
if label.is_empty() || label.len() > 50 {
return Err(AppError::BadRequest("label must be 1–50 characters".into()));
}
@@ -32,20 +33,12 @@ fn canonicalize_label(raw: &str) -> std::result::Result<String, AppError> {
"label must contain only alphanumeric characters, hyphens, and colons".into(),
));
}
Ok(label)
}
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = canonicalize_label(&req.label)?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
let added = state.db.add_label(&record.id, &label).await?;
@@ -66,16 +59,14 @@ pub async fn remove_label(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, label)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let label = canonicalize_label(&label)?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
let removed = state.db.remove_label(&record.id, &label).await?;
if !removed {
return Err(AppError::NotFound(format!("label '{label}'")));
}
state.db.remove_label(&record.id, &label).await?;
Ok(Json(serde_json::json!({ "label": label, "removed": true })))
}
+20 -662
View File
@@ -22,7 +22,6 @@ pub mod replicas;
pub mod repos;
pub mod resolve;
pub mod stars;
pub mod task_cursor;
pub mod tasks;
pub mod visibility;
pub mod webhooks;
@@ -69,25 +68,27 @@ pub(crate) async fn authorize_repo_read(
/// `did:web` / `did:gitlawb` share the base58 space with `did:key`, so a
/// trailing-segment compare would treat `did:key:X` and `did:gitlawb:X` as equal.
pub(crate) fn did_matches(a: &str, b: &str) -> bool {
// One implementation, shared with `gl` and `git-remote-gitlawb` through core:
// every boundary that compares a DID applies the same collapse rule.
gitlawb_core::ucan::push::did_key_eq(a, b)
if a == b {
return true;
}
fn key_id(d: &str) -> &str {
d.strip_prefix("did:key:").unwrap_or(d)
}
let (ka, kb) = (key_id(a), key_id(b));
// After stripping `did:key:`, a value still containing ':' is a non-key full
// DID — do not let it match a bare `did:key` id.
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// Client-facing denial shared by every repo-owner gate copy. `require_repo_owner`
/// renders it as `AppError::Forbidden`; a handler that emits its own
/// `(StatusCode, Json)` envelope instead of `AppError` (e.g.
/// `api::tasks::create_task`) must use this constant too, so two copies of one
/// gate cannot drift in observable output.
pub(crate) const REPO_OWNER_ONLY_MESSAGE: &str = "only the repo owner can perform this action";
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
/// and the author check (close policy) share one normalization.
pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> {
if did_matches(caller, &record.owner_did) {
Ok(())
} else {
Err(AppError::Forbidden(REPO_OWNER_ONLY_MESSAGE.into()))
Err(AppError::Forbidden(
"only the repo owner can perform this action".into(),
))
}
}
@@ -170,28 +171,16 @@ mod authz_guard {
"\nasync fn ",
"\npub fn ",
"\nfn ",
"#[cfg(test)]",
]
.iter()
.filter_map(|p| rest[1..].find(p).map(|i| i + 1))
.min()
.unwrap_or(rest.len());
// Strip block comments too: a marker surviving only inside /* */ would
// otherwise satisfy a row with the real gate deleted.
// Line comments first: a `//` line containing `/*` must not open a
// block range or real code (and gate markers) gets stripped with it.
let mut body = rest[..end]
rest[..end]
.lines()
.filter(|l| !l.trim_start().starts_with("//"))
.collect::<Vec<_>>()
.join("\n");
while let Some(open) = body.find("/*") {
match body[open..].find("*/").map(|i| open + i + 2) {
Some(close) => body.replace_range(open..close, ""),
None => break,
}
}
body
.join("\n")
}
#[test]
@@ -214,14 +203,9 @@ mod authz_guard {
// (source, handler, expected gate marker)
let rows: &[(&str, &str, &str)] = &[
// Bucket A — owner-gate (require_repo_owner -> 403), now behind
// the shared read gate first so a hidden repo 404s for every
// caller class before the owner check can answer 403.
(pulls, "merge_pr", "authorize_repo_read("),
// Bucket A — owner-gate (require_repo_owner -> 403)
(pulls, "merge_pr", "require_repo_owner("),
(webhooks, "create_webhook", "authorize_repo_read("),
(webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "authorize_repo_read("),
(webhooks, "delete_webhook", "require_repo_owner("),
// Bucket A/B hybrid — list_webhooks is read-visibility THEN owner:
// authorize_repo_read 404s a non-reader of a private repo, then
@@ -230,15 +214,10 @@ mod authz_guard {
// (existence hiding) and require_repo_owner guards the owner half.
(webhooks, "list_webhooks", "authorize_repo_read("),
(webhooks, "list_webhooks", "require_repo_owner("),
(labels, "add_label", "authorize_repo_read("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "authorize_repo_read("),
(labels, "remove_label", "require_repo_owner("),
// Bucket A' — owner OR author (did_matches against the author),
// also behind the shared read gate first.
(pulls, "close_pr", "authorize_repo_read("),
// Bucket A' — owner OR author (did_matches against the author)
(pulls, "close_pr", "did_matches("),
(issues, "close_issue", "authorize_repo_read("),
(issues, "close_issue", "did_matches("),
// Bucket B — read-gate (authorize_repo_read)
(pulls, "create_review", "authorize_repo_read("),
@@ -263,11 +242,6 @@ mod authz_guard {
(events, "list_repo_events", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
// #496: create_task is signer-self AND owner-when-repo-scoped — a
// repo_id naming a hosted repo admits tasks only from its owner.
// Both halves are pinned: the did_matches row guards the signer
// binding, this one the repo-ownership gate.
(tasks, "create_task", "require_repo_owner("),
(tasks, "claim_task", "did_matches("),
(tasks, "complete_task", "did_matches("),
(tasks, "fail_task", "did_matches("),
@@ -284,66 +258,21 @@ mod authz_guard {
(bounties, "dispute_bounty", "did_matches("),
(replicas, "register_replica", "did_matches("),
(replicas, "unregister_replica", "replica_did = &auth.0"),
// PRE-GATED — owner-gated behind the shared read gate; guard both
(protect, "protect_branch", "authorize_repo_read("),
// PRE-GATED — already owner-gated, in-scope group; guard the gate itself
(protect, "protect_branch", "did_matches("),
(protect, "unprotect_branch", "authorize_repo_read("),
(protect, "unprotect_branch", "did_matches("),
(visibility, "set_visibility", "authorize_repo_read("),
(visibility, "set_visibility", "require_owner("),
(visibility, "remove_visibility", "authorize_repo_read("),
(visibility, "remove_visibility", "require_owner("),
(visibility, "list_visibility", "authorize_repo_read("),
(visibility, "list_visibility", "require_owner("),
];
// Rows above prove each visibility handler calls authorize_repo_read and
// require_owner; this proves the helper itself does DID-safe matching.
// The visibility rows prove require_owner is CALLED; this proves the helper
// itself does DID-safe matching, not a raw/trailing-segment compare.
assert!(
fn_body(visibility, "require_owner").contains("did_matches("),
"visibility::require_owner must use did_matches for DID-safe owner matching"
);
// Order matters as much as presence: hoisting the owner/author check
// above the read gate would 403 a non-owner on a hidden repo, an
// existence oracle. Pin read-gate-then-owner on every handler that
// takes both; the owner marker differs per bucket (require_repo_owner
// for owner-only, require_owner for visibility, did_matches for the
// owner-or-author and protect paths).
for (src, func, owner_marker) in [
(visibility, "set_visibility", "require_owner("),
(visibility, "remove_visibility", "require_owner("),
(visibility, "list_visibility", "require_owner("),
(pulls, "merge_pr", "require_repo_owner("),
(webhooks, "list_webhooks", "require_repo_owner("),
(webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "require_repo_owner("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "require_repo_owner("),
(pulls, "close_pr", "did_matches("),
(issues, "close_issue", "did_matches("),
(protect, "protect_branch", "did_matches("),
(protect, "unprotect_branch", "did_matches("),
] {
let body = fn_body(src, func);
let gate = body.find("authorize_repo_read(").expect("gate marker");
let owner = body.find(owner_marker).expect("owner marker");
assert!(
gate < owner,
"{func} must run authorize_repo_read before {owner_marker}"
);
// The owner-or-author and protect handlers also carry
// require_repo_owner: a hoist of that call above the read gate is
// the same 403-vs-404 oracle, so pin it too where it exists.
if func == "close_pr" || func == "close_issue" {
let owner_gate = body.find("require_repo_owner(").expect("owner marker");
assert!(
gate < owner_gate,
"{func} must run authorize_repo_read before require_repo_owner("
);
}
}
// The CID read surface (#173) enforces its gate inside the shared
// `gate_and_serve`, which BOTH the provenance path and the legacy scan call, so
// the markers must live there (the get_by_cid row above only proves delegation).
@@ -369,577 +298,6 @@ mod authz_guard {
}
}
/// Completeness fence over the GraphQL mutation surface: the mirror of
/// `every_in_scope_mutation_has_its_gate` for `graphql/mutation.rs`. Kept a
/// SEPARATE guard with its own table (never folded into the REST rows above):
/// a guard over a heterogeneous set must classify per member, and these
/// same-named task rows belong to the GraphQL surface, not `api/tasks.rs`.
///
/// COMPLETENESS comes from the engine's own SDL (`schema.sdl()`), parsed as a
/// structured AST (#219), NOT introspection and NOT a text parse of source.
/// This matters: async-graphql OMITS `#[graphql(visible = false)]` fields from
/// introspection while leaving them executable, so an introspection-sourced fence
/// could be defeated by a hidden mutation. SDL includes those fields, so no
/// visibility flag (and, via the AST parse, no brace-in-description quirk) can hide
/// a live mutation. The SDL field set must equal the registered set: a new field with
/// no row fails (a repo-write cannot slip in unlisted); a registered name absent from
/// the schema fails (renamed or removed). The set is also cross-checked against
/// introspection, so any field present in SDL but hidden from introspection fails
/// named as a forbidden hidden mutation.
///
/// The MARKER check is the only source-scraped part, and it is not the
/// completeness guarantee. Three honest limits, so a green check is not misread
/// as full authorization coverage:
/// 1. A present marker is not load-bearingness (the gate may not redden on a
/// hostile probe: the #203 present-but-vacuous case). The adversarial tests
/// in `graphql/mutation.rs` are the load-bearing layer.
/// 2. A present marker is not gate-correctness (it may be the wrong bucket for
/// the operation's resource: a repo-write registered as signer-self would
/// pass, so INV-1 review remains the defense).
/// 3. The marker check scrapes the comment-masked, brace-bounded resolver body;
/// a `{`/`}` or a `did_matches(` inside a STRING literal is not masked, so it
/// could mis-slice that body or satisfy the marker vacuously (the same
/// vacuous-marker class as limit 1). Completeness is unaffected: it comes
/// from the SDL AST, not from this scrape.
#[test]
fn every_graphql_mutation_has_its_gate() {
// (Rust snake name, schema field in camelCase, expected gate markers). The
// four are Bucket C signer-self: `/graphql` uses optional_signature, so each
// resolver must call `require_signer(ctx)?` and bind the acting DID with
// `did_matches(`. A repo-write would be Bucket A (`require_repo_owner(`).
// Same-named REST rows live in the guard above against `api/tasks.rs`.
let registered: &[(&str, &str, &[&str])] = &[
(
"create_task",
"createTask",
&[
"require_signer(ctx)?",
"did_matches(",
"require_repo_owner(",
],
),
(
"claim_task",
"claimTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"complete_task",
"completeTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"fail_task",
"failTask",
&["require_signer(ctx)?", "did_matches("],
),
];
// Completeness + hidden-mutation cross-check against the REAL schema (#219),
// delegated to the generic `assert_mutation_fence` so the SDL sourcing and the
// cross-check are ALSO exercised by a synthetic hidden schema in
// `fence_rejects_a_hidden_mutation` (INV-21). Because the real MutationRoot has no
// hidden field today, that synthetic test — not a hypothetical production canary —
// is what turns reverting the SDL source or deleting the cross-check RED.
{
use crate::graphql::mutation::MutationRoot;
use crate::graphql::query::QueryRoot;
use crate::graphql::subscription::SubscriptionRoot;
let schema =
async_graphql::Schema::build(QueryRoot, MutationRoot, SubscriptionRoot).finish();
let registered_camel: Vec<&str> = registered.iter().map(|(_, c, _)| *c).collect();
assert_mutation_fence(&schema, &registered_camel);
}
// Marker present (source scrape; the vacuity limit, backstopped by the
// adversarial tests): each resolver body carries every required gate marker.
let masked = mask_comments(include_str!("../graphql/mutation.rs"));
for (snake, _, markers) in registered {
let body = resolver_body(&masked, snake);
for marker in *markers {
assert!(
body.contains(marker),
"GraphQL mutation `{snake}` is missing its gate marker `{marker}`: gate removed"
);
}
}
}
/// Version-safety self-check (#219, INV-21): the fence's completeness guarantee rests
/// on `schema.sdl()` INCLUDING `visible = false` fields while introspection omits
/// them. That is observed async-graphql behavior, not a documented contract, so pin
/// it here: a synthetic schema with a hidden mutation must have that field in its
/// SDL-derived set and NOT in its introspection-derived set. If a future
/// async-graphql upgrade ever filters SDL by visibility, the first assertion flips
/// RED — the assumption is load-bearing, not trusted.
#[test]
fn sdl_source_sees_hidden_mutations_that_introspection_omits() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(visible = false)]
async fn hidden_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// SDL source (the fix): includes the hidden field.
let sdl_fields = mutation_field_names_from_sdl(&schema.sdl());
assert!(
sdl_fields.iter().any(|f| f == "hiddenMut"),
"SDL-sourced set MUST include the visible=false mutation; if this is RED, \
async-graphql started filtering SDL by visibility and the fence's \
completeness guarantee is broken: {sdl_fields:?}"
);
// Introspection (the fence's own helper): OMITS the hidden field. This is the
// exact gap the fix closes — an introspection-sourced fence never sees `hiddenMut`.
let introspected = introspection_mutation_fields(&schema);
assert!(
!introspected.iter().any(|f| f == "hiddenMut"),
"introspection is expected to OMIT the visible=false mutation (that is the \
gap the SDL source closes); got {introspected:?}"
);
// The exact diff the fence computes: hidden ∈ (sdl \ introspection).
let hidden: Vec<&String> = sdl_fields
.iter()
.filter(|f| !introspected.iter().any(|i| i == *f))
.collect();
assert_eq!(
hidden,
vec!["hiddenMut"],
"the SDL-vs-introspection diff must surface exactly the hidden mutation"
);
}
/// INV-21 wiring guard (#219): drives the generic `assert_mutation_fence` — the SAME
/// helper the real fence calls — against a synthetic schema whose mutation is
/// `visible = false` and IS registered, so completeness passes and ONLY the
/// hidden-from-introspection cross-check can fail. This is what makes the fence's SDL
/// sourcing load-bearing without a production canary: reverting the source to
/// introspection (the hidden field then absent from the SDL set) or deleting the
/// cross-check removes this panic (or changes its message), turning the
/// `should_panic` test RED. The real MutationRoot has no hidden field, so nothing on
/// the real schema could catch that regression.
#[test]
#[should_panic(expected = "hidden from introspection")]
fn fence_rejects_a_hidden_mutation() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(visible = false)]
async fn hidden_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// Both fields registered, so completeness passes and only the cross-check fires.
assert_mutation_fence(&schema, &["visibleMut", "hiddenMut"]);
}
/// Deprecated-mutation guard (#219): a `#[graphql(deprecation = "...")]` mutation is
/// executable and present in SDL, but GraphQL introspection omits it by default. The
/// fence's introspection helper uses `includeDeprecated: true`, so a deprecated (but
/// visible) mutation is NOT mistaken for a hidden one. With it registered, the fence
/// must NOT panic. Remove `includeDeprecated: true` and this goes RED — the deprecated
/// field would look hidden and false-red as a forbidden visible=false mutation.
#[test]
fn fence_admits_a_deprecated_mutation() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(deprecation = "use visibleMut")]
async fn deprecated_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// Must not panic: the deprecated field is visible (present in introspection with
// includeDeprecated) and registered, so it is neither hidden nor unclassified.
assert_mutation_fence(&schema, &["visibleMut", "deprecatedMut"]);
}
/// Completeness + hidden-mutation cross-check for a BUILT schema (#219). Generic over
/// the schema type so the real fence AND the synthetic `fence_rejects_a_hidden_mutation`
/// / `fence_admits_a_deprecated_mutation` tests drive the SAME sourcing and cross-check.
/// Completeness is sourced from SDL (which includes `visible = false` fields); the
/// cross-check fails any SDL field hidden from introspection. This genericity is the
/// INV-21 fix: because the real MutationRoot has no hidden field, only a synthetic
/// hidden schema through THIS helper makes reverting the SDL source or removing the
/// cross-check RED, rather than leaving the class latent behind a production canary.
fn assert_mutation_fence<Q, M, S>(
schema: &async_graphql::Schema<Q, M, S>,
registered_camel: &[&str],
) where
Q: async_graphql::ObjectType + 'static,
M: async_graphql::ObjectType + 'static,
S: async_graphql::SubscriptionType + 'static,
{
let sdl_fields = mutation_field_names_from_sdl(&schema.sdl());
let introspected = introspection_mutation_fields(schema);
// No SDL field may be hidden from introspection (a visible=false mutation).
for f in &sdl_fields {
assert!(
introspected.iter().any(|i| i == f),
"GraphQL mutation `{f}` is in the schema but hidden from introspection \
(a `#[graphql(visible = false)]` field): hidden mutations are forbidden \
by this fence — make it visible so its classification is enforceable, or \
remove it"
);
}
// Completeness both directions against the registered (camelCase) set.
for f in &sdl_fields {
assert!(
registered_camel.contains(&f.as_str()),
"unclassified GraphQL mutation `{f}`: add it to \
every_graphql_mutation_has_its_gate with a deliberate gate \
(`require_repo_owner(` for a repo-write, `did_matches(` for a task op)"
);
}
for c in registered_camel {
assert!(
sdl_fields.iter().any(|f| f == c),
"registered GraphQL mutation `{c}` is not in the schema \
(renamed or removed); update the table"
);
}
}
/// The mutation field names via the engine's introspection, run against a passed
/// schema. STRUCTURED (names arrive as data). Uses `fields(includeDeprecated: true)`:
/// GraphQL's default omits `@deprecated` fields, but a `#[graphql(deprecation = "...")]`
/// mutation is still executable and present in SDL, so without this the cross-check
/// above would false-red a deprecated mutation as a hidden (visible=false) one. Only a
/// real visibility filter should trip that arm; deprecation is a lifecycle marker.
fn introspection_mutation_fields<Q, M, S>(
schema: &async_graphql::Schema<Q, M, S>,
) -> Vec<String>
where
Q: async_graphql::ObjectType + 'static,
M: async_graphql::ObjectType + 'static,
S: async_graphql::SubscriptionType + 'static,
{
use async_graphql::Value;
let resp = tokio::runtime::Runtime::new()
.expect("tokio runtime")
.block_on(schema.execute(
"{ __schema { mutationType { fields(includeDeprecated: true) { name } } } }",
));
assert!(
resp.errors.is_empty(),
"schema introspection failed: {:?}",
resp.errors
);
let mut names = Vec::new();
if let Value::Object(root) = &resp.data {
if let Some(Value::Object(sc)) = root.get("__schema") {
if let Some(Value::Object(mt)) = sc.get("mutationType") {
if let Some(Value::List(field_list)) = mt.get("fields") {
for field in field_list {
if let Value::Object(fo) = field {
if let Some(Value::String(n)) = fo.get("name") {
names.push(n.clone());
}
}
}
}
}
}
}
names
}
/// The mutation field names sourced from the built schema's SDL — the completeness
/// source (#219). Builds the same DB-free schema as the introspection helper and runs
/// its `.sdl()` (sync, no runtime needed) through [`mutation_field_names_from_sdl`].
/// SDL includes `#[graphql(visible = false)]` fields, so this source cannot be
/// defeated by hiding a mutation from introspection.
fn graphql_mutation_fields_from_sdl() -> Vec<String> {
use crate::graphql::mutation::MutationRoot;
use crate::graphql::query::QueryRoot;
use crate::graphql::subscription::SubscriptionRoot;
let schema =
async_graphql::Schema::build(QueryRoot, MutationRoot, SubscriptionRoot).finish();
mutation_field_names_from_sdl(&schema.sdl())
}
/// The mutation type's field names extracted from an async-graphql SDL string via
/// the engine's OWN parser AST (`async_graphql::parser::parse_schema`), NOT
/// introspection. This is the completeness source (#219): unlike introspection,
/// which async-graphql filters by `visible` (a `#[graphql(visible = false)]`
/// mutation is executable but ABSENT from introspection), `schema.sdl()` includes
/// every registered field, so a hidden mutation still appears here and must carry a
/// classified row. Parsing the AST (not a brace/line text scan) means a `}` inside a
/// field description cannot mis-slice the type block. camelCase, as async-graphql
/// emits. An empty result on a navigation miss is safe: the set-equality in
/// `every_graphql_mutation_has_its_gate` turns "no fields" into a loud failure for
/// every registered name, never a vacuous pass.
fn mutation_field_names_from_sdl(sdl: &str) -> Vec<String> {
use async_graphql::parser::parse_schema;
use async_graphql::parser::types::{TypeKind, TypeSystemDefinition};
let doc = parse_schema(sdl).expect("async-graphql SDL must parse");
// The mutation root type name: from the `schema { mutation: X }` block if
// present, else the GraphQL default "Mutation".
let mutation_type = doc
.definitions
.iter()
.find_map(|def| match def {
TypeSystemDefinition::Schema(s) => s
.node
.mutation
.as_ref()
.map(|m| m.node.as_str().to_string()),
_ => None,
})
.unwrap_or_else(|| "Mutation".to_string());
// The fields of that object type.
doc.definitions
.iter()
.find_map(|def| match def {
TypeSystemDefinition::Type(t) if t.node.name.node.as_str() == mutation_type => {
match &t.node.kind {
TypeKind::Object(obj) => Some(
obj.fields
.iter()
.map(|f| f.node.name.node.as_str().to_string())
.collect::<Vec<_>>(),
),
_ => None,
}
}
_ => None,
})
.unwrap_or_default()
}
#[test]
fn sdl_extractor_includes_a_hidden_visible_false_field() {
// A field that would be hidden from introspection must still be listed from SDL.
let sdl = "schema { query: QueryRoot mutation: MutationRoot }\n\
type QueryRoot { ping: Int! }\n\
type MutationRoot { createTask: Int! hiddenMut: Int! }\n";
let fields = mutation_field_names_from_sdl(sdl);
assert!(
fields.iter().any(|f| f == "createTask"),
"visible field must be listed: {fields:?}"
);
assert!(
fields.iter().any(|f| f == "hiddenMut"),
"a visible=false field (present in SDL, hidden from introspection) MUST be \
listed by the completeness source: {fields:?}"
);
}
#[test]
fn sdl_extractor_is_not_fooled_by_a_brace_in_a_field_description() {
// A `}` inside a field description must not truncate the type block (the AST
// parse is immune to the text-parse quirk the old design feared).
let sdl = "schema { query: QueryRoot mutation: MutationRoot }\n\
type QueryRoot { ping: Int! }\n\
type MutationRoot {\n\
\"\"\"a description with a } brace and { another\"\"\"\n\
createTask: Int!\n\
claimTask: Int!\n\
}\n";
let fields = mutation_field_names_from_sdl(sdl);
assert!(
fields.iter().any(|f| f == "createTask") && fields.iter().any(|f| f == "claimTask"),
"both fields must be extracted despite a brace in a description: {fields:?}"
);
}
#[test]
fn sdl_extractor_returns_empty_for_a_query_only_schema() {
let sdl = "schema { query: QueryRoot }\ntype QueryRoot { ping: Int! }\n";
assert!(
mutation_field_names_from_sdl(sdl).is_empty(),
"a schema with no mutation root yields no mutation fields (no panic)"
);
}
/// The comment-masked body of `async fn <snake>` in `graphql/mutation.rs`, bounded
/// to the resolver's OWN braces (its `{ .. }`), for the marker check. Brace-matching
/// on the comment-masked source means a `}` in a trailing helper cannot extend the
/// body and a comment marker does not count. A `{`/`}` inside a STRING literal is
/// not masked, a documented marker-check-only residual (limit 3); completeness does
/// not use this.
fn resolver_body(masked_src: &str, snake: &str) -> String {
let needle = format!("async fn {snake}(");
let start = masked_src.find(&needle).unwrap_or_else(|| {
panic!("resolver `async fn {snake}(` not found in graphql/mutation.rs")
});
let open = start
+ masked_src[start..]
.find('{')
.expect("resolver body opening brace");
let mut depth = 0i32;
let mut end = masked_src.len();
for (i, c) in masked_src[open..].char_indices() {
match c {
'{' => depth += 1,
'}' => {
depth -= 1;
if depth == 0 {
end = open + i + 1;
break;
}
}
_ => {}
}
}
masked_src[open..end].to_string()
}
/// `src` with `//` line comments and `/* */` block comments (nested) blanked to
/// spaces, WITHOUT opening a comment inside a `"..."` string (so a `"https://"` does
/// not erase the rest of its line). String CONTENTS are left intact, so a brace or
/// a gate marker inside a string is not masked; for the marker check that is a
/// documented vacuity residual (limit 3), and completeness does not use this at
/// all. Char literals and raw/byte strings are not tracked (absent in resolver
/// bodies); a `'"'` char or a raw string is the remaining narrow residual.
fn mask_comments(src: &str) -> String {
let chars: Vec<char> = src.chars().collect();
let mut out = String::with_capacity(src.len());
let mut i = 0;
let mut block_depth = 0usize;
let mut in_string = false;
while i < chars.len() {
let c = chars[i];
let next = chars.get(i + 1).copied();
if block_depth > 0 {
if c == '/' && next == Some('*') {
block_depth += 1;
out.push_str(" ");
i += 2;
} else if c == '*' && next == Some('/') {
block_depth -= 1;
out.push_str(" ");
i += 2;
} else {
out.push(if c == '\n' { '\n' } else { ' ' });
i += 1;
}
} else if in_string {
out.push(c);
if c == '\\' {
if let Some(n) = next {
out.push(n);
i += 2;
} else {
i += 1;
}
} else {
if c == '"' {
in_string = false;
}
i += 1;
}
} else if c == '"' {
in_string = true;
out.push('"');
i += 1;
} else if c == '/' && next == Some('/') {
while i < chars.len() && chars[i] != '\n' {
out.push(' ');
i += 1;
}
} else if c == '/' && next == Some('*') {
block_depth = 1;
out.push_str(" ");
i += 2;
} else {
out.push(c);
i += 1;
}
}
out
}
/// The SDL field set (the completeness source of truth, #219) is exactly the
/// registered mutations. If async-graphql adds or renames a field, this reddens
/// and forces the `every_graphql_mutation_has_its_gate` table to be updated. Uses
/// SDL, not introspection: introspection omits `visible = false` fields, so an
/// introspection-based pin here would go green while a hidden mutation existed.
#[test]
fn schema_lists_exactly_the_registered_mutations() {
let mut fields = graphql_mutation_fields_from_sdl();
fields.sort();
assert_eq!(
fields,
vec!["claimTask", "completeTask", "createTask", "failTask"],
"the SDL mutation field set is the authoritative discovered set"
);
}
/// The marker check is bounded to a resolver's OWN body and is comment- and
/// string-aware: a `//` inside a `"https://"` string does not erase a real gate on
/// that line (no false-red), and a trailing module helper's `did_matches(` does not
/// rescue a resolver whose own gate was removed (no false-green).
#[test]
fn resolver_body_is_bounded_and_comment_and_string_aware() {
let src = "\
impl MutationRoot {
async fn ok(&self) -> Result<u32> {
let _ = (\"see https://x/y\", did_matches(a, b)); // URL string then gate, one line
Ok(0)
}
async fn last(&self) -> Result<u32> {
// did_matches( only in a comment
Ok(0)
}
}
fn helper() { let _ = did_matches(z, z); }
#[cfg(test)]
mod tests { async fn t() { let _ = did_matches(x, y); } }
";
let masked = mask_comments(src);
assert!(
resolver_body(&masked, "ok").contains("did_matches("),
"a `//` inside a string must not erase the real gate on that line"
);
assert!(
!resolver_body(&masked, "last").contains("did_matches("),
"the marker check must be bounded to the resolver's own body (a trailing \
helper's marker must not rescue a resolver whose gate was removed)"
);
}
/// Proves the comment-stripping that GUARD-1 added: a marker that appears only
/// in a full-line comment (the real `replicas.rs` false-pass shape) must NOT
/// satisfy a row.
+65 -158
View File
@@ -70,8 +70,6 @@ pub struct PeerResponse {
pub reachable: bool,
}
pub(crate) const PUBLIC_HTTP_URL_REQUIREMENT: &str = "must be a public http(s) URL (no loopback, private, localhost, .localhost, .internal, or .local hosts)";
/// Extract an IPv4 address embedded in an IPv6 literal across the transition
/// formats that carry one: IPv4-mapped (`::ffff:a.b.c.d`), IPv4-compatible
/// (`::a.b.c.d`), 6to4 (`2002:WWXX:YYZZ::/16`), and the NAT64 well-known prefix
@@ -84,7 +82,7 @@ pub(crate) const PUBLIC_HTTP_URL_REQUIREMENT: &str = "must be a public http(s) U
/// sits at a prefix-length-dependent offset (RFC 6052 §2.2) — so they return
/// `None`. Any caller that needs them blocked must reject the wider
/// `64:ff9b::/32` itself; `is_public_http_url` does this in its native-v6 arm.
pub(crate) fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr> {
fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr> {
if let Some(v4) = v6.to_ipv4_mapped().or_else(|| v6.to_ipv4()) {
return Some(v4);
}
@@ -104,120 +102,10 @@ pub(crate) fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr
None
}
/// Whether an IP literal is outside the local/private ranges that an
/// attacker-controlled outbound URL must never reach. Kept separate from URL
/// parsing so callers that resolve a DNS name themselves can apply the same
/// classification to every answer and then pin the approved addresses.
pub(crate) fn is_public_ip(ip: std::net::IpAddr) -> bool {
// Reject loopback/unspecified on the literal as given — catches `::1` and
// `::` before the IPv4-folding below (`::1`.to_ipv4() would otherwise map
// to a non-loopback `0.0.0.1`).
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
// Fold any IPv6 literal that embeds an IPv4 address (mapped, compatible,
// 6to4, NAT64) down to that IPv4 so the v4 range checks catch
// loopback/private addresses smuggled in via an IPv6 encoding, then
// re-check loopback/unspecified.
let ip = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
v4 => v4,
};
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
match ip {
std::net::IpAddr::V4(v4) => {
let o = v4.octets();
// RFC1918 private, link-local, CGNAT (100.64.0.0/10), or the
// RFC1122 "this host" block 0.0.0.0/8 (never a valid destination;
// 0.0.0.0 itself is already caught by the is_unspecified check).
if v4.is_private()
|| v4.is_link_local()
|| (o[0] == 100 && (o[1] & 0xc0) == 64)
|| o[0] == 0
{
return false;
}
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// fc00::/7 (unique-local), fe80::/10 (link-local), or fec0::/10
// (site-local, deprecated by RFC 3879; the link-local mask does
// not cover it).
if (s[0] & 0xfe00) == 0xfc00 || (s[0] & 0xffc0) == 0xfe80 || (s[0] & 0xffc0) == 0xfec0 {
return false;
}
// Any NAT64 address (64:ff9b::/32) that is not the cleanly
// decodable well-known /96 — e.g. the RFC 8215 local-use
// 64:ff9b:1::/48 — carries a translated target whose embedded v4
// sits at a prefix-length-dependent offset (RFC 6052 §2.2).
// Rather than risk a wrong decode across every prefix length we
// reject the whole NAT64 space here. The well-known /96 was already
// folded to its v4 above and never reaches this arm.
if s[0] == 0x0064 && s[1] == 0xff9b {
return false;
}
}
}
true
}
/// Mirror-upstream variant of [`is_public_ip`]: in addition to the shared
/// peer classification, rejects IANA special-purpose ranges that are
/// non-global, locally meaningful, or unsafe as owner-controlled outbound
/// fetch endpoints. For IP-literal upstreams, configure-time URL validation
/// and the fetch-time egress policy share this classifier, so a stored
/// literal can never fail the worker it is scheduled for. Hostname upstreams
/// are only shape-checked at configure time: each fetch resolves DNS and
/// applies this classifier to the answers, so a hostname whose answers turn
/// non-public still fails at fetch time.
pub(crate) fn mirror_upstream_ip_permitted(ip: std::net::IpAddr) -> bool {
if !is_public_ip(ip) {
return false;
}
let classified = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
_ => ip,
};
match classified {
std::net::IpAddr::V4(v4) => {
let [a, b, c, _] = v4.octets();
// IANA special-purpose ranges that are not globally reachable but
// may be routed inside labs, clouds, or operator networks. They are
// unsuitable for an owner-controlled outbound fetch even though
// `Ipv4Addr::is_private` does not classify them as RFC1918.
!(a >= 224
|| (a == 192 && b == 0 && c == 0)
|| (a == 192 && b == 0 && c == 2)
|| (a == 192 && b == 88 && c == 99)
|| (a == 198 && matches!(b, 18 | 19))
|| (a == 198 && b == 51 && c == 100)
|| (a == 203 && b == 0 && c == 113))
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// IANA special-purpose ranges that are non-global, locally
// meaningful, or unsafe to treat as ordinary forge endpoints.
// Reject the complete IETF protocol-assignment /23 rather than
// trying to maintain its narrow globally reachable exceptions;
// a public forge has no reason to depend on those anycast ranges.
!(v6.is_multicast()
|| (s[0] & 0xffc0) == 0xfec0
|| (s[0] == 0x0100 && s[1] == 0 && s[2] == 0 && s[3] == 0)
|| (s[0] == 0x0100 && s[1] == 0 && s[2] == 0 && s[3] == 1)
|| (s[0] == 0x2001 && (s[1] & 0xfe00) == 0)
|| (s[0] == 0x2001 && s[1] == 0x0db8)
|| (s[0] == 0x3fff && (s[1] & 0xf000) == 0)
|| s[0] == 0x5f00)
}
}
}
/// Whether a peer `http_url` is a public http(s) endpoint safe to register.
/// Rejects non-http(s) schemes, loopback/unspecified/private/link-local IPs,
/// and `localhost` / `.localhost` / `.local` / `.internal` hostnames. Used at
/// announce time and by the boot-time prune of already-poisoned rows.
/// and `localhost` / `.local` / `.internal` hostnames. Used at announce time
/// and by the boot-time prune of already-poisoned rows.
pub fn is_public_http_url(raw: &str) -> bool {
let url = match reqwest::Url::parse(raw) {
Ok(u) => u,
@@ -237,7 +125,6 @@ pub fn is_public_http_url(raw: &str) -> bool {
host.truncate(host.trim_end_matches('.').len());
if host.is_empty()
|| host == "localhost"
|| host.ends_with(".localhost")
|| host.ends_with(".local")
|| host.ends_with(".internal")
{
@@ -247,9 +134,60 @@ pub fn is_public_http_url(raw: &str) -> bool {
// before parsing as an IP.
let ip_candidate = host.trim_start_matches('[').trim_end_matches(']');
if let Ok(ip) = ip_candidate.parse::<std::net::IpAddr>() {
// IP literals share the single DNS/literal classification in
// `is_public_ip`; hostname-only rules stay local to this function.
return is_public_ip(ip);
// Reject loopback/unspecified on the literal as given — catches `::1`
// and `::` before the IPv4-folding below (`::1`.to_ipv4() would
// otherwise map to a non-loopback `0.0.0.1`).
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
// Fold any IPv6 literal that embeds an IPv4 address (mapped, compatible,
// 6to4, NAT64) down to that IPv4 so the v4 range checks catch
// loopback/private addresses smuggled in via an IPv6 encoding, then
// re-check loopback/unspecified.
let ip = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
v4 => v4,
};
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
match ip {
std::net::IpAddr::V4(v4) => {
let o = v4.octets();
// RFC1918 private, link-local, CGNAT (100.64.0.0/10), or the
// RFC1122 "this host" block 0.0.0.0/8 (never a valid destination;
// 0.0.0.0 itself is already caught by the is_unspecified check).
if v4.is_private()
|| v4.is_link_local()
|| (o[0] == 100 && (o[1] & 0xc0) == 64)
|| o[0] == 0
{
return false;
}
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// fc00::/7 (unique-local), fe80::/10 (link-local), or fec0::/10
// (site-local, deprecated by RFC 3879 and so never a legitimate
// peer address, but still routable on networks that kept it).
if (s[0] & 0xfe00) == 0xfc00
|| (s[0] & 0xffc0) == 0xfe80
|| (s[0] & 0xffc0) == 0xfec0
{
return false;
}
// Any NAT64 address (64:ff9b::/32) that is not the cleanly
// decodable well-known /96 — e.g. the RFC 8215 local-use
// 64:ff9b:1::/48 — carries a translated target whose embedded v4
// sits at a prefix-length-dependent offset (RFC 6052 §2.2).
// Rather than risk a wrong decode across every prefix length we
// reject the whole NAT64 space here. The well-known /96 was
// already folded to its v4 above and never reaches this arm.
if s[0] == 0x0064 && s[1] == 0xff9b {
return false;
}
}
}
} else if !host.contains('.') {
// A name with no dot at all. The `.local` / `.internal` rules above are
// suffix checks, so a single label slips both, and the resolver decides
@@ -322,9 +260,9 @@ pub async fn announce(
// and turn our outbound sync-notify fan-out into an SSRF probe — and bury
// the real peers under junk so node-origin repos stop replicating.
if !is_public_http_url(&req.http_url) {
return Err(AppError::BadRequest(format!(
"http_url {PUBLIC_HTTP_URL_REQUIREMENT}"
)));
return Err(AppError::BadRequest(
"http_url must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(),
));
}
// Reject self-announcements: a peer row whose http_url is our own public
@@ -603,7 +541,6 @@ mod tests {
fn rejects_loopback_private_and_internal() {
for bad in [
"http://localhost:7545",
"http://node.localhost:7545",
"http://127.0.0.1:5432/",
"http://localhost:22/",
"http://0.0.0.0:7545",
@@ -675,8 +612,6 @@ mod tests {
fn rejects_deprecated_site_local_v6() {
// fec0::/10, deprecated by RFC 3879. The link-local mask (0xffc0 ==
// 0xfe80) does not cover it: 0xfec0 & 0xffc0 is 0xfec0.
assert!(!super::is_public_ip("fec0::1".parse().unwrap()));
assert!(!super::is_public_ip("feff:ffff::1".parse().unwrap()));
for bad in ["http://[fec0::1]/", "http://[feff:ffff::1]:7545"] {
assert!(!is_public_http_url(bad), "{bad:?} must be rejected");
}
@@ -685,14 +620,11 @@ mod tests {
#[test]
fn rejects_repeated_trailing_root_dots() {
// Stripping a single root dot leaves `localhost.`, which matches
// neither the equality nor the suffix check. The subdomain form now
// exercises the `.localhost` suffix rejection together with the extra
// trailing dot.
for bad in [
"http://localhost../",
"http://localhost...:7545",
"http://node.localhost../",
] {
// neither the equality nor the suffix check. The subdomain form
// (`node.localhost..`) is not asserted here: `.localhost` is not yet a
// rejected suffix on this base, so that case belongs to whichever
// change adds it rather than to this one.
for bad in ["http://localhost../", "http://localhost...:7545"] {
assert!(!is_public_http_url(bad), "{bad:?} must be rejected");
}
}
@@ -1650,31 +1582,6 @@ mod tests {
)
}
#[sqlx::test]
async fn announce_rejects_localhost_subdomains_with_an_accurate_error(pool: PgPool) {
let state = test_state(pool).await;
let did = Keypair::generate().did().to_string();
let resp = announce_only(state.clone())
.oneshot(announce_as(
&did,
&announce_body(&did, "https://node.localhost:7545"),
))
.await
.unwrap();
let (status, error, message) = status_and_error(resp).await;
assert_eq!(status, StatusCode::BAD_REQUEST);
assert_eq!(error, "bad_request");
assert_eq!(
message,
"http_url must be a public http(s) URL (no loopback, private, localhost, .localhost, .internal, or .local hosts)"
);
assert!(
snapshot(&state.db, &did).await.is_none(),
"a rejected .localhost announce must leave no row behind"
);
}
/// U4 scenario 1, and the first test the keyid branch has ever had: a caller
/// who proved control of one DID must not announce another's. Kills
/// neutralizing the handler's keyid comparison, and kills demoting this
+10 -4
View File
@@ -17,8 +17,11 @@ pub async fn protect_branch(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, branch)): Path<(String, String, String)>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Only the repo owner can protect branches (DID-safe match, shared idiom).
let caller = &auth.0;
@@ -48,8 +51,11 @@ pub async fn unprotect_branch(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, branch)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = &auth.0;
if !crate::api::did_matches(caller, &record.owner_did) {
+10 -8
View File
@@ -95,7 +95,6 @@ pub async fn create_pr(
webhooks::fire_event(
std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id,
"pull_request.opened",
serde_json::json!({
@@ -200,8 +199,11 @@ pub async fn merge_pr(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, number)): Path<(String, String, i64)>,
) -> Result<Json<serde_json::Value>> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Owner-only merge (N7). Merging writes the served tree, so this is the same
// trust boundary as owner-only push; it subsumes branch protection (a
@@ -247,7 +249,6 @@ pub async fn merge_pr(
webhooks::fire_event(
std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id,
"pull_request.merged",
serde_json::json!({
@@ -273,8 +274,11 @@ pub async fn close_pr(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, number)): Path<(String, String, i64)>,
) -> Result<Json<serde_json::Value>> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let pr = state
.db
@@ -296,7 +300,6 @@ pub async fn close_pr(
webhooks::fire_event(
std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id,
"pull_request.closed",
serde_json::json!({
@@ -347,7 +350,6 @@ pub async fn create_review(
webhooks::fire_event(
std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id,
"pull_request.reviewed",
serde_json::json!({
+10 -9
View File
@@ -39,11 +39,11 @@ pub async fn register_replica(
) -> Result<(StatusCode, Json<serde_json::Value>)> {
validate_replica_url(&req.url)?;
// The mutation response carries repo metadata (name, replica count), so
// registration applies the same read-visibility decision as the listing:
// a caller who may not read the repo gets the same 404 as a missing one.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replica_did = &auth.0;
@@ -93,10 +93,11 @@ pub async fn unregister_replica(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
// Same gate as register_replica: removal mutates repo-scoped metadata and
// its response discloses the replica count.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replica_did = &auth.0;
state.db.unregister_replica(&record.id, replica_did).await?;
File diff suppressed because it is too large. Load diff
-782
View File
@@ -1,782 +0,0 @@
//! Opaque, integrity-protected continuation tokens for the task read surfaces.
//!
//! The task list pages by keyset over `(created_at, id)`, but the rows a
//! caller may *see* are a filtered subset of the rows the query has to
//! *examine*. Handing the caller a raw `(created_at, id)` cursor therefore
//! forced a choice between two broken options (#327 review): anchor the cursor
//! on the last visible row, and a window of denied tasks longer than the scan
//! budget stalls paging forever; or anchor it on the last examined row, and a
//! denied read leaks the id and timestamp of a task `GET /tasks/{id}` other-
//! wise 404s.
//!
//! A server-issued token removes the choice. The position it carries is the
//! last *examined* candidate, so paging always advances, and the payload is
//! opaque and MAC'd, so the caller learns nothing from it and cannot forge one
//! naming a row of their choosing.
//!
//! The position must be *confidential*, not merely unforgeable: a token that
//! merely signed a base64 payload would still let its holder read the id and
//! timestamp of the denied row it names, which is the disclosure the token
//! exists to prevent. The payload is therefore encrypted under a
//! synthetic-IV construction (SIV): the tag is an HMAC over the filter and the
//! plaintext, and it doubles as the IV seeding the keystream the plaintext is
//! XORed with. That needs no randomness source and no dependency beyond the
//! `hmac`/`sha2` pair already used for webhook signatures and blob recipient
//! tags, and it is decrypt-last: nothing is parsed until the tag verifies.
//!
//! Making the token the *only* accepted cursor also fixes the ordering-domain
//! bug the same review found. `agent_tasks.created_at` is TEXT and compared as
//! TEXT, so `...Z` and `...+00:00` denote one instant but sort differently. A
//! caller-typed timestamp could silently skip or repeat same-time rows. The
//! token instead carries the stored string verbatim, so the value compared is
//! always a value the server wrote.
//!
//! A token is bound to the *caller* as well as the filter. The position it
//! names is the last examined candidate, not the last visible one, so it
//! encodes how far a scan got under one caller's visibility. Resuming it as a
//! different caller would start the scan past rows that caller is entitled to
//! read, silently dropping them from the answer (#327 review). The presenting
//! caller's normalized DID (empty and flagged absent when anonymous) is
//! therefore part of the MAC input, so a token presented by anyone else fails
//! to verify rather than under-reporting.
//!
//! Wire form: `v1.<iv>.<body>`, both parts base64url unpadded: the 16-byte
//! synthetic IV (the truncated MAC, which also seeds the keystream) and the
//! padded, encrypted JSON payload.
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use base64::Engine;
use hmac::{Hmac, Mac};
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use crate::error::AppError;
type HmacSha256 = Hmac<Sha256>;
const CURSOR_PREFIX: &str = "v1";
const KEY_DERIVATION_LABEL: &[u8] = b"gitlawb/tasks-cursor-key/v1";
const TAG_DOMAIN: &[u8] = b"gitlawb/tasks-cursor-tag/v1";
const STREAM_DOMAIN: &[u8] = b"gitlawb/tasks-cursor-stream/v1";
/// Truncated MAC length, and the synthetic IV width. 128 bits is far beyond
/// forgery reach for a token that carries no authority of its own (every page
/// re-runs the visibility gate against the presenting caller), and keeps the
/// token short enough to sit in a query string.
const TAG_LEN: usize = 16;
/// Padding bucket for the serialized cursor payload, in bytes. The keystream
/// XORs the plaintext byte for byte, so token length would otherwise track
/// payload length and leak the named row's field widths to a holder who
/// cannot decrypt anything. 128 comfortably covers the widest realistic
/// payload: a 35-char RFC3339 timestamp (`to_rfc3339()` renders 29 chars at
/// millisecond precision and 32 at microsecond, 35 only with all nine
/// fractional digits), a 36-char server-minted UUID, and the fixed JSON
/// envelope plus a 10-digit expiry, which together total 101 bytes. Should a future payload ever exceed the bucket, the length
/// rounds up to the next multiple instead of failing, so `encode` never
/// breaks — it just mints a longer token.
const PAYLOAD_BUCKET: usize = 128;
/// How long an issued cursor stays acceptable. Keyset positions never go stale
/// on their own — `created_at`/`id` are immutable — so this is not a
/// correctness bound. It bounds how long a token stays valid across a node
/// restart-and-rotate and keeps an abandoned page from being resumed
/// indefinitely.
const CURSOR_TTL_SECS: i64 = 24 * 60 * 60;
/// Node-keyed MAC key for continuation tokens, derived from the node keypair
/// seed so it needs no configuration and survives restarts of the same node.
/// Derived rather than used directly so a token forgery oracle could not
/// bear on the signing key itself.
#[derive(Clone)]
pub struct TaskCursorKey([u8; 32]);
impl std::fmt::Debug for TaskCursorKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("TaskCursorKey(<redacted>)")
}
}
impl TaskCursorKey {
pub fn derive(node_seed: &[u8; 32]) -> Self {
let mut mac = HmacSha256::new_from_slice(node_seed).expect("HMAC accepts any key length");
mac.update(KEY_DERIVATION_LABEL);
let mut key = [0u8; 32];
key.copy_from_slice(&mac.finalize().into_bytes());
Self(key)
}
}
/// The keyset position a token carries: the last candidate row the previous
/// request examined, whether or not the caller was allowed to see it.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TaskPosition {
pub created_at: String,
pub id: String,
}
impl TaskPosition {
pub fn new(created_at: impl Into<String>, id: impl Into<String>) -> Self {
Self {
created_at: created_at.into(),
id: id.into(),
}
}
pub fn as_pair(&self) -> (&str, &str) {
(self.created_at.as_str(), self.id.as_str())
}
}
#[derive(Serialize, Deserialize)]
struct CursorPayload<'a> {
/// `created_at` of the last examined candidate, verbatim as stored.
t: &'a str,
/// `id` of the last examined candidate.
i: &'a str,
/// Unix expiry.
e: i64,
}
/// The filter a page was issued under. A cursor is only meaningful against the
/// same `status`/`assignee_did` filter that produced it: resuming a filtered
/// scan from an unfiltered position (or the reverse) silently skips rows.
/// Bound into the MAC rather than stored in the payload, so it costs no token
/// length and cannot be edited without invalidating the tag.
#[derive(Debug, Clone, Copy)]
pub struct TaskFilter<'a> {
pub status: Option<&'a str>,
pub assignee_did: Option<&'a str>,
}
/// The visibility identity a token was issued to. Normalized through
/// `normalize_owner_key` so the two spellings of one `did:key` identity
/// (`did:key:X` and bare `X`) bind identically, matching `did_matches` on the
/// read path: a caller who authenticates with the other spelling of their own
/// DID must be able to resume their own page.
fn caller_binding(caller: Option<&str>) -> &str {
caller.map(crate::db::normalize_owner_key).unwrap_or("")
}
/// The assignee filter a token was issued under. Normalized through
/// `normalize_owner_key` so the two spellings of one `did:key` identity
/// (`did:key:X` and bare `X`) bind identically, matching `list_tasks_keyset`
/// in SQL: minting under `did:key:X` and resuming with bare `X` (or the reverse)
/// hits the same rows and must verify under the MAC.
fn assignee_binding(assignee: Option<&str>) -> &str {
assignee.map(crate::db::normalize_owner_key).unwrap_or("")
}
fn cursor_mac(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
plaintext: &[u8],
) -> HmacSha256 {
let mut mac = HmacSha256::new_from_slice(&key.0).expect("HMAC accepts any key length");
mac.update(TAG_DOMAIN);
// Length-prefix every field so no two distinct filter/caller/plaintext
// tuples can produce the same MAC input.
for field in [
filter.status.unwrap_or("").as_bytes(),
assignee_binding(filter.assignee_did).as_bytes(),
caller_binding(caller).as_bytes(),
plaintext,
] {
mac.update(&(field.len() as u64).to_be_bytes());
mac.update(field);
}
// Presence is distinct from emptiness for all three optional fields, so an
// anonymous token cannot collide with one issued to a caller whose
// normalized DID is the empty string.
mac.update(&[
u8::from(filter.status.is_some()),
u8::from(filter.assignee_did.is_some()),
u8::from(caller.is_some()),
]);
mac
}
/// Synthetic IV: the authentication tag over the filter and plaintext, which
/// also seeds the keystream. Deterministic by construction, so no randomness
/// source is needed and two tokens for the same page are byte-identical.
fn siv(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
plaintext: &[u8],
) -> [u8; TAG_LEN] {
let mut out = [0u8; TAG_LEN];
out.copy_from_slice(
&cursor_mac(key, filter, caller, plaintext)
.finalize()
.into_bytes()[..TAG_LEN],
);
out
}
/// XOR `buf` with the keystream for `iv`. Its own inverse, so encrypt and
/// decrypt are the same call.
fn apply_keystream(key: &TaskCursorKey, iv: &[u8; TAG_LEN], buf: &mut [u8]) {
for (block_index, chunk) in buf.chunks_mut(32).enumerate() {
let mut mac = HmacSha256::new_from_slice(&key.0).expect("HMAC accepts any key length");
mac.update(STREAM_DOMAIN);
mac.update(iv);
mac.update(&(block_index as u64).to_be_bytes());
let block = mac.finalize().into_bytes();
for (byte, k) in chunk.iter_mut().zip(block.iter()) {
*byte ^= k;
}
}
}
/// Mint a token resuming at `position` for `filter`, usable only by `caller`.
///
/// The serialized payload is padded with trailing whitespace to a fixed
/// bucket before the SIV tag is computed: the keystream preserves plaintext
/// length, so an unpadded payload would let the token's length leak the
/// named row's field widths (a 25-char `to_rfc3339()` rendering mints a
/// shorter token than a 35-char one). `decode` tolerates the padding because
/// `serde_json` ignores trailing whitespace, and the tag covers the padded
/// bytes, so a token minted before padding landed still verifies.
pub fn encode(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
position: &TaskPosition,
) -> String {
let mut payload = serde_json::to_vec(&CursorPayload {
t: &position.created_at,
i: &position.id,
e: chrono::Utc::now().timestamp() + CURSOR_TTL_SECS,
})
.expect("cursor payload is plain strings and an integer");
let padded_len = payload.len().next_multiple_of(PAYLOAD_BUCKET);
payload.resize(padded_len, b' ');
let iv = siv(key, filter, caller, &payload);
apply_keystream(key, &iv, &mut payload);
format!(
"{CURSOR_PREFIX}.{}.{}",
URL_SAFE_NO_PAD.encode(iv),
URL_SAFE_NO_PAD.encode(&payload)
)
}
/// One rejection message for every way a token can fail to verify. A caller
/// who mangled a token, replayed an expired one, or tried to move one to a
/// different filter or a different presenting identity learns only that the
/// cursor is not usable — never which of those it was, and never anything
/// about the row it named.
const INVALID_CURSOR: &str = "invalid or expired cursor";
fn reject() -> AppError {
AppError::BadRequest(INVALID_CURSOR.into())
}
/// Verify a token against `filter` and the presenting `caller`, and return the
/// position it carries.
pub fn decode(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
token: &str,
) -> crate::error::Result<TaskPosition> {
let mut parts = token.split('.');
let (Some(version), Some(iv_b64), Some(body_b64), None) =
(parts.next(), parts.next(), parts.next(), parts.next())
else {
return Err(reject());
};
if version != CURSOR_PREFIX {
return Err(reject());
}
let iv_bytes = URL_SAFE_NO_PAD.decode(iv_b64).map_err(|_| reject())?;
let mut plaintext = URL_SAFE_NO_PAD.decode(body_b64).map_err(|_| reject())?;
let iv: [u8; TAG_LEN] = iv_bytes.try_into().map_err(|_| reject())?;
apply_keystream(key, &iv, &mut plaintext);
// Authenticate before parsing: until the tag matches, `plaintext` is just
// attacker-chosen bytes run through a keystream. `verify_truncated_left`
// is the constant-time compare, so a forgery attempt cannot be steered by
// timing the first differing byte.
cursor_mac(key, filter, caller, &plaintext)
.verify_truncated_left(&iv)
.map_err(|_| reject())?;
let decoded: CursorPayload<'_> = serde_json::from_slice(&plaintext).map_err(|_| reject())?;
if decoded.e < chrono::Utc::now().timestamp() {
return Err(reject());
}
Ok(TaskPosition::new(decoded.t, decoded.i))
}
#[cfg(test)]
mod tests {
use super::*;
fn key() -> TaskCursorKey {
TaskCursorKey::derive(&[7u8; 32])
}
fn unfiltered() -> TaskFilter<'static> {
TaskFilter {
status: None,
assignee_did: None,
}
}
#[test]
fn round_trips_position_verbatim() {
let k = key();
// A stored timestamp the server wrote, fractional digits and all.
let pos = TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", "task-a");
let token = encode(&k, unfiltered(), None, &pos);
assert_eq!(decode(&k, unfiltered(), None, &token).unwrap(), pos);
// A sub-bucket payload pads to exactly one bucket: the padding
// claim is an exact width, and every other width assertion here is
// bucket-relative, so only an absolute check catches a pad that
// skips "next multiple" or a drifted bucket constant.
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
assert_eq!(
raw.len(),
PAYLOAD_BUCKET,
"normal token must pad to exactly one bucket"
);
}
/// The whole point of the token is that the caller may hold it without
/// learning the denied row it names. A signed-but-plaintext payload would
/// pass every other test here and still fail this one.
#[test]
fn token_does_not_expose_the_row_it_names() {
let k = key();
let token = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00+00:00", "denied-task-id"),
);
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
let as_text = String::from_utf8_lossy(&raw);
for secret in ["denied-task-id", "2026-01-03", "\"t\"", "\"i\""] {
assert!(
!as_text.contains(secret),
"token body must not carry {secret:?} in the clear: {as_text:?}"
);
}
// Also assert it is not merely reordered or whitespace-mangled JSON.
assert!(
serde_json::from_slice::<serde_json::Value>(&raw).is_err(),
"token body must not be parseable as JSON"
);
}
/// The keystream preserves plaintext length, so without the fixed-width
/// padding a token naming a 35-char timestamp would be measurably longer
/// than one naming a 32-char one: the holder would learn the withheld
/// row's field widths without decrypting anything. Mint the same shape
/// both ways and require identical token length.
#[test]
fn token_length_does_not_vary_with_position_width() {
let k = key();
// Ids are server-minted fixed-length UUIDs; the timestamp is the
// width that varies in production (`to_rfc3339()` renders 35 chars
// with nanoseconds, 25 without fractional seconds).
let id = "b3f7a1c2-4d5e-4f6a-8b9c-0d1e2f3a4b5c";
let narrow = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00+00:00", id),
);
let wide = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", id),
);
assert_eq!(
narrow.len(),
wide.len(),
"tokens must not leak the named row's field widths: {narrow} vs {wide}"
);
// The padded token still decodes to the exact stored string.
assert_eq!(
decode(&k, unfiltered(), None, &wide).unwrap(),
TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", id)
);
}
/// A token minted before the fixed-width padding landed (unpadded payload,
/// tag over the raw bytes) must still verify: `decode` MACs whatever
/// plaintext the keystream recovers and `serde_json` tolerates the
/// missing trailing whitespace. This pins the back-compat claim in the
/// `encode` docstring with a frozen v1 artifact, so a change to the v1
/// recipe pins (domains, MAC input, keystream, segment layout, field
/// names) breaks this test instead of silently re-deriving under the
/// new internals. It does not pin the padding: `decode` never consults
/// `PAYLOAD_BUCKET` and the fixture is unpadded, so the padding-width
/// claim belongs to `oversized_payload_rounds_up_to_next_bucket`.
///
/// Fixture expiry is fixed at 2100-01-01 (`4102444800`): it must stay in
/// the future whenever the test runs.
#[test]
fn unpadded_legacy_token_still_verifies() {
let k = key();
let filter = unfiltered();
let position = TaskPosition::new("2026-01-03T00:00:00+00:00", "legacy-task");
// Pre-padding recipe minted once and frozen: serialize with no
// padding, tag, and encrypt. Deterministic by construction (SIV needs
// no randomness), so this literal is stable across runs.
let token = "v1.ILJyy7JZScm_h-HM1-CggA.l4kJUoHTbqSGTVKL0DP4hORnbOS1_gHYV2fwJYXd2QtM-6TBMCKh-wvluIiaEYC9rUSUPVSteAKkp7jBCmK_bR2D";
assert_eq!(
decode(&k, filter, None, token).unwrap(),
position,
"pre-padding token must verify under the current decode"
);
// The fixture is a genuine pre-padding artifact, so its decoded
// body must stay below one bucket: a literal re-minted under the
// padded recipe would retire this back-compat pin silently.
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
assert!(
raw.len() < PAYLOAD_BUCKET,
"legacy fixture must be the unpadded shape: {} bytes",
raw.len()
);
}
/// The `PAYLOAD_BUCKET` docstring promises an oversized payload rounds up
/// to the next multiple instead of failing. Mint a position whose id
/// pushes the serialized payload past one bucket and require the token
/// to round-trip (at two buckets).
#[test]
fn oversized_payload_rounds_up_to_next_bucket() {
let k = key();
// ~140-char id: payload crosses the 128-byte bucket.
let long_id = "a".repeat(140);
let position = TaskPosition::new("2026-01-03T00:00:00+00:00", long_id.clone());
let probe = serde_json::to_vec(&CursorPayload {
t: &position.created_at,
i: &position.id,
e: 1_700_000_000,
})
.expect("cursor payload is plain strings and an integer");
assert!(
probe.len() > PAYLOAD_BUCKET,
"fixture payload must exceed one bucket: {} bytes",
probe.len()
);
assert!(
probe.len() <= 2 * PAYLOAD_BUCKET,
"fixture payload must fit in two buckets: {} bytes",
probe.len()
);
let token = encode(&k, unfiltered(), None, &position);
assert_eq!(
decode(&k, unfiltered(), None, &token).unwrap(),
position,
"oversized payload must round-trip at two buckets"
);
// The padded body must be exactly two buckets, not merely longer
// than one: the padding claim is an exact width, not a direction.
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
assert_eq!(
raw.len(),
2 * PAYLOAD_BUCKET,
"oversized payload must pad to exactly two buckets"
);
}
/// Callers paste the token straight into a query string, so it must carry
/// no character that needs percent-encoding.
#[test]
fn token_is_url_safe_verbatim() {
let token = encode(
&key(),
TaskFilter {
status: Some("pending"),
assignee_did: Some("did:key:z6MkAssignee"),
},
None,
&TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", "task-a"),
);
assert!(
token
.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.')),
"token must be query-safe without encoding: {token}"
);
}
/// Flipping any byte of a token must fail the tag, not decode to a
/// different position: the keystream is malleable on its own, the tag is
/// what stops a chosen-position forgery.
#[test]
fn rejects_any_tampered_byte() {
let k = key();
let token = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a"),
);
let parts: Vec<&str> = token.split('.').collect();
for part in [1usize, 2] {
for position in [0usize, 3] {
let mut bytes = parts[part].as_bytes().to_vec();
bytes[position] = if bytes[position] == b'A' { b'B' } else { b'A' };
let mut mangled: Vec<String> = parts.iter().map(|p| p.to_string()).collect();
mangled[part] = String::from_utf8(bytes).unwrap();
assert!(
decode(&k, unfiltered(), None, &mangled.join(".")).is_err(),
"byte {position} of part {part} must not be malleable"
);
}
}
// Sanity: the untouched token still decodes, so the loop above is not
// passing because every token is rejected.
assert!(decode(&k, unfiltered(), None, &token).is_ok());
}
#[test]
fn rejects_token_from_another_node() {
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&TaskCursorKey::derive(&[1u8; 32]), unfiltered(), None, &pos);
assert!(decode(
&TaskCursorKey::derive(&[2u8; 32]),
unfiltered(),
None,
&token
)
.is_err());
}
#[test]
fn rejects_cursor_moved_to_a_different_filter() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(
&k,
TaskFilter {
status: Some("pending"),
assignee_did: None,
},
None,
&pos,
);
assert!(decode(&k, unfiltered(), None, &token).is_err());
assert!(decode(
&k,
TaskFilter {
status: Some("claimed"),
assignee_did: None
},
None,
&token
)
.is_err());
assert!(decode(
&k,
TaskFilter {
status: Some("pending"),
assignee_did: None
},
None,
&token
)
.is_ok());
}
/// A filter of `Some("")` must not verify a token minted with `None`.
#[test]
fn distinguishes_absent_filter_from_empty_filter() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&k, unfiltered(), None, &pos);
assert!(decode(
&k,
TaskFilter {
status: Some(""),
assignee_did: None
},
None,
&token
)
.is_err());
}
/// A cursor names how far a scan got under one caller's visibility. Moved
/// to a different presenting identity it would start that caller's scan
/// past rows they are entitled to read, so it must fail to verify rather
/// than silently under-report (#327 review).
#[test]
fn rejects_cursor_moved_to_a_different_caller() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let alice = Some("did:key:z6MkAlice");
let bob = Some("did:key:z6MkBob");
let anon_token = encode(&k, unfiltered(), None, &pos);
assert!(
decode(&k, unfiltered(), alice, &anon_token).is_err(),
"an anonymously minted cursor must not resume an authenticated scan"
);
assert!(decode(&k, unfiltered(), None, &anon_token).is_ok());
let alice_token = encode(&k, unfiltered(), alice, &pos);
assert!(
decode(&k, unfiltered(), bob, &alice_token).is_err(),
"one caller's cursor must not resume another caller's scan"
);
assert!(
decode(&k, unfiltered(), None, &alice_token).is_err(),
"an authenticated cursor must not resume an anonymous scan"
);
assert!(decode(&k, unfiltered(), alice, &alice_token).is_ok());
}
/// The binding is on identity, not spelling: `did:key:X` and bare `X` are
/// one caller everywhere else on the read path (`did_matches`), so a caller
/// who presents the other form of their own DID must keep their own page.
#[test]
fn caller_binding_is_normalized_across_did_key_spellings() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&k, unfiltered(), Some("did:key:z6MkAlice"), &pos);
assert_eq!(
decode(&k, unfiltered(), Some("z6MkAlice"), &token).unwrap(),
pos
);
// A different method sharing the base58 tail is a different principal.
assert!(decode(&k, unfiltered(), Some("did:web:z6MkAlice"), &token).is_err());
}
/// The filter binding is on identity, not spelling: `assignee_did` matches
/// normalized keys in SQL (`normalize_owner_key`), so minting under
/// `did:key:X` and resuming under bare `X` (or vice versa) must succeed,
/// while `did:web:X` remains distinct.
#[test]
fn assignee_filter_binding_is_normalized_across_did_key_spellings() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let filter_did_key = TaskFilter {
status: Some("pending"),
assignee_did: Some("did:key:z6MkAssignee"),
};
let filter_bare = TaskFilter {
status: Some("pending"),
assignee_did: Some("z6MkAssignee"),
};
let filter_web = TaskFilter {
status: Some("pending"),
assignee_did: Some("did:web:z6MkAssignee"),
};
let token_from_did_key = encode(&k, filter_did_key, None, &pos);
assert_eq!(
decode(&k, filter_bare, None, &token_from_did_key).unwrap(),
pos
);
assert!(decode(&k, filter_web, None, &token_from_did_key).is_err());
let token_from_bare = encode(&k, filter_bare, None, &pos);
assert_eq!(
decode(&k, filter_did_key, None, &token_from_bare).unwrap(),
pos
);
assert!(decode(&k, filter_web, None, &token_from_bare).is_err());
}
/// Anonymous is a distinct binding from a caller whose normalized DID is
/// the empty string, the same way `Some("")` is distinct from `None` for
/// the filter fields.
#[test]
fn distinguishes_anonymous_from_empty_caller() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&k, unfiltered(), None, &pos);
assert!(decode(&k, unfiltered(), Some(""), &token).is_err());
}
/// Length-prefixing must stop a status/assignee pair from being re-split.
#[test]
fn rejects_field_boundary_shift() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(
&k,
TaskFilter {
status: Some("pend"),
assignee_did: Some("ing"),
},
None,
&pos,
);
assert!(decode(
&k,
TaskFilter {
status: Some("pending"),
assignee_did: Some("")
},
None,
&token
)
.is_err());
}
#[test]
fn rejects_expired_token() {
let k = key();
// Minted the same way `encode` does, but already past its expiry.
let mut payload = serde_json::to_vec(&CursorPayload {
t: "2026-01-03T00:00:00+00:00",
i: "task-a",
e: chrono::Utc::now().timestamp() - 1,
})
.unwrap();
let iv = siv(&k, unfiltered(), None, &payload);
apply_keystream(&k, &iv, &mut payload);
let expired = format!(
"v1.{}.{}",
URL_SAFE_NO_PAD.encode(iv),
URL_SAFE_NO_PAD.encode(&payload)
);
let err = decode(&k, unfiltered(), None, &expired).unwrap_err();
assert!(err.to_string().contains(INVALID_CURSOR));
}
#[test]
fn rejects_malformed_shapes() {
let k = key();
for bad in [
"",
"v1",
"v1.",
"v1.abc",
"v2.abc.def",
"v1.abc.def.ghi",
"v1.!!!.def",
] {
assert!(
decode(&k, unfiltered(), None, bad).is_err(),
"must reject {bad:?}"
);
}
}
/// Every rejection path renders the same text, so a caller cannot tell a
/// forged token from an expired one from a filter mismatch.
#[test]
fn every_rejection_is_indistinguishable() {
let k = key();
for bad in ["v1.abc.def", "not-a-cursor", "v1..", "v9.a.b"] {
assert_eq!(
decode(&k, unfiltered(), None, bad).unwrap_err().to_string(),
format!("invalid request: {INVALID_CURSOR}")
);
}
}
}
File diff suppressed because it is too large. Load diff
+32 -16
View File
@@ -85,10 +85,11 @@ pub async fn set_visibility(
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<SetVisibilityRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
// Quarantine first (via authorize_repo_read), then owner — same posture as
// list_visibility so a quarantined repo cannot be mutated while reads 404.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(&auth.0), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
validate_path_glob(&req.path_glob)?;
@@ -140,8 +141,11 @@ pub async fn remove_visibility(
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<RemoveVisibilityRequest>,
) -> Result<Json<serde_json::Value>> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(&auth.0), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
state
@@ -167,12 +171,14 @@ pub async fn list_visibility(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
// Quarantine first (via authorize_repo_read), then owner — a quarantined
// mirror must be opaque even to a caller matching owner_did.
let (record, rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(&auth.0), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let rules_json: Vec<_> = rules
.into_iter()
.map(|r| {
@@ -199,18 +205,28 @@ pub async fn list_visibility(
/// denied one (`reinclude`), so a clean-clone client can sparse-exclude the
/// denied subtrees while re-including the allowed nested paths. Unlike
/// `list_visibility` this is not owner-gated and never exposes reader_dids.
///
/// Quarantined repos are opaque 404 via [`crate::api::authorize_repo_read`] —
/// same posture as encrypted-blob discovery — so admission and private-subtree
/// layout are not disclosed to anon, owner, or peers.
pub async fn withheld_paths(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
// Whole-repo read gate: a caller who cannot read "/" gets repo-not-found,
// matching the git read endpoints, so this never discloses a private repo's
// existence or its path layout to an unauthorized caller.
if crate::visibility::visibility_check(&rules, record.is_public, &record.owner_did, caller, "/")
== crate::visibility::Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let withheld =
crate::visibility::withheld_globs(&rules, record.is_public, &record.owner_did, caller);
+27 -217
View File
@@ -33,8 +33,11 @@ pub async fn create_webhook(
Path((owner, name)): Path<(String, String)>,
Json(req): Json<CreateWebhookRequest>,
) -> Result<(StatusCode, Json<Webhook>)> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
// Gate the target through the same hardened public-host validator the peer
@@ -43,10 +46,9 @@ pub async fn create_webhook(
// endpoints (SSRF). Delivery runs on the shared no-redirect client
// (main.rs), which closes the 3xx-to-internal bounce.
if !crate::api::peers::is_public_http_url(&req.url) {
return Err(AppError::BadRequest(format!(
"webhook URL {}",
crate::api::peers::PUBLIC_HTTP_URL_REQUIREMENT
)));
return Err(AppError::BadRequest(
"webhook URL must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(),
));
}
let events = req.events.unwrap_or_else(|| vec!["*".into()]);
@@ -56,14 +58,7 @@ pub async fn create_webhook(
id: Uuid::new_v4().to_string(),
repo_id: record.id,
url: req.url,
// Seal the secret before it touches the database (#506): the column and
// any dump/backup of it hold ciphertext, never the caller's signing key.
secret: req.secret.map(|plain| {
crate::webhook_secret::WebhookSecret::new(crate::webhook_secret::seal(
state.webhook_secret_key.as_ref(),
&plain,
))
}),
secret: req.secret,
events,
created_by_did,
created_at: Utc::now().to_rfc3339(),
@@ -100,11 +95,19 @@ pub async fn list_webhooks(
crate::api::require_repo_owner(&record, &caller)?;
let hooks = state.db.list_webhooks(&record.id).await?;
// `WebhookSecret` redacts itself on serialize, so a set secret is emitted as
// "***" and the signing material never appears in the body (see webhook_secret).
let count = hooks.len();
// Redact secrets in list response
let redacted: Vec<_> = hooks
.into_iter()
.map(|mut h| {
if h.secret.is_some() {
h.secret = Some("***".into());
}
h
})
.collect();
Ok(Json(
serde_json::json!({ "webhooks": hooks, "count": count }),
serde_json::json!({ "webhooks": redacted, "count": redacted.len() }),
))
}
@@ -114,8 +117,11 @@ pub async fn delete_webhook(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
// Verify the webhook belongs to this repo
@@ -135,20 +141,7 @@ pub async fn delete_webhook(
#[cfg(test)]
mod tests {
use axum::extract::{Extension, Path, State};
use axum::Json;
use chrono::Utc;
use gitlawb_core::identity::Keypair;
use sqlx::PgPool;
use uuid::Uuid;
use super::{create_webhook, CreateWebhookRequest};
use crate::api::peers::{is_public_http_url, PUBLIC_HTTP_URL_REQUIREMENT};
use crate::auth::AuthenticatedDid;
use crate::db::RepoRecord;
use crate::error::AppError;
use crate::test_support::test_state;
use rand::Rng;
use crate::api::peers::is_public_http_url;
// create_webhook gates req.url through is_public_http_url. Pin the exact
// SSRF targets from issue #81 so the webhook path can never regress to the
@@ -159,7 +152,6 @@ mod tests {
"http://127.0.0.1:5432/",
"http://169.254.169.254/latest/meta-data/",
"http://localhost/",
"http://node.localhost/",
"http://10.0.0.5/",
"http://[::1]/",
// IPv6 transition encodings smuggling loopback v4 (6to4 / NAT64).
@@ -177,186 +169,4 @@ mod tests {
assert!(is_public_http_url("https://hooks.example.com/gitlawb"));
assert!(is_public_http_url("http://203.0.113.10:7545/"));
}
#[sqlx::test]
async fn webhook_localhost_rejection_uses_shared_public_url_contract(pool: PgPool) {
let state = test_state(pool).await;
let owner = Keypair::generate().did().to_string();
let repo_id = Uuid::new_v4().to_string();
state
.db
.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "webhook-contract".to_string(),
owner_did: owner.clone(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/webhook-contract.git".to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
let result = create_webhook(
State(state.clone()),
Extension(AuthenticatedDid(owner.clone())),
Path((owner, "webhook-contract".to_string())),
Json(CreateWebhookRequest {
url: "https://node.localhost/hook".to_string(),
secret: None,
events: None,
}),
)
.await;
match result {
Err(AppError::BadRequest(message)) => assert_eq!(
message,
format!("webhook URL {PUBLIC_HTTP_URL_REQUIREMENT}")
),
Err(error) => panic!("unexpected webhook rejection: {error}"),
Ok(_) => panic!("a .localhost webhook must be rejected"),
}
assert!(
state.db.list_webhooks(&repo_id).await.unwrap().is_empty(),
"a rejected .localhost webhook must leave no row behind"
);
}
/// The startup re-encryption pass must seal any webhook secret still resting
/// as plaintext (a row written before at-rest encryption shipped), and leave
/// a second pass with nothing to do.
#[sqlx::test]
async fn reencrypt_legacy_secrets_encrypts_existing_cleartext(pool: PgPool) {
let state = test_state(pool).await;
// Seed a webhook the way a pre-encryption node would have: plaintext
// secret written straight into the row, bypassing the handler's seal.
let hook = crate::db::Webhook {
id: Uuid::new_v4().to_string(),
repo_id: "owner/legacy-repo".to_string(),
url: "https://hooks.example.com/legacy".to_string(),
secret: Some(crate::webhook_secret::WebhookSecret::new(
"legacy-plaintext".to_string(),
)),
events: vec!["*".to_string()],
created_by_did: "did:key:zLEGACYOWNER".to_string(),
created_at: Utc::now().to_rfc3339(),
active: true,
};
state.db.create_webhook(&hook).await.unwrap();
let rewritten = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(rewritten, 1, "exactly the one plaintext row is rewritten");
let stored = state.db.get_webhook(&hook.id).await.unwrap().unwrap();
let secret = stored.secret.expect("secret retained after re-encryption");
assert!(secret.is_sealed(), "re-encrypted secret must be sealed");
assert_eq!(
crate::webhook_secret::open(state.webhook_secret_key.as_ref(), secret.as_str())
.as_deref(),
Some("legacy-plaintext"),
"the node-held key must recover the original secret"
);
// Idempotent: a second pass finds nothing left to rewrite.
let again = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(again, 0);
}
/// A legacy plaintext secret that literally begins with the seal prefix is
/// not ciphertext: prefix-matching alone would skip it forever while
/// `reveal` fails to open it, wedging the hook. Classification is by
/// `open`, so the pass seals it and the literal secret survives.
#[sqlx::test]
async fn reencrypt_seals_a_prefixed_plaintext_secret(pool: PgPool) {
let state = test_state(pool).await;
let literal = format!(
"{}not-a-real-ciphertext",
crate::webhook_secret::SEAL_PREFIX
);
let hook = crate::db::Webhook {
id: Uuid::new_v4().to_string(),
repo_id: "owner/prefixed-repo".to_string(),
url: "https://hooks.example.com/prefixed".to_string(),
secret: Some(crate::webhook_secret::WebhookSecret::new(literal.clone())),
events: vec!["*".to_string()],
created_by_did: "did:key:zLEGACYOWNER".to_string(),
created_at: Utc::now().to_rfc3339(),
active: true,
};
state.db.create_webhook(&hook).await.unwrap();
let rewritten = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(rewritten, 1, "the prefixed plaintext row is rewritten");
let stored = state.db.get_webhook(&hook.id).await.unwrap().unwrap();
let secret = stored.secret.unwrap();
assert!(secret.is_sealed());
assert_eq!(
crate::webhook_secret::reveal(state.webhook_secret_key.as_ref(), &secret).as_deref(),
Some(literal.as_str()),
"the literal plaintext, prefix included, is the recovered secret"
);
}
/// A well-formed sealed payload that does not open under this node's key
/// (foreign or rotated ciphertext) must survive the re-encrypt pass
/// untouched: re-wrapping it would make `reveal` return the outer blob as
/// a signing secret and flip a broken hook from withheld into garbage
/// signatures.
#[sqlx::test]
async fn reencrypt_leaves_foreign_ciphertext_untouched(pool: PgPool) {
let state = test_state(pool).await;
let foreign_key: [u8; 32] = chacha20poly1305::aead::OsRng.gen();
let foreign_ciphertext = crate::webhook_secret::seal(&foreign_key, "owner-real-secret");
let hook = crate::db::Webhook {
id: Uuid::new_v4().to_string(),
repo_id: "owner/foreign-repo".to_string(),
url: "https://hooks.example.com/foreign".to_string(),
secret: Some(crate::webhook_secret::WebhookSecret::new(
foreign_ciphertext.clone(),
)),
events: vec!["*".to_string()],
created_by_did: "did:key:zLEGACYOWNER".to_string(),
created_at: Utc::now().to_rfc3339(),
active: true,
};
state.db.create_webhook(&hook).await.unwrap();
let rewritten = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(rewritten, 0, "foreign ciphertext is left unchanged");
let stored = state.db.get_webhook(&hook.id).await.unwrap().unwrap();
let secret = stored.secret.unwrap();
assert_eq!(
secret.as_str(),
foreign_ciphertext,
"the stored value is still the unopenable ciphertext, not a re-wrap"
);
assert!(
crate::webhook_secret::reveal(state.webhook_secret_key.as_ref(), &secret).is_none(),
"delivery still withholds: reveal cannot open the foreign blob"
);
}
}
-62
View File
@@ -1,62 +0,0 @@
//! #26 Split PR 2 — three-outcome recovery policy types.
//!
//! This module owns the recovery classification policy the reviewer
//! demanded: only a trustworthy, protocol-defined absence authorizes
//! a paid re-upload. Everything else stays non-terminal.
//!
//! Deliberately narrow: the ANS-104 codec lives in `ans104`, and the
//! gateway probe, the `verify_anchor` path, and the public verify
//! endpoint are deferred until the uploader, retrieval proof, and
//! recovery consumer can be proven as one vertical slice (reviewer 2,
//! round 5). A probe against a real provider contract — same-network
//! upload/read pair, redirect-safe client, envelope-supplying read
//! API, provider-documented absence evidence — belongs in that slice,
//! not here. What ships here is the policy type plus the rule that
//! only `DefinitivelyAbsent` permits spending another upload.
/// Outcome of a gateway probe for a persisted `item_id`.
// Policy surface for the recovery slice: no construction on this
// head (the probe arrives with the provider contract); the unit
// test pins the rule meanwhile.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ProbeOutcome {
/// The persisted item was served back with a verifiable envelope
/// bound to the requested id. No re-upload allowed.
Present,
/// Trustworthy, protocol-defined proof the item was never
/// accepted. Authorizes re-upload. The exact evidence that
/// establishes this is defined by the provider contract in the
/// vertical slice that introduces the probe — never by a
/// mock-only body shape.
DefinitivelyAbsent,
/// Anything else: transport failure, oversized body, bad
/// signature, id mismatch, ambiguous gateway response. The
/// outbox stays non-terminal.
Indeterminate,
}
impl ProbeOutcome {
/// True iff the recovery code is allowed to spend another paid
/// upload request. Only `DefinitivelyAbsent` qualifies.
#[allow(dead_code)] // the recovery consumer lives in a later slice
pub fn permits_reupload(self) -> bool {
matches!(self, ProbeOutcome::DefinitivelyAbsent)
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Only `DefinitivelyAbsent` authorizes a paid re-upload. This is
/// the policy the reviewer named: collapsing any ambiguous outcome
/// to absent spends a second immutable upload for the same
/// transition.
#[test]
fn only_definitively_absent_authorizes_reupload() {
assert!(!ProbeOutcome::Present.permits_reupload());
assert!(ProbeOutcome::DefinitivelyAbsent.permits_reupload());
assert!(!ProbeOutcome::Indeterminate.permits_reupload());
}
}
+28 -691
View File
@@ -17,74 +17,16 @@ use crate::state::AppState;
#[derive(Clone, Debug)]
pub struct AuthenticatedDid(pub String);
/// A UCAN that passed full chain validation, with the root issuer the chain
/// rests on. Inserted into request extensions by [`require_ucan_chain`] when
/// `X-Ucan` is present; absent when the header is.
///
/// `root` is carried rather than recomputed so the chain is walked once per
/// request. Holding this is not itself an authorization decision — a caller must
/// still compare `root` against an identity it independently trusts, because
/// `did:key` is self-certifying and anyone can mint a chain that verifies.
#[derive(Clone, Debug)]
pub struct VerifiedUcan {
pub ucan: Ucan,
pub root: Did,
}
/// Whether `caller` is authorized to push to `record`.
///
/// The repo owner, or a caller presenting a verified UCAN whose chain roots at
/// that owner and which carries `git/push` for this repo.
///
/// `verified` is optional because `X-Ucan` is: a push carrying no token reaches
/// the same owner-only decision it always did. The owner check is unconditional
/// and runs first, so this can only ever turn a refusal into an acceptance,
/// never the reverse.
pub fn caller_authorized_to_push(
record: &crate::db::RepoRecord,
caller: &str,
verified: Option<&VerifiedUcan>,
) -> bool {
/// Phase 1 (`GITLAWB_ENFORCE_OWNER_PUSH`): owner-only, via the canonical
/// [`crate::api::did_matches`] owner comparison (DID-safe on both sides). This is
/// intentionally a distinct, intent-named gate rather than a bare owner check so
/// that Phase 2 can extend it to honor a verified UCAN `git/push` capability as a
/// pure addition (`did_matches(..) || ucan_grants_push(..)`) without rewriting
/// call sites.
pub fn caller_authorized_to_push(record: &crate::db::RepoRecord, caller: &str) -> bool {
crate::api::did_matches(caller, &record.owner_did)
|| verified.is_some_and(|v| ucan_grants_push(record, v))
}
/// Whether a verified UCAN authorizes a push to `record`.
///
/// Three conditions, all required:
/// 1. The chain roots at this repo's owner. This is the trust anchor — the
/// repo record is data the node holds independently of the token, so a
/// self-minted chain cannot satisfy it.
/// 2. Every link is bounded. There is no revocation path, so an unbounded link
/// is a permanent grant.
/// 3. Every link — the leaf and every proof behind it — carries an
/// unconstrained push-class capability naming this repository. That is
/// [`gitlawb_core::ucan::Ucan::chain_grants_push_to`], the same rule
/// `gl ucan import` applies before storing a token and `git-remote-gitlawb`
/// applies before minting an invocation, so what one boundary accepts the
/// next does not refuse.
///
/// (3) walks the whole chain on purpose. Refusing a wildcard leaf alone did
/// nothing: `is_attenuated_by` accepts a concrete child under a `*` parent, and
/// that narrowing was exactly what the helper performed, so one owner-issued
/// `with: "*"` proof let a delegate mint a concrete leaf for ANY repository the
/// owner had — or created later — and both the chain check and the owner-root
/// check passed. A delegation's scope is fixed when it is issued.
pub fn ucan_grants_push(record: &crate::db::RepoRecord, verified: &VerifiedUcan) -> bool {
if !crate::api::did_matches(&verified.root.to_string(), &record.owner_did) {
return false;
}
// A write capability must lapse on its own. `exp` is optional in the format and
// there is no revocation path, so a chain with an unbounded link is a permanent
// grant: once the token leaks, the owner cannot withdraw it short of rotating
// the DID the repo is keyed on. Refusing here is what makes "the damage window
// is the token's expiry" a true statement rather than an aspiration.
if !verified.ucan.chain_lifetime_is_bounded() {
return false;
}
verified
.ucan
.chain_grants_push_to(&record.owner_did, &record.name)
}
use gitlawb_core::http_sig::{
@@ -92,38 +34,6 @@ use gitlawb_core::http_sig::{
};
use gitlawb_core::identity::verify;
/// Per-route-group ceiling on how much request body [`require_signature`] will
/// buffer BEFORE it has authenticated anything.
///
/// This is middleware state rather than a constant because a single number cannot
/// serve both a 2 GB git push and a 2 MB profile update: the git write group
/// passes `GITLAWB_MAX_PACK_BYTES`, every other signed group passes
/// [`crate::server::SIGNED_BODY_LIMIT`]. Making it a required piece of state means
/// a new signed route group cannot be wired up without choosing a limit.
#[derive(Clone, Copy, Debug)]
pub struct SignatureBodyLimit(pub usize);
/// The hint for an unresolvable DID, chosen by method. The hint is only true
/// for a method we have no resolver for: a did:key that parsed and then failed
/// on its key material (a small-order key, wrong multicodec, wrong length)
/// would be sent looking for the wrong problem by the alpha-support wording.
fn unresolvable_did_hint(key_id: &Did) -> &'static str {
if key_id.is_did_key() {
"the DID is a did:key whose key material did not resolve"
} else {
"only did:key is supported in alpha"
}
}
/// Bound the echoed keyid in the unresolvable-DID message. `key_id` is
/// unauthenticated request input, so reflecting it back at whatever length the
/// caller chose would amplify attacker input into the response body. The bound
/// is slack (a did:key method-id is a fixed 48 characters) and only defends
/// the echo, not the resolution.
fn bounded_did_echo(key_id: &str) -> String {
key_id.chars().take(96).collect()
}
/// Axum middleware that enforces HTTP Signature authentication (RFC 9421).
///
/// Every write request must carry:
@@ -138,52 +48,20 @@ fn bounded_did_echo(key_id: &str) -> String {
/// 4. Resolves the did:key to an Ed25519 VerifyingKey
/// 5. Rebuilds the signing string and verifies the Ed25519 signature
/// 6. Verifies Content-Digest matches the request body
pub async fn require_signature(
State(SignatureBodyLimit(limit)): State<SignatureBodyLimit>,
request: Request,
next: Next,
) -> Response {
verify_signature(limit, request, next).await
}
/// The signature check itself, shared by [`require_signature`] and the optional
/// variant. `limit` bounds the pre-auth buffer; see [`SignatureBodyLimit`].
async fn verify_signature(limit: usize, request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream.
//
// The buffer is bounded, because this runs BEFORE any identity check: an
// unauthenticated caller with no DID, no signature and no repo reaches this
// line. Nothing upstream supplies the bound. On the git write routes
// `RequestBodyLimitLayer` sits on the inner router, and a later `.layer()` is
// the OUTER service, so this middleware runs first and sees the raw body. On
// every other signed group axum's 2 MB `DefaultBodyLimit` is enforced by the
// `Bytes`/`Json` extractors through a request extension, which collecting the
// body here bypasses. `Limited` stops reading at the limit rather than reading
// everything and then measuring it, which is the property that matters.
pub async fn require_signature(request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream
let (parts, body) = request.into_parts();
let body_bytes = match http_body_util::Limited::new(body, limit).collect().await {
Ok(collected) => collected.to_bytes(),
// Over the limit: a distinct status from the 401 an unsigned request gets
// and the 400 below, so a client can tell "too big" from "not authenticated"
// and from "your body did not arrive".
Err(e) if e.is::<http_body_util::LengthLimitError>() => return (
StatusCode::PAYLOAD_TOO_LARGE,
Json(json!({
"error": "body_too_large",
"message": format!("request body exceeds the {limit}-byte limit for this route"),
})),
)
.into_response(),
Err(_) => {
return (
let body_bytes =
match body.collect().await {
Ok(collected) => collected.to_bytes(),
Err(_) => return (
StatusCode::BAD_REQUEST,
Json(
json!({ "error": "unreadable_body", "message": "could not read request body" }),
),
)
.into_response()
}
};
.into_response(),
};
let sig_input = parts
.headers
@@ -267,14 +145,11 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
StatusCode::BAD_REQUEST,
Json(json!({
"error": "unresolvable_did",
"message": format!(
"cannot resolve DID '{}': {e}",
bounded_did_echo(&sig.key_id.to_string())
),
"hint": unresolvable_did_hint(&sig.key_id),
"message": format!("cannot resolve DID '{}': {e}", sig.key_id),
"hint": "only did:key is supported in alpha",
})),
)
.into_response();
.into_response()
}
};
@@ -385,17 +260,11 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
/// Optional variant for rolling upgrades: verify and inject `AuthenticatedDid` when
/// RFC 9421 signature headers are present, but allow legacy unsigned requests to
/// continue when no signature attempt was made.
pub async fn optional_signature(
State(SignatureBodyLimit(limit)): State<SignatureBodyLimit>,
request: Request,
next: Next,
) -> Response {
pub async fn optional_signature(request: Request, next: Next) -> Response {
let has_signature_headers = request.headers().contains_key("signature-input")
|| request.headers().contains_key("signature");
if has_signature_headers {
// Same pre-auth buffer, same bound: presenting signature headers is not
// authentication, so this path is as reachable as the required one.
return verify_signature(limit, request, next).await;
return require_signature(request, next).await;
}
next.run(request).await
}
@@ -413,7 +282,7 @@ fn validate_ucan_chain(
token: &str,
expected_aud: &Did,
signer_did: &Did,
) -> Result<VerifiedUcan, (StatusCode, Json<serde_json::Value>)> {
) -> Result<(), (StatusCode, Json<serde_json::Value>)> {
let ucan = Ucan::decode(token).map_err(|e| {
(
StatusCode::UNAUTHORIZED,
@@ -441,14 +310,14 @@ fn validate_ucan_chain(
)
})?;
let root = ucan.verify_chain().map_err(|e| {
ucan.verify_chain().map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
Ok(VerifiedUcan { ucan, root })
Ok(())
}
/// Axum middleware that validates a UCAN chain when `X-Ucan` is present.
@@ -501,18 +370,11 @@ pub async fn require_ucan_chain(
}
};
let verified = match validate_ucan_chain(&token, &state.node_did, &signer_did) {
Ok(v) => v,
Err((status, body)) => return (status, body).into_response(),
};
if let Err((status, body)) = validate_ucan_chain(&token, &state.node_did, &signer_did) {
return (status, body).into_response();
}
tracing::debug!(did = %signer_did, root = %verified.root, "UCAN chain validated");
// Park the verified token where a handler can reach it. Validation alone
// grants nothing; the authorization decision is made downstream, by a caller
// that knows which identity it trusts for the resource being touched.
let mut request = request;
request.extensions_mut().insert(verified);
tracing::debug!(did = %signer_did, "UCAN chain validated");
next.run(request).await
}
@@ -548,37 +410,6 @@ mod tests {
Ucan::bootstrap(node, agent_did).unwrap()
}
/// The middleware validated a token and threw the result away, so no handler
/// could ever read it and `Ucan::can` had no call site in the node. Validation
/// must hand back both the token and the root the chain rests on.
#[test]
fn validate_ucan_chain_hands_back_the_root_and_the_token() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let caps_vec = vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let delegation =
Ucan::issue(&owner, agent.did(), caps_vec.clone(), None).expect("issue delegation");
let invocation = Ucan::delegate(&agent, node.did(), caps_vec, None, &delegation)
.expect("wrap invocation");
let token = invocation.encode().expect("encode");
let verified = validate_ucan_chain(&token, &node.did(), &agent.did())
.expect("a well-formed owner-rooted invocation must validate");
assert_eq!(
verified.root,
owner.did(),
"the root must be the owner, so a caller can anchor against the repo record"
);
assert_eq!(
verified.ucan.payload.iss,
agent.did(),
"the token itself must come back so a caller can read its capabilities"
);
}
fn delegation_ucan(agent: &Keypair, node_did: Did, proof: &Ucan) -> Ucan {
Ucan::delegate(
agent,
@@ -604,58 +435,6 @@ mod tests {
assert!(validate_ucan_chain(&token, &node_did, &agent_did).is_ok());
}
/// The two-branch hint selection is the point of the unresolvable-DID
/// change: a did:key that failed on its key material must be told the key
/// material did not resolve, and any other method must be told only did:key
/// is supported. A regression that drops the is_did_key() check would send
/// a did:key caller after the wrong problem, so both branches are pinned.
#[test]
fn unresolvable_did_hint_distinguishes_key_material_from_method() {
// A real did:key whose key material fails to resolve (small-order):
// key-material hint.
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let weak_vk = ed25519_dalek::VerifyingKey::from_bytes(&weak).unwrap();
let weak_did = Did::from_verifying_key(&weak_vk);
assert!(
weak_did.to_verifying_key().is_err(),
"fixture precondition: small-order did:key must not resolve"
);
assert_eq!(
unresolvable_did_hint(&weak_did),
"the DID is a did:key whose key material did not resolve"
);
// A method with no resolver: alpha-support hint.
let web = Did::web("example.com");
assert!(
web.to_verifying_key().is_err(),
"fixture precondition: did:web must not resolve locally"
);
assert_eq!(
unresolvable_did_hint(&web),
"only did:key is supported in alpha"
);
}
/// The unresolvable-DID message echoes the keyid, which is unauthenticated
/// request input. The echoed value must be bounded so a hostile caller
/// cannot amplify arbitrary-length input into the response body.
#[test]
fn unresolvable_did_message_bounds_the_echoed_keyid() {
let long_keyid = format!("did:key:z{}", "A".repeat(2000));
let shown = bounded_did_echo(&long_keyid);
assert!(
shown.len() <= 96,
"echoed keyid must be bounded, got {} chars",
shown.len()
);
assert!(
!shown.contains(&"A".repeat(1000)),
"the full keyid must not survive into the message"
);
}
#[test]
fn validate_ucan_chain_wrong_issuer() {
let node = Keypair::generate();
@@ -724,19 +503,16 @@ mod tests {
let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let webhook_secret_key = crate::state::AppState::derive_webhook_secret_key(&keypair);
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/gitlawb_test_placeholder")
.expect("lazy pool creation should not fail");
let db = Arc::new(crate::db::Db::for_testing(pool.clone()));
let task_cursor_key = crate::api::task_cursor::TaskCursorKey::derive(&keypair.to_seed());
let schema = Arc::new(graphql::build_schema(
db.clone(),
ref_tx.clone(),
task_tx.clone(),
task_cursor_key.clone(),
));
crate::state::AppState {
config: Arc::new(Config::parse_from(["gitlawb-node"])),
@@ -748,7 +524,6 @@ mod tests {
ref_update_tx: ref_tx,
task_event_tx: task_tx,
graphql_schema: schema,
task_cursor_key,
machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
@@ -764,12 +539,10 @@ mod tests {
ipfs_max_legacy_scan_rule_bytes:
crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
webhook_secret_key: Arc::new(webhook_secret_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new_bounded(12, Duration::from_secs(60), 10_000),
shutdown_tx: tokio::sync::watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
@@ -785,7 +558,6 @@ mod tests {
git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_ipfs_walk_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
task_read_rate_limiter: RateLimiter::new(1200, Duration::from_secs(3600)),
git_bin: "git".to_string(),
}
}
@@ -921,10 +693,7 @@ mod tests {
StatusCode::OK
}),
)
.layer(middleware::from_fn_with_state(
SignatureBodyLimit(crate::server::SIGNED_BODY_LIMIT),
require_signature,
));
.layer(middleware::from_fn(require_signature));
let req = Request::builder()
.method("POST")
@@ -952,435 +721,3 @@ mod tests {
);
}
}
#[cfg(test)]
mod ucan_push_tests {
use super::*;
use gitlawb_core::identity::Keypair;
use gitlawb_core::ucan::{caps, Capability, Ucan};
const OWNER_KEY: &str = "z6MkOwnerAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
/// `RepoRecord` does not derive `Default`, and adding the derive to a
/// production DB type purely to serve a test is the wrong direction.
fn repo(owner_did: &str, name: &str) -> crate::db::RepoRecord {
crate::db::RepoRecord {
id: "repo-id".to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
disk_path: "/unused".to_string(),
forked_from: None,
machine_id: None,
}
}
/// The token's own issuer and audience are irrelevant to this predicate: the
/// middleware has already bound `iss` to the request signer and `aud` to this
/// node. Only the capabilities and the chain's root matter here.
fn verified(root: &str, caps_vec: Vec<Capability>) -> VerifiedUcan {
verified_with_exp(
root,
caps_vec,
Some(chrono::Utc::now() + chrono::Duration::hours(1)),
)
}
fn verified_with_exp(
root: &str,
caps_vec: Vec<Capability>,
exp: Option<chrono::DateTime<chrono::Utc>>,
) -> VerifiedUcan {
let agent = Keypair::generate();
let node = Keypair::generate();
let ucan = Ucan::issue(&agent, node.did(), caps_vec, exp).expect("issue");
VerifiedUcan {
ucan,
root: root.parse().expect("root DID must parse"),
}
}
fn owner_full() -> String {
format!("did:key:{OWNER_KEY}")
}
fn push_cap_for(owner: &str, name: &str) -> Capability {
Capability::new(format!("gitlawb://repos/{owner}/{name}"), caps::GIT_PUSH)
}
#[test]
fn grants_push_when_the_chain_roots_at_the_owner_and_names_the_repo() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(&owner_full(), vec![push_cap_for(&owner_full(), "myrepo")]);
assert!(ucan_grants_push(&rec, &v));
}
#[test]
fn matches_a_bare_owner_key_against_a_full_did_record() {
// Mirror rows store the bare key. A literal string compare would fail
// here, denying a delegation that is in fact valid.
let rec = repo(OWNER_KEY, "myrepo");
let v = verified(&owner_full(), vec![push_cap_for(&owner_full(), "myrepo")]);
assert!(ucan_grants_push(&rec, &v));
}
/// A perpetual grant is refused even when it is otherwise perfectly valid:
/// owner-rooted, right repo, right action. Without revocation, an unbounded
/// delegation cannot be withdrawn once it leaks.
#[test]
fn refuses_a_delegation_that_never_expires() {
let rec = repo(&owner_full(), "myrepo");
let v = verified_with_exp(
&owner_full(),
vec![push_cap_for(&owner_full(), "myrepo")],
None,
);
assert!(!ucan_grants_push(&rec, &v));
}
#[test]
fn refuses_a_self_minted_root() {
// The whole point: a token nobody delegated grants nothing, however
// permissive its capabilities look.
let stranger = Keypair::generate();
let rec = repo(&owner_full(), "myrepo");
let v = verified(&stranger.did().to_string(), vec![Capability::new("*", "*")]);
assert!(!ucan_grants_push(&rec, &v));
}
#[test]
fn refuses_a_capability_for_a_different_repo() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "otherrepo")],
);
assert!(!ucan_grants_push(&rec, &v));
}
/// A delegation can be issued against the remote's spelling (`myrepo.git`):
/// the matcher strips the same single `.git` the smart-HTTP routes do, so
/// the capability lands on the canonical record name.
#[test]
fn grants_push_when_the_capability_carries_the_git_suffix() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "myrepo.git")],
);
assert!(ucan_grants_push(&rec, &v));
}
/// Mirror-created rows can carry a literal `.git` name; reaching one takes
/// `foo.git.git`, the same spelling the remote URL uses. A single `.git`
/// capability resolves to `foo` and must not fall through to `foo.git`.
#[test]
fn a_repo_literally_named_dot_git_needs_the_doubled_suffix() {
let rec = repo(&owner_full(), "foo.git");
let single = verified(&owner_full(), vec![push_cap_for(&owner_full(), "foo.git")]);
assert!(!ucan_grants_push(&rec, &single));
let doubled = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "foo.git.git")],
);
assert!(ucan_grants_push(&rec, &doubled));
}
#[test]
fn refuses_a_capability_carrying_constraints() {
// `nb` is not interpreted yet. An owner who writes {"refs": [...]} means
// to restrict; honouring the capability while ignoring nb would grant
// strictly more than they intended, so it authorizes nothing.
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "myrepo")
.with_constraints(serde_json::json!({ "refs": ["refs/heads/feat/*"] }))],
);
assert!(!ucan_grants_push(&rec, &v));
}
#[test]
fn refuses_a_non_push_capability() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![Capability::new(
format!("gitlawb://repos/{}/myrepo", owner_full()),
caps::ISSUE_CREATE,
)],
);
assert!(!ucan_grants_push(&rec, &v));
}
/// A resource wildcard must NOT authorize a push, even though attenuation
/// accepts it. The helper narrows a `*` delegation before signing, but the node
/// cannot rely on that: a delegate can sign an invocation that keeps the
/// wildcard, and it would otherwise reach every repository the owner has — or
/// will later create. This test previously asserted the opposite.
#[test]
fn refuses_a_resource_wildcard_and_honours_repo_admin() {
let rec = repo(&owner_full(), "myrepo");
let wildcard = verified(&owner_full(), vec![Capability::new("*", caps::GIT_PUSH)]);
assert!(
!ucan_grants_push(&rec, &wildcard),
"a wildcard resource must not authorize a push at the node"
);
// The ACTION wildcard is a different axis and stays: attenuation bounds it,
// and it still has to name a concrete repository.
let action_wildcard = verified(
&owner_full(),
vec![Capability::new(
format!("gitlawb://repos/{}/myrepo", owner_full()),
"*",
)],
);
assert!(ucan_grants_push(&rec, &action_wildcard));
let admin = verified(
&owner_full(),
vec![Capability::new(
format!("gitlawb://repos/{}/myrepo", owner_full()),
caps::REPO_ADMIN,
)],
);
assert!(ucan_grants_push(&rec, &admin));
}
/// The attack the wildcard refusal exists to stop: one `*` delegation reaching a
/// repository it was never issued against, including one created afterwards.
#[test]
fn a_wildcard_delegation_cannot_reach_a_second_repository() {
let other = repo(&owner_full(), "a-repo-created-later");
let wildcard = verified(&owner_full(), vec![Capability::new("*", caps::GIT_PUSH)]);
assert!(!ucan_grants_push(&other, &wildcard));
}
/// The round-8 P1, executed rather than reasoned: an owner-issued `*` PROOF with
/// a concrete leaf for a repository that did not exist at issuance. Refusing a
/// wildcard *leaf* did nothing here — `is_attenuated_by` accepts a concrete child
/// under a `*` parent, and that narrowing is exactly what `build_invocation` did,
/// so the first-party helper was the working mint path.
#[test]
fn a_wildcard_proof_cannot_reach_a_repo_created_later() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let parent = Ucan::issue(
&owner,
agent.did(),
vec![Capability::new("*", caps::GIT_PUSH)],
Some(hour),
)
.unwrap();
let later = format!("gitlawb://repos/{}/a-repo-created-later", owner.did());
let invocation = Ucan::delegate(
&agent,
node.did(),
vec![Capability::new(&later, caps::GIT_PUSH)],
Some(hour),
&parent,
)
.unwrap();
let root = invocation.verify_chain().expect("the chain still verifies");
let rec = repo(&owner.did().to_string(), "a-repo-created-later");
assert!(
!ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
),
"a wildcard proof must not authorize a repo it never named"
);
}
/// The shipping flow must keep working: a proof that names the repository
/// authorizes a push to it. Guards against fixing the wildcard by refusing
/// everything with a `prf`.
#[test]
fn a_concrete_proof_still_authorizes_the_repo_it_names() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let parent = Ucan::issue(
&owner,
agent.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
)
.unwrap();
let invocation = Ucan::delegate(
&agent,
node.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
&parent,
)
.unwrap();
let root = invocation.verify_chain().expect("chain verifies");
let rec = repo(&owner.did().to_string(), "myrepo");
assert!(ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
));
}
/// A `*` two links up. The immediate proof names the repository, so a walk that
/// checks only one level of `prf` stays green here — this is the case that
/// makes the recursion load-bearing rather than incidental.
#[test]
fn a_wildcard_grandparent_cannot_reach_the_repo_through_a_concrete_parent() {
let owner = Keypair::generate();
let intermediary = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let grandparent = Ucan::issue(
&owner,
intermediary.did(),
vec![Capability::new("*", caps::GIT_PUSH)],
Some(hour),
)
.unwrap();
let parent = Ucan::delegate(
&intermediary,
agent.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
&grandparent,
)
.unwrap();
let invocation = Ucan::delegate(
&agent,
node.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
&parent,
)
.unwrap();
let root = invocation.verify_chain().expect("chain verifies");
assert_eq!(root, owner.did());
let rec = repo(&owner.did().to_string(), "myrepo");
assert!(
!ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
),
"a wildcard anywhere in the chain must deny, not just in the immediate proof"
);
}
/// And the same three-link shape with every link concrete must still grant, so
/// the walk is refusing the wildcard and not the depth.
#[test]
fn a_three_link_concrete_chain_still_authorizes() {
let owner = Keypair::generate();
let intermediary = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let cap = || vec![Capability::new(&resource, caps::GIT_PUSH)];
let grandparent = Ucan::issue(&owner, intermediary.did(), cap(), Some(hour)).unwrap();
let parent =
Ucan::delegate(&intermediary, agent.did(), cap(), Some(hour), &grandparent).unwrap();
let invocation = Ucan::delegate(&agent, node.did(), cap(), Some(hour), &parent).unwrap();
let root = invocation.verify_chain().unwrap();
let rec = repo(&owner.did().to_string(), "myrepo");
assert!(ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
));
}
/// The round-ten P1, pinned where it bites. The middleware verifies the chain
/// on the `X-Ucan` header of any signed request before the scope walk runs,
/// so a depth bound that lived only in `chain_grants_push_to` protected
/// nothing: a hand-built header reached the unbounded recursion first. A
/// chain at `MAX_CHAIN_DEPTH` links validates; one link more is refused here,
/// with 401 and a message that names depth, before any authorization runs.
#[test]
fn validate_ucan_chain_stops_at_the_depth_bound() {
use gitlawb_core::ucan::MAX_CHAIN_DEPTH;
let owner = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let cap = || vec![Capability::new(&resource, caps::GIT_PUSH)];
// owner -> k1 -> ... -> signer: MAX_CHAIN_DEPTH - 1 links, all concrete,
// all signed, all expiring. Only the length is in question.
let mut signer = Keypair::generate();
let mut cur = Ucan::issue(&owner, signer.did(), cap(), Some(hour)).unwrap();
for _ in 2..MAX_CHAIN_DEPTH {
let next = Keypair::generate();
cur = Ucan::delegate(&signer, next.did(), cap(), Some(hour), &cur).unwrap();
signer = next;
}
// Wrapped into an invocation for the node: exactly MAX_CHAIN_DEPTH links.
let at_bound = Ucan::delegate(&signer, node.did(), cap(), Some(hour), &cur).unwrap();
let verified = validate_ucan_chain(&at_bound.encode().unwrap(), &node.did(), &signer.did())
.expect("a chain at the bound must still validate");
assert_eq!(verified.root, owner.did());
// One more link and it is past the bound.
let next = Keypair::generate();
let deeper = Ucan::delegate(&signer, next.did(), cap(), Some(hour), &cur).unwrap();
let past_bound = Ucan::delegate(&next, node.did(), cap(), Some(hour), &deeper).unwrap();
let (status, body) =
match validate_ucan_chain(&past_bound.encode().unwrap(), &node.did(), &next.did()) {
Ok(_) => panic!("a chain past the bound must be refused, not walked"),
Err(refusal) => refusal,
};
assert_eq!(status, StatusCode::UNAUTHORIZED);
let message = body.0["message"].as_str().unwrap_or_default();
assert!(
message.contains("deeper than"),
"the refusal must name depth as the reason: {message}"
);
}
#[test]
fn refuses_a_malformed_resource_uri() {
let rec = repo(&owner_full(), "myrepo");
for bad in [
"",
"myrepo",
"https://repos/x/myrepo",
"gitlawb://repos/myrepo",
] {
let v = verified(&owner_full(), vec![Capability::new(bad, caps::GIT_PUSH)]);
assert!(!ucan_grants_push(&rec, &v), "{bad} must not grant push");
}
}
}
+14 -536
View File
@@ -11,108 +11,10 @@ use uuid::Uuid;
use crate::db::RefCertificate;
use crate::state::AppState;
/// Issue a signed ref-update certificate for a successful push. The
/// live receive-pack handler calls this on every successful push.
/// Build the v1 signing payload — the EXACT bytes the node signs and
/// gl's `verify_signature` reconstructs. This is the single source
/// of truth shared by `issue_ref_certificate` and the frozen
/// canonical-form tests below; drift here is what would break every
/// existing cert.
///
/// Field order and key set are fixed: a default `serde_json::Value`
/// serializes `Map<String, Value>` with sorted keys, so any change to
/// the literal is observable as a different byte sequence on the
/// wire.
///
/// v1 is the unversioned legacy shape: no `version` key in the signed
/// JSON. From v2 onward the version belongs INSIDE the signed bytes
/// (see `v2_signing_payload`): leaving it as an unsigned sibling
/// would let a future v2 cert be downgraded to v1 and still verify
/// cleanly, since the bytes would be identical.
pub(crate) fn v1_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
})
}
/// Build the v2 signing payload — the future shape for versioned
/// certs. Identical to v1 except the wire `version` is part of the
/// signed bytes, so stripping or flipping the `version` column
/// invalidates the signature instead of verifying cleanly under the
/// other version's path.
///
/// NOT yet issued: `issue_ref_certificate` still stamps v1 because
/// the shipped `gl` verifier only supports v1. This builder exists
/// to pin the downgrade-resistant shape now, before any v2 cert is
/// ever signed.
#[allow(dead_code)]
pub(crate) fn v2_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
"version": 2,
})
}
/// Issue a signed ref-update certificate for a successful push.
///
/// `cert_id` is the deterministic id derived from `(request_id,
/// ref_name)` (see [`crate::db::ref_cert_id_for`]). It is required so
/// the recovery drain and the live handler produce the same primary
/// key: a live push followed by a recovery pass collapses to a
/// single cert row, and a re-push to the same `(repo, ref)` updates
/// the existing row's `old_sha` / `new_sha` / `pusher_did` /
/// `issued_at` / `signature` to the new transition while preserving
/// the original `id` (the `insert_ref_certificate` upsert is
/// keyed on `(repo_id, ref_name)` and only updates fields when the
/// new `issued_at` is strictly greater).
///
/// #26 Split PR 1 P1-B: the live handler routes through this
/// function (the upsert). After the reviewer-1 round-2 fix, the
/// recovery drain also routes through this function (P1: refresh a
/// stale cert), so both paths use the same deterministic `cert_id`
/// and the same upsert. A re-pass is always safe:
///
/// - Live handler → live upsert: re-push updates the row, preserves
/// the original `id`. The contract pinned by
/// `insert_ref_certificate_upserts_on_repo_ref` is restored.
/// - Live handler → recovery: live's `ON CONFLICT (id) DO UPDATE`
/// preserves the original `id`; the recovery's same upsert is
/// a no-op for an equal-`issued_at` re-run and a refresh for a
/// strictly-newer one.
/// - Recovery → live handler: the recovery wrote a row with the
/// deterministic `id`; the live upsert (which preserves `id` and
/// only updates other fields when `issued_at` is strictly newer)
/// is a no-op for an equal-`issued_at` re-run and a refresh for
/// a strictly-newer one.
#[allow(dead_code)] // round-trip test in db/mod.rs pins the upsert contract; the live path and the drain use issue_ref_certificate_with_issued_at
/// Builds a canonical JSON payload, signs it with the node's Ed25519 key,
/// persists the certificate, and returns it.
pub async fn issue_ref_certificate(
state: &AppState,
repo_id: &str,
@@ -120,106 +22,26 @@ pub async fn issue_ref_certificate(
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id: &str,
) -> Result<RefCertificate> {
issue_ref_certificate_with_issued_at(
state, repo_id, ref_name, old_sha, new_sha, pusher_did, cert_id, None,
)
.await
}
/// #26 Split PR 1 round 4 — variant that lets the caller stamp the
/// cert's `issued_at` with a transition-time timestamp instead of
/// `Utc::now()`. The recovery drain passes the persisted
/// `row.created_at` so a replay after a later live cert does not
/// outrank the live cert in the `EXCLUDED.issued_at >
/// ref_certificates.issued_at` upsert guard.
///
/// The live handler uses the default `issue_ref_certificate` (no
/// override), which keeps `Utc::now()` — the reviewer's invariant
/// is that `issued_at` reflects the transition time, and for a
/// live push the transition time and the wall-clock are the same.
///
/// `issued_at_override` is honored verbatim; passing a value not in
/// RFC 3339 form is a logic bug (the upsert will mis-order), so
/// callers must use the row's persisted `created_at`.
///
/// # Clippy allow — too many arguments
/// This is the explicit "stamp a transition-time `issued_at`"
/// variant of `issue_ref_certificate`. The drain
/// (`durable_outbox::derive_one`) is the in-crate caller; the
/// test `replay_of_stale_row_does_not_overwrite_live_cert_b` pins
/// the contract that a recovery replay's `issued_at` does NOT
/// outrank a later live cert. Adding a struct-arg would be a
/// larger refactor for two callers (live + drain) and obscure the
/// parallel to `issue_ref_certificate` (which is `#[allow]`'d for
/// the same reason historically).
#[allow(clippy::too_many_arguments)]
pub async fn issue_ref_certificate_with_issued_at(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id: &str,
issued_at_override: Option<String>,
) -> Result<RefCertificate> {
let cert = build_ref_certificate(
state,
repo_id,
ref_name,
old_sha,
new_sha,
pusher_did,
Some(cert_id.to_string()),
issued_at_override,
)
.await?;
state.db.insert_ref_certificate(&cert).await
}
/// Shared cert construction: build the JSON payload, sign it with the
/// node key, and assemble the `RefCertificate` row. `cert_id_override`
/// lets the recovery path plug in a deterministic id; the live path
/// passes `None` and gets a fresh UUID. `issued_at_override` lets
/// the recovery path stamp the cert with the original transition
/// time so the upsert's `issued_at > issued_at` guard correctly
/// orders transitions regardless of write order.
#[allow(clippy::too_many_arguments)]
async fn build_ref_certificate(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id_override: Option<String>,
issued_at_override: Option<String>,
) -> Result<RefCertificate> {
let node_did = state.node_did.to_string();
// P1 (reviewer-1 round 4): when the caller passes a transition-
// time `issued_at` (the recovery drain passes `row.created_at`),
// use it verbatim so the upsert's per-column guard
// `EXCLUDED.issued_at > ref_certificates.issued_at` correctly
// orders transitions regardless of write order. The live handler
// passes `None` and gets `Utc::now()` — for a live push the
// transition time and the wall-clock are the same.
let issued_at = issued_at_override.unwrap_or_else(|| Utc::now().to_rfc3339());
let issued_at = Utc::now().to_rfc3339();
// Build the canonical signing payload via the shared builder so
// the frozen-vector test below and the live signer cannot drift
// apart — a regression in either side fails both.
let payload = v1_signing_payload(
repo_id, ref_name, old_sha, new_sha, pusher_did, &node_did, &issued_at,
);
// Build the canonical signing payload.
let payload = serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
});
let payload_bytes = serde_json::to_vec(&payload)?;
let signature = state.node_keypair.sign_b64(&payload_bytes);
let id = cert_id_override.unwrap_or_else(|| Uuid::new_v4().to_string());
let cert = RefCertificate {
id,
id: Uuid::new_v4().to_string(),
repo_id: repo_id.to_string(),
ref_name: ref_name.to_string(),
old_sha: old_sha.to_string(),
@@ -228,353 +50,9 @@ async fn build_ref_certificate(
node_did,
signature,
issued_at,
// #26 Split PR 3: the wire-format version. New certs stamp
// v1 — the pre-versioning 7-field payload with no `version`
// key in the signed JSON — because the shipped `gl`
// verifier only supports v1. Stamping v2 here would issue
// certificates the shipped client refuses (round-3
// finding: node pinned `version == 2` while gl pinned
// "2 is refused", with nothing exercising the
// composition). v2 is reserved for the future shape
// defined by `v2_signing_payload`, which carries the
// version INSIDE the signed bytes; land the v2 verify
// path before stamping v2.
version: 1,
};
// Persist and return the row as it exists in the database (on a
// conflict the existing row survives when it is newer).
state.db.insert_ref_certificate(&cert).await
}
#[cfg(test)]
mod v1_payload_tests {
//! #26 Split PR 3 — cert payload version compat.
//!
//! The v1 cert payload is the frozen canonical byte form the
//! node signs. gl's `verify_signature` reconstructs the same
//! payload and verifies. Any drift in field order, key set, or
//! whitespace breaks every existing cert. The tests here pin
//! the v1 payload shape, the round-trip, the cert id
//! determinism per the v1 version, and the v2 read forward
//! compat (an unknown version refuses to verify rather than
//! guessing).
//!
//! Reviewer 1 finding: the live signer (`issue_ref_certificate`)
//! and the frozen-vector test (`v1_payload_matches_frozen_canonical_form`)
//! previously held two separate `serde_json::json!` literals,
//! so a regression in the signer could not fail this test.
//! Both now go through the shared `v1_signing_payload` builder.
use super::{v1_signing_payload, v2_signing_payload};
use crate::db::RefCertificate;
use gitlawb_core::identity::Keypair;
/// v1 payload is the same shape the legacy code signed, byte
/// for byte. Reverting this assertion to the v2 shape (which
/// includes a `version` key) is what the versioned format
/// forbids: an old client reading a v1 cert must not see a
/// `version` key in the signed JSON.
///
/// The literal is built via the shared `v1_signing_payload`
/// helper that `issue_ref_certificate` also uses, so the live
/// signer and the frozen vector share one source of truth.
#[test]
fn v1_payload_matches_frozen_canonical_form() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let frozen = format!(
r#"{{"new":"{}","node":"did:key:z6MkNode","old":"{}","pusher":"did:key:z6MkPusher","ref":"refs/heads/main","repo_id":"repo-1","ts":"2026-07-22T00:00:00+00:00"}}"#,
"a".repeat(40),
"0".repeat(40),
);
assert_eq!(serde_json::to_string(&payload).unwrap(), frozen);
}
/// Round-trip: sign the v1 payload with a keypair, build a
/// RefCertificate with version: 1, and verify the structure.
/// This is the construction the live code does on every push.
///
/// Uses the shared builder for both halves so a regression in
/// the live signer (adding/removing/renaming a field) cannot
/// pass this test by accident.
#[test]
fn v1_ref_certificate_structure_is_well_formed() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
);
let payload_bytes = serde_json::to_vec(&payload).unwrap();
let sig = kp.sign_b64(&payload_bytes);
let cert = RefCertificate {
id: "cert-id".into(),
repo_id: "repo-1".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "a".repeat(40),
pusher_did: "did:key:z6MkPusher".into(),
node_did: node_did.clone(),
signature: sig.clone(),
issued_at: "2026-07-22T00:00:00+00:00".into(),
version: 1,
};
assert_eq!(cert.version, 1, "new certs carry version: 1");
// The signed payload reconstructs identically: gl's
// verify_signature would build the same JSON, hash the
// same bytes, and verify the same signature.
let reconstructed = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let reconstructed_bytes = serde_json::to_vec(&reconstructed).unwrap();
assert_eq!(
reconstructed_bytes, payload_bytes,
"the round-trip serialization must be byte-identical; \
this is what makes gl's verify_signature succeed"
);
// Round 3 (P3 reviewer): byte-equality alone never calls
// `identity::verify`, so a bad signature helper would pass.
// Verify the computed signature against the reconstructed
// bytes — the same check gl performs.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &reconstructed_bytes, &sig_bytes)
.expect("well-formed v1 cert signature must verify");
}
/// Reviewer 1 finding: the live signer must share the
/// canonical-form literal so a regression in either side
/// fails both. This test pins the shared builder's output
/// against the literal a future v2 byte stream would have
/// to break — if someone adds a `version` key to
/// `v1_signing_payload`, every existing cert breaks verify,
/// and this test is one of the canaries that fires.
#[test]
fn v1_signing_payload_has_no_version_key() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let obj = payload.as_object().expect("payload is a JSON object");
assert!(
!obj.contains_key("version"),
"v1 signing payload must not carry a `version` key — adding \
one changes the bytes and breaks every existing cert's \
signature. v2 certs build a different payload, not this one"
);
// Pin the exact set so a future field addition is caught.
let expected: std::collections::BTreeSet<&str> =
["new", "node", "old", "pusher", "ref", "repo_id", "ts"]
.into_iter()
.collect();
let actual: std::collections::BTreeSet<&str> = obj.keys().map(String::as_str).collect();
assert_eq!(actual, expected, "v1 payload key set is frozen");
}
/// Round 3 (P2 reviewer): from v2 onward the version belongs
/// INSIDE the signed bytes. v1 stays the unversioned legacy
/// shape; v2 carries `"version": 2` in the payload so a
/// version flip without re-signing invalidates the signature
/// instead of verifying cleanly under the other path.
#[test]
fn v2_signing_payload_binds_version_and_resists_downgrade() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let ts = "2026-07-22T00:00:00+00:00";
let v1 = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2 = v2_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2_obj = v2.as_object().expect("v2 payload is a JSON object");
assert_eq!(
v2_obj.get("version"),
Some(&serde_json::json!(2)),
"v2 payload must carry the version inside the signed bytes"
);
let v1_bytes = serde_json::to_vec(&v1).unwrap();
let v2_bytes = serde_json::to_vec(&v2).unwrap();
assert_ne!(
v1_bytes, v2_bytes,
"v1 and v2 payloads must differ, otherwise a version flip is cryptographically invisible"
);
// A signature over v1 must NOT verify as v2 and vice versa.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_v1 = kp.sign_b64(&v1_bytes);
let sig_v1_bytes: [u8; 64] = URL_SAFE_NO_PAD.decode(&sig_v1).unwrap().try_into().unwrap();
assert!(
gitlawb_core::identity::verify(&vk, &v1_bytes, &sig_v1_bytes).is_ok(),
"v1 signature must verify under v1 bytes"
);
assert!(
gitlawb_core::identity::verify(&vk, &v2_bytes, &sig_v1_bytes).is_err(),
"a v1 signature presented as v2 (downgrade/upgrade flip) must not verify"
);
}
/// The live issuer stamps the version the shipped client can
/// verify. The gl client refuses to verify v2 certs explicitly;
/// this test pins that the version the ISSUING PATH claims
/// agrees with the payload shape it actually signs.
///
/// Round 2: the prior form built a `RefCertificate` literal with
/// `version: 1` and asserted `version == 1` — true no matter what the
/// signer does, so flipping the issuer's version (or its payload shape)
/// left it green. This one drives the actual
/// `issue_ref_certificate` so a regression in the issuer flips the test.
/// The DB row is created against `test_support::test_state`, which
/// gives us a real `AppState` with a real node keypair.
///
/// Round 3: the issuer stamps v1 because the shipped `gl`
/// verifier only supports v1 (stamping v2 shipped certs the
/// client refuses). The binding to the live issuer is kept —
/// flipping the stamp still turns this test red — without
/// changing what production issues.
#[sqlx::test]
#[allow(clippy::async_yields_async)]
async fn issuer_stamps_v1_over_v1_payload(pool: sqlx::PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let repo_id = "repo-issuer-observe";
let ref_name = "refs/heads/main";
let old = "0".repeat(40);
let new = "a".repeat(40);
let pusher = "did:key:z6MkPusher";
let cert_id = "cert-issuer-observe";
// Drive the live issuer and read the persisted row back by
// id, so the test covers the stored `version` column and
// not just the in-process return value.
let cert = crate::cert::issue_ref_certificate(
&state, repo_id, ref_name, &old, &new, pusher, cert_id,
)
.await
.expect("issue_ref_certificate must succeed");
// The cert returned by the issuer is the source of truth
// for both the version claim and the signed bytes.
assert_eq!(cert.repo_id, repo_id);
assert_eq!(cert.ref_name, ref_name);
// Cross-check 1: the issuer's claimed version must match
// what the live function stamps. This assertion is bound
// to the live function, not a hand-built literal:
// flipping the stamp in `issue_ref_certificate` breaks it
// through the actual call path.
assert_eq!(
cert.version, 1,
"the live issuer must stamp v1 until a v2 verifier ships; \
flipping the stamp breaks this assertion through the actual call path"
);
// Cross-check 1b: the persisted row carries the same stamp.
let stored = state
.db
.get_ref_certificate(&cert.id)
.await
.expect("persisted cert must be readable")
.expect("persisted cert must exist");
assert_eq!(
stored.version, cert.version,
"the stored version column must match the issued claim"
);
// Cross-check 2: the signature on the cert must verify
// against the v1 payload builder's bytes, not against some
// other shape the issuer might have introduced. This is
// what gl's `verify_signature` does for a v1 cert: rebuild
// the payload from the cert's own fields and verify the
// signature. If the issuer ever signs a different shape
// while still claiming v1, every shipped client breaks —
// and so does this.
let rebuilt = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let vk = cert
.node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &serde_json::to_vec(&rebuilt).unwrap(), &sig_bytes)
.expect("a v1 cert's signature must verify against the v1 payload shape");
// Cross-check 3: the v1 signed payload must NOT contain a
// version key — v2 is the future shape that adds one inside
// the signed bytes (see `v2_signing_payload`). The v1
// payload (which the cert is signed over) is frozen at 7
// fields. A change that adds `version` to the v1 payload
// breaks every existing cert and this test.
let obj = rebuilt
.as_object()
.expect("rebuilt v1 payload is a JSON object");
assert!(
!obj.contains_key("version"),
"the v1 signing payload must not carry a version key — \
adding one changes the bytes and breaks every existing \
cert's signature"
);
}
}
+8 -480
View File
@@ -55,10 +55,7 @@ pub struct Config {
#[arg(long, env = "GITLAWB_KEY", default_value = "~/.gitlawb/identity.pem")]
pub key_path: String,
/// Reserved and currently inert: read access is enforced per repository
/// through `is_public` and path-scoped visibility rules, not this flag.
/// Setting it to false changes nothing; make a repo private through its
/// own visibility instead.
/// Reserved for private-read mode; per-repo read enforcement is not wired in alpha
#[arg(long, env = "GITLAWB_PUBLIC_READ", default_value_t = true)]
pub public_read: bool,
@@ -66,12 +63,6 @@ pub struct Config {
#[arg(long, env = "GITLAWB_PUBLIC_URL")]
pub public_url: Option<String>,
/// Base URL for the web view of repos on this node (e.g. https://gitlawb.com).
/// When set, `GET /` advertises it as `web_url` so the CLI can print a
/// working `View:` link. Omit for nodes that have no web front-end.
#[arg(long, env = "GITLAWB_WEB_URL")]
pub web_url: Option<String>,
/// Comma-separated list of bootstrap peer URLs to announce to on startup
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>,
@@ -152,57 +143,6 @@ pub struct Config {
#[arg(long, env = "GITLAWB_AUTO_SYNC", default_value_t = false)]
pub auto_sync: bool,
/// Periodically fetch configured INBOUND repositories from their external
/// HTTPS upstream. Disabled by default: enabling this also requires
/// `GITLAWB_ENFORCE_OWNER_PUSH=true`, so an upstream-authoritative mirror
/// cannot accept writes from arbitrary signed identities between fetches.
#[arg(long, env = "GITLAWB_UPSTREAM_MIRROR_ENABLED", default_value_t = false)]
pub upstream_mirror_enabled: bool,
/// Pause between complete upstream-mirror scans. A scan is sequential and
/// paginated; the pause begins only after the prior scan has finished, so a
/// slow forge cannot create overlapping worker cycles.
#[arg(
long,
env = "GITLAWB_UPSTREAM_MIRROR_INTERVAL_SECS",
default_value_t = 300,
value_parser = clap::value_parser!(u64).range(5..=86_400)
)]
pub upstream_mirror_interval_secs: u64,
/// Maximum wall-clock time for one external `git fetch`. The child runs in
/// its own process group and is fully reaped on timeout. The preflight
/// URL-rewrite and local-config probes run on their own short bounds and
/// do not consume this budget.
#[arg(
long,
env = "GITLAWB_UPSTREAM_MIRROR_FETCH_TIMEOUT_SECS",
default_value_t = 600,
value_parser = clap::value_parser!(u64).range(1..=GIT_SERVICE_TIMEOUT_SECS_MAX)
)]
pub upstream_mirror_fetch_timeout_secs: u64,
/// Number of durable INBOUND rows read per database page. Every page is
/// processed before the scan sleeps; this bounds query memory without
/// permanently starving repositories beyond the first page.
#[arg(
long,
env = "GITLAWB_UPSTREAM_MIRROR_PAGE_SIZE",
default_value_t = 100,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_000)
)]
pub upstream_mirror_page_size: usize,
/// Enable the periodic reconciliation sweep that re-derives pin/seal sets
/// and fills durability gaps. Defaults to true; set to false to disable
/// the sweep even when a pin backend (IPFS/Pinata) is configured.
#[arg(
long,
env = "GITLAWB_RECONCILIATION_SWEEP",
default_value_t = true,
action = clap::ArgAction::Set
)]
pub reconciliation_sweep: bool,
/// Irys URL for Arweave permanent anchoring.
/// Leave empty to disable. Use https://devnet.irys.xyz for free devnet.
#[arg(long, env = "GITLAWB_IRYS_URL", default_value = "")]
@@ -284,29 +224,6 @@ pub struct Config {
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
pub shutdown_grace_secs: u64,
/// Maximum number of creation requests (repo create, issue create, PR
/// create, register, fork) allowed per DID within the rate-limit window.
/// 0 is rejected at parse time: it would silently disable the limiter
/// (RateLimiter treats max_requests == 0 as allow-all), not block every
/// creation request.
#[arg(
long,
env = "GITLAWB_RATE_LIMIT_MAX",
default_value_t = 10,
value_parser = clap::value_parser!(u32).range(1..)
)]
pub rate_limit_max_requests: u32,
/// Rate-limit window size, in seconds, over which
/// `rate_limit_max_requests` applies. 0 is rejected at parse time: every
/// timestamp would prune immediately, silently disabling the limiter.
#[arg(
long,
env = "GITLAWB_RATE_LIMIT_WINDOW_SECS",
default_value_t = 3600,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub rate_limit_window_secs: u64,
/// Maximum wall-clock time a single served git operation (upload-pack /
/// receive-pack through `run_git_service`) may run before it is aborted and
/// its process group torn down, in seconds. Bounds a git that neither
@@ -836,60 +753,6 @@ pub struct Config {
value_parser = clap::builder::RangedU64ValueParser::<u64>::new().range(0..=86_400)
)]
pub pin_repair_sweep_delay_secs: u64,
/// #26 Split PR 1 step 4 — receive-pack queue retention window.
/// Terminal `complete` and `rejected_at_git` rows older than
/// this are eligible for the periodic purge. `quarantined`
/// rows are never purged on a timer. The v30 partial index
/// `idx_receive_pack_requests_completed_at` keeps the scan
/// cheap regardless of the value.
#[arg(
long,
env = "GITLAWB_QUEUE_RETENTION_DAYS",
default_value_t = 7,
value_parser = clap::builder::RangedU64ValueParser::<i64>::new().range(1..=365)
)]
pub queue_retention_days: i64,
/// #26 Split PR 1 step 4 — receive-pack queue purge batch size.
/// Each periodic purge pass deletes at most this many terminal
/// rows per batch. The drain and the purge share the same
/// `DRAIN_PER_PASS_LIMIT` budget; see the spawn function in
/// `main.rs` for the wiring.
#[arg(
long,
env = "GITLAWB_QUEUE_PURGE_BATCH",
default_value_t = 1000,
value_parser = clap::builder::RangedU64ValueParser::<i64>::new().range(1..=100_000)
)]
pub queue_purge_batch: i64,
/// #26 Split PR 1 step 5 — effects-executor retry bound. The
/// drain flips a request to `quarantined` after this many
/// `EffectsOutcome::Retry` returns, closing the
/// infinite-retry DoS window. Default 8 matches the spec; set
/// lower in tests.
#[arg(
long,
env = "GITLAWB_EFFECTS_MAX_ATTEMPTS",
default_value_t = 8,
value_parser = clap::builder::RangedU64ValueParser::<i32>::new().range(1..=1000)
)]
pub effects_max_attempts: i32,
/// Per-client-IP rate limit for the anonymous task read routes
/// (`GET /api/v1/tasks`, `GET /api/v1/tasks/{id}`), in requests per hour.
/// Both are publicly reachable (`optional_signature`). `GET /api/v1/tasks`
/// runs `collect_visible_tasks` and returns a visibility-filtered page;
/// `GET /api/v1/tasks/{id}` runs `get_visible_task` and returns an opaque 404
/// when the task is hidden or missing. These reads can require task,
/// repository, and visibility-rule queries even when no task is returned,
/// so the brake bounds the cost of anonymous probes. Keyed on the resolved
/// client IP via `GITLAWB_TRUSTED_PROXY`. `0` disables. Default: 1200 (a list page
/// followed by per-task reads is a normal client pattern, so this sits above
/// the `/ipfs` budget). GraphQL `tasks` / `task` queries, including WebSocket
/// operations, share this per-IP budget with REST task reads.
#[arg(long, env = "GITLAWB_TASK_READ_RATE_LIMIT", default_value_t = 1200)]
pub task_read_rate_limit: usize,
}
impl Config {
@@ -937,13 +800,7 @@ impl Config {
/// Cross-field boot validation. Single-field ranges are enforced by clap; this
/// catches combinations that ship a denial-of-service under otherwise-valid
/// values. Call once at startup and fail fast on `Err`.
///
/// `&mut self` is required so the validator can normalize `web_url` to
/// its canonical form after validation: the configured value is
/// advertised on public `GET /`, and any consumer reading
/// `config.web_url` later must see the same string that was actually
/// validated.
pub fn validate(&mut self) -> Result<(), String> {
pub fn validate(&self) -> Result<(), String> {
// Concurrent git writes pin the dedicated lock pool for their whole
// duration (connection-affine advisory lock in acquire_write). The main
// pool no longer carries that occupancy.
@@ -960,13 +817,6 @@ impl Config {
Self::DB_LOCK_POOL_NON_PUSH_HEADROOM
));
}
if self.upstream_mirror_enabled && !self.enforce_owner_push {
return Err(
"GITLAWB_UPSTREAM_MIRROR_ENABLED requires GITLAWB_ENFORCE_OWNER_PUSH=true: \
an upstream-authoritative mirror must not accept writes from arbitrary signed identities"
.to_string(),
);
}
let main_floor = Self::DB_POOL_APP_HEADROOM as u64;
if (self.db_max_connections as u64) < main_floor {
return Err(format!(
@@ -1067,150 +917,14 @@ impl Config {
.map_err(|e| e.to_string())?;
}
// GITLAWB_WEB_URL is advertised in GET / so the CLI can print a working
// View: link. Clap maps "" to Some("") which would produce a broken URL;
// reject early instead of serving a malformed link. Non-blank values must
// additionally parse as an absolute http(s) URL — the CLI appends
// `/{owner}/{repo}` to it, so scheme-less hosts ("gitlawb.com") and other
// garbage would render links no browser can follow.
if let Some(raw) = &self.web_url {
match validate_web_url(raw) {
Ok(()) => {}
Err(reason) => {
return Err(format!(
"GITLAWB_WEB_URL {reason} — \
set it to an absolute URL like https://gitlawb.com or leave it unset."
));
}
}
// Normalize to one canonical form so every consumer (the regular
// and degraded `GET /` handlers, anyone who reads
// `config.web_url` directly) sees the same string that was
// actually validated. The CLI's own defensive trim would paper
// over the divergence on the `View:` line, but that hides the
// bug from any other consumer that parses the value verbatim.
let canonical = raw
.trim()
.parse::<url::Url>()
.map(|u| u.to_string())
.map_err(|_| "is not a valid absolute URL".to_string())?;
if canonical != *raw {
self.web_url = Some(canonical);
}
}
Ok(())
}
}
/// Validate a `web_url` value for use as a browser-reachable base URL.
/// Blank values are rejected (clap maps `--web-url ""` to `Some("")`), and
/// non-blank values must parse as absolute `http`/`https` URLs with no
/// query, fragment, or userinfo — the CLI treats this as a string prefix
/// to append paths to, so anything else produces links no browser can
/// follow (`?a=1/owner/repo` puts the repo path inside the query string);
/// and userinfo is rejected outright because the configured value is
/// published on public `GET /` and reprinted on the CLI's `View:` line,
/// so an accidentally configured password would land in public metadata,
/// terminal scrollback, and CI logs.
pub(crate) fn validate_web_url(raw: &str) -> Result<(), String> {
if raw.trim().is_empty() {
return Err("must not be empty or whitespace-only".into());
}
let parsed: url::Url = raw
.trim()
.parse()
.map_err(|_| "is not a valid absolute URL".to_string())?;
match parsed.scheme() {
"http" | "https" => {}
other => return Err(format!("must use http or https, got '{other}:'")),
}
// url::Url cannot represent a URL without a host for http/https schemes, so
// reaching here guarantees an absolute browser-usable base.
if parsed.query().is_some() {
return Err(
"must not contain a query string — it is used as a path prefix for View links".into(),
);
}
if parsed.fragment().is_some() {
return Err(
"must not contain a fragment — it is used as a path prefix for View links".into(),
);
}
if !parsed.username().is_empty() || parsed.password().is_some() {
return Err(
"must not contain a username or password — it is advertised publicly and printed on the View: line"
.into(),
);
}
// Bidi/control bytes can hide behind percent-encoding: a literal scan of
// the wire form misses `.../%E2%80%AE` even though a browser decodes it
// into a reordering character on the rendered View: link. Check the
// decoded host and path before the value is advertised on GET / (INV-6).
let host = parsed.host_str().unwrap_or("");
let decoded_path = percent_encoding::percent_decode_str(parsed.path()).decode_utf8_lossy();
if host
.chars()
.chain(decoded_path.chars())
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c))
{
return Err(
"must not contain control or bidi characters, even percent-encoded — the View: line is terminal output"
.into(),
);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn upstream_mirror_worker_is_default_off_and_bounded() {
let mut config = Config::parse_from(["gitlawb-node"]);
assert!(!config.upstream_mirror_enabled);
assert_eq!(config.upstream_mirror_interval_secs, 300);
assert_eq!(config.upstream_mirror_fetch_timeout_secs, 600);
assert_eq!(config.upstream_mirror_page_size, 100);
config.validate().unwrap();
assert!(
Config::try_parse_from(["gitlawb-node", "--upstream-mirror-interval-secs", "4",])
.is_err()
);
assert!(
Config::try_parse_from(["gitlawb-node", "--upstream-mirror-page-size", "0",]).is_err()
);
}
#[test]
fn upstream_mirror_worker_requires_owner_push_enforcement() {
// enforce_owner_push defaults true on main, so the only unsafe shape
// is the explicit opt-out: mirror enabled plus enforcement refused.
let mut unsafe_config = Config::parse_from([
"gitlawb-node",
"--upstream-mirror-enabled",
"--enforce-owner-push",
"false",
]);
let error = unsafe_config.validate().unwrap_err();
assert!(error.contains("GITLAWB_ENFORCE_OWNER_PUSH=true"));
Config::parse_from([
"gitlawb-node",
"--upstream-mirror-enabled",
"--enforce-owner-push",
"true",
])
.validate()
.unwrap();
// The default posture alone also validates: enforcement is on.
Config::parse_from(["gitlawb-node", "--upstream-mirror-enabled"])
.validate()
.unwrap();
}
#[test]
fn lock_pool_size_defaults_to_40_and_rejects_zero() {
assert_eq!(
@@ -1439,30 +1153,6 @@ mod tests {
);
}
#[test]
fn queue_lifecycle_knobs_default_conservatively() {
let c = Config::parse_from(["gitlawb-node"]);
// 7-day retention matches the v30 partial index comment
// and the spec at .gravirei/plans/state-model-durable-post-receive.md.
assert_eq!(c.queue_retention_days, 7);
// 1000 rows per pass matches DRAIN_PER_PASS_LIMIT in durable_outbox.
assert_eq!(c.queue_purge_batch, 1000);
assert_eq!(
Config::parse_from(["gitlawb-node", "--queue-retention-days", "30"])
.queue_retention_days,
30
);
assert!(Config::try_parse_from(["gitlawb-node", "--queue-retention-days", "0"]).is_err());
assert!(Config::try_parse_from(["gitlawb-node", "--queue-retention-days", "366"]).is_err());
assert_eq!(
Config::parse_from(["gitlawb-node", "--queue-purge-batch", "500"]).queue_purge_batch,
500
);
assert!(Config::try_parse_from(["gitlawb-node", "--queue-purge-batch", "0"]).is_err());
}
#[test]
fn ipfs_walk_per_source_defaults_and_rejects_out_of_range() {
assert_eq!(Config::parse_from(["gitlawb-node"]).ipfs_walk_per_source, 4);
@@ -1873,7 +1563,7 @@ mod tests {
.expect("default config must validate");
// An under-sized lock pool relative to the push cap is rejected.
let mut under_lock = Config::parse_from([
let under_lock = Config::parse_from([
"gitlawb-node",
"--db-lock-pool-max-connections",
"16",
@@ -1886,7 +1576,7 @@ mod tests {
);
// Exactly at the push cap without non-push headroom is rejected.
let mut push_only = Config::parse_from([
let push_only = Config::parse_from([
"gitlawb-node",
"--db-lock-pool-max-connections",
"32",
@@ -1899,7 +1589,7 @@ mod tests {
);
// Main pool can be smaller than pushes + headroom when the lock pool carries writes.
let mut split = Config::parse_from([
let split = Config::parse_from([
"gitlawb-node",
"--db-max-connections",
"16",
@@ -1914,153 +1604,13 @@ mod tests {
);
// Main pool still needs the app headroom floor.
let mut under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
let under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
assert!(
under_main.validate().is_err(),
"db_max_connections below DB_POOL_APP_HEADROOM must be rejected"
);
}
#[test]
fn web_url_rejects_empty_and_whitespace() {
// Unset is fine.
Config::parse_from(["gitlawb-node"])
.validate()
.expect("no web_url must validate");
// A real URL validates.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://gitlawb.com".into());
assert!(cfg.validate().is_ok());
// Empty string is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("".into());
assert!(cfg.validate().is_err(), "empty web_url must be rejected");
// Whitespace-only is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" ".into());
assert!(
cfg.validate().is_err(),
"whitespace-only web_url must be rejected"
);
}
#[test]
fn web_url_rejects_non_absolute_or_non_browser_urls() {
// Scheme-less host: parses nowhere, renders a broken View: link.
for bad in [
"gitlawb.com",
"not a url",
"ftp://files.example.com",
"javascript:alert(1)",
"https://", // no host
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(cfg.validate().is_err(), "web_url {bad:?} must be rejected");
}
// Query strings and fragments corrupt the appended repo path, since
// web_url is used as a raw string prefix (`?a=1` would swallow
// `/{owner}/{repo}` into the query).
for bad in [
"https://gitlawb.com?a=1",
"https://gitlawb.com/?utm_source=docs",
"https://gitlawb.com#section",
"https://gitlawb.com/#faq",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (query or fragment) must be rejected"
);
}
// Surrounding whitespace around a valid URL is tolerated.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://gitlawb.com ".into());
assert!(
cfg.validate().is_ok(),
"padded-but-valid web_url must validate"
);
// Percent-encoded bidi must not boot: the wire form carries no Cf
// code point, but a browser decodes %E2%80%AE into a reordering
// character on the rendered View: link (INV-6). Literal bidi and
// control bytes are rejected the same way.
for bad in [
"https://example.com/%E2%80%AE",
"https://example.com/\u{202e}spoof",
"https://example.com/%1b[31m",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (control or bidi, even encoded) must be rejected"
);
}
// Non-default ports and paths are fine.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("http://localhost:8080/ui".into());
assert!(cfg.validate().is_ok(), "port+path web_url must validate");
// Userinfo must be rejected: `web_url` is published on public
// `GET /` and reprinted on the CLI's `View:` line, so an
// accidentally configured password would land in public metadata,
// terminal scrollback, and CI logs. Reject username-only and
// user+password forms.
for bad in [
"https://[email protected]",
"https://user:[email protected]",
"https://:[email protected]",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (userinfo) must be rejected"
);
}
}
/// `validate()` accepts whitespace-padded input but the stored value
/// must match the canonical form actually validated, so every consumer
/// (the regular and degraded `GET /` handlers, anyone who reads
/// `config.web_url` directly) sees the same string. Without this, an
/// operator that sets `GITLAWB_WEB_URL=" https://git.example "` boots
/// a node that advertises `" https://git.example "` on public
/// metadata — different from what was validated, different from what
/// any other consumer would parse.
#[test]
fn web_url_is_normalized_to_canonical_form_after_validation() {
// Padded input is rewritten to the canonical form. `url::Url`'s
// own `to_string` is what we use for the canonicalization, and
// it always appends a path-separator slash to a bare-host URL,
// so the stored value is `https://git.example/` (which is also
// what every browser would normalize the operator-typed form to).
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://git.example ".into());
cfg.validate().unwrap();
assert_eq!(
cfg.web_url.as_deref(),
Some("https://git.example/"),
"stored web_url must be the canonical form, not the operator-typed one"
);
// Already-canonical input is left alone after a no-op
// re-normalization — same form comes back out, so any consumer
// reading the stored value sees a stable string across calls.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://git.example/".into());
cfg.validate().unwrap();
assert_eq!(cfg.web_url.as_deref(), Some("https://git.example/"));
}
/// The DECLARED default, read off the parser rather than out of a parse.
///
/// `Config::parse_from` consults the process environment, so on a host that
@@ -2084,28 +1634,6 @@ mod tests {
);
}
/// #338: GITLAWB_PUBLIC_READ is inert, so its help must not claim read
/// enforcement is missing; it must tell the operator the flag changes
/// nothing and where the real control lives.
#[test]
fn public_read_help_describes_the_flag_as_inert() {
use clap::CommandFactory;
let cmd = Config::command();
let arg = cmd
.get_arguments()
.find(|a| a.get_id() == "public_read")
.expect("the argument must exist");
let help = arg.get_help().map(|h| h.to_string()).unwrap_or_default();
assert!(
!help.contains("not wired"),
"the help still claims read enforcement is unwired: {help}"
);
assert!(
help.contains("inert"),
"the help must say the flag is inert so an operator does not rely on it: {help}"
);
}
/// The flip must not strand an operator mid-upgrade.
///
/// Turning the gate on is a breaking change for any deployment whose pushers are
@@ -2325,7 +1853,7 @@ mod tests {
std::os::unix::fs::symlink(&target, &link).unwrap();
let key_path = link.join("p2p.key");
let mut config = Config::parse_from([
let config = Config::parse_from([
"gitlawb-node",
"--p2p-port",
"0",
@@ -2420,7 +1948,7 @@ mod tests {
let key_path = build(base.path());
let before = snapshot(base.path());
let mut config = Config::parse_from([
let config = Config::parse_from([
"gitlawb-node",
"--p2p-port",
"7546",
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+8 -509
View File
@@ -6,7 +6,6 @@
use std::collections::{BTreeSet, HashMap};
use std::path::Path;
use std::str::FromStr;
use std::time::Duration;
use ed25519_dalek::VerifyingKey;
use gitlawb_core::did::Did;
@@ -70,12 +69,6 @@ fn resolve_all_recipients(dids: &BTreeSet<String>) -> Result<Vec<VerifyingKey>,
enum SealPlan {
/// An existing envelope already covers exactly this recipient set; nothing to do.
SkipUnchanged,
/// The stored tag matches, but the stored reader set contains a DID that
/// must not resolve (e.g. a small-order did:key sealed before the choke
/// point landed). The old envelope is already public and cannot be
/// withdrawn, but it must be surfaced for operator action rather than
/// silently skipped as unchanged.
SkipUnresolvableStored(Vec<String>),
/// No recipient DID resolved to a key, so there is nothing to seal to.
SkipNoRecipients,
/// At least one recipient DID is unresolvable. Fail closed: never seal to a
@@ -97,30 +90,15 @@ enum SealPlan {
/// recover the blob; reader removal is not retroactive (the old envelope is
/// already public). The comparison is on the opaque node-keyed tag, never the
/// DID list.
///
/// A matching stored tag is normally a skip, but the set is still resolved
/// before skipping: a reader that used to resolve and now must not (a
/// small-order did:key sealed before the choke point landed) is surfaced as
/// [`SealPlan::SkipUnresolvableStored`] instead of being silently treated as
/// unchanged. The old envelope is already public and cannot be withdrawn, but
/// the operator must be able to see that a persisted envelope was sealed to a
/// weak recipient.
fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&str>) -> SealPlan {
let tag = recipients_tag(node_seed, dids);
if stored_tag == Some(tag.as_str()) {
// The stored envelope already covers exactly this set, so there is
// nothing to re-seal. Resolve anyway so a now-unresolvable reader in an
// unchanged set is surfaced rather than hidden by the tag match.
match resolve_all_recipients(dids) {
Ok(_) => SealPlan::SkipUnchanged,
Err(unresolved) => SealPlan::SkipUnresolvableStored(unresolved),
}
} else {
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
}
return SealPlan::SkipUnchanged;
}
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
}
}
@@ -128,60 +106,17 @@ fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&
/// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob
/// actually sealed and recorded this call (the per-push delta), used by Option B3
/// to anchor a manifest. Recipient identities are never stored or returned.
///
/// Nine args (the fence joins the seal's eight) but grouping them would churn
/// both callers and the race/hung-git tests for no behavioral gain.
#[allow(clippy::too_many_arguments)]
pub async fn encrypt_and_pin(
ipfs_api: &str,
repo_path: &Path,
db: &Db,
repo_id: &str,
node_seed: &[u8; 32],
git_bin: &str,
batch_budget: Duration,
recipients: &HashMap<String, BTreeSet<String>>,
fence: Option<&crate::ipfs_pin::PolicyFence>,
) -> Vec<(String, String)> {
let mut sealed = Vec::new();
let mut skipped_unresolvable = 0usize;
// One shared read deadline for the whole batch, like `pin_new_objects`: a
// hung git child is watchdog-reaped at this bound, so the outer
// `PIN_PHASE_DEADLINE` timeout cannot be held open by a blocking read
// (R1-P2). Each read runs under `spawn_blocking` — it is synchronous child
// spawn + pipe drain + watchdog join.
let read_deadline = std::time::Instant::now() + batch_budget;
let total = recipients.len();
for (attempted, (oid, dids)) in recipients.iter().enumerate() {
// Batch budget gate (R2-P3), mirroring the public pin loops: an object
// is never started with a remainder too small to cover a bounded read's
// teardown. This is consistency (the seal is bounded by the outer
// `PIN_PHASE_DEADLINE` either way), but it keeps the three loops from
// drifting apart in how they report a truncated batch.
if crate::ipfs_pin::batch_budget_gate(
"encrypted-seal",
read_deadline,
sealed.len(),
total - attempted,
)
.is_none()
{
break;
}
// Policy fence (R1-P1): the recipients snapshot was derived before the
// long withheld-blob walk; if a visibility rule moved while that walk
// ran (a reader added or removed), stop sealing instead of pinning to a
// stale recipient set. Checked FIRST so a changed policy costs nothing.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
oid = %oid,
"visibility policy changed after the recipients snapshot; stopping the seal loop"
);
break;
}
}
for (oid, dids) in recipients {
// A DB read failure is not a cache miss: re-sealing here would do an
// avoidable IPFS write during a partial outage. Skip and retry next push.
let stored_tag = match db.encrypted_blob_recipients_tag(repo_id, oid).await {
@@ -196,21 +131,6 @@ pub async fn encrypt_and_pin(
// covered (which would permanently lock out the dropped readers, #47).
let (keys, tag) = match plan_seal(node_seed, dids, stored_tag.as_deref()) {
SealPlan::SkipUnchanged => continue,
SealPlan::SkipUnresolvableStored(unresolved) => {
// A stored envelope already covers this exact reader set, and
// the set now contains a DID that must not resolve (a
// small-order did:key sealed before the choke point landed).
// The old envelope is already public and cannot be withdrawn
// or re-sealed; surface it so the operator can act.
let sample: Vec<&String> = unresolved.iter().take(3).collect();
tracing::warn!(
oid = %oid,
unresolved_count = unresolved.len(),
unresolved_sample = ?sample,
"stored envelope was sealed to a recipient that must not resolve; operator action may be required"
);
continue;
}
SealPlan::SkipNoRecipients => {
tracing::warn!(oid = %oid, "no resolvable recipient keys; skipping encrypted pin");
continue;
@@ -232,9 +152,7 @@ pub async fn encrypt_and_pin(
}
SealPlan::Seal { keys, tag } => (keys, tag),
};
let data = match read_object_bounded_spawn_blocking(git_bin, repo_path, oid, read_deadline)
.await
{
let data = match crate::git::store::read_object(repo_path, oid) {
Ok(Some((_t, bytes))) => bytes,
Ok(None) => {
tracing::warn!(oid = %oid, "git object not found; skipping encrypted pin");
@@ -252,22 +170,6 @@ pub async fn encrypt_and_pin(
continue;
}
};
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the tag lookup, recipient resolution, git read, or seal — all
// of which can take seconds. Without this, a reader removed during
// preparation can still receive a newly published envelope.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
oid = %oid,
"visibility policy changed during encrypted seal preparation; aborting upload"
);
break;
}
}
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
@@ -299,76 +201,10 @@ pub async fn encrypt_and_pin(
sealed
}
/// Bounded, reaped git object read for the seal loop, run off the async thread:
/// `read_object_bounded` is synchronous child spawn + pipe drain + watchdog
/// join, so blocking the runtime task on it would let a hung git hold a worker
/// thread (R1-P2). The `deadline` is the batch's shared read deadline; a child
/// still alive at it is SIGTERM/SIGKILL group-reaped by the watchdog.
async fn read_object_bounded_spawn_blocking(
git_bin: &str,
repo_path: &Path,
sha256_hex: &str,
deadline: std::time::Instant,
) -> anyhow::Result<Option<(String, Vec<u8>)>> {
let git_bin = git_bin.to_string();
let repo_path = repo_path.to_path_buf();
let sha256_hex = sha256_hex.to_string();
tokio::task::spawn_blocking(move || {
crate::git::store::read_object_bounded(&git_bin, &repo_path, &sha256_hex, deadline)
.map_err(anyhow::Error::from)
})
.await
.map_err(|e| anyhow::anyhow!("read_object spawn_blocking join failed: {e}"))?
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use std::time::Duration;
/// A did:key encoding a small-order point. Well-formed, indistinguishable
/// from a real one by eye, and refused at resolution since the choke-point
/// guard landed.
fn weak_did_key() -> String {
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let vk = ed25519_dalek::VerifyingKey::from_bytes(&weak).expect("decompresses");
Did::from_verifying_key(&vk).to_string()
}
/// The #47 fail-closed arm already refuses to seal to a partial recipient
/// set. Because a small-order did:key no longer RESOLVES, it lands in the
/// unresolved set and that existing arm catches it: nothing is sealed, and
/// no new code path was added to make that happen.
#[test]
fn weak_recipient_falls_into_the_existing_fail_closed_arm() {
let weak = weak_did_key();
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(weak.clone());
match plan_seal(&[0u8; 32], &dids, None) {
SealPlan::SkipUnresolvable(unresolved) => assert!(
unresolved.contains(&weak),
"the weak DID must be named in the unresolved set, got {unresolved:?}"
),
other => panic!("expected SkipUnresolvable, got {other:?}"),
}
}
/// Must-not-over-reject control: an all-legitimate recipient set still seals.
#[test]
fn legitimate_recipient_set_still_seals() {
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(did_key(11));
match plan_seal(&[0u8; 32], &dids, None) {
SealPlan::Seal { keys, .. } => assert_eq!(keys.len(), 2, "both recipients resolve"),
other => panic!("expected Seal, got {other:?}"),
}
}
fn did_key(seed: u8) -> String {
let vk = SigningKey::from_bytes(&[seed; 32]).verifying_key();
@@ -512,43 +348,6 @@ mod tests {
));
}
/// P1 remediation (review): an unchanged reader set is normally a skip, but
/// the skip must not hide a recipient that now fails to resolve. A blob
/// sealed to a small-order did:key before the choke-point guard landed keeps
/// its old CID and stays decryptable-without-a-private-key unless the
/// matching-tag path surfaces it. This pins that the stored-tag match still
/// resolves the set instead of short-circuiting.
#[test]
fn plan_seal_surfaces_unresolvable_recipient_in_unchanged_reader_set() {
let weak = weak_did_key();
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(weak.clone());
let stored = recipients_tag(&SEED, &dids);
match plan_seal(&SEED, &dids, Some(&stored)) {
SealPlan::SkipUnresolvableStored(unresolved) => assert!(
unresolved.contains(&weak),
"the weak DID must be named in the unresolved set, got {unresolved:?}"
),
other => panic!("expected SkipUnresolvableStored, got {other:?}"),
}
}
/// Degenerate control: an empty set with a matching stored tag stays a
/// silent skip. The state is unreachable in practice (a seal is never
/// recorded for an empty reader set), so this pins only that the
/// resolve-on-match path must not turn it into a re-seal or a warning.
#[test]
fn plan_seal_unchanged_empty_set_still_skips() {
let empty = BTreeSet::new();
let empty_tag = recipients_tag(&SEED, &empty);
assert!(matches!(
plan_seal(&SEED, &empty, Some(&empty_tag)),
SealPlan::SkipUnchanged
));
}
#[test]
fn plan_seal_reseals_when_recipient_set_changed() {
// A stored tag for a DIFFERENT set is a miss: a newly added reader must
@@ -560,304 +359,4 @@ mod tests {
other => panic!("changed recipient set must re-seal; got {other:?}"),
}
}
/// A reader removed mid-seal must stop the seal loop (R1-P1 "race test for
/// reader removal"): `encrypt_and_pin` re-checks the policy fence before
/// each blob, so a `remove_visibility_rule` landing while the first seal is
/// in flight aborts before a later blob is pinned to a stale recipient set.
#[sqlx::test]
async fn encrypt_and_pin_stops_sealing_when_reader_removed_mid_batch(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("seal-race.git");
// Three loose blobs, each withheld (path-scoped deny exists so the sweep
// would have derived recipients for them).
let oids: Vec<String> = {
crate::git::store::init_bare(&repo_path).expect("init bare repo");
(0..3)
.map(|i| {
let mut cmd = std::process::Command::new("git");
cmd.args(["hash-object", "-w", "--stdin"])
.current_dir(&repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped());
let mut child = cmd.spawn().expect("spawn git hash-object");
{
use std::io::Write;
child
.stdin
.as_mut()
.expect("stdin")
.write_all(format!("secret blob {i}\n").as_bytes())
.expect("write stdin");
}
let out = child.wait_with_output().expect("hash-object output");
assert!(out.status.success());
String::from_utf8_lossy(&out.stdout).trim().to_string()
})
.collect()
};
// A real repos row so the fence has an epoch and a reader can be removed.
let now = chrono::Utc::now();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.clone(),
name: "seal-race-repo".into(),
owner_did: "did:key:zSealRaceOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.expect("create repo");
// A rule whose removal is the "reader removed" mutation: one reader per
// blob, all under the same path glob.
let reader = did_key(1);
db.set_visibility_rule(
&repo_id,
"**/secret/*",
crate::db::VisibilityMode::B,
std::slice::from_ref(&reader),
"did:key:zSealRaceOwner",
)
.await
.expect("set rule");
// IPFS endpoint that delays the FIRST add 2s so the removal lands while
// that seal is in flight, then answers immediately.
let endpoint = delaying_cid_endpoint(vec![Duration::from_secs(2)]).await;
let recipients: HashMap<String, BTreeSet<String>> = oids
.iter()
.cloned()
.map(|oid| {
let mut s = BTreeSet::new();
s.insert(reader.clone());
(oid, s)
})
.collect();
let fence = crate::ipfs_pin::PolicyFence::capture(&db, &repo_id)
.await
.expect("fence captures");
let sealed = tokio::time::timeout(Duration::from_secs(30), async {
let seal_db = db.clone();
let seal_repo = repo_path.clone();
let seal_endpoint = endpoint.clone();
let seal_repo_id = repo_id.clone();
let handle = tokio::spawn(async move {
encrypt_and_pin(
&seal_endpoint,
&seal_repo,
&seal_db,
&seal_repo_id,
&SEED,
"git",
Duration::from_secs(60),
&recipients,
Some(&fence),
)
.await
});
// Let the first add start (endpoint sleeps 2s), then remove the
// reader so the fence is stale before the loop checks again.
// The window has to cover blob 1's encrypt phase: if the removal
// lands before the loop reaches dispatch, the fence aborts with
// nothing sealed, so give the in-flight add room to start under
// a loaded box while staying inside the 2s endpoint delay.
tokio::time::sleep(Duration::from_millis(1200)).await;
db.remove_visibility_rule(&repo_id, "**/secret/*")
.await
.expect("remove rule");
handle.await.expect("seal task")
})
.await
.expect("wedge guard: the fence abort must not take 30s");
assert!(
sealed.len() < oids.len(),
"a reader removal landing mid-batch must abort before every blob is sealed: {}",
sealed.len()
);
assert!(
!sealed.is_empty(),
"at least the blob already in flight before the removal completes"
);
}
/// A hung git must not hold the seal loop past its read budget (R1-P2): the
/// git read runs under `spawn_blocking` against `read_object_bounded`, so
/// the watchdog reaps a wedged child at the batch deadline and the loop
/// keeps its shape instead of blocking a runtime worker indefinitely.
#[cfg(unix)]
#[sqlx::test]
async fn encrypt_and_pin_returns_by_budget_with_a_hung_git(pool: sqlx::PgPool) {
use std::os::unix::fs::PermissionsExt;
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("seal-hung.git");
let oids: Vec<String> = {
crate::git::store::init_bare(&repo_path).expect("init bare repo");
(0..2)
.map(|i| {
let mut cmd = std::process::Command::new("git");
cmd.args(["hash-object", "-w", "--stdin"])
.current_dir(&repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped());
let mut child = cmd.spawn().expect("spawn git hash-object");
{
use std::io::Write;
child
.stdin
.as_mut()
.expect("stdin")
.write_all(format!("secret blob {i}\n").as_bytes())
.expect("write stdin");
}
let out = child.wait_with_output().expect("hash-object output");
assert!(out.status.success());
String::from_utf8_lossy(&out.stdout).trim().to_string()
})
.collect()
};
// A git that wedges forever, ignoring SIGTERM, so only the watchdog's
// SIGKILL can reap it.
let fake = tmp.path().join("hanging-git");
std::fs::write(&fake, "#!/bin/sh\ntrap '' TERM\necho $$ > pid\nsleep 30\n").unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let now = chrono::Utc::now();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.clone(),
name: "seal-hung-repo".into(),
owner_did: "did:key:zSealHungOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.expect("create repo");
db.set_visibility_rule(
&repo_id,
"**/secret/*",
crate::db::VisibilityMode::B,
&[did_key(1)],
"did:key:zSealHungOwner",
)
.await
.expect("set rule");
let recipients: HashMap<String, BTreeSet<String>> = oids
.iter()
.cloned()
.map(|oid| {
let mut s = BTreeSet::new();
s.insert(did_key(1));
(oid, s)
})
.collect();
// Unreachable endpoint: even if a read somehow succeeded, the pin would
// fail; the read itself is the thing under test.
let started = std::time::Instant::now();
let sealed = tokio::time::timeout(
Duration::from_secs(60),
encrypt_and_pin(
"http://127.0.0.1:9",
&repo_path,
&db,
&repo_id,
&SEED,
fake.to_str().unwrap(),
Duration::from_secs(2),
&recipients,
None,
),
)
.await
.expect("a hung git must not hold the seal past the outer wedge guard");
let elapsed = started.elapsed();
assert!(
elapsed < Duration::from_secs(10),
"a hung git must be watchdog-reaped inside the read budget, not block the loop for ~10s+ (took {elapsed:?})"
);
assert!(
sealed.is_empty(),
"with a hung git no blob can be read, so nothing may be reported sealed"
);
}
/// Local TCP endpoint that answers `{ "Hash": "QmMock" }` after an optional
/// per-request delay, so a seal can be made to straddle a policy mutation.
async fn delaying_cid_endpoint(delays: Vec<Duration>) -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
let mut seen = 0usize;
while let Ok((mut sock, _)) = listener.accept().await {
let delay = *delays
.get(seen)
.or_else(|| delays.last())
.unwrap_or(&Duration::ZERO);
seen += 1;
tokio::spawn(async move {
let mut acc = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
acc.extend_from_slice(&buf[..n]);
if let Some(hdr_end) =
acc.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4)
{
let headers = String::from_utf8_lossy(&acc[..hdr_end]).to_lowercase();
let len: usize = headers
.lines()
.find_map(|l| l.strip_prefix("content-length:"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0);
if acc.len() >= hdr_end + len {
break;
}
}
}
tokio::time::sleep(delay).await;
let body = br#"{"Hash":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4"}"#;
let _ = sock
.write_all(
format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n", body.len())
.as_bytes(),
)
.await;
let _ = sock.write_all(body).await;
let _ = sock.flush().await;
});
}
});
endpoint
}
}
+1 -15
View File
@@ -12,9 +12,6 @@ pub enum AppError {
#[error("repo already exists: {0}")]
RepoExists(String),
#[error("conflict: {0}")]
Conflict(String),
#[error("not found: {0}")]
NotFound(String),
@@ -104,7 +101,7 @@ pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
pub(crate) fn db_unavailable(e: &sqlx::Error) -> bool {
fn db_unavailable(e: &sqlx::Error) -> bool {
matches!(
e,
sqlx::Error::PoolTimedOut
@@ -192,7 +189,6 @@ impl IntoResponse for AppError {
"repo_exists",
format!("repository '{r}' already exists"),
),
AppError::Conflict(msg) => (StatusCode::CONFLICT, "conflict", msg.clone()),
AppError::NotFound(msg) => (StatusCode::NOT_FOUND, "not_found", msg.clone()),
AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, "not_an_agent", msg.clone()),
AppError::Forbidden(msg) => (StatusCode::FORBIDDEN, "forbidden", msg.clone()),
@@ -325,16 +321,6 @@ pub type Result<T> = std::result::Result<T, AppError>;
mod tests {
use super::*;
#[test]
fn conflict_maps_to_409() {
assert_eq!(
AppError::Conflict("task not claimable".into())
.into_response()
.status(),
StatusCode::CONFLICT
);
}
#[test]
fn timeout_maps_to_504_distinct_from_git_500() {
assert_eq!(
+28 -116
View File
@@ -129,22 +129,19 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
/// Returns Ok(Some(full_id)) on unique match, Ok(None) if not found,
/// Err if the prefix is ambiguous (matches more than one issue).
pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<String>> {
// The id comes from the request path and is spliced into a git revision and a
// for-each-ref pattern below. Issue ids are UUIDs, so accept only ASCII
// alphanumerics and '-': anything else (`^{blob}`, `@{0}`, `*`, `/`, `..`) is
// revision or glob syntax, never an issue id, and is reported as not found.
if id_or_prefix.is_empty()
|| !id_or_prefix
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'-')
{
return Ok(None);
// Try exact match first — fast path for callers passing the full UUID.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let check = Command::new("git")
.args(["cat-file", "-e", &exact_ref])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
if check.status.success() {
return Ok(Some(id_or_prefix.to_string()));
}
// List all refs that start with the given string; the glob also matches the exact
// ref. The id handed back is always git's own refname, never the request input.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let prefix_glob = format!("{exact_ref}*");
// Prefix search: list all refs that start with the given string.
let prefix_glob = format!("refs/gitlawb/issues/{id_or_prefix}*");
let list = Command::new("git")
.args(["for-each-ref", "--format=%(refname)", &prefix_glob])
.current_dir(repo_path)
@@ -156,44 +153,25 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
}
let output = String::from_utf8_lossy(&list.stdout);
let matches: Vec<&str> = output
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
let matches: Vec<&str> = output.lines().filter(|l| !l.trim().is_empty()).collect();
// An exact id wins over longer ids that share it as a prefix, but only if its
// object exists: for-each-ref also lists a ref whose blob is missing, so a
// dangling exact ref falls through to the prefix rules below.
let exact = match matches.iter().find(|m| **m == exact_ref) {
Some(m) => {
let check = Command::new("git")
.args(["cat-file", "-e", m])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
check.status.success().then_some(*m)
match matches.len() {
0 => Ok(None),
1 => {
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = matches[0]
.trim()
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(matches[0].trim())
.to_string();
Ok(Some(full_id))
}
None => None,
};
let found = match exact {
Some(m) => m,
None => match matches.as_slice() {
[] => return Ok(None),
[m] => *m,
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
},
};
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = found
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(found)
.to_string();
Ok(Some(full_id))
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
}
}
/// Close an issue by updating its status to "closed" in the git ref.
@@ -277,43 +255,6 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string()));
}
#[test]
fn test_resolve_exact_id_wins_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open"}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open"}"#).unwrap();
let resolved = resolve_issue_id(dir.path(), "abc12345").unwrap();
assert_eq!(resolved, Some("abc12345".to_string()));
}
#[test]
fn test_resolve_dangling_exact_id_does_not_win_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open","n":1}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open","n":2}"#).unwrap();
// Dangle the exact ref: the ref stays listed by for-each-ref, but its blob is gone.
let out = Command::new("git")
.args(["rev-parse", "refs/gitlawb/issues/abc12345"])
.current_dir(dir.path())
.output()
.unwrap();
let hash = String::from_utf8(out.stdout).unwrap();
let hash = hash.trim();
std::fs::remove_file(
dir.path()
.join(".git/objects")
.join(&hash[..2])
.join(&hash[2..]),
)
.unwrap();
let result = resolve_issue_id(dir.path(), "abc12345");
assert!(result.unwrap_err().to_string().contains("ambiguous"));
assert!(close_issue(dir.path(), "abc12345").is_err());
}
#[test]
fn test_resolve_prefix_matches_unique() {
let dir = TempDir::new().unwrap();
@@ -329,35 +270,6 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string()));
}
#[test]
fn test_resolve_rejects_revision_syntax() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "abc12345-0000-0000-0000-000000000000";
create_issue(
dir.path(),
full_id,
r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
// Each of these resolves to (or globs over) the issue blob in git, but none
// is an issue id, so the lookup must not accept it.
for crafted in [
"abc12345-0000-0000-0000-000000000000^{blob}",
"abc12345-0000-0000-0000-000000000000^{}",
"abc12345-0000-0000-0000-000000000000~0",
"*",
"",
] {
assert_eq!(
resolve_issue_id(dir.path(), crafted).unwrap(),
None,
"{crafted:?}"
);
assert_eq!(get_issue(dir.path(), crafted).unwrap(), None, "{crafted:?}");
}
}
#[test]
fn test_resolve_prefix_not_found() {
let dir = TempDir::new().unwrap();
+8 -15
View File
@@ -185,13 +185,10 @@ fn rev_list_delta(
/// List every object in the repository via
/// `git cat-file --batch-all-objects --batch-check='%(objectname)'`.
///
/// This is the whole-repo enumeration the push path falls back to. It
/// returns *all* objects (including unreachable/dangling ones), which is
/// what the push path needs to catch stragglers — do not swap it for a
/// reachability walk. The reconciliation sweep no longer uses it: sweep
/// candidates come from the windowed walk (window commits plus walked
/// pairs), which are reachable by construction, so dangling objects are
/// absent rather than filtered.
/// This is the whole-repo enumeration the push path falls back to and the
/// reconciliation sweep relies on. It returns *all* objects (including
/// unreachable/dangling ones), which is what the sweep needs to catch
/// stragglers — do not swap it for a reachability walk.
pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<Vec<String>> {
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
@@ -217,7 +214,6 @@ pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> R
/// filter needs to tell blobs (content, withholdable) and trees (content,
/// withholdable since #135/#173) from commits/tags (structural, never withheld)
/// without typing the candidate list itself.
#[allow(dead_code)] // used by tests and all_blob_oids
pub fn list_all_objects_with_type(
repo_path: &Path,
git_bin: &str,
@@ -265,7 +261,6 @@ pub fn all_blob_oids(
/// candidate blob or tree absent from the matching reachable visibility-allowed
/// set; a dangling or withheld object is in this universe but not the allowed
/// set, so it never replicates (#99, #172).
#[allow(dead_code)] // production went to the typed sweep path; tests still use it
pub fn all_blob_and_tree_oids(
repo_path: &Path,
git_bin: &str,
@@ -308,12 +303,10 @@ pub struct PinCandidateSet {
/// Every degraded path is **logged**, not silent: a full-scan fallback, a
/// failed full scan, and a panicked blocking task each emit a warning. On a
/// failed full scan or a task panic the candidate set is empty (pin nothing
/// this push); that is a durability gap the reconciliation sweep backstops
/// when it is enabled and a pin backend is configured (a node running with the
/// sweep disabled or with no IPFS/Pinata backend has no backstop), and it can
/// never leak because the withheld/fail-closed filter still runs on whatever
/// set is returned. `full_scan` rides on the returned set so the caller knows
/// when the dangling-inclusive filter is required.
/// this push); that is a durability gap the reconciliation sweep backstops, and
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
+4 -7
View File
@@ -118,7 +118,7 @@ impl RepoStore {
/// Test-only: every guard from this store parks in `release` right before the
/// `pg_advisory_unlock` await, until `gate` is notified. Dropping the future
/// while it is parked reproduces a client disconnect inside `release`.
#[cfg(all(test, unix))]
#[cfg(test)]
pub fn with_pre_unlock_gate(mut self, gate: Arc<tokio::sync::Notify>) -> Self {
self.pre_unlock_gate = Some(gate);
self
@@ -126,7 +126,7 @@ impl RepoStore {
/// Test-only: how many write guards from this store have reached the Tigris upload
/// site. See [`RepoWriteGuard::release`].
#[cfg(all(test, unix))]
#[cfg(test)]
pub fn tigris_upload_site_reached(&self) -> usize {
self.upload_site_reached
.load(std::sync::atomic::Ordering::SeqCst)
@@ -1717,11 +1717,8 @@ fn validate_repo_name(repo_name: &str) -> Result<()> {
if repo_name.is_empty() {
anyhow::bail!("repo_name is empty");
}
// Byte bound shared with the API-side validator via `MAX_REPO_NAME_LEN`:
// the two validators differ on charset by design, but must not drift on
// length, or an API-admitted name fails here after the proof is spent.
if repo_name.len() > crate::db::MAX_REPO_NAME_LEN {
anyhow::bail!("repo_name exceeds {} chars", crate::db::MAX_REPO_NAME_LEN);
if repo_name.len() > 100 {
anyhow::bail!("repo_name exceeds 100 chars");
}
// Repo names are `[A-Za-z0-9._-]+` minus path-traversal traps.
if repo_name.contains("..") {
+6 -491
View File
@@ -106,7 +106,7 @@ pub async fn info_refs(
let mut command = Command::new(git_bin);
command
.arg(service_to_command(service)?)
.arg(service_to_command(service))
.arg("--stateless-rpc")
.arg("--advertise-refs")
.arg(repo_path);
@@ -220,7 +220,6 @@ pub fn response_served_pack(output: &[u8]) -> bool {
///
/// Accepts a push. The caller MUST verify HTTP Signature auth before
/// calling this function.
#[allow(dead_code)] // used by tests; production uses receive_pack_raw
pub async fn receive_pack(
git_bin: &str,
repo_path: &Path,
@@ -245,219 +244,6 @@ pub async fn receive_pack(
.body(Body::from(output))?)
}
/// Run `git-receive-pack` and return the raw stdout bytes together with
/// the process exit status. Unlike [`receive_pack`], this does NOT bail
/// on a non-zero exit: the caller needs the stdout (which contains the
/// report-status with per-ref ok/ng results) even when the process
/// exits non-zero. A timeout still returns `Err`.
///
/// `reflog_action` (typically `gitlawb-request:<request_id>`) is
/// exported as `GIT_REFLOG_ACTION` so the ref transaction writes
/// request-bound landing evidence into the reflog. Recovery requires
/// that evidence before auto-promoting a stranded row; without it a
/// later request recreating the same old/new tuple is
/// indistinguishable from the stranded one.
#[allow(dead_code)]
pub async fn receive_pack_raw(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Vec<u8>, bool)> {
receive_pack_raw_with_reflog(git_bin, repo_path, request_body, timeout, admission, None).await
}
/// [`receive_pack_raw`] with explicit reflog binding.
pub async fn receive_pack_raw_with_reflog(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
reflog_action: Option<&str>,
) -> Result<(Vec<u8>, bool)> {
let mut command = Command::new(git_bin);
command
.arg("receive-pack")
.arg("--stateless-rpc")
.arg(repo_path);
if let Some(action) = reflog_action {
command.env("GIT_REFLOG_ACTION", action);
}
let (out, err, status, _admission) = drive_git_child_raw(
command,
request_body,
timeout,
"git-receive-pack",
admission,
)
.await?;
// On timeout, drive_git_child_raw returns Err — we never reach here.
// On success/non-zero exit, we have the stdout and exit status.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
tracing::warn!(stderr = %stderr, "git-receive-pack exited non-zero");
}
Ok((out, status.success()))
}
/// Parse the git-receive-pack report-status output to determine per-ref
/// success/failure. Returns `(unpack_ok, per_ref_results)` where
/// `per_ref_results` is a list of `(ref_name, is_ok)`.
///
/// The report-status format (after the sideband framing) is:
/// ```text
/// unpack ok\n (or: unpack fail\n)
/// ok <refname>\n (per successful ref)
/// ng <refname> <reason>\n (per rejected ref)
/// \n (empty line terminates)
/// ```
///
/// Returns `None` if the output cannot be parsed (e.g. the client did
/// not request report-status, or the output is truncated). In that
/// case the caller should treat all refs as uncertain.
pub fn parse_report_status(output: &[u8]) -> Option<(bool, Vec<(String, bool)>)> {
let text = std::str::from_utf8(output).ok()?;
// Strip sideband framing: each line starts with a pkt-line length
// prefix and a channel byte (1 = stdout, 2 = stderr). The actual
// data starts after the first `0000` flush packet or after we
// strip sideband bytes.
//
// Framing completeness is part of authority, at whichever level the
// framing lives. A double-framed capture carries the report stream
// (itself pkt-line framed, flush-terminated) inside the sideband
// envelope — real git output has been observed with NO outer flush,
// so the outer level cannot require one; the inner frame's flush is
// authoritative there. A single-framed (or unframed) payload has no
// inner frame, so its only completion marker is the outer flush:
// output ending after a complete `unpack ok` / `ok <ref>` packet but
// before the outer `0000` is a truncated report and yields `None`
// (uncertain), not a successful report.
let (stripped, outer_flush) = strip_sideband(text, false)?;
// The report-status vocabulary (`unpack ...`, `ok ...`, `ng ...`)
// never opens with four hex digits, so a hex quartet here is inner
// pkt-line framing, not report text.
let stripped = if stripped.len() >= 4
&& stripped.as_bytes()[..4]
.iter()
.all(|b| b.is_ascii_hexdigit())
{
strip_sideband(&stripped, true)?.0
} else if !outer_flush {
return None;
} else {
stripped
};
// The report is framed TWICE when the client negotiated side-band-64k,
// which `git push` over smart HTTP does — so this is the common case, not
// an exotic one. The outer frame is the side-band envelope; band 1 carries
// the report-status stream, which is ITSELF pkt-line encoded. Real bytes
// from git 2.50.1 rejecting one ref of a two-ref push:
//
// 0057\x01000eunpack ok\n0028ng refs/heads/main non-fast-forward\n...
//
// After one pass the first line reads `000eunpack ok`, which fails the
// `unpack ok` check below and makes this return None — i.e. "no report",
// which the caller treats as inconclusive and keeps every declared ref.
// The per-ref gate would then be inert for exactly the pushes it exists to
// filter. The second pass above already removed the inner pkt-line
// framing where present (and required its flush); it is a no-op
// on the single-framed shape, because plain report text does not begin
// with four hex digits.
let lines: Vec<&str> = stripped.lines().collect();
if lines.is_empty() {
return None;
}
// First non-empty line is "unpack ok" or "unpack fail".
let unpack_line = lines.iter().find(|l| !l.is_empty())?;
let unpack_ok = if unpack_line.starts_with("unpack ok") {
true
} else if unpack_line.starts_with("unpack fail") {
false
} else {
return None;
};
let mut results = Vec::new();
for line in &lines[1..] {
let line = line.trim();
if line.is_empty() {
break;
}
if let Some(rest) = line.strip_prefix("ok ") {
results.push((rest.to_string(), true));
} else if let Some(rest) = line.strip_prefix("ng ") {
// "ng <refname> <reason>" — skip the reason
let ref_name = rest.split_whitespace().next()?.to_string();
results.push((ref_name, false));
}
}
Some((unpack_ok, results))
}
/// Strip git sideband framing from a pkt-line encoded output.
/// Sideband-encoded lines start with a 4-hex-digit length, then a
/// channel byte (0x01=stdout, 0x02=stderr), then payload. Returns the
/// decoded payload plus whether the stream saw its terminating flush
/// packet, or `None` if the framing is malformed. With `require_flush`,
/// a stream that never sees its flush is also `None` — the caller uses
/// this for the level whose completion marker is authoritative (see
/// [`parse_report_status`]).
fn strip_sideband(text: &str, require_flush: bool) -> Option<(String, bool)> {
let mut output = String::new();
let mut pos = 0;
let bytes = text.as_bytes();
let mut saw_flush = false;
loop {
if pos == bytes.len() {
break;
}
if pos + 4 > bytes.len() {
// Truncated framing mid-header: fail closed.
return None;
}
let len_str = std::str::from_utf8(&bytes[pos..pos + 4]).ok()?;
let len = usize::from_str_radix(len_str, 16).ok()?;
if len == 0 {
// Flush packet — end of sideband stream
saw_flush = true;
break;
}
if len < 4 || pos + len > bytes.len() {
// Truncated pkt-line extends past EOF: incomplete framing.
return None;
}
let pkt_data = std::str::from_utf8(&bytes[pos + 4..pos + len]).ok()?;
pos += len;
// Sideband: first byte is channel (1=stdout, 2=stderr)
if let Some(payload) = pkt_data.strip_prefix('\x01') {
output.push_str(payload);
} else if pkt_data.starts_with('\x02') {
// stderr — skip (git error messages)
} else {
// Not sideband encoded — pass through
output.push_str(pkt_data);
}
}
if require_flush && !saw_flush {
// Truncated at a packet boundary: every packet parsed, but the
// stream never completed. Indistinguishable from a report git was
// killed mid-write, so no authority.
return None;
}
if output.is_empty() {
None
} else {
Some((output, saw_flush))
}
}
/// Sends SIGTERM to a child's whole process group on drop, unless disarmed first.
///
/// A served `git upload-pack`/`receive-pack` forks helpers such as `pack-objects`.
@@ -640,7 +426,7 @@ async fn run_git_service(
) -> Result<Vec<u8>> {
let mut command = Command::new(git_bin);
command
.arg(service_to_command(service)?)
.arg(service_to_command(service))
.arg("--stateless-rpc")
.arg(repo_path);
let (out, admission) = drive_git_child(command, input, timeout, service, admission).await?;
@@ -792,128 +578,11 @@ async fn drive_git_child(
Ok((out, admission))
}
/// Like [`drive_git_child`], but returns stdout and stderr even on a
/// non-zero exit status. Used by [`receive_pack_raw`] so the caller
/// can parse the report-status output from a failed `git-receive-pack`.
async fn drive_git_child_raw(
mut command: Command,
input: Bytes,
timeout: Duration,
what: &str,
admission: Option<AdmissionGuard>,
) -> Result<(
Vec<u8>,
Vec<u8>,
std::process::ExitStatus,
Option<AdmissionGuard>,
)> {
command
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
#[cfg(unix)]
command.process_group(0);
let mut child = command.spawn()?;
let mut stdin = child.stdin.take();
let mut stdout = child.stdout.take().context("git stdout was not piped")?;
let mut stderr = child.stderr.take().context("git stderr was not piped")?;
#[cfg(unix)]
let pgid = child.id().map(|id| id as i32);
#[cfg(unix)]
let mut group_guard = KillGroupOnDrop {
child: Some(child),
pgid,
admission,
};
let mut out = Vec::new();
let mut err = Vec::new();
let interact = async {
let write = async {
match stdin.take() {
Some(mut s) => s.write_all(&input).await,
None => Ok(()),
}
};
#[cfg(unix)]
let child_ref = group_guard.child_mut();
#[cfg(not(unix))]
let child_ref = &mut child;
let (write_result, r_out, r_err, status) = tokio::join!(
write,
stdout.read_to_end(&mut out),
stderr.read_to_end(&mut err),
child_ref.wait(),
);
r_out?;
r_err?;
Ok::<_, anyhow::Error>((write_result, status?))
};
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status, admission) = match timed {
Ok(result) => {
#[cfg(unix)]
let admission = group_guard.disarm();
let (write_result, status) = result?;
(write_result, status, admission)
}
Err(_elapsed) => {
#[cfg(unix)]
{
reap_group_on_timeout(group_guard.child_mut()).await;
drop(group_guard.disarm());
}
#[cfg(not(unix))]
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
drop(admission);
}
return Err(GitServiceTimeout.into());
}
};
// Prefer the child's collected answer over a stdin-write error: an early
// rejection (unpack failure, command-level deny) makes git print a complete
// report-status and exit WITHOUT draining stdin, so a body larger than the
// pipe buffer EPIPEs the write while the answer is already on stdout. The
// child is reaped (the join above waited on it), so `out`/`err`/`status`
// are complete and authoritative — returning Err here would discard them:
// `receive_pack_raw_with_reflog` would propagate the write error and the
// handler would mark every prepared row `uncertain` instead of driving
// `cancelled` per the report. That durable fate is the effect: on a
// non-zero exit the client still gets `Err(AppError::Git)` (the handler
// maps the failed run itself, not this write), while on exit 0 an error
// here would answer a push git actually landed with a server error. The
// non-raw `drive_git_child` orders the same way (exit status and git's own
// failure first, stdin-write error last — pinned by
// `run_git_service_surfaces_git_stderr_over_a_stdin_epipe`), but it must
// fail on non-zero because its callers do not parse output from a failed
// child; the raw driver exists precisely to parse that output.
if let Err(e) = &write_result {
tracing::warn!(
err = %e,
what = %what,
"git stdin write failed after child exit; surfacing collected output"
);
}
Ok((out, err, status, admission))
}
/// Map a validated service name to its git subcommand. Only the two fixed
/// subcommands can come out, so an unrecognized service can never reach git's argv.
fn service_to_command(service: &str) -> Result<&'static str> {
fn service_to_command(service: &str) -> &str {
match service {
"git-upload-pack" => Ok("upload-pack"),
"git-receive-pack" => Ok("receive-pack"),
other => bail!("unknown git service: {other}"),
"git-upload-pack" => "upload-pack",
"git-receive-pack" => "receive-pack",
_ => service,
}
}
@@ -1197,104 +866,6 @@ mod tests {
use std::process::Command;
use tempfile::TempDir;
/// Byte-for-byte capture from `git receive-pack` 2.50.1 rejecting one ref of
/// a two-ref push, with side-band-64k negotiated — what `git push` over
/// smart HTTP actually sends. The report is framed twice: the outer
/// side-band envelope, then the report-status stream's own pkt-lines.
///
/// This fixture is the point of the test. A hand-written single-framed
/// string parses fine with only one strip pass, so a fixture invented to
/// match the parser hides the exact case the parser is for.
const REAL_SIDEBAND_REPORT: &[u8] =
b"0057\x01000eunpack ok\n0028ng refs/heads/main non-fast-forward\n0018ok refs/heads/third\n0000";
#[test]
fn a_real_double_framed_sideband_report_is_read_not_treated_as_absent() {
let (unpack_ok, refs) =
parse_report_status(REAL_SIDEBAND_REPORT).expect("a real git report must parse");
assert!(unpack_ok, "unpack line must be read through both frames");
assert_eq!(
refs,
vec![
("refs/heads/main".to_string(), false),
("refs/heads/third".to_string(), true),
],
"the rejected ref must be distinguished from the accepted one"
);
}
/// The single-framed shape must keep working: the second pass has to be a
/// no-op there, not a corruption. Plain report text does not begin with
/// four hex digits, which is what makes that safe.
#[test]
fn a_single_framed_report_still_parses_after_the_second_pass() {
// Framed programmatically rather than by a hand-counted hex prefix: a
// wrong length makes the parser return None, which would look exactly
// like the regression this pair of tests exists to catch.
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let (unpack_ok, refs) =
parse_report_status(single.as_bytes()).expect("single-framed must parse");
assert!(unpack_ok);
assert_eq!(refs, vec![("refs/heads/x".to_string(), true)]);
}
#[test]
fn truncated_mid_packet_report_is_indeterminate_not_partial() {
// Truncate a valid single-framed report mid-packet (overrun):
// must return None so the handler persists the whole request as
// indeterminate instead of committing a partial ok set.
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let truncated = &single.as_bytes()[..single.len() - 6];
assert!(
parse_report_status(truncated).is_none(),
"mid-packet truncation must be indeterminate"
);
}
/// Truncation exactly at a packet boundary (every packet complete, but
/// the outer flush never arrived — e.g. git killed after the last `ok`
/// line) must also be indeterminate: the payload is complete but the
/// frame is not, and framing completeness is part of authority. The
/// mid-packet sibling above cannot catch this; only the flush gate can.
/// Removing the `require_flush` enforcement returns `Some` and turns
/// this red.
#[test]
fn truncated_at_packet_boundary_report_is_indeterminate_not_partial() {
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let no_flush = &single.as_bytes()[..single.len() - 4];
assert!(
parse_report_status(no_flush).is_none(),
"a report missing its outer flush must be indeterminate, \
even with every packet intact"
);
// Sanity: the untruncated fixture still parses (flush present).
assert!(
parse_report_status(single.as_bytes()).is_some(),
"the same bytes with the flush must parse"
);
}
#[test]
fn service_to_command_maps_allowlist_and_rejects_passthrough() {
assert_eq!(
service_to_command("git-upload-pack").unwrap(),
"upload-pack"
);
assert_eq!(
service_to_command("git-receive-pack").unwrap(),
"receive-pack"
);
for bad in ["git-fetch-pack", "--version", "upload-pack", ""] {
assert!(
service_to_command(bad).is_err(),
"must reject passthrough for {bad:?}"
);
}
}
/// List OIDs in a pack by writing it to a temp dir and running verify-pack.
pub(super) fn pack_object_ids(pack: &[u8]) -> std::collections::HashSet<String> {
let dir = TempDir::new().unwrap();
@@ -3050,62 +2621,6 @@ mod tests {
);
}
// The raw driver's counterpart: a rejection-shaped early exit where git
// prints a COMPLETE report-status and exits without draining stdin, so a
// body larger than the pipe buffer (~64 KiB) EPIPEs the write while the
// answer is already on stdout. `receive_pack_raw` must surface that report
// (Ok + stdout + exit status), not discard it as a stdin-write error — an
// Err here loses parseable output: `receive_pack_raw_with_reflog`
// propagates it and the handler marks every prepared row `uncertain`
// instead of driving `cancelled` from the report (the durable fate is the
// effect; the client-visible status on a failed run is `AppError::Git`
// either way). Goes RED if the stdin-write check returns Err after the
// join. Both exit codes are pinned: receive-pack rejections and successes
// both carry reports the handler must parse.
#[cfg(unix)]
#[tokio::test]
async fn receive_pack_raw_surfaces_report_status_over_a_stdin_epipe() {
for exit_code in [0, 1] {
let tmp = tempfile::TempDir::new().unwrap();
std::fs::write(tmp.path().join("report.bin"), REAL_SIDEBAND_REPORT).unwrap();
// Never reads stdin: the pending body blocks the pipe, then EPIPEs
// once the fake exits. `cat` delivers the report first.
let git_bin = write_fake_git(
tmp.path(),
&format!("#!/bin/sh\ncat \"$(dirname \"$0\")/report.bin\"\nexit {exit_code}\n"),
);
let body = Bytes::from(vec![0u8; 256 * 1024]);
let (out, ok) = receive_pack_raw(
git_bin.to_str().unwrap(),
tmp.path(),
body,
Duration::from_secs(60),
None,
)
.await
.expect(
"a report-status printed before exit must not be discarded as a \
stdin-write error",
);
assert_eq!(
ok,
exit_code == 0,
"the exit status rides the Ok return for both report shapes"
);
let (unpack_ok, results) = parse_report_status(&out)
.expect("git's report survives the EPIPE and stays parseable");
assert!(unpack_ok, "unpack ok parses");
assert_eq!(
results,
vec![
("refs/heads/main".to_string(), false),
("refs/heads/third".to_string(), true),
],
"the per-ref report reaches the handler instead of an HTTP error"
);
}
}
// ── #192 F1: response_served_pack detector (real git output) ────────────
// A completing request (want+flush+done) yields NAK + a real packfile;
File diff suppressed because it is too large. Load diff
+1 -3
View File
@@ -627,9 +627,7 @@ impl TigrisClient {
}
/// Compress a bare repo directory into a tar.zst byte vector.
/// `pub(crate)` for sweep tests that seed a fake object store with a
/// byte-identical archive (rather than duplicating the format).
pub(crate) fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
let buf = Vec::new();
let encoder = zstd::stream::Encoder::new(buf, 3)?; // level 3 = fast + decent ratio
let mut tar = tar::Builder::new(encoder);
File diff suppressed because it is too large. Load diff
+4 -767
View File
@@ -3,7 +3,7 @@ pub mod query;
pub mod subscription;
pub mod types;
use async_graphql::{Schema, SchemaBuilder};
use async_graphql::Schema;
use std::sync::Arc;
use crate::db::Db;
@@ -14,22 +14,6 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
// Keep ordinary schema discovery and application queries usable while
// bounding validation work and the number of resolver selections per request.
const GRAPHQL_MAX_COMPLEXITY: usize = 400;
// The current public schema is shallow; this leaves headroom for composed
// clients and the canonical getIntrospectionQuery without allowing
// recursively nested documents to grow unchecked.
const GRAPHQL_MAX_DEPTH: usize = 14;
fn apply_query_limits<Query, Mutation, Subscription>(
builder: SchemaBuilder<Query, Mutation, Subscription>,
) -> SchemaBuilder<Query, Mutation, Subscription> {
builder
.limit_complexity(GRAPHQL_MAX_COMPLEXITY)
.limit_depth(GRAPHQL_MAX_DEPTH)
}
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
@@ -76,7 +60,6 @@ pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error
// Curated client-safe variants — `Display` is intentional API text.
safe @ (crate::error::AppError::RepoNotFound(_)
| crate::error::AppError::RepoExists(_)
| crate::error::AppError::Conflict(_)
| crate::error::AppError::NotFound(_)
| crate::error::AppError::Unauthorized(_)
| crate::error::AppError::Forbidden(_)
@@ -93,91 +76,21 @@ pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error
}
}
/// Classify a failed `claim_task` for GraphQL exactly as REST's claim handler
/// classifies it: a lost race is a client-safe conflict, a real sqlx fault
/// stays opaque. Lives here rather than at the three call sites so the
/// classification cannot drift between transports (#327 review), and so
/// `every_graphql_map_err_uses_opaque_helpers` can keep whitelisting by name.
pub(crate) fn graphql_claim_conflict(e: anyhow::Error) -> async_graphql::Error {
graphql_app_err(crate::api::tasks::task_write_conflict(
e,
"task not claimable: not found or already claimed",
))
}
/// The `finish_task` half of [`graphql_claim_conflict`], covering both
/// `completeTask` and `failTask`.
pub(crate) fn graphql_finish_conflict(e: anyhow::Error) -> async_graphql::Error {
graphql_app_err(crate::api::tasks::task_write_conflict(
e,
"task not found or not in claimed state",
))
}
pub struct TaskReadBrakeExtension;
impl async_graphql::extensions::ExtensionFactory for TaskReadBrakeExtension {
fn create(&self) -> Arc<dyn async_graphql::extensions::Extension> {
Arc::new(TaskReadBrakeExtensionImpl)
}
}
use async_graphql::async_trait::async_trait;
struct TaskReadBrakeExtensionImpl;
#[async_trait]
impl async_graphql::extensions::Extension for TaskReadBrakeExtensionImpl {
async fn prepare_request(
&self,
ctx: &async_graphql::extensions::ExtensionContext<'_>,
mut request: async_graphql::Request,
next: async_graphql::extensions::NextPrepareRequest<'_>,
) -> async_graphql::ServerResult<async_graphql::Request> {
if let Some(session_brake) = ctx
.session_data
.get(&std::any::TypeId::of::<crate::rate_limit::TaskReadBrake>())
.and_then(|d| d.downcast_ref::<crate::rate_limit::TaskReadBrake>())
{
request = request.data(crate::rate_limit::TaskReadBrake {
limiter: session_brake.limiter.clone(),
key: session_brake.key.clone(),
request_count: Arc::new(std::sync::atomic::AtomicUsize::new(0)),
});
}
if let Some(did) = ctx
.session_data
.get(&std::any::TypeId::of::<crate::auth::AuthenticatedDid>())
.and_then(|d| d.downcast_ref::<crate::auth::AuthenticatedDid>())
{
request = request.data(did.clone());
}
next.run(ctx, request).await
}
}
pub fn build_schema(
db: Arc<Db>,
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>,
task_cursor_key: crate::api::task_cursor::TaskCursorKey,
) -> GitlawbSchema {
apply_query_limits(Schema::build(QueryRoot, MutationRoot, SubscriptionRoot))
Schema::build(QueryRoot, MutationRoot, SubscriptionRoot)
.data(db)
.data(ref_update_tx)
.data(task_event_tx)
.data(task_cursor_key)
.extension(TaskReadBrakeExtension)
.finish()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
use crate::db::{MAX_VISIBLE_REPO_PAGE_SIZE, MAX_VISIBLE_TASK_PAGE_SIZE};
use async_graphql::{EmptyMutation, EmptySubscription, Object, Value};
use std::sync::atomic::{AtomicUsize, Ordering};
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
@@ -247,683 +160,9 @@ mod tests {
assert!(!err.message.contains("failed to open"));
}
fn production_test_schema() -> GitlawbSchema {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
production_test_schema_with_db(Arc::new(Db::for_testing(pool)))
}
fn production_test_schema_with_db(db: Arc<Db>) -> GitlawbSchema {
build_schema(
db,
tokio::sync::broadcast::channel(1).0,
tokio::sync::broadcast::channel(1).0,
crate::api::task_cursor::TaskCursorKey::derive(&[42u8; 32]),
)
}
#[tokio::test]
async fn ref_update_subscription_alias_budget() {
assert_subscription_alias_budget("refUpdates { repo }").await;
}
#[tokio::test]
async fn task_event_subscription_alias_budget() {
assert_subscription_alias_budget("taskEvents { taskId }").await;
}
async fn assert_subscription_alias_budget(field: &str) {
use futures::StreamExt;
use std::time::Duration;
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(
Arc::new(Db::for_testing(pool)),
ref_tx.clone(),
task_tx.clone(),
crate::api::task_cursor::TaskCursorKey::derive(&[42u8; 32]),
);
for count in [7, 8] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let query = format!("subscription {{ {fields} }}");
let mut responses = Box::pin(schema.execute_stream(query));
if count == 8 {
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.expect("over-budget subscriptions must reject before waiting for events")
.expect("the validation error must be returned");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
continue;
}
// Poll the accepted operation far enough to register its receivers.
assert!(
tokio::time::timeout(Duration::from_millis(30), responses.next())
.await
.is_err()
);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 7);
let _ = ref_tx.send(RefUpdateBroadcast {
repo: "owner/repo".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "1".repeat(40),
pusher_did: "did:key:test".into(),
node_did: "did:key:node".into(),
timestamp: "2026-01-01T00:00:00Z".into(),
owner_did: "did:key:test".into(),
});
let _ = task_tx.send(TaskEventBroadcast {
task_id: "task".into(),
old_status: "pending".into(),
new_status: "claimed".into(),
by_did: "did:key:test".into(),
at: "2026-01-01T00:00:00Z".into(),
});
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.unwrap()
.unwrap();
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_ne!(response.data, Value::Null);
drop(responses);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
}
}
#[sqlx::test]
async fn production_mutation_budget_accepts_seven_and_rejects_eight(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let caller = "did:key:reviewer";
let now = chrono::Utc::now().to_rfc3339();
for index in 0..8 {
db.create_task(&crate::db::AgentTask {
id: format!("task-{index}"),
repo_id: None,
kind: "test".into(),
status: "pending".into(),
delegator_did: caller.into(),
assignee_did: None,
capability: "test".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = production_test_schema_with_db(db.clone());
for count in [8, 7] {
let fields = (0..count)
.map(|n| {
format!("r{n}: claimTask(id: \"task-{n}\", assigneeDid: \"{caller}\") {{ id }}")
})
.collect::<Vec<_>>()
.join(" ");
let response = schema
.execute(
async_graphql::Request::new(format!("mutation {{ {fields} }}"))
.data(crate::auth::AuthenticatedDid(caller.into())),
)
.await;
if count == 8 {
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(
db.list_tasks_keyset(Some("pending"), None, 8, None)
.await
.unwrap()
.len(),
8
);
continue;
}
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
7
);
assert_eq!(
db.list_tasks_keyset(Some("claimed"), Some(caller), 8, None)
.await
.unwrap()
.len(),
7
);
assert_eq!(
db.get_task("task-7").await.unwrap().unwrap().status,
"pending"
);
}
}
#[sqlx::test]
async fn production_list_cost_scales_with_the_same_alias_count(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
// Bounds are per-root named constants so the cases move with the
// documented maximums instead of going stale as literals.
for (field, selection, max) in [
("refUpdates", "repo", MAX_VISIBLE_REF_UPDATES),
("tasks", "items { id }", MAX_VISIBLE_TASK_PAGE_SIZE),
(
"reposPage",
"nodes { name }",
MAX_VISIBLE_REPO_PAGE_SIZE as i64,
),
] {
// Two aliases at the maximum always exceed the budget.
let fields = (0..2)
.map(|n| format!("r{n}: {field}(limit: {max}) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
// At limit 1 two aliases fit for the row-priced roots; reposPage
// prices the full corpus scan, so a single alias is the most that
// fits under the budget.
let aliases = if field == "reposPage" { 1 } else { 2 };
let fields = (0..aliases)
.map(|n| format!("r{n}: {field}(limit: 1) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{field}: {:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
aliases
);
}
}
#[tokio::test]
async fn production_limits_reject_mutation_aliases_and_large_lists() {
let schema = production_test_schema();
// Limits are derived from the MAX_* constants so the boundaries move
// with the documented maximums instead of going stale as literals.
for (prefix, count, field) in [
("mutation", 8, "claimTask(id: \"missing\", assigneeDid: \"did:key:test\") { id }".to_owned()),
("mutation", 8, "createTask(delegatorDid: \"did:key:test\", input: { kind: \"test\", capability: \"test\" }) { id }".to_owned()),
("mutation", 8, "completeTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("mutation", 8, "failTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("query", 8, "task(id: \"missing\") { id }".to_owned()),
("query", 2, format!("refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }}")),
("query", 2, format!("tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id }}")),
("query", 2, format!("reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ nodes {{ name }} }}")),
("query", 8, "reposPage(limit: 1) { nodes { name } }".to_owned()),
] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{prefix} {{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
}
#[sqlx::test]
async fn production_repos_page_serves_the_documented_maximum(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
for selection in [
"nodes { name }",
"nodes { name ownerDid } hasNextPage endCursor",
] {
let response = schema
.execute(format!(
"{{ reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ {selection} }} }}"
))
.await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response.data.into_json().unwrap()["reposPage"]["nodes"],
serde_json::json!([])
);
}
}
#[tokio::test]
async fn seven_root_aliases_are_accepted() {
struct Root(Arc<AtomicUsize>);
#[Object]
impl Root {
#[graphql(complexity = "50 + child_complexity")]
async fn repos(&self) -> Vec<Nested> {
self.0.fetch_add(1, Ordering::Relaxed);
vec![Nested]
}
}
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
Root(calls.clone()),
EmptyMutation,
EmptySubscription,
))
.finish();
let fields = (0..7)
.map(|n| format!("r{n}: repos {{ value }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(calls.load(Ordering::Relaxed), 7);
}
#[tokio::test]
async fn expensive_root_aliases_are_rejected_before_database_access() {
// Use the production builder with a lazy pool: rejection must precede DB access.
let schema = production_test_schema();
let fields = (0..8)
.map(|n| format!("r{n}: repos {{ name }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.data, Value::Null);
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ordinary_schema_introspection_remains_available() {
let schema = production_test_schema();
let canonical_introspection = r#"
query IntrospectionQuery {
__schema {
queryType { name }
mutationType { name }
subscriptionType { name }
types {
...FullType
}
directives {
name
description
locations
args {
...InputValue
}
}
}
}
fragment FullType on __Type {
kind
name
description
fields(includeDeprecated: true) {
name
description
args {
...InputValue
}
type {
...TypeRef
}
isDeprecated
deprecationReason
}
inputFields {
...InputValue
}
interfaces {
...TypeRef
}
enumValues(includeDeprecated: true) {
name
description
isDeprecated
deprecationReason
}
possibleTypes {
...TypeRef
}
}
fragment InputValue on __InputValue {
name
description
type { ...TypeRef }
defaultValue
}
fragment TypeRef on __Type {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
}
}
}
}
}
}
}
}
"#;
let response = schema.execute(canonical_introspection).await;
assert!(
response.errors.is_empty(),
"canonical introspection query failed against production schema: {:?}",
response.errors
);
assert_ne!(response.data, Value::Null);
}
#[tokio::test]
async fn ref_updates_and_tasks_single_alias_max_limit_complexity_contract() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
pool.close().await;
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(
Arc::new(Db::for_testing(pool)),
ref_tx,
task_tx,
crate::api::task_cursor::TaskCursorKey::derive(&[42u8; 32]),
);
// refUpdates(limit: MAX_VISIBLE_REF_UPDATES):
// Single-field selection has child_complexity 1 -> cost 100 + 200 + 1 = 301 <= 400.
// It passes validation and reaches the resolver (which yields db error on lazy pool).
let single_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }} }}"
))
.await;
assert!(
!single_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field refUpdates at max limit must pass complexity validation: {:?}",
single_ref.errors
);
assert!(
single_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field refUpdates at max limit must reach the resolver: {:?}",
single_ref.errors
);
// Two-field selection has child_complexity 2 -> cost 100 + 200 + 2 = 302 <= 400.
// The documented max-200 request stays reachable with multiple fields
// (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo refName }} }}"
))
.await;
assert!(
!multi_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field refUpdates at max limit must pass complexity validation: {:?}",
multi_ref.errors
);
assert!(
multi_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field refUpdates at max limit must reach the resolver: {:?}",
multi_ref.errors
);
// tasks(limit: MAX_VISIBLE_TASK_PAGE_SIZE):
// Selection items { id } has child_complexity 2, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 2 <= 400.
// It passes validation and reaches the resolver.
let single_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ items {{ id }} }} }}"
))
.await;
assert!(
!single_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field tasks at the maximum page size must pass complexity validation: {:?}",
single_task.errors
);
assert!(
single_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field tasks at the maximum page size must reach the resolver: {:?}",
single_task.errors
);
// Selection items { id status } has child_complexity 3, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 3 <= 400.
// A request at the documented maximum stays reachable with multiple
// fields (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ items {{ id status }} }} }}"
))
.await;
assert!(
!multi_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field tasks at the maximum page size must pass complexity validation: {:?}",
multi_task.errors
);
assert!(
multi_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field tasks at the maximum page size must reach the resolver: {:?}",
multi_task.errors
);
}
#[tokio::test]
async fn two_repos_aliases_exceed_the_budget() {
// `repos` is priced as the full page it scans
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child), so two `repos { name }`
// aliases cost 2 * 251 = 502 > 400 and the document is rejected before
// any resolver runs. If the price reverts to `50 + child_complexity`
// the aliases cost 2 * 51 = 102, validation passes, and the failure
// mode changes (resolver DB errors on the lazy pool): this test goes
// red on revert.
let schema = production_test_schema();
let response = schema
.execute("{ r0: repos { name } r1: repos { name } }")
.await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn seven_repos_page_aliases_exceed_the_budget() {
// `reposPage` is priced as the full corpus scan
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity), so seven
// `reposPage(limit: 1) { nodes { name } }` aliases cost
// 7 * (50 + 200 + 2) = 1764 > 400 and the document is rejected
// before any resolver runs. Under the previous row-priced formula
// the same aliases cost 7 * (50 + 1 + 2) = 371, validation passes,
// and the failure mode changes (resolver DB errors on the lazy
// pool): this test goes red on revert. Eight aliases are rejected
// under both formulas, so seven is the load-bearing boundary.
let schema = production_test_schema();
let fields = (0..7)
.map(|n| format!("r{n}: reposPage(limit: 1) {{ nodes {{ name }} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ref_updates_fixed_base_is_charged() {
// The 100 base in the refUpdates complexity formula charges for the
// collector's fixed work (loading the deduped repo set, the
// quarantined set, and the visibility rules, then walking up to 16
// keyset pages) even at limit: 1. Four aliases at limit: 1 cost
// 4 * (100 + 1 + 1) = 408 > 400 and must be rejected. If the base
// disappears, the cost drops to 4 * (1 + 1) = 8 and this test fails.
let schema = production_test_schema();
let fields = (0..4)
.map(|n| format!("r{n}: refUpdates(limit: 1) {{ repo }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[derive(Clone, Copy)]
struct Nested;
#[Object]
impl Nested {
async fn child(&self) -> Nested {
Nested
}
async fn value(&self) -> i32 {
1
}
}
struct CountingQuery(Arc<AtomicUsize>);
#[Object]
impl CountingQuery {
async fn nested(&self) -> Nested {
self.0.fetch_add(1, Ordering::Relaxed);
Nested
}
}
#[tokio::test]
async fn query_depth_limit_accepts_fourteen_and_rejects_fifteen() {
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
CountingQuery(Arc::clone(&calls)),
EmptyMutation,
EmptySubscription,
))
.finish();
let query_at_depth = |depth: usize| {
let selection = (2..depth).fold("value".to_string(), |selection, _| {
format!("child {{ {selection} }}")
});
format!("{{ nested {{ {selection} }} }}")
};
let accepted = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH)).await;
assert!(
accepted.errors.is_empty(),
"depth {GRAPHQL_MAX_DEPTH} should be accepted: {:?}",
accepted.errors
);
assert_eq!(calls.swap(0, Ordering::Relaxed), 1);
let rejected = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH + 1)).await;
assert_eq!(rejected.data, Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
assert_eq!(calls.load(Ordering::Relaxed), 0);
}
#[tokio::test]
async fn production_schema_enforces_depth_limit() {
let schema = production_test_schema();
// Depth 14 is accepted through the production build_schema path.
let of_types_14 = (0..9).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query_14 =
format!("{{ __schema {{ types {{ fields {{ type {{ {of_types_14} }} }} }} }} }}");
let accepted = schema.execute(&query_14).await;
assert!(accepted.errors.is_empty(), "{:?}", accepted.errors);
// Construct an introspection query of depth 15 using __schema.
// 1: __schema
// 2: types
// 3: fields
// 4: type
// 5..14: ofType (10 times)
// 15: name
// Total depth = 15 > GRAPHQL_MAX_DEPTH (14).
let of_types = (0..10).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query = format!("{{ __schema {{ types {{ fields {{ type {{ {of_types} }} }} }} }} }}");
let rejected = schema.execute(&query).await;
assert_eq!(rejected.data, async_graphql::Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through one of the curated helpers in this module, or discard the error
/// (`|_|`). Same source-scrape pattern as `api::authz_guard` (#255
/// review). The list is deliberately a whitelist of names rather than a
/// prefix match, so adding a new mapper is a decision a reviewer sees.
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
#[test]
fn every_graphql_map_err_uses_opaque_helpers() {
for (file, src) in [
@@ -938,8 +177,6 @@ mod tests {
let after = code[idx + ".map_err(".len()..].trim_start();
let ok = after.starts_with("crate::graphql::graphql_db_err")
|| after.starts_with("crate::graphql::graphql_app_err")
|| after.starts_with("crate::graphql::graphql_claim_conflict")
|| after.starts_with("crate::graphql::graphql_finish_conflict")
|| after.starts_with("|_|")
|| after.starts_with("|_ ");
assert!(
+87 -804
View File
@@ -22,7 +22,6 @@ pub struct MutationRoot;
#[Object]
impl MutationRoot {
#[graphql(complexity = "50 + child_complexity")]
async fn create_task(
&self,
ctx: &Context<'_>,
@@ -37,33 +36,6 @@ impl MutationRoot {
}
let delegator_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
// #496: same write gate as the REST handler — a repo_id naming a
// hosted, non-quarantined repo admits tasks only from its owner, so a
// stranger cannot file under a foreign repo id. An id naming no hosted
// repo (unknown, or quarantined — hidden as if it did not exist) is
// stored verbatim as an opaque label; the gate does not bind an
// unresolved id to a repo. Task reads are caller-independent on this
// base (the tasks resolver is not repo-scoped), before and after this
// change. As in tasks.rs, the success-vs-denial split between a hosted
// repo id and an unknown one is itself a one-bit oracle — a stranger
// holding a private repo's id learns it is hosted here — but denying
// foreign repos requires drawing exactly that line.
if let Some(repo_id) = input.repo_id.as_deref() {
let record = db
.get_repo_by_id(repo_id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if let Some(record) = record {
let quarantined = db
.is_repo_quarantined(&record.id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if !quarantined {
crate::api::require_repo_owner(&record, caller)
.map_err(crate::graphql::graphql_app_err)?;
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -80,17 +52,12 @@ impl MutationRoot {
updated_at: now,
deadline: input.deadline,
};
// Return the stored row, not the pre-insert input: create_task
// normalizes the assignee (whitespace-only becomes NULL/open), and a
// response that shows the raw input lies about the reservation.
let stored = db
.create_task(&task)
db.create_task(&task)
.await
.map_err(crate::graphql::graphql_db_err)?;
Ok(AgentTaskType::from(stored))
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn claim_task(
&self,
ctx: &Context<'_>,
@@ -106,41 +73,20 @@ impl MutationRoot {
let assignee_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
crate::api::tasks::get_claimable_task(db, &id, caller)
.await
.map_err(crate::graphql::graphql_app_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
// Same classification REST's claim handler applies: a claim race is a
// client-safe conflict, a genuine sqlx failure stays opaque. Routing
// both transports through `task_write_conflict` is what stops a normal
// race from reaching a GraphQL caller as a generic database error
// (#327 review).
let task = db
.claim_task(&id, &assignee_did)
.await
.map_err(crate::graphql::graphql_claim_conflict)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(assignee_did.as_str())
.to_string();
crate::api::tasks::announce_task_event(
db,
tx,
TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
},
)
.await;
.map_err(crate::graphql::graphql_db_err)?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did: assignee_did,
at: Utc::now().to_rfc3339(),
});
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn complete_task(
&self,
ctx: &Context<'_>,
@@ -157,47 +103,32 @@ impl MutationRoot {
let by_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
// Authorize the actor: the task must be visible to the caller (returning
// not found for invisible tasks so existence is not leaked), and only
// the task's assignee may finish it.
let existing = crate::api::tasks::get_visible_task(db, &id, Some(caller))
// Authorize the actor: binding by_did to the signer is necessary but not
// sufficient — only the task's assignee may finish it.
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_app_err)?
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
"only the task assignee can complete it",
));
}
let task = db
.finish_task(&id, "completed", input.result.as_deref(), caller)
.finish_task(&id, "completed", input.result.as_deref())
.await
.map_err(crate::graphql::graphql_finish_conflict)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(by_did.as_str())
.to_string();
crate::api::tasks::announce_task_event(
db,
tx,
TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
new_status: "completed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
},
)
.await;
.map_err(crate::graphql::graphql_db_err)?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
new_status: "completed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
});
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn fail_task(
&self,
ctx: &Context<'_>,
@@ -214,44 +145,29 @@ impl MutationRoot {
let by_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
// Authorize the actor: the task must be visible to the caller (returning
// not found for invisible tasks so existence is not leaked), and only
// the task's assignee may fail it.
let existing = crate::api::tasks::get_visible_task(db, &id, Some(caller))
// Authorize the actor: only the task's assignee may fail it.
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_app_err)?
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
"only the task assignee can fail it",
));
}
let reason = input.reason.unwrap_or_default();
let task = db
.finish_task(&id, "failed", Some(&reason), caller)
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(crate::graphql::graphql_finish_conflict)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(by_did.as_str())
.to_string();
crate::api::tasks::announce_task_event(
db,
tx,
TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
new_status: "failed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
},
)
.await;
.map_err(crate::graphql::graphql_db_err)?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
new_status: "failed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
});
Ok(AgentTaskType::from(task))
}
}
@@ -270,94 +186,57 @@ mod tests {
.join("; ")
}
/// The `id` scalar returned by a successful `createTask` mutation.
fn created_id(resp: &Response) -> String {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::Object(task) = obj.get("createTask").expect("createTask key")
else {
panic!("createTask not an object");
};
let async_graphql::Value::String(id) = task.get("id").expect("id key") else {
panic!("id not a string");
};
id.clone()
}
/// N2: every GraphQL mutation requires a verified signer and binds the acting
/// DID to it. Unsigned is rejected; a signer other than the claimed actor is
/// N2: GraphQL mutations require a verified signer and bind the acting DID to
/// it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate.
#[sqlx::test]
async fn each_mutation_requires_and_binds_signer(pool: PgPool) {
async fn mutation_requires_and_binds_signer(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let schema = state.graphql_schema.as_ref();
let actor = "did:key:zGQLACTORAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let other = "did:key:zGQLOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let cases: Vec<(&str, String)> = vec![
(
"createTask",
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write" }}) {{ id }} }}"#
),
),
(
"claimTask",
format!(
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{actor}") {{ id }} }}"#
),
),
(
"completeTask",
format!(
r#"mutation {{ completeTask(id: "no-such-task", byDid: "{actor}", input: {{}}) {{ id }} }}"#
),
),
(
"failTask",
format!(
r#"mutation {{ failTask(id: "no-such-task", byDid: "{actor}", input: {{}}) {{ id }} }}"#
),
),
];
let assignee = "did:key:zASSIGNEEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let q = format!(
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{assignee}") {{ id }} }}"#
);
for (name, q) in &cases {
let resp = schema.execute(Request::new(q.as_str())).await;
assert!(
errors(&resp).contains("authentication required"),
"{name}: unsigned mutation must be rejected: {}",
errors(&resp)
);
// 1. Unsigned → rejected before any DB work.
let resp = schema.execute(Request::new(&q)).await;
assert!(
errors(&resp).contains("authentication required"),
"unsigned mutation must be rejected: {}",
errors(&resp)
);
let resp = schema
.execute(Request::new(q.as_str()).data(AuthenticatedDid(other.into())))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"{name}: DID mismatch must be rejected: {}",
errors(&resp)
);
// 2. Signed as someone other than the claimed assignee → rejected.
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(
"did:key:zOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB".into(),
)))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"DID mismatch must be rejected: {}",
errors(&resp)
);
let resp = schema
.execute(Request::new(q.as_str()).data(AuthenticatedDid(actor.into())))
.await;
let errs = errors(&resp);
assert!(
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"{name}: matching signer must pass the auth gate: {errs}"
);
if *name == "claimTask" {
assert!(
errs.contains("task not found"),
"claim race / missing task must keep its business message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"business error must not be rewritten as opaque DB error: {errs}"
);
}
}
// 3. Signed as the claimed assignee → passes the auth gate. The missing
// task is a business error from claim_task, not a sqlx fault, so the
// actionable message must survive (not the opaque DB string) (#250).
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"matching signer must pass the auth gate: {errs}"
);
assert!(
errs.contains("task not claimable"),
"claim race / missing task must keep its business message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"business error must not be rewritten as opaque DB error: {errs}"
);
}
/// #250: mutation DB faults must be opaque; create_task hits agent_tasks.
@@ -393,210 +272,6 @@ mod tests {
);
}
/// #327 review: `claimTask`/`completeTask`/`failTask` called
/// `graphql_db_err` directly on db-layer business failures, so an ordinary
/// claim race or stale finish reached GraphQL clients as a generic
/// database error while REST clients got an actionable conflict. All three
/// now route through the shared `task_write_conflict` classifier.
///
/// Deleting either `map_err` from a mutation turns this red: the message
/// becomes the raw anyhow text instead of the fixed conflict form.
#[sqlx::test]
async fn task_write_races_surface_as_conflicts_not_db_errors(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let schema = state.graphql_schema.as_ref();
let assignee = "did:key:zGQLRACEASSIGNEEAAAAAAAAAAAAAAAAAAAAAAA";
let rival = "did:key:zGQLRACERIVALBBBBBBBBBBBBBBBBBBBBBBBBBB";
let delegator = "did:key:zGQLRACEDELEGATORCCCCCCCCCCCCCCCCCCCCC";
let now = chrono::Utc::now().to_rfc3339();
state
.db
.create_repo(&crate::db::RepoRecord {
id: "race-repo".into(),
name: "race".into(),
owner_did: delegator.into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
disk_path: "/tmp/race-repo".into(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
state
.db
.create_task(&crate::db::AgentTask {
id: "race-task".into(),
repo_id: Some("race-repo".into()),
kind: "build".into(),
status: "pending".into(),
delegator_did: delegator.into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
})
.await
.unwrap();
// The assignee wins the claim.
let claim = |did: &str| {
format!(r#"mutation {{ claimTask(id: "race-task", assigneeDid: "{did}") {{ id }} }}"#)
};
let resp = schema
.execute(Request::new(claim(assignee)).data(AuthenticatedDid(assignee.into())))
.await;
assert!(resp.errors.is_empty(), "first claim: {}", errors(&resp));
// The rival loses the race: ineligible caller receives opaque not-found.
let resp = schema
.execute(Request::new(claim(rival)).data(AuthenticatedDid(rival.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not found"),
"ineligible caller after task is claimed must receive opaque not-found: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"ineligible claim is not a database failure: {errs}"
);
// A second claim that reaches the conditional SQL write (e.g. assignee claiming
// again on an already-claimed task) exercises the conflict mapper:
let resp = schema
.execute(Request::new(claim(assignee)).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not claimable: not found or already claimed"),
"a lost claim race must render the fixed conflict message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a claim race is not a database failure: {errs}"
);
// A stale finish on an already-completed task is the same class.
let complete = format!(
r#"mutation {{ completeTask(id: "race-task", byDid: "{assignee}", input: {{ result: "ok" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(&complete).data(AuthenticatedDid(assignee.into())))
.await;
assert!(resp.errors.is_empty(), "first complete: {}", errors(&resp));
let resp = schema
.execute(Request::new(&complete).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not found or not in claimed state"),
"a stale complete must render the fixed conflict message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a stale complete is not a database failure: {errs}"
);
let fail = format!(
r#"mutation {{ failTask(id: "race-task", byDid: "{assignee}", input: {{ reason: "nope" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(&fail).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not found or not in claimed state"),
"a stale fail must render the fixed conflict message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a stale fail is not a database failure: {errs}"
);
}
/// The other half of the same classification: a genuine SQL fault on the
/// write itself must stay opaque. Without this, routing conflicts through
/// `task_write_conflict` could be "fixed" by making every db failure a
/// client-visible conflict message.
#[sqlx::test]
async fn task_write_sql_faults_stay_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let assignee = "did:key:zGQLOPAQUEASSIGNEEAAAAAAAAAAAAAAAAAAAAA";
let delegator = "did:key:zGQLOPAQUEDELEGATORBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
state
.db
.create_task(&crate::db::AgentTask {
id: "opaque-task".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: delegator.into(),
assignee_did: Some(assignee.into()),
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
})
.await
.unwrap();
// Fault the write itself, not the schema. Dropping a column the
// visibility pre-check also SELECTs would fail in `get_visible_task`
// and never reach `graphql_claim_conflict`, so this test would pass
// against a claim mapper that leaks (#327 review). A BEFORE UPDATE
// trigger keeps every read valid and faults only inside
// `Db::claim_task`.
sqlx::raw_sql(
"CREATE FUNCTION gl_test_fault_task_update() RETURNS trigger
LANGUAGE plpgsql AS $fn$
BEGIN RAISE EXCEPTION 'gl_test_forced_update_fault'; END;
$fn$;
CREATE TRIGGER gl_test_fault_task_update
BEFORE UPDATE ON agent_tasks
FOR EACH ROW EXECUTE FUNCTION gl_test_fault_task_update();",
)
.execute(&pool)
.await
.unwrap();
let resp = state
.graphql_schema
.execute(
Request::new(format!(
r#"mutation {{ claimTask(id: "opaque-task", assigneeDid: "{assignee}") {{ id }} }}"#
))
.data(AuthenticatedDid(assignee.into())),
)
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a real sqlx fault on the write must stay opaque: {errs}"
);
assert!(
!errs.contains("task not claimable"),
"a write-time sqlx fault must not be reclassified as a claim race: {errs}"
);
assert!(
!errs.contains("gl_test_forced_update_fault") && !errs.contains("trigger"),
"database text leaked through the conflict mapper: {errs}"
);
}
/// Adversarial-review GATE-1 (GraphQL): completing a task requires being its
/// assignee, not merely signing as the by_did you pass. A signer who is not
/// the assignee is rejected even though the by_did binding passes; the
@@ -619,7 +294,7 @@ mod tests {
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed task");
@@ -636,69 +311,14 @@ mod tests {
)
};
// Stranger signs as themselves on a repo-less task they cannot see:
// invisible task returns "task not found" so existence is not leaked.
// Stranger signs as themselves and passes byDid=self (so the signer
// binding passes), but is not the assignee → rejected by authorization.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).contains("task not found"),
"an invisible task must return not found, got: {}",
errors(&resp)
);
// Seed a task on a public repo that stranger CAN see, but is not assignee of:
let pub_repo = crate::db::RepoRecord {
id: "pub-r".into(),
name: "pub-r".into(),
owner_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
disk_path: "/tmp/pub-r".into(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&pub_repo).await.expect("create repo");
let pub_task = crate::db::AgentTask {
id: "task-pub".into(),
repo_id: Some("pub-r".into()),
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state
.db
.create_task(&pub_task)
.await
.expect("seed pub task");
state
.db
.claim_task("task-pub", assignee)
.await
.expect("claim pub task");
let q_pub = |actor: &str| {
format!(
r#"mutation {{ completeTask(id: "task-pub", byDid: "{actor}", input: {{}}) {{ id status }} }}"#
)
};
let resp = schema
.execute(Request::new(q_pub(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).contains("assignee"),
"a non-assignee signer on a visible task must be rejected: {}",
"a non-assignee signer must be rejected: {}",
errors(&resp)
);
@@ -712,341 +332,4 @@ mod tests {
errors(&resp)
);
}
/// #496 (GraphQL): createTask applies the same repo-ownership gate as the
/// REST handler — a stranger naming a hosted repo is rejected without
/// persisting anything, while the owner files. The quarantined and
/// unknown-id arms are pinned too, including that each accepted write
/// stores the supplied id verbatim, so this copy of the gate cannot regress
/// silently.
#[sqlx::test]
async fn create_task_rejects_foreign_repo_id(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let owner = "did:key:zGQLTASKOWNERAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zGQLTASKSTRANGERBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
)
};
// Stranger signs as themselves (so the signer binding passes) but
// names the victim's repo → rejected by the ownership gate, leaking
// nothing about the repo.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::api::REPO_OWNER_ONLY_MESSAGE),
"a non-owner must not file under a foreign repo id (shared denial text): {errs}"
);
// The denial must not persist: no task row may carry the victim's
// repo id afterwards.
let stored = state
.db
.list_tasks_keyset(None, None, 50, None)
.await
.expect("list tasks");
assert!(
stored
.iter()
.all(|t| t.repo_id.as_deref() != Some(repo.id.as_str())),
"a denied write must not leave a task under the foreign repo id"
);
// The owner files under their own repo id.
let resp = schema
.execute(Request::new(q(owner)).data(AuthenticatedDid(owner.into())))
.await;
assert!(
errors(&resp).is_empty(),
"the owner should file against their repo: {}",
errors(&resp)
);
// Unknown repo ids stay accepted as opaque labels.
let q_unknown = format!(
r#"mutation {{ createTask(delegatorDid: "{stranger}", input: {{ kind: "build", capability: "repo:write", repoId: "no-such-repo-id" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(q_unknown).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"an id naming no hosted repo stays an opaque label: {}",
errors(&resp)
);
// Read back the row: the unresolved id must be stored verbatim, not
// dropped or rewritten to a repo it never named.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some("no-such-repo-id"),
"an unresolved id must be stored verbatim as a label"
);
// A quarantined repo id is treated as unknown (accepted, never a 403
// that would confirm the id names a real repo). This pins the
// `!quarantined` arm on this copy of the gate.
state
.db
.set_repo_quarantine(&repo.id, true)
.await
.expect("quarantine");
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"a quarantined repo id must not 403 (existence oracle): {}",
errors(&resp)
);
// Read back the row: the quarantined id is stored verbatim as a label
// too, so the accepted write is not silently dropped.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some(repo.id.as_str()),
"a quarantined id must be stored verbatim as a label"
);
}
/// #496 (GraphQL): a repo-lookup fault in createTask's gate must stay
/// opaque. Dropping a column `get_repo_by_id` selects faults that lookup;
/// the resolver must return the fixed DB message and leak no schema text
/// (#250), as `create_task_db_error_message_is_opaque` pins for the
/// agent_tasks fault.
#[sqlx::test]
async fn create_task_gate_repo_lookup_fault_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner = "did:key:zGQLTASKFAULTOWNERAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-fault-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-fault-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
sqlx::query("ALTER TABLE repos DROP COLUMN machine_id")
.execute(&pool)
.await
.expect("drop machine_id");
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{owner}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(owner.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"gate sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("machine_id") && !errs.contains("column"),
"schema text leaked through the gate: {errs}"
);
}
/// Pre-assigned GraphQL claimTask must reject a stranger (no UCAN leak)
/// and admit the reserved assignee.
#[sqlx::test]
async fn claim_task_honors_preassigned_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let reserved = "did:key:zGQLRESERVEDAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let thief = "did:key:zGQLTHIEFBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-reserved".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: Some(reserved.into()),
capability: "repo:write".into(),
ucan_token: Some("gql-ucan-secret".into()),
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ claimTask(id: "task-gql-reserved", assigneeDid: "{actor}") {{ id status ucanToken }} }}"#
)
};
let resp = schema
.execute(Request::new(q(thief)).data(AuthenticatedDid(thief.into())))
.await;
let errs = errors(&resp);
// The claim gate hides existence: an ineligible claimer gets the same
// not-found as a missing task, not a reserved deny that would leak it.
assert!(
errs.contains("task not found"),
"stranger claim must read as not-found, got: {errs}"
);
assert!(
!errs.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL errors: {errs}"
);
if let Ok(data) = resp.data.into_json() {
let s = data.to_string();
assert!(
!s.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL data on failed claim: {s}"
);
}
// Parity with REST: failed steal must leave the row untouched.
let still = state
.db
.get_task("task-gql-reserved")
.await
.expect("get")
.expect("task exists");
assert_eq!(
still.status, "pending",
"status must stay pending after steal"
);
assert_eq!(
still.assignee_did.as_deref(),
Some(reserved),
"assignee_did must stay reserved after steal"
);
let resp = schema
.execute(Request::new(q(reserved)).data(AuthenticatedDid(reserved.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim: {}",
errors(&resp)
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["claimTask"]["status"], "claimed");
assert_eq!(data["claimTask"]["ucanToken"], "gql-ucan-secret");
}
/// create_task normalizes a whitespace-only assignee to NULL (open task),
/// so the create response must show the stored row, not echo the input: a
/// client trusting the echoed "reservation" would believe a claimable task
/// is reserved. Before the fix the response was built from the pre-insert
/// struct and returned the padded input verbatim.
#[sqlx::test]
async fn create_task_reports_stored_assignee_not_input(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let caller = "did:key:zGQLCREATORAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let schema = state.graphql_schema.as_ref();
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{caller}", input: {{ kind: "build", capability: "repo:write", assigneeDid: " " }}) {{ id assigneeDid }} }}"#
);
let resp = schema
.execute(Request::new(q).data(AuthenticatedDid(caller.into())))
.await;
let data = resp.data.into_json().expect("json data");
assert!(
data["createTask"]["assigneeDid"].is_null(),
"whitespace-only assigneeDid must come back null (stored shape): {data}"
);
}
/// A task seeded with a whitespace-padded assignee must still be
/// completable by that assignee over GraphQL: REST and Db::finish_task
/// trim the stored value before comparing, and the GraphQL gate must do
/// the same or padded reservations finish only through REST.
#[sqlx::test]
async fn graphql_finish_accepts_padded_stored_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let assignee = "did:key:zGQLPADDEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-padded".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORDDDDDDDDDDDDDDDDDDDDDDDDDD".into(),
assignee_did: Some(format!("\t{assignee}\n")),
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let claim = format!(
r#"mutation {{ claimTask(id: "task-gql-padded", assigneeDid: "{assignee}") {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(claim).data(AuthenticatedDid(assignee.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim a padded reservation: {}",
errors(&resp)
);
let finish = format!(
r#"mutation {{ completeTask(id: "task-gql-padded", byDid: "{assignee}", input: {{}}) {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(finish).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.is_empty(),
"assignee must complete a padded reservation: {errs}"
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["completeTask"]["status"], "completed");
}
}
File diff suppressed because it is too large. Load diff
@@ -11,16 +11,15 @@ pub struct SubscriptionRoot;
#[Subscription]
impl SubscriptionRoot {
/// Live ref-update stream. `/graphql/ws` is mounted under the
/// `optional_signature` layer and carries optional caller identity, but this
/// resolver does not inspect it — it relays whatever enters the broadcast
/// Live ref-update stream. `/graphql/ws` is mounted outside the
/// `optional_signature` layer, so this resolver has NO caller identity and
/// cannot gate per-subscriber — it relays whatever enters the broadcast
/// channel to any anonymous client. Its visibility safety therefore rests
/// entirely on the WRITE side: the push handler only sends a
/// `RefUpdateBroadcast` for repos the anonymous public may read (inside its
/// `if announce` block, `api/repos.rs`). This is a single-point invariant —
/// any new sender to `ref_update_tx` MUST be `announce`-gated, or private-repo
/// ref metadata leaks here to unauthenticated subscribers (#112/#114 class).
#[graphql(complexity = "50 + child_complexity")]
async fn ref_updates(
&self,
ctx: &Context<'_>,
@@ -47,7 +46,6 @@ impl SubscriptionRoot {
})
}
#[graphql(complexity = "50 + child_complexity")]
async fn task_events(
&self,
ctx: &Context<'_>,
-67
View File
@@ -11,14 +11,6 @@ pub struct RepoType {
pub created_at: String,
}
#[derive(SimpleObject)]
pub struct RepoPageType {
pub nodes: Vec<RepoType>,
pub has_next_page: bool,
/// Position after the last returned visible repository; null for an empty page.
pub end_cursor: Option<String>,
}
#[derive(SimpleObject, Clone)]
pub struct AgentTaskType {
pub id: String,
@@ -56,65 +48,6 @@ impl From<AgentTask> for AgentTaskType {
}
}
/// Read-only projection of `AgentTask` for the `tasks`/`task` queries, as
/// opposed to `AgentTaskType`, which the task mutations (`createTask`,
/// `claimTask`, `completeTask`, `failTask` — all `require_signer`-gated)
/// return. Identical except for the missing `ucan_token` (#268): a read
/// surface never needs to echo it back, since the assignee already received it
/// at delegation/claim time via the mutation response.
#[derive(SimpleObject, Clone)]
pub struct AgentTaskReadType {
pub id: String,
pub repo_id: Option<String>,
pub kind: String,
pub status: String,
pub delegator_did: String,
pub assignee_did: Option<String>,
pub capability: String,
pub payload: Option<String>,
pub result: Option<String>,
pub created_at: String,
pub updated_at: String,
pub deadline: Option<String>,
}
/// Wraps a `tasks` page with the same completion signals the REST list route
/// exposes, so the two surfaces answer pagination identically.
///
/// `hasMore` and `incomplete` are separate because they are separate facts
/// (#327 review): `hasMore` says more candidates remain, `incomplete` says
/// this page is short *only* because the authorization scan hit its safety
/// wall. `nextCursor` is present exactly when `hasMore` is true and is an
/// opaque MAC'd token — it can name the last examined candidate, denied or
/// not, without disclosing it, which is what lets a caller page past a denied
/// window instead of stalling on it.
#[derive(SimpleObject, Clone)]
pub struct TaskPageType {
pub items: Vec<AgentTaskReadType>,
pub has_more: bool,
pub incomplete: bool,
pub next_cursor: Option<String>,
}
impl From<AgentTask> for AgentTaskReadType {
fn from(t: AgentTask) -> Self {
Self {
id: t.id,
repo_id: t.repo_id,
kind: t.kind,
status: t.status,
delegator_did: t.delegator_did,
assignee_did: t.assignee_did,
capability: t.capability,
payload: t.payload,
result: t.result,
created_at: t.created_at,
updated_at: t.updated_at,
deadline: t.deadline,
}
}
}
#[derive(SimpleObject, Clone)]
pub struct RefUpdateType {
pub repo: String,
+1 -166
View File
@@ -126,55 +126,15 @@ fn decode_key(b64url: &str) -> Option<VerifyingKey> {
VerifyingKey::from_bytes(&arr).ok()
}
/// Upper bound on the iCaptcha `/v1/pubkey` response body. The document is a
/// handful of keys at most; anything larger is refused rather than buffered
/// into memory (the unbounded `.json()` read this replaces, #507).
const KEY_MAX_BODY_BYTES: usize = 8 * 1024;
/// Read at most [`KEY_MAX_BODY_BYTES`] bytes of a key response. Returns `None`
/// when the body exceeds the cap, so an oversized document is treated as absent
/// exactly like a network error or a timeout (fail safe, never a partial key).
async fn read_key_body(mut resp: reqwest::Response) -> Option<Vec<u8>> {
let mut buf = Vec::with_capacity(1024);
loop {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (KEY_MAX_BODY_BYTES - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
return None; // exceeded the cap mid-chunk
}
}
Ok(None) => return Some(buf),
Err(_) => return None,
}
}
}
async fn fetch_key(url: &str) -> Option<VerifyingKey> {
let endpoint = format!("{}/v1/pubkey", url.trim_end_matches('/'));
// Bounded request: a hung /v1/pubkey must never block node startup. On
// timeout/error we return None and the gate stays inert (fail safe).
//
// No redirects and a capped body read: the key this fetch produces is the
// TRUST ANCHOR for the whole proof gate, so a 3xx must not be allowed to move
// where it comes from (anchor substitution / SSRF, matching the node's other
// outbound clients — `build_http_client` in main.rs, the sync worker), and
// the body is bounded rather than handed to `.json()`'s unbounded buffering.
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(5))
.redirect(reqwest::redirect::Policy::none())
.build()
.ok()?;
let resp = client.get(&endpoint).send().await.ok()?;
// A non-2xx response body is not a key: with Policy::none() a 3xx is
// surfaced rather than followed, but its own body must not be parsed
// either, or the redirect would move the anchor after all.
if !resp.status().is_success() {
return None;
}
let body = read_key_body(resp).await?;
let jwks: Jwks = serde_json::from_slice(&body).ok()?;
let jwks: Jwks = client.get(&endpoint).send().await.ok()?.json().await.ok()?;
decode_key(&jwks.keys.first()?.x)
}
@@ -687,129 +647,4 @@ mod tests {
let tampered = format!("{}.{}", payload, chars.into_iter().collect::<String>());
assert!(verify_token(&v, &tampered, SUB, IAT, false).is_err());
}
/// `{"keys":[{"x":"<base64url ed25519 point>"}]}` for the live `PUBKEY_X`.
fn jwks_body() -> Vec<u8> {
format!(r#"{{"keys":[{{"x":"{PUBKEY_X}"}}]}}"#).into_bytes()
}
/// The no-redirect / bounded-body client still parses a valid JWKS, so the
/// hardening did not break the happy path the gate depends on at startup.
#[tokio::test]
async fn fetch_key_parses_a_valid_jwks() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(jwks_body())
.create_async()
.await;
let key = fetch_key(&server.url()).await.expect("valid JWKS parses");
assert_eq!(key, decode_key(PUBKEY_X).unwrap());
}
/// A 302 must not move the trust anchor. The redirect target serves a REAL
/// key, so following the redirect would make `fetch_key` adopt it; instead
/// `Policy::none()` leaves the 302's (non-JWKS) body as the answer → None.
#[tokio::test]
async fn fetch_key_does_not_follow_a_redirect() {
let mut target = mockito::Server::new_async().await;
let _t = target
.mock("GET", "/v1/pubkey")
.expect(0)
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(jwks_body())
.create_async()
.await;
let mut redirector = mockito::Server::new_async().await;
// Location points at the target's /v1/pubkey, so a client that DID
// follow the redirect would fetch the real key and return Some: the
// assertion below only holds while Policy::none() is in place.
let _r = redirector
.mock("GET", "/v1/pubkey")
.with_status(302)
.with_header("Location", &format!("{}/v1/pubkey", target.url()))
.create_async()
.await;
assert!(
fetch_key(&redirector.url()).await.is_none(),
"a redirect must not be followed to a host serving a valid key"
);
}
/// A body past the cap is refused rather than buffered, so a broken
/// endpoint cannot make the node materialize an arbitrarily large document.
/// The body is a VALID JWKS (`jwks_body()`) padded with JSON whitespace past
/// the cap, so an unbounded read would still parse it; this test only passes
/// because the byte cap refuses it.
#[tokio::test]
async fn fetch_key_rejects_an_oversized_body() {
let mut server = mockito::Server::new_async().await;
let mut oversized = jwks_body();
oversized.resize(KEY_MAX_BODY_BYTES + 1, b' ');
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(oversized)
.create_async()
.await;
assert!(
fetch_key(&server.url()).await.is_none(),
"an oversized body must be refused even when it is a valid JWKS"
);
}
/// A body of exactly the cap still parses: the contract refuses bodies that
/// EXCEED it, not bodies that reach it (the padded JWKS is exactly
/// KEY_MAX_BODY_BYTES bytes of otherwise-valid JSON).
#[tokio::test]
async fn fetch_key_accepts_a_body_at_the_cap() {
let mut server = mockito::Server::new_async().await;
let mut at_cap = jwks_body();
at_cap.resize(KEY_MAX_BODY_BYTES, b' ');
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(at_cap)
.create_async()
.await;
let key = fetch_key(&server.url()).await.expect("at-cap JWKS parses");
assert_eq!(key, decode_key(PUBKEY_X).unwrap());
}
/// A non-2xx response never supplies the anchor, even when its body is a
/// perfectly valid JWKS: the redirect pin only stops the hop, and the
/// status gate is what stops the 3xx's own body from being adopted.
#[tokio::test]
async fn fetch_key_rejects_a_non_success_status() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(500)
.with_header("Content-Type", "application/json")
.with_body(jwks_body())
.create_async()
.await;
assert!(
fetch_key(&server.url()).await.is_none(),
"a non-2xx body must not be adopted as the anchor even when it parses"
);
}
/// An unreachable endpoint is the same fail-safe None as every other fetch
/// error: the gate stays inert rather than the node failing to start.
#[tokio::test]
async fn fetch_key_returns_none_when_the_endpoint_is_unreachable() {
assert!(fetch_key("http://127.0.0.1:1").await.is_none());
}
}
+58 -497
View File
@@ -922,12 +922,6 @@ async fn sweep_pass(
// Advance FIRST: every path below this line may skip the row, and none of them
// may wedge the walk (scenario 7).
last = sha.clone();
// Round-3 P1: skip Pinata-only rows whose local cid is NULL.
// The legacy repair walk re-keys `cid` from a provider CID to
// the raw resolver CID; a row with no local cid has no string
// to re-key, and skipping is the natural behavior. The cursor
// still advances so the walk does not loop on this row.
let Some(stored) = stored else { continue };
// Same cost gate as the skip-path repair: a canonical raw CIDv1 key is already
// the resolver key, so it reads no bytes and resolves no repo.
if gitlawb_core::cid::is_raw_cidv1(&stored) {
@@ -1463,97 +1457,6 @@ fn note_legacy_repair_read() {
/// have to be documented, validated, and kept meaningful.
pub const PIN_BATCH_BUDGET: Duration = Duration::from_secs(120);
/// A captured per-repo visibility-policy epoch that fences a pin batch.
///
/// The reconciliation sweep reads the epoch immediately before dispatching a
/// pin loop and passes a fence in; the loop re-reads the epoch before every
/// upload and aborts the batch the moment it moves. A visibility narrow that
/// lands mid-batch (a rule made private, a repo quarantined) must not let the
/// remaining pre-authorized objects still go to a public content-addressed
/// backend — the narrow is a policy change, and dispatching against the stale
/// snapshot is the exact irreversible-publication class this fence exists for
/// (R1-P1). `None` (the push path) means "no fence": the push derives its own
/// object list at admission and holds a write lease, so no sweep-style batch
/// snapshot crosses the dispatch boundary.
///
/// The fence deliberately does NOT hold any lock across the batch: a
/// visibility narrow (rule insert/remove, quarantine set) must commit
/// immediately, and the batch aborts on its next `is_current` check. Holding
/// a per-repo mutex from capture through drop would invert this — the narrow
/// would block behind the background sweep and every object in the batch
/// would still be sealed/posted to the reader being removed. The accepted
/// residual is a single in-flight object: a narrow that commits between the
/// pre-POST `is_current` check and the HTTP POST cannot be recalled, but the
/// next iteration aborts and the fenced DB record (row-locked against the
/// narrow's epoch bump) refuses to land the raced row as durable.
/// Multi-process / multi-node narrows are ordered by the same epoch column;
/// no in-process registry is involved.
///
/// Product decision (revocation model): prompt revocation wins over
/// publication atomicity. The suite intentionally allows one provider
/// upload already in flight when a rule narrows and requires every
/// LATER object to stop (`encrypt_and_pin_stops_sealing_when_reader_
/// removed_mid_batch`). A stronger contract — revocation commit
/// linearizing against every concurrent publication, or compensation
/// (unpin/delete) for an envelope that finishes after removal — is
/// explicitly OUT of scope until decided and documented here. Do not
/// reintroduce batch-spanning mutual exclusion to close the
/// single-in-flight window without that decision: it trades one
/// possibly-raced object for sealing the whole batch to a removed
/// reader.
#[derive(Clone)]
pub struct PolicyFence {
db: crate::db::Db,
repo_id: String,
epoch: i64,
}
impl PolicyFence {
/// Capture the current policy epoch for `repo_id`. A read failure is a
/// skip, not a retry-with-zero: the caller must not dispatch a batch it
/// cannot fence (fail closed on a stale allow).
pub async fn capture(db: &crate::db::Db, repo_id: &str) -> Option<Self> {
match db.repo_policy_epoch(repo_id).await {
Ok(epoch) => Some(PolicyFence {
db: db.clone(),
repo_id: repo_id.to_string(),
epoch,
}),
Err(e) => {
tracing::warn!(repo = %repo_id, err = %e, "policy-epoch read failed; not fencing pin batch");
None
}
}
}
/// Whether the repo's policy epoch is unchanged since capture. A read
/// failure is treated as "changed": never dispatch on a policy we cannot
/// prove current.
pub async fn is_current(&self) -> bool {
match self.db.repo_policy_epoch(&self.repo_id).await {
Ok(epoch) => epoch == self.epoch,
Err(_) => false,
}
}
/// The epoch value captured at `capture` time. Exposed so the
/// pinner can pass it to
/// `Db::record_pinned_cid_with_source_fenced` — the third
/// fence in the same transaction as the row insert.
/// Returning the field directly (rather than a `Option`)
/// matches the contract: a `PolicyFence` always has a
/// captured epoch; `is_current()` reports whether it still
/// matches.
pub fn captured_epoch(&self) -> i64 {
self.epoch
}
/// The repo this fence guards, for log correlation.
pub fn repo_id(&self) -> &str {
&self.repo_id
}
}
/// The smallest remainder worth starting a bounded git read (or an add) with.
///
/// A 1ms remainder otherwise buys a child spawned already past its deadline, which
@@ -1605,7 +1508,9 @@ pub async fn pin_git_object(
return Ok(String::new());
}
// Compute the expected CIDv1 from the content bytes
let expected_cid = Cid::from_git_object_bytes(data).to_string();
let url = format!(
"{}/api/v0/add?cid-version=1&raw-leaves=true&pin=true",
ipfs_api.trim_end_matches('/')
@@ -1648,23 +1553,10 @@ pub async fn pin_git_object(
// Kubo returns newline-delimited JSON; we only care about the last object
// (there's typically just one for a single-file add).
//
// The response MUST carry a real `Hash`: a misconfigured `GITLAWB_IPFS_API`
// (proxy returning HTML, health check on the wrong port, truncated gateway)
// can otherwise answer 2xx with no JSON, and falling back to the locally
// computed `expected_cid` would record a row for bytes the backend never
// stored. The reconciliation sweep trusts `pinned_cids` rows as durability
// evidence, so a silent false positive at pin time becomes a permanent blind
// spot for the backstop. A missing `Hash` fails the pin rather than recording
// a phantom row (mirrors Pinata's `data.cid` check).
let body = resp
.text()
.await
.map_err(|e| anyhow::anyhow!("IPFS add response body read failed: {e}"))?;
// A 200 without a parseable Hash is not a successful pin. Fabricating the
// caller-computed CID would record an address the backend never confirmed;
// above its chunk threshold that raw CID is not even a stored block, so
// every later cat fails permanently.
let cid = body
.lines()
.filter(|l| !l.trim().is_empty())
@@ -1672,31 +1564,8 @@ pub async fn pin_git_object(
let v: serde_json::Value = serde_json::from_str(line).ok()?;
v["Hash"].as_str().map(|s| s.to_string())
})
.next_back();
let cid = match cid {
Some(cid) => {
if cid != expected_cid {
tracing::warn!(
sha256 = %sha256_hex,
returned = %cid,
expected = %expected_cid,
"IPFS returned a different CID than computed locally (Kubo chunking may differ); recording the backend's answer"
);
}
cid
}
None => {
return Err(anyhow::anyhow!(
"IPFS /api/v0/add returned 2xx without a Hash field; refusing to record \
a CID the backend never acknowledged (misconfigured GITLAWB_IPFS_API?)"
));
}
};
// The string still has to be a CID. A garbage Hash would be recorded and
// later cat-ed the same way a missing one was, so reject it here.
Cid::from_str(&cid).map_err(|e| {
anyhow::anyhow!("IPFS /api/v0/add returned a Hash that is not a CID ({cid:.200}): {e}")
})?;
.next_back()
.unwrap_or(expected_cid.clone());
tracing::debug!(sha256 = %sha256_hex, %cid, "pinned git object to IPFS");
Ok(cid)
@@ -1774,8 +1643,8 @@ pub(crate) fn batch_budget_gate(
/// than [`PIN_READ_FLOOR`] left. It is a gate, not a hard ceiling, since a started
/// iteration still runs to completion;
/// - the git read: `store::read_object_bounded` runs under `spawn_blocking` against the
/// ABSOLUTE batch deadline (not the loop-top remainder, which the `has_ipfs_cid`
/// round-trip sitting between the two would push past it), with SIGTERM-then-SIGKILL
/// ABSOLUTE batch deadline (not the loop-top remainder, which the `is_pinned` round-trip
/// sitting between the two would push past it), with SIGTERM-then-SIGKILL
/// process-group teardown, so a hung `git cat-file` costs this batch its remaining
/// budget plus one watchdog teardown instead of holding the permit for the child's
/// whole lifetime and blocking a runtime worker while it does;
@@ -1819,11 +1688,10 @@ pub(crate) fn batch_budget_gate(
/// # Truncation semantics
///
/// A batch stopped at the deadline leaves its remaining objects unpinned, and
/// nothing sweeps them up afterwards on the push path; recovery is opportunistic
/// (a later full-scan push re-offers the skipped OIDs). The reconciliation
/// sweep is the systematic backstop: when it is enabled and a pin backend is
/// configured, it re-derives the public object set each pass and fills any
/// remaining gap.
/// nothing sweeps them up afterwards. There is no reconciliation pass over
/// `pinned_cids`; recovery is opportunistic, happening only if some later push
/// on the repo takes the full-scan fallback (`push_delta::list_all_objects`) and
/// re-derives the whole object set, which then re-offers the skipped OIDs.
///
/// The twin in `pinata.rs` is back at parity on everything that bounds or repairs an
/// object: it runs the same shared budget gate at the top of every iteration, the same
@@ -1840,29 +1708,6 @@ pub(crate) fn batch_budget_gate(
///
/// Returns a list of `(sha256_hex, cid)` pairs pinned AND durably recorded this
/// call.
/// What one `pin_new_objects` call (IPFS or Pinata twin) did, with the
/// three backend states kept apart instead of inferred from each other:
///
/// - `confirmed`: `(sha, provider_cid)` pairs whose DB record durably
/// landed. ONLY these may advance `gaps_filled`, branch/gossip CID
/// state, or any "repaired" bookkeeping. A provider upload whose
/// record timed out, failed, or was refused by the fence is absent
/// here even though the bytes may sit on the provider.
/// - `last_attempted`: the last OID whose loop body was entered — i.e.
/// the backend did real work for it (reads, an upload attempt, or a
/// skip-branch decision), whether that work succeeded, failed, or hit
/// an unknown outcome. `None` when nothing was entered (empty input,
/// immediate fence abort, or immediate budget gate). Durable
/// continuation cursors advance to this, never to the tail of the
/// planned vector: the tail may never have been visited, and
/// promoting it would rotate an untouched suffix behind the backlog
/// forever.
#[derive(Debug)]
pub struct PinBatchOutcome {
pub confirmed: Vec<(String, String)>,
pub last_attempted: Option<String>,
}
// Eight because #173's git seam (`git_bin`, `git_timeout`) and pin provenance
// (`repo_id`) sit alongside #174's batch budget. All four callers pass every one, and
// grouping them into a context struct would add a type whose only job is to be
@@ -1877,34 +1722,16 @@ pub async fn pin_new_objects(
db: &crate::db::Db,
repo_id: &str,
batch_budget: Duration,
fence: Option<&PolicyFence>,
) -> PinBatchOutcome {
) -> Vec<(String, String)> {
if ipfs_api.is_empty() {
return PinBatchOutcome {
confirmed: Vec::new(),
last_attempted: None,
};
return vec![];
}
let deadline = Instant::now() + batch_budget;
let total = object_list.len();
let mut pinned = Vec::new();
let mut last_attempted: Option<String> = None;
for (attempted, sha) in object_list.into_iter().enumerate() {
// Policy fence (R1-P1): a visibility narrow that lands after the caller
// built this batch must abort it before the next irreversible upload.
// Checked FIRST so a changed policy costs nothing beyond the read.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id,
unattempted = total - attempted,
"visibility policy changed mid-batch; stopping the pin loop"
);
break;
}
}
// Top of the iteration, before any of this object's work: an object is
// never started with a remainder too small to cover a bounded read's
// teardown. Consumed as a guard only: the read below runs against the
@@ -1913,40 +1740,19 @@ pub async fn pin_new_objects(
if batch_budget_gate("IPFS", deadline, pinned.len(), total - attempted).is_none() {
break;
}
// Attempt progress (not a durability claim): from here the loop does
// real work for this OID — skip-branch reads, airgapped git reads,
// an upload attempt — whatever the outcome. The caller persists
// this as the continuation, never the planned vector's tail.
last_attempted = Some(sha.clone());
// Skip if the object is ALREADY a real local IPFS pin, but first
// backfill provenance if the existing pin has none. A legacy pin
// (recorded before repo_id existed, #173, jatmn) is skipped here
// before record_pinned_cid ever runs, so its NULL provenance would
// never resolve to one repo and known CIDs keep hitting the scan.
// The backfill only sets repo_id (AND repo_id IS NULL guard
// preserves first-pinner-owns) and never re-pins the bytes: the
// object is already on IPFS.
//
// #218 review P1a: this check keys on `has_ipfs_cid` (writer-owned
// `local_ipfs_provenance = TRUE`), NOT on row existence
// (`is_pinned`). A Pinata-only row is `is_pinned = true` but
// `has_ipfs_cid = false`: the bytes never reached the local IPFS
// daemon, only Pinata, and we MUST fall through to the local
// writer path so a real local pin lands. Using `is_pinned` here
// was the gap that made the Pinata-only → local-IPFS repair
// path inert: every sweep pass re-entered this arm, recorded
// the source, and continued without ever calling
// `pin_git_object` or `record_pinned_cid_with_source`. The flag
// stayed FALSE forever.
//
// Every DB call from here to the end of the iteration is bounded
// by the ABSOLUTE batch deadline (F3, #173): the loop runs under
// a global pin permit and a bare await parked it for the whole
// stall. The elapsed arm is mapped per site below, never as a
// blanket "existing error arm": a timeout cancels the client
// future but not the statement Postgres is running, so it
// reports an UNKNOWN outcome, not a failed write.
match db_bounded(deadline, db.has_ipfs_cid(&sha)).await {
// Skip if already pinned, but first backfill provenance if the existing
// pin has none. A legacy pin (recorded before repo_id existed, #173, jatmn)
// is skipped here before record_pinned_cid ever runs, so its NULL provenance
// would never resolve to one repo and known CIDs keep hitting the scan. The
// backfill only sets repo_id (AND repo_id IS NULL guard preserves
// first-pinner-owns) and never re-pins the bytes: the object is already on IPFS.
// Every DB call from here to the end of the iteration is bounded by the
// ABSOLUTE batch deadline (F3, #173): the loop runs under a global pin permit
// and a bare await parked it for the whole stall. The elapsed arm is mapped per
// site below, never as a blanket "existing error arm": a timeout cancels the
// client future but not the statement Postgres is running, so it reports an
// UNKNOWN outcome, not a failed write.
match db_bounded(deadline, db.is_pinned(&sha)).await {
Ok(true) => {
// Elapsed here is free to skip: these are reads, so a late server-side
// completion costs nothing, and the backfill's own `AND repo_id IS NULL`
@@ -2056,7 +1862,7 @@ pub async fn pin_new_objects(
}
Ok(false) => {}
Err(e) => {
tracing::warn!(sha = %sha, err = %e, "DB error checking IPFS pinned status");
tracing::warn!(sha = %sha, err = %e, "DB error checking pinned status");
continue;
}
}
@@ -2070,7 +1876,7 @@ pub async fn pin_new_objects(
// own deadline regardless.
//
// The read runs against the ABSOLUTE batch deadline, not against the remainder
// measured at the top of the iteration: the `has_ipfs_cid` round-trip above sits
// measured at the top of the iteration: the `is_pinned` round-trip above sits
// between the two, so `Instant::now() + budget_left` would land past `deadline`
// by however long the DB took, and under a saturated pool that is the dominant
// term. A slow DB check must not push the read's own bound out.
@@ -2155,24 +1961,6 @@ pub async fn pin_new_objects(
break;
};
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the has_ipfs_cid round-trip or the bounded Git read — both of
// which can take seconds and during which a quarantine or rule change may
// have committed. Without this, stale plaintext can start uploading
// under authorization that is no longer current.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id,
unattempted = total - attempted,
"visibility policy changed during preparation; aborting IPFS upload"
);
break;
}
}
// Pin to IPFS
match pin_git_object(ipfs_api, &sha, &data, Some(add_timeout)).await {
Ok(cid) if !cid.is_empty() => {
@@ -2222,31 +2010,7 @@ pub async fn pin_new_objects(
// per-object failure.
match db_bounded(
db_record_deadline(deadline),
retry_db_record(|| {
// #218 review round 9 (guidance #3 —
// linearization): always go through the
// fenced form. The fence is either
// captured (sweep / public-pin path: the
// third fence is the linearization point
// that closes the rule-write /
// record-write race) or absent
// (push-side admission where the
// decision is made at request time —
// we pass `i64::MAX` as a sentinel that
// the fenced form treats as "no fence
// check"). The 3-arg
// `record_pinned_cid_with_source` is
// still available for tests that don't
// own a fence, but the production
// pinner routes through here.
let fence_epoch = fence.map(|f| f.captured_epoch()).unwrap_or(i64::MAX);
db.record_pinned_cid_with_source_fenced(
&sha,
&raw_cid,
repo_id,
fence_epoch,
)
}),
retry_db_record(|| db.record_pinned_cid_with_source(&sha, &raw_cid, repo_id)),
)
.await
{
@@ -2263,10 +2027,7 @@ pub async fn pin_new_objects(
}
}
PinBatchOutcome {
confirmed: pinned,
last_attempted,
}
pinned
}
#[cfg(test)]
@@ -2462,16 +2223,16 @@ mod tests {
endpoint
}
/// A sleeping-but-live endpoint. Answers `200` with a conformant NDJSON add
/// response after `delays[i]` for the i-th request it accepts (the last
/// entry repeats), so a test can make one add slow and the next fast.
/// Drains the full request, headers plus the declared `Content-Length`
/// body, before sleeping: exactly as in `rejecting_endpoint`, answering
/// early and closing would surface as a write failure on the client and
/// turn a slow-but-healthy add into a different failure shape.
/// A sleeping-but-live endpoint. Answers `200` with an empty body after
/// `delays[i]` for the i-th request it accepts (the last entry repeats), so
/// a test can make one add slow and the next fast. Drains the full request,
/// headers plus the declared `Content-Length` body, before sleeping: exactly
/// as in `rejecting_endpoint`, answering early and closing would surface as
/// a write failure on the client and turn a slow-but-healthy add into a
/// different failure shape.
///
/// The Hash is a fixed, real CID: the timing tests assert on pin counts,
/// not on the recorded address.
/// An empty body is a successful pin: `pin_git_object` falls back to the CID
/// it computed from the bytes when the response carries no `Hash`.
async fn delaying_endpoint(delays: Vec<Duration>) -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
@@ -2508,14 +2269,9 @@ mod tests {
}
}
tokio::time::sleep(delay).await;
let body = br#"{"Hash":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4","Size":"12"}
"#;
let resp = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/x-ndjson\r\nContent-Length: {}\r\n\r\n",
body.len()
);
let _ = sock.write_all(resp.as_bytes()).await;
let _ = sock.write_all(body).await;
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n")
.await;
let _ = sock.flush().await;
});
}
@@ -2602,74 +2358,6 @@ mod tests {
);
}
/// The misconfigured-`GITLAWB_IPFS_API` false positive (P3): a 2xx response
/// that carries no `Hash` field (proxy returning HTML, health check on the
/// wrong port, truncated gateway) must FAIL the pin, not fall back to the
/// locally computed `expected_cid`. Falling back records a `pinned_cids`
/// row for bytes the backend never stored, and the reconciliation sweep
/// trusts rows as durability evidence — so the false positive becomes a
/// permanent blind spot for the backstop. A missing `Hash` must surface as
/// an explicit error, never a successful pin.
#[tokio::test]
async fn pin_git_object_rejects_a_2xx_without_a_hash_field() {
let endpoint = empty_ok_endpoint().await;
let inner = tokio::time::timeout(
Duration::from_secs(30),
pin_git_object(&endpoint, "deadbeef", b"some object bytes\n", None),
)
.await
.expect("wedge guard: an immediate empty 200 cannot take 30s");
let err = inner.expect_err(
"a 2xx without a Hash field must not surface as a successful pin \
(would record a phantom pinned_cids row the sweep then trusts)",
);
assert!(
err.to_string().contains("without a Hash field"),
"the error must name the missing Hash so operators diagnose the endpoint: {err:#}"
);
}
/// A 200 that answers with an empty body and no `Hash` — the exact shape of
/// a proxy or health-check endpoint mistaken for a Kubo API.
async fn empty_ok_endpoint() -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
while let Ok((mut sock, _)) = listener.accept().await {
tokio::spawn(async move {
let mut acc = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
acc.extend_from_slice(&buf[..n]);
if let Some(hdr_end) =
acc.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4)
{
let headers = String::from_utf8_lossy(&acc[..hdr_end]).to_lowercase();
let len: usize = headers
.lines()
.find_map(|l| l.strip_prefix("content-length:"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0);
if acc.len() >= hdr_end + len {
break;
}
}
}
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n")
.await;
let _ = sock.flush().await;
});
}
});
endpoint
}
/// The second unhardened sink, reached from `sync.rs`. Same shape as above.
#[tokio::test]
async fn cat_against_silent_endpoint_errors_within_its_own_timeout() {
@@ -2685,71 +2373,6 @@ mod tests {
);
}
/// A 200 add response with no parseable Hash must surface as an add
/// failure. Falling back to the caller-computed CID records an
/// unverifiable address: above the backend's chunk threshold that raw
/// CID is not a stored block, so every later cat 500s.
#[tokio::test]
async fn pin_git_object_rejects_a_200_without_a_hash() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", mockito::Matcher::Regex(r"^/api/v0/add".to_string()))
.with_status(200)
.with_body(r#"{"Name":"object","Size":"19"}"#)
.create_async()
.await;
let result = pin_git_object(&server.url(), "deadbeef", b"some object bytes\n", None).await;
assert!(
result.is_err(),
"a 200 without a Hash must be an add failure, not a fabricated CID: {result:?}"
);
}
/// A Hash that is a string but not a CID must also fail. It would be
/// recorded into `encrypted_blobs.cid` and every later cat would miss.
#[tokio::test]
async fn pin_git_object_rejects_a_malformed_hash() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", mockito::Matcher::Regex(r"^/api/v0/add".to_string()))
.with_status(200)
.with_body(r#"{"Hash":"not-a-cid","Size":"19"}"#)
.create_async()
.await;
let result = pin_git_object(&server.url(), "deadbeef", b"some object bytes\n", None).await;
assert!(
result.is_err(),
"a Hash that is not a CID must be an add failure: {result:?}"
);
}
/// The Hash text goes into the error and `encrypted_pin` logs that error,
/// so a backend answering with a huge invalid Hash must not produce an
/// unbounded log entry.
#[tokio::test]
async fn pin_git_object_bounds_an_oversized_invalid_hash_in_the_error() {
let long = "x".repeat(5000);
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", mockito::Matcher::Regex(r"^/api/v0/add".to_string()))
.with_status(200)
.with_body(format!(r#"{{"Hash":"{long}","Size":"19"}}"#))
.create_async()
.await;
let result = pin_git_object(&server.url(), "deadbeef", b"some object bytes\n", None).await;
let err = result.expect_err("an invalid Hash must be an add failure");
let msg = err.to_string();
assert!(
msg.len() < 1000,
"the error must bound the echoed Hash, got {} bytes",
msg.len()
);
assert!(
!msg.contains(&long),
"the error must not carry the full invalid Hash"
);
}
/// The permit-hold bound. `pin_new_objects` runs under a deferring
/// `pin_semaphore`, so without a batch deadline the hold is O(N) with N
/// chosen by the pusher. Five objects against an endpoint that takes 2s
@@ -2782,16 +2405,15 @@ mod tests {
&db,
"repo-batch-budget",
Duration::from_millis(5500),
None,
),
)
.await
.expect("wedge guard: a 5.5s budget cannot take 30s");
assert!(
(1..=3).contains(&pinned.confirmed.len()),
(1..=3).contains(&pinned.len()),
"the batch must stop partway, not pin all five and not stall on the first: pinned {}",
pinned.confirmed.len()
pinned.len()
);
let text = logs.text();
let warns: Vec<&str> = text
@@ -2813,52 +2435,9 @@ mod tests {
})
.unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}"));
assert!(
unattempted >= 1 && unattempted + pinned.confirmed.len() <= 5,
unattempted >= 1 && unattempted + pinned.len() <= 5,
"unattempted={unattempted} with {} pinned is not a partial batch of five",
pinned.confirmed.len()
);
}
/// The outcome reports the last object actually ENTERED, not the planned
/// tail: the first upload hangs 6s against a ~2s per-request timeout, so
/// it fails, and the loop-top gate breaks the batch before the second
/// object starts. `last_attempted` must be the first OID — the only one
/// the loop body entered — even though nothing was confirmed and two
/// OIDs were never visited. A cursor persisted from `to_pin.last()`
/// would rotate the untouched suffix behind the backlog forever.
#[sqlx::test]
async fn pin_new_objects_reports_last_attempted_not_planned_tail(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("attempted.git");
let oids = seed_loose_blobs(&repo_path, 3);
let endpoint = delaying_endpoint(vec![Duration::from_secs(6)]).await;
let outcome = tokio::time::timeout(
Duration::from_secs(30),
pin_new_objects(
&endpoint,
&repo_path,
"git",
Duration::from_secs(30),
oids.clone(),
&db,
"repo-attempted",
Duration::from_secs(2),
None,
),
)
.await
.expect("a 2s budget cannot take 30s");
assert_eq!(
outcome.last_attempted,
Some(oids[0].clone()),
"only the first object was entered; the planned tail was never visited"
);
assert!(
outcome.confirmed.is_empty(),
"the hung upload timed out, so nothing was confirmed"
pinned.len()
);
}
@@ -2893,13 +2472,12 @@ mod tests {
&db,
"repo-batch-continues",
Duration::from_secs(90),
None,
),
)
.await
.expect("wedge guard: a 13s add plus an immediate one cannot take 60s");
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
2,
"a slow but progressing endpoint must pin both objects: an upload past the client's \
10s default is not a dead endpoint"
@@ -2932,12 +2510,11 @@ mod tests {
&db,
"repo-batch-rejects",
Duration::from_secs(60),
None,
),
)
.await
.expect("a rejecting endpoint answers immediately, so this cannot take 30s");
assert!(pinned.confirmed.is_empty(), "every add was rejected");
assert!(pinned.is_empty(), "every add was rejected");
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
4,
@@ -2980,7 +2557,6 @@ mod tests {
/// type probe carries its oid on stdin rather than in argv, so an oid appears in
/// the log only once an object has already got past its probe, and a healthy
/// object costs two invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log)
.unwrap_or_default()
@@ -3031,7 +2607,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(2),
None,
),
)
.await
@@ -3042,7 +2617,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -3119,7 +2694,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_millis(1500),
None,
),
)
.await
@@ -3134,7 +2708,7 @@ mod tests {
that can only be spawned and reaped"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"the first object is inside the budget and must pin: {pinned:?}"
);
@@ -3193,7 +2767,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -3203,7 +2776,7 @@ mod tests {
if genuinely_unreadable {
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"nothing can be pinned through a store that cannot be read: {pinned:?}"
);
assert_eq!(
@@ -3268,7 +2841,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -3284,7 +2856,7 @@ mod tests {
must still be attempted, got {attempted} of {}",
oids.len()
);
let pinned_shas: Vec<&String> = pinned.confirmed.iter().map(|(sha, _)| sha).collect();
let pinned_shas: Vec<&String> = pinned.iter().map(|(sha, _)| sha).collect();
let expected: Vec<&String> = oids.iter().filter(|o| !tainted.contains(o)).collect();
assert_eq!(
pinned_shas, expected,
@@ -3337,7 +2909,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -3349,7 +2920,7 @@ mod tests {
"an object-scoped fault must not stop the batch: every object must be read"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
4,
"one corrupt object must cost only itself: the other four must still pin"
);
@@ -3572,7 +3143,6 @@ mod tests {
&db,
"repo-stalled-db",
Duration::from_millis(1500),
None,
),
)
.await
@@ -3583,7 +3153,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a stalled pinned-status check cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -3643,7 +3213,6 @@ mod tests {
&db,
"repo-skip-stalled",
Duration::from_millis(1500),
None,
),
)
.await
@@ -3654,7 +3223,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"an already-pinned object is skipped, never re-pinned: {pinned:?}"
);
assert!(
@@ -3710,17 +3279,13 @@ mod tests {
&db,
"repo-multi-stalled",
Duration::from_millis(1500),
None,
),
)
.await
.expect("three stalled objects must still cost one budget, not three");
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
"nothing can pin against a stalled DB"
);
assert!(pinned.is_empty(), "nothing can pin against a stalled DB");
assert!(
elapsed < Duration::from_secs(3),
"three stalled objects must charge ONE budget (1.5s), not one each; got {elapsed:?}"
@@ -3743,7 +3308,7 @@ mod tests {
///
/// The lock time is a MARGIN, not a boundary: taking it at 100ms left `is_pinned`
/// racing it on a loaded box, and losing that race makes the read block, time out,
/// and break the batch, which fails on `pinned.confirmed.len() == 1` for a reason that has
/// and break the batch, which fails on `pinned.len() == 1` for a reason that has
/// nothing to do with the floor. Any time between the `is_pinned` round trip and
/// the add's 1.7s return proves the same thing.
#[sqlx::test]
@@ -3781,7 +3346,6 @@ mod tests {
&db,
"repo-spent-budget",
Duration::from_millis(2000),
None,
),
);
let (pinned, ()) = tokio::join!(pin, locker);
@@ -3794,7 +3358,7 @@ mod tests {
nothing can resolve the CID"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"the durably recorded pin must be returned: {pinned:?}"
);
@@ -3855,7 +3419,6 @@ mod tests {
&db,
"repo-definite-error",
Duration::from_millis(1200),
None,
),
);
let (pinned, ()) = tokio::join!(pin, commit);
@@ -3937,7 +3500,6 @@ mod tests {
&db,
"repo-marker-floor",
Duration::from_millis(1500),
None,
),
);
let (pinned, ()) = tokio::join!(pin, controller);
@@ -3948,7 +3510,7 @@ mod tests {
drop(sources_lock);
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"an already-pinned object is skipped, never re-pinned: {pinned:?}"
);
assert!(
@@ -4042,7 +3604,6 @@ mod tests {
&db,
"repo-repair-stalled",
Duration::from_millis(2200),
None,
),
);
let (pinned, mut cids_lock) = tokio::join!(pin, controller);
+55 -352
View File
@@ -1,13 +1,10 @@
mod ans104;
mod api;
mod arweave;
mod arweave_v2;
mod auth;
mod bootstrap;
mod cert;
mod config;
mod db;
mod durable_outbox;
mod encrypted_pin;
mod error;
mod git;
@@ -19,15 +16,12 @@ mod operator;
mod p2p;
mod pinata;
mod rate_limit;
mod reconciliation;
mod server;
mod state;
mod sync;
#[cfg(test)]
mod test_support;
mod upstream_mirror;
mod visibility;
mod webhook_secret;
mod webhooks;
use anyhow::{anyhow, Context, Result};
@@ -55,7 +49,6 @@ use state::AppState;
struct DegradedState {
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
}
/// Two independent counters with no cross-field invariant — atomics, not a
@@ -91,7 +84,7 @@ async fn main() -> Result<()> {
if !config.public_read {
warn!(
"GITLAWB_PUBLIC_READ=false has no effect; reads are gated per repository by is_public and path-scoped visibility rules, not this flag"
"GITLAWB_PUBLIC_READ=false is reserved; per-repository private-read enforcement is not wired in alpha"
);
}
@@ -100,7 +93,6 @@ async fn main() -> Result<()> {
// Sealing key for the legacy-scan continuation tokens, DERIVED from the identity
// just loaded so it is the same key after a restart (see `derive_scan_token_key`).
let scan_token_key = AppState::derive_scan_token_key(&keypair);
let webhook_secret_key = AppState::derive_webhook_secret_key(&keypair);
let node_did = keypair.did();
// One-time metrics init. Must run before any handler that calls into
@@ -165,7 +157,6 @@ async fn main() -> Result<()> {
degraded_listener,
node_did.to_string(),
Arc::clone(&db_startup),
config.web_url.clone(),
db_ready_rx,
shutdown_tx.subscribe(),
));
@@ -311,15 +302,10 @@ async fn main() -> Result<()> {
let (ref_update_tx, _) = tokio::sync::broadcast::channel::<state::RefUpdateBroadcast>(256);
let (task_event_tx, _) = tokio::sync::broadcast::channel::<state::TaskEventBroadcast>(256);
// Node-keyed, so continuation tokens this node mints are only accepted by
// this node and survive its restarts without any configured secret.
let task_cursor_key = api::task_cursor::TaskCursorKey::derive(&keypair.to_seed());
let graphql_schema = Arc::new(graphql::build_schema(
Arc::clone(&db),
ref_update_tx.clone(),
task_event_tx.clone(),
task_cursor_key.clone(),
));
let machine_id = std::env::var("FLY_MACHINE_ID").ok();
@@ -360,14 +346,9 @@ async fn main() -> Result<()> {
);
// Per-DID limiter for the creation endpoints. Keyed on the authenticated
// DID (attacker-varied), so bound its key set to cap memory. Max and window
// are operator-configurable via GITLAWB_RATE_LIMIT_MAX /
// GITLAWB_RATE_LIMIT_WINDOW_SECS; both reject 0 at parse time.
let rate_limiter = rate_limit::RateLimiter::new_bounded(
config.rate_limit_max_requests as usize,
std::time::Duration::from_secs(config.rate_limit_window_secs),
200_000,
);
// DID (attacker-varied), so bound its key set to cap memory.
let rate_limiter =
rate_limit::RateLimiter::new_bounded(10, std::time::Duration::from_secs(3600), 200_000);
// Per-client-IP flood brake for the creation endpoints. The per-DID limiter
// above is bypassed by a DID farm (one throwaway did:key per repo), which is
@@ -446,8 +427,6 @@ async fn main() -> Result<()> {
std::time::Duration::from_secs(3600),
200_000,
);
let federated_rate_limiter =
rate_limit::RateLimiter::new_bounded(12, std::time::Duration::from_secs(60), 10_000);
if config.sync_trigger_rate_limit == 0 {
tracing::warn!(
"GITLAWB_SYNC_TRIGGER_RATE_LIMIT=0 — /sync/trigger IP rate limiting disabled"
@@ -465,7 +444,6 @@ async fn main() -> Result<()> {
db,
node_did: node_did.clone(),
node_keypair: keypair,
task_cursor_key,
p2p: p2p_handle,
http_client,
ref_update_tx,
@@ -483,12 +461,10 @@ async fn main() -> Result<()> {
ipfs_max_legacy_scan_rows: AppState::ipfs_legacy_scan_row_budget(&config),
ipfs_max_legacy_scan_rule_bytes: crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
webhook_secret_key: Arc::new(webhook_secret_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust,
sync_trigger_rate_limiter,
peer_write_rate_limiter,
federated_rate_limiter,
shutdown_tx: shutdown_tx.clone(),
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(config.max_concurrent_git_ops)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(
@@ -565,36 +541,11 @@ async fn main() -> Result<()> {
std::time::Duration::from_secs(3600),
200_000,
),
task_read_rate_limiter: rate_limit::RateLimiter::new_bounded(
config.task_read_rate_limit,
std::time::Duration::from_secs(3600),
200_000,
),
git_bin: "git".to_string(),
};
// One-time re-encryption of any webhook HMAC secrets still resting as
// plaintext (written before at-rest encryption shipped). A SQL migration
// cannot do it — Postgres never sees the node-held key — so it runs here.
match state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
{
Ok(0) => {}
Ok(n) => info!(
count = n,
"re-encrypted legacy webhook HMAC secrets at rest"
),
Err(e) => warn!(err = %e, "failed to re-encrypt legacy webhook secrets"),
}
if config.ipfs_rate_limit == 0 {
tracing::warn!("GITLAWB_IPFS_RATE_LIMIT=0 — per-IP /ipfs rate limiting disabled");
}
if config.task_read_rate_limit == 0 {
tracing::warn!("GITLAWB_TASK_READ_RATE_LIMIT=0 — per-IP task read rate limiting disabled");
}
// Periodic peer-count poll for the metrics gauge. If p2p is disabled
// we still set the gauge to 0 so dashboards don't show "no data".
@@ -651,13 +602,6 @@ async fn main() -> Result<()> {
}
let _legacy_cid_sweep = spawn_legacy_cid_sweep(&state, &config);
let _queue_lifecycle_sweep = spawn_queue_lifecycle_sweep(&state, &config);
// NOTE: the due-request worker is spawned AFTER the one-shot startup
// reconcile/drain below, not here: its 5s interval would otherwise tick
// during operator/PoS setup and interleave with boot recovery on backlog
// rows (state gates prevent data loss, but the interleave costs recovery
// latency and widens the reconcile/drain race window).
let _marker_cleanup_worker = spawn_marker_cleanup_worker(&state);
let router = server::build_router(state.clone());
// Re-register the socket bound at startup — same fd, so there was never a
@@ -715,46 +659,6 @@ async fn main() -> Result<()> {
info!("auto-sync worker started");
}
// External-forge mirroring is a separate, default-off worker. Startup
// validation already requires owner-only pushes when it is enabled; the
// worker additionally verifies the Git runtime can pin DNS answers before
// any URL reaches `git fetch`.
if config.upstream_mirror_enabled {
upstream_mirror::start(
Arc::clone(&state.db),
Arc::clone(&state.config),
state.repo_store.clone(),
state.repo_write_leases.clone(),
Arc::clone(&state.git_write_semaphore),
state.subscribe_shutdown(),
)
.context("starting upstream mirror worker")?;
}
// Periodic reconciliation sweep: re-derives pin/seal sets and fills gaps
// so a dropped replication job never means data loss.
{
let db = state.db.clone();
let config = Arc::clone(&state.config);
let http_client = Arc::clone(&state.http_client);
let node_keypair = Arc::clone(&state.node_keypair);
let node_did = state.node_did.clone();
let pin_sem = Arc::clone(&state.pin_semaphore);
let shutdown_rx = state.subscribe_shutdown();
if reconciliation::spawn(
db,
config,
http_client,
node_keypair,
node_did,
pin_sem,
shutdown_rx,
Some(state.repo_store.clone()),
) {
info!("reconciliation sweep worker started");
}
}
// On-chain operator setup: verify stake + spawn heartbeat loop
if !state.config.contract_node_staking.is_empty()
&& !state.config.operator_private_key.is_empty()
@@ -789,63 +693,6 @@ async fn main() -> Result<()> {
let grace = std::time::Duration::from_secs(config.shutdown_grace_secs);
info!(grace_secs = config.shutdown_grace_secs, "axum server ready");
// #26 Split PR 1: drain any `applied` rows left by a previous
// process that crashed after Git applied a ref but before the
// bookkeeping landed. Runs once, BEFORE the server accepts new
// pushes, so a recovery re-derivation does not race a fresh push.
// Non-fatal: a transient drain failure is logged and the rows
// remain `applied` for the next startup to pick up.
//
// P1-A: the reconcile step runs FIRST and promotes any `prepared`
// or `uncertain` row whose target SHA actually landed on disk.
// This is the path that recovers a ref when the post-receive
// `mark_pending_ref_transitions_applied` call errored or was
// interrupted after `receive_pack` returned Ok, or when
// receive-pack returned Err but some refs may have landed
// (the `uncertain` state). Without this step, the drain (gated
// on `state = 'applied'`) would never see those rows.
//
// P2 (reviewer-1/2 round 3): use the multi-pass reconcile so
// prepared/uncertain rows beyond the first 1000-row page are
// processed in the same startup, rather than waiting for the
// next restart (where they might age out of MAX_RECONCILE_AGE).
match durable_outbox::reconcile_prepared_from_disk_all(
state.clone(),
durable_outbox::DRAIN_PER_PASS_LIMIT,
durable_outbox::DRAIN_MAX_PASSES,
)
.await
{
Ok(0) => {}
Ok(n) => info!(
n,
"reconciled prepared/uncertain -> applied via on-disk ref match"
),
Err(e) => warn!(
err = %e,
"pending ref transition reconcile failed at startup (non-fatal; will retry on next start)"
),
}
match durable_outbox::drain_receive_pack_requests_all(
state.clone(),
durable_outbox::DRAIN_PER_PASS_LIMIT,
durable_outbox::DRAIN_MAX_PASSES,
)
.await
{
Ok(0) => {}
Ok(n) => info!(n, "drained pending ref transitions from prior run"),
Err(e) => warn!(
err = %e,
"pending ref transition drain failed at startup (non-fatal; will retry on next start)"
),
}
// Start the due-request worker only now that boot recovery is done (see
// the note at the other spawns above): its first tick must not interleave
// with the one-shot reconcile/drain on backlog rows.
let _due_request_worker = spawn_due_request_worker(&state);
// `into_make_service_with_connect_info` exposes the socket peer address as
// `ConnectInfo<SocketAddr>` so the push limiter can key on the real client
// when no trusted proxy header applies (see `rate_limit::client_key`).
@@ -878,158 +725,6 @@ async fn main() -> Result<()> {
Ok(())
}
/// U4 (#173): spawn the periodic legacy provider-CID repair sweep. Releases before this
/// version stored the PROVIDER CID (Kubo dag-pb / Pinata CIDv0) in `pinned_cids.cid`,
/// and this version's `/ipfs/{cid}` resolver withholds any row whose stored key is not
/// the raw-content CID. The opportunistic repair on the pin path only fires when a push
/// re-carries the object, which normal git negotiation makes it not do, so those rows
/// need a walk. DETACHED, never on the boot path: the caller keeps serving while this
/// runs, and the sweep's own batch bound plus inter-batch delay keep it off the DB's
/// critical path. Its cursor is durable, so a restart mid-walk resumes instead of
/// rewinding. It re-arms after every run, including a failed one, so it never returns
/// and there is no awaited value to log here; the shutdown watcher below is what ends
/// it.
///
/// A named function rather than an inline block in `main` so the WIRING has a seam a
/// test can call: that the task is spawned at all, that it reads its batch and delay
/// from the config knobs rather than some other field, that the caller is not blocked
/// on it, and that the shutdown watcher actually ends it mid-walk. The sweep's own
/// behavior is covered elsewhere; this is the boot-path half.
fn spawn_legacy_cid_sweep(state: &AppState, config: &Config) -> tokio::task::JoinHandle<()> {
let db = state.db.clone();
let repos_dir = config.repos_dir.clone();
let git_bin = state.git_bin.clone();
let git_timeout = std::time::Duration::from_secs(config.git_service_timeout_secs);
let batch = config.pin_repair_sweep_batch;
let delay = std::time::Duration::from_secs(config.pin_repair_sweep_delay_secs);
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
tokio::select! {
_ = ipfs_pin::run_sweep_rearmed(
&repos_dir, &git_bin, git_timeout, batch, delay,
ipfs_pin::SWEEP_REARM_DELAY, &db,
) => {}
// Shutdown mid-walk simply drops the run; the persisted cursor means the
// next boot picks up where this one stopped.
_ = shutdown_rx.changed() => {}
}
})
}
/// #26 Split PR 1 step 4 — periodic queue-lifecycle purge. Runs on
/// the same detached task pattern as `spawn_legacy_cid_sweep`:
/// tokio::spawn with a shutdown watcher, never on the boot path. The
/// interval is fixed at 24 hours (the spec calls for "one per
/// cluster per day"); the inter-batch delay is implicit in the
/// batch size plus the wall-clock cost of each pass. The drain
/// (`drain_receive_pack_requests_all`) and the purge
/// (`purge_request_queue`) share the same `DRAIN_PER_PASS_LIMIT`
/// budget so a 1000-row purge pass takes roughly the same time as
/// a 1000-row drain pass.
fn spawn_queue_lifecycle_sweep(state: &AppState, config: &Config) -> tokio::task::JoinHandle<()> {
let app_state = state.clone();
let retention_days = config.queue_retention_days;
let batch = config.queue_purge_batch;
let git_timeout = std::time::Duration::from_secs(config.git_service_timeout_secs.min(30));
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(24 * 3600));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
// Consume the immediate first tick: Tokio intervals fire
// immediately on creation, which would run the purge
// concurrently with startup reconcile/drain. The child purge
// is terminal-parent-gated in SQL, but delaying the first
// sweep a full period removes the race entirely.
interval.tick().await;
loop {
tokio::select! {
_ = interval.tick() => {
if let Err(e) = durable_outbox::purge_request_queue(
&app_state.db,
retention_days,
batch,
).await {
tracing::warn!(err = %e, "queue lifecycle purge failed; will retry on next tick");
}
// Drain marker tombstones with the bounded deleter;
// failures retain the tombstone for the next tick.
if let Err(e) = durable_outbox::drain_marker_cleanup_queue(
&app_state,
batch,
git_timeout,
).await {
tracing::warn!(err = %e, "marker cleanup drain failed");
}
}
_ = shutdown_rx.changed() => {
break;
}
}
}
})
}
/// Shutdown-aware background due-request loop. The startup drain runs
/// once; without this, `effects_pending` rows with a future
/// `next_attempt_at` would wait until the next process restart.
/// Polls the indexed due query with bounded batches and failure
/// isolation (one request's error never aborts the batch).
fn spawn_due_request_worker(state: &AppState) -> tokio::task::JoinHandle<()> {
let app_state = state.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(5));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
tokio::select! {
_ = interval.tick() => {
match durable_outbox::drain_receive_pack_requests(
app_state.clone(),
durable_outbox::DRAIN_PER_PASS_LIMIT,
)
.await
{
Ok((0, _)) => {}
Ok((n, _)) => tracing::info!(n, "due request worker drained"),
Err(e) => tracing::warn!(err = %e, "due request worker drain failed"),
}
}
_ = shutdown_rx.changed() => break,
}
}
})
}
/// Dedicated marker-tombstone worker at a shorter cadence than the
/// daily purge, so external Git refs do not linger a full period
/// after their SQL owner is gone.
fn spawn_marker_cleanup_worker(state: &AppState) -> tokio::task::JoinHandle<()> {
let app_state = state.clone();
let git_timeout =
std::time::Duration::from_secs(app_state.config.git_service_timeout_secs.min(30));
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(60));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
tokio::select! {
_ = interval.tick() => {
if let Err(e) = durable_outbox::drain_marker_cleanup_queue(
&app_state,
durable_outbox::DRAIN_PER_PASS_LIMIT,
git_timeout,
)
.await
{
tracing::warn!(err = %e, "marker cleanup worker failed");
}
}
_ = shutdown_rx.changed() => break,
}
}
})
}
fn spawn_shutdown_signal(tx: watch::Sender<bool>) {
tokio::spawn(async move {
#[cfg(unix)]
@@ -1172,12 +867,11 @@ async fn run_degraded_server(
listener: TcpListener,
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
mut db_ready_rx: watch::Receiver<bool>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<()> {
let addr = listener.local_addr().ok();
let router = build_degraded_router(node_did, db_startup, web_url);
let router = build_degraded_router(node_did, db_startup);
info!(?addr, "degraded HTTP server ready");
axum::serve(listener, router)
@@ -1194,15 +888,10 @@ async fn run_degraded_server(
Ok(())
}
fn build_degraded_router(
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
) -> Router {
fn build_degraded_router(node_did: String, db_startup: Arc<DbStartupStatus>) -> Router {
let state = DegradedState {
node_did,
db_startup,
web_url,
};
// Everything answers 503 with the same body — including /health and
// /ready, so peer readiness probes and uptime monitors correctly see a
@@ -1234,9 +923,6 @@ async fn degraded_node_info(State(state): State<DegradedState>) -> impl IntoResp
obj.insert("name".into(), "gitlawb-node".into());
obj.insert("version".into(), env!("CARGO_PKG_VERSION").into());
obj.insert("did".into(), state.node_did.clone().into());
if let Some(web_url) = &state.web_url {
obj.insert("web_url".into(), web_url.clone().into());
}
}
(StatusCode::SERVICE_UNAVAILABLE, Json(body))
}
@@ -1483,10 +1169,7 @@ mod rate_limiter_sweep_tests {
state.close_issue_rate_limiter = RateLimiter::new(10, window);
state.sync_trigger_rate_limiter = RateLimiter::new(10, window);
state.peer_write_rate_limiter = RateLimiter::new(10, window);
state.federated_rate_limiter = RateLimiter::new(10, window);
state.ipfs_rate_limiter = RateLimiter::new(10, window);
state.ipfs_work_rate_limiter = RateLimiter::new(10, window);
state.task_read_rate_limiter = RateLimiter::new(10, window);
let limiters = |s: &crate::state::AppState| {
[
@@ -1496,10 +1179,7 @@ mod rate_limiter_sweep_tests {
s.close_issue_rate_limiter.clone(),
s.sync_trigger_rate_limiter.clone(),
s.peer_write_rate_limiter.clone(),
s.federated_rate_limiter.clone(),
s.ipfs_rate_limiter.clone(),
s.ipfs_work_rate_limiter.clone(),
s.task_read_rate_limiter.clone(),
]
};
for l in limiters(&state) {
@@ -1517,6 +1197,55 @@ mod rate_limiter_sweep_tests {
}
/// it.
fn spawn_legacy_cid_sweep(state: &AppState, config: &Config) -> tokio::task::JoinHandle<()> {
let db = state.db.clone();
let repos_dir = config.repos_dir.clone();
let git_bin = state.git_bin.clone();
let git_timeout = std::time::Duration::from_secs(config.git_service_timeout_secs);
let batch = config.pin_repair_sweep_batch;
let delay = std::time::Duration::from_secs(config.pin_repair_sweep_delay_secs);
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
tokio::select! {
_ = ipfs_pin::run_sweep_rearmed(
&repos_dir, &git_bin, git_timeout, batch, delay,
ipfs_pin::SWEEP_REARM_DELAY, &db,
) => {}
_ = shutdown_rx.changed() => {}
}
})
}
async fn gossip_ping_round(
db: &Db,
client: &reqwest::Client,
failed_once: &mut HashSet<String>,
peers: Vec<db::PeerRecord>,
) {
{
let current_dids: HashSet<&str> = peers.iter().map(|peer| peer.did.as_str()).collect();
failed_once.retain(|did| current_dids.contains(did.as_str()));
}
for peer in peers {
let ok = ping_peer_readiness(client, &peer.http_url).await;
match peer_ping_db_update(failed_once, &peer.did, ok) {
Some(reachable) => {
if let Err(error) = db.mark_peer_ping(&peer.did, reachable).await {
warn!(did = %peer.did, err = %error, "failed to persist peer readiness");
}
}
None => warn!(
did = %peer.did,
"peer readiness probe failed once; preserving previous federation status"
),
}
}
}
/// Decide whether one readiness sample should update the persisted federation
/// gate. A success is authoritative immediately. A failure must repeat on the
/// next gossip tick before it can mark a peer unreachable, so one transient
/// database hiccup does not hide the peer's repositories for five minutes.
fn peer_ping_db_update(failed_once: &mut HashSet<String>, did: &str, ready: bool) -> Option<bool> {
if ready {
failed_once.remove(did);
@@ -1693,32 +1422,6 @@ fn load_or_create_keypair_at(key_path: &std::path::Path) -> Result<Keypair> {
Ok(kp)
}
async fn gossip_ping_round(
db: &Db,
client: &reqwest::Client,
failed_once: &mut HashSet<String>,
peers: Vec<db::PeerRecord>,
) {
{
let current_dids: HashSet<&str> = peers.iter().map(|peer| peer.did.as_str()).collect();
failed_once.retain(|did| current_dids.contains(did.as_str()));
}
for peer in peers {
let ok = ping_peer_readiness(client, &peer.http_url).await;
match peer_ping_db_update(failed_once, &peer.did, ok) {
Some(reachable) => {
if let Err(error) = db.mark_peer_ping(&peer.did, reachable).await {
warn!(did = %peer.did, err = %error, "failed to persist peer readiness");
}
}
None => warn!(
did = %peer.did,
"peer readiness probe failed once; preserving previous federation status"
),
}
}
}
#[cfg(test)]
mod gossip_ssrf_tests {
use super::{
+1 -57
View File
@@ -20,9 +20,6 @@
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
//! * currently-connected peer count — `gitlawb_peers_connected`
//! * reconciliation sweep gaps found and filled —
//! `gitlawb_reconciliation_gaps_found_total` /
//! `gitlawb_reconciliation_gaps_filled_total`
//!
//! All metrics live in a single process-wide registry initialized by
//! [`init`]. Increment helpers (`record_push`, `record_auth_failure`, ...)
@@ -61,8 +58,6 @@ static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
static RECONCILIATION_GAPS_FOUND: OnceLock<IntCounter> = OnceLock::new();
static RECONCILIATION_GAPS_FILLED: OnceLock<IntCounter> = OnceLock::new();
/// 1 when the node started without its p2p identity because the key storage
/// could not be used. The node keeps serving HTTP in that state and /health
/// stays green, so without this the outage is visible only to whoever reads
@@ -246,29 +241,6 @@ fn init_inner(version: &str, node_did: &str) {
.set(peers_connected)
.expect("set PEERS_CONNECTED once");
let gaps_found = IntCounter::with_opts(Opts::new(
"gitlawb_reconciliation_gaps_found_total",
"Total reconciliation sweep gaps detected (objects that should be pinned but are not)",
))
.expect("gitlawb_reconciliation_gaps_found_total definition");
registry
.register(Box::new(gaps_found.clone()))
.expect("register gitlawb_reconciliation_gaps_found_total");
RECONCILIATION_GAPS_FOUND
.set(gaps_found)
.expect("set RECONCILIATION_GAPS_FOUND once");
let gaps_filled = IntCounter::with_opts(Opts::new(
"gitlawb_reconciliation_gaps_filled_total",
"Total reconciliation sweep gaps successfully filled (objects pinned by the sweep)",
))
.expect("gitlawb_reconciliation_gaps_filled_total definition");
registry
.register(Box::new(gaps_filled.clone()))
.expect("register gitlawb_reconciliation_gaps_filled_total");
RECONCILIATION_GAPS_FILLED
.set(gaps_filled)
.expect("set RECONCILIATION_GAPS_FILLED once");
let p2p_key_load_failed = IntGauge::with_opts(Opts::new(
"gitlawb_p2p_identity_key_load_failed",
"1 if the node is running without p2p because its identity key could not be loaded",
@@ -342,11 +314,7 @@ pub fn record_sync_processed(status: &str) {
}
/// Record one webhook delivery attempt outcome. `result` ∈
/// {ok, http_error, network_error, skipped, invalid_url, secret_unopenable}.
/// `invalid_url` is a policy skip: the stored hook URL failed the public-URL
/// check at delivery time and the POST was never sent. `secret_unopenable` is
/// a withheld delivery: the hook carries a secret the node could not open, so
/// the POST was never sent rather than going out unsigned.
/// {ok, http_error, network_error, skipped}.
pub fn record_webhook_delivery(result: &str) {
if let Some(c) = WEBHOOK_DELIVERIES.get() {
c.with_label_values(&[result]).inc();
@@ -427,20 +395,6 @@ pub fn set_peers_connected(count: i64) {
}
}
/// Record reconciliation sweep gaps found (objects that should be pinned but are not).
pub fn record_reconciliation_gaps_found(count: u64) {
if let Some(c) = RECONCILIATION_GAPS_FOUND.get() {
c.inc_by(count);
}
}
/// Record reconciliation sweep gaps filled (objects successfully pinned by the sweep).
pub fn record_reconciliation_gaps_filled(count: u64) {
if let Some(c) = RECONCILIATION_GAPS_FILLED.get() {
c.inc_by(count);
}
}
/// Record whether the node is running without its p2p identity.
pub fn set_p2p_key_load_failed(failed: bool) {
if let Some(g) = P2P_KEY_LOAD_FAILED.get() {
@@ -485,8 +439,6 @@ mod tests {
.expect("PUSHES set after init")
.with_label_values(&["alice/repo"])
.inc();
record_reconciliation_gaps_found(7);
record_reconciliation_gaps_filled(3);
let body = encode().expect("encode should succeed after init");
assert!(
@@ -501,14 +453,6 @@ mod tests {
body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"),
"expected the incremented counter to be visible in: {body}"
);
assert!(
body.contains("gitlawb_reconciliation_gaps_found_total 7"),
"expected the reconciliation gaps-found counter to be visible in: {body}"
);
assert!(
body.contains("gitlawb_reconciliation_gaps_filled_total 3"),
"expected the reconciliation gaps-filled counter to be visible in: {body}"
);
}
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
+3 -692
View File
@@ -7,11 +7,11 @@
//! The node's PeerId comes from an Ed25519 keypair loaded from a persistent
//! key file, so the PeerId is stable across restarts.
use std::collections::{hash_map::DefaultHasher, HashMap, HashSet, VecDeque};
use std::collections::{hash_map::DefaultHasher, HashMap};
use std::hash::{Hash, Hasher};
use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant};
use std::time::Duration;
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
@@ -315,17 +315,6 @@ fn is_zero(v: &u32) -> bool {
*v == 0
}
// Identify is an untrusted address source. Keep enough entries for normal
// multi-homed peers while bounding both one peer and the whole routing table.
const IDENTIFY_ADDRESS_LIMIT: usize = 8;
// Bound canonicalization and sorting too, including rejected/duplicate entries.
const IDENTIFY_REPORT_ADDRESS_LIMIT: usize = 64;
const IDENTIFY_NEW_ADDRESS_LIMIT: usize = 8;
const IDENTIFY_GLOBAL_ADDRESS_LIMIT: usize = 1024;
const IDENTIFY_ADDRESS_WINDOW: Duration = Duration::from_secs(60);
const IDENTIFY_ADDRESS_TTL: Duration = Duration::from_secs(30 * 60);
const IDENTIFY_ADDRESS_CLEANUP_INTERVAL: Duration = Duration::from_secs(60);
/// A ref-update event published to Gossipsub when a push lands.
///
/// The signing bytes are this struct serialized with `sig` set to None (see
@@ -1642,200 +1631,6 @@ fn did_to_kad_key(did: &str) -> kad::RecordKey {
kad::RecordKey::new(&format!("/gitlawb/did/{did}").as_bytes())
}
/// Returns true when `address` was explicitly configured for `peer` through
/// AddKnownPeer. Explicit addresses are retained in Kademlia even when an
/// overlapping Identify lease expires or is evicted.
fn is_explicit_address(
explicit_addresses: &HashMap<PeerId, HashSet<Multiaddr>>,
peer: &PeerId,
address: &Multiaddr,
) -> bool {
explicit_addresses
.get(peer)
.is_some_and(|addresses| addresses.contains(address))
}
#[derive(Debug)]
struct IdentifyAddress {
address: Multiaddr,
expires_at: Instant,
}
#[derive(Debug)]
struct IdentifyPeerAddresses {
addresses: VecDeque<IdentifyAddress>,
window_started: Instant,
new_addresses: usize,
}
#[derive(Debug, Default, PartialEq, Eq)]
struct IdentifyAddressChanges {
added: Vec<Multiaddr>,
refreshed: Vec<Multiaddr>,
removed: Vec<(PeerId, Multiaddr)>,
}
#[derive(Debug, Default)]
struct IdentifyAddressBook {
peers: HashMap<PeerId, IdentifyPeerAddresses>,
address_count: usize,
}
impl IdentifyAddressBook {
fn update(
&mut self,
peer_id: PeerId,
now: Instant,
addresses: &[Multiaddr],
) -> IdentifyAddressChanges {
let mut changes = IdentifyAddressChanges::default();
if self.peers.contains_key(&peer_id) {
changes.removed.extend(self.expire_peer(peer_id, now));
}
let reported = addresses
.iter()
.take(IDENTIFY_REPORT_ADDRESS_LIMIT)
.filter_map(|address| address.clone().with_p2p(peer_id).ok())
.collect::<HashSet<_>>();
if reported.is_empty() && !self.peers.contains_key(&peer_id) {
return changes;
}
let state = self
.peers
.entry(peer_id)
.or_insert_with(|| IdentifyPeerAddresses {
addresses: VecDeque::new(),
window_started: now,
new_addresses: 0,
});
if now.saturating_duration_since(state.window_started) >= IDENTIFY_ADDRESS_WINDOW {
state.window_started = now;
state.new_addresses = 0;
}
// Refresh every retained address in the bounded report before admission.
// Refreshed addresses are reported so the Identify handler can re-admit
// them to Kademlia. Kademlia may have dropped a retained address
// after a failed dial or never kept it for a full k-bucket, and a
// TTL refresh alone would never re-add it.
for existing in &mut state.addresses {
if reported.contains(&existing.address) {
existing.expires_at = now + IDENTIFY_ADDRESS_TTL;
changes.refreshed.push(existing.address.clone());
}
}
let mut new_addresses: Vec<Multiaddr> = reported
.iter()
.filter(|address| {
!state
.addresses
.iter()
.any(|existing| &existing.address == *address)
})
.cloned()
.collect();
// HashSet iteration is deliberately unordered. Sort new candidates so
// an oversized Identify report has deterministic admission behavior.
new_addresses.sort();
for address in new_addresses {
if state.new_addresses >= IDENTIFY_NEW_ADDRESS_LIMIT {
break;
}
let evicted = if state.addresses.len() < IDENTIFY_ADDRESS_LIMIT {
None
} else {
state
.addresses
.iter()
.position(|existing| !reported.contains(&existing.address))
.and_then(|index| state.addresses.remove(index))
};
if let Some(evicted) = evicted {
self.address_count -= 1;
changes.removed.push((peer_id, evicted.address));
} else if state.addresses.len() >= IDENTIFY_ADDRESS_LIMIT {
// Every retained address was refreshed in this report. Keep
// that stable subset and drop surplus new candidates.
break;
}
if self.address_count >= IDENTIFY_GLOBAL_ADDRESS_LIMIT {
// Reject the admission instead of evicting another peer's
// entry: cross-peer eviction lets a flood of new identities
// push honest peers' addresses out of the book and, when the
// evicted address is their last one, out of the Kademlia
// routing table. Per-peer eviction above still applies to the
// reporting peer's own addresses.
break;
}
state.new_addresses += 1;
state.addresses.push_back(IdentifyAddress {
address: address.clone(),
expires_at: now + IDENTIFY_ADDRESS_TTL,
});
self.address_count += 1;
changes.added.push(address);
}
// Do not keep a row when every candidate was rejected: a zero-address
// row is invisible to address_count and would otherwise linger until
// the expiry tick. This matches expire_peer, which drops empty rows.
if state.addresses.is_empty() {
self.peers.remove(&peer_id);
}
changes
}
fn expire(&mut self, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let mut removed = Vec::new();
let peers = self.peers.keys().copied().collect::<Vec<_>>();
for peer_id in peers {
removed.extend(self.expire_peer(peer_id, now));
}
removed
}
fn expire_peer(&mut self, peer_id: PeerId, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let expired = self
.peers
.get_mut(&peer_id)
.map(|state| Self::expire_state(state, now))
.unwrap_or_default();
self.address_count -= expired.len();
if self
.peers
.get(&peer_id)
.is_some_and(|state| state.addresses.is_empty())
{
self.peers.remove(&peer_id);
}
expired
.into_iter()
.map(|address| (peer_id, address))
.collect()
}
fn expire_state(state: &mut IdentifyPeerAddresses, now: Instant) -> Vec<Multiaddr> {
let mut expired = Vec::new();
let mut retained = VecDeque::with_capacity(state.addresses.len());
while let Some(address) = state.addresses.pop_front() {
if address.expires_at <= now {
expired.push(address.address);
} else {
retained.push_back(address);
}
}
state.addresses = retained;
expired
}
}
/// Combined libp2p behaviour.
#[derive(NetworkBehaviour)]
#[behaviour(prelude = "libp2p_swarm::derive_prelude")]
@@ -4165,14 +3960,6 @@ pub async fn start(
// Track in-flight GetRecord queries → reply channels
let mut pending_get_did: HashMap<kad::QueryId, oneshot::Sender<Option<DidRecord>>> =
HashMap::new();
// Keep Identify-derived entries across disconnects so a bootstrap address
// learned through Identify remains usable for redial. The TTL cleanup below
// removes entries that are no longer refreshed without touching explicit
// AddKnownPeer addresses.
let mut identify_addresses = IdentifyAddressBook::default();
let mut explicit_addresses: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
let mut identify_cleanup = tokio::time::interval(IDENTIFY_ADDRESS_CLEANUP_INTERVAL);
identify_cleanup.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
// Start the event loop as a background task
tokio::spawn(async move {
@@ -4192,13 +3979,6 @@ pub async fn start(
_ = ingest_sweep.tick() => {
run_ingest_sweep(&ingest_limiters, &replay_guard, ingest_now()).await;
}
_ = identify_cleanup.tick() => {
for (peer_id, address) in identify_addresses.expire(Instant::now()) {
if !is_explicit_address(&explicit_addresses, &peer_id, &address) {
swarm.behaviour_mut().kademlia.remove_address(&peer_id, &address);
}
}
}
// Graceful shutdown: exit the swarm loop when the
// process-wide signal flips. This drops the Swarm
// which closes all libp2p connections cleanly.
@@ -4333,21 +4113,7 @@ pub async fn start(
identify::Event::Received { peer_id, info, .. }
)) => {
debug!(peer = %peer_id, "identify received");
let changes = identify_addresses.update(
peer_id,
Instant::now(),
&info.listen_addrs,
);
for (removed_peer, addr) in changes.removed {
if !is_explicit_address(&explicit_addresses, &removed_peer, &addr)
{
swarm
.behaviour_mut()
.kademlia
.remove_address(&removed_peer, &addr);
}
}
for addr in changes.refreshed.into_iter().chain(changes.added) {
for addr in info.listen_addrs {
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
}
}
@@ -4384,14 +4150,6 @@ pub async fn start(
}
}
P2pCommand::AddKnownPeer { peer_id, addr } => {
let Ok(addr) = addr.clone().with_p2p(peer_id) else {
warn!(%peer_id, %addr, "dropping known-peer address with a foreign peer suffix");
continue;
};
explicit_addresses
.entry(peer_id)
.or_default()
.insert(addr.clone());
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
}
P2pCommand::Dial(addr) => {
@@ -4443,453 +4201,6 @@ pub async fn start(
mod tests {
use super::*;
fn identify_address(peer_id: PeerId, octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
.with_p2p(peer_id)
.unwrap()
}
fn identify_address_without_peer(octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
}
#[test]
fn identify_empty_reports_do_not_retain_peer_entries() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let foreign = identify_address(PeerId::random(), 1, 10_000);
for _ in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT + 1 {
let peer = PeerId::random();
assert_eq!(
book.update(peer, now, &[]),
IdentifyAddressChanges::default()
);
assert_eq!(
book.update(peer, now, std::slice::from_ref(&foreign)),
IdentifyAddressChanges::default()
);
}
assert!(book.peers.is_empty());
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_fully_rejected_peer_leaves_no_row_at_full_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// Fill the global budget with one-address peers.
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// A fresh peer reporting at saturation has every candidate rejected
// by the global-budget check. The row created by or_insert_with must
// be dropped: a zero-address row is invisible to address_count and
// would otherwise linger until the expiry tick.
let fresh = PeerId::random();
let fresh_address = identify_address(fresh, 2, 30_000);
let changes = book.update(fresh, now, std::slice::from_ref(&fresh_address));
assert!(changes.added.is_empty());
assert!(!book.peers.contains_key(&fresh));
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_update_reports_expired_addresses_as_removed() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
// An update past the TTL expires the stale address through
// expire_peer and reports it in changes.removed so the Identify
// handler can drop it from Kademlia.
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1),
std::slice::from_ref(&address),
);
assert_eq!(changes.removed, vec![(peer, address.clone())]);
assert_eq!(changes.added, vec![address]);
}
#[test]
fn identify_expire_drops_rows_emptied_by_expiry() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
assert!(book.peers.contains_key(&peer));
// Once every address in the row has expired, expire() must drop the
// row itself, matching the empty-row drop in update().
let removed = book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1));
assert_eq!(removed, vec![(peer, address)]);
assert!(!book.peers.contains_key(&peer));
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_report_processing_stops_at_the_input_limit() {
let peer = PeerId::random();
let now = Instant::now();
let foreign = identify_address(PeerId::random(), 1, 10_000);
let accepted = identify_address(peer, 2, 10_001);
let ignored = identify_address(peer, 3, 10_002);
let mut report = vec![foreign; IDENTIFY_REPORT_ADDRESS_LIMIT - 1];
report.push(accepted.clone());
report.push(ignored);
let mut book = IdentifyAddressBook::default();
let changes = book.update(peer, now, &report);
assert_eq!(changes.added, vec![accepted]);
assert!(changes.removed.is_empty());
assert_eq!(book.address_count, 1);
}
#[test]
fn identify_addresses_grow_without_eviction_below_the_peer_cap() {
let peer = PeerId::random();
let now = Instant::now();
let first = identify_address(peer, 1, 10_000);
let second = identify_address(peer, 2, 10_001);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&first));
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&second),
);
assert_eq!(changes.added, vec![second]);
assert!(changes.removed.is_empty());
assert_eq!(book.peers[&peer].addresses.len(), 2);
assert_eq!(book.address_count, 2);
}
#[test]
fn identify_global_budget_rejects_new_admissions() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let peers = (0..IDENTIFY_GLOBAL_ADDRESS_LIMIT)
.map(|index| {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
(peer, address)
})
.collect::<Vec<_>>();
let (peer, refreshed) = &peers[0];
let new_address = identify_address(*peer, 2, 30_000);
let changes = book.update(
*peer,
now + IDENTIFY_ADDRESS_WINDOW,
&[new_address.clone(), refreshed.clone()],
);
// The budget is full: the new address is rejected rather than
// evicting another peer's entry, and the refreshed address keeps its
// slot and is reported for re-admission to Kademlia.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![(*refreshed).clone()]);
assert_eq!(book.peers.len(), IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(
book.peers[peer]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![refreshed]
);
}
#[test]
fn identify_per_peer_eviction_applies_at_full_global_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// One filler peer goes in before the victim so the evicted address
// is not also the globally-oldest insertion. If saturation evicted
// the globally-oldest entry instead of the victim's own oldest,
// the sentinel's address would be the one removed.
let sentinel = PeerId::random();
let sentinel_address = identify_address(sentinel, 200, 19_999);
book.update(sentinel, now, std::slice::from_ref(&sentinel_address));
// The victim peer holds a full per-peer row while the rest of the
// global budget is filled by one-address peers.
let victim = PeerId::random();
let victim_addresses: Vec<Multiaddr> = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(victim, index as u8 + 1, 10_000 + index as u16))
.collect();
book.update(victim, now, &victim_addresses);
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT - IDENTIFY_ADDRESS_LIMIT - 1 {
let peer = PeerId::random();
let address = identify_address(peer, 200, 20_000 + index as u16);
book.update(peer, now, &[address]);
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// The victim reports a ninth address after the rate window rolls over.
// Per-peer eviction must still apply at global saturation: the oldest
// non-reported address is evicted to make room, and the global count
// is unchanged. This ordering only holds because eviction runs before
// the global-budget check; hoisting that check above the eviction
// block makes this test fail.
let ninth = identify_address(victim, 100, 30_000);
let changes = book.update(
victim,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&ninth),
);
assert_eq!(changes.added, vec![ninth]);
assert_eq!(changes.removed, vec![(victim, victim_addresses[0].clone())]);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.peers[&victim].addresses.len(), IDENTIFY_ADDRESS_LIMIT);
// The sentinel was inserted first; per-peer eviction must leave it
// alone. A global-oldest eviction policy would have removed it.
assert_eq!(
book.peers[&sentinel]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&sentinel_address]
);
}
#[test]
fn explicit_address_helper_detects_retained_addresses() {
let peer = PeerId::random();
let other = PeerId::random();
let address = identify_address(peer, 1, 10_000);
let mut explicit: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
// Unknown peer: nothing is explicit.
assert!(!is_explicit_address(&explicit, &peer, &address));
explicit.entry(peer).or_default().insert(address.clone());
assert!(is_explicit_address(&explicit, &peer, &address));
// The same address under a different peer is not retained.
assert!(!is_explicit_address(&explicit, &other, &address));
// A different address for the same peer is not retained.
assert!(!is_explicit_address(
&explicit,
&peer,
&identify_address(peer, 2, 10_001)
));
}
#[test]
fn identify_addresses_are_capped_with_fifo_eviction() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &initial);
assert_eq!(first.added, initial);
assert!(first.removed.is_empty());
let second = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
);
assert_eq!(second.added, vec![replacement]);
assert_eq!(second.removed, vec![(peer_id, initial[0].clone())]);
}
#[test]
fn identify_oversized_reports_keep_the_same_refreshed_subset() {
let peer_id = PeerId::random();
let now = Instant::now();
let mut report = (0..IDENTIFY_ADDRESS_LIMIT + 2)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &report);
assert_eq!(first.added.len(), IDENTIFY_ADDRESS_LIMIT);
assert_eq!(first.removed.len(), 0);
let retained = first.added.clone();
report.reverse();
let second = book.update(peer_id, now + IDENTIFY_ADDRESS_WINDOW, &report);
assert!(second.added.is_empty());
assert!(second.removed.is_empty());
let current = book
.peers
.get(&peer_id)
.unwrap()
.addresses
.iter()
.map(|entry| entry.address.clone())
.collect::<Vec<_>>();
assert_eq!(current, retained);
}
#[test]
fn identify_addresses_rate_limit_new_entries_across_updates() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_NEW_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(book.update(peer_id, now, &initial).added, initial);
assert!(book
.update(peer_id, now, std::slice::from_ref(&replacement))
.added
.is_empty());
assert_eq!(
book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
)
.added,
vec![replacement]
);
}
#[test]
fn identify_addresses_have_a_global_budget_across_peers() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let mut first = None;
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer_id = PeerId::random();
let address = identify_address(peer_id, (index % 254) as u8 + 1, 10_000 + index as u16);
if first.is_none() {
first = Some((peer_id, address.clone()));
}
assert!(book.update(peer_id, now, &[address]).removed.is_empty());
}
let (peer_id, first_address) = first.unwrap();
let address = identify_address(peer_id, 250, 30_000);
let changes = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&address),
);
// The global budget is full, so the admission is rejected and the
// peer's existing entry is left alone.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(
book.peers[&peer_id]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&first_address]
);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_addresses_refresh_and_expire() {
let peer_id = PeerId::random();
let now = Instant::now();
let address = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, now, std::slice::from_ref(&address))
.added,
vec![address.clone()]
);
let refreshed = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let refresh = book.update(peer_id, refreshed, std::slice::from_ref(&address));
assert!(refresh.added.is_empty());
assert!(refresh.removed.is_empty());
// Refreshed addresses are reported so the Identify handler re-adds
// them to Kademlia even though they were not newly admitted.
assert_eq!(refresh.refreshed, vec![address.clone()]);
assert!(book
.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1))
.is_empty());
assert_eq!(
book.expire(refreshed + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, address)]
);
}
#[test]
fn identify_refresh_reports_only_addresses_in_the_report() {
let peer_id = PeerId::random();
let now = Instant::now();
let reported = identify_address(peer_id, 1, 10_000);
let unreported = identify_address(peer_id, 2, 20_000);
let mut book = IdentifyAddressBook::default();
// Admit both addresses up front so the peer holds two.
assert_eq!(
book.update(peer_id, now, &[reported.clone(), unreported.clone()])
.added
.len(),
2
);
// A later report carrying only one of the two addresses must refresh
// just that one: pins the negative direction of the refresh
// predicate, so an address the peer stopped advertising keeps its
// original TTL instead of being refreshed back onto the Kademlia
// re-admit path on every report.
let report_at = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let changes = book.update(peer_id, report_at, std::slice::from_ref(&reported));
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![reported.clone()]);
// The unreported address still expires at its original TTL, while
// the refreshed address survives until its refreshed TTL.
assert_eq!(
book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, unreported)]
);
assert!(book.expire(report_at + Duration::from_secs(1)).is_empty());
assert_eq!(
book.expire(report_at + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, reported)]
);
}
#[test]
fn identify_addresses_reject_foreign_peer_suffixes() {
let peer_id = PeerId::random();
let other_peer_id = PeerId::random();
let address = identify_address(other_peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert!(book
.update(peer_id, Instant::now(), &[address])
.added
.is_empty());
}
#[test]
fn identify_addresses_canonicalize_the_peer_suffix_once() {
let peer_id = PeerId::random();
let raw = identify_address_without_peer(1, 10_000);
let canonical = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, Instant::now(), &[raw]).added,
vec![canonical]
);
}
#[test]
fn dir_mode_replace_authority_is_the_write_bits() {
assert!(!dir_mode_allows_untrusted_replace(0o755));
+41 -490
View File
@@ -153,26 +153,6 @@ pub async fn pin_object(
/// `upsert_branch_cid` and the p2p `publish_ref_update` gossip CID. The twin's return is
/// log-only, so it omits a record-failed pin rather than logging a pin the resolver
/// cannot serve. Moving this side to match would need that consumer moved first.
/// Whether the committed `pinned_cids` row already names `repo_id` as
/// first pinner, making a failed redundant `record_pin_source` insert
/// irrelevant to resolvability (`pin_sources_for_oid` unions the
/// primary row). Bounded read; any failure (including its own
/// timeout) answers false — an unproven row never confirms.
/// Used only after the primary `record_pinata_cid` for this call
/// succeeded or verified: callers must not promote a row this call
/// did not establish.
async fn primary_covers_repo(
db: &crate::db::Db,
sha: &str,
repo_id: &str,
deadline: std::time::Instant,
) -> bool {
matches!(
crate::ipfs_pin::db_bounded(deadline, db.provenance_for_oid(sha)).await,
Ok(Some(owner)) if owner.as_str() == repo_id
)
}
// Ten arguments, over clippy's threshold: the three the budget and the git seam add
// (`git_bin`, `git_timeout`, `batch_budget`) plus #173's `repo_id` are what put the read
// under test injection and under a deadline, and grouping them into a struct would only
@@ -190,33 +170,16 @@ pub async fn pin_new_objects(
db: &crate::db::Db,
repo_id: &str,
batch_budget: Duration,
fence: Option<&crate::ipfs_pin::PolicyFence>,
) -> crate::ipfs_pin::PinBatchOutcome {
) -> Vec<(String, String)> {
if jwt.is_empty() {
return crate::ipfs_pin::PinBatchOutcome {
confirmed: Vec::new(),
last_attempted: None,
};
return vec![];
}
let deadline = Instant::now() + batch_budget;
let total = object_list.len();
let mut pinned = Vec::new();
let mut last_attempted: Option<String> = None;
for (attempted, sha) in object_list.into_iter().enumerate() {
// Policy fence (R1-P1): a visibility narrow that lands after the caller
// built this batch must abort it before the next irreversible upload.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
unattempted = total - attempted,
"visibility policy changed mid-batch; stopping the Pinata pin loop"
);
break;
}
}
// Top of the iteration, before any of this object's work: an object is never
// started with a remainder too small to cover a bounded read's teardown. The
// gate is shared with the IPFS loop so the two cannot drift apart in how they
@@ -228,10 +191,6 @@ pub async fn pin_new_objects(
{
break;
}
// Attempt progress (not a durability claim): same contract as the
// IPFS twin — the continuation persists this, never the planned
// vector's tail.
last_attempted = Some(sha.clone());
// Every DB call from here to the end of the iteration is bounded by the
// ABSOLUTE batch deadline (F3, #173), through the same `db_bounded` helper the
@@ -436,21 +395,6 @@ pub async fn pin_new_objects(
}
};
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the has_pinata_cid round-trip or the bounded Git read.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
unattempted = total - attempted,
"visibility policy changed during preparation; aborting Pinata upload"
);
break;
}
}
match pin_object(client, upload_url, jwt, &sha, &data).await {
Ok(cid) if !cid.is_empty() => {
// The resolver key (`pinned_cids.cid`) must be the locally-computed
@@ -466,109 +410,30 @@ pub async fn pin_new_objects(
// runs under the shared client's own ceiling, so a successful one can
// return with ~0 of the batch budget left, and an unfloored bound would
// fail a write that today completes in milliseconds. That costs more on
// this side than on the twin: `pinned.push` below feeds
// `api/repos.rs`, which builds its `cid_map` from the pair and drives
// this side than on the twin: `pinned.push` below is UNCONDITIONAL, so
// `api/repos.rs` builds its `cid_map` from the pair either way and drives
// `upsert_branch_cid` plus the p2p `publish_ref_update` gossip from it. A
// dropped record therefore makes the node ADVERTISE a CID whose `/ipfs`
// read 404s. Only durably recorded pairs are pushed, so the map never
// advertises an unconfirmed row.
//
// Round 10 P2: a closed/failed DB write means the (sha, cid)
// pair is not durable; we suppress the `pinned.push` for this
// sha so the reconcile cannot count a Pinata gap as filled
// when no row exists. The source-record failure arms are also
// hard failures (multi-statement transaction never committed)
// and suppress the push in the same way.
//
// `record_pinata_cid` is an explicit multi-statement
// transaction (row lock + insert + commit), NOT a single
// autocommit upsert, so a timed-out future proves nothing:
// cancellation before COMMIT wrote nothing, during COMMIT is
// unknown. The Elapsed arm therefore verifies by content
// (`verify_pinata_record`: exact row + unchanged fence epoch)
// under its own bound and pushes only on proof; anything
// else stays suppressed and the gap is re-offered.
let fence_epoch = fence.map(|f| f.captured_epoch()).unwrap_or(i64::MAX);
let mut db_record_durable = false;
match crate::ipfs_pin::db_bounded(
// read 404s. If the floored bound still fires, THIS site's outcome really
// is unknown, and unlike the source record below that is a property of
// the operation: `record_pinata_cid` is a single autocommit upsert, so
// the statement Postgres already started can still land after the client
// future is cancelled. The warn names the arm through the error's own
// Display and the site keeps its existing behavior: the pair is still
// returned, and the row may or may not exist.
if let Err(e) = crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
crate::ipfs_pin::retry_db_record(|| {
// P2 (reviewer round 9): Pinata's POST is
// irreversible exactly like IPFS's, so
// route the record through the fenced
// variant when a fence is in scope. The
// `i64::MAX` sentinel tells the helper
// to skip the lock + comparison (the
// unfenced caller path).
db.record_pinata_cid(&sha, &raw_cid, &cid, Some(repo_id), fence_epoch)
db.record_pinata_cid(&sha, &raw_cid, &cid, Some(repo_id))
}),
)
.await
{
Ok(()) => db_record_durable = true,
Err(crate::ipfs_pin::BoundedDbError::Elapsed) => {
tracing::warn!(
sha = %sha,
"record_pinata_cid deadline elapsed on a multi-statement transaction; \
verifying the exact row before treating it as durable"
);
match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
db.verify_pinata_record(&sha, &raw_cid, &cid, repo_id, fence_epoch),
)
.await
{
Ok(true) => {
tracing::warn!(
sha = %sha,
"timed-out pinata record verified present with unchanged epoch; \
treating as durable"
);
db_record_durable = true;
}
Ok(false) => {
tracing::warn!(
sha = %sha,
"timed-out pinata record has no matching row or the epoch moved; \
suppressing the push so the gap is re-offered"
);
}
Err(e) => {
tracing::warn!(
sha = %sha,
err = %e,
"pinata record verification did not complete; \
suppressing the push so the gap is re-offered"
);
}
}
}
Err(e) => {
tracing::warn!(
sha = %sha,
err = %e,
"failed to record pinata_cid in DB; suppressing the (sha, cid) push \
so the reconcile cannot count this gap as filled"
);
// db_record_durable stays false → push suppressed
}
tracing::warn!(sha = %sha, err = %e, "failed to record pinata_cid in DB");
}
// F1 (#173 round 8): also record the first pinner in pin_repo_sources.
// U3: an exhausted retry marks the set incomplete so the resolver keeps
// the scan fallback rather than 404ing a copy it could serve.
//
// Whether the pair stays confirmed depends on what the
// PRIMARY `record_pinata_cid` above established: that row
// already names this repo as first pinner
// (`pin_sources_for_oid` unions it), so the resolver
// reaches this copy with or without the redundant
// `pin_repo_sources` insert. A source-write failure then
// keeps the pair confirmed (but still marks the set
// incomplete, preserving the U3 compensation). If the
// primary row names ANOTHER repo — or is absent — the
// source write was load-bearing and the push stays
// suppressed. (The IPFS twin needs no equivalent: its
// pin and source land in ONE transaction.)
match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
crate::ipfs_pin::retry_db_record(|| db.record_pin_source(&sha, repo_id)),
@@ -580,7 +445,9 @@ pub async fn pin_new_objects(
// wraps `record_pin_source`, an explicit transaction, so a timed-out
// call definitely never committed and the source is definitely
// missing. Mark the set incomplete rather than leaving it incomplete
// and unmarked.
// and unmarked. Note the contrast with `record_pinata_cid` a few
// lines up: that one is a single autocommit statement, so its
// timeout genuinely is an unknown outcome and it is warn-only.
Err(e @ crate::ipfs_pin::BoundedDbError::Elapsed) => {
tracing::warn!(
sha = %sha,
@@ -597,16 +464,6 @@ pub async fn pin_new_objects(
{
tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete");
}
if !primary_covers_repo(
db,
&sha,
repo_id,
crate::ipfs_pin::db_record_deadline(deadline),
)
.await
{
db_record_durable = false;
}
}
Err(e) => {
tracing::warn!(sha = %sha, err = %e, "failed to record pin source");
@@ -618,21 +475,9 @@ pub async fn pin_new_objects(
{
tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete");
}
if !primary_covers_repo(
db,
&sha,
repo_id,
crate::ipfs_pin::db_record_deadline(deadline),
)
.await
{
db_record_durable = false;
}
}
}
if db_record_durable {
pinned.push((sha, cid));
}
pinned.push((sha, cid));
}
Ok(_) => {}
Err(e) => {
@@ -641,10 +486,7 @@ pub async fn pin_new_objects(
}
}
crate::ipfs_pin::PinBatchOutcome {
confirmed: pinned,
last_attempted,
}
pinned
}
// ── Tests ─────────────────────────────────────────────────────────────────────
@@ -846,20 +688,19 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_millis(5500),
None,
),
)
.await
.expect("wedge guard: a 5.5s budget cannot take 30s");
assert!(
(1..=3).contains(&pinned.confirmed.len()),
(1..=3).contains(&pinned.len()),
"the batch must stop partway, not pin all five and not stall on the first: pinned {}",
pinned.confirmed.len()
pinned.len()
);
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
pinned.confirmed.len(),
pinned.len(),
"no upload may be issued for an object the budget stopped short of"
);
let text = logs.text();
@@ -887,9 +728,9 @@ mod tests {
})
.unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}"));
assert!(
unattempted >= 1 && unattempted + pinned.confirmed.len() <= 5,
unattempted >= 1 && unattempted + pinned.len() <= 5,
"unattempted={unattempted} with {} pinned is not a partial batch of five",
pinned.confirmed.len()
pinned.len()
);
}
@@ -938,7 +779,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(2),
None,
),
)
.await
@@ -949,7 +789,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -1028,7 +868,6 @@ mod tests {
"repo-git-timeout",
// Generous, so a call that ends on time ended on `git_timeout`.
Duration::from_secs(60),
None,
),
)
.await
@@ -1039,7 +878,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -1073,7 +912,6 @@ mod tests {
/// probe carries its oid on stdin rather than in argv, so an oid appears in the log only
/// once an object has already got past its probe, and a healthy object costs two
/// invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log)
.unwrap_or_default()
@@ -1131,7 +969,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -1141,7 +978,7 @@ mod tests {
if genuinely_unreadable {
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"nothing can be pinned through a store that cannot be read: {pinned:?}"
);
assert_eq!(
@@ -1206,7 +1043,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -1218,7 +1054,7 @@ mod tests {
"an object-scoped fault must not stop the batch: every object must be read"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
4,
"one corrupt object must cost only itself: the other four must still pin"
);
@@ -1258,18 +1094,13 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
.expect("an immediate endpoint and three healthy objects cannot take 30s");
assert_eq!(
pinned.confirmed.len(),
3,
"every healthy object must pin: {pinned:?}"
);
for (i, (sha, cid)) in pinned.confirmed.iter().enumerate() {
assert_eq!(pinned.len(), 3, "every healthy object must pin: {pinned:?}");
for (i, (sha, cid)) in pinned.iter().enumerate() {
assert_eq!(sha, &oids[i], "the pairs must carry the objects' own oids");
assert_eq!(cid, "QmPinataBatchTestCid");
assert!(
@@ -1292,15 +1123,11 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
.expect("a fully deduped batch cannot take 30s");
assert!(
again.confirmed.is_empty(),
"already-recorded objects must be skipped"
);
assert!(again.is_empty(), "already-recorded objects must be skipped");
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
3,
@@ -1308,63 +1135,6 @@ mod tests {
);
}
/// The outcome reports the last object actually ENTERED, not the planned
/// tail (Pinata twin of the IPFS test above): the first upload takes 6s
/// against a 2s batch budget, so it still completes — the Pinata loop
/// has no per-request timeout — but the loop-top gate then breaks the
/// batch before the second object starts. `last_attempted` must be the
/// first OID even though two OIDs were never visited; the 6s server
/// sleep makes the break deterministic on any box.
#[sqlx::test]
async fn pin_new_objects_reports_last_attempted_not_planned_tail(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("attempted.git");
let oids = seed_loose_blobs(&repo_path, 3);
let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let endpoint = delaying_pinata_endpoint(
vec![Duration::from_secs(6)],
std::sync::Arc::clone(&requests),
)
.await;
let client = reqwest::Client::new();
let outcome = tokio::time::timeout(
Duration::from_secs(60),
pin_new_objects(
&client,
&endpoint,
"test-jwt",
&repo_path,
"git",
Duration::from_secs(60),
oids.clone(),
&db,
"repo-attempted",
Duration::from_secs(2),
None,
),
)
.await
.expect("one 6s upload plus bounded gates cannot take 60s");
assert_eq!(
outcome.last_attempted,
Some(oids[0].clone()),
"only the first object was entered; the planned tail was never visited"
);
assert_eq!(
outcome.confirmed.len(),
1,
"the slow first upload still completed and recorded"
);
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
1,
"exactly one upload was attempted"
);
}
/// The no-op configuration still short-circuits with the budgeted signature: an
/// empty JWT must return before any git child is spawned and before any request
/// is issued. The `git_bin` here records every invocation, so "git was never
@@ -1406,13 +1176,12 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
.expect("an unconfigured sink returns immediately");
assert!(pinned.confirmed.is_empty(), "an empty JWT pins nothing");
assert!(pinned.is_empty(), "an empty JWT pins nothing");
assert!(
!log.exists(),
"no git child may be spawned when the sink is not configured"
@@ -1512,7 +1281,6 @@ mod tests {
&db,
"repo-pinata-stalled",
Duration::from_millis(1500),
None,
),
)
.await
@@ -1523,7 +1291,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a stalled pinata-status check cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -1569,15 +1337,9 @@ mod tests {
// bytes.
let raw_cid =
gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata skip seed").to_string();
db.record_pinata_cid(
&sha,
&raw_cid,
"QmSeedProviderCid",
Some("repo-seed"),
i64::MAX,
)
.await
.unwrap();
db.record_pinata_cid(&sha, &raw_cid, "QmSeedProviderCid", Some("repo-seed"))
.await
.unwrap();
db.record_pin_source(&sha, "repo-seed").await.unwrap();
let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let endpoint = delaying_pinata_endpoint(
@@ -1609,7 +1371,6 @@ mod tests {
&db,
"repo-pinata-skip-stalled",
Duration::from_millis(1500),
None,
),
)
.await
@@ -1620,7 +1381,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"an already-pinned object is skipped, never re-uploaded: {pinned:?}"
);
assert_eq!(
@@ -1664,7 +1425,7 @@ mod tests {
///
/// The lock time is a MARGIN, not a boundary: taking it at 100ms left
/// `has_pinata_cid` racing it on a loaded box, and losing that race makes the read
/// block, time out, and break the batch, which fails on `pinned.confirmed.len() == 1` for a
/// block, time out, and break the batch, which fails on `pinned.len() == 1` for a
/// reason that has nothing to do with the floor. Any time between the
/// `has_pinata_cid` round trip and the upload's 1.7s return proves the same thing.
#[sqlx::test]
@@ -1710,7 +1471,6 @@ mod tests {
&db,
"repo-pinata-spent-budget",
Duration::from_millis(2000),
None,
),
);
let (pinned, ()) = tokio::join!(pin, locker);
@@ -1728,7 +1488,7 @@ mod tests {
record makes api/repos.rs advertise a CID the resolver cannot serve"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"the uploaded pin must still be returned: {pinned:?}"
);
@@ -1796,7 +1556,6 @@ mod tests {
&db,
"repo-pinata-post-upload",
Duration::from_millis(1500),
None,
),
)
.await
@@ -1810,21 +1569,10 @@ mod tests {
drop(lock);
upload.assert_async().await;
// The primary `record_pinata_cid` committed above naming this
// repo as first pinner, so `pin_sources_for_oid` resolves the
// copy with or without the redundant `pin_repo_sources`
// insert: the pair stays confirmed even though the source
// record timed out. The incomplete marker below is still set
// (compensation preserved); only the suppression is gone.
// Prior to the round 10 fix, the push fired regardless of
// every post-upload outcome; the correction since is that
// confirmation follows the durable primary row, not the
// redundant source write.
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"record_pin_source timed out (pin_repo_sources locked) but the primary row \
covers this repo, so the pair stays confirmed: {pinned:?}"
"the upload succeeded, so this lane still returns the pair: {pinned:?}"
);
assert!(
elapsed < Duration::from_secs(8),
@@ -1841,203 +1589,6 @@ mod tests {
);
}
/// The other half of the source-write contract: when the committed
/// primary row names ANOTHER repo as first pinner, the failed
/// `record_pin_source` was load-bearing (not redundant) and the
/// push stays suppressed. Fixture: a local-only row owned by
/// repo-first (no Pinata CID yet), then a Pinata loop as
/// repo-second with `pin_repo_sources` locked. The upload lands,
/// the primary upsert keeps repo-first, the source insert stalls,
/// and coverage fails — so no pair, but the incomplete marker
/// still lands and the provider CID is still recorded.
#[sqlx::test]
async fn pinata_post_upload_stalled_source_without_coverage_suppresses_push(
pool: sqlx::PgPool,
) {
let db = crate::db::Db::for_testing(pool.clone());
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("pinata_second_source.git");
let oids = seed_loose_blobs(&repo_path, 1);
let sha = oids[0].clone();
// A local-only row owned by repo-first: the writer path below
// (repo-second, Pinata-only so far) must not claim it.
let raw =
gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata loop object 0\n").to_string();
db.record_pinned_cid_with_source(&sha, &raw, "repo-first")
.await
.unwrap();
let mut server = mockito::Server::new_async().await;
let upload = server
.mock("POST", mockito::Matcher::Any)
.with_status(200)
.with_body(r#"{"data":{"cid":"QmSecondSourceProviderCid"}}"#)
.expect(1)
.create_async()
.await;
let (_logs, _log_guard) = capture_logs();
let mut lock = lock_table(&pool, "pin_repo_sources").await;
let client = reqwest::Client::new();
let pinned = tokio::time::timeout(
Duration::from_secs(20),
pin_new_objects(
&client,
&server.url(),
"test-jwt",
&repo_path,
"git",
Duration::from_secs(30),
oids,
&db,
"repo-second",
Duration::from_millis(1500),
None,
),
)
.await
.expect("record + coverage ladders are each floored");
rollback(&mut lock).await;
drop(lock);
upload.assert_async().await;
assert!(
pinned.confirmed.is_empty(),
"repo-second is not covered by the repo-first primary row, so the \
failed source write stays suppressing: {pinned:?}"
);
assert!(
db.pin_sources_incomplete(&sha).await.unwrap(),
"the compensation marker still lands"
);
// The primary upsert itself landed (provider CID recorded under
// the first-pinner row); only the pair was suppressed.
assert!(
db.has_pinata_cid(&sha).await.unwrap(),
"the committed provider row survives the suppressed push"
);
}
/// Hold a row-level lock on one repos row in an open transaction: plain
/// `SELECT`s (fence reads, skip checks) proceed, but any `SELECT ...
/// FOR UPDATE` on the row — exactly what the fenced record takes —
/// blocks until rollback. A table-level lock would also stall the
/// fence's own epoch reads and deadlock the loop under test, proving
/// nothing about the record arm.
async fn hold_repo_row_lock(
pool: &sqlx::PgPool,
repo_id: &str,
) -> sqlx::pool::PoolConnection<sqlx::Postgres> {
let mut conn = pool.acquire().await.unwrap();
sqlx::raw_sql("BEGIN").execute(&mut *conn).await.unwrap();
sqlx::query("SELECT policy_epoch FROM repos WHERE id = $1 FOR UPDATE")
.bind(repo_id)
.fetch_one(&mut *conn)
.await
.unwrap();
conn
}
/// A timed-out `record_pinata_cid` with no verifiable row suppresses the
/// push: `record_pinata_cid` is an explicit transaction, so Elapsed proves
/// nothing, and the verify read finds no row, so nothing is proven that
/// way either. The upload mock still expects the POST — Pinata accepted
/// the bytes — but without a durable row the pair must not be returned,
/// or the reconcile counts a fill that never happened and the push path
/// advertises an unresolvable CID.
#[sqlx::test]
async fn pinata_post_upload_elapsed_record_without_row_suppresses_push(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool.clone());
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("pinata_elapsed_verify.git");
let oids = seed_loose_blobs(&repo_path, 1);
let sha = oids[0].clone();
// A real repo row: the fenced record's `FOR UPDATE` must have a row
// to block on. Against a missing row the predicate takes no lock and
// the record would sail through instead of stalling.
let repo_id = "repo-pinata-elapsed-verify";
let now = chrono::Utc::now();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.to_string(),
name: "elapsed-verify".into(),
owner_did: "did:key:zElapsedVerifyOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
// Capture the fence BEFORE locking: the capture itself is a plain
// epoch read and must succeed for the fenced record path to run.
let fence = crate::ipfs_pin::PolicyFence::capture(&db, repo_id)
.await
.expect("fence captures");
let mut server = mockito::Server::new_async().await;
let upload = server
.mock("POST", mockito::Matcher::Any)
.with_status(200)
.with_body(r#"{"data":{"cid":"QmElapsedVerifyProviderCid"}}"#)
.expect(1)
.create_async()
.await;
let (_logs, _log_guard) = capture_logs();
// Stall only the fenced record's `FOR UPDATE`: fence reads and the
// skip check are plain SELECTs and proceed, the upload lands, then
// the record elapses. The verify read proceeds too and finds no row.
let mut lock = hold_repo_row_lock(&pool, repo_id).await;
let client = reqwest::Client::new();
let pinned = tokio::time::timeout(
Duration::from_secs(30),
pin_new_objects(
&client,
&server.url(),
"test-jwt",
&repo_path,
"git",
Duration::from_secs(30),
oids,
&db,
repo_id,
Duration::from_millis(1500),
Some(&fence),
),
)
.await
.expect(
"record + verify ladders are each floored, so the call returns in seconds, \
never at the lock's lifetime",
);
rollback(&mut lock).await;
drop(lock);
upload.assert_async().await;
assert!(
pinned.confirmed.is_empty(),
"an unverified timed-out record must not return the pair: {pinned:?}"
);
assert!(
!db.has_pinata_cid(&sha).await.unwrap(),
"the cancelled record transaction must not have landed a row"
);
}
#[tokio::test]
async fn test_pin_skipped_when_jwt_empty() {
let client = reqwest::Client::new();
+10 -67
View File
@@ -70,14 +70,6 @@ impl RateLimiter {
self.window.min(Duration::from_secs(1))
}
/// The configured window, exposed so a 429 can carry an honest
/// `retry-after`: a hardcoded value lies when the operator retunes the
/// window, and a client that honors the header would retry into another
/// 429 until the real window elapses.
pub fn window(&self) -> Duration {
self.window
}
pub(crate) async fn check(&self, key: &str) -> bool {
// max_requests == 0 means the limiter is disabled, not "block all".
if self.max_requests == 0 {
@@ -301,7 +293,12 @@ pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
if let (Some(limiter), Some(did)) = (limiter, did) {
if !limiter.check(&did).await {
return too_many_requests(limiter.window().as_secs());
return (
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response();
}
}
@@ -404,72 +401,18 @@ impl<S: Send + Sync> axum::extract::FromRequestParts<S> for PeerAddr {
}
}
/// The message every per-IP brake answers with, shared so the GraphQL surface
/// (which cannot return a 429 status inside a 200 GraphQL envelope) says the
/// same thing as the REST routes.
pub const RATE_LIMIT_MESSAGE: &str = "rate limit exceeded — try again later";
/// The shared 429 response for the per-IP flood brakes. Route-agnostic: this
/// middleware now serves the push path AND the peer-sync routes, so the message
/// stays generic (the offending path is recorded in the warn log below).
pub fn too_many_requests(retry_after_secs: u64) -> Response {
pub fn too_many_requests() -> Response {
(
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", retry_after_secs.to_string())],
RATE_LIMIT_MESSAGE,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response()
}
/// The per-IP task-read brake carried as GraphQL request data: the limiter plus
/// the client key the transport already resolved.
///
/// `/graphql` is a single POST endpoint, so this brake cannot sit in middleware
/// the way `task_read_routes` mounts `rate_limit_by_ip`: that would charge every
/// unrelated query and every mutation against the task-read bucket. The two task
/// resolvers debit it instead, which also prices an aliased query honestly — ten
/// aliased `tasks` fields run the visibility gate ten times and pay ten slots,
/// where one request-scoped debit would pay one (#327 review).
/// Default cap on the number of task read fields (across all aliases) permitted
/// in a single GraphQL request before rejecting further field executions (#327 review).
pub const MAX_GRAPHQL_TASK_READS_PER_REQUEST: usize = 5;
#[derive(Clone)]
pub struct TaskReadBrake {
pub limiter: RateLimiter,
/// `None` when no key could be resolved at all (no trusted header and no
/// `ConnectInfo`, e.g. a synthetic test request). Never braked, matching
/// [`rate_limit_by_ip`].
pub key: Option<String>,
pub request_count: Arc<std::sync::atomic::AtomicUsize>,
}
impl TaskReadBrake {
/// Debit one slot for one task-read field. `true` = proceed.
pub async fn check(&self, field: &str) -> bool {
let Some(key) = self.key.as_deref() else {
return true;
};
let count = self
.request_count
.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
if count >= MAX_GRAPHQL_TASK_READS_PER_REQUEST {
tracing::warn!(
key = %key,
field = %field,
count = count + 1,
"per-request GraphQL task field limit exceeded"
);
return false;
}
if self.limiter.check(key).await {
return true;
}
tracing::warn!(key = %key, field = %field, "per-IP rate limit exceeded");
false
}
}
/// Throttle the git push path by resolved client IP. The socket peer address is
/// read from `ConnectInfo` (see `into_make_service_with_connect_info` in
/// `main`). Only skips the limiter when no key can be resolved at all.
@@ -484,7 +427,7 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
if let Some(key) = client_key(request.headers(), peer, limiter.trust) {
if !limiter.limiter.check(&key).await {
tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded");
return too_many_requests(limiter.limiter.window().as_secs());
return too_many_requests();
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+8 -73
View File
@@ -21,19 +21,6 @@ const SCAN_TOKEN_KEY_SALT: &[u8] = b"gitlawb/hkdf-salt/ipfs-scan-token";
/// front) and needs no migration, no config, and no change to this function.
const SCAN_TOKEN_KEY_INFO: &[u8] = b"gitlawb/ipfs-scan-token/v1";
/// HKDF salt for [`AppState::derive_webhook_secret_key`]. A constant, not a
/// secret: the confidentiality of the derived key rests on the node's private
/// seed being the input keying material, exactly as for the scan-token key.
const WEBHOOK_SECRET_KEY_SALT: &[u8] = b"gitlawb/hkdf-salt/webhook-secret";
/// HKDF `info` for [`AppState::derive_webhook_secret_key`]: the domain
/// separation from the scan-token key that shares the same input keying
/// material. Unlike the scan-token key, bumping the version is NOT a safe
/// rotation: sealed secrets persist, `open` has no old-key fallback, and a new
/// version renders every stored ciphertext permanently unopenable until a
/// re-seal path exists.
const WEBHOOK_SECRET_KEY_INFO: &[u8] = b"gitlawb/webhook-secret/v1";
#[derive(Clone, Debug)]
pub struct RefUpdateBroadcast {
pub repo: String,
@@ -72,10 +59,6 @@ pub struct AppState {
pub task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>,
/// GraphQL schema (queries + mutations + subscriptions)
pub graphql_schema: Arc<crate::graphql::GitlawbSchema>,
/// MAC key for task-list continuation tokens, derived from the node
/// keypair seed. Held here (and handed to the GraphQL schema) so REST and
/// GraphQL mint and accept the same tokens.
pub task_cursor_key: crate::api::task_cursor::TaskCursorKey,
/// Fly.io machine ID — used for fly-replay routing in multi-machine deployments
pub machine_id: Option<String>,
/// Centralized repo storage: local disk cache + optional Tigris backend
@@ -186,11 +169,6 @@ pub struct AppState {
/// already has on disk, so derivation is also the smaller mechanism. Rotation is a
/// bump of the version in [`SCAN_TOKEN_KEY_INFO`].
pub ipfs_scan_token_key: Arc<[u8; 32]>,
/// Key sealing webhook HMAC secrets at rest (#506), derived from the node's
/// persistent identity by [`AppState::derive_webhook_secret_key`]. The
/// webhook delivery path opens a secret with it only at signing time; the
/// `webhooks.secret` column stores the ciphertext (see `webhook_secret`).
pub webhook_secret_key: Arc<[u8; 32]>,
/// Hard ceiling on the byte size of an object `GET /ipfs/{cid}` will buffer and
/// serve (default `api::ipfs::MAX_SERVED_OBJECT_BYTES`). The serve reads via a
/// blocking `git cat-file` and buffers the whole object; without a bound a large
@@ -212,9 +190,6 @@ pub struct AppState {
/// sink as trigger and accepts unsigned requests from known peers, so it is
/// braked too; each peer's distinct IP gets its own bucket.
pub peer_write_rate_limiter: RateLimiter,
/// Per-client-IP federation budget: 12 requests per minute, with at most
/// 10,000 tracked source keys. Shared by router instances and swept below.
pub federated_rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
@@ -348,16 +323,6 @@ pub struct AppState {
/// (`with_default_max_keys`, reject-before-insert) so a source-key farm cannot grow
/// it (INV-15).
pub git_ipfs_walk_per_caller: crate::rate_limit::PerCallerConcurrency,
/// Per-client-IP rate limiter for the task read routes. `GET /api/v1/tasks`
/// and `GET /api/v1/tasks/{id}` are anonymously reachable and run the #268
/// visibility gate (a task lookup plus deduped-repo and visibility-rule
/// queries) before returning the opaque 404, so an unauthenticated prober
/// costs the node real work per request whether or not anything is visible.
/// Keyed on the resolved client IP via `push_limiter_trust`. Layered on
/// `task_read_routes` via `rate_limit_by_ip`.
/// The same per-IP hourly budget applies to GraphQL `tasks` / `task`
/// queries and WebSocket task queries via `TaskReadBrake`.
pub task_read_rate_limiter: RateLimiter,
/// The `git` executable the served-git withheld-blob walk spawns. Production is
/// `"git"` (resolved via PATH); injectable so a fake `git` can drive the walk's
/// process-group teardown in handler tests without mutating the process-global
@@ -386,8 +351,6 @@ impl AppState {
self.ipfs_work_rate_limiter.cleanup().await;
self.sync_trigger_rate_limiter.cleanup().await;
self.peer_write_rate_limiter.cleanup().await;
self.task_read_rate_limiter.cleanup().await;
self.federated_rate_limiter.cleanup().await;
}
/// Trigger graceful shutdown. Idempotent — calling more than once
@@ -449,48 +412,20 @@ impl AppState {
/// identity, the token format, or any caller, so rotation is a constant change
/// rather than a fork of this derivation.
pub(crate) fn derive_scan_token_key(keypair: &Keypair) -> [u8; 32] {
Self::hkdf_sha256(
SCAN_TOKEN_KEY_SALT,
SCAN_TOKEN_KEY_INFO,
keypair.to_seed().as_slice(),
)
}
/// Derive the at-rest webhook-secret sealing key from the node identity.
///
/// Same HKDF construction as [`AppState::derive_scan_token_key`] under its
/// own [`WEBHOOK_SECRET_KEY_INFO`]/[`WEBHOOK_SECRET_KEY_SALT`] pair, so it
/// is derived (a restart re-derives the same key, and old secrets remain
/// readable) yet domain-separated from the signing key and the scan-token
/// key. The doc on [`SCAN_TOKEN_KEY_INFO`] argues why derivation (not a
/// boot-random key) is the right trade: the secret's confidentiality still
/// rests on the node's private seed. Rotation is NOT a version bump here,
/// unlike for the scan-token key: [`WEBHOOK_SECRET_KEY_INFO`] explains that
/// bumping renders every stored ciphertext unopenable until a re-seal path
/// exists.
pub(crate) fn derive_webhook_secret_key(keypair: &Keypair) -> [u8; 32] {
Self::hkdf_sha256(
WEBHOOK_SECRET_KEY_SALT,
WEBHOOK_SECRET_KEY_INFO,
keypair.to_seed().as_slice(),
)
}
/// HKDF-Extract-then-Expand (RFC 5869) over the node seed: PRK =
/// HMAC(salt, ikm), then one expand block T(1) = HMAC(PRK, info || 0x01).
/// One 32-byte output needs exactly one block of SHA-256, so there is no
/// counter loop to get wrong. Salt and info are the caller's domain
/// separation; shared by every identity-derived key in this module.
fn hkdf_sha256(salt: &[u8], info: &[u8], ikm: &[u8]) -> [u8; 32] {
use hmac::{Hmac, Mac};
type HmacSha256 = Hmac<sha2::Sha256>;
let mut extract = HmacSha256::new_from_slice(salt).expect("HMAC takes a key of any size");
extract.update(ikm);
let seed = keypair.to_seed();
// HKDF-Extract: PRK = HMAC(salt, ikm).
let mut extract =
HmacSha256::new_from_slice(SCAN_TOKEN_KEY_SALT).expect("HMAC takes a key of any size");
extract.update(seed.as_slice());
let prk = extract.finalize().into_bytes();
// HKDF-Expand, one block: T(1) = HMAC(PRK, info || 0x01). One 32-byte output
// needs exactly one block of SHA-256, so there is no counter loop to get wrong.
let mut expand =
HmacSha256::new_from_slice(prk.as_slice()).expect("HMAC takes a key of any size");
expand.update(info);
expand.update(SCAN_TOKEN_KEY_INFO);
expand.update(&[0x01]);
expand.finalize().into_bytes().into()
}
+11 -874
View File
@@ -14,7 +14,6 @@
use std::collections::HashMap;
use std::path::Path;
use std::path::PathBuf;
use std::sync::Arc;
use gitlawb_core::identity::Keypair;
@@ -436,10 +435,9 @@ async fn process_batch(
.await;
}
Err(e) => {
// Prune vs terminal classification lives in
// `handle_sync_item_error` (defer = stays pending for the
// next tick; anything else = failed).
handle_sync_item_error(db, &item, &origin_url, e).await;
warn!(repo = %item.repo, origin = %origin_url, err = %e, "repo sync failed");
let _ = db.mark_sync_failed(&item.id).await;
crate::metrics::record_sync_processed("failed");
}
}
}
@@ -578,18 +576,7 @@ async fn replicate_encrypted_blobs(
let url = format!("{origin_url}/api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate");
let resp = match client.get(&url).send().await {
Ok(r) if r.status().is_success() => r,
// A peer that predates the replicate route answers 404: expected and
// quiet. Anything else is the route existing and refusing or failing,
// which should be observable rather than read as route-absent.
Ok(r) if r.status().as_u16() == 404 => return,
Ok(r) => {
tracing::warn!(repo = %repo, status = %r.status(), "encrypted-blobs replicate refused; skipping");
return;
}
Err(e) => {
tracing::warn!(repo = %repo, error = %e, "encrypted-blobs replicate request failed; skipping");
return;
}
_ => return,
};
let parsed: ReplicateResponse = match resp.json().await {
Ok(p) => p,
@@ -704,284 +691,6 @@ async fn existing_promisor_state(repo: &str) -> PromisorProbe {
}
}
/// Byte-prefix check for the two node-managed blob namespaces exempt from
/// the fail-closed commit walk (`visibility_pack.rs`): request markers
/// (`refs/gitlawb/requests/`) and issue records (`refs/gitlawb/issues/`).
/// Bytewise so non-UTF-8 refnames (which git accepts) are classified, not
/// lossy-decoded into a false exempt match. Extracted so the predicate is
/// unit-testable; [`prune_non_exempt_gitlawb_refs`] enforces it on import.
fn gitlawb_ref_is_exempt(raw_refname: &[u8]) -> bool {
raw_refname.starts_with(b"refs/gitlawb/requests/")
|| raw_refname.starts_with(b"refs/gitlawb/issues/")
}
/// Parse one `git show-ref` output line into its refname bytes: `<hex-sha>`
/// (40-hex SHA-1 or 64-hex SHA-256, which a sha256 mirror inherits from its
/// origin), one space, then the refname. Empty lines (trailing newline) yield
/// `Ok(None)`; anything else-shaped is an anomaly the caller fails on rather
/// than skips — silently passing an unclassifiable line could leave an evil
/// ref imported. Extracted so malformed input is unit-testable without git.
fn show_ref_refname(line: &[u8]) -> anyhow::Result<Option<&[u8]>> {
if line.is_empty() {
return Ok(None);
}
let Some(sp) = line.iter().position(|b| *b == b' ') else {
return Err(anyhow::anyhow!("git show-ref emitted an unparsable line"));
};
let (sha, raw) = (&line[..sp], &line[sp + 1..]);
let sha_ok = (sha.len() == 40 || sha.len() == 64) && sha.iter().all(|b| b.is_ascii_hexdigit());
if !sha_ok || raw.is_empty() {
return Err(anyhow::anyhow!("git show-ref emitted an unparsable line"));
}
Ok(Some(raw))
}
/// Delete imported `refs/gitlawb/*` refs outside the two exempt subtrees.
///
/// Mirrors import whatever the origin advertises (`+refs/*:refs/*`), and
/// `transfer.hideRefs` is a serve-side knob: a repo's own `git fetch` never
/// consults local hideRefs, and origins hide only `requests/` anyway. So a
/// planted `refs/gitlawb/<other>` blob ref — creatable on any pre-gate node —
/// lands on every mirror and then trips `assert_all_refs_are_commits` closed,
/// wedging serving on each one. Prune at import instead: after clone/fetch,
/// list `refs/gitlawb` and delete every ref outside the exempt subtrees
/// (negative refspecs cannot re-include them, so post-import deletion is the
/// mechanism that works).
///
/// Callers map a transient prune error to [`PruneFailed`] so the batch
/// defers the row (stays `pending`, retried next tick) instead of failing
/// it: a failed row would leave a possibly-wedged mirror unretried until
/// the next peer announcement. Deterministic refusals (identity mismatch,
/// malformed enumeration) map to [`PruneInvalid`], which fails terminally
/// since no retry can clear them.
///
/// Enumeration is `git show-ref` (no pattern), parsed as raw bytes: refnames
/// cannot contain newlines, so one line per ref is lossless even for
/// non-UTF-8 names, which a lossy decode would miss. The `<sha> SP <refname>`
/// structure (not `for-each-ref --format=%(refname)`, which carries no object
/// id) is what lets the parser width-check 40-hex SHA-1 vs 64-hex SHA-256
/// and reject anything else-shaped instead of skipping it.
///
/// Fail-closed throughout: the exit code is classified (only `1` with empty
/// output means "no refs"; anything else non-zero, e.g. 128 on a missing
/// repo, is fatal, since a silent skip would leave an imported evil ref in
/// place while the sync reports success), and any line that is not
/// `<hex-sha> SP <refname>` — 40-hex SHA-1 or 64-hex SHA-256, which a sha256
/// mirror inherits from its origin — fails the sync rather than being
/// skipped into an unpruned wedge.
/// Bounded git spawn for the prune below: `kill_on_drop` plus a hard timeout
/// so a wedged child cannot stall a mirror import or sweep forever. The
/// refname delete runs on exact bytes, so args arrive as `OsStr`, not `str`.
/// Non-PruneInvalid failures (including a timeout) wrap as PruneFailed and
/// defer for retry, matching the transient classification below.
async fn prune_git(
local_path: &Path,
args: &[&std::ffi::OsStr],
label: &str,
) -> anyhow::Result<std::process::Output> {
let fut = async {
// allow-unbounded-git: bounded by the tokio::time::timeout wrapping
// this future below, plus kill_on_drop.
let mut cmd = tokio::process::Command::new("git");
cmd.kill_on_drop(true);
cmd.arg("-C").arg(local_path).args(args);
cmd.output().await
};
tokio::time::timeout(std::time::Duration::from_secs(30), fut)
.await
.map_err(|_| anyhow::anyhow!("{label} timed out"))?
.map_err(|e| anyhow::anyhow!("{label} failed to spawn: {e}"))
}
async fn prune_non_exempt_gitlawb_refs(local_path: &Path) -> anyhow::Result<()> {
let local_str = local_path.to_str().unwrap_or(".");
// Prove the path is itself a repo before trusting the enumeration:
// `git -C` walks up to the nearest ancestor repo, so inside a non-repo
// child (a partially-created mirror) `show-ref` would exit 0 with the
// ANCESTOR's refs and `update-ref -d` would then run against that repo.
// Require `rev-parse --absolute-git-dir` to resolve to the path itself
// (mirrors are bare repos, where the git dir IS the path).
let git_dir_out = prune_git(
local_path,
&[
std::ffi::OsStr::new("rev-parse"),
std::ffi::OsStr::new("--absolute-git-dir"),
],
"git rev-parse",
)
.await?;
if !git_dir_out.status.success() {
let stderr = String::from_utf8_lossy(&git_dir_out.stderr);
return Err(anyhow::anyhow!(PruneInvalid(format!(
"mirror path is not a git repo (rev-parse failed): {stderr}"
))));
}
let actual_dir = PathBuf::from(
String::from_utf8_lossy(&git_dir_out.stdout)
.trim()
.to_string(),
);
let canonical_path = local_path
.canonicalize()
.map_err(|e| anyhow::anyhow!(PruneInvalid(format!("mirror path does not resolve: {e}"))))?;
let canonical_git_dir = actual_dir
.canonicalize()
.map_err(|e| anyhow::anyhow!(PruneInvalid(format!("git dir does not resolve: {e}"))))?;
if canonical_git_dir != canonical_path {
return Err(anyhow::anyhow!(PruneInvalid(format!(
"mirror path {local_str} is not itself a repo (git dir resolves elsewhere); refusing to enumerate"
))));
}
let out = prune_git(
local_path,
&[std::ffi::OsStr::new("show-ref")],
"git show-ref",
)
.await?;
// Classify on the exit code (the `existing_promisor_state` precedent):
// 0 parses, 1 with empty output is an empty repo, anything else is a
// fatal enumeration failure that must not read as "nothing to prune".
match out.status.code() {
Some(0) => {}
Some(1) if out.stdout.is_empty() => return Ok(()),
_ => {
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!(
"git show-ref failed with code {:?}: {stderr}",
out.status.code()
));
}
}
for line in out.stdout.split(|b| *b == b'\n') {
// Malformed lines are deterministic (same git, same output every
// tick): fail terminally via PruneInvalid, not transiently.
let Some(raw) =
show_ref_refname(line).map_err(|e| anyhow::anyhow!(PruneInvalid(e.to_string())))?
else {
continue;
};
if !(raw == b"refs/gitlawb" || raw.starts_with(b"refs/gitlawb/"))
|| gitlawb_ref_is_exempt(raw)
{
continue;
}
// Delete by exact bytes (not the lossy display string below): a
// non-UTF-8 refname would otherwise survive the prune.
let refname_os = os_from_raw_refname(raw);
let shown = refname_os.to_string_lossy().into_owned();
let delete = prune_git(
local_path,
&[
std::ffi::OsStr::new("update-ref"),
std::ffi::OsStr::new("-d"),
refname_os.as_os_str(),
std::ffi::OsStr::new("--no-deref"),
],
"git update-ref -d",
)
.await?;
if !delete.status.success() {
let stderr = String::from_utf8_lossy(&delete.stderr);
return Err(anyhow::anyhow!(
"git update-ref -d {shown} failed (mirror still carries a non-exempt refs/gitlawb ref): {stderr}"
));
}
tracing::info!(
gitlawb_ref = %shown,
"mirror: pruned non-exempt refs/gitlawb ref on import"
);
}
Ok(())
}
/// Lossless bytes → OsString for `update-ref -d`: non-UTF-8 refnames must be
/// deleted by exact bytes. Unix is the serving platform; elsewhere fall back
/// to a lossy decode (best effort where exact bytes are unrepresentable).
#[cfg(unix)]
fn os_from_raw_refname(raw: &[u8]) -> std::ffi::OsString {
use std::os::unix::ffi::OsStringExt;
std::ffi::OsString::from_vec(raw.to_vec())
}
#[cfg(not(unix))]
fn os_from_raw_refname(raw: &[u8]) -> std::ffi::OsString {
String::from_utf8_lossy(raw).into_owned().into()
}
/// Marker for mirror-prune failures: the clone/fetch succeeded, so the
/// mirror is on disk, but a non-exempt `refs/gitlawb/*` ref may still be
/// imported. Local-git conditions are transient (a retry can succeed), while
/// `mark_sync_failed` is terminal until the next peer announcement — and a
/// failed row would leave a possibly-wedged mirror unretried in the
/// meantime. The batch error arm downcasts to this and defers the row
/// (stays `pending`, retried on the next 30s tick, where fetch+prune are
/// idempotent) instead of failing it.
#[derive(Debug)]
struct PruneFailed(String);
impl std::fmt::Display for PruneFailed {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "mirror prune failed: {}", self.0)
}
}
impl std::error::Error for PruneFailed {}
/// Marker for deterministic mirror-prune refusals: the identity check (path
/// is not itself a repo) and malformed `show-ref` output cannot clear on
/// retry — re-running fetch+prune every tick would spin forever on a
/// permanently bad path. Unlike [`PruneFailed`], these fail the row
/// terminally so the condition surfaces instead of looping silently.
#[derive(Debug)]
struct PruneInvalid(String);
/// Preserve a deterministic prune refusal through the clone/fetch call
/// sites: `prune_non_exempt_gitlawb_refs` already returns `PruneInvalid`
/// for identity mismatches and malformed enumeration, and re-wrapping
/// every error as `PruneFailed` would stringify that type away before
/// `handle_sync_item_error` downcasts it — leaving a permanently invalid
/// mirror deferred (fetch+prune every tick) instead of failed. Transient
/// failures keep the `PruneFailed` deferral.
fn wrap_prune_error(e: anyhow::Error) -> anyhow::Error {
match e.downcast::<PruneInvalid>() {
Ok(invalid) => anyhow::Error::new(invalid),
Err(other) => anyhow::anyhow!(PruneFailed(other.to_string())),
}
}
impl std::fmt::Display for PruneInvalid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "mirror prune refused: {}", self.0)
}
}
impl std::error::Error for PruneInvalid {}
/// Classify a mirror sync failure for one queue row. Transient prune
/// failures ([`PruneFailed`]) defer: the mirror is on disk but may still
/// carry a non-exempt refs/gitlawb ref, a transient local-git condition a
/// retry can clear — failing the row would leave it unretried until the
/// next peer announcement, wedging serving in the meantime. Deterministic
/// refusals ([`PruneInvalid`]) and everything else fail the row terminally:
/// retrying a permanently bad path every tick would spin forever instead of
/// surfacing. Extracted so the deferral (not just the prune itself) is
/// unit-testable.
async fn handle_sync_item_error(
db: &Db,
item: &crate::db::SyncQueueItem,
origin_url: &str,
err: anyhow::Error,
) {
if err.downcast_ref::<PruneFailed>().is_some() {
warn!(repo = %item.repo, origin = %origin_url, err = %err, "mirror prune failed; leaving sync row pending for retry");
crate::metrics::record_sync_processed("deferred");
return;
}
warn!(repo = %item.repo, origin = %origin_url, err = %err, "repo sync failed");
let _ = db.mark_sync_failed(&item.id).await;
crate::metrics::record_sync_processed("failed");
}
/// Mirror-clone a repo from a remote URL into a local bare repo.
/// `Promisor` mode adds `--filter=blob:limit=10g`, which marks the repo a git
/// promisor (so a pack with origin-omitted withheld blobs is accepted) while
@@ -992,7 +701,6 @@ async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> an
if mode == MirrorMode::Promisor {
args.push("--filter=blob:limit=10g");
}
args.push("--");
args.push(remote_url);
args.push(local_str);
@@ -1006,13 +714,6 @@ async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> an
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!("git clone --mirror failed: {stderr}"));
}
// A fresh mirror imports everything the origin advertises, including any
// planted non-exempt `refs/gitlawb/*` ref; prune before serving from it.
// Prune outcome flows through `wrap_prune_error`: transient failures
// defer (stays pending), deterministic refusals fail terminally.
prune_non_exempt_gitlawb_refs(local_path)
.await
.map_err(wrap_prune_error)?;
Ok(())
}
@@ -1028,16 +729,9 @@ async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> an
async fn fetch_repo(local_path: &Path, remote_url: &str, mode: MirrorMode) -> anyhow::Result<()> {
let local_str = local_path.to_str().unwrap_or(".");
git_run(&[
"-C", local_str, "remote", "set-url", "origin", "--", remote_url,
])
.await?;
git_run(&["-C", local_str, "remote", "set-url", "origin", remote_url]).await?;
// The fetch imports whatever the origin advertises via the stored
// `+refs/*:refs/*` refspec; prune non-exempt `refs/gitlawb/*` refs after
// every fetch so one planted origin ref cannot wedge serving on the
// mirror (see `prune_non_exempt_gitlawb_refs`).
let fetch_result = match mode {
match mode {
MirrorMode::Promisor => {
git_run(&["-C", local_str, "config", "remote.origin.promisor", "true"]).await?;
git_run(&[
@@ -1077,13 +771,7 @@ async fn fetch_repo(local_path: &Path, remote_url: &str, mode: MirrorMode) -> an
git_run(&["-C", local_str, "fetch", "--prune", "origin"]).await
}
}
};
fetch_result?;
// Prune after every fetch (see above); outcome flows through
// `wrap_prune_error` like the clone path.
prune_non_exempt_gitlawb_refs(local_path)
.await
.map_err(wrap_prune_error)
}
}
#[cfg(test)]
@@ -1318,82 +1006,6 @@ mod tests {
assert_eq!(probe, PromisorProbe::NotPromisor);
}
/// #374: `clone_repo` interpolates the peer's origin URL into a `git
/// clone --mirror` argument list. Without a `--` delimiter git parses a
/// remote beginning with `-` as an option instead of a repository.
///
/// The delimiter is what forces the same string to be a repository, and
/// git says so in its own words: it reports the whole argument as a
/// repository it cannot find. Undelimited, git consumes it as an option
/// and fails against the *destination* instead (`fatal: repository '<dest>' does not exist`),
/// never naming the injected string — which is why this assertion is RED
/// before the fix and GREEN after.
///
/// The payload is deliberately free of `:` and spaces: a colon would make
/// git read the argument as an ssh-style `host:path` URL and report a
/// hostname rather than a repository, which is a different code path.
#[tokio::test]
async fn clone_never_parses_a_remote_as_a_git_option() {
let td = TempDir::new().unwrap();
let dest = td.path().join("mirror.git");
let injected = "--upload-pack=false";
let err = clone_repo(injected, &dest, MirrorMode::Plain)
.await
.expect_err("a remote that is not a repository must fail the clone")
.to_string();
assert!(
err.contains(injected),
"git must report the argument as a repository, not consume it as an option: {err}"
);
}
/// #374: `fetch_repo` interpolates the peer's origin URL into
/// `git remote set-url`, a second argv construction with the same contract as
/// the clone. It needs its own proof: the clone test cannot reach this path.
///
/// The assertion is on stored state rather than on the error string, and that
/// is deliberate. `git_run` formats its failure as `git {args:?} failed:
/// {stderr}`, so the injected value is in the message via the debug-printed
/// argv whether or not the delimiter is present — asserting on containment
/// would pass with `--` deleted. Delimited, `set-url` succeeds and stores the
/// value verbatim; undelimited, git exits 129 with `unknown option` and leaves
/// the previous URL in place, so this goes red without the fix.
///
/// The fetch that follows the rewrite fails, because the stored URL is not a
/// repository. That is expected and ignored: the property under test is what
/// `set-url` wrote, not whether the subsequent fetch could succeed.
#[tokio::test]
async fn fetch_never_parses_a_remote_as_a_git_option() {
let td = TempDir::new().unwrap();
let local = td.path().join("mirror.git");
let local_str = local.to_str().unwrap();
let placeholder = "http://example.invalid/x.git";
assert!(Command::new("git")
.args(["init", "-q", "--bare", local_str])
.status()
.unwrap()
.success());
assert!(Command::new("git")
.args(["-C", local_str, "remote", "add", "origin", placeholder])
.status()
.unwrap()
.success());
let injected = "--upload-pack=false";
let _ = fetch_repo(&local, injected, MirrorMode::Plain).await;
assert_eq!(
git_config_get(local_str, "remote.origin.url")
.await
.as_deref(),
Some(injected),
"set-url must store an option-shaped remote verbatim, not reject it as an option"
);
}
#[tokio::test]
async fn probe_reports_unknown_on_git_error() {
// A path git cannot resolve as a repo at all (exit 128) is an indeterminate
@@ -1638,18 +1250,6 @@ mod tests {
.await;
}
/// Use a rooted path without a Windows drive prefix in the remote fixture.
/// The destination still resolves on the temp directory's current drive,
/// while the remote's relative path contains no illegal colon component.
fn absolute_slug_path(path: &Path) -> String {
assert!(path.is_absolute());
path.components()
.filter(|part| !matches!(part, std::path::Component::Prefix(_)))
.collect::<std::path::PathBuf>()
.to_string_lossy()
.replace('\\', "/")
}
#[sqlx::test]
async fn process_batch_rejects_slug_escaping_repos_dir(pool: PgPool) {
// The verified escape from #272: `PathBuf::join` discards everything
@@ -1665,13 +1265,12 @@ mod tests {
// fails on but leaves the parent it created, so assert on both.
let outside = TempDir::new().unwrap();
let escape_dir = outside.path().join("nest");
let escape_path = absolute_slug_path(&escape_dir);
let slug = format!("a/{escape_path}/escape");
let slug = format!("a/{}/escape", escape_dir.display());
let escape_target = escape_dir.join("escape.git");
// Serve the composed URL for real, so a run without the guard genuinely
// clones outside the root rather than merely failing at git.
let rel = format!("a{escape_path}/escape");
let rel = format!("a{}/escape", escape_dir.display());
let (_remote, peer_url) = rooted_remote(&[&rel]);
let did = "did:key:z6MkAttacker";
@@ -1791,7 +1390,6 @@ mod tests {
// ── canonical containment before the git call (issue #272) ───────────────
/// Every ref in `repo`, as one string, for a before/after comparison.
#[cfg(unix)]
fn refs_of(repo: &Path) -> String {
let out = Command::new("git")
.args(["-C", repo.to_str().unwrap(), "for-each-ref"])
@@ -2124,7 +1722,6 @@ mod tests {
/// starvation tests is fixed rather than dependent on how fast the loop
/// runs. Two rows enqueued in the same microsecond would otherwise order
/// arbitrarily.
#[cfg(unix)]
async fn enqueue_at(db: &Db, pool: &PgPool, repo: &str, did: &str, enqueued_at: &str) {
enqueue(db, repo, did).await;
sqlx::query("UPDATE sync_queue SET enqueued_at = $1 WHERE repo = $2")
@@ -2318,8 +1915,7 @@ mod tests {
let outside = TempDir::new().unwrap();
let escape_dir = outside.path().join("nest");
let escape_path = absolute_slug_path(&escape_dir);
let slug = format!("a/{escape_path}/gitlawb-probe");
let slug = format!("a/{}/gitlawb-probe", escape_dir.display());
let escape_target = escape_dir.join("gitlawb-probe.git");
// Two things this fixture must get right or the test is green for the
@@ -2330,7 +1926,7 @@ mod tests {
// run without the guard genuinely clones outside the root instead of
// just failing at git. Db::upsert_peer cannot seed this row: it gates on
// is_public_http_url, which rejects file://.
let rel = format!("a{escape_path}/gitlawb-probe");
let rel = format!("a{}/gitlawb-probe", escape_dir.display());
let (_remote, peer_url) = rooted_remote(&[&rel]);
let did = "did:key:z6MkAttacker";
seed_local_peer(&pool, did, &peer_url).await;
@@ -2416,463 +2012,4 @@ mod tests {
assert!(object_count(&mirror) > 0, "mirror has no objects");
assert_eq!(sync_status(&pool, "z6Mkfoo/hello").await, "done");
}
/// The import-exemption predicate is bytewise over both node-managed
/// namespaces and nothing else. Non-UTF-8 bytes never match (git accepts
/// them, so a lossy decode could false-exempt). Narrowing either prefix
/// turns these red.
#[test]
fn gitlawb_import_exemption_covers_both_namespaces_by_bytes() {
for exempt in [
&b"refs/gitlawb/requests/550e8400-e29b-41d4-a716-446655440000"[..],
&b"refs/gitlawb/issues/abc123"[..],
] {
assert!(
gitlawb_ref_is_exempt(exempt),
"import must keep {}",
String::from_utf8_lossy(exempt)
);
}
for pruned in [
&b"refs/gitlawb/evil"[..],
&b"refs/gitlawb"[..],
&b"refs/gitlawbfoo"[..],
&b"refs/heads/main"[..],
&b""[..],
// Raw 0xFF byte: never an exempt prefix, always pruned.
&b"refs/gitlawb/\xffvil"[..],
] {
assert!(
!gitlawb_ref_is_exempt(pruned),
"import must prune {}",
String::from_utf8_lossy(pruned)
);
}
}
/// End-to-end prune on a scratch bare repo: a planted non-exempt
/// `refs/gitlawb/*` blob ref is deleted, exempt `requests/`/`issues/`
/// blob refs and content refs survive. Removing the prune call from
/// `clone_repo`/`fetch_repo` leaves the evil ref in place, which is what
/// wedges serving on mirrors — this test plants exactly that state and
/// proves the prune removes it.
#[test]
fn prune_removes_non_exempt_gitlawb_refs_and_keeps_exempt() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("m.git");
std::fs::create_dir_all(&work).unwrap();
// A real commit for the content branch (branch refs must point
// at commits; the evil/exempt refs below point at blobs).
std::fs::write(work.join("f.txt"), b"x\n").unwrap();
let run_work = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&work)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
run_work(&["init", "-q"]);
run_work(&["config", "user.email", "t@t"]);
run_work(&["config", "user.name", "t"]);
run_work(&["add", "f.txt"]);
run_work(&["commit", "-qm", "init"]);
assert!(
Command::new("git")
.args([
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap()
])
.current_dir(td.path())
.status()
.unwrap()
.success(),
"git clone --bare failed"
);
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&bare)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
let run_out = |args: &[&str]| {
let out = Command::new("git")
.args(args)
.current_dir(&bare)
.output()
.unwrap();
assert!(out.status.success(), "git {args:?} failed");
out.stdout
};
run(&["init", "-q", "--bare", "."]);
std::fs::write(bare.join("blob-body"), b"marker\n").unwrap();
let blob = String::from_utf8_lossy(&run_out(&["hash-object", "-w", "blob-body"]))
.trim()
.to_string();
// The clone carries the content branch; plant the evil ref and
// both exempt refs (all blob-pointing, like node metadata).
run(&["update-ref", "refs/gitlawb/evil", &blob]);
run(&["update-ref", "refs/gitlawb/requests/r1", &blob]);
run(&["update-ref", "refs/gitlawb/issues/i1", &blob]);
prune_non_exempt_gitlawb_refs(&bare)
.await
.expect("prune succeeds");
let raw_refs = run_out(&["for-each-ref", "--format=%(refname)"]);
let refs = String::from_utf8_lossy(&raw_refs);
let set: std::collections::HashSet<&str> = refs
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
assert!(
!set.contains("refs/gitlawb/evil"),
"non-exempt blob ref must be pruned, still present: {set:?}"
);
for keep in ["refs/gitlawb/requests/r1", "refs/gitlawb/issues/i1"] {
assert!(set.contains(keep), "{keep} must survive the prune: {set:?}");
}
assert!(
set.iter().any(|r| r.starts_with("refs/heads/")),
"content branch ref must survive the prune: {set:?}"
);
});
}
/// `show-ref` line parser: 40-hex SHA-1 and 64-hex SHA-256 lines yield
/// the refname, empty lines yield None, and anything else-shaped errors
/// (the caller fails the sync rather than skipping into an unpruned
/// wedge). Reverting to a 40-only width check turns the sha256 case red.
#[test]
fn show_ref_line_parser_accepts_both_hash_widths_and_rejects_junk() {
// 40-hex SHA-1 line.
let sha1_line = format!("{} refs/gitlawb/evil", "a".repeat(40));
assert_eq!(
show_ref_refname(sha1_line.as_bytes()).unwrap(),
Some(b"refs/gitlawb/evil".as_slice())
);
// 64-hex SHA-256 line (a sha256 mirror inherits the width).
let sha256_line = format!("{} refs/gitlawb/evil", "b".repeat(64));
assert_eq!(
show_ref_refname(sha256_line.as_bytes()).unwrap(),
Some(b"refs/gitlawb/evil".as_slice())
);
// Non-UTF-8 refname passes through as exact bytes.
let mut raw = vec![b'c'; 40];
raw.extend_from_slice(b" refs/gitlawb/\xffvil");
assert_eq!(
show_ref_refname(&raw).unwrap(),
Some(b"refs/gitlawb/\xffvil".as_slice())
);
// Trailing newline residue.
assert_eq!(show_ref_refname(b"").unwrap(), None);
// No space, short sha, non-hex sha, empty refname: all errors.
let bads: Vec<Vec<u8>> = vec![
b"nospaceshere".to_vec(),
format!("{}refs/x", "a".repeat(40)).into_bytes(),
[b'z'; 40]
.iter()
.chain(b" refs/x".iter())
.copied()
.collect(),
format!("{} ", "a".repeat(40)).into_bytes(),
format!("{} refs/x", "a".repeat(41)).into_bytes(),
];
for bad in &bads {
assert!(
show_ref_refname(bad).is_err(),
"malformed show-ref line must error, not skip"
);
}
}
/// Pruning a path that is not a repo errors. Two shapes: a missing path
/// fails `rev-parse` outright (exit 128), while a plain subdirectory of
/// a repo would otherwise enumerate the ancestor's refs — covered by the
/// identity test below. Either way the prune refuses instead of pruning
/// the wrong repo or silently skipping an evil ref.
#[test]
fn prune_on_non_repo_path_errors() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let err =
prune_non_exempt_gitlawb_refs(Path::new("/tmp/gitlawb-definitely-not-a-repo-xyz"))
.await
.expect_err("non-repo path must fail the prune, not skip it");
assert!(
err.to_string().contains("not a git repo")
|| err.to_string().contains("does not resolve"),
"unexpected error shape: {err:#}"
);
});
}
/// Identity check: pruning a plain subdirectory of a repo refuses instead
/// of enumerating the ancestor. `git -C` on the child exits 0 from
/// `show-ref` with the ANCESTOR's refs, so without the
/// `rev-parse --absolute-git-dir` identity check the delete loop would
/// enumerate and delete the wrong repository's refs. Deleting the check
/// makes the prune succeed and removes the ancestor's evil ref — both
/// assertions go red.
#[test]
fn prune_on_non_repo_child_of_repo_refuses_and_touches_nothing() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let td = TempDir::new().unwrap();
let ancestor = td.path().join("ancestor.git");
std::fs::create_dir_all(&ancestor).unwrap();
let run_ancestor = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&ancestor)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
run_ancestor(&["init", "-q", "--bare", "."]);
// Blob target for the planted ref (mirrors the node-metadata shape).
std::fs::write(ancestor.join("blob-body"), b"marker\n").unwrap();
let out = Command::new("git")
.args(["hash-object", "-w", "blob-body"])
.current_dir(&ancestor)
.output()
.unwrap();
assert!(out.status.success(), "git hash-object failed");
let blob = String::from_utf8_lossy(&out.stdout).trim().to_string();
run_ancestor(&["update-ref", "refs/gitlawb/evil", &blob]);
// A plain child directory: not itself a repo.
let child = ancestor.join("not-a-repo");
std::fs::create_dir_all(&child).unwrap();
let err = prune_non_exempt_gitlawb_refs(&child)
.await
.expect_err("pruning a non-repo child must refuse, not enumerate the ancestor");
assert!(
err.to_string().contains("not itself a repo"),
"refusal must name the identity mismatch, got: {err:#}"
);
// The ancestor's evil ref is untouched.
let refs_out = Command::new("git")
.args(["for-each-ref", "--format=%(refname)"])
.current_dir(&ancestor)
.output()
.unwrap();
let refs = String::from_utf8_lossy(&refs_out.stdout);
assert!(
refs.lines().any(|l| l.trim() == "refs/gitlawb/evil"),
"ancestor evil ref must survive the refused prune"
);
});
}
/// sha256 mirror: a 64-hex `show-ref` line parses and the planted ref is
/// pruned. A 40-only parser skips every line and leaves the evil ref.
#[test]
fn prune_parses_sha256_show_ref_and_prunes() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("test runtime");
rt.block_on(async {
let td = TempDir::new().unwrap();
let bare = td.path().join("m256.git");
std::fs::create_dir_all(&bare).unwrap();
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&bare)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
let run_out = |args: &[&str]| {
let out = Command::new("git")
.args(args)
.current_dir(&bare)
.output()
.unwrap();
assert!(out.status.success(), "git {args:?} failed");
out.stdout
};
run(&["init", "-q", "--bare", "--object-format=sha256", "."]);
std::fs::write(bare.join("blob-body"), b"marker\n").unwrap();
let blob = String::from_utf8_lossy(&run_out(&["hash-object", "-w", "blob-body"]))
.trim()
.to_string();
assert_eq!(
blob.len(),
64,
"sanity: sha256 repo must mint 64-hex objects"
);
run(&["update-ref", "refs/gitlawb/evil", &blob]);
run(&["update-ref", "refs/gitlawb/issues/i1", &blob]);
prune_non_exempt_gitlawb_refs(&bare)
.await
.expect("prune succeeds on a sha256 repo");
let raw_refs = run_out(&["for-each-ref", "--format=%(refname)"]);
let refs = String::from_utf8_lossy(&raw_refs);
let set: std::collections::HashSet<&str> = refs
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
assert!(
!set.contains("refs/gitlawb/evil"),
"evil ref must be pruned from sha256 mirror: {set:?}"
);
assert!(
set.contains("refs/gitlawb/issues/i1"),
"exempt ref must survive: {set:?}"
);
});
}
/// Prune failures defer the queue row (stays `pending` for the next
/// tick); deterministic prune refusals and any other sync error fail it
/// terminally. Deleting the `downcast_ref::<PruneFailed>` arm restores
/// terminal failure and turns the first assertion red.
/// Prune failures defer the queue row (stays `pending` for the next
/// tick); deterministic prune refusals and any other sync error fail it
/// terminally. Deleting the `downcast_ref::<PruneFailed>` arm restores
/// terminal failure and turns the first assertion red.
///
/// The wrapper below is what the clone/fetch call sites run through:
/// a `PruneInvalid` from the helper must still downcast as
/// `PruneInvalid` afterwards (not stringified into `PruneFailed`),
/// and anything else must arrive as `PruneFailed`.
#[test]
fn wrap_prune_error_preserves_deterministic_type() {
let invalid = wrap_prune_error(anyhow::anyhow!(PruneInvalid("x".to_string())));
assert!(
invalid.downcast_ref::<PruneInvalid>().is_some(),
"deterministic refusal must survive the call-site wrap"
);
assert!(
invalid.downcast_ref::<PruneFailed>().is_none(),
"deterministic refusal must not also read as transient"
);
let transient = wrap_prune_error(anyhow::anyhow!("boom"));
assert!(
transient.downcast_ref::<PruneFailed>().is_some(),
"generic failure must arrive as transient"
);
assert!(
transient.downcast_ref::<PruneInvalid>().is_none(),
"generic failure must not read as deterministic"
);
}
#[sqlx::test]
async fn prune_failure_defers_row_other_failures_fail_it(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let item = |id: &str| crate::db::SyncQueueItem {
id: id.to_string(),
repo: "z6Mkfoo/hello".to_string(),
node_did: "did:key:z6MkOrigin".to_string(),
ref_name: "refs/heads/main".to_string(),
new_sha: "0".repeat(40),
cid: None,
status: "pending".to_string(),
enqueued_at: chrono::Utc::now().to_rfc3339(),
};
async fn status_of(pool: &sqlx::PgPool, id: &str) -> String {
let row: (String,) = sqlx::query_as("SELECT status FROM sync_queue WHERE id = $1")
.bind(id)
.fetch_one(pool)
.await
.unwrap();
row.0
}
// Seed three pending rows directly (enqueue_sync mints random ids).
for id in ["sync-prune-defer", "sync-prune-invalid", "sync-other-fail"] {
sqlx::query(
"INSERT INTO sync_queue (id, repo, node_did, ref_name, new_sha, cid, status, enqueued_at)
VALUES ($1, $2, $3, $4, $5, NULL, 'pending', $6)",
)
.bind(id)
.bind("z6Mkfoo/hello")
.bind("did:key:z6MkOrigin")
.bind("refs/heads/main")
.bind("0".repeat(40))
.bind(chrono::Utc::now().to_rfc3339())
.execute(state.db.pool())
.await
.unwrap();
}
handle_sync_item_error(
&state.db,
&item("sync-prune-defer"),
"http://origin.example",
anyhow::anyhow!(PruneFailed("boom".to_string())),
)
.await;
assert_eq!(
status_of(state.db.pool(), "sync-prune-defer").await,
"pending",
"prune failure must leave the row pending for retry"
);
handle_sync_item_error(
&state.db,
&item("sync-other-fail"),
"http://origin.example",
anyhow::anyhow!("boom"),
)
.await;
assert_eq!(
status_of(state.db.pool(), "sync-other-fail").await,
"failed",
"non-prune failure must fail the row"
);
// Deterministic refusal: no retry can clear it, so it fails
// terminally instead of spinning every tick forever.
handle_sync_item_error(
&state.db,
&item("sync-prune-invalid"),
"http://origin.example",
anyhow::anyhow!(PruneInvalid("not itself a repo".to_string())),
)
.await;
assert_eq!(
status_of(state.db.pool(), "sync-prune-invalid").await,
"failed",
"deterministic prune refusal must fail the row, not defer forever"
);
}
}
-75
View File
@@ -1,75 +0,0 @@
//! Native Git fixtures shared by timing tests on Unix and Windows.
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// The fixture delays or hangs in its own process, so the production watchdog
/// can terminate it without depending on a shell or descendant process support.
pub(super) enum Behavior<'a> {
Delay(u64),
Hang,
HangOid(&'a str),
HangRepo(&'a str),
LogTypes(&'a Path),
}
pub(super) struct GitShim {
_directory: tempfile::TempDir,
executable: PathBuf,
}
impl Deref for GitShim {
type Target = Path;
fn deref(&self) -> &Path {
&self.executable
}
}
pub(super) fn create(name: &str, behavior: Behavior<'_>) -> GitShim {
static COMPILED: OnceLock<tempfile::TempDir> = OnceLock::new();
let compiled = COMPILED.get_or_init(|| {
let directory = tempfile::tempdir().expect("native Git fixture directory");
let source = directory.path().join("git_shim.rs");
std::fs::write(&source, include_str!("../tests/fixtures/git_shim.rs"))
.expect("write native Git fixture source");
let executable = directory
.path()
.join(format!("git-shim{}", std::env::consts::EXE_SUFFIX));
let output =
std::process::Command::new(std::env::var_os("RUSTC").unwrap_or_else(|| "rustc".into()))
.arg("--edition=2021")
.arg("-Dwarnings")
.arg(&source)
.arg("-o")
.arg(executable)
.output()
.expect("compile native Git fixture with the installed Rust toolchain");
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
directory
});
let directory = tempfile::Builder::new()
.prefix(name)
.tempdir()
.expect("Git fixture instance");
let file_name = format!("git-shim{}", std::env::consts::EXE_SUFFIX);
let executable = directory.path().join(&file_name);
std::fs::copy(compiled.path().join(file_name), &executable).expect("copy native Git fixture");
let config = match behavior {
Behavior::Delay(milliseconds) => format!("delay\n{milliseconds}"),
Behavior::Hang => "hang\n".to_owned(),
Behavior::HangOid(oid) => format!("hang-oid\n{oid}"),
Behavior::HangRepo(repo) => format!("hang-repo\n{repo}"),
Behavior::LogTypes(path) => format!("log-types\n{}", path.display()),
};
std::fs::write(directory.path().join("config"), config).expect("configure native Git fixture");
GitShim {
_directory: directory,
executable,
}
}
File diff suppressed because it is too large. Load diff
-968
View File
@@ -1,968 +0,0 @@
//! Scheduled external-forge fetches for canonical INBOUND mirrors.
//!
//! This worker is deliberately separate from `sync`, which mirrors repositories
//! between Gitlawb peers. External upstream URLs are owner-controlled data that
//! eventually reach `git fetch`, so each cycle resolves DNS itself, validates
//! every address, and pins the approved answers into Git/libcurl with
//! `http.curloptResolve`. Redirects, proxies, credential helpers, submodule
//! recursion, and non-HTTPS protocols are disabled at the command boundary.
use anyhow::{Context, Result};
use std::collections::BTreeSet;
use std::net::IpAddr;
use std::path::Path;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tracing::{info, warn};
use crate::config::Config;
use crate::db::{Db, InboundMirrorTarget, MirrorStatus, MirrorTransitionPhase};
use crate::git::repo_store::RepoStore;
use crate::state::{repo_identity_key, RepoWriteLeases};
const DNS_TIMEOUT: Duration = Duration::from_secs(10);
const MAX_PINNED_ADDRESSES: usize = 32;
const MIN_CURL_OPT_RESOLVE_GIT: (u64, u64) = (2, 37);
#[derive(Debug, Clone, Copy)]
struct EgressPolicy;
#[derive(Debug, Clone, PartialEq, Eq)]
struct PinnedUpstream {
url: String,
curlopt_resolve: Option<String>,
}
impl EgressPolicy {
async fn resolve(&self, url: reqwest::Url) -> Result<PinnedUpstream> {
let raw_host = url.host_str().context("mirror upstream URL has no host")?;
let connect_host = raw_host
.trim_start_matches('[')
.trim_end_matches(']')
.to_ascii_lowercase();
let policy_host = normalize_policy_host(&connect_host);
reject_always_local_hostname(&policy_host)?;
let port = url
.port_or_known_default()
.context("mirror upstream URL has no known port")?;
if let Ok(ip) = connect_host.parse::<IpAddr>() {
self.validate_addresses(&policy_host, &[ip])?;
return Ok(PinnedUpstream {
url: url.to_string(),
// An IP literal performs no DNS lookup, so there is no rebinding
// window to pin. TLS still verifies the literal from the URL.
curlopt_resolve: None,
});
}
let resolved = tokio::time::timeout(
DNS_TIMEOUT,
tokio::net::lookup_host((connect_host.as_str(), port)),
)
.await
.context("mirror upstream DNS lookup timed out")?
.context("resolving mirror upstream host")?;
let addresses: BTreeSet<IpAddr> = resolved.map(|addr| addr.ip()).collect();
if addresses.is_empty() {
anyhow::bail!("mirror upstream host resolved to no addresses");
}
let addresses: Vec<IpAddr> = addresses.into_iter().collect();
self.validate_addresses(&policy_host, &addresses)?;
let pinned = addresses
.iter()
.map(|ip| match ip {
IpAddr::V4(v4) => v4.to_string(),
IpAddr::V6(v6) => format!("[{v6}]"),
})
.collect::<Vec<_>>()
.join(",");
Ok(PinnedUpstream {
url: url.to_string(),
curlopt_resolve: Some(format!("+{connect_host}:{port}:{pinned}")),
})
}
fn validate_addresses(&self, host: &str, addresses: &[IpAddr]) -> Result<()> {
if addresses.is_empty() || addresses.len() > MAX_PINNED_ADDRESSES {
anyhow::bail!(
"mirror upstream host must resolve to 1..={MAX_PINNED_ADDRESSES} addresses"
);
}
for &ip in addresses {
if !address_is_permitted(ip) {
anyhow::bail!("mirror upstream host {host:?} resolved to disallowed address {ip}");
}
}
Ok(())
}
}
fn normalize_policy_host(host: &str) -> String {
host.trim_end_matches('.').to_ascii_lowercase()
}
fn reject_always_local_hostname(host: &str) -> Result<()> {
if host.is_empty()
|| host == "localhost"
|| host.ends_with(".localhost")
|| host.ends_with(".local")
|| host.ends_with(".internal")
{
anyhow::bail!("mirror upstream host is local-only and cannot be fetched");
}
Ok(())
}
fn address_is_permitted(ip: IpAddr) -> bool {
crate::api::peers::mirror_upstream_ip_permitted(ip)
}
fn parse_git_version(output: &str) -> Option<(u64, u64)> {
let version = output.split_whitespace().find(|part| {
part.as_bytes()
.first()
.is_some_and(|byte| byte.is_ascii_digit())
})?;
let mut parts = version.split('.');
Some((parts.next()?.parse().ok()?, parts.next()?.parse().ok()?))
}
fn ensure_safe_git_runtime(git_bin: &str) -> Result<()> {
// Bounded runner like every other git spawn on this path: --version does
// no I/O of its own, but the child still takes a permit and the deadline.
let (status, stdout, stderr) = crate::git::visibility_pack::run_bounded_git_raw_isolated_https(
git_bin,
&["--version"],
&std::env::temp_dir(),
b"",
Instant::now() + Duration::from_secs(10),
)?;
if !status.success() {
anyhow::bail!(
"{git_bin} --version failed: {}",
String::from_utf8_lossy(&stderr)
.chars()
.take(256)
.collect::<String>()
);
}
let stdout = String::from_utf8_lossy(&stdout);
let version = parse_git_version(&stdout).context("parsing git version")?;
if version < MIN_CURL_OPT_RESOLVE_GIT {
anyhow::bail!(
"upstream mirroring requires Git 2.37+ for DNS pinning; found {}.{}",
version.0,
version.1
);
}
Ok(())
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct FetchCommand {
config: Vec<(String, String)>,
args: Vec<String>,
}
fn fetch_command(upstream: &PinnedUpstream) -> FetchCommand {
let http_scope = |name: &str| format!("http.{}.{name}", upstream.url);
let credential_scope = |name: &str| format!("credential.{}.{name}", upstream.url);
let mut config = vec![
("http.proxy".to_string(), "".to_string()),
("http.followRedirects".to_string(), "false".to_string()),
// Reset any inherited multi-valued resolver entries before adding the
// address set validated for this exact fetch.
("http.curloptResolve".to_string(), "".to_string()),
// Never forward operator or repository-scoped HTTP credentials to an
// owner-selected forge. Empty values reset Git's multi-valued headers
// and disable cookie state for this invocation.
("http.extraHeader".to_string(), "".to_string()),
("http.cookieFile".to_string(), "".to_string()),
("http.saveCookies".to_string(), "false".to_string()),
("http.sslVerify".to_string(), "true".to_string()),
// URL-specific configuration outranks generic `http.*` values even
// when the generic value is command-scoped. Repeat every security
// control at the exact validated URL so a repository-local scoped
// setting cannot re-enable redirects/proxies/headers/cookies or disable
// TLS. Key/value environment entries avoid `git -c` splitting a valid
// URL path that contains `=`.
(http_scope("proxy"), "".to_string()),
(http_scope("followRedirects"), "false".to_string()),
(http_scope("curloptResolve"), "".to_string()),
(http_scope("extraHeader"), "".to_string()),
(http_scope("cookieFile"), "".to_string()),
(http_scope("saveCookies"), "false".to_string()),
(http_scope("sslVerify"), "true".to_string()),
];
if let Some(resolve) = &upstream.curlopt_resolve {
config.extend([
("http.curloptResolve".to_string(), resolve.clone()),
(http_scope("curloptResolve"), resolve.clone()),
]);
}
config.extend([
("credential.helper".to_string(), "".to_string()),
(credential_scope("helper"), "".to_string()),
("credential.interactive".to_string(), "false".to_string()),
("core.askPass".to_string(), "false".to_string()),
("fetch.recurseSubmodules".to_string(), "false".to_string()),
// Do not let a smart server or repository-local config introduce a
// second, unvalidated pack/bundle URL outside the pinned upstream.
("fetch.uriprotocols".to_string(), "".to_string()),
("fetch.bundleURI".to_string(), "".to_string()),
("protocol.allow".to_string(), "never".to_string()),
("protocol.https.allow".to_string(), "always".to_string()),
]);
let args = vec![
"fetch".to_string(),
"--atomic".to_string(),
"--force".to_string(),
"--prune".to_string(),
"--prune-tags".to_string(),
"--no-auto-maintenance".to_string(),
"--no-recurse-submodules".to_string(),
"--no-write-fetch-head".to_string(),
upstream.url.clone(),
"+refs/heads/*:refs/heads/*".to_string(),
"+refs/tags/*:refs/tags/*".to_string(),
];
FetchCommand { config, args }
}
fn reject_local_url_rewrite(
git_bin: &str,
repo_path: &Path,
upstream_url: &str,
deadline: Instant,
) -> Result<()> {
let args = ["ls-remote", "--get-url", upstream_url];
let (status, stdout, stderr) = crate::git::visibility_pack::run_bounded_git_raw_isolated_https(
git_bin, &args, repo_path, b"", deadline,
)?;
if !status.success() {
let stderr = String::from_utf8_lossy(&stderr);
anyhow::bail!(
"checking mirror upstream URL rewrite failed: {}",
stderr.chars().take(4_096).collect::<String>()
);
}
let expanded = std::str::from_utf8(&stdout)
.context("Git returned a non-UTF-8 expanded mirror upstream URL")?
.trim_end_matches(['\r', '\n']);
if expanded != upstream_url {
anyhow::bail!("repository-local Git config attempted to rewrite the mirror upstream URL");
}
Ok(())
}
fn reject_sensitive_local_config(git_bin: &str, repo_path: &Path, deadline: Instant) -> Result<()> {
let args = [
"config",
"--local",
"--includes",
"--get-regexp",
r"^(http|credential|url)\.",
];
let (status, stdout, stderr) = crate::git::visibility_pack::run_bounded_git_raw_isolated_https(
git_bin, &args, repo_path, b"", deadline,
)?;
match status.code() {
Some(1) if stdout.is_empty() => Ok(()),
Some(0) => anyhow::bail!(
"repository-local network or credential Git config is not allowed for upstream mirrors"
),
_ => {
let stderr = String::from_utf8_lossy(&stderr);
anyhow::bail!(
"checking repository-local Git config failed: {}",
stderr.chars().take(4_096).collect::<String>()
)
}
}
}
/// Budget for each preflight probe inside [`run_fetch`] (the URL-rewrite
/// check and the sensitive-local-config scan). Both run local git children
/// against the on-disk repo, so a short bound is generous; keeping them on
/// their own deadlines means `upstream_mirror_fetch_timeout_secs` covers
/// only the external pack transfer it documents.
const PREFLIGHT_PROBE_TIMEOUT: Duration = Duration::from_secs(30);
fn run_fetch(
git_bin: &str,
repo_path: &Path,
upstream: &PinnedUpstream,
timeout: Duration,
) -> Result<()> {
let command = fetch_command(upstream);
let args: Vec<&str> = command.args.iter().map(String::as_str).collect();
let config: Vec<(&str, &str)> = command
.config
.iter()
.map(|(key, value)| (key.as_str(), value.as_str()))
.collect();
let rewrite_deadline = Instant::now()
.checked_add(PREFLIGHT_PROBE_TIMEOUT)
.context("mirror URL-rewrite probe timeout is not representable")?;
reject_local_url_rewrite(git_bin, repo_path, &upstream.url, rewrite_deadline)?;
let config_deadline = Instant::now()
.checked_add(PREFLIGHT_PROBE_TIMEOUT)
.context("mirror local-config probe timeout is not representable")?;
reject_sensitive_local_config(git_bin, repo_path, config_deadline)?;
let deadline = Instant::now()
.checked_add(timeout)
.context("upstream mirror fetch timeout is not representable")?;
let (status, _stdout, stderr) =
crate::git::visibility_pack::run_bounded_git_raw_isolated_https_with_config(
git_bin, &args, repo_path, b"", deadline, &config,
)?;
if !status.success() {
let stderr = String::from_utf8_lossy(&stderr);
let stderr = stderr.chars().take(4_096).collect::<String>();
anyhow::bail!("git upstream fetch failed: {stderr}");
}
Ok(())
}
#[derive(Clone)]
struct Worker {
db: Arc<Db>,
config: Arc<Config>,
repo_store: RepoStore,
repo_write_leases: RepoWriteLeases,
git_write_semaphore: Arc<tokio::sync::Semaphore>,
policy: EgressPolicy,
git_bin: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum FetchOutcome {
Updated,
SkippedStale,
}
impl Worker {
async fn fetch_target(&self, target: InboundMirrorTarget) -> Result<FetchOutcome> {
let url = crate::db::validate_mirror_upstream_url(&target.upstream_url)?;
let upstream = self.policy.resolve(url).await?;
let repo_identity = repo_identity_key(&target.owner_did, &target.name);
let steal_after = Duration::from_secs(
self.config
.upstream_mirror_fetch_timeout_secs
.saturating_mul(2)
.saturating_add(60),
);
let _lease = self
.repo_write_leases
.acquire(&repo_identity, steal_after)
.await
.context("upstream mirror write-lease waiter cap reached")?;
let _write_permit = Arc::clone(&self.git_write_semaphore)
.acquire_owned()
.await
.context("upstream mirror write semaphore closed")?;
let acquire_timeout = Duration::from_secs(self.config.git_acquire_timeout_secs);
let guard = tokio::time::timeout(
acquire_timeout,
self.repo_store
.acquire_write(&target.owner_did, &target.name),
)
.await
.context("upstream mirror write-lock acquisition timed out")??;
let repo_path = guard.path().to_path_buf();
// A row can leave INBOUND state after selection. Recheck only after
// both the process-local lease and the cluster-wide advisory write lock
// are held. Every future transition path must take these same locks
// before changing authority, so no other node can commit a transition
// between this check and the fetch.
let current = match self.db.get_repo_mirror_state(&target.repo_id).await {
Ok(Some(current)) => current,
Ok(None) => {
let _ = guard.release(false).await;
anyhow::bail!("inbound mirror state disappeared before fetch");
}
Err(error) => {
let _ = guard.release(false).await;
return Err(error);
}
};
// Both stable shapes the CHECK admits: never transitioned, or a
// completed transition whose metadata is retained as audit. Only an
// in-flight transition makes the target stale.
let stable = (current.transition_id.is_none() && current.transition_phase.is_none())
|| (current.transition_id.is_some()
&& current.transition_phase == Some(MirrorTransitionPhase::Completed));
if current.status != MirrorStatus::Inbound
|| !stable
|| current.upstream_url != target.upstream_url
{
let _ = guard.release(false).await;
info!(repo_id = %target.repo_id, "upstream mirror state changed before fetch; skipping stale target");
return Ok(FetchOutcome::SkippedStale);
}
let git_bin = self.git_bin.clone();
let fetch_timeout = Duration::from_secs(self.config.upstream_mirror_fetch_timeout_secs);
let fetch = tokio::task::spawn_blocking(move || {
run_fetch(&git_bin, &repo_path, &upstream, fetch_timeout)
})
.await
.context("upstream mirror fetch task panicked")?;
let success = fetch.is_ok();
let outcome = guard.release(success).await;
// A refused publish (fenced or unknowable upload) must not report
// Updated; surface it as the failure it is before mapping the fetch.
outcome.into_result()?;
fetch.map(|()| FetchOutcome::Updated)
}
async fn scan_once(&self) {
let mut cursor: Option<String> = None;
loop {
let targets = match self
.db
.list_inbound_mirror_targets(
cursor.as_deref(),
self.config.upstream_mirror_page_size as i64,
)
.await
{
Ok(targets) => targets,
Err(error) => {
warn!(err = %error, "failed to list inbound mirror targets");
return;
}
};
if targets.is_empty() {
return;
}
let page_len = targets.len();
cursor = targets.last().map(|target| target.repo_id.clone());
for target in targets {
let repo_id = target.repo_id.clone();
let upstream_url = target.upstream_url.clone();
match self.fetch_target(target).await {
Ok(FetchOutcome::Updated) => {
info!(repo_id, upstream = %upstream_url, "upstream mirror fetch completed")
}
Ok(FetchOutcome::SkippedStale) => {}
Err(error) => {
warn!(repo_id, upstream = %upstream_url, err = %error, "upstream mirror fetch failed")
}
}
}
if page_len < self.config.upstream_mirror_page_size {
return;
}
}
}
}
/// Validate the opt-in runtime and spawn the scheduled worker. The worker runs
/// one scan immediately, sleeps only after a complete scan, and exits between
/// scans when graceful shutdown is signalled. An in-flight Git child is bounded
/// and reaped before the worker observes shutdown.
pub fn start(
db: Arc<Db>,
config: Arc<Config>,
repo_store: RepoStore,
repo_write_leases: RepoWriteLeases,
git_write_semaphore: Arc<tokio::sync::Semaphore>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) -> Result<()> {
ensure_safe_git_runtime("git")?;
let policy = EgressPolicy;
let worker = Worker {
db,
config: Arc::clone(&config),
repo_store,
repo_write_leases,
git_write_semaphore,
policy,
git_bin: "git".to_string(),
};
tokio::spawn(async move {
info!(
interval_secs = config.upstream_mirror_interval_secs,
"upstream mirror worker started"
);
loop {
worker.scan_once().await;
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(config.upstream_mirror_interval_secs)) => {}
changed = shutdown_rx.changed() => {
if changed.is_err() || *shutdown_rx.borrow() {
info!("upstream mirror worker stopped");
return;
}
}
}
}
});
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use clap::Parser;
use std::path::PathBuf;
async fn test_worker(
pool: sqlx::PgPool,
git_bin: PathBuf,
page_size: usize,
) -> (Worker, Arc<Db>, tempfile::TempDir) {
let db = Arc::new(Db::for_testing(pool.clone()));
db.run_migrations().await.unwrap();
let repos = tempfile::tempdir().unwrap();
let mut config = Config::parse_from([
"gitlawb-node".to_string(),
"--upstream-mirror-enabled".to_string(),
"--enforce-owner-push".to_string(),
"--upstream-mirror-fetch-timeout-secs".to_string(),
"5".to_string(),
"--upstream-mirror-page-size".to_string(),
page_size.to_string(),
]);
config.validate().unwrap();
let config = Arc::new(config);
let worker = Worker {
db: Arc::clone(&db),
config,
repo_store: RepoStore::for_testing(repos.path().to_path_buf(), pool),
repo_write_leases: RepoWriteLeases::new(8),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(4)),
policy: EgressPolicy,
git_bin: git_bin.to_string_lossy().into_owned(),
};
(worker, db, repos)
}
async fn add_inbound_mirror(
worker: &Worker,
db: &Db,
id: &str,
name: &str,
upstream_url: &str,
) -> InboundMirrorTarget {
let now = chrono::Utc::now();
let record = crate::db::RepoRecord {
id: id.to_string(),
name: name.to_string(),
owner_did: "did:key:z6MkMirrorWorker".to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/test/{id}.git"),
forked_from: None,
machine_id: None,
};
db.create_repo(&record).await.unwrap();
worker
.repo_store
.init(&record.owner_did, &record.name)
.await
.unwrap();
let state = db
.configure_inbound_mirror(&record.id, upstream_url)
.await
.unwrap();
InboundMirrorTarget {
repo_id: record.id,
owner_did: record.owner_did,
name: record.name,
upstream_url: state.upstream_url,
}
}
#[cfg(unix)]
fn fake_git(repos: &tempfile::TempDir) -> (PathBuf, PathBuf) {
use std::os::unix::fs::PermissionsExt;
let git = repos.path().join("fake-git");
let calls = repos.path().join("fetch-calls");
let script = format!(
r#"#!/bin/sh
if [ "$1" = "ls-remote" ]; then
printf '%s\n' "$3"
exit 0
fi
if [ "$1" = "config" ]; then
# Match real Git when no sensitive repository-local keys are configured.
exit 1
fi
case "$*" in
*fail.git*) printf '%s\n' fail >> '{}'; exit 7 ;;
*) printf '%s\n' ok >> '{}'; exit 0 ;;
esac
"#,
calls.display(),
calls.display()
);
std::fs::write(&git, script).unwrap();
let mut permissions = std::fs::metadata(&git).unwrap().permissions();
permissions.set_mode(0o700);
std::fs::set_permissions(&git, permissions).unwrap();
(git, calls)
}
#[test]
fn git_version_gate_requires_curlopt_resolve_support() {
assert_eq!(parse_git_version("git version 2.37.0"), Some((2, 37)));
assert_eq!(
parse_git_version("git version 2.39.5 (Apple Git-154)"),
Some((2, 39))
);
assert_eq!(parse_git_version("not git"), None);
assert!((2, 36) < MIN_CURL_OPT_RESOLVE_GIT);
}
#[test]
fn every_resolved_address_must_be_public() {
let policy = EgressPolicy;
let public_v4: IpAddr = "8.8.8.8".parse().unwrap();
let private_v4: IpAddr = "10.2.3.4".parse().unwrap();
let private_v6: IpAddr = "fd00::20".parse().unwrap();
assert!(policy
.validate_addresses("public.example", &[public_v4])
.is_ok());
assert!(policy
.validate_addresses("private.example", &[private_v4, private_v6])
.is_err());
assert!(policy
.validate_addresses("mixed.example", &[public_v4, private_v4])
.is_err());
let too_many = (1..=MAX_PINNED_ADDRESSES + 1)
.map(|last| IpAddr::V4(std::net::Ipv4Addr::new(8, 8, 0, last as u8)))
.collect::<Vec<_>>();
assert!(policy
.validate_addresses("oversized.example", &too_many)
.is_err());
for never in [
"127.0.0.1",
"169.254.169.254",
"192.0.2.1",
"198.18.0.1",
"198.51.100.1",
"203.0.113.10",
"::1",
"::ffff:198.18.0.1",
"2002:c612:1::",
"64:ff9b::c612:1",
"100::1",
"100:0:0:1::1",
"2001::1",
"2001:2::1",
"2001:db8::1",
"fec0::1",
"fe80::1",
"3fff::1",
"5f00::1",
] {
let ip = never.parse().unwrap();
assert!(!address_is_permitted(ip), "{never} must stay denied");
}
}
#[test]
fn local_only_hostnames_are_always_rejected() {
for invalid in [
"",
"localhost",
"forge.localhost",
"forge.local",
"ghe.internal",
] {
assert!(
reject_always_local_hostname(invalid).is_err(),
"{invalid:?}"
);
}
assert!(reject_always_local_hostname("github.example").is_ok());
}
#[test]
fn fetch_command_pins_dns_and_only_updates_branches_and_tags() {
let upstream = PinnedUpstream {
url: "https://github.example/org/repo.git".to_string(),
curlopt_resolve: Some("+github.example:443:8.8.8.8,[2606:4700:4700::1111]".to_string()),
};
let command = fetch_command(&upstream);
let has_config = |key: &str, value: &str| {
command
.config
.contains(&(key.to_string(), value.to_string()))
};
assert!(has_config("http.followRedirects", "false"));
assert!(has_config("http.proxy", ""));
assert!(has_config("http.extraHeader", ""));
assert!(has_config("http.cookieFile", ""));
assert!(has_config("http.saveCookies", "false"));
assert!(has_config("http.sslVerify", "true"));
assert!(has_config(
"http.curloptResolve",
"+github.example:443:8.8.8.8,[2606:4700:4700::1111]"
));
assert!(has_config(
"http.https://github.example/org/repo.git.followRedirects",
"false"
));
assert!(has_config(
"http.https://github.example/org/repo.git.proxy",
""
));
assert!(has_config(
"http.https://github.example/org/repo.git.extraHeader",
""
));
assert!(has_config(
"http.https://github.example/org/repo.git.curloptResolve",
"+github.example:443:8.8.8.8,[2606:4700:4700::1111]"
));
assert!(has_config(
"credential.https://github.example/org/repo.git.helper",
""
));
assert!(has_config("protocol.allow", "never"));
assert!(has_config("protocol.https.allow", "always"));
assert!(has_config("fetch.uriprotocols", ""));
assert!(has_config("fetch.bundleURI", ""));
assert!(!command
.config
.iter()
.any(|(key, _)| key.ends_with("sslCert") || key.ends_with("sslKey")));
assert!(command.args.contains(&"--atomic".to_string()));
assert!(command
.args
.contains(&"+refs/heads/*:refs/heads/*".to_string()));
assert!(command
.args
.contains(&"+refs/tags/*:refs/tags/*".to_string()));
assert!(!command.args.iter().any(|arg| arg == "+refs/*:refs/*"));
}
/// Spawn `git` inside a fixture repo with inherited repository-location
/// and config environment scrubbed, so a test run inside an outer Git
/// checkout or custom config can never leak into the fixture.
fn fixture_git(dir: &std::path::Path) -> std::process::Command {
let mut cmd = std::process::Command::new("git");
cmd.current_dir(dir);
for var in [
"GIT_DIR",
"GIT_WORK_TREE",
"GIT_CONFIG",
"GIT_CONFIG_GLOBAL",
"GIT_CONFIG_SYSTEM",
"GIT_CONFIG_NOSYSTEM",
"GIT_CONFIG_COUNT",
"GIT_EXEC_PATH",
"GIT_SSH",
"GIT_SSH_COMMAND",
"GIT_ASKPASS",
"SSH_ASKPASS",
] {
cmd.env_remove(var);
}
cmd
}
#[test]
fn repository_local_instead_of_rewrite_is_rejected() {
let repo = tempfile::tempdir().unwrap();
let init = fixture_git(repo.path())
.args(["init", "--bare"])
.output()
.unwrap();
assert!(init.status.success());
let config = fixture_git(repo.path())
.args([
"config",
"url.https://evil.example/.insteadOf",
"https://github.example/",
])
.output()
.unwrap();
assert!(config.status.success());
let error = reject_local_url_rewrite(
"git",
repo.path(),
"https://github.example/org/repo.git",
Instant::now() + Duration::from_secs(5),
)
.unwrap_err();
assert!(error.to_string().contains("attempted to rewrite"));
}
#[test]
fn repository_local_network_and_credential_config_is_rejected() {
let repo = tempfile::tempdir().unwrap();
let init = fixture_git(repo.path())
.args(["init", "--bare"])
.output()
.unwrap();
assert!(init.status.success());
reject_sensitive_local_config("git", repo.path(), Instant::now() + Duration::from_secs(5))
.unwrap();
let config = fixture_git(repo.path())
.args(["config", "http.sslCert", "/tmp/untrusted-client.pem"])
.output()
.unwrap();
assert!(config.status.success());
let error = reject_sensitive_local_config(
"git",
repo.path(),
Instant::now() + Duration::from_secs(5),
)
.unwrap_err();
assert!(error.to_string().contains("network or credential"));
}
#[cfg(unix)]
#[test]
fn hardened_real_git_reaches_https_transport_for_equals_path() {
let repo = tempfile::tempdir().unwrap();
let init = fixture_git(repo.path())
.args(["init", "--bare"])
.output()
.unwrap();
assert!(init.status.success());
// This is intentionally a loopback test seam around `run_fetch`, below
// the egress-policy layer. Reaching the listener proves the generated
// Git config neither dies on an empty client certificate nor splits the
// valid `=` path at the command-config boundary. The listener closes
// immediately, so the expected final result is a TLS transport error.
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
listener.set_nonblocking(true).unwrap();
let accepted = std::thread::spawn(move || {
let deadline = Instant::now() + Duration::from_secs(3);
while Instant::now() < deadline {
match listener.accept() {
Ok((_stream, _address)) => return true,
Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(Duration::from_millis(10));
}
Err(error) => panic!("accepting real-Git transport probe: {error}"),
}
}
false
});
let upstream = PinnedUpstream {
url: format!("https://127.0.0.1:{port}/org/repo=mirror.git"),
curlopt_resolve: None,
};
let error = run_fetch("git", repo.path(), &upstream, Duration::from_secs(5)).unwrap_err();
assert!(
accepted.join().unwrap(),
"real Git must reach the HTTPS transport; got {error:#}"
);
let error = error.to_string();
assert!(!error.contains("credential missing host field"));
assert!(!error.contains("invalid key"));
}
#[tokio::test]
async fn literal_addresses_are_checked_without_dns() {
let policy = EgressPolicy;
let allowed = policy
.resolve(reqwest::Url::parse("https://8.8.8.8/org/repo.git").unwrap())
.await
.unwrap();
assert_eq!(allowed.curlopt_resolve, None);
for denied in [
"127.0.0.1",
"10.2.3.4",
"169.254.169.254",
"192.0.2.1",
"198.18.0.1",
"203.0.113.10",
"[::1]",
"[100::1]",
"[100:0:0:1::1]",
"[2001::1]",
"[2001:db8::1]",
"[fec0::1]",
"[fd00::20]",
"[3fff::1]",
"[5f00::1]",
] {
let url = reqwest::Url::parse(&format!("https://{denied}/org/repo.git")).unwrap();
assert!(
policy.resolve(url).await.is_err(),
"{denied} must stay denied"
);
}
}
#[sqlx::test]
async fn stale_target_is_rechecked_under_the_write_locks(pool: sqlx::PgPool) {
let missing_git = PathBuf::from("/definitely/missing/git");
let (worker, db, _repos) = test_worker(pool, missing_git, 1).await;
let target = add_inbound_mirror(
&worker,
&db,
"00000000-0000-0000-0000-000000000001",
"stale",
"https://8.8.8.8/org/stale.git",
)
.await;
sqlx::query(
"UPDATE repos
SET mirror_status = 'outbound'
WHERE id = $1",
)
.bind(&target.repo_id)
.execute(db.pool())
.await
.unwrap();
let outcome = worker.fetch_target(target).await.unwrap();
assert_eq!(outcome, FetchOutcome::SkippedStale);
assert!(worker.repo_write_leases.is_empty());
}
#[cfg(unix)]
#[sqlx::test]
async fn scan_pages_past_one_failure_and_fetches_the_next_repo(pool: sqlx::PgPool) {
let bootstrap = tempfile::tempdir().unwrap();
let (git, calls) = fake_git(&bootstrap);
let (worker, db, _repos) = test_worker(pool, git, 1).await;
add_inbound_mirror(
&worker,
&db,
"00000000-0000-0000-0000-000000000001",
"fails",
"https://8.8.8.8/org/fail.git",
)
.await;
add_inbound_mirror(
&worker,
&db,
"00000000-0000-0000-0000-000000000002",
"succeeds",
"https://8.8.8.8/org/ok.git",
)
.await;
worker.scan_once().await;
let calls = std::fs::read_to_string(calls).unwrap();
assert_eq!(calls.lines().collect::<Vec<_>>(), ["fail", "ok"]);
assert!(worker.repo_write_leases.is_empty());
}
}
-221
View File
@@ -1,221 +0,0 @@
//! At-rest protection for webhook HMAC secrets (#506).
//!
//! Webhook secrets are signing keys: whoever holds one can forge delivery
//! payloads to the receiver the hook points at. They are persisted in the
//! `webhooks.secret` column and used to compute the `X-Gitlawb-Signature-256`
//! header on each delivery. That secret needs two properties:
//!
//! * It must never reach a log line or an API response. [`WebhookSecret`]
//! redacts both `Debug` and `Serialize`, so no derived `{:?}` and no JSON
//! body ever renders the value, whatever it contains.
//! * It must not rest as plaintext where a database dump or backup can read
//! it. The node seals the secret with a key derived from its persistent
//! identity (see `AppState::derive_webhook_secret_key`), so a dump alone
//! reveals nothing.
//!
//! Storage format: `enc:v1:<base64url(nonce || ciphertext)>` — a 24-byte
//! XChaCha20-Poly1305 nonce followed by the ciphertext (which carries the tag).
//! Anything without the `enc:v1:` prefix is a legacy plaintext value, written
//! before this protection shipped; the delivery path signs with those unchanged
//! and they are re-encrypted once at startup by
//! `Db::reencrypt_legacy_webhook_secrets`.
use std::fmt;
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64URL, Engine};
use chacha20poly1305::{
aead::{Aead, AeadCore, KeyInit, OsRng},
XChaCha20Poly1305, XNonce,
};
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// Prefix marking a sealed-at-rest secret. Legacy plaintext values lack it.
pub const SEAL_PREFIX: &str = "enc:v1:";
const NONCE_LEN: usize = 24;
/// An HMAC signing secret that never renders its contents in debug or JSON
/// output. Used as `Webhook.secret`, so that field is safe to `Debug`-print and
/// `Serialize` by construction rather than by discipline at each call site.
#[derive(Clone, PartialEq, Eq)]
pub struct WebhookSecret(String);
impl WebhookSecret {
pub fn new(inner: String) -> Self {
Self(inner)
}
pub fn as_str(&self) -> &str {
&self.0
}
/// Whether the value is a sealed ciphertext (as opposed to legacy plaintext).
pub fn is_sealed(&self) -> bool {
self.0.starts_with(SEAL_PREFIX)
}
}
impl fmt::Debug for WebhookSecret {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("\"***\"")
}
}
impl Serialize for WebhookSecret {
/// Emit a fixed marker, never the contents. On the wire a set secret is
/// `"secret": "***"` and an unset one is `"secret": null` (`Option`), which
/// the CLI already reads as "is a secret set".
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str("***")
}
}
impl<'de> Deserialize<'de> for WebhookSecret {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
Ok(WebhookSecret(String::deserialize(deserializer)?))
}
}
/// Seal a plaintext secret under a node-held key. A fresh `OsRng` nonce on every
/// seal guarantees repeated seals of the same secret never collide.
pub fn seal(key: &[u8; 32], plaintext: &str) -> String {
let cipher = XChaCha20Poly1305::new_from_slice(key).expect("webhook secret key is 32 bytes");
let nonce = XChaCha20Poly1305::generate_nonce(&mut OsRng);
let ciphertext = cipher
.encrypt(&nonce, plaintext.as_bytes())
.expect("XChaCha20-Poly1305 encryption is infallible for in-memory inputs");
let mut out = Vec::with_capacity(NONCE_LEN + ciphertext.len());
out.extend_from_slice(&nonce);
out.extend_from_slice(&ciphertext);
format!("{SEAL_PREFIX}{}", B64URL.encode(out))
}
/// Open a sealed secret. Returns `None` for anything that is not a well-formed
/// sealed value — legacy plaintext, a foreign key, tampered or truncated bytes —
/// so a caller that needs the legacy path checks [`WebhookSecret::is_sealed`]
/// first (see [`reveal`]).
pub fn open(key: &[u8; 32], stored: &str) -> Option<String> {
let raw = B64URL.decode(stored.strip_prefix(SEAL_PREFIX)?).ok()?;
if raw.len() <= NONCE_LEN {
return None;
}
let (nonce, ciphertext) = raw.split_at(NONCE_LEN);
let cipher = XChaCha20Poly1305::new_from_slice(key).ok()?;
let plaintext = cipher.decrypt(XNonce::from_slice(nonce), ciphertext).ok()?;
String::from_utf8(plaintext).ok()
}
/// Whether a stored value has the structural shape of a sealed payload:
/// [`SEAL_PREFIX`] plus a base64url blob long enough to hold a nonce and
/// ciphertext. This is a shape check, not an authentication check: a
/// well-formed blob sealed under a different key satisfies it while [`open`]
/// still fails. The re-encryption pass uses the distinction to leave
/// well-formed-but-foreign ciphertext alone instead of re-wrapping it as
/// plaintext, which would let `reveal` return the outer blob as a signing key.
pub fn has_seal_shape(stored: &str) -> bool {
stored
.strip_prefix(SEAL_PREFIX)
.and_then(|rest| B64URL.decode(rest).ok())
.is_some_and(|raw| raw.len() > NONCE_LEN)
}
/// Resolve a stored secret to its plaintext for signing. Sealed values are opened
/// with the node key; legacy plaintext is returned unchanged so a hook created
/// before this fix keeps signing until the startup re-encryption pass runs.
pub fn reveal(key: &[u8; 32], stored: &WebhookSecret) -> Option<String> {
if stored.is_sealed() {
open(key, stored.as_str())
} else {
Some(stored.as_str().to_string())
}
}
#[cfg(test)]
mod tests {
use super::*;
use rand::Rng;
/// Test keys need no secrecy, but they still come from the OS CSPRNG so no
/// hard-coded key material sits in the tree. A wrong key fails XChaCha20-Poly1305
/// authentication with overwhelming probability, so the "wrong key" case stays
/// deterministic in practice.
fn new_key() -> [u8; 32] {
OsRng.gen()
}
fn other_key() -> [u8; 32] {
OsRng.gen()
}
#[test]
fn seal_open_round_trip() {
let k = new_key();
let sealed = seal(&k, "s3cr3t-value");
assert!(sealed.starts_with(SEAL_PREFIX));
assert_eq!(open(&k, &sealed).as_deref(), Some("s3cr3t-value"));
}
#[test]
fn sealed_value_is_never_the_plaintext() {
let k = new_key();
let sealed = seal(&k, "s3cr3t-value");
assert!(!sealed.contains("s3cr3t-value"));
}
#[test]
fn wrong_key_does_not_open() {
let sealed = seal(&new_key(), "s3cr3t-value");
assert_eq!(open(&other_key(), &sealed), None);
}
#[test]
fn tampered_or_truncated_value_does_not_open() {
let k = new_key();
let sealed = seal(&k, "s3cr3t-value");
// Flip a byte inside the base64 payload.
let mut chars: Vec<char> = sealed.chars().collect();
let idx = SEAL_PREFIX.len();
chars[idx] = if chars[idx] == 'A' { 'B' } else { 'A' };
let tampered: String = chars.into_iter().collect();
assert_eq!(open(&k, &tampered), None);
// A value that is just the prefix (no nonce/ciphertext) is malformed.
assert_eq!(open(&k, SEAL_PREFIX), None);
}
#[test]
fn reveal_opens_a_sealed_value() {
let k = new_key();
let sealed = WebhookSecret::new(seal(&k, "s3cr3t-value"));
assert_eq!(reveal(&k, &sealed).as_deref(), Some("s3cr3t-value"));
}
#[test]
fn legacy_plaintext_is_revealed_unchanged() {
let k = new_key();
let legacy = WebhookSecret::new("plaintext-secret".into());
assert!(!legacy.is_sealed());
assert_eq!(reveal(&k, &legacy).as_deref(), Some("plaintext-secret"));
}
#[test]
fn debug_is_redacted() {
let s = WebhookSecret::new("s3cr3t-value".into());
let rendered = format!("{s:?}");
assert!(!rendered.contains("s3cr3t-value"));
assert_eq!(rendered, "\"***\"");
}
#[test]
fn serialize_is_redacted() {
let s = WebhookSecret::new("s3cr3t-value".into());
let json = serde_json::to_string(&s).unwrap();
assert!(!json.contains("s3cr3t-value"));
assert_eq!(json, "\"***\"");
}
#[test]
fn deserialize_reads_the_underlying_string() {
let s: WebhookSecret = serde_json::from_str("\"s3cr3t-value\"").unwrap();
assert_eq!(s.as_str(), "s3cr3t-value");
}
}
+9 -293
View File
@@ -30,200 +30,34 @@ fn sign_payload(secret: &str, payload: &[u8]) -> String {
format!("sha256={}", hex::encode(mac.finalize().into_bytes()))
}
/// What the signing step decided for one hook. Keeping this a decision (not a
/// bare `Option`) lets the delivery path tell "hook has no secret" (send
/// unsigned) apart from "hook has a secret that could not be opened" (withhold:
/// an unsigned POST would present a broken hook as working while the ledger
/// marks the event delivered).
enum SigningOutcome {
NoSecret,
Signed(String),
Withhold,
}
fn signing_decision(
secret_key: &[u8; 32],
secret: Option<&crate::webhook_secret::WebhookSecret>,
bytes: &[u8],
) -> SigningOutcome {
match secret {
None => SigningOutcome::NoSecret,
Some(s) => match crate::webhook_secret::reveal(secret_key, s) {
Some(plain) => SigningOutcome::Signed(sign_payload(&plain, bytes)),
None => SigningOutcome::Withhold,
},
}
}
/// Fire webhooks for `event` on `repo_id`. Spawns background tasks — never blocks.
pub fn fire_event(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
) {
fire_event_occurrence(
db,
http_client,
secret_key,
repo_id,
event,
payload,
None,
None,
);
}
/// Occurrence-keyed variant: at most one delivery per
/// `(request_id, ref_name, hook)` is spawned; concurrent executors
/// collapse via the `webhook_deliveries` ledger. Best-effort on crash
/// (ledger is claimed before spawn), which the PR scope allows.
///
/// For durable outbox usage, call `claim_webhook_delivery_before_spawn` first
/// to claim deliveries synchronously, then call
/// `fire_event_occurrence_with_claimed_hooks` with the claimed hook IDs to
/// spawn HTTP POST only for claimed hooks.
#[allow(clippy::too_many_arguments)]
pub fn fire_event_occurrence(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
) {
let repo_id = repo_id.to_string();
let event = event.to_string();
let request_id = request_id.map(|s| s.to_string());
let ref_name = ref_name.map(|s| s.to_string());
tokio::spawn(async move {
fire_event_async_occurrence(
db,
http_client,
secret_key,
&repo_id,
&event,
payload,
request_id.as_deref(),
ref_name.as_deref(),
None,
)
.await;
fire_event_async(db, http_client, &repo_id, &event, payload).await;
});
}
/// Variant for durable outbox: only spawn HTTP POST for previously claimed hooks.
#[allow(clippy::too_many_arguments)]
pub fn fire_event_occurrence_with_claimed_hooks(
async fn fire_event_async(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
claimed_hooks: Vec<crate::db::Webhook>,
) {
let repo_id = repo_id.to_string();
let event = event.to_string();
let request_id = request_id.map(|s| s.to_string());
let ref_name = ref_name.map(|s| s.to_string());
tokio::spawn(async move {
fire_event_async_occurrence(
db,
http_client,
secret_key,
&repo_id,
&event,
payload,
request_id.as_deref(),
ref_name.as_deref(),
Some(claimed_hooks),
)
.await;
});
}
/// Claim webhook delivery before spawning HTTP POST.
/// Returns `Some(hook ids)` successfully claimed, or `None` when the hook
/// list itself could not be read (transient DB error, unknown membership).
/// A `None` caller must fall back to best-effort send rather than dropping
/// the delivery: children are still deleted downstream, so an empty claim
/// set on DB error would lose the webhook permanently.
/// This synchronous claim phase must be awaited before child deletion to prevent
/// permanent webhook loss on crash between deletion and claim.
pub async fn claim_webhook_delivery_before_spawn(
db: Arc<Db>,
repo_id: &str,
event: &str,
request_id: Option<&str>,
ref_name: Option<&str>,
) -> Option<Vec<crate::db::Webhook>> {
let mut claimed_hooks = Vec::new();
// Only claim when we have occurrence context (request_id + ref_name)
if let (Some(req), Some(r)) = (request_id, ref_name) {
let hooks = match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(err = %e, "failed to list webhooks for event {event}");
return None;
}
};
for hook in hooks {
let delivery_id = crate::db::deterministic_id(&["webhook", req, r, &hook.id]);
match db
.claim_webhook_delivery(&delivery_id, req, repo_id, event)
.await
{
Ok(true) => claimed_hooks.push(hook),
Ok(false) => continue, // Already claimed
// Send fallback (legacy best-effort): a transient claim
// error must not drop the delivery while children are
// still deleted. The in-task claimer this replaced sent
// anyway on claim error; keep that here.
Err(e) => {
tracing::warn!(err = %e, hook_id = %hook.id, "webhook claim failed; sending anyway");
claimed_hooks.push(hook);
}
}
let hooks = match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(err = %e, "failed to list webhooks for event {event}");
return;
}
}
Some(claimed_hooks)
}
#[allow(clippy::too_many_arguments)]
async fn fire_event_async_occurrence(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
claimed_hooks: Option<Vec<crate::db::Webhook>>,
) {
// A caller that already claimed deliveries hands the claimed hook rows
// through; the send path must not re-list, because a list failure here
// would silently drop deliveries already claimed in the ledger.
let already_claimed = claimed_hooks.is_some();
let hooks = match claimed_hooks {
Some(h) => h,
None => match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(err = %e, "failed to list webhooks for event {event}");
return;
}
},
};
if hooks.is_empty() {
@@ -242,67 +76,10 @@ async fn fire_event_async_occurrence(
let client = Arc::clone(&http_client);
let event_name = event.to_string();
let bytes = payload_bytes.clone();
// Stable occurrence delivery key when request context exists;
// random UUID otherwise (legacy callers without occurrence).
let delivery_id = match (request_id, ref_name) {
(Some(req), Some(r)) => crate::db::deterministic_id(&["webhook", req, r, &hook.id]),
_ => uuid::Uuid::new_v4().to_string(),
};
// Claim before spawn so concurrent executors collapse to one
// delivery per occurrence. Failures to record fall back to
// sending (legacy best-effort) rather than dropping.
// Skip claim if already done by caller (durable_outbox path).
if !already_claimed && request_id.is_some() {
match db
.clone()
.claim_webhook_delivery(&delivery_id, request_id.unwrap_or(""), repo_id, event)
.await
{
Ok(true) => {}
Ok(false) => continue,
Err(_) => {}
}
}
let delivery_id = uuid::Uuid::new_v4().to_string();
let db_for_mark = db.clone();
let secret_key = Arc::clone(&secret_key);
tokio::spawn(async move {
// Policy skip, not a delivery failure: the stored URL predates the
// create-time gate. The "invalid_url" label is documented on
// record_webhook_delivery in metrics.rs.
if !crate::api::peers::is_public_http_url(&hook.url) {
crate::metrics::record_webhook_delivery("invalid_url");
tracing::warn!(
url = %hook.url,
event = %event_name,
"webhook delivery skipped: stored URL is not a public http(s) target"
);
return;
}
let delivery_for_mark = delivery_id.clone();
// Open the secret only at signing time; at rest it stays sealed.
let signature =
match signing_decision(secret_key.as_ref(), hook.secret.as_ref(), &bytes) {
SigningOutcome::NoSecret => None,
SigningOutcome::Signed(sig) => Some(sig),
// A hook that HAS a secret must sign with it or not deliver at
// all: `reveal` returning None means the stored value could
// not be opened (a foreign or rotated key, a truncated
// ciphertext), and sending unsigned would present a broken
// hook as a working one while the ledger marks the event
// delivered. The row stays pending, like a network error, so
// a fixed key can still land it.
SigningOutcome::Withhold => {
crate::metrics::record_webhook_delivery("secret_unopenable");
tracing::warn!(
hook_id = %hook.id,
delivery_id = %delivery_for_mark,
"webhook secret could not be opened; delivery withheld"
);
return;
}
};
let signature = hook.secret.as_deref().map(|s| sign_payload(s, &bytes));
let mut req = client
.post(&hook.url)
@@ -323,10 +100,6 @@ async fn fire_event_async_occurrence(
"http_error"
};
crate::metrics::record_webhook_delivery(result_label);
// Any HTTP response proves the delivery fired; mark
// sent so the ledger never claims an unsent delivery.
// Network errors stay pending for stale-reclaim.
let _ = db_for_mark.mark_webhook_sent(&delivery_for_mark).await;
tracing::info!(
url = %hook.url,
event = %event_name,
@@ -347,60 +120,3 @@ async fn fire_event_async_occurrence(
});
}
}
#[cfg(test)]
mod tests {
use super::*;
use rand::Rng;
fn key() -> [u8; 32] {
chacha20poly1305::aead::OsRng.gen()
}
/// A hook with no secret delivers unsigned, as before.
#[test]
fn signing_decision_no_secret_is_unsigned() {
assert!(matches!(
signing_decision(&key(), None, b"{}"),
SigningOutcome::NoSecret
));
}
/// A sealed secret that opens signs with the revealed plaintext.
#[test]
fn signing_decision_sealed_secret_signs() {
let k = key();
let sealed =
crate::webhook_secret::WebhookSecret::new(crate::webhook_secret::seal(&k, "topsecret"));
match signing_decision(&k, Some(&sealed), b"{}") {
SigningOutcome::Signed(sig) => {
assert_eq!(sig, sign_payload("topsecret", b"{}"))
}
_ => panic!("an openable sealed secret must produce a signature"),
}
}
/// A stored value that carries the sealed prefix but cannot be opened
/// (foreign key, truncated ciphertext) must withhold, never send unsigned.
#[test]
fn signing_decision_unopenable_secret_withholds() {
let garbage =
crate::webhook_secret::WebhookSecret::new("enc:v1:bm90LWNoaXBoZXJ0ZXh0".to_string());
assert!(matches!(
signing_decision(&key(), Some(&garbage), b"{}"),
SigningOutcome::Withhold
));
}
/// A legacy plaintext value signs with itself unchanged.
#[test]
fn signing_decision_legacy_plaintext_signs() {
let legacy = crate::webhook_secret::WebhookSecret::new("plainsecret".to_string());
match signing_decision(&key(), Some(&legacy), b"{}") {
SigningOutcome::Signed(sig) => {
assert_eq!(sig, sign_payload("plainsecret", b"{}"))
}
_ => panic!("legacy plaintext must keep signing"),
}
}
}
-59
View File
@@ -1,59 +0,0 @@
use std::io::{Read, Write};
use std::process::{Command, Stdio};
use std::time::Duration;
fn main() -> std::io::Result<()> {
let executable = std::env::current_exe()?;
let config = std::fs::read_to_string(executable.parent().unwrap().join("config"))?;
let (mode, parameter) = config.split_once('\n').unwrap_or((&config, ""));
let args: Vec<_> = std::env::args_os().skip(1).collect();
if mode == "delay" {
std::thread::sleep(Duration::from_millis(parameter.parse().unwrap()));
}
let batch = args.get(1).is_some_and(|arg| arg == "--batch-check");
let input = if batch {
let mut bytes = Vec::new();
std::io::stdin().read_to_end(&mut bytes)?;
Some(bytes)
} else {
None
};
let cwd = std::env::current_dir()?;
let hang = mode == "hang"
|| (mode == "hang-repo" && cwd.file_name().is_some_and(|name| name == parameter))
|| (mode == "hang-oid"
&& (args.iter().any(|arg| arg == parameter)
|| input
.as_ref()
.is_some_and(|bytes| String::from_utf8_lossy(bytes).trim() == parameter)));
if hang {
std::thread::sleep(Duration::from_secs(30));
std::process::exit(1);
}
if mode == "log-types" && batch {
let mut log = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(parameter)?;
writeln!(
log,
"{} {}",
String::from_utf8_lossy(input.as_ref().unwrap()).trim(),
cwd.file_name().unwrap().to_string_lossy()
)?;
}
let mut child = Command::new("git")
.args(&args)
.stdin(if input.is_some() {
Stdio::piped()
} else {
Stdio::inherit()
})
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()?;
if let Some(bytes) = input {
child.stdin.take().unwrap().write_all(&bytes)?;
}
std::process::exit(child.wait()?.code().unwrap_or(1));
}
+27 -577
View File
@@ -416,8 +416,8 @@ fn f3_second_writer_leased_until_reap() {
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
);
let receive = repos_production
.find("smart_http::receive_pack_raw")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack_raw");
.find("smart_http::receive_pack(")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack");
assert!(
lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \
@@ -509,8 +509,8 @@ fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if disposition.spawn_tail`, and `release` must consume the same
/// disposition (`disposition.release_ok`) so the two cannot drift apart.
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so
/// the two cannot drift apart.
///
/// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
@@ -519,9 +519,7 @@ fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
/// `guard.release(` and the ordering assertion fails; take it out of the
/// `if disposition.spawn_tail` block and the failed-push assertion fails; gate
/// `spawn_tail`/`release_ok` on the outcome commit and
/// `post_git_disposition_replication_carve_out` fails with it.
/// `if push_succeeded` block and the failed-push assertion fails.
#[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs");
@@ -536,589 +534,41 @@ fn inv22_replication_tail_spawns_at_the_durability_boundary() {
.expect("split always yields a first chunk");
let success_flag = production
.find("post_git_disposition(")
.expect("U5 gate missing: the tail's disposition must be bound before the gate");
// One tail per push, on the proven-landed ref set. Either spawn
// position (carve-out early, or durability-bound at release) must sit
// inside a `disposition.spawn_tail` gate and feed `landed_refs` —
// never the unfiltered `ref_updates`.
let spawn_count = production
.matches("tokio::spawn(post_receive_replication_tail(")
.count();
assert!(
spawn_count >= 1,
"U5 gate missing: the replication tail must be spawned by git_receive_pack"
);
for (i, (pos, _)) in production
.match_indices("tokio::spawn(post_receive_replication_tail(")
.enumerate()
{
let gate = production[..pos].rfind("if disposition.spawn_tail").expect(
"U5 gate missing: every tail spawn must be gated on `if disposition.spawn_tail`",
.find("let push_succeeded = receive_result.is_ok();")
.expect(
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()",
);
// The spawn must sit inside the gate's block: brace depth at the
// spawn position must still be positive measured from the `if`.
// An ungated spawn after the block closes fails this, as does a
// stray `disposition.spawn_tail` mention in a comment.
let mut depth = 0i32;
for ch in production[gate..pos].chars() {
match ch {
'{' => depth += 1,
'}' => depth -= 1,
_ => {}
}
}
assert!(
depth >= 1,
"U5 gate bypassed: tail #{i} sits outside the `if disposition.spawn_tail` block"
);
assert!(
!production[gate..pos].contains("post_receive_replication_tail"),
"U5 gate bypassed: a second tail spawn sits between the gate and its spawn"
);
let call_end = production[pos..].find(");").unwrap() + pos;
let call = &production[pos..call_end];
assert!(
call.contains("landed_refs"),
"U5 gate bypassed: tail #{i} must carry the proven-landed ref set, not unfiltered ref_updates"
);
}
let gate_open = production
.find("if push_succeeded {")
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded");
let spawn = production
.find("tokio::spawn(post_receive_replication_tail(")
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production
.find(".release(disposition.release_ok)")
.expect("U5 gate stale: release must consume the same disposition as the tail gate");
.find(".release(push_succeeded)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate");
let touch = production
.find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
// #26 Split PR 1 step 3 — the webhook fan-out moved into
// `durable_outbox::apply_request_effects`, which the live
// handler calls inline (the recovery drain calls the same
// function on the next startup). The gate now pins that the
// handler is wired to the executor, not to a per-ref inline
// webhook call.
let effects_executor = production
.find("apply_request_effects(&state, &request_id)")
.expect("U5 gate stale: git_receive_pack no longer calls apply_request_effects inline");
let webhook = production
.find("webhooks::fire_event(")
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks");
assert!(
success_flag < spawn,
"U5 gate bypassed: the disposition must be bound before any tail spawn"
success_flag < gate_open && gate_open < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
rejected push now spawns a tail"
);
assert!(
spawn < release && spawn < touch && spawn < effects_executor,
spawn < release && spawn < touch && spawn < webhook,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the effect executor, so a disconnect in any of those windows cannot drop \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements"
);
}
/// #26 Split PR 1 step 3 — drain + handler share a single effect
/// executor. The live handler (api/repos.rs) and the recovery
/// drain (`durable_outbox::drain_receive_pack_requests_with`) both
/// call `apply_request_effects`, so the per-ref effects fan-out
/// lives in exactly one place. The v29 per-ref walk
/// (`derive_one`, `drain_pending_ref_transitions_all`,
/// `lookup_accepted_ordinal`) is dead code: any caller reintroduced
/// would be the per-ref walk the step-3 PR removed. This gate
/// fails if a call site slips back in or the new seam is bypassed.
#[test]
fn inv26_step3_live_and_drain_share_apply_request_effects() {
let repos = src("api/repos.rs");
let outbox = src("durable_outbox.rs");
// The live handler calls `apply_request_effects`. Split the
// file at the test attribute so test code can't satisfy the
// gate by itself.
let production_repos = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
production_repos.contains("apply_request_effects(&state, &request_id)"),
"live handler must call `apply_request_effects(&state, &request_id)`; \
reverting to a per-ref inline fan-out splits live and recovery"
);
// The durable intent insert closes the pre-outbox crash window this
// PR exists to fix. Wrapping it in `if false` must turn this gate
// red; drain-only tests cannot prove the handler creates the rows.
assert!(
production_repos.contains("insert_receive_pack_request_with_children("),
"live handler must persist durable intent via \
`insert_receive_pack_request_with_children`; disabling it must fail this gate"
);
// The drain's per-request seam calls the same executor. Test
// code lives below `mod drain_tests`, so split there too.
let production_outbox = outbox
.split("\nmod drain_tests {")
.next()
.expect("split always yields a first chunk");
assert!(
production_outbox.contains("drain_receive_pack_requests_with"),
"drain seam `drain_receive_pack_requests_with` must exist; \
removing it forces a per-ref walk back into the drain"
);
assert!(
production_outbox.contains("apply_request_effects"),
"durable_outbox production code must define `apply_request_effects`; \
removing it splits the executor between live and recovery"
);
// The drain's per-request walker is wired to the executor.
// The closure body of `drain_receive_pack_requests_with` is the
// only call site — if a future change calls `derive_one`
// instead, this assertion fires.
let drain_seam_open = production_outbox
.find("pub async fn drain_receive_pack_requests_with<F, Fut>")
.expect("drain seam must be defined in the production half");
let drain_seam_close = production_outbox[drain_seam_open..]
.find("\n}\n")
.expect("drain seam body must close");
let drain_seam_body = &production_outbox[drain_seam_open..drain_seam_open + drain_seam_close];
assert!(
drain_seam_body.contains("apply_request_effects"),
"drain_receive_pack_requests_with must call apply_request_effects; \
wiring it to a per-ref helper reintroduces the v29 walk"
);
// The deleted per-ref drain must have zero call sites in the
// production half. A regression that re-adds a caller would
// bring back the per-ref fan-out.
assert!(
!production_outbox.contains("drain_pending_ref_transitions_all("),
"deleted `drain_pending_ref_transitions_all` must have zero production call sites; \
a per-ref walk is reintroduced"
);
assert!(
!production_outbox.contains("derive_one("),
"deleted `derive_one` must have zero production call sites; \
the per-ref fan-out is reintroduced"
);
assert!(
!production_outbox.contains("lookup_accepted_ordinal("),
"deleted `lookup_accepted_ordinal` must have zero production call sites; \
the per-ref ordinal lookup is reintroduced"
);
assert!(
!production_repos.contains("derive_one("),
"deleted `derive_one` must have zero live-handler call sites"
);
assert!(
!production_repos.contains("drain_pending_ref_transitions"),
"deleted per-ref drain functions must have zero live-handler call sites"
);
}
/// #26 Split PR 1 step 4 — the periodic queue-lifecycle purge is
/// wired in `main.rs` and the contract is enforced by the `idx_receive_pack_requests_completed_at`
/// partial index from v41.
///
/// Assertions:
/// 1. `main.rs` calls `purge_request_queue` once at boot (well, on
/// the spawn-task interval) and the spawn function exists.
/// 2. The DB helpers `purge_completed_receive_pack_requests` and
/// `purge_completed_pending_ref_transitions` exist in `db/mod.rs`.
/// 3. The config knobs `queue_retention_days` and `queue_purge_batch` exist
/// on `Config`.
/// 4. `quarantined` is NOT in the purge WHERE clause (step 4 does not
/// introduce the state, but the invariant holds for the future).
#[test]
fn inv26_step4_queue_lifecycle_purge_is_wired() {
let main_src = src("main.rs");
assert!(
main_src.contains("spawn_queue_lifecycle_sweep"),
"main.rs must spawn the periodic queue-lifecycle purge"
);
assert!(
main_src.contains("purge_request_queue"),
"main.rs must call purge_request_queue on the periodic sweep"
);
let db_src = src("db/mod.rs");
assert!(
db_src.contains("purge_completed_receive_pack_requests"),
"db/mod.rs must expose purge_completed_receive_pack_requests"
);
assert!(
db_src.contains("purge_completed_pending_ref_transitions"),
"db/mod.rs must expose purge_completed_pending_ref_transitions"
);
let config_src = src("config.rs");
assert!(
config_src.contains("queue_retention_days"),
"Config must expose queue_retention_days"
);
assert!(
config_src.contains("queue_purge_batch"),
"Config must expose queue_purge_batch"
);
}
/// #26 Split PR 1 step 5 — the marker gate and the retry-bound
/// quarantine are wired end-to-end. A missing or hash-mismatched
/// marker on disk quarantines the request; a Retry over the bound
/// does too. This gate pins every load-bearing seam, against the
/// production half of each file (the test modules name the same
/// identifiers in their own harnesses).
///
/// Assertions:
/// 1. `reconcile_prepared_page` calls `mark_request_quarantined`
/// and `mark_children_rejected_for_quarantined_parent` on
/// marker-gate failure.
/// 2. The drain's `EffectsOutcome::Retry` arm checks
/// `effects_max_attempts`.
/// 3. `git::store::read_ref` exists in `git/store.rs`.
/// 4. `db::mark_request_quarantined`,
/// `db::mark_children_rejected_for_quarantined_parent`,
/// `db::get_receive_pack_requests_by_ids`, and
/// `db::request_state::QUARANTINED` all exist.
/// 5. The handler writes the marker ref BEFORE calling
/// `smart_http::receive_pack_raw` (the durability window).
#[test]
fn inv26_step5_marker_quarantine_and_bound_are_wired() {
let outbox = src("durable_outbox.rs");
let store = src("git/store.rs");
let db = src("db/mod.rs");
let repos = src("api/repos.rs");
// Split at the TEST MODULE for the production-only assertions.
let production_outbox = outbox
.split("\nmod drain_tests {")
.next()
.expect("split always yields a first chunk");
let production_repos = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (1) The reconcile's marker gate quarantines via the DB helpers.
assert!(
production_outbox.contains("mark_request_quarantined"),
"reconcile_prepared_page must call mark_request_quarantined on marker-gate failure"
);
assert!(
production_outbox.contains("mark_children_rejected_for_quarantined_parent"),
"reconcile_prepared_page must call mark_children_rejected_for_quarantined_parent \
so quarantined parents cancel their children"
);
// The gate reads the marker ref and compares against the parent's
// `request_bytes_hash` via `git::store::read_ref` /
// `git::store::marker_value_for`. Reverting either reintroduces
// the DoS window the marker gate exists to close.
assert!(
production_outbox.contains("git::store::read_ref"),
"reconcile_prepared_page must read the marker ref via git::store::read_ref"
);
assert!(
production_outbox.contains("marker_value_for"),
"reconcile_prepared_page must compute the expected marker value via \
git::store::marker_value_for"
);
// (2) The drain's `EffectsOutcome::Retry` arm checks the bound.
assert!(
production_outbox.contains("effects_max_attempts"),
"drain_receive_pack_requests_with must consult effects_max_attempts on Retry"
);
// (3) `git::store::read_ref` is the read seam the gate depends on.
assert!(
store.contains("pub fn read_ref("),
"git::store::read_ref must exist; the marker gate reads through it"
);
assert!(
store.contains("pub fn marker_value_for("),
"git::store::marker_value_for must exist; the marker gate computes the expected \
value with it (and the live handler writes the value via the same helper)"
);
// (4) DB-side seams the gate depends on.
assert!(
db.contains("pub async fn mark_request_quarantined"),
"Db::mark_request_quarantined must exist"
);
assert!(
db.contains("pub async fn mark_children_rejected_for_quarantined_parent"),
"Db::mark_children_rejected_for_quarantined_parent must exist"
);
assert!(
db.contains("pub async fn get_receive_pack_requests_by_ids"),
"Db::get_receive_pack_requests_by_ids must exist (avoids N+1 in the marker gate)"
);
assert!(
db.contains("pub const QUARANTINED: &str = \"quarantined\""),
"request_state::QUARANTINED must be defined"
);
// (5) The handler writes the marker ref BEFORE the durability
// boundary (smart_http::receive_pack_raw). Severing the
// ordering re-opens the marker-gate DoS window for live pushes.
let marker_write = production_repos
.find("git::store::marker_value_for")
.expect("U5 gate stale: the live handler no longer computes the marker value");
let receive_raw = production_repos
.find("smart_http::receive_pack_raw")
.expect("U5 gate stale: git_receive_pack no longer calls smart_http::receive_pack_raw");
assert!(
marker_write < receive_raw,
"U5 gate bypassed: the marker ref must be written BEFORE receive_pack_raw so the \
reconcile's gate has evidence of the live push"
);
}
/// Issue #26 split 1 — namespace gate and pre-git refusal lifecycle.
///
/// The push gate (repos.rs) must deny:
/// 1. Any ref-update line that fails UTF-8 decode (the raw body is passed verbatim
/// to git receive-pack, and git accepts refnames with bytes >= 0x80; the gate
/// never sees these refs, no durable child row exists, and under requests/issues/
/// the lossy for-each-ref decode still matches the narrowed exemption).
/// 2. The bare `refs/gitlawb` name (the `starts_with("refs/gitlawb/")` check misses it).
///
/// Every explicit pre-git refusal must call `refuse_receive_pack_before_git` to
/// terminalize the intent aggregate in one transaction, or a dropped future between
/// the intent insert and `receive_pack` strands a `received` + `prepared` aggregate
/// that is never purged, never promotable, and poisons its tuple.
///
/// The visibility exemption (visibility_pack.rs) was narrowed to `requests/` and
/// `issues/` only. The push gate is the premise that narrowing relies on (the comment
/// at visibility_pack.rs:340), so tests must prove:
/// - A `refs/gitlawb/<other>` ref pointing at a blob is denied on push.
/// - A stray `refs/gitlawb/<other>` blob ref planted in the pack path fails closed.
///
/// Source gates below pin the wiring; behavioral proofs live next to the code:
/// `parse_ref_updates_refuses_non_utf8_command_line` and
/// `internal_namespace_gate_covers_bare_name_and_subtree` (api/repos.rs),
/// `planted_non_exempt_gitlawb_blob_ref_fails_closed` (git/visibility_pack.rs),
/// and the mirror import prune
/// `prune_removes_non_exempt_gitlawb_refs_and_keeps_exempt` (sync.rs).
#[test]
fn issue_26_namespace_gate_and_refusal_wiring() {
let repos = src("api/repos.rs");
let vis = src("git/visibility_pack.rs");
// P1: fail the push on ref-update lines the handler cannot decode.
// The gate at parse_ref_updates must refuse when any command line in the
// ref-update section is undecodable UTF-8. The check is an Err arm on
// `from_utf8(data)` that returns BadRequest, so removing it allows non-UTF-8
// refs to bypass the namespace gate.
assert!(
repos.contains("ref-update line is not valid UTF-8; push refused"),
"P1 gate missing: parse_ref_updates must refuse when any ref-update line is \
non-UTF-8 (git accepts bytes >= 0x80; the gate never sees them, no durable \
child exists, and the lossy for-each-ref decode still matches the exemption)"
);
// P1 (cont): the bare `refs/gitlawb` name must also be caught. The
// predicate is extracted as `ref_is_internal_namespace` so it is
// unit-testable; pin the bare-name arm inside it and its use at the
// push gate (a `starts_with("refs/gitlawb/")`-only check misses the
// bare name, and it lands on repos with no internal refs yet).
assert!(
repos.contains("ref_name == \"refs/gitlawb\"")
&& repos.contains(".find(|r| ref_is_internal_namespace(r))"),
"P1 gate missing: ref_is_internal_namespace must match the bare \
'refs/gitlawb' name and the push gate must consult it \
(starts_with('refs/gitlawb/') misses it, and it lands on repos with \
no internal refs yet)"
);
// P1 (cont): every explicit pre-git refusal must call refuse_receive_pack_before_git
// to terminalize the intent. Exactly 6 occurrences: the five inline call
// sites (lease cap, per-source cap, write pool, acquire timeout, acquire
// error) plus the drop-guard's own pre-git spawn. The post-git guard phase
// calls mark_receive_pack_interrupted (a different name), so deleting any
// one inline site drops the count to 5 and turns this red. The parse and
// namespace refusals correctly carry no call: they return before the
// intent insert, so there is no aggregate to terminalize.
assert_eq!(
repos.matches("refuse_receive_pack_before_git(").count(),
6,
"P1 wiring: exactly 6 refuse_receive_pack_before_git calls expected (5 \
inline refusal sites + guard pre-git spawn); deleting any inline site \
must turn this red"
);
// Pin the five inline sites by their reason literals, so a site that stops
// terminalizing (or is renumbered away) fails by name, not just by count.
for reason in [
"repo write-lease waiter cap reached",
"admission per-source cap reached",
"git write pool saturated",
"acquire_write timed out",
"acquire_write failed",
"handler dropped before git started",
] {
assert!(
repos.contains(reason),
"P1 wiring: refusal reason literal missing: {reason}"
);
}
// P1 (cont): the intent drop-guard must be phase-aware. It is armed before
// the insert resolves (covering the cancellation window), flipped to
// git-started immediately before the receive_pack await (a drop from there
// is a mid-git disconnect, terminalized via mark_receive_pack_interrupted
// because the Err arm never runs on a drop), and disarmed once the outcome
// commit resolves. Assert the ordering, not just presence: the phase flip
// must precede the receive_pack call.
assert!(
repos.contains("IntentDropGuard::new") && repos.contains("notify_git_started()"),
"P1 gate missing: IntentDropGuard must exist and flip to git-started phase \
before receive_pack (a dropped future between intent insert and git start \
strands a received+prepared aggregate)"
);
let flip_pos = repos
.find("notify_git_started()")
.expect("notify_git_started call must exist");
let git_pos = repos
.find("receive_pack_raw_with_reflog(")
.expect("receive_pack call must exist");
assert!(
flip_pos < git_pos,
"P1 ordering: notify_git_started() must precede the receive_pack call, \
or a mid-git drop is misclassified as a pre-git shed"
);
assert!(
repos.contains("_intent_guard.disarm()"),
"P1 gate missing: the guard must disarm once the outcome commit resolves, \
or every push spawns a wasteful cleanup txn at scope end"
);
// A sync Drop cannot await: cleanup is spawned via the current runtime
// handle with an off-runtime fallback. A bare tokio::spawn panics
// mid-unwind with no runtime (off-runtime drop, process teardown).
assert!(
repos.contains("Handle::try_current()"),
"P1 gate missing: guard Drop must spawn via Handle::try_current with an \
off-runtime arm, not a bare tokio::spawn"
);
// P2: the post-git drop arm must exist and be wired. The guard's
// GIT_STARTED phase spawns `mark_receive_pack_interrupted` (parent to
// `rejected_at_git`, children to `uncertain` — the Err arm never runs on
// a drop). Deleting that spawn keeps every count above green, so pin the
// method name; its behavior is proved by
// `post_git_interruption_marks_uncertain_and_terminalizes_parent`.
assert!(
repos.contains("mark_receive_pack_interrupted("),
"P2 wiring: the guard's post-git phase must spawn \
mark_receive_pack_interrupted (a mid-git drop otherwise strands \
received+prepared with no Err arm to mark them uncertain)"
);
// The guard must commit computed fates rather than a blanket uncertain
// once the report is parsed, and only disarm on a resolved commit: pin
// both the stash call and the `outcome_commit_ok` gate. Making disarm()
// unconditional strands the commit-failure path and misfiles proven
// rejections as uncertain.
assert!(
repos.contains("set_fates(ComputedFates"),
"P2 wiring: computed fates must be stashed on the guard once parsed, \
or a post-commit drop can only apply a context-free uncertain"
);
assert!(
repos.contains("if outcome_commit_ok"),
"P2 wiring: disarm must be gated on outcome_commit_ok; unconditional \
disarm strands the persistent-commit-failure path"
);
// The live-handler `AwaitingSiblings` arm is unreachable through the
// current outcome builder (documented at the arm itself) and therefore
// has no behavioral test; pin its presence so removing it is conscious.
// The drain consumer IS behaviorally driven (sibling-wait tests).
assert_eq!(
repos.matches("EffectsOutcome::AwaitingSiblings").count(),
1,
"live AwaitingSiblings arm must exist (defensive symmetry with the \
drain consumer); deleting it must turn this red"
);
// P2: pin the mirror prune call sites. The prune helper is called from
// both `clone_repo` and `fetch_repo` (plus its own definition) — deleting
// either call leaves the direct-helper test green while mirrors import
// evil refs again. Count occurrences of the call with its argument.
let sync_src = src("sync.rs");
assert_eq!(
sync_src
.matches("prune_non_exempt_gitlawb_refs(local_path)")
.count(),
2,
"P2 wiring: prune must be called from both clone_repo and fetch_repo; \
deleting either call must turn this red"
);
// Both call sites must route through `wrap_prune_error`, which preserves
// deterministic `PruneInvalid` refusals instead of stringifying every
// error into transient `PruneFailed` (an every-tick deferral spin for
// permanently invalid mirrors). Pin the exact call shape at both sites.
assert_eq!(
sync_src.matches(".map_err(wrap_prune_error)").count(),
2,
"P2 wiring: both prune call sites must use wrap_prune_error; inline \
rewrap must turn this red"
);
// P2: the visibility exemption narrowing removed a serving path. The exemption
// is now scoped to `requests/` and `issues/` only (visibility_pack.rs:371), and
// the comment at :340 states the push gate is the premise. Tests must prove the
// gate works, which the P1 findings above covered (non-UTF-8 line, bare name,
// refusal wiring). This check verifies the exemption narrowing landed.
assert!(
vis.contains(
r#"!r.starts_with("refs/gitlawb/requests/") && !r.starts_with("refs/gitlawb/issues/")"#
),
"P2 gate missing: the visibility exemption must be narrowed to requests/ and \
issues/ only (the push gate is the premise; any other refs/gitlawb/* ref is \
denied on push and must fail closed in the pack path)"
);
}
/// Claimed-webhook delivery must not re-list hooks: the durable path claims
/// synchronously, hands the claimed hook rows to the sender, and a list
/// failure in the send task would silently drop already-claimed deliveries.
#[test]
fn inv26_claimed_hook_delivery_never_relists() {
let webhooks = src("webhooks.rs");
let production = webhooks
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// The claimed entry point exists and passes hook ROWS (not ids) so the
// sender never needs the hook table again.
let entry = production
.find("fire_event_occurrence_with_claimed_hooks(")
.expect("claimed-hooks sender missing");
let claimed_rows = production
.find("claimed_hooks: Vec<crate::db::Webhook>")
.expect("claimed path must carry Webhook rows, not ids");
let list_call = production
.find("db.list_webhooks_for_event(repo_id, event)")
.expect("unclaimed path still lists hooks");
assert!(
entry < claimed_rows && claimed_rows < list_call,
"claimed sender must hand rows through before any list fallback"
);
// Inside the async worker, the claimed branch must bypass the list.
// Inside the async worker (which starts after the claim helper), the
// claimed branch must be selected before the list fallback can run.
let worker = production
.find("async fn fire_event_async_occurrence(")
.expect("occurrence worker missing");
let already_claimed = production[worker..]
.find("let already_claimed = claimed_hooks.is_some();")
.map(|i| worker + i)
.expect("claimed path must short-circuit before re-listing");
let worker_list = production[worker..]
.find("db.list_webhooks_for_event(repo_id, event)")
.map(|i| worker + i)
.expect("unclaimed path still lists hooks");
assert!(
already_claimed < worker_list,
"the claimed branch must be selected before list_webhooks_for_event can run"
);
}
@@ -1,177 +0,0 @@
//! #26 Split PR 4 — `GITLAWB_ENFORCE_OWNER_PUSH` default is consistent across surfaces.
//!
//! Reviewer 2 (2026-08-28) closed PR #224 with a P2 finding: SECURITY.md and
//! README.md both stated the default was `false`, while `Config::enforce_owner_push`
//! declared `default_value_t = true` and the runtime, .env.example, and the env
//! var table on the README all agreed with the code. Two operator-facing
//! surfaces lied about the authorization policy. PR #330 flipped the code
//! default and updated .env.example / README env table / docs/RUN-A-NODE.md,
//! but missed these two prose mentions.
//!
//! This guard pins the documentation and example surfaces together. Each one
//! is read off the working tree at test time, not a copy in the binary, so a
//! revert of the doc edit turns it red:
//!
//! 1. `SECURITY.md` says the default is `true`.
//! 2. `README.md` says the default is `true` in the known-limitations prose.
//! 3. `README.md` says the default is `true` in the env-var table row.
//! 4. `.env.example` sets `GITLAWB_ENFORCE_OWNER_PUSH=true`.
//!
//! The runtime declared default (`Config::enforce_owner_push`'s
//! `default_value_t = true`) is protected by the
//! `enforce_owner_push_is_declared_true_independent_of_the_environment`
//! unit test in `config.rs`, which reads the parser directly and is
//! env-independent. The two checks together cover the surfaces that
//! produced the original doc-vs-code drift; this guard cannot reach into
//! the binary crate's `Config` type (it is private to the bin target,
//! which has no `lib.rs`) without a larger refactor, so the contract is
//! stated as the four documentation/example surfaces above.
//!
//! The four are checked independently. A check that just verified "at least
//! one surface says true" would pass while a stale SECURITY.md lied; a check
//! that only verified the binary would not catch the doc drift that
//! produced this finding in the first place.
use std::path::Path;
fn read(rel: &str) -> String {
let p = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join(rel);
std::fs::read_to_string(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()))
}
/// Pin the prose mentions of `GITLAWB_ENFORCE_OWNER_PUSH` so a future doc
/// edit cannot reintroduce the "defaults to false" wording the reviewer
/// flagged. The wording in the test is what the docs MUST say about the
/// default; absence of any prose mention is also a fail because operators
/// rely on the docs to learn the default.
#[test]
fn security_md_states_owner_push_default_is_true() {
let security = read("SECURITY.md");
let section = section_after(&security, "### Repository write authorization defaults")
.unwrap_or_else(|| panic!("SECURITY.md is missing the owner-push section"));
assert!(
section.contains("defaults to `true`"),
"SECURITY.md owner-push section must state the default is true. \
Found:\n{section}"
);
assert!(
!section_claims_default_is_false(&section),
"SECURITY.md owner-push section must not claim the default is false. \
Found:\n{section}"
);
}
#[test]
fn readme_known_limitations_states_owner_push_default_is_true() {
let readme = read("README.md");
let limitations = section_after(&readme, "Known limitations:")
.unwrap_or_else(|| panic!("README.md is missing the Known limitations: section"));
// The owner-push line is one bullet in the limitations list. Pull the
// single line that mentions the env var so the assertion is local to
// the claim, not a substring of unrelated text.
let line = limitations
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| {
panic!(
"README.md Known limitations must mention GITLAWB_ENFORCE_OWNER_PUSH.\n\
Got:\n{limitations}"
)
});
assert!(
line.contains("defaults to `true`") || line.contains("defaults to true"),
"README Known-limitations owner-push bullet must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README Known-limitations owner-push bullet must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn readme_env_var_table_states_owner_push_default_is_true() {
let readme = read("README.md");
let line = readme
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH") && l.contains("|"))
.unwrap_or_else(|| {
panic!("README.md is missing the env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH")
});
assert!(
line.contains("Defaults to `true`") || line.contains("defaults to `true`"),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn env_example_sets_owner_push_true() {
let env = read(".env.example");
let line = env
.lines()
.find(|l| l.trim_start().starts_with("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| panic!(".env.example is missing GITLAWB_ENFORCE_OWNER_PUSH"));
assert!(
line.contains("=true"),
".env.example GITLAWB_ENFORCE_OWNER_PUSH must be set to true. Found:\n{line}"
);
}
/// A claim that the default is false can hide in many phrasings. Match the
/// forms that have appeared in the wild on this branch: "defaults to `false`",
/// "defaults to false", "default is false", "is `false` for compatibility",
/// "is false for compatibility". The match is intentionally narrow — false
/// positives would suppress a real doc edit.
fn section_claims_default_is_false(section: &str) -> bool {
let lower = section.to_ascii_lowercase();
lower.contains("defaults to `false`")
|| lower.contains("defaults to false")
|| lower.contains("default is `false`")
|| lower.contains("default is false")
|| (lower.contains("`false`") && lower.contains("for compatibility"))
}
/// Return the prose after a marker up to the next heading boundary. The
/// marker is normally a markdown heading (e.g. `### Section`); if it has
/// no `#` prefix it is treated as a plain-text marker and the slice is
/// bounded by the next heading of any level. This matters because a
/// plain-text marker would otherwise produce `header_level = 0`, making
/// the same-or-higher-level stop condition unsatisfiable and letting the
/// slice run to EOF.
fn section_after(doc: &str, heading: &str) -> Option<String> {
let start = doc.find(heading)?;
let after = &doc[start..];
let header_level = heading.chars().take_while(|c| *c == '#').count();
// Skip past the heading line itself.
let body_start = after.find('\n')? + 1;
let body = &after[body_start..];
// Stop at the next heading of the same or higher level, or at the next
// heading of any level when the marker had no `#` prefix.
let mut end = body.len();
for line in body.lines() {
let level = line.chars().take_while(|c| *c == '#').count();
if level > 0 && (header_level == 0 || level <= header_level) {
end = line.as_ptr() as usize - body.as_ptr() as usize;
break;
}
}
Some(body[..end].to_string())
}
-4
View File
@@ -26,10 +26,7 @@ clap = { version = "4", features = ["derive", "env"] }
dirs = "5"
reqwest = { workspace = true }
uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
urlencoding = "2"
libc = "0.2"
alloy = { version = "1", default-features = false, features = [
"contract",
"provider-http",
@@ -45,4 +42,3 @@ alloy = { version = "1", default-features = false, features = [
mockito = "1"
tempfile = "3"
tokio = { workspace = true }
ed25519-dalek = { workspace = true }
+26 -17
View File
@@ -2,6 +2,7 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -64,16 +65,24 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
.get(&path)
.await
.context("failed to connect to node")?;
// Keep the bespoke 404 hint (older node without the agents API) ahead of
// read_json; every other status routes through it (capped + sanitized error).
if resp.status() == reqwest::StatusCode::NOT_FOUND {
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!(
"this node does not yet support the agents API (v0.3+)\n\
upgrade the node or check GITLAWB_NODE is pointing to the right server"
);
}
let body = crate::http::read_json(resp, "list agents").await?;
let body: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("list agents failed ({status}): {msg}");
}
let agents = body["agents"].as_array().cloned().unwrap_or_default();
@@ -88,7 +97,7 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
let short = did
.split(':')
.next_back()
.map(|s| crate::text::truncate(s, 16))
.map(|s| &s[..s.len().min(16)])
.unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
let caps = agent["capabilities"]
@@ -116,15 +125,24 @@ async fn cmd_show(did: String, node: String) -> Result<()> {
.get(&format!("/api/v1/agents/{did}"))
.await
.context("failed to connect to node")?;
// Keep the bespoke 404 hint ahead of read_json; other statuses route through it.
if resp.status() == reqwest::StatusCode::NOT_FOUND {
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!(
"agent not found, or this node does not yet support the agents API (v0.3+)\n\
upgrade the node or check GITLAWB_NODE is pointing to the right server"
);
}
let agent = crate::http::read_json(resp, "agent").await?;
let agent: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = agent["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("agent not found ({status}): {msg}");
}
let full_did = agent["did"].as_str().unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
@@ -205,14 +223,11 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not found"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_list(server.url(), None).await.unwrap_err();
assert!(err.to_string().contains("agents API"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -223,14 +238,11 @@ mod tests {
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"internal error"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_list(server.url(), None).await.unwrap_err();
assert!(err.to_string().contains("list agents failed"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -257,7 +269,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"agent not found"}"#)
.expect(1)
.create_async()
.await;
@@ -265,8 +276,6 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("agents API") || err.to_string().contains("not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
+52 -106
View File
@@ -2,6 +2,7 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -220,7 +221,13 @@ async fn cmd_create(
.await
.context("failed to connect to node")?;
let body = crate::http::read_json(resp, "create bounty").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("create bounty failed ({status}): {msg}");
}
let id = body["id"].as_str().unwrap_or("?");
println!("Bounty created: {id}");
@@ -256,14 +263,11 @@ async fn cmd_list(
u
};
let body = crate::http::read_json(
client
.get_authed(&url)
.await
.context("failed to connect to node")?,
"bounties",
)
.await?;
let resp = client
.get_authed(&url)
.await
.context("failed to connect to node")?;
let body: Value = resp.json().await.unwrap_or_default();
let bounties = body["bounties"].as_array();
if let Some(arr) = bounties {
@@ -275,7 +279,7 @@ async fn cmd_list(
let title = b["title"].as_str().unwrap_or("?");
let amount = b["amount"].as_i64().unwrap_or(0);
let st = b["status"].as_str().unwrap_or("?");
let short_id = crate::text::truncate(id, 8);
let short_id = &id[..8.min(id.len())];
println!("{short_id} {st:<10} {amount:>12} $GITLAWB {title}");
}
}
@@ -287,7 +291,13 @@ async fn cmd_show(id: String, node: String, dir: Option<PathBuf>) -> Result<()>
let client = signed_client(&node, dir.as_deref());
let resp = client.get_authed(&format!("/api/v1/bounties/{id}")).await?;
let body = crate::http::read_json(resp, "show bounty").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("not found");
anyhow::bail!("bounty not found: {msg}");
}
println!("{}", serde_json::to_string_pretty(&body)?);
Ok(())
@@ -311,7 +321,13 @@ async fn cmd_claim(
)
.await?;
let _ = crate::http::read_json(resp, "claim").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("claim failed ({status}): {msg}");
}
println!("Bounty {id} claimed. Deadline starts now.");
Ok(())
@@ -330,7 +346,13 @@ async fn cmd_submit(id: String, pr: String, node: String, dir: Option<PathBuf>)
)
.await?;
let _ = crate::http::read_json(resp, "submit").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("submit failed ({status}): {msg}");
}
println!("Bounty {id} submitted with PR {pr}. Awaiting creator approval.");
Ok(())
@@ -354,7 +376,13 @@ async fn cmd_approve(
)
.await?;
let _ = crate::http::read_json(resp, "approve").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("approve failed ({status}): {msg}");
}
println!("Bounty {id} approved! Payout released to agent.");
Ok(())
@@ -373,7 +401,13 @@ async fn cmd_cancel(id: String, node: String, dir: Option<PathBuf>) -> Result<()
)
.await?;
let _ = crate::http::read_json(resp, "cancel").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("cancel failed ({status}): {msg}");
}
println!("Bounty {id} cancelled. Tokens refunded.");
Ok(())
@@ -381,11 +415,8 @@ async fn cmd_cancel(id: String, node: String, dir: Option<PathBuf>) -> Result<()
async fn cmd_stats(node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
// Route through read_json so a node denial (403/404/5xx, incl. a non-JSON
// body) surfaces as an error instead of a fake `Bounty Stats` with every
// counter zeroed (#186, the denial-as-success class #123 fixes elsewhere).
let body =
crate::http::read_json(client.get("/api/v1/bounties/stats").await?, "bounty stats").await?;
let resp = client.get("/api/v1/bounties/stats").await?;
let body: Value = resp.json().await.unwrap_or_default();
let open = body["open"].as_i64().unwrap_or(0);
let claimed = body["claimed"].as_i64().unwrap_or(0);
@@ -403,7 +434,7 @@ async fn cmd_stats(node: String) -> Result<()> {
let did = entry["did"].as_str().unwrap_or("?");
let count = entry["completed"].as_i64().unwrap_or(0);
let earned = entry["total_earned"].as_i64().unwrap_or(0);
let short = crate::text::truncate_suffix(did, 8);
let short = &did[did.len().saturating_sub(8)..];
println!(
" {}. ...{short} {count} bounties {earned} $GITLAWB",
i + 1
@@ -522,7 +553,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not found"}"#)
.expect(1)
.create_async()
.await;
@@ -530,8 +560,6 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -548,86 +576,4 @@ mod tests {
cmd_stats(server.url()).await.unwrap();
}
/// #186 (F1): a node denial on the stats endpoint must surface as an error,
/// not print a fake `Bounty Stats` with every counter zeroed. RED before the
/// fix (unwrap_or_default + no status check -> Ok with zeros), GREEN after
/// (read_json bails on the non-2xx).
#[tokio::test]
async fn cmd_stats_surfaces_denial_not_fake_result() {
let mut server = mockito::Server::new_async().await;
let m = server
.mock("GET", mockito::Matcher::Regex(r"/stats$".to_string()))
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"forbidden"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_stats(server.url())
.await
.expect_err("a 403 must be an error, not a zeroed fake result");
assert!(err.to_string().contains("403"), "err={err}");
// Pin the mock hit explicitly. The status assertion above already fails
// when the route does not match (mockito answers an unmatched route with
// 501, so "403" is absent), but that proof is incidental to mockito's
// fallback status; assert the hit directly so it survives a reworded
// assertion.
m.assert_async().await;
}
/// #186 (F1): the same denial-as-error behavior across the non-2xx response
/// classes, not just 403 — a 404 and a 500 (incl. a non-JSON body) must each
/// surface as an error with the status, never a zeroed fake result.
#[tokio::test]
async fn cmd_stats_surfaces_denial_across_status_classes() {
for (status, body, json) in [
(404, r#"{"message":"not found"}"#, true),
(500, "internal boom", false),
] {
let mut server = mockito::Server::new_async().await;
let mut m = server
.mock("GET", mockito::Matcher::Regex(r"/stats$".to_string()))
.with_status(status)
.with_body(body);
if json {
m = m.with_header("content-type", "application/json");
}
let m = m.expect(1).create_async().await;
let err = cmd_stats(server.url()).await.unwrap_err().to_string();
assert!(
err.contains(&status.to_string()),
"status {status} must surface as an error, got: {err}"
);
m.assert_async().await;
}
}
#[tokio::test]
async fn cmd_list_repo_scoped_surfaces_denial_not_empty() {
// `gl bounty list --repo owner/name` hits the gated /repos/{o}/{n}/bounties;
// a 404 must Err, not silently print nothing.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"/repos/alice/secret/bounties".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list(Some("alice/secret".to_string()), None, server.url(), None).await;
assert!(
result.is_err(),
"bounty list --repo must Err on a gated 404"
);
// Prove the gated repo-scoped path was actually requested: without this,
// an unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
}
+61 -1149
View File
File diff suppressed because it is too large. Load diff
+16 -34
View File
@@ -2,6 +2,7 @@
use anyhow::{Context, Result};
use clap::Args;
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -45,7 +46,12 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(&args.node, None);
let info = crate::http::read_json(client.get("/").await?, "node info").await?;
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let did = info["did"].as_str().context("node missing DID")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -62,7 +68,13 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
.await
.context("failed to connect to node")?;
let body = crate::http::read_json(resp, "changelog").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("changelog failed ({status}): {msg}");
}
let events = body["events"].as_array().cloned().unwrap_or_default();
@@ -75,14 +87,14 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
for event in &events {
let ts = event["timestamp"].as_str().unwrap_or("?");
let date = crate::text::truncate(ts, 10);
let date = &ts[..ts.len().min(10)];
match event["type"].as_str().unwrap_or("") {
"commit" => {
let sha = event["sha"].as_str().unwrap_or("?");
let msg = event["message"].as_str().unwrap_or("?");
let first_line = msg.lines().next().unwrap_or(msg);
let short_sha = crate::text::truncate(sha, 8);
let short_sha = &sha[..sha.len().min(8)];
println!(" {date} commit {short_sha} {first_line}");
}
"pr_merged" => {
@@ -204,7 +216,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.expect(1)
.create_async()
.await;
@@ -216,8 +227,6 @@ mod tests {
};
let err = run(args).await.unwrap_err();
assert!(err.to_string().contains("changelog failed"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -265,31 +274,4 @@ mod tests {
let err = rt.block_on(run(args)).unwrap_err();
assert!(err.to_string().contains("no repo specified"));
}
#[tokio::test]
async fn resolve_via_run_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the inline GET /
// node-info fetch during resolution. A gated 404 there must Err (surfacing
// the status), proving the read_json conversion is load-bearing.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = run(ChangelogArgs {
repo: Some("noslash".to_string()),
limit: 20,
node: server.url(),
dir: Some(dir.path().to_path_buf()),
})
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
}
+7 -120
View File
@@ -163,8 +163,8 @@ pub fn setup_partial_clone(
if withheld_globs.is_empty() {
match branch {
Some(b) => git_global(&["clone", "-q", "--branch", b, "--", remote_url, dest_str])?,
None => git_global(&["clone", "-q", "--", remote_url, dest_str])?,
Some(b) => git_global(&["clone", "-q", "--branch", b, remote_url, dest_str])?,
None => git_global(&["clone", "-q", remote_url, dest_str])?,
}
return Ok(());
}
@@ -174,7 +174,6 @@ pub fn setup_partial_clone(
"-q",
"--filter=blob:none",
"--no-checkout",
"--",
remote_url,
dest_str,
])?;
@@ -267,31 +266,7 @@ async fn fetch_withheld(node: &str, owner: &str, name: &str) -> Result<(Vec<Stri
};
let resp = match resp {
Ok(r) if r.status().is_success() => r,
Ok(r) if matches!(r.status().as_u16(), 404 | 501) => {
// Two 404s share this status and must be told apart before the
// legacy empty-glob fallback is taken: a route miss on a node that
// predates withheld-paths has an empty body, while a current node
// returns the JSON AppError body for absent/denied/quarantined
// repos. The JSON body proves the route exists, so the repo is
// unavailable to this caller and cloning cannot proceed.
if r.status().as_u16() == 404 {
let status = r.status();
let body = crate::http::read_body_capped(r, 4096).await;
if body.read_failed {
// The read died part-way: we cannot classify the denial,
// and a failed read must not collapse into the legacy
// route-miss arm and start a clone.
bail!("withheld-paths 404 body from this node could not be read");
}
if !body.text.trim().is_empty() {
bail!(
"repository {owner}/{name} is not available on this node ({status}): {}",
sanitize_node_msg(&body.text)
);
}
}
return Ok((Vec::new(), Vec::new()));
}
Ok(r) if matches!(r.status().as_u16(), 404 | 501) => return Ok((Vec::new(), Vec::new())),
Ok(r) => bail!("withheld-paths lookup failed: {}", r.status()),
Err(err) => return Err(err).context("fetching withheld paths"),
};
@@ -351,12 +326,7 @@ async fn recover_encrypted_blobs(
.await
{
Ok(r) if r.status().is_success() => r,
// Discovery failure is not an empty inventory: a denied, quarantined,
// erroring, or unreachable node must surface so the caller can warn
// and still run the Arweave fallback rather than report a clean clone
// with authorized blobs silently missing.
Ok(r) => bail!("encrypted-blobs inventory unavailable: {}", r.status()),
Err(err) => return Err(err).context("fetching encrypted-blobs inventory"),
_ => return Ok(vec![]),
};
let body: EncryptedBlobsResponse = resp.json().await.context("parsing encrypted-blobs")?;
if body.blobs.is_empty() {
@@ -418,7 +388,7 @@ async fn recover_encrypted_blobs(
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
Err(e) => {
emit_warning(&format!("warning: could not decrypt {oid}: {e}"));
eprintln!("warning: could not decrypt {oid}: {e}");
continue;
}
};
@@ -436,9 +406,7 @@ async fn recover_encrypted_blobs(
recovered.push(p.clone());
}
} else {
emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
}
}
Ok(recovered)
@@ -836,9 +804,7 @@ async fn recover_from_arweave(
recovered.push(p.clone());
}
} else {
emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
}
}
Ok(recovered)
@@ -1124,85 +1090,6 @@ mod tests {
);
}
#[test]
fn setup_partial_clone_plain_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], None).unwrap();
assert!(dest.join("file.txt").exists());
}
#[test]
fn setup_partial_clone_with_branch_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let branch_out = Command::new("git")
.args([
"-C",
td.path().join("origin").to_str().unwrap(),
"branch",
"--show-current",
])
.output()
.unwrap();
let branch = String::from_utf8(branch_out.stdout)
.unwrap()
.trim()
.to_string();
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], Some(&branch)).unwrap();
assert!(dest.join("file.txt").exists());
}
/// #374: every `setup_partial_clone` shape interpolates `remote_url` into a
/// `git clone` argv. The `--` delimiter is what forces an option-shaped
/// remote to be read as a repository rather than parsed as an option.
///
/// The assertion is on git's own words — `repository '<value>'`, single-quoted
/// — and not merely on the value appearing somewhere in the error. `git_global`
/// formats its failure as `git {args:?} failed: {stderr}`, so the injected
/// string is already present in the debug-printed argv whether or not the
/// delimiter is there; asserting on containment alone would pass with `--`
/// deleted. Undelimited, git consumes the value as an option, leaving the
/// destination as the only positional, and names *that* instead
/// (`repository '<dest>' does not exist`) — so this goes red without the fix.
///
/// The payload carries no `:` or space: a colon would make git read the
/// argument as an ssh-style `host:path` URL and report a hostname, which is a
/// different parse.
fn assert_option_shaped_remote_is_a_repository(withheld: &[String], branch: Option<&str>) {
let td = TempDir::new().unwrap();
let dest = td.path().join("dest");
let injected = "--upload-pack=false";
let err = setup_partial_clone(&dest, injected, withheld, &[], branch)
.expect_err("an option-shaped remote is not a repository and must fail")
.to_string();
assert!(
err.contains(&format!("repository '{injected}'")),
"git must name the argument as the repository, not consume it as an option: {err}"
);
}
#[test]
fn plain_clone_never_parses_a_remote_as_a_git_option() {
assert_option_shaped_remote_is_a_repository(&[], None);
}
#[test]
fn branch_clone_never_parses_a_remote_as_a_git_option() {
// Pins that `--branch <b>` stays ahead of the delimiter: if `--` were moved
// before `--branch`, git would read the branch name as a positional instead.
assert_option_shaped_remote_is_a_repository(&[], Some("main"));
}
#[test]
fn sparse_clone_never_parses_a_remote_as_a_git_option() {
// The withheld-globs arm is a separate argv construction
// (`--filter=blob:none --no-checkout`) and needs its own proof.
assert_option_shaped_remote_is_a_repository(&["/secret/**".to_string()], None);
}
#[test]
fn sparse_patterns_subtree_and_exact() {
assert_eq!(sparse_patterns("/secret/**"), vec!["/secret/".to_string()]);
+73 -275
View File
@@ -3,7 +3,7 @@
//! Checks:
//! 1. Identity — ~/.gitlawb/identity.pem exists, DID parseable
//! 2. Registration — ~/.gitlawb/ucan.json exists (registered with a node)
//! 3. GITLAWB_NODE — env var points at a public node (unset is advisory)
//! 3. GITLAWB_NODE — env var is set to a non-localhost URL
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.twigpine.com)
//! 5. git helper — git-remote-gitlawb is in PATH
//! 6. git — git is installed
@@ -24,7 +24,7 @@ pub struct DoctorArgs {
#[arg(long, default_value = PUBLIC_NODE, env = "GITLAWB_NODE")]
pub node: String,
/// Identity directory (default: the parent of $GITLAWB_KEY, else ~/.gitlawb)
/// Identity directory (default: ~/.gitlawb)
#[arg(long)]
pub dir: Option<PathBuf>,
}
@@ -69,63 +69,23 @@ impl Check {
}
}
/// Resolve the config dir and key path without aborting on a refused resolver.
/// The whole point of doctor is to explain a broken setup, so a `GITLAWB_KEY`
/// or `--dir` the resolvers reject becomes `Check::fail` rows for the checks
/// that depend on them, while node/git checks still run.
fn resolve_config_paths(
args_dir: Option<PathBuf>,
) -> (Option<PathBuf>, Option<PathBuf>, Vec<Check>) {
let dir = crate::identity::gitlawb_dir(args_dir.clone());
let mut checks = Vec::new();
let dir = match dir {
Ok(d) => Some(d),
Err(e) => {
checks.push(Check::fail(
"identity",
format!("could not resolve config dir: {e}"),
"fix GITLAWB_KEY or --dir",
));
checks.push(Check::fail(
"registration",
"skipped: config dir unresolved",
"fix GITLAWB_KEY or --dir",
));
None
}
};
let pem_path = match crate::identity::key_path_for(args_dir.as_deref()) {
Ok(p) => Some(p),
Err(e) => {
if dir.is_some() {
checks.push(Check::fail(
"identity",
format!("could not resolve key path: {e}"),
"fix GITLAWB_KEY or --dir",
));
}
None
}
};
(dir, pem_path, checks)
}
pub async fn run(args: DoctorArgs) -> Result<()> {
println!("gl doctor — checking your Twigpine setup");
println!();
// The same resolver every other command uses. `doctor` reporting on
// `~/.gitlawb` while `gl register` writes to the parent of `GITLAWB_KEY` would
// make the one command whose job is to explain a broken setup the one that
// misreports it.
let args_dir = args.dir.clone();
let (dir, pem_path, resolver_rows) = resolve_config_paths(args_dir.clone());
let mut checks = resolver_rows;
let dir = args.dir.unwrap_or_else(|| {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".gitlawb")
});
let mut checks = Vec::new();
let mut all_ok = true;
// ── 1. Identity ───────────────────────────────────────────────────────
if pem_path.as_ref().is_some_and(|p| p.exists()) {
let pem_path = pem_path.as_ref().unwrap();
match std::fs::read_to_string(pem_path)
let pem_path = dir.join("identity.pem");
if pem_path.exists() {
match std::fs::read_to_string(&pem_path)
.ok()
.and_then(|pem| gitlawb_core::identity::Keypair::from_pem(&pem).ok())
{
@@ -142,7 +102,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
));
}
}
} else if let Some(pem_path) = &pem_path {
} else {
checks.push(Check::fail(
"identity",
format!("not found at {}", pem_path.display()),
@@ -151,48 +111,62 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
// ── 2. Registration ───────────────────────────────────────────────────
if let Some(dir) = &dir {
let ucan_path = dir.join("ucan.json");
if ucan_path.exists() {
match std::fs::read_to_string(&ucan_path)
.ok()
.and_then(|s| serde_json::from_str::<serde_json::Value>(&s).ok())
{
Some(v) => {
let node = v["node"].as_str().unwrap_or("unknown");
checks.push(Check::pass(
"registration",
format!("registered with {node}"),
));
}
None => {
checks.push(Check::fail(
"registration",
"ucan.json exists but is malformed",
"gl register",
));
}
let ucan_path = dir.join("ucan.json");
if ucan_path.exists() {
match std::fs::read_to_string(&ucan_path)
.ok()
.and_then(|s| serde_json::from_str::<serde_json::Value>(&s).ok())
{
Some(v) => {
let node = v["node"].as_str().unwrap_or("unknown");
checks.push(Check::pass(
"registration",
format!("registered with {node}"),
));
}
} else {
None => {
checks.push(Check::fail(
"registration",
"ucan.json exists but is malformed",
"gl register",
));
}
}
} else {
checks.push(Check::fail(
"registration",
"not registered with any node",
"gl register",
));
}
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
match std::env::var("GITLAWB_NODE") {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Ok(v) if is_loopback_url(&v) => {
checks.push(Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
));
}
Ok(v) if !v.is_empty() => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
_ => {
checks.push(Check::fail(
"registration",
"not registered with any node",
"gl register",
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
"export GITLAWB_NODE=https://node.twigpine.com",
));
}
}
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
checks.push(gitlawb_node_env_check(
std::env::var("GITLAWB_NODE").ok().as_deref(),
));
// ── 4. Node connectivity ──────────────────────────────────────────────
// GITLAWB_NODE="" binds args.node to "" through clap, and check 3 groups
// empty with unset. The probe must apply the same normalization or it
// would dial "" and Fail while the env row only Warns.
let node_url = effective_node_url(&args.node);
let client = NodeClient::new(node_url, None);
let client = NodeClient::new(&args.node, None);
match client.get("/").await {
Ok(resp) if resp.status().is_success() => {
let info = resp.json::<serde_json::Value>().await.unwrap_or_default();
@@ -201,7 +175,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
let short_did = did.chars().take(30).collect::<String>();
checks.push(Check::pass(
"node",
format!("{} — v{version} ({short_did}…)", node_url),
format!("{} — v{version} ({short_did}…)", args.node),
));
// Capability drift: a node newer than this CLI may require features
// (RFC 9421 signing, iCaptcha) the CLI doesn't speak.
@@ -220,14 +194,14 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
Ok(resp) => {
checks.push(Check::fail(
"node",
format!("{} returned HTTP {}", node_url, resp.status()),
format!("{} returned HTTP {}", args.node, resp.status()),
"check GITLAWB_NODE env var or try gl register --node <url>",
));
}
Err(e) => {
checks.push(Check::fail(
"node",
format!("{} unreachable: {e}", node_url),
format!("{} unreachable: {e}", args.node),
"check your internet connection or set GITLAWB_NODE",
));
}
@@ -311,21 +285,23 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
CheckState::Fail => "✗",
};
println!(" {icon} {:<24} {}", check.label, check.detail);
if matches!(check.state, CheckState::Fail) {
all_ok = false;
}
}
println!();
let has_failures_now = has_failures(&checks);
let has_issues = checks
.iter()
.any(|c| matches!(c.state, CheckState::Fail | CheckState::Warn));
if !has_issues {
println!("Everything looks good. Run `gl quickstart` to create your first repo.");
} else {
if has_failures_now {
println!("Some checks failed. Suggested fixes:");
} else {
if all_ok {
println!("Setup looks good with some warnings:");
} else {
println!("Some checks failed. Suggested fixes:");
}
for check in &checks {
if matches!(check.state, CheckState::Fail | CheckState::Warn) {
@@ -335,75 +311,14 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
}
println!();
if has_failures_now {
if !all_ok {
println!("Run `gl quickstart` for a guided setup.");
}
}
// #357: a non-zero exit is the only signal scripting and CI have. Warn-only
// rows stay exit 0 (iCaptcha offline, version drift, shell-alias shadowing
// are all non-fatal); Fail-class rows (identity, registration, node
// reachable, git-remote-gitlawb present, git present) flip to exit 1.
// Direct `process::exit` rather than returning Err: the prose above is the
// user-facing summary, and an anyhow Error frame would just duplicate it.
if has_failures(&checks) {
std::process::exit(1);
}
Ok(())
}
/// Classify the `GITLAWB_NODE` env var for `gl doctor`. Extracted so unit
/// tests can pin the Warn tier directly (#357): an unset env is advisory, not
/// a failure, because `--node` defaults to `https://node.twigpine.com` and the
/// CLI works without it. Flipping this back to `Check::fail` must break a
/// test, not just change prose.
fn gitlawb_node_env_check(env_val: Option<&str>) -> Check {
match env_val {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Some(v) if is_loopback_url(v) => Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
),
Some(v) if !v.is_empty() => Check::pass("GITLAWB_NODE", v.to_string()),
// `--node` defaults to `https://node.twigpine.com` and the CLI works
// fine without the variable, so an unset env is advisory, not a
// failure. Warn-class keeps the exit code 0 for a stock working
// install; this used to be Fail, which the obvious "return non-zero
// on any failure" change would have flipped to exit 1 on every
// default-config install (#357).
_ => Check::warn(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
format!("export GITLAWB_NODE={PUBLIC_NODE}"),
),
}
}
/// The URL the connectivity probe should dial. `GITLAWB_NODE` set but empty
/// (or whitespace-only) binds `args.node` to `""` through clap rather than
/// to the `default_value`, and check 3 already groups empty with unset, so
/// the probe applies the same normalization: an empty effective URL means
/// the public node, same as unset.
fn effective_node_url(node: &str) -> &str {
if node.trim().is_empty() {
PUBLIC_NODE
} else {
node
}
}
/// True when at least one check is Fail-class. Warn-only runs (iCaptcha offline,
/// version drift, shell-alias shadowing, GITLAWB_NODE unset) keep exit 0;
/// this is the gating predicate for #357's exit-status fix.
fn has_failures(checks: &[Check]) -> bool {
checks.iter().any(|c| matches!(c.state, CheckState::Fail))
}
/// Check if a binary name exists anywhere on PATH.
/// True when the rc file contains a real `unalias` command naming `gl` —
/// not a comment, and not a longer word like `unalias global`. Ordering
@@ -648,24 +563,6 @@ mod tests {
assert!(check_shell_alias_shadowing(None).is_none());
}
#[test]
fn refused_resolver_becomes_check_rows_not_abort() {
// A relative GITLAWB_KEY makes both resolvers Err; doctor must turn
// that into fail rows for the dependent checks and keep going, not
// exit before the node/git checks ever run.
crate::identity::test_env::with_key(Some("relative/path"), || {
let (dir, pem_path, rows) = resolve_config_paths(None);
assert!(dir.is_none() && pem_path.is_none());
let fails: Vec<_> = rows
.iter()
.filter(|c| matches!(c.state, CheckState::Fail))
.map(|c| c.label)
.collect();
assert!(fails.contains(&"identity"), "rows: {fails:?}");
assert!(fails.contains(&"registration"), "rows: {fails:?}");
});
}
#[test]
fn test_is_newer_minor_bump() {
assert!(is_newer("0.2.0", "0.1.0"));
@@ -747,103 +644,4 @@ mod tests {
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("GitHub API returned HTTP 403"));
}
/// #357: an all-Ok run does not flip the exit code to non-zero — a healthy
/// install must keep returning 0 so scripts like `gl doctor && gl push`
/// keep working.
#[test]
fn exit_predicate_is_false_for_all_ok() {
let checks = vec![
Check::pass("identity", "ok"),
Check::pass("registration", "ok"),
Check::pass("git", "ok"),
];
assert!(!has_failures(&checks));
}
/// #357: Warn-class rows (iCaptcha offline, version drift, shell-alias
/// shadowing, GITLAWB_NODE unset on a default-config install) are NOT
/// exit-status failures — they are still printed to the user but the
/// process exits 0. This is the rule that the previous "any
/// Fail-or-Warn row → exit 1" change would have broken for stock
/// installs that simply have not set the env var.
#[test]
fn exit_predicate_is_false_for_warn_only() {
let checks = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export GITLAWB_NODE=..."),
Check::warn("iCaptcha", "offline", "check connectivity"),
Check::warn("version", "drift", "upgrade"),
Check::warn("shell alias", "omz git plugin", "unalias gl"),
];
assert!(!has_failures(&checks));
}
/// #357: a single Fail-class row trips the exit code. The set of Fail
/// rows is whatever `run` writes through `Check::fail`: identity missing
/// or unparseable, registration missing/malformed, node unreachable,
/// git-remote-gitlawb absent, git absent.
#[test]
fn exit_predicate_is_true_when_a_fail_row_is_present() {
let checks = vec![
Check::pass("registration", "ok"),
Check::fail("identity", "missing", "gl identity new"),
Check::pass("git", "ok"),
];
assert!(has_failures(&checks));
let mixed_warns = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export"),
Check::fail("node", "unreachable", "check network"),
Check::warn("iCaptcha", "offline", "check"),
];
assert!(has_failures(&mixed_warns));
}
/// #357: `GITLAWB_NODE` unset must be Warn, not Fail. This pins the
/// re-tier so flipping it back to `Check::fail` breaks the suite — the
/// previous `has_failures`-only tests built their own `Check::warn` rows
/// and could not see the re-tier at all.
#[test]
fn gitlawb_node_env_unset_is_warn() {
let check = gitlawb_node_env_check(None);
assert!(matches!(check.state, CheckState::Warn));
assert!(check.detail.contains("not set"));
let empty = gitlawb_node_env_check(Some(""));
assert!(matches!(empty.state, CheckState::Warn));
}
/// `GITLAWB_NODE=""` binds `args.node` to `""` through clap, and check 3
/// groups empty with unset. The probe must dial the same URL unset
/// would — removing the normalization makes check 4 probe `""` and Fail
/// while the env row only Warns, so an empty env install exits 1.
#[test]
fn effective_node_url_treats_empty_and_blank_as_unset() {
assert_eq!(effective_node_url(""), PUBLIC_NODE);
assert_eq!(effective_node_url(" "), PUBLIC_NODE);
assert_eq!(effective_node_url("\t\n"), PUBLIC_NODE);
assert_eq!(
effective_node_url("http://127.0.0.1:7545"),
"http://127.0.0.1:7545"
);
}
#[test]
fn gitlawb_node_env_set_is_pass() {
let check = gitlawb_node_env_check(Some("https://node.twigpine.com"));
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("https://node.twigpine.com"));
}
#[test]
fn gitlawb_node_env_loopback_is_pass() {
for url in ["http://127.0.0.1:7545", "http://localhost:7545"] {
let check = gitlawb_node_env_check(Some(url));
assert!(
matches!(check.state, CheckState::Ok),
"loopback {url} must be Ok, not Warn/Fail"
);
}
}
}
+3 -271
View File
@@ -10,7 +10,6 @@ use anyhow::{Context, Result};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use icaptcha_client::IcaptchaCfg;
use serde_json::Value;
/// Max times we'll fetch a fresh proof and retry a 403-iCaptcha response
/// (absorbs proof expiry / first-seen replay).
@@ -320,70 +319,16 @@ pub(crate) async fn read_body_capped(mut resp: reqwest::Response, cap: usize) ->
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim. We drop the C0/C1 control bytes (which
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
pub(crate) fn sanitize_node_msg(s: &str) -> String {
gitlawb_core::sanitize::strip_terminal_controls(s)
.chars()
s.chars()
.filter(|c| !c.is_control() && !gitlawb_core::sanitize::is_bidi_format(*c))
.take(200)
.collect()
}
/// Read a JSON response, surfacing a node denial/error instead of parsing it as
/// the requested resource. On a non-2xx status it returns an `Err` carrying the
/// node's sanitized `message` plus the status; on success it parses the
/// body and propagates a parse error, so a truncated/garbage 2xx body is an error
/// rather than a silently-empty success (the denial-as-success bug #123 fixes).
/// `what` names the resource for the error text (e.g. "repo", "commits").
///
/// Callers must route gated reads through this rather than `resp.json().await?`:
/// the bare parse renders a gated 404/5xx body back as the resource.
/// Cap on the error body `read_json` reads before parsing a node's `message`.
/// Matches the capped-read bound used on the sync error path; large enough for any
/// legitimate error, small enough that a hostile node cannot exhaust memory.
const ERROR_BODY_CAP: usize = 8 * 1024;
pub(crate) async fn read_json(resp: reqwest::Response, what: &str) -> Result<Value> {
let status = resp.status();
if !status.is_success() {
// Bound the error read: `resp.json()` would buffer and parse the WHOLE
// body, so a hostile node could stream an arbitrarily large valid JSON
// error and exhaust this process's memory (#186). Read a small capped body
// first, then best-effort extract `message`. The error body may also be
// non-JSON (503 degraded, 413 body-limit from middleware), so a parse miss
// falls back to the capped raw text; an empty body to the status alone.
let raw = read_body_capped(resp, ERROR_BODY_CAP).await.text;
// Preserve the prior contract: surface the JSON `message` when present in
// the capped body, else the status alone ("request failed") — a non-JSON
// or message-less body (503 degraded, 413 body-limit) never surfaces raw
// node bytes. A body whose `message` sits past the cap parses as truncated
// garbage here and correctly falls back rather than being buffered whole.
//
// Fall back to `error` before giving up: the node's shared AppError
// envelope carries both keys, but the task API answers with `error` alone
// (`{"error":"task not found"}` in gitlawb-node/src/api/tasks.rs), so
// reading only `message` blanked every `gl task` denial to the generic
// "request failed". `sync.rs`'s hand-rolled reader already reads both; this
// matches it.
let msg = serde_json::from_str::<Value>(&raw)
.ok()
.and_then(|b| {
b.get("message")
.or_else(|| b.get("error"))
.and_then(|m| m.as_str())
.map(str::to_owned)
})
.unwrap_or_else(|| "request failed".to_owned());
anyhow::bail!("{what} failed ({status}): {}", sanitize_node_msg(&msg));
}
// Success: a truncated/garbage 2xx body must be an `Err`, not `Ok(Null)` that
// a caller then renders as an empty success.
resp.json()
.await
.with_context(|| format!("invalid JSON in {what} response"))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -1323,217 +1268,4 @@ mod tests {
let out = sanitize_node_msg("ok \u{0627}\u{200D}b");
assert_eq!(out, "ok \u{0627}\u{200D}b");
}
// ── read_json (status-checked, bounded, sanitized read; #123) ────────
/// Drive a real `reqwest::Response` off a mockito mock so `read_json` sees an
/// actual HTTP status + body, the same shape the gated read arms produce.
async fn response_for(
server: &mut Server,
status: usize,
body: &str,
json: bool,
) -> reqwest::Response {
let mut m = server.mock("GET", "/x").with_status(status).with_body(body);
if json {
m = m.with_header("content-type", "application/json");
}
let _m = m.create_async().await;
NodeClient::new(server.url(), None).get("/x").await.unwrap()
}
#[tokio::test]
async fn read_json_returns_body_on_2xx() {
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
200,
r#"{"name":"r","owner_did":"did:gitlawb:z"}"#,
true,
)
.await;
let v = read_json(resp, "repo").await.unwrap();
assert_eq!(v["name"], "r");
}
#[tokio::test]
async fn read_json_errs_on_404_surfacing_message_and_status() {
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
404,
r#"{"message":"repository 'o/r' not found"}"#,
true,
)
.await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("404"), "err={err}");
assert!(err.contains("not found"), "err={err}");
}
#[tokio::test]
async fn read_json_errs_on_500_surfacing_message() {
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 500, r#"{"message":"internal boom"}"#, true).await;
let err = read_json(resp, "commits").await.unwrap_err().to_string();
assert!(err.contains("500"), "err={err}");
assert!(err.contains("internal boom"), "err={err}");
}
#[tokio::test]
async fn read_json_errs_on_non_json_error_body_with_fallback() {
// 503 with a plain-text (middleware) body: no `message` field to surface.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 503, "service unavailable", false).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("503"), "err={err}");
assert!(err.contains("request failed"), "err={err}");
}
#[tokio::test]
async fn read_json_sanitizes_control_and_bidi_in_message() {
// A hostile node embeds ESC, BEL, and a right-to-left override in
// the error message; none may reach the terminal verbatim.
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
404,
r#"{"message":"a\u001b[31mb\u0007c\u202ed"}"#,
true,
)
.await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(!err.contains('\u{1b}'), "ESC leaked: {err:?}");
assert!(!err.contains('\u{7}'), "BEL leaked: {err:?}");
assert!(!err.contains('\u{202e}'), "bidi override leaked: {err:?}");
}
#[tokio::test]
async fn read_json_errs_on_garbage_2xx_body() {
// The #123 correctness point: a 200 with a non-JSON/truncated body must be
// an `Err`, NOT `Ok(Null)` that a caller renders as an empty success.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 200, "this is not json", false).await;
assert!(read_json(resp, "repo").await.is_err());
}
#[tokio::test]
async fn read_json_errs_on_empty_2xx_body() {
// A zero-byte 200 body must be an `Err`, not a silent `Ok(Null)`.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 200, "", false).await;
assert!(read_json(resp, "repo").await.is_err());
}
#[tokio::test]
async fn read_json_errs_on_non_2xx_json_without_message_or_error() {
// A non-2xx JSON body carrying NEITHER key is the real fallback case.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 403, r#"{"detail":"forbidden"}"#, true).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("403"), "err={err}");
assert!(err.contains("request failed"), "err={err}");
// The body's own text must not leak when no recognised key carries it.
assert!(!err.contains("forbidden"), "raw body leaked: {err}");
}
#[tokio::test]
async fn read_json_surfaces_error_key_when_message_absent() {
// The task API answers with `error` alone — see
// gitlawb-node/src/api/tasks.rs `{"error":"task not found"}`. Reading only
// `message` blanked those denials to the generic "request failed", so the
// user was told nothing about why the call failed.
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 404, r#"{"error":"task not found"}"#, true).await;
let err = read_json(resp, "task show").await.unwrap_err().to_string();
assert!(err.contains("404"), "err={err}");
assert!(err.contains("task not found"), "reason not surfaced: {err}");
}
#[tokio::test]
async fn read_json_prefers_message_over_error_key() {
// The shared AppError envelope carries both; `message` is the human text
// and must win over the machine code in `error`.
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
500,
r#"{"error":"db_error","message":"database unavailable"}"#,
true,
)
.await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("database unavailable"), "err={err}");
assert!(
!err.contains("db_error"),
"machine code won over message: {err}"
);
}
#[tokio::test]
async fn read_json_sanitizes_error_key_text() {
// The `error` fallback goes through the same sanitizer as `message`; a
// hostile node must not reach the terminal through the new path. The
// controls are JSON \u escapes so the body stays VALID JSON — embedding
// them raw would fail the parse and pass this test vacuously.
let mut server = Server::new_async().await;
let resp = response_for(
&mut server,
404,
r#"{"error":"a\u001b[31mb\u0007c\u202ed"}"#,
true,
)
.await;
let err = read_json(resp, "task show").await.unwrap_err().to_string();
// Proves the parse succeeded and the `error` key was read, not the
// "request failed" fallback (which would make the asserts below vacuous).
assert!(err.contains("task show failed (404"), "err={err}");
assert!(
!err.contains("request failed"),
"fell back, sanitizer untested: {err}"
);
assert!(
err.contains('a') && err.contains('d'),
"text dropped: {err}"
);
assert!(!err.contains('\u{1b}'), "ESC leaked: {err:?}");
assert!(!err.contains('\u{7}'), "BEL leaked: {err:?}");
assert!(!err.contains('\u{202e}'), "bidi override leaked: {err:?}");
}
/// #186 (F2): the non-2xx error path must read a CAPPED body, not buffer and
/// parse the whole thing — a hostile node can stream an arbitrarily large valid
/// JSON error. Place the `message` field AFTER more than the cap of padding: the
/// bounded read stops before it, so it must be ABSENT from the surfaced error.
/// RED before the fix (`resp.json()` parses the full body → the far message
/// appears), GREEN after (capped read truncates → fallback → far message gone).
/// The status is still surfaced, so a legitimate small error is unaffected.
#[tokio::test]
async fn read_json_bounds_the_error_body_read() {
let mut server = Server::new_async().await;
// >8 KiB of padding, then the marker message at the very end of the body.
let big = format!(
r#"{{"padding":"{}","message":"FARMARKER_BEYOND_CAP"}}"#,
"A".repeat(16 * 1024)
);
let resp = response_for(&mut server, 500, &big, true).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("500"), "bounded read still errors: {err}");
assert!(
!err.contains("FARMARKER_BEYOND_CAP"),
"the error body read must be capped: a message beyond the cap must not be surfaced, err={err}"
);
}
/// #186 (F2): an EMPTY non-2xx body is its own input class — the capped read
/// yields "", the JSON parse fails, and it falls back to the status alone
/// ("request failed"), never a panic or a spurious success.
#[tokio::test]
async fn read_json_errs_on_empty_non_2xx_body() {
let mut server = Server::new_async().await;
let resp = response_for(&mut server, 502, "", false).await;
let err = read_json(resp, "repo").await.unwrap_err().to_string();
assert!(err.contains("502"), "err={err}");
assert!(err.contains("request failed"), "err={err}");
}
}
+53 -308
View File
@@ -9,8 +9,7 @@ use std::path::{Path, PathBuf};
pub enum IdentityCmd {
/// Generate a new Ed25519 keypair and DID
New {
/// Output directory for key files
/// (default: the parent of $GITLAWB_KEY, else ~/.gitlawb)
/// Output directory for key files (default: ~/.gitlawb)
#[arg(long)]
dir: Option<PathBuf>,
/// Overwrite existing keys if present
@@ -64,51 +63,16 @@ pub async fn run(cmd: IdentityCmd) -> Result<()> {
}
}
/// Resolve the identity directory, honouring an explicit override.
/// Public so sibling commands (`gl ucan import`, `gl doctor`) look in the same
/// place the identity itself lives.
///
/// Without an override this is [`gitlawb_core::identity_path::identity_dir`] — the
/// parent of `GITLAWB_KEY`, else `~/.gitlawb`. The rules live in `gitlawb-core`
/// because `git-remote-gitlawb` needs the identical answer: an operator who moved
/// their key (`GITLAWB_KEY=/data/keys/identity.pem`, the shape `.env.example`
/// documents) would otherwise have `gl ucan import` write the delegation to
/// `~/.gitlawb/delegations` while the helper reads `/data/keys/delegations` and
/// finds it empty. The push then goes out with no `X-Ucan` and the delegate is
/// refused, with nothing on either side to indicate why.
pub fn gitlawb_dir(override_dir: Option<PathBuf>) -> Result<PathBuf> {
fn gitlawb_dir(override_dir: Option<PathBuf>) -> Result<PathBuf> {
if let Some(d) = override_dir {
return Ok(d);
}
// `home_dir()` is passed as an Option rather than demanded here: an absolute
// GITLAWB_KEY resolves without a home directory, and a host that has none is a
// normal container shape, not a reason to refuse a correctly-configured key.
let home = dirs::home_dir();
gitlawb_core::identity_path::identity_dir(home.as_deref()).map_err(|e| anyhow::anyhow!("{e}"))
let home = dirs::home_dir().context("could not determine home directory")?;
Ok(home.join(".gitlawb"))
}
/// The identity PEM to read or write.
///
/// With an explicit `--dir` this is `<dir>/identity.pem`, the conventional layout.
/// Without one it is [`gitlawb_core::identity_path::identity_key_path`] — the whole
/// of `GITLAWB_KEY`, basename included.
///
/// Taking only the parent and re-appending `identity.pem` was a real divergence,
/// not a tidy-up: `GITLAWB_KEY` is documented as a path to a PEM, and
/// `git-remote-gitlawb` opens exactly that path. With
/// `GITLAWB_KEY=/data/keys/ci-agent.pem`, `gl identity new` wrote
/// `/data/keys/identity.pem` while every push loaded `/data/keys/ci-agent.pem`, so
/// the two either disagreed on identity or the helper found no key at all — and
/// owner enforcement and the delegation proof both key off that identity.
pub(crate) fn key_path_for(dir: Option<&Path>) -> Result<PathBuf> {
match dir {
Some(d) => Ok(d.join(gitlawb_core::identity_path::KEY_FILE_NAME)),
None => {
let home = dirs::home_dir();
gitlawb_core::identity_path::identity_key_path(home.as_deref())
.map_err(|e| anyhow::anyhow!("{e}"))
}
}
fn key_path(dir: &Path) -> PathBuf {
dir.join("identity.pem")
}
fn load_keypair(dir: Option<PathBuf>) -> Result<Keypair> {
@@ -117,14 +81,15 @@ fn load_keypair(dir: Option<PathBuf>) -> Result<Keypair> {
/// Load keypair from an optional directory override.
/// Used by other modules (register, repo, mcp).
///
/// Routed through [`gitlawb_dir`] rather than reaching for `~/.gitlawb` directly:
/// `gl identity new` writes the key wherever `GITLAWB_KEY` points, so a second
/// resolver here would have every other command read a different file than the one
/// just created — `gl ucan delegate` would either fail to find an identity or sign
/// with a stale DID that is not the repo owner.
pub fn load_keypair_from_dir(dir: Option<&std::path::Path>) -> Result<Keypair> {
let path = key_path_for(dir)?;
let base = if let Some(d) = dir {
d.to_path_buf()
} else {
dirs::home_dir()
.context("could not determine home directory")?
.join(".gitlawb")
};
let path = key_path(&base);
let pem = fs::read_to_string(&path).with_context(|| {
format!(
"no identity found at {}\nRun `gl identity new` to create one",
@@ -134,36 +99,6 @@ pub fn load_keypair_from_dir(dir: Option<&std::path::Path>) -> Result<Keypair> {
Keypair::from_pem(&pem).context("failed to load keypair from PEM")
}
/// Load keypair from an optional directory override, distinguishing missing default
/// identity (clean anonymous) from an explicit directory error or corrupt PEM.
pub fn load_optional_keypair(dir: Option<&std::path::Path>) -> Result<Option<Keypair>> {
if let Some(d) = dir {
return load_keypair_from_dir(Some(d)).map(Some);
}
// Route the default lookup through key_path_for(None), not a hand-built
// ~/.gitlawb/identity.pem: that path pinned the conventional basename and
// ignored GITLAWB_KEY, so callers on this helper (task list/view, the MCP
// identity fallback) signed as a stale DID or went anonymous while every
// sibling command read the configured key. A host with no home and no
// GITLAWB_KEY still resolves to anonymous rather than refusing.
// An empty GITLAWB_KEY counts as unconfigured, matching identity_key_path:
// set-but-empty should not force an anonymous resolution through a path
// that cannot resolve.
let key_configured = std::env::var_os("GITLAWB_KEY").is_some_and(|v| !v.is_empty());
if !key_configured && dirs::home_dir().is_none() {
return Ok(None);
}
let path = key_path_for(None)?;
match fs::read_to_string(&path) {
Ok(pem) => Keypair::from_pem(&pem)
.context("failed to load keypair from PEM")
.map(Some),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(e) => Err(anyhow::Error::from(e)
.context(format!("failed to read identity from {}", path.display()))),
}
}
async fn cmd_new(dir: Option<PathBuf>, force: bool) -> Result<()> {
cmd_new_with_reader(dir, force, &mut std::io::stdin().lock()).await
}
@@ -173,11 +108,8 @@ async fn cmd_new_with_reader(
force: bool,
reader: &mut impl std::io::BufRead,
) -> Result<()> {
let path = key_path_for(dir.as_deref())?;
let dir = path
.parent()
.map(Path::to_path_buf)
.unwrap_or_else(|| PathBuf::from("."));
let dir = gitlawb_dir(dir)?;
let path = key_path(&dir);
if path.exists() {
if force {
@@ -199,13 +131,23 @@ async fn cmd_new_with_reader(
}
}
crate::secret_file::create_dir(&dir)
fs::create_dir_all(&dir)
.with_context(|| format!("failed to create directory {}", dir.display()))?;
let keypair = Keypair::generate();
let pem = keypair.to_pem()?;
crate::secret_file::write(&path, pem.as_bytes())?;
// Write with restricted permissions
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::write(&path, pem.as_bytes())?;
fs::set_permissions(&path, fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
fs::write(&path, pem.as_bytes())?;
}
let did = keypair.did();
println!("✓ Generated new identity");
@@ -241,7 +183,8 @@ async fn cmd_sign(message: String, dir: Option<PathBuf>) -> Result<()> {
}
async fn cmd_backup(out: Option<PathBuf>, dir: Option<PathBuf>) -> Result<()> {
let src = key_path_for(dir.as_deref())?;
let base = gitlawb_dir(dir)?;
let src = key_path(&base);
let pem = fs::read_to_string(&src).with_context(|| {
format!(
@@ -259,7 +202,16 @@ async fn cmd_backup(out: Option<PathBuf>, dir: Option<PathBuf>) -> Result<()> {
.join("identity.pem.bak")
});
crate::secret_file::write(&dest, pem.as_bytes())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::write(&dest, pem.as_bytes())?;
fs::set_permissions(&dest, fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
fs::write(&dest, pem.as_bytes())?;
}
println!("✓ Identity backed up");
println!(" DID: {}", keypair.did());
@@ -287,11 +239,8 @@ async fn cmd_restore_with_reader(
// Verify it's a valid keypair before writing anything
let keypair = Keypair::from_pem(&pem).context("backup file is not a valid identity PEM")?;
let dest = key_path_for(dir.as_deref())?;
let base = dest
.parent()
.map(Path::to_path_buf)
.unwrap_or_else(|| PathBuf::from("."));
let base = gitlawb_dir(dir)?;
let dest = key_path(&base);
if dest.exists() {
if force {
@@ -313,10 +262,19 @@ async fn cmd_restore_with_reader(
}
}
crate::secret_file::create_dir(&base)
fs::create_dir_all(&base)
.with_context(|| format!("failed to create directory {}", base.display()))?;
crate::secret_file::write(&dest, pem.as_bytes())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::write(&dest, pem.as_bytes())?;
fs::set_permissions(&dest, fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
fs::write(&dest, pem.as_bytes())?;
}
println!("✓ Identity restored");
println!(" DID: {}", keypair.did());
@@ -330,36 +288,6 @@ mod tests {
use super::*;
use tempfile::TempDir;
#[test]
fn load_optional_keypair_honors_gitlawb_key_basename() {
let dir = TempDir::new().unwrap();
let kp = Keypair::generate();
let custom = dir.path().join("ci-agent.pem");
std::fs::write(&custom, kp.to_pem().unwrap()).unwrap();
crate::identity::test_env::with_key(Some(custom.as_os_str()), || {
let loaded = load_optional_keypair(None).unwrap();
assert_eq!(
loaded.map(|k| k.did().to_string()),
Some(kp.did().to_string()),
"the configured basename must be read, not ~/.gitlawb/identity.pem"
);
});
}
#[test]
fn load_optional_keypair_none_when_unset_and_missing() {
let dir = TempDir::new().unwrap();
crate::identity::test_env::with_key(
Some(dir.path().join("absent.pem").as_os_str()),
|| {
assert!(
load_optional_keypair(None).unwrap().is_none(),
"a configured-but-absent key resolves anonymous, not an error"
);
},
);
}
#[tokio::test]
async fn test_cmd_new_creates_pem() {
let dir = TempDir::new().unwrap();
@@ -581,186 +509,3 @@ mod tests {
assert_eq!(original_did, dst_kp.did());
}
}
/// Scoped `GITLAWB_KEY` for tests, shared crate-wide.
///
/// The process environment is global and more than one suite in this crate
/// depends on it — the resolver's own cases here, and `gl register`'s check that
/// the bootstrap token lands beside the key. They take one lock rather than each
/// declaring its own, which would not serialise them against each other.
#[cfg(test)]
pub(crate) mod test_env {
use std::ffi::{OsStr, OsString};
use std::sync::{Mutex, MutexGuard};
static ENV_LOCK: Mutex<()> = Mutex::new(());
/// Restores the previous value and releases the lock on drop.
pub(crate) struct KeyEnv {
_guard: MutexGuard<'static, ()>,
restore: Option<OsString>,
}
impl Drop for KeyEnv {
fn drop(&mut self) {
match self.restore.take() {
Some(v) => std::env::set_var("GITLAWB_KEY", v),
None => std::env::remove_var("GITLAWB_KEY"),
}
}
}
/// Run `f` with `GITLAWB_KEY` set, restoring it afterwards.
pub(crate) fn with_key<T, V: AsRef<OsStr>>(value: Option<V>, f: impl FnOnce() -> T) -> T {
let _guard = set_key(value);
f()
}
/// Set `GITLAWB_KEY` (or remove it, for `None`) until the guard drops.
pub(crate) fn set_key<V: AsRef<OsStr>>(value: Option<V>) -> KeyEnv {
let guard = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let restore = std::env::var_os("GITLAWB_KEY");
match value {
Some(v) => std::env::set_var("GITLAWB_KEY", v),
None => std::env::remove_var("GITLAWB_KEY"),
}
KeyEnv {
_guard: guard,
restore,
}
}
}
#[cfg(test)]
mod gitlawb_dir_tests {
use super::{gitlawb_dir, load_keypair_from_dir};
use std::ffi::OsString;
use std::path::PathBuf;
/// Run `f` with `GITLAWB_KEY` set to `value` (or removed for `None`), restoring
/// whatever was there before.
fn with_key_env<T>(value: Option<OsString>, f: impl FnOnce() -> T) -> T {
let _env = crate::identity::test_env::set_key(value);
f()
}
/// An explicit --dir always wins and is never validated against GITLAWB_KEY.
#[test]
fn explicit_override_wins() {
let d = PathBuf::from("/tmp/explicit");
assert_eq!(gitlawb_dir(Some(d.clone())).unwrap(), d);
}
/// A relative GITLAWB_KEY must fail loudly. `gl` and `git-remote-gitlawb` run
/// from different working directories, so resolving one relatively sends the
/// import and the lookup to different stores; a one-component value yields an
/// empty parent and puts the store in `./delegations`.
#[test]
fn relative_key_paths_are_refused() {
for raw in ["identity.pem", "keys/identity.pem"] {
let result = with_key_env(Some(OsString::from(raw)), || gitlawb_dir(None));
assert!(result.is_err(), "{raw} is relative and must be refused");
}
}
/// An empty value is what a shell leaves behind for `FOO=` and for an unset
/// variable expanded into a wrapper script. Treated as unset, not as an error.
#[test]
fn an_empty_key_path_is_treated_as_unset() {
// A host without a home dir is a legitimate CI shape; the empty-is-unset
// rule resolves to anonymous there, so this assertion has nothing to
// compare against and is a pass, not a panic.
let Some(home) = dirs::home_dir() else { return };
let result = with_key_env(Some(OsString::new()), || gitlawb_dir(None));
assert_eq!(
result.unwrap(),
home.join(".gitlawb"),
"an empty value selects the default directory"
);
}
/// The reason `gitlawb_dir` reads through `var_os`: `env::var` folds a non-UTF-8
/// value into the same `Err` as unset, so a bad path would silently resolve to
/// `~/.gitlawb` instead of being reported. Byte 0xFF is not valid UTF-8 in any
/// position, so this value is unreachable through `env::var`.
#[cfg(unix)]
#[test]
fn a_non_utf8_key_path_is_not_mistaken_for_unset() {
use std::os::unix::ffi::OsStringExt;
let raw = OsString::from_vec(b"keys/\xFF/identity.pem".to_vec());
let result = with_key_env(Some(raw), || gitlawb_dir(None));
let err = result.expect_err("a non-UTF-8 relative path must be refused");
assert!(
err.to_string().contains("absolute"),
"the error must name the real problem, not fall back to the default: {err}"
);
let mut absolute = OsString::from("/data/");
absolute.push(OsString::from_vec(vec![0xFF]));
absolute.push("/identity.pem");
let resolved = with_key_env(Some(absolute.clone()), || gitlawb_dir(None)).unwrap();
assert_eq!(
resolved,
PathBuf::from(&absolute).parent().unwrap(),
"an absolute non-UTF-8 path resolves to its own parent, not to ~/.gitlawb"
);
}
/// `gl identity new` writes the key wherever `GITLAWB_KEY` points, so every
/// other command has to read it back from there. `load_keypair_from_dir(None)`
/// used to hardcode `~/.gitlawb`, which made `gl ucan delegate` sign with a
/// stale DID — or fail outright — for exactly the operators who moved the key.
#[test]
fn load_keypair_from_dir_honours_the_key_env() {
let dir = tempfile::tempdir().unwrap();
let key = dir.path().join("identity.pem");
let expected = gitlawb_core::identity::Keypair::generate();
std::fs::write(&key, expected.to_pem().unwrap()).unwrap();
let loaded = with_key_env(Some(key.into_os_string()), || load_keypair_from_dir(None))
.expect("the identity beside GITLAWB_KEY must be found");
assert_eq!(loaded.did(), expected.did());
}
}
#[cfg(test)]
mod key_basename_tests {
use super::*;
/// `GITLAWB_KEY` names a FILE. `gl` used to keep only its parent and re-append
/// `identity.pem`, while `git-remote-gitlawb` opened the configured path — so
/// with `GITLAWB_KEY=/data/keys/ci-agent.pem` the CLI and the push path loaded
/// different files, and owner enforcement and the delegation proof both key off
/// whichever identity that was.
#[test]
fn a_non_default_key_basename_is_honoured() {
let dir = tempfile::tempdir().unwrap();
let key = dir.path().join("ci-agent.pem");
let expected = gitlawb_core::identity::Keypair::generate();
std::fs::write(&key, expected.to_pem().unwrap()).unwrap();
let (resolved, loaded) = crate::identity::test_env::with_key(Some(key.clone()), || {
(key_path_for(None).unwrap(), load_keypair_from_dir(None))
});
assert_eq!(resolved, key, "the configured basename must survive");
assert_eq!(
loaded.expect("the key at GITLAWB_KEY must load").did(),
expected.did(),
"gl must load the same file the helper opens"
);
}
/// An explicit --dir keeps the conventional layout.
#[test]
fn an_explicit_dir_still_uses_identity_pem() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(
key_path_for(Some(dir.path())).unwrap(),
dir.path().join("identity.pem")
);
}
}
+28 -22
View File
@@ -26,7 +26,7 @@ pub struct InitArgs {
)]
pub node: String,
/// Identity directory (default: the parent of $GITLAWB_KEY, else ~/.gitlawb)
/// Identity directory (default: ~/.gitlawb)
#[arg(long)]
pub dir: Option<PathBuf>,
@@ -105,17 +105,20 @@ pub async fn run(args: InitArgs) -> Result<()> {
// Save UCAN if returned
if let Some(ucan) = payload.get("ucan").and_then(|v| v.as_str()) {
if !ucan.is_empty() {
let ucan_dir = crate::identity::gitlawb_dir(args.dir.clone())?;
crate::secret_file::create_dir(&ucan_dir)?;
let ucan_dir = args
.dir
.clone()
.unwrap_or_else(|| dirs::home_dir().unwrap_or_default().join(".gitlawb"));
std::fs::create_dir_all(&ucan_dir)?;
let record = json!({
"ucan": ucan,
"node": args.node,
"did": did.to_string(),
"saved_at": chrono::Utc::now().to_rfc3339(),
});
crate::secret_file::write(
&ucan_dir.join("ucan.json"),
serde_json::to_string_pretty(&record)?.as_bytes(),
std::fs::write(
ucan_dir.join("ucan.json"),
serde_json::to_string_pretty(&record)?,
)?;
}
}
@@ -222,26 +225,29 @@ pub async fn run(args: InitArgs) -> Result<()> {
}
fn generate_identity(dir: Option<&std::path::Path>) -> Result<gitlawb_core::identity::Keypair> {
let path = crate::identity::key_path_for(dir)?;
let base = path
.parent()
.map(std::path::Path::to_path_buf)
.unwrap_or_else(|| std::path::PathBuf::from("."));
crate::secret_file::create_dir(&base)?;
let base = if let Some(d) = dir {
d.to_path_buf()
} else {
dirs::home_dir()
.context("could not determine home directory")?
.join(".gitlawb")
};
std::fs::create_dir_all(&base)?;
let keypair = gitlawb_core::identity::Keypair::generate();
let pem = keypair.to_pem()?;
let path = base.join("identity.pem");
crate::secret_file::write_new(&path, pem.as_bytes()).map_err(|e| {
if e.kind() == std::io::ErrorKind::AlreadyExists {
anyhow::anyhow!(
"identity key already exists at {} — refusing to overwrite; pass --dir for a fresh location or move the key",
path.display()
)
} else {
anyhow::anyhow!(e)
}
})?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::write(&path, pem.as_bytes())?;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
std::fs::write(&path, pem.as_bytes())?;
}
Ok(keypair)
}
+14 -180
View File
@@ -28,7 +28,7 @@ pub enum IpfsCmd {
env = "GITLAWB_NODE"
)]
node: String,
/// Identity directory (default: the parent of $GITLAWB_KEY, else ~/.gitlawb)
/// Identity directory (default: ~/.gitlawb)
#[arg(long)]
dir: Option<PathBuf>,
},
@@ -79,7 +79,7 @@ pub enum IpfsCmd {
env = "GITLAWB_NODE"
)]
node: String,
/// Identity directory (default: the parent of $GITLAWB_KEY, else ~/.gitlawb)
/// Identity directory (default: ~/.gitlawb)
#[arg(long)]
dir: Option<PathBuf>,
/// Resume token from a scan that stopped at a bound, as printed by a
@@ -127,93 +127,23 @@ async fn cmd_list(node: String, dir: Option<PathBuf>) -> Result<()> {
println!("IPFS pins ({count}) on {node}");
println!();
for pin in &pins {
for line in render_pin(pin) {
println!("{line}");
}
let cid = pin["cid"].as_str().unwrap_or("?");
let sha = pin["sha256_hex"].as_str().unwrap_or("?");
let pinned_at = pin["pinned_at"].as_str().unwrap_or("?");
// Trim pinned_at to date+time without subseconds
let ts = if pinned_at.len() >= 19 {
&pinned_at[..19]
} else {
pinned_at
};
println!(" {cid}");
println!(" sha256: {sha}");
println!(" pinned: {ts}");
println!();
}
Ok(())
}
/// The rendered lines for one pins-listing entry.
///
/// #218 review round 8 P2 — why this is not just `pin["cid"]`: the wire format
/// gained a provenance split, and `cid` is now NULLABLE. The node stopped
/// aliasing a Pinata provider CID into `cid` (round 3) because `cid` is the
/// key `GET /ipfs/{cid}` resolves against, and a Pinata provider CID is not
/// resolvable there — advertising one would send clients to a guaranteed 404.
/// The truthful representation of a Pinata-only row is therefore `cid: null`
/// plus a `pinata_cid`. This function was reading only `cid`, so such a row
/// printed as a bare `?` — the CLI showed nothing usable for an object that is
/// in fact durably stored, and nothing in the client read `pinata_cid` at all.
///
/// So: the heading is the node-resolvable CID when there is one, and otherwise
/// the Pinata provider CID, labelled as such so nobody feeds it back to this
/// node's resolver. A `backends:` line names where the bytes actually are,
/// taken from the writer-owned `local_pinned` / `pinata_pinned` booleans rather
/// than re-inferred from CID shape (which is what the node's own comment warns
/// against). Older nodes send neither flag; their `cid`-only rows fall back to
/// CID presence and render exactly as they did before.
fn render_pin(pin: &Value) -> Vec<String> {
let sha = pin["sha256_hex"].as_str().unwrap_or("?");
let pinned_at = pin["pinned_at"].as_str().unwrap_or("?");
// Trim pinned_at to date+time without subseconds; truncate() is
// boundary-safe (a raw &s[..19] panics on a split code point).
let ts = if pinned_at.len() >= 19 {
crate::text::truncate(pinned_at, 19)
} else {
pinned_at
};
// `cid` and `local_cid` carry the same raw CID; read both so a node that
// ships only one of them still renders.
let local_cid = pin["cid"]
.as_str()
.or_else(|| pin["local_cid"].as_str())
.filter(|s| !s.is_empty());
let pinata_cid = pin["pinata_cid"].as_str().filter(|s| !s.is_empty());
// Writer-owned flags when present; CID presence is the pre-split fallback.
let local_pinned = pin["local_pinned"]
.as_bool()
.unwrap_or_else(|| local_cid.is_some());
let pinata_pinned = pin["pinata_pinned"]
.as_bool()
.unwrap_or_else(|| pinata_cid.is_some());
let mut lines = Vec::new();
match (local_cid, pinata_cid) {
// Node-resolvable CID present: it leads, as it always has.
(Some(cid), _) => lines.push(format!(" {cid}")),
// Pinata-only: show the provider CID rather than "?", and say plainly
// that this node's resolver will not serve it.
(None, Some(p)) => lines.push(format!(" {p} (pinata provider CID)")),
(None, None) => lines.push(" ?".to_string()),
}
lines.push(format!(" sha256: {sha}"));
lines.push(format!(" pinned: {ts}"));
// A dual row keeps the provider CID visible too; it differs from the local
// CID and is the only key that resolves on Pinata's gateway.
if local_cid.is_some() {
if let Some(p) = pinata_cid {
lines.push(format!(" pinata: {p}"));
}
}
let backends = match (local_pinned, pinata_pinned) {
(true, true) => "local ipfs, pinata",
(true, false) => "local ipfs",
(false, true) => "pinata",
// Neither flag set: the node filters such rows out, so this is only
// reachable from a hand-rolled response. Say so rather than imply
// durability the row does not claim.
(false, false) => "none recorded",
};
lines.push(format!(" backends: {backends}"));
lines
}
/// Automatic resumes attempted after the initial request when the node reports a
/// truncated legacy scan, so at most `MAX_SCAN_RESUMES + 1` node calls per invocation.
const MAX_SCAN_RESUMES: usize = 8;
@@ -707,102 +637,6 @@ mod tests {
dir
}
/// #218 review round 8 P2: a Pinata-only row must render its provider CID,
/// not a bare `?`. The node stopped aliasing that CID into `cid` because
/// `cid` is the key `GET /ipfs/{cid}` resolves on and a provider CID 404s
/// there — so `cid` is now legitimately null for such a row, and a client
/// that reads only `cid` shows the user nothing for an object that IS
/// durably stored. The heading must carry the provider CID and label it,
/// and the row must not claim a local pin.
#[test]
fn render_pin_shows_provider_cid_for_a_pinata_only_row() {
let pin: Value = serde_json::from_str(
r#"{"sha256_hex":"abc123","cid":null,"local_cid":null,
"pinata_cid":"QmProviderOnly","local_pinned":false,
"pinata_pinned":true,"pinned_at":"2026-07-02T12:00:00.123456Z"}"#,
)
.unwrap();
let out = render_pin(&pin).join("\n");
assert!(
out.contains("QmProviderOnly"),
"the provider CID must be shown; got:\n{out}"
);
assert!(
!out.contains(" ?"),
"a Pinata-only row must not render as a bare `?`; got:\n{out}"
);
assert!(
out.contains("backends: pinata"),
"the row must say where the bytes are; got:\n{out}"
);
assert!(
out.contains("pinata provider CID"),
"the heading must be labelled so it is not fed back to this node's \
resolver, which cannot serve it; got:\n{out}"
);
}
/// A dual row leads with the node-resolvable CID (unchanged behavior) and
/// additionally surfaces the provider CID, which is a different key and the
/// only one Pinata's gateway answers for.
#[test]
fn render_pin_shows_both_cids_for_a_dual_row() {
let pin: Value = serde_json::from_str(
r#"{"sha256_hex":"abc123","cid":"bafyLocal","local_cid":"bafyLocal",
"pinata_cid":"QmProvider","local_pinned":true,
"pinata_pinned":true,"pinned_at":"2026-07-02T12:00:00.123456Z"}"#,
)
.unwrap();
let out = render_pin(&pin).join("\n");
assert!(
out.starts_with(" bafyLocal"),
"the resolver key leads; got:\n{out}"
);
assert!(
out.contains("pinata: QmProvider"),
"the provider CID must still be reachable from the listing; got:\n{out}"
);
assert!(out.contains("backends: local ipfs, pinata"), "got:\n{out}");
}
/// Backward compatibility: a pre-provenance node sends `cid` alone, with no
/// `local_pinned` / `pinata_pinned` flags. That row must render as it always
/// did, with the backends line inferred from CID presence.
#[test]
fn render_pin_handles_a_legacy_cid_only_row() {
let pin: Value = serde_json::from_str(
r#"{"sha256_hex":"abc123","cid":"bafyone",
"pinned_at":"2026-07-02T12:00:00.123456Z"}"#,
)
.unwrap();
let out = render_pin(&pin).join("\n");
assert!(out.starts_with(" bafyone"), "got:\n{out}");
assert!(out.contains("sha256: abc123"), "got:\n{out}");
assert!(
out.contains("pinned: 2026-07-02T12:00:00"),
"the timestamp is still trimmed to seconds; got:\n{out}"
);
assert!(out.contains("backends: local ipfs"), "got:\n{out}");
}
/// A node-supplied timestamp whose 19th byte splits a multi-byte code
/// point must not panic the CLI — `&s[..19]` panics there; `truncate`
/// returns the boundary-safe prefix.
#[test]
fn render_pin_does_not_panic_on_a_non_boundary_timestamp() {
let mut ts = "2026-07-02T12:00:0".to_string();
ts.push('é'); // 2-byte char starting at byte 18: byte 19 is its tail
assert_eq!(ts.len(), 20);
assert!(std::panic::catch_unwind(|| &ts[..19]).is_err());
let pin: Value = serde_json::json!({
"sha256_hex":"abc123","cid":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4",
"local_cid":null,"pinata_cid":null,"local_pinned":true,
"pinata_pinned":false,"pinned_at":ts,
});
let out = render_pin(&pin).join("\n");
assert!(out.contains("pinned: 2026-07-02T12:00:0"), "got:\n{out}");
}
#[tokio::test]
async fn test_cmd_list_signs_request_and_renders_pins() {
let mut server = mockito::Server::new_async().await;
+52 -106
View File
@@ -3,7 +3,7 @@
use anyhow::{Context, Result};
use chrono::Utc;
use clap::{Args, Subcommand};
use serde_json::json;
use serde_json::{json, Value};
use std::path::PathBuf;
use uuid::Uuid;
@@ -166,7 +166,12 @@ async fn resolve_repo(
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(node, None);
let info = crate::http::read_json(client.get("/").await?, "node info").await?;
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let did = info["did"].as_str().context("node info missing 'did'")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -217,7 +222,13 @@ async fn cmd_create(
.post(&path, &request_body)
.await
.context("failed to connect to node")?;
let result = crate::http::read_json(resp, "create issue").await?;
let status = resp.status();
let result: Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("create issue failed ({status}): {msg}");
}
let id = result["id"].as_str().unwrap_or("?");
println!("✓ Created issue #{id}");
@@ -234,7 +245,12 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/issues");
let resp = crate::http::read_json(client.get_authed(&path).await?, "issues").await?;
let resp: Value = client
.get_authed(&path)
.await?
.json()
.await
.context("failed to list issues")?;
let issues = resp["issues"].as_array().cloned().unwrap_or_default();
@@ -251,7 +267,7 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let status = issue["status"].as_str().unwrap_or("?");
let created = issue["created_at"]
.as_str()
.map(|s| crate::text::truncate(s, 10))
.map(|s| &s[..10])
.unwrap_or("?");
let icon = match status {
"open" => "○",
@@ -272,7 +288,13 @@ async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>)
.get_authed(&path)
.await
.context("failed to connect to node")?;
let issue = crate::http::read_json(resp, "show").await?;
let status = resp.status();
let issue: Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = issue["message"].as_str().unwrap_or("issue not found");
anyhow::bail!("show failed ({status}): {msg}");
}
let title = issue["title"].as_str().unwrap_or("(no title)");
let status = issue["status"].as_str().unwrap_or("?");
@@ -303,7 +325,13 @@ async fn cmd_close(repo: String, id: String, node: String, dir: Option<PathBuf>)
.post(&format!("{path}/close"), &body)
.await
.context("failed to connect to node")?;
let _ = crate::http::read_json(resp, "close issue").await?;
let status = resp.status();
let result: Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("close issue failed ({status}): {msg}");
}
println!("✗ Closed issue {id}");
Ok(())
@@ -328,7 +356,13 @@ async fn cmd_issue_comment(
)
.await
.context("failed to connect to node")?;
let _ = crate::http::read_json(resp, "comment").await?;
let code = resp.status();
let result: Value = resp.json().await.context("invalid JSON")?;
if !code.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("comment failed ({code}): {msg}");
}
println!("· Comment posted on issue {id}");
Ok(())
@@ -343,15 +377,14 @@ async fn cmd_issue_comments(
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = signed_client(&node, dir.as_deref());
let resp = crate::http::read_json(
client
.get_authed(&format!(
"/api/v1/repos/{owner}/{name}/issues/{id}/comments"
))
.await?,
"issue comments",
)
.await?;
let resp: Value = client
.get_authed(&format!(
"/api/v1/repos/{owner}/{name}/issues/{id}/comments"
))
.await?
.json()
.await
.context("invalid JSON")?;
let comments = resp["comments"].as_array().cloned().unwrap_or_default();
if comments.is_empty() {
@@ -365,13 +398,10 @@ async fn cmd_issue_comments(
let author_short = author
.split(':')
.next_back()
.map(|s| crate::text::truncate(s, 8))
.map(|s| &s[..s.len().min(8)])
.unwrap_or("?");
let cbody = c["body"].as_str().unwrap_or("");
let created = c["created_at"]
.as_str()
.map(|s| crate::text::truncate(s, 10))
.unwrap_or("?");
let created = c["created_at"].as_str().map(|s| &s[..10]).unwrap_or("?");
println!(" · {author_short} ({created})");
println!(" {cbody}");
println!();
@@ -442,34 +472,6 @@ mod tests {
.unwrap();
}
#[tokio::test]
async fn test_cmd_list_survives_malformed_timestamps() {
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
// A short timestamp used to panic on `&s[..10]`; a timestamp whose
// byte 10 is mid-char panics even with a len() guard.
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/issues$".to_string()),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"issues":[{"id":"a","title":"short ts","status":"open","created_at":"2026"},{"id":"b","title":"mb ts","status":"open","created_at":"2026-08-1é5T00:00:00Z"},{"id":"c","title":"empty ts","status":"open","created_at":""}]}"#)
.create_async()
.await;
cmd_list(
"myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_create_success() {
let dir = TempDir::new().unwrap();
@@ -512,7 +514,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.expect(1)
.create_async()
.await;
@@ -527,8 +528,6 @@ mod tests {
assert!(result.is_err());
assert!(result.unwrap_err().to_string().contains("repo not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy is_err() vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -576,7 +575,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"issue not found"}"#)
.expect(1)
.create_async()
.await;
@@ -590,8 +588,6 @@ mod tests {
.unwrap_err();
assert!(err.to_string().contains("show failed"), "got: {err}");
assert!(err.to_string().contains("issue not found"), "got: {err}");
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -640,7 +636,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"issue not found"}"#)
.expect(1)
.create_async()
.await;
@@ -653,8 +648,6 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("close issue failed"), "got: {err}");
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -701,7 +694,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"issue not found"}"#)
.expect(1)
.create_async()
.await;
@@ -715,8 +707,6 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("issue not found"), "got: {err}");
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -774,48 +764,4 @@ mod tests {
.await
.unwrap();
}
#[tokio::test]
async fn cmd_list_surfaces_denial_not_empty() {
// A gated 404 on the issue-list read must Err, not print "No issues".
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/alice/secret/issues$".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list("alice/secret".to_string(), server.url(), None).await;
assert!(result.is_err(), "issue list must Err on a gated 404");
// Prove the gated issues path was actually requested: without this, an
// unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
#[tokio::test]
async fn resolve_repo_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the GET / node-info
// fetch. A gated 404 there must Err (surfacing the status), proving the
// read_json conversion is load-bearing rather than silently ignored.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = resolve_repo("noslash", &server.url(), Some(dir.path()))
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
}
Loaded 100 of 142 files, more files were not shown because too many files have changed in this diff. Show more