Compare commits

..
Author SHA1 Message Date
beardthelion 7e9a5be6d4 test(node): complete listing-denial regressions for the new gates 2026-11-07 15:08:26 -06:00
beardthelion 0140c9cd86 fix(node): pass SignatureBodyLimit state to optional_signature layers 2026-11-07 06:57:14 -06:00
beardthelion 5274433503 Merge remote-tracking branch 'origin/main' into bug_fix_2
# Conflicts:
#	crates/gitlawb-node/src/server.rs
2026-11-07 06:50:06 -06:00
Gravirei ac9ea61927 All three findings addressed on bug_fix_2:
P2 — signed_scoped_anchors_full_did_form_matches_stored_slug_through_build_router now seeds
  a second repo under a different owner and asserts irys-other-repo-tx is absent. Negative
  control: flipping the production filter to None made the test go RED with count: 2 showing
  the cross-repo leak.

  P3 — let short = owner_did.split(':').next_back().unwrap() replaced with let short =
  crate::db::normalize_owner_key(&owner_did).to_string(). Reader and writer now share one
  convention.

  P3 — PR #134 description updated: pins bullet now describes the actual pre-merge layering
  (and cites the axum-layer-vs-merge-pitfall memory entry), and round 2's stored-slug
  normalization is added as its own bullet with the regression's coverage.

  5/5 anchor-related tests pass, cargo fmt --check clean.
2026-08-30 17:13:53 +06:00
Gravirei 4242bb85ad fix(node): normalize scoped anchor SQL filter to stored repo slug
After authorize_repo_read succeeds, the SQL filter used the raw ?repo=
string (format!("{owner}/{name}")). Anchor rows are stored with
normalize_owner_key(owner_did)/name (short key), so a caller using
?repo=did🔑.../name passed authz but got anchors: [] — a false
empty page that broke callers that legitimately pass the full DID
form. Build the filter from record.owner_did normalized and
record.name instead, matching the shape the seeder uses.
2026-08-30 11:24:12 +06:00
Gravirei 8dd745e98b fix(node): keep /api/v1/ipfs/pins out of the CID resolver's per-IP flood brake
Follow-up to #134: the previous reordering put `/api/v1/ipfs/pins` and
`/ipfs/{cid}` under the same `rate_limit_by_ip` + `IpRateLimiter` layer
stack, so both routes shared the resolver's flood brake bucket. That
bucket is documented as the once-per-request brake for `GET /ipfs/{cid}`
because the resolver can drive bounded-but-expensive repo walks. The
pin listing is a single `list_pinned_cids()` call and has no reason to
share the bucket. Sharing meant /ipfs/{cid} traffic could exhaust the
bucket and 429 the pins endpoint, and signed pin polling could exhaust
the bucket and 429 legitimate CID reads.

Compose `ipfs_routes` from two sub-routers with independent policies,
then merge:
  * `/ipfs/{cid}` keeps `optional_signature` + `rate_limit_by_ip` + the
    `IpRateLimiter` extension. Same behavior as before #134.
  * `/api/v1/ipfs/pins` gets only `optional_signature`. Auth-required
    (handler denies with 401), but no rate limit — it has nothing to
    flood.

Two new `build_router` regressions exercise the bucket independence
through `TrustedProxy::XForwardedFor`:
  * 5 signed pin requests against a size-2 CID bucket all 200, then a
    `/ipfs/{cid}` from the same IP also 200 (pins did not debit the
    bucket).
  * A `/ipfs/{cid}` exhausts a size-1 CID bucket (200, then 429), then
    a signed pin from the same IP is 200 (the bucket does not gate
    pins).

All 65 tests in `server::tests::` + `test_support::tests::ipfs_*`
still pass, including the existing 429 tests for the CID resolver.
2026-08-29 10:52:55 +06:00
Gravirei f49ed79336 fix(node): gate scoped /api/v1/arweave/anchors reads on repository visibility
Closes the review follow-up on #134: a signed non-reader could pass the
auth-layer check on `?repo=<owner>/<name>` and obtain private-repo anchor
metadata (ref names, old/new SHAs, CIDs, irys tx ids). The auth check
proved only that the signature was valid; it did not bind the request to
the canonical repo-read policy.

* Parse the slug through `validate_repo_slug` (same helper the sync path
  uses) and call `authorize_repo_read` before the SQL query. Missing
  repos, quarantined mirrors, and signed non-readers all collapse to the
  standard 404 (`repo_not_found`).
* The unscoped global listing stays auth-only (#121 narrowing; the wider
  permissionless-identity enumeration is the #136 class, out of scope).
* Clamp the limit to `[0, 200]` so `?limit=-1` no longer reaches Postgres
  as `LIMIT -1`.

Five new `build_router` regressions cover the full production contract
through the real `optional_signature` middleware: scoped unsigned 401,
scoped owner 200, scoped non-reader 404 with no anchor metadata leaked
in the body, scoped missing-repo 404 indistinguishable from non-reader,
and `?limit=-1` clamped to an empty 200.
2026-08-29 03:32:01 +06:00
Gravirei 752e117f07 fix(node): format arweave routes for consistency in build_router function 2026-08-28 16:09:52 +06:00
Gravirei f70bb0008e fix(node): enforce authentication on /api/v1/ipfs/pins and /api/v1/arweave/anchors routes 2026-08-28 16:04:51 +06:00
Gravirei 063a5e689f fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#134)
Add auth rejection to list_pins and list_anchors handlers. The pin index
spans the entire node and would expose metadata for every object ever
pushed; anonymous callers must not see it.

- Require AuthenticatedDid extension in both handlers, return 401 when absent
- Add server.rs regression test for anonymous rejection through build_router
- Fix closed-pool tests to pass authenticated requests (test 503 path)
2026-08-26 16:22:29 +06:00
66 changed files with 973 additions and 22326 deletions

No files matched your search

-31
View File
@@ -21,25 +21,6 @@ ignore = [
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
# lru 0.16.3 (use-after-free in pop()). Reachable via alloy. The
# lockfile holds only 0.16.3 — the aws-sdk-s3 path that once pulled
# 0.12.5 no longer does. The RUSTSEC-2026-0253 advisory covers this
# range, so the ignore stays load-bearing until a fixed 0.16.x lands;
# REMOVE when the locked version is fixed or lru leaves the tree.
# Retirement owner: node maintainers, enforced by the weekly
# audit-schedule drift guard ("Fail if lru moved while its ignore
# remains"), which fails if either locked version moves or lru
# leaves the tree. No separate tracking issue exists; file one
# against the lru upgrade if this waiver needs discussion outside
# the schedule job.
"RUSTSEC-2026-0253",
#
# Round-3 P2 (reviewer 2): the previous version of this file
# ignored RUSTSEC-2026-0258 for h2 0.4.13, but the lockfile
# already has h2 0.4.18 (verified 2026-08-31: cargo tree -p h2
# shows 0.4.18) — the fix is in. The ignore was stale and would
# have silently re-accepted the advisory if h2 dropped back to
# 0.4.13. Removed.
# crypto-bigint 0.5.5 (secret-dependent branches in operations meant
# to be constant-time). REACHABLE accepted risk, on two paths:
# - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256
@@ -58,16 +39,4 @@ ignore = [
# (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint
# below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`.
"RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix)
# libsqlite3-sys 0.30.1 (bundled SQLite WAL uninitialized-memory disclosure,
# CVE-2026-119375). Present in Cargo.lock because sqlx resolves sqlx-sqlite
# unconditionally; sqlx is built postgres-only
# (crates/gitlawb-node/Cargo.toml), so sqlx-sqlite and its bundled SQLite C
# source are never compiled or linked (`cargo tree -i sqlx-sqlite` prints
# nothing) and there is no reachable WAL open to attack. The fix requires
# libsqlite3-sys >= 0.39, but sqlx-sqlite 0.8.6 pins ^0.30.1; a compatible
# bump needs a sqlx upgrade, not a lockfile refresh. REMOVE this ignore if
# sqlx is ever built with the `sqlite` feature or any other libsqlite3-sys
# consumer enters the build.
"RUSTSEC-2026-0459", # libsqlite3-sys bundled SQLite CVE (not linked in our build)
]
-20
View File
@@ -39,12 +39,6 @@ GITLAWB_KEY=/data/keys/identity.pem
# Publicly reachable URL of this node (used in peer announcements)
GITLAWB_PUBLIC_URL=https://your-node.example.com
# Base URL for the web view of repos on this node (used by `gl` to print a
# working View: link after repo creation). Omit for nodes with no web front-end.
# Distinct from GITLAWB_PUBLIC_URL: that is API reachability for peers;
# this is browser reachability for humans.
# GITLAWB_WEB_URL=https://gitlawb.com
# ── Server ────────────────────────────────────────────────────────────────
GITLAWB_HOST=0.0.0.0
GITLAWB_PORT=7545
@@ -359,20 +353,6 @@ GITLAWB_TRUSTED_PROXY=
# Enable automatic background sync from known peers
GITLAWB_AUTO_SYNC=false
# ── Reconciliation sweep ─────────────────────────────────────────────────
# Periodic durability sweep: re-derives the public pin set and the withheld-blob
# recovery set each hour and fills gaps so a dropped replication job never means
# data loss. Defaults to true; set to false to disable the sweep even when a pin
# backend (IPFS/Pinata) is configured.
#
# Phase-capability matrix:
# - Public pin repair: IPFS-only, Pinata-only, or both (requires the
# respective backend to be configured).
# - Encrypted recovery repair: requires local IPFS (GITLAWB_IPFS_API).
# Pinata-only nodes reconcile public pins only; encrypted recovery
# reconciliation is not performed.
GITLAWB_RECONCILIATION_SWEEP=true
# ── iCaptcha proof-of-intelligence gate ───────────────────────────────────
# Optional gate on create_repo + register: require callers to present an
# iCaptcha proof (X-ICaptcha-Proof header) earned at icaptcha.gitlawb.com.
-24
View File
@@ -86,27 +86,3 @@ jobs:
exit 1
fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version."
# Drift guard: the RUSTSEC-2026-0253 (lru) ignore is only valid while
# the lockfile holds exactly the version it was written for (0.16.3
# via alloy). Any move — an upgrade that may carry the upstream fix,
# or removal of the dependency — must revisit the ignore. Retirement
# owner: node maintainers, via this weekly job (no separate tracking
# issue; see the waiver comment in .cargo/audit.toml).
- name: Fail if lru moved while its ignore remains
run: |
set -euo pipefail
versions="$(grep -A1 'name = "lru"' Cargo.lock | grep '^version' | cut -d'"' -f2 | sort -u | tr '\n' ' ')"
echo "lru versions in Cargo.lock: ${versions:-not present}"
lru_ignore_present=false
if grep -q 'RUSTSEC-2026-0253' .cargo/audit.toml; then
lru_ignore_present=true
fi
# Drift = the lockfile no longer matches what the ignore assumes:
# the version moved (possible fix) or lru left the tree
# (dead ignore). Both must fail for explicit re-triage.
if [ "$lru_ignore_present" = true ] && [ "${versions}" != "0.16.3 " ]; then
echo "::error::lru versions are [${versions:-absent from Cargo.lock}] but the RUSTSEC-2026-0253 ignore is still in .cargo/audit.toml. Re-triage the waiver against the advisory's fixed range and either drop the ignore or update the pinned set."
exit 1
fi
echo "No drift: lru ignore is consistent with the pinned versions."
+1 -1
View File
@@ -81,7 +81,7 @@ time once updated.
**Requirements:**
- Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs)
- PostgreSQL 16+ — required for the node (it uses `pg_input_is_valid`, which exists only on PostgreSQL 16 and later). Use the bundled `docker-compose.yml` for local dev.
- PostgreSQL — required for the node. Use the bundled `docker-compose.yml` for local dev.
- Docker (optional, for full-stack local testing)
**Environment variables:**
Generated
+8 -14
View File
@@ -2208,7 +2208,7 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -2701,7 +2701,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de"
dependencies = [
"data-encoding",
"syn 1.0.109",
"syn 2.0.117",
]
[[package]]
@@ -3085,7 +3085,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -3563,9 +3563,7 @@ dependencies = [
"libp2p-swarm",
"mockito",
"multiaddr",
"percent-encoding",
"prometheus",
"rand 0.8.6",
"reqwest",
"serde",
"serde_json",
@@ -3582,7 +3580,6 @@ dependencies = [
"tracing",
"tracing-subscriber",
"unicode-normalization",
"url",
"uuid",
"zeroize",
"zstd",
@@ -3601,9 +3598,7 @@ dependencies = [
"ed25519-dalek 2.2.0",
"gitlawb-core",
"icaptcha-client",
"libc",
"mockito",
"percent-encoding",
"reqwest",
"serde",
"serde_json",
@@ -3612,7 +3607,6 @@ dependencies = [
"tokio",
"tracing",
"tracing-subscriber",
"url",
"urlencoding",
"uuid",
]
@@ -3648,9 +3642,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.19"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
checksum = "839c0e8a181239723652be9062bb56ca5bf5f64011f73b623f6f4fc59086a228"
dependencies = [
"atomic-waker",
"bytes",
@@ -6229,7 +6223,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -7163,7 +7157,7 @@ dependencies = [
"getrandom 0.4.2",
"once_cell",
"rustix",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -7955,7 +7949,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.61.2",
]
[[package]]
+1 -3
View File
@@ -54,10 +54,8 @@ uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core; also used for the
# absolute-browser-url validation on GITLAWB_WEB_URL in the node and CLI)
# take a parsed URL without pulling reqwest into gitlawb-core)
url = "2"
percent-encoding = "2"
# HMAC
hmac = "0.12"
-3
View File
@@ -53,7 +53,6 @@ Good today:
- Local or Docker node startup.
- Postgres-backed repo metadata.
- Bounded GraphQL queries with [repository pagination](docs/graphql-pagination.md).
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
@@ -312,7 +311,6 @@ metadata local disk / optional S3
| DID | A user, agent, or node identity derived from an Ed25519 public key. |
| HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. |
| Ref certificate | Signed record of a ref update. Useful for audit and replication. |
| Ref certificate versioning | The wire format is versioned; new certificates are currently issued as v1. `gl cert show <repo> <id> --verify` requires `--expect-node <did>`, and the queried node's self-reported DID is **not** accepted as a trust anchor. v2 is reserved for a future payload shape that carries the version inside the signed bytes; this client refuses to verify v2+ rather than guessing. |
| UCAN | Delegation token for future capability-based workflows. |
| Peer announce | Node-to-node HTTP announcement of DID + public URL. |
| Gossipsub | libp2p topic for ref-update events. |
@@ -404,7 +402,6 @@ Important node settings:
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require peer writes to prove their DID on every transport: RFC 9421 signatures on the announce/sync routes, and an Ed25519 payload signature on gossip ref-update events. Defaults to `false` during the staged rollout below. Upgrade every gossip publisher before enabling, or their updates are dropped with no error on their side. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization (anyone can mint a key and sign), so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_RECONCILIATION_SWEEP` | Enable the hourly durability sweep that re-pins/backstops missing objects (default `true`; disabled when no IPFS/Pinata backend is configured). Public pin repair runs against any configured backend (IPFS, Pinata, or both). Encrypted recovery repair requires local IPFS (`GITLAWB_IPFS_API`); Pinata-only nodes reconcile public pins only. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
+1 -9
View File
@@ -1,7 +1,7 @@
{
"$comment": "Canonical seed list for the Twigpine network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1,
"updated": "2026-10-02",
"updated": "2026-06-30",
"peers": [
{
"name": "gitlawb",
@@ -42,14 +42,6 @@
"http_url": "https://rapybus-gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-05-16"
},
{
"name": "moonwake-ledger",
"operator": "Moonwake Ledger",
"did": "did:key:z6Mkh9AZ46Z3RkGZfFCom5vzX2535f54d6G2o2ZGkCqjwW8K",
"http_url": "https://node.moonwakeledger.com",
"p2p_multiaddr": null,
"added": "2026-10-02"
}
]
}
@@ -137,11 +137,6 @@ enum ShimMode {
struct Shim {
base_url: String,
posts: Arc<AtomicUsize>,
/// Advertisement (GET /info/refs) count. One `git fetch` performs exactly
/// one advertisement, so tests assert this is EXACTLY one: any fetch-level
/// retry that comes back re-advertises and goes red on the spot, which the
/// lower-bound POST assertions alone cannot see (#275 round 6).
gets: Arc<AtomicUsize>,
stop: Arc<AtomicBool>,
handle: Option<std::thread::JoinHandle<()>>,
}
@@ -171,17 +166,15 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
let addr = listener.local_addr().unwrap();
let base_url = format!("http://{addr}");
let posts = Arc::new(AtomicUsize::new(0));
let gets = Arc::new(AtomicUsize::new(0));
let stop = Arc::new(AtomicBool::new(false));
let posts_t = posts.clone();
let gets_t = gets.clone();
let stop_t = stop.clone();
let handle = std::thread::spawn(move || {
while !stop_t.load(Ordering::SeqCst) {
match listener.accept() {
Ok((stream, _)) => {
handle_conn(stream, &repo, mode, &posts_t, &gets_t);
handle_conn(stream, &repo, mode, &posts_t);
}
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(Duration::from_millis(5));
@@ -194,7 +187,6 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
Shim {
base_url,
posts,
gets,
stop,
handle: Some(handle),
}
@@ -220,13 +212,7 @@ fn normalize_accepted_stream(stream: &TcpStream) {
stream.set_read_timeout(Some(Duration::from_secs(30))).ok();
}
fn handle_conn(
stream: TcpStream,
repo: &Path,
mode: ShimMode,
posts: &AtomicUsize,
gets: &AtomicUsize,
) {
fn handle_conn(stream: TcpStream, repo: &Path, mode: ShimMode, posts: &AtomicUsize) {
normalize_accepted_stream(&stream);
let mut reader = BufReader::new(stream);
@@ -277,7 +263,6 @@ fn handle_conn(
receive_pack(repo, false, &body),
)
} else if method == "GET" && target.contains("/info/refs") {
gets.fetch_add(1, Ordering::SeqCst);
// v0 advertisement, wrapped exactly as the node's info_refs does.
let adv = upload_pack(repo, true, b"");
let mut wrapped = pkt(b"# service=git-upload-pack\n");
@@ -1022,7 +1007,6 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
let server = std::thread::spawn(move || {
let posts = AtomicUsize::new(0);
let gets = AtomicUsize::new(0);
let stream = loop {
match listener.accept() {
Ok((s, _)) => break s,
@@ -1037,7 +1021,7 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
// the assertions below run on every platform.
stream.set_nonblocking(true).unwrap();
accepted_tx.send(()).unwrap();
handle_conn(stream, &repo, ShimMode::Normal, &posts, &gets);
handle_conn(stream, &repo, ShimMode::Normal, &posts);
});
let mut client = TcpStream::connect(addr).unwrap();
@@ -1101,17 +1085,6 @@ fn real_git_multi_round_fetch_completes() {
posts >= 2,
"fixture did not force multi-round negotiation (observed {posts} POST(s)); the bridging path was not exercised"
);
// Exactly one advertisement: the POST bound above is a floor, so a
// reintroduced fetch-level retry could satisfy it by running a second full
// fetch instead of multi-round negotiation. Re-advertising is what a
// second fetch cannot avoid, so this is the committed guard on the
// retry's absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
// The fetched tip is present and the clone's object graph is intact.
let server_head = String::from_utf8(git(&server, &["rev-parse", "HEAD"])).unwrap();
@@ -1204,17 +1177,6 @@ fn real_git_withheld_shaped_first_post() {
"helper hung on a withheld-shaped response (should forward-and-terminate, not deadlock). stderr:\n{stderr}"
);
// Exactly one advertisement, pass or fail: the POST assertions below are
// floors, so a reintroduced fetch-level retry could mask the withheld
// shape behind a second full fetch. A retry cannot avoid re-advertising,
// so this is the committed guard on its absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
if out.status.success() {
// Real git accepted the mid-negotiation pack: the withheld multi-round
// path works end to end with no extra handling.
-37
View File
@@ -539,43 +539,6 @@ mod tests {
assert!(matches!(err, Error::Signature(_)));
}
/// The identity-point forgery must be rejected: the shared attestation
/// verifier is a cert-bound provenance gate, so accepting the weak-key
/// signature would let anyone mint a forged attestation that verifies.
/// Strict verification rejects small-order public keys and R (the identity
/// point here), which ordinary verification does not.
#[test]
fn verify_rejects_identity_point_forgery() {
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&fresh(), cert_hash);
// Public key A = identity point (0,1); signature R = identity, S = 0.
// The equation `[S]B = R + [k]A` then holds for any k and any message.
let identity = [
1u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
let mut buf = Vec::with_capacity(34);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&identity);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let mut sig = [0u8; 64];
sig[..32].copy_from_slice(&identity);
att.sig = B64U.encode(sig);
let err = att.verify_signature(cert_hash).unwrap_err();
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"identity-point signer must be refused as a small-order key, got: {msg}"
),
other => panic!("expected Error::Did for an identity-point signer, got {other:?}"),
}
}
/// A payload that happens to contain a `cert_hash` field of its own does
/// not interfere with the outer binding: the attestation envelope's
/// `cert_hash` is the only field consulted by `verify_signature`, and the
+11 -381
View File
@@ -5,28 +5,6 @@
//!
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
//!
//! ## Replay
//!
//! A certificate proves *who* asserted a ref update and *what* they asserted.
//! It does not prove the assertion is fresh. [`RefUpdateBody::seq`] and
//! [`RefUpdateBody::nonce`] are inside the signed bytes (see
//! [`RefUpdateBody::to_signing_bytes`]), so a verifier can trust the values,
//! but deciding whether a nonce is being reused or a `seq` is going backwards
//! needs state.
//!
//! This crate is deliberately stateless — it is embedded in both nodes and
//! clients — so replay is **not** enforced here and cannot be. Ownership of
//! the check sits with the verifier that accepts and persists certificates,
//! which must:
//!
//! - reject a `nonce` it has already accepted for the same `(repo, ref_name)`;
//! - reject a `seq` that does not advance past the last accepted `seq` for
//! that `(repo, ref_name)`.
//!
//! Until such a verifier exists, replay of ref-update certificates is not
//! enforced anywhere in this workspace. This contract exists so the first one
//! inherits the requirement instead of re-deriving it.
use std::collections::HashSet;
@@ -41,30 +19,6 @@ use crate::{Error, Result};
/// The certificate type discriminant. Always `"gitlawb/ref-update/v1"`.
pub const CERT_TYPE: &str = "gitlawb/ref-update/v1";
/// Exact url-safe unpadded base64 length of a 64-byte Ed25519 signature.
/// The tolerant threshold scan requires this canonical unpadded length before
/// attempting decode: the length gate is a DoS bound that keeps an
/// attacker-sized sig string from costing allocation before the signature is
/// rejected. A longer string is not meaningless (the same 64 bytes can appear
/// in padded or non-url-safe encodings) but no in-tree signer produces one,
/// so the scan skips non-canonical entries rather than decoding them.
const ENCODED_SIG_LEN: usize = 86;
/// Hard ceiling on signature entries a certificate may carry into the
/// tolerant threshold scan. A legitimate certificate holds one entry per
/// maintainer (a re-sign after key rotation may briefly leave a second), so
/// this is far above any real signer set, while turning the scan's worst case
/// from "one curve verification per attacker-appended entry" into a small
/// constant. Rejection is loud (`Err`), not a silent prefix scan: scanning
/// only the first N entries would let junk placed ahead of the real
/// signatures starve them out of the window, which is the #349 denial again
/// in positional form. An attacker positioned to inflate the list past this
/// ceiling can already suppress the certificate outright. The loud
/// rejection does deny a legitimately over-long certificate; that is an
/// explicit, detectable denial above the cap rather than a silent window
/// junk placement could exploit.
const MAX_SIGNATURE_ENTRIES: usize = 64;
/// A signature on a ref-update certificate.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CertSignature {
@@ -87,19 +41,11 @@ pub struct RefUpdateBody {
pub from: String,
/// The new commit hash (SHA-256 hex).
pub to: String,
/// Monotonically increasing sequence number, scoped to `(repo, ref_name)`.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a `seq` that does not
/// advance for the same repo and ref. See the module docs on replay.
/// Monotonically increasing sequence number. Prevents replay.
pub seq: u64,
/// RFC 3339 timestamp of this update.
pub timestamp: DateTime<Utc>,
/// Random nonce, unique per certificate.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a nonce it has already
/// accepted for the same repo and ref. See the module docs on replay.
/// Random nonce for deduplication.
pub nonce: String,
}
@@ -162,15 +108,9 @@ impl RefUpdateCert {
Ok(())
}
/// Verify all signatures on this certificate, failing closed.
/// Verify all signatures on this certificate.
///
/// On success returns the list of DIDs whose signatures are valid. This is
/// strict: it returns an error if *any* signature entry is malformed or
/// invalid. Because signature entries live outside the signed body (anyone
/// can append one without key material), do not use this to make a
/// threshold decision on an untrusted certificate — a single appended junk
/// entry would reject the whole cert. Use [`Self::satisfies_threshold`],
/// which ignores invalid entries and counts only the valid signers.
/// Returns the list of DIDs whose signatures are valid.
pub fn verify_all(&self) -> Result<Vec<Did>> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let signing_bytes = self.body.to_signing_bytes()?;
@@ -199,65 +139,12 @@ impl RefUpdateCert {
/// from the provided set of authorized maintainer DIDs.
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once. Malformed and invalid
/// signature entries are skipped, not fatal — entries live outside the
/// signed body, so any third party can append junk without key material,
/// and short-circuiting on the first bad one (as [`Self::verify_all`]
/// does) would let that junk deny an otherwise valid certificate (#349).
///
/// Tolerance must not become an unbounded verification path (#349 review
/// round 2): entries are attacker-extensible, so the work done for them is
/// bounded BEFORE the loop, not discovered inside it:
///
/// - a list over [`MAX_SIGNATURE_ENTRIES`] is rejected outright (see the
/// constant for why loud rejection beats a silent prefix scan);
/// - an encoded signature is length-gated before base64 work — a 64-byte
/// Ed25519 signature is exactly [`ENCODED_SIG_LEN`] characters in
/// url-safe unpadded base64, so an oversized blob never allocates;
/// - entries whose signer is not in `maintainers`, or is already counted,
/// are skipped on a string compare, before any decode or curve work —
/// so even within the cap, junk naming strangers costs no crypto;
/// - the scan returns as soon as the threshold is met.
/// signature from the same maintainer counts once.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
if threshold == 0 {
return Err(Error::RefCert("threshold must be at least 1".to_string()));
}
if self.signatures.len() > MAX_SIGNATURE_ENTRIES {
return Err(Error::RefCert(format!(
"certificate carries {} signature entries (max {})",
self.signatures.len(),
MAX_SIGNATURE_ENTRIES
)));
}
let signing_bytes = self.body.to_signing_bytes()?;
let mut counted: HashSet<&Did> = HashSet::new();
for cert_sig in &self.signatures {
if counted.contains(&cert_sig.signer) || !maintainers.contains(&cert_sig.signer) {
continue;
}
if cert_sig.sig.len() != ENCODED_SIG_LEN {
continue;
}
let Ok(vk) = cert_sig.signer.to_verifying_key() else {
continue;
};
let Ok(sig_bytes_vec) = URL_SAFE_NO_PAD.decode(&cert_sig.sig) else {
continue;
};
let Ok(sig_bytes) = <[u8; 64]>::try_from(sig_bytes_vec) else {
continue;
};
if verify(&vk, &signing_bytes, &sig_bytes).is_ok() {
counted.insert(&cert_sig.signer);
if counted.len() >= threshold {
return Ok(true);
}
}
}
Ok(counted.len() >= threshold)
let valid = self.verify_all()?;
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
Ok(distinct_signers.len() >= threshold)
}
/// Validate the certificate structure (not signatures).
@@ -490,151 +377,8 @@ mod tests {
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
/// An attacker can append junk signature entries to a cert (they live
/// outside the signed body, so no key material is needed). `satisfies_threshold`
/// must ignore them and still count the real signers — before the fix,
/// `verify_all`'s `?` on the junk made the whole check error out, denying a
/// valid 2-of-2 cert.
#[test]
fn satisfies_threshold_ignores_appended_junk_signatures() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
cert.countersign(&kp2).unwrap();
// Junk sits AHEAD of the real entries so the scan must actually skip
// it: appended last, the threshold-2 early return at the second real
// signature would fire before any junk is examined and the test would
// stay green even if the skip logic never ran.
cert.signatures.insert(
0,
CertSignature {
signer: kp1.did(),
sig: "!!!not-base64!!!".to_string(),
},
);
// Valid base64 but only 3 bytes — fails the 64-byte length check.
cert.signatures.insert(
0,
CertSignature {
signer: kp2.did(),
sig: "AAAA".to_string(),
},
);
// Exactly ENCODED_SIG_LEN of valid base64 decoding to garbage —
// exercises the decode+verify path before the skip, not just the
// length gate.
cert.signatures.insert(
0,
CertSignature {
signer: kp1.did(),
sig: "A".repeat(ENCODED_SIG_LEN),
},
);
let maintainers = vec![kp1.did(), kp2.did()];
assert!(cert.satisfies_threshold(&maintainers, 2).unwrap());
}
/// A junk entry that names a real maintainer DID but carries a signature
/// that does not verify must not be counted — otherwise appending garbage
/// under a maintainer's DID could forge a threshold.
#[test]
fn satisfies_threshold_does_not_count_a_forged_maintainer_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Well-formed 64-byte signature from kp2, but over unrelated bytes, so
// it fails verification against the cert body.
cert.signatures.push(CertSignature {
signer: kp2.did(),
sig: kp2.sign_b64(b"unrelated bytes"),
});
let maintainers = vec![kp1.did(), kp2.did()];
// Only kp1 actually signed the body: 2-of-2 must fail, 1-of-2 holds.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
}
/// The tolerant scan must not be an unbounded verification path: a list
/// past MAX_SIGNATURE_ENTRIES is rejected loudly before any curve work,
/// while a list at the cap still verifies normally.
#[test]
fn satisfies_threshold_rejects_an_oversized_signature_list() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let junk = || CertSignature {
signer: Keypair::generate().did(),
sig: "AAAA".to_string(),
};
while cert.signatures.len() < MAX_SIGNATURE_ENTRIES {
cert.signatures.push(junk());
}
let maintainers = vec![kp.did()];
// At the cap: still fine, the real signature counts.
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
// One past the cap: loud rejection, no tolerant scan.
cert.signatures.push(junk());
let err = cert.satisfies_threshold(&maintainers, 1).unwrap_err();
assert!(err.to_string().contains("signature entries"));
}
/// An entry whose encoded signature is not the exact base64 length of a
/// 64-byte Ed25519 signature is skipped before any base64 decode — an
/// attacker-sized sig string must cost neither allocation nor curve work,
/// and must not affect the verdict.
#[test]
fn satisfies_threshold_skips_an_oversized_encoded_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Names a real maintainer, valid base64 alphabet, but megabytes long:
// the length gate must refuse it without decoding.
cert.signatures.push(CertSignature {
signer: kp2.did(),
sig: "A".repeat(2 * 1024 * 1024),
});
let maintainers = vec![kp1.did(), kp2.did()];
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_rejected() {
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
@@ -645,120 +389,6 @@ mod tests {
&kp,
)
.unwrap();
// Zero must never read as "no signatures required", even with an empty
// maintainer set.
let err = cert.satisfies_threshold(&[], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"zero must be refused as an invalid threshold"
);
}
/// The threshold is checked before any signature work, so a zero is
/// reported as a threshold error even when the certificate's signatures
/// are also invalid. Otherwise a broken certificate would let a caller
/// mistake the failure for a legitimate verification result.
#[test]
fn threshold_zero_is_rejected_before_signature_verification() {
let kp = Keypair::generate();
let other = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures[0].sig = other.sign_b64(b"unrelated bytes");
let err = cert.satisfies_threshold(&[kp.did()], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"the invalid threshold must win over the invalid signature"
);
}
/// `seq` and `nonce` are the verifier's replay inputs, so they have to be
/// inside the signed body; tapping either one must invalidate the
/// certificate rather than leave it verifying with different values.
#[test]
fn replay_fields_are_covered_by_the_signature() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
7,
&kp,
)
.unwrap();
cert.verify_all().expect("baseline must verify");
let mut bumped = cert.clone();
bumped.body.seq = 8;
assert!(
matches!(bumped.verify_all(), Err(Error::SignatureInvalid)),
"changing seq must invalidate the signature"
);
let mut renamed = cert.clone();
renamed.body.nonce = "different-nonce".to_string();
assert!(
matches!(renamed.verify_all(), Err(Error::SignatureInvalid)),
"changing nonce must invalidate the signature"
);
}
/// A signature minted by a real key must not verify when the signer is
/// spelled in another multibase alphabet. Otherwise the same identity
/// would satisfy a maintainer allowlist under several DID strings.
///
/// Base64url is deliberate: its 47-character method-id fits under the
/// pre-existing length bound, so this test reaches the canonical-prefix
/// rule and fails on the pre-fix code instead of being refused for length.
#[test]
fn verify_all_rejects_a_non_canonical_did_spelling() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let mut prefixed = vec![0xed, 0x01];
prefixed.extend_from_slice(&kp.verifying_key().to_bytes());
cert.signatures[0].signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base64Url, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = cert
.verify_all()
.expect_err("a non-canonical did:key spelling must not resolve");
assert!(
err.to_string().contains("base58btc"),
"rejection must come from the canonical-prefix rule, not the length guard"
);
assert!(
!matches!(
cert.satisfies_threshold(&[cert.signatures[0].signer.clone()], 1),
Ok(true)
),
"the alternate spelling must not satisfy a threshold"
);
// The canonical spelling of the same key is unaffected.
cert.signatures[0].signer = kp.did();
assert!(cert.satisfies_threshold(&[kp.did()], 1).unwrap());
assert!(cert.satisfies_threshold(&[], 0).unwrap());
}
}
+3 -158
View File
@@ -20,19 +20,6 @@ use crate::{Error, Result};
/// Value: 0xed01 encoded as a varint.
const ED25519_MULTICODEC: &[u8] = &[0xed, 0x01];
/// The only multibase prefix a `did:key` method-specific id may carry.
///
/// The DID:key spec encodes Ed25519 keys as base58btc, which is multibase
/// `z`. `multibase::decode` happily accepts every other alphabet, and each
/// spelling of the same key bytes would produce a distinct DID string while
/// gates, allowlists, and database rows compare DIDs as strings — one
/// identity would read as several. Rejecting the alternatives here keeps the
/// mapping from `did:key` to key single-valued for every caller of
/// [`Did::to_verifying_key`] (HTTP signatures, UCAN, ref certs, peer
/// announces). `gitlawb-attest` enforces the same rule for attestation
/// signers.
const DID_KEY_MULTIBASE_PREFIX: char = 'z';
/// A Decentralized Identifier.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)]
@@ -84,14 +71,8 @@ impl Did {
/// Resolve the Ed25519 verifying key from a `did:key`.
///
/// Only the canonical base58btc (`z`) spelling is accepted. Other
/// multibase encodings decode to the same key but spell a different DID
/// string, so accepting them would let one identity appear under several
/// identifiers to string-comparing gates and allowlists.
///
/// Returns `Err` if the DID is not a `did:key`, uses a non-canonical
/// multibase prefix, the key bytes are invalid, or the key is a
/// small-order (weak) point.
/// Returns `Err` if the DID is not a `did:key`, the key bytes are malformed,
/// or the key is a small-order (weak) point.
pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() {
return Err(Error::InvalidDid(format!(
@@ -113,27 +94,9 @@ impl Did {
));
}
// Cheap prefix gate before the decode, then the base check on what
// multibase actually parsed. The first keeps a clearly non-canonical id
// off the decoder. The second is belt-and-braces — multibase maps a
// leading `z` only to `Base58Btc` — but it states the requirement
// directly instead of relying on that mapping, and `gitlawb-attest`
// makes the same pair of checks.
if !self.method_id().starts_with(DID_KEY_MULTIBASE_PREFIX) {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
let (base, bytes) =
let (_, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
if base != Base::Base58Btc {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
if !bytes.starts_with(ED25519_MULTICODEC) {
return Err(Error::InvalidDid(
"not an ed25519 multicodec key".to_string(),
@@ -426,124 +389,6 @@ mod tests {
did.to_verifying_key().expect("a real did:key must resolve");
}
/// `multibase::decode` accepts every alphabet, so the same Ed25519 key can
/// be spelled as several `did:key` strings. Only base58btc is canonical;
/// the rest must be refused so one identity cannot appear as several to
/// gates and allowlists that compare DID strings.
#[test]
fn to_verifying_key_rejects_non_base58btc_multibase() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[9u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let canonical = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
canonical.to_verifying_key().unwrap().to_bytes(),
signing.verifying_key().to_bytes(),
"base58btc is the accepted spelling"
);
// These spellings fit inside the method-id bound, so they reach the
// canonical-prefix check and must be refused there.
for base in [Base::Base64Url, Base::Base32Lower] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert_ne!(
did.as_str(),
canonical.as_str(),
"the same key must not compare equal under another spelling"
);
let err = did
.to_verifying_key()
.expect_err("non-canonical multibase must be refused");
assert!(
matches!(err, Error::InvalidDid(_)),
"must be an InvalidDid, got {err:?}"
);
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
// A base16 spelling of the same key is longer than the method-id
// bound, so it is refused by the length guard before the prefix check.
// Either way it must not resolve.
for base in [Base::Base16Lower, Base::Base16Upper] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert!(
matches!(did.to_verifying_key(), Err(Error::InvalidDid(_))),
"a base16 spelling of a real key must not resolve"
);
}
}
/// base58flickr (`Z`) is the non-bitcoin sibling alphabet. Its spelling of
/// a real key fits under the length bound, so it must be refused for the
/// prefix, not merely for its size.
#[test]
fn to_verifying_key_rejects_a_base58flickr_spelling() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[11u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!(
"did:key:{}",
multibase::encode(Base::Base58Flickr, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("base58flickr must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// An empty method-specific id must be refused by the prefix rule without
/// panicking on a missing first character.
#[test]
fn to_verifying_key_rejects_an_empty_method_id() {
let did: Did = "did:key:".parse().expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("an empty method id must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// A canonical base58btc spelling is not enough on its own: the bytes must
/// also carry the ed25519-pub multicodec. This pins the check that stops a
/// key of another type from resolving as an Ed25519 key.
#[test]
fn to_verifying_key_rejects_base58btc_with_a_wrong_multicodec() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[12u8; 32]);
let mut buf = vec![0x12, 0x00]; // sha2-256, not ed25519-pub
buf.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!("did:key:{}", multibase::encode(Base::Base58Btc, &buf))
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("a non-ed25519 multicodec must be refused");
assert!(
err.to_string().contains("ed25519"),
"the error must name the expected multicodec: {err}"
);
}
/// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key
/// satisfies the verification equation for any message and, converted to
/// Montgomery form for X25519, yields the all-zero shared secret, so an
-20
View File
@@ -566,26 +566,6 @@ mod tests {
assert!(sig.check_created().is_ok());
}
#[test]
fn extreme_created_timestamps_reject_without_wrapping() {
let kp = Keypair::generate();
let did = kp.did();
let parse = |created: i64| {
let sig_input =
format!(r#"sig1=("@method");keyid="{did}";alg="ed25519";created={created}"#);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
};
// i64::MIN used to panic in debug (subtraction overflow) and wrap to a
// passing skew in release; now - 2^63 wrapped to i64::MIN, whose abs()
// wrapped back to itself and read as fresh. Both must reject.
for created in [i64::MIN, i64::MAX, Utc::now().timestamp() - i64::MAX] {
assert!(
parse(created).check_created().is_err(),
"created={created} must not pass the freshness gate"
);
}
}
#[test]
fn parse_error_missing_sig1_prefix() {
let err = HttpSignature::parse(
-40
View File
@@ -77,14 +77,6 @@ impl Keypair {
}
/// Verify an Ed25519 signature.
///
/// Strict verification: rejects small-order `R` and small-order public keys
/// (the identity point, and any point of low order). Ordinary `verify` accepts
/// a signature forged with the identity point as the public key plus
/// `R = identity, S = 0`, which verifies for *any* message. `identity::verify`
/// is the shared primitive behind HTTP request authentication, UCANs, and
/// certificates, so weak-key acceptance is an authentication bypass, not a
/// malleability nuance.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
let sig = Signature::from_bytes(sig_bytes);
verifying_key
@@ -216,38 +208,6 @@ mod tests {
);
}
/// The identity-point forgery: with public key A = identity, R = identity,
/// and S = 0, the equation `[S]B = R + [k]A` holds for every message,
/// because `[k]·identity = identity`. Ordinary (non-strict) Ed25519
/// verification accepts it, so the shared `verify` primitive must use
/// strict verification, which rejects small-order R and public keys.
#[test]
fn verify_rejects_identity_point_forgery() {
use ed25519_dalek::Verifier;
// The identity point (0,1) compresses to y = 1 with sign bit 0.
let identity = [
1u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
let vk = VerifyingKey::from_bytes(&identity).expect("identity point is on the curve");
let mut sig_bytes = [0u8; 64];
sig_bytes[..32].copy_from_slice(&identity);
let msg = b"arbitrary message the key owner never signed";
// Prove the forged signature satisfies the ordinary verification
// equation, so the strict check below is what actually defends the
// boundary (not a signature that was already invalid everywhere).
assert!(
vk.verify(msg, &Signature::from_bytes(&sig_bytes)).is_ok(),
"identity-point forgery must satisfy ordinary verification (this is why strict is needed)"
);
assert!(
verify(&vk, msg, &sig_bytes).is_err(),
"strict verification must reject the identity-point forgery"
);
}
#[test]
fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a
-3
View File
@@ -23,8 +23,6 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
axum = { version = "0.8", features = ["http1", "http2", "json", "ws"] }
async-graphql = { version = "7", features = ["chrono", "uuid", "tracing"] }
async-graphql-axum = "7"
@@ -75,7 +73,6 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
rand = { workspace = true }
libp2p-dns = { version = "0.45.0", features = ["tokio"] }
[dev-dependencies]
-2
View File
@@ -52,7 +52,6 @@ pub async fn list_certs(
"node_did": c.node_did,
"signature": c.signature,
"issued_at": c.issued_at,
"version": c.version,
})
})
.collect();
@@ -93,6 +92,5 @@ pub async fn get_cert(
"node_did": cert.node_did,
"signature": cert.signature,
"issued_at": cert.issued_at,
"version": cert.version,
})))
}
+1 -3
View File
@@ -11,8 +11,7 @@ use crate::state::AppState;
/// Hard ceiling on rows any ref-update feed returns in one request. Shared by the
/// shared collector's clamp and the per-handler request caps so they can't drift.
/// `pub(crate)` so the GraphQL complexity meter can price against the same bound.
pub(crate) const MAX_VISIBLE_REF_UPDATES: i64 = 200;
const MAX_VISIBLE_REF_UPDATES: i64 = 200;
/// Collect up to `limit` ref-update rows visible to `caller`, newest first,
/// paging past rows the feed gate drops. Filtering after a plain SQL `LIMIT`
@@ -438,7 +437,6 @@ mod ref_updates_feed_tests {
node_did: "did:key:z6MkNode".into(),
signature: "sig".into(),
issued_at: Utc::now().to_rfc3339(),
version: 1,
}
}
+8 -434
View File
@@ -2130,32 +2130,9 @@ async fn gate_and_serve(
/// GET /api/v1/ipfs/pins
///
/// Returns all CIDs that have been pinned from git objects received via push.
/// Each entry includes the git SHA-256 hex and the timestamp when it was
/// pinned. The wire contract is nullable by design:
///
/// - `cid` / `local_cid` — node-local resolver keys: the raw CIDv1 the
/// `GET /ipfs/{cid}` endpoint serves. Non-null ONLY when the stored value
/// parses as a raw CIDv1 AND writer-owned local provenance confirms the
/// bytes went through this node's IPFS daemon. Otherwise null, even when
/// the database holds some other CID-shaped string for the row.
/// - `pinata_cid` — the provider identifier Pinata returned (dag-pb/UnixFS,
/// not a node resolver key). The only usable identifier for remote-only
/// rows; `gl` consumers must read this field, never `cid`, for those.
/// - `local_pinned` — `true` iff the local-IPFS writer path
/// (`record_pinned_cid_with_source`) pushed the bytes into the local
/// daemon. Writer-owned; never inferred from CID shape. A row can carry
/// `local_pinned = true` with a null `cid` when its stored key predates
/// the raw-key contract (legacy provider key awaiting repair).
/// - `pinata_pinned` — `true` iff the row has a non-null `pinata_cid`.
///
/// Rows with neither a local nor a Pinata CID are omitted so the response
/// only contains rows with at least one backend. Remote-only rows stay
/// visible through `pinata_cid`/`pinata_pinned`; nothing is hidden.
///
/// #218 review P2: the response surfaces writer-owned provenance so a
/// `gl` consumer can distinguish local-only, remote-only, and dual rows
/// without re-inferring semantics from nullability.
/// Returns all CIDs that have been pinned to the local IPFS node from git
/// objects received via push. Each entry includes the git SHA-256 hex, the
/// CIDv1 string, and the timestamp when it was pinned.
pub async fn list_pins(
State(state): State<AppState>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
@@ -2169,38 +2146,6 @@ pub async fn list_pins(
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251).
let pins = state.db.list_pinned_cids().await?;
let pins: Vec<serde_json::Value> = pins
.into_iter()
.filter(|p| p.cid.is_some() || p.pinata_cid.is_some())
.map(|p| {
// A stored `cid` is advertised as a node-local resolver key
// only when it passes the raw-CIDv1 contract AND local
// provenance is confirmed. The historical `cid` column has
// held more than one namespace (Kubo dag-pb, Pinata CIDv0),
// and the `/ipfs/{cid}` resolver recomputes the raw CID
// from object bytes and rejects anything else — so a
// legacy provider key in `cid`/`local_cid` would promise a
// local object the node immediately refuses to serve.
// Such rows stay listed (with their provenance flags) but
// carry no local key until repair rewrites them.
let local_key = match &p.cid {
Some(c) if gitlawb_core::cid::is_raw_cidv1(c) && p.local_ipfs_provenance => {
Some(c.clone())
}
_ => None,
};
serde_json::json!({
"sha256_hex": p.sha256_hex,
"cid": local_key.clone(),
"local_cid": local_key,
"pinata_cid": p.pinata_cid,
"local_pinned": p.local_ipfs_provenance,
"pinata_pinned": p.pinata_cid.is_some(),
"pinned_at": p.pinned_at,
})
})
.collect();
Ok(Json(serde_json::json!({
"pins": pins,
"count": pins.len(),
@@ -2514,369 +2459,6 @@ mod closed_pool_tests {
);
}
/// #218 review P2: the `list_pins` API response surfaces writer-owned
/// provenance (`local_pinned`, `pinata_pinned`) so a `gl` consumer
/// can distinguish local-only, remote-only, and dual rows without
/// re-inferring semantics from nullability. `cid`/`local_cid` carry a
/// value ONLY for rows whose stored key parses as a raw CIDv1 with
/// confirmed local provenance; anything else (Pinata-only rows,
/// legacy provider keys) exposes its usable identifier through
/// `pinata_cid` alone, and every non-null local key must resolve
/// through `GET /ipfs/{cid}`.
#[sqlx::test]
async fn list_pins_reports_writer_owned_provenance_for_all_shapes(pool: sqlx::PgPool) {
use sqlx::Row as _;
let state = crate::test_support::test_state(pool.clone()).await;
let db = &state.db;
// Raw resolver keys must be REAL raw CIDv1 strings: the handler
// gates `cid`/`local_cid` on `is_raw_cidv1`, so placeholder text
// would fail the gate for the wrong reason. Provider CIDs stay
// opaque Qm… strings (never syntax-checked, only echoed).
fn raw_cid_for(content: &[u8]) -> String {
gitlawb_core::cid::Cid::from_git_object_bytes(content).to_string()
}
// Two real blobs on disk in one public repo, so the local and
// dual rows below can be followed through `GET /ipfs/{cid}`.
// The resolver locates repos through `repo_store.acquire`, not
// the row's `disk_path`, so the recipe overwrites the acquired
// bare path with a real clone: workdir commit of both files,
// cloned --bare into the store path, oids via rev-parse, raw
// keys recomputed from the file bytes exactly as the serve
// path does.
fn run_git(args: &[&str], cwd: &std::path::Path) {
let out = std::process::Command::new("git")
.args(args)
.current_dir(cwd)
.output()
.expect("git runs");
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
let content1 = b"pins follow-through one\n";
let content2 = b"pins follow-through two\n";
let tmp = tempfile::TempDir::new().unwrap();
let work = tmp.path().join("work-pins-list");
std::fs::create_dir_all(&work).unwrap();
std::fs::write(work.join("one.txt"), content1).unwrap();
std::fs::write(work.join("two.txt"), content2).unwrap();
run_git(
&["init", "-q", "--object-format=sha256", "-b", "main"],
&work,
);
run_git(&["config", "user.email", "t@t"], &work);
run_git(&["config", "user.name", "t"], &work);
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "seed"], &work);
let oid_of = |spec: &str| {
let out = std::process::Command::new("git")
.args(["rev-parse", spec])
.current_dir(&work)
.output()
.expect("git rev-parse runs");
assert!(out.status.success(), "rev-parse {spec} failed");
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let sha_local = oid_of("HEAD:one.txt");
let sha_dual = oid_of("HEAD:two.txt");
let raw1 = raw_cid_for(content1);
let raw4 = raw_cid_for(content2);
db.upsert_mirror_repo(
"zPinsListOwner",
"pins-list",
"/unused-pins-list",
None,
false,
)
.await
.unwrap();
let rec = db
.get_repo("zPinsListOwner", "pins-list")
.await
.unwrap()
.expect("mirror repo row");
let bare = state
.repo_store
.acquire(&rec.owner_did, &rec.name)
.await
.unwrap();
let _ = std::fs::remove_dir_all(&bare);
std::fs::create_dir_all(&bare).unwrap();
run_git(
&[
"clone",
"--bare",
"-q",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
tmp.path(),
);
// The combination table for what the response must carry:
//
// shape | local_pinned | pinata_pinned | local_cid | pinata_cid
// -------------------+--------------+---------------+-----------+-----------
// local-only | true | false | raw | null
// pinata-only (raw) | false | true | null | provider
// pinata-only (null) | false | true | null | raw(=provider)
// dual | true | true | raw | provider
// legacy provider key| true | false | null | null
//
// The pinata-only (raw) row keeps a genuine raw key in the
// column but must NOT surface it: provenance is unconfirmed,
// so no local field may carry it. The legacy row predates the
// raw-key contract (dag-pb key, provenance claimed) and must
// surface no local key until repair rewrites it — while the
// row itself (and its `local_pinned` flag) stays visible.
//
// Distinct shas (the table is keyed on sha256_hex) keep the
// rows independent.
// (1) local-only: real local pin via the production writer.
db.record_pinned_cid_with_source(&sha_local, &raw1, &rec.id)
.await
.unwrap();
// (2) pinata-only (raw != provider): the post-v27/v30 row shape
// produced by `record_pinata_cid`. The stored raw key is genuine
// but local provenance is unconfirmed, so neither local field
// may carry it; the provider field is the usable identifier.
let sha_pinata_raw = "sha_p2_pinata_only_distinct_cids";
let raw2 = raw_cid_for(b"pins pinata-only bytes\n");
let pinata2 = "QmPinataProviderCidForRawOnlyRow";
assert_ne!(raw2, pinata2);
db.record_pinata_cid(
sha_pinata_raw,
&raw2,
pinata2,
Some("repo-p2-pinata-raw"),
i64::MAX,
)
.await
.unwrap();
// (3) pinata-only (raw == provider): `record_pinata_cid` stores
// `cid = NULL` so the resolver key isn't aliased to a dag-pb
// provider CID that doesn't hash to the raw bytes (#173).
let sha_pinata_null = "sha_p2_pinata_only_null_cid";
let same = "QmPinataOnlyCidRawEqualsProvider";
db.record_pinata_cid(
sha_pinata_null,
same,
same,
Some("repo-p2-pinata-null"),
i64::MAX,
)
.await
.unwrap();
// (4) dual: local first (sets the flag and `cid`), then Pinata
// (sets `pinata_cid` and preserves `cid` and `local_ipfs_provenance`).
let pinata4 = "QmPinataProviderCidForDualRow";
db.record_pinned_cid_with_source(&sha_dual, &raw4, &rec.id)
.await
.unwrap();
db.record_pinata_cid(&sha_dual, &raw4, pinata4, Some(&rec.id), i64::MAX)
.await
.unwrap();
// (5) legacy provider key: a pre-fix row whose `cid` is a dag-pb
// provider identifier with claimed local provenance. Production
// writers can no longer produce this shape, so it is seeded with
// raw SQL — the point is that the projection must not trust column
// shape as proof of a raw local key.
let sha_legacy = "sha_p2_legacy_provider_key";
sqlx::query(
"INSERT INTO pinned_cids (sha256_hex, cid, pinned_at, pinata_cid, repo_id, local_ipfs_provenance)
VALUES ($1, $2, $3, NULL, $4, TRUE)",
)
.bind(sha_legacy)
.bind("QmLegacyProviderKeyPredatesRawContract")
.bind("2026-07-01T00:00:00Z")
.bind(&rec.id)
.execute(&pool)
.await
.unwrap();
// Hit the production handler end-to-end through a one-shot router
// request, so the test exercises the actual response shape
// (not a unit test on PinnedCidRecord fields).
let resp = Router::new()
.route("/api/v1/ipfs/pins", axum::routing::get(list_pins))
.layer(axum::Extension(crate::auth::AuthenticatedDid(
"did:key:zPinListTest".to_string(),
)))
.with_state(state.clone())
.oneshot(
axum::http::Request::builder()
.uri("/api/v1/ipfs/pins")
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(resp.status(), axum::http::StatusCode::OK);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
let pins = v
.get("pins")
.and_then(|p| p.as_array())
.expect("pins array");
assert_eq!(
pins.len(),
5,
"all five shapes must appear in the response (every row has at least one of cid/pinata_cid set)"
);
// Build a map sha -> pin object for ergonomic assertions.
let by_sha: std::collections::HashMap<String, &Value> = pins
.iter()
.map(|p| {
(
p.get("sha256_hex")
.and_then(|s| s.as_str())
.unwrap()
.to_string(),
p,
)
})
.collect();
// Helper: assert a single pin's fields.
let assert_pin = |sha: &str,
local_pinned: bool,
pinata_pinned: bool,
local_cid: Option<&str>,
pinata_cid: Option<&str>| {
let pin = by_sha.get(sha).unwrap_or_else(|| {
panic!(
"pin row for {sha} missing; got shas {:?}",
by_sha.keys().collect::<Vec<_>>()
)
});
assert_eq!(
pin.get("local_pinned").and_then(|v| v.as_bool()),
Some(local_pinned),
"{sha}: local_pinned mismatch",
);
assert_eq!(
pin.get("pinata_pinned").and_then(|v| v.as_bool()),
Some(pinata_pinned),
"{sha}: pinata_pinned mismatch",
);
assert_eq!(
pin.get("local_cid").and_then(|v| v.as_str()),
local_cid,
"{sha}: local_cid mismatch",
);
assert_eq!(
pin.get("pinata_cid").and_then(|v| v.as_str()),
pinata_cid,
"{sha}: pinata_cid mismatch",
);
// Round-3 P2: `cid` is the LOCAL resolver key, full
// stop. It is the raw CID when the row has a local pin
// (local-only or dual shape) and `None` for Pinata-only
// rows. The previous contract aliased the Pinata provider
// CID into `cid` for Pinata-only rows, which made
// `gl ipfs list` advertise a CID the node's own
// `/ipfs/{cid}` resolver cannot serve (404). The
// provenance split is exposed via `local_cid`,
// `pinata_cid`, and the boolean `local_pinned` /
// `pinata_pinned` flags.
let cid_value = pin.get("cid").cloned().unwrap_or(serde_json::Value::Null);
let expected_cid = local_cid
.map(|s| serde_json::Value::String(s.to_string()))
.unwrap_or(serde_json::Value::Null);
assert_eq!(cid_value, expected_cid, "{sha}: cid mismatch");
};
// (1) local-only
assert_pin(&sha_local, true, false, Some(raw1.as_str()), None);
// (2) pinata-only (raw != provider): the stored raw key is
// genuine but provenance is unconfirmed, so NO local field may
// carry it — remote-only state is visible through `pinata_cid`
// alone.
assert_pin(sha_pinata_raw, false, true, None, Some(pinata2));
// (3) pinata-only (raw == provider) — cid NULL, pinata_cid set
assert_pin(sha_pinata_null, false, true, None, Some(same));
// (4) dual
assert_pin(&sha_dual, true, true, Some(raw4.as_str()), Some(pinata4));
// (5) legacy provider key: no local field carries the dag-pb
// key, but the row and its claimed provenance stay visible so
// the repair gap is observable rather than hidden.
assert_pin(sha_legacy, true, false, None, None);
// Sanity: the `local_ipfs_provenance` column itself is what
// powers `local_pinned`, so the response field must agree
// with the database column. Reading directly avoids any
// confusion if the writer path changes.
let rows = sqlx::query("SELECT sha256_hex, local_ipfs_provenance FROM pinned_cids")
.fetch_all(&pool)
.await
.unwrap();
let mut db_provenance: std::collections::HashMap<String, bool> = Default::default();
for r in rows {
let sha: String = r.get("sha256_hex");
let p: bool = r.get("local_ipfs_provenance");
db_provenance.insert(sha, p);
}
for (sha, expected) in [
(sha_local.as_str(), true),
(sha_pinata_raw, false),
(sha_pinata_null, false),
(sha_dual.as_str(), true),
(sha_legacy, true),
] {
assert_eq!(
db_provenance.get(sha).copied(),
Some(expected),
"db column for {sha} does not match expected writer-owned provenance"
);
}
// Follow-through: every non-null local key in the response must
// resolve through `GET /ipfs/{cid}`. The local-only and dual rows
// above name real blobs on disk in a public repo, so both must
// serve their exact bytes; the Pinata-only and legacy rows carry
// no local key and have nothing to follow.
let follow_router = Router::new()
.route("/ipfs/{cid}", axum::routing::get(get_by_cid))
.with_state(state.clone());
for (raw, content) in [(&raw1, &content1[..]), (&raw4, &content2[..])] {
let resp = follow_router
.clone()
.oneshot(
axum::http::Request::builder()
.uri(format!("/ipfs/{raw}"))
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
axum::http::StatusCode::OK,
"advertised local key {raw} must resolve"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
assert_eq!(
body.as_ref(),
content,
"resolved bytes for {raw} must match the pinned blob"
);
}
}
/// #251 / CodeRabbit nit: cover `get_by_cid`'s DB-error conversion path — a
/// valid CID must still yield 503 on a closed pool.
#[sqlx::test]
@@ -8785,7 +8367,6 @@ mod tests {
#[tokio::test]
async fn get_by_cid_per_source_cap_sheds_same_source_admits_other() {
let mut state = crate::test_support::test_state_lazy();
state.db.pool().close().await;
// Global pool has room; the per-source cap is 1.
state.git_ipfs_walk_semaphore = Arc::new(Semaphore::new(8));
state.git_ipfs_walk_per_caller = crate::rate_limit::PerCallerConcurrency::new(1, 100);
@@ -8812,23 +8393,16 @@ mod tests {
"a source at its per-source /ipfs walk cap must shed 503 with global capacity free"
);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(body["error"], "overloaded");
// A DIFFERENT source is NOT shed by the per-source cap: it clears admission and
// proceeds to the closed DB, which has a distinct db_unavailable error code.
// proceeds (then errors on the lazy DB, which is not a 503).
let resp = ipfs_router(state)
.oneshot(get_cid(&cid, Some(other)))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(
body["error"],
crate::error::DB_UNAVAILABLE_CODE,
"a different source must clear admission and reach the closed database"
assert_ne!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"a different source must not be shed by the per-source cap"
);
}
+4 -18
View File
@@ -80,20 +80,15 @@ pub(crate) fn did_matches(a: &str, b: &str) -> bool {
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// Client-facing denial shared by every repo-owner gate copy. `require_repo_owner`
/// renders it as `AppError::Forbidden`; a handler that emits its own
/// `(StatusCode, Json)` envelope instead of `AppError` (e.g.
/// `api::tasks::create_task`) must use this constant too, so two copies of one
/// gate cannot drift in observable output.
pub(crate) const REPO_OWNER_ONLY_MESSAGE: &str = "only the repo owner can perform this action";
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
/// and the author check (close policy) share one normalization.
pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> {
if did_matches(caller, &record.owner_did) {
Ok(())
} else {
Err(AppError::Forbidden(REPO_OWNER_ONLY_MESSAGE.into()))
Err(AppError::Forbidden(
"only the repo owner can perform this action".into(),
))
}
}
@@ -247,11 +242,6 @@ mod authz_guard {
(events, "list_repo_events", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
// #496: create_task is signer-self AND owner-when-repo-scoped — a
// repo_id naming a hosted repo admits tasks only from its owner.
// Both halves are pinned: the did_matches row guards the signer
// binding, this one the repo-ownership gate.
(tasks, "create_task", "require_repo_owner("),
(tasks, "claim_task", "did_matches("),
(tasks, "complete_task", "did_matches("),
(tasks, "fail_task", "did_matches("),
@@ -351,11 +341,7 @@ mod authz_guard {
(
"create_task",
"createTask",
&[
"require_signer(ctx)?",
"did_matches(",
"require_repo_owner(",
],
&["require_signer(ctx)?", "did_matches("],
),
(
"claim_task",
File diff suppressed because it is too large. Load diff
+11 -123
View File
@@ -30,33 +30,6 @@ fn forbidden(msg: &str) -> (StatusCode, Json<Value>) {
)
}
/// This module's error shape with an opaque body: the real error is logged
/// server-side and never serialized (#250), matching the envelope
/// `AppError::Db` renders. Also mirrors its status split: pool-class failures
/// are a retryable 503 `db_unavailable`, everything else the opaque 500
/// `db_error`, so a transient outage is not reported as permanent.
fn db_error(e: anyhow::Error) -> (StatusCode, Json<Value>) {
tracing::error!(error = %format!("{e:#}"), "task database error");
if let Some(sql) = e.chain().find_map(|c| c.downcast_ref::<sqlx::Error>()) {
if crate::error::db_unavailable(sql) {
return (
StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})),
);
}
}
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})),
)
}
// ── Request / response types ──────────────────────────────────────────────────
#[derive(Deserialize)]
@@ -128,37 +101,6 @@ pub async fn create_task(
if !crate::api::did_matches(&auth.0, &body.delegator_did) {
return Err(forbidden("delegator_did must be the authenticated signer"));
}
// #496: a caller-supplied repo_id must name a repo the caller owns. Without
// this, any signed caller can plant a task — payload and UCAN included —
// under a foreign repo id, where the named assignee can claim it. Resolve
// the id against hosted repos: a hosted, non-quarantined repo admits tasks
// only from its owner (403 otherwise). An id naming no hosted repo
// (unknown, or quarantined — which is hidden as if it did not exist, so
// 403ing it would confirm a real id) is stored verbatim as an opaque
// label: this is a write gate and does not bind an unresolved id to a
// repo. Task reads are caller-independent on this base (the list endpoints
// are not repo-scoped), before and after this change. The 403-vs-201 split
// between a hosted repo id and an unknown one is itself a one-bit oracle —
// a stranger holding a private repo's id learns it is hosted here — but
// denying foreign repos requires drawing exactly that line, and repo ids
// already surface via the task list on this base. Repo-less tasks are
// unaffected.
if let Some(repo_id) = body.repo_id.as_deref() {
let record = state.db.get_repo_by_id(repo_id).await.map_err(db_error)?;
if let Some(record) = record {
let quarantined = state
.db
.is_repo_quarantined(&record.id)
.await
.map_err(db_error)?;
if !quarantined && crate::api::require_repo_owner(&record, &auth.0).is_err() {
// Same denial text the GraphQL copy relays from
// `require_repo_owner`, so the two copies of this gate do not
// disagree in observable output.
return Err(forbidden(crate::api::REPO_OWNER_ONLY_MESSAGE));
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -175,7 +117,12 @@ pub async fn create_task(
updated_at: now,
deadline: body.deadline,
};
let task = state.db.create_task(&task).await.map_err(db_error)?;
state.db.create_task(&task).await.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "error": e.to_string() })),
)
})?;
Ok((StatusCode::CREATED, Json(task_to_json(&task))))
}
@@ -228,28 +175,16 @@ pub async fn claim_task(
return Err(forbidden("assignee_did must be the authenticated signer"));
}
let task = state.db.claim_task(&id, &auth.0).await.map_err(|e| {
if e.downcast_ref::<crate::db::TaskReservedForOtherAssignee>()
.is_some()
{
return forbidden("task not claimable: reserved for another assignee");
}
(
StatusCode::CONFLICT,
Json(json!({ "error": e.to_string() })),
)
})?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.filter(|s| !s.is_empty())
.map(str::to_string)
.unwrap_or_else(|| auth.0.clone());
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did,
by_did: auth.0,
at: Utc::now().to_rfc3339(),
});
Ok(Json(task_to_json(&task)))
@@ -284,14 +219,14 @@ pub async fn complete_task(
})?;
if !crate::api::did_matches(
&auth.0,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
existing.assignee_did.as_deref().unwrap_or_default(),
) {
return Err(forbidden("only the task assignee can complete it"));
}
let by_did = auth.0;
let task = state
.db
.finish_task(&id, "completed", body.result.as_deref(), &by_did)
.finish_task(&id, "completed", body.result.as_deref())
.await
.map_err(|e| {
(
@@ -299,13 +234,6 @@ pub async fn complete_task(
Json(json!({ "error": e.to_string() })),
)
})?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.filter(|s| !s.is_empty())
.map(str::to_string)
.unwrap_or(by_did);
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -344,7 +272,7 @@ pub async fn fail_task(
})?;
if !crate::api::did_matches(
&auth.0,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
existing.assignee_did.as_deref().unwrap_or_default(),
) {
return Err(forbidden("only the task assignee can fail it"));
}
@@ -352,7 +280,7 @@ pub async fn fail_task(
let reason = body.reason.unwrap_or_default();
let task = state
.db
.finish_task(&id, "failed", Some(&reason), &by_did)
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(|e| {
(
@@ -360,13 +288,6 @@ pub async fn fail_task(
Json(json!({ "error": e.to_string() })),
)
})?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.filter(|s| !s.is_empty())
.map(str::to_string)
.unwrap_or(by_did);
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -376,36 +297,3 @@ pub async fn fail_task(
});
Ok(Json(task_to_json(&task)))
}
#[cfg(test)]
mod tests {
use super::*;
/// The gate's DB helper must mirror `AppError::Db`: a pool-level failure is
/// the retryable 503 `db_unavailable`, a query fault the opaque 500.
#[test]
fn db_error_matches_app_error_status_split() {
let unavailable = db_error(anyhow::Error::from(sqlx::Error::PoolTimedOut));
assert_eq!(unavailable.0, StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(unavailable.1 .0["error"], crate::error::DB_UNAVAILABLE_CODE);
assert_eq!(
unavailable.1 .0["message"],
crate::error::DB_UNAVAILABLE_MESSAGE
);
let query = db_error(anyhow::Error::from(sqlx::Error::Protocol(
"column \"machine_id\" does not exist".into(),
)));
assert_eq!(query.0, StatusCode::INTERNAL_SERVER_ERROR);
assert_eq!(query.1 .0["error"], "db_error");
assert_eq!(query.1 .0["message"], crate::error::DB_ERROR_MESSAGE);
assert!(!query.1 .0.to_string().contains("machine_id"));
}
/// `.context()` layers must not hide the sqlx error from the status split.
#[test]
fn db_error_sees_sqlx_through_context() {
let wrapped = anyhow::Error::from(sqlx::Error::PoolClosed).context("repo lookup failed");
assert_eq!(db_error(wrapped).0, StatusCode::SERVICE_UNAVAILABLE);
}
}
-1
View File
@@ -668,7 +668,6 @@ mod tests {
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new_bounded(12, Duration::from_secs(60), 10_000),
shutdown_tx: tokio::sync::watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
+10 -416
View File
@@ -11,74 +11,6 @@ use uuid::Uuid;
use crate::db::RefCertificate;
use crate::state::AppState;
/// Build the v1 signing payload — the EXACT bytes the node signs and
/// gl's `verify_signature` reconstructs. This is the single source
/// of truth shared by `issue_ref_certificate` and the frozen
/// canonical-form tests below; drift here is what would break every
/// existing cert.
///
/// Field order and key set are fixed: a default `serde_json::Value`
/// serializes `Map<String, Value>` with sorted keys, so any change to
/// the literal is observable as a different byte sequence on the
/// wire.
///
/// v1 is the unversioned legacy shape: no `version` key in the signed
/// JSON. From v2 onward the version belongs INSIDE the signed bytes
/// (see `v2_signing_payload`): leaving it as an unsigned sibling
/// would let a future v2 cert be downgraded to v1 and still verify
/// cleanly, since the bytes would be identical.
pub(crate) fn v1_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
})
}
/// Build the v2 signing payload — the future shape for versioned
/// certs. Identical to v1 except the wire `version` is part of the
/// signed bytes, so stripping or flipping the `version` column
/// invalidates the signature instead of verifying cleanly under the
/// other version's path.
///
/// NOT yet issued: `issue_ref_certificate` still stamps v1 because
/// the shipped `gl` verifier only supports v1. This builder exists
/// to pin the downgrade-resistant shape now, before any v2 cert is
/// ever signed.
#[allow(dead_code)]
pub(crate) fn v2_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
"version": 2,
})
}
/// Issue a signed ref-update certificate for a successful push.
///
/// Builds a canonical JSON payload, signs it with the node's Ed25519 key,
@@ -94,12 +26,16 @@ pub async fn issue_ref_certificate(
let node_did = state.node_did.to_string();
let issued_at = Utc::now().to_rfc3339();
// Build the canonical signing payload via the shared builder so
// the frozen-vector test below and the live signer cannot drift
// apart — a regression in either side fails both.
let payload = v1_signing_payload(
repo_id, ref_name, old_sha, new_sha, pusher_did, &node_did, &issued_at,
);
// Build the canonical signing payload.
let payload = serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
});
let payload_bytes = serde_json::to_vec(&payload)?;
let signature = state.node_keypair.sign_b64(&payload_bytes);
@@ -114,351 +50,9 @@ pub async fn issue_ref_certificate(
node_did,
signature,
issued_at,
// #26 Split PR 3: the wire-format version. New certs stamp
// v1 — the pre-versioning 7-field payload with no `version`
// key in the signed JSON — because the shipped `gl`
// verifier only supports v1. Stamping v2 here would issue
// certificates the shipped client refuses (round-3
// finding: node pinned `version == 2` while gl pinned
// "2 is refused", with nothing exercising the
// composition). v2 is reserved for the future shape
// defined by `v2_signing_payload`, which carries the
// version INSIDE the signed bytes; land the v2 verify
// path before stamping v2.
version: 1,
};
// Persist and return the row as it exists in the database (on a
// conflict the existing row survives when it is newer).
state.db.insert_ref_certificate(&cert).await
}
#[cfg(test)]
mod v1_payload_tests {
//! #26 Split PR 3 — cert payload version compat.
//!
//! The v1 cert payload is the frozen canonical byte form the
//! node signs. gl's `verify_signature` reconstructs the same
//! payload and verifies. Any drift in field order, key set, or
//! whitespace breaks every existing cert. The tests here pin
//! the v1 payload shape, the round-trip, the cert id
//! determinism per the v1 version, and the v2 read forward
//! compat (an unknown version refuses to verify rather than
//! guessing).
//!
//! Reviewer 1 finding: the live signer (`issue_ref_certificate`)
//! and the frozen-vector test (`v1_payload_matches_frozen_canonical_form`)
//! previously held two separate `serde_json::json!` literals,
//! so a regression in the signer could not fail this test.
//! Both now go through the shared `v1_signing_payload` builder.
use super::{v1_signing_payload, v2_signing_payload};
use crate::db::RefCertificate;
use gitlawb_core::identity::Keypair;
/// v1 payload is the same shape the legacy code signed, byte
/// for byte. Reverting this assertion to the v2 shape (which
/// includes a `version` key) is what the versioned format
/// forbids: an old client reading a v1 cert must not see a
/// `version` key in the signed JSON.
///
/// The literal is built via the shared `v1_signing_payload`
/// helper that `issue_ref_certificate` also uses, so the live
/// signer and the frozen vector share one source of truth.
#[test]
fn v1_payload_matches_frozen_canonical_form() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let frozen = format!(
r#"{{"new":"{}","node":"did:key:z6MkNode","old":"{}","pusher":"did:key:z6MkPusher","ref":"refs/heads/main","repo_id":"repo-1","ts":"2026-07-22T00:00:00+00:00"}}"#,
"a".repeat(40),
"0".repeat(40),
);
assert_eq!(serde_json::to_string(&payload).unwrap(), frozen);
}
/// Round-trip: sign the v1 payload with a keypair, build a
/// RefCertificate with version: 1, and verify the structure.
/// This is the construction the live code does on every push.
///
/// Uses the shared builder for both halves so a regression in
/// the live signer (adding/removing/renaming a field) cannot
/// pass this test by accident.
#[test]
fn v1_ref_certificate_structure_is_well_formed() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
);
let payload_bytes = serde_json::to_vec(&payload).unwrap();
let sig = kp.sign_b64(&payload_bytes);
let cert = RefCertificate {
id: "cert-id".into(),
repo_id: "repo-1".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "a".repeat(40),
pusher_did: "did:key:z6MkPusher".into(),
node_did: node_did.clone(),
signature: sig.clone(),
issued_at: "2026-07-22T00:00:00+00:00".into(),
version: 1,
};
assert_eq!(cert.version, 1, "new certs carry version: 1");
// The signed payload reconstructs identically: gl's
// verify_signature would build the same JSON, hash the
// same bytes, and verify the same signature.
let reconstructed = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let reconstructed_bytes = serde_json::to_vec(&reconstructed).unwrap();
assert_eq!(
reconstructed_bytes, payload_bytes,
"the round-trip serialization must be byte-identical; \
this is what makes gl's verify_signature succeed"
);
// Round 3 (P3 reviewer): byte-equality alone never calls
// `identity::verify`, so a bad signature helper would pass.
// Verify the computed signature against the reconstructed
// bytes — the same check gl performs.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &reconstructed_bytes, &sig_bytes)
.expect("well-formed v1 cert signature must verify");
}
/// Reviewer 1 finding: the live signer must share the
/// canonical-form literal so a regression in either side
/// fails both. This test pins the shared builder's output
/// against the literal a future v2 byte stream would have
/// to break — if someone adds a `version` key to
/// `v1_signing_payload`, every existing cert breaks verify,
/// and this test is one of the canaries that fires.
#[test]
fn v1_signing_payload_has_no_version_key() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let obj = payload.as_object().expect("payload is a JSON object");
assert!(
!obj.contains_key("version"),
"v1 signing payload must not carry a `version` key — adding \
one changes the bytes and breaks every existing cert's \
signature. v2 certs build a different payload, not this one"
);
// Pin the exact set so a future field addition is caught.
let expected: std::collections::BTreeSet<&str> =
["new", "node", "old", "pusher", "ref", "repo_id", "ts"]
.into_iter()
.collect();
let actual: std::collections::BTreeSet<&str> = obj.keys().map(String::as_str).collect();
assert_eq!(actual, expected, "v1 payload key set is frozen");
}
/// Round 3 (P2 reviewer): from v2 onward the version belongs
/// INSIDE the signed bytes. v1 stays the unversioned legacy
/// shape; v2 carries `"version": 2` in the payload so a
/// version flip without re-signing invalidates the signature
/// instead of verifying cleanly under the other path.
#[test]
fn v2_signing_payload_binds_version_and_resists_downgrade() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let ts = "2026-07-22T00:00:00+00:00";
let v1 = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2 = v2_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2_obj = v2.as_object().expect("v2 payload is a JSON object");
assert_eq!(
v2_obj.get("version"),
Some(&serde_json::json!(2)),
"v2 payload must carry the version inside the signed bytes"
);
let v1_bytes = serde_json::to_vec(&v1).unwrap();
let v2_bytes = serde_json::to_vec(&v2).unwrap();
assert_ne!(
v1_bytes, v2_bytes,
"v1 and v2 payloads must differ, otherwise a version flip is cryptographically invisible"
);
// A signature over v1 must NOT verify as v2 and vice versa.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_v1 = kp.sign_b64(&v1_bytes);
let sig_v1_bytes: [u8; 64] = URL_SAFE_NO_PAD.decode(&sig_v1).unwrap().try_into().unwrap();
assert!(
gitlawb_core::identity::verify(&vk, &v1_bytes, &sig_v1_bytes).is_ok(),
"v1 signature must verify under v1 bytes"
);
assert!(
gitlawb_core::identity::verify(&vk, &v2_bytes, &sig_v1_bytes).is_err(),
"a v1 signature presented as v2 (downgrade/upgrade flip) must not verify"
);
}
/// The live issuer stamps the version the shipped client can
/// verify. The gl client refuses to verify v2 certs explicitly;
/// this test pins that the version the ISSUING PATH claims
/// agrees with the payload shape it actually signs.
///
/// Round 2: the prior form built a `RefCertificate` literal with
/// `version: 1` and asserted `version == 1` — true no matter what the
/// signer does, so flipping the issuer's version (or its payload shape)
/// left it green. This one drives the actual
/// `issue_ref_certificate` so a regression in the issuer flips the test.
/// The DB row is created against `test_support::test_state`, which
/// gives us a real `AppState` with a real node keypair.
///
/// Round 3: the issuer stamps v1 because the shipped `gl`
/// verifier only supports v1 (stamping v2 shipped certs the
/// client refuses). The binding to the live issuer is kept —
/// flipping the stamp still turns this test red — without
/// changing what production issues.
#[sqlx::test]
#[allow(clippy::async_yields_async)]
async fn issuer_stamps_v1_over_v1_payload(pool: sqlx::PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let repo_id = "repo-issuer-observe";
let ref_name = "refs/heads/main";
let old = "0".repeat(40);
let new = "a".repeat(40);
let pusher = "did:key:z6MkPusher";
// Drive the live issuer and read the persisted row back by
// id, so the test covers the stored `version` column and
// not just the in-process return value.
let cert =
crate::cert::issue_ref_certificate(&state, repo_id, ref_name, &old, &new, pusher)
.await
.expect("issue_ref_certificate must succeed");
// The cert returned by the issuer is the source of truth
// for both the version claim and the signed bytes.
assert_eq!(cert.repo_id, repo_id);
assert_eq!(cert.ref_name, ref_name);
// Cross-check 1: the issuer's claimed version must match
// what the live function stamps. This assertion is bound
// to the live function, not a hand-built literal:
// flipping the stamp in `issue_ref_certificate` breaks it
// through the actual call path.
assert_eq!(
cert.version, 1,
"the live issuer must stamp v1 until a v2 verifier ships; \
flipping the stamp breaks this assertion through the actual call path"
);
// Cross-check 1b: the persisted row carries the same stamp.
let stored = state
.db
.get_ref_certificate(&cert.id)
.await
.expect("persisted cert must be readable")
.expect("persisted cert must exist");
assert_eq!(
stored.version, cert.version,
"the stored version column must match the issued claim"
);
// Cross-check 2: the signature on the cert must verify
// against the v1 payload builder's bytes, not against some
// other shape the issuer might have introduced. This is
// what gl's `verify_signature` does for a v1 cert: rebuild
// the payload from the cert's own fields and verify the
// signature. If the issuer ever signs a different shape
// while still claiming v1, every shipped client breaks —
// and so does this.
let rebuilt = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let vk = cert
.node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &serde_json::to_vec(&rebuilt).unwrap(), &sig_bytes)
.expect("a v1 cert's signature must verify against the v1 payload shape");
// Cross-check 3: the v1 signed payload must NOT contain a
// version key — v2 is the future shape that adds one inside
// the signed bytes (see `v2_signing_payload`). The v1
// payload (which the cert is signed over) is frozen at 7
// fields. A change that adds `version` to the v1 payload
// breaks every existing cert and this test.
let obj = rebuilt
.as_object()
.expect("rebuilt v1 payload is a JSON object");
assert!(
!obj.contains_key("version"),
"the v1 signing payload must not carry a version key — \
adding one changes the bytes and breaks every existing \
cert's signature"
);
}
}
+7 -261
View File
@@ -66,12 +66,6 @@ pub struct Config {
#[arg(long, env = "GITLAWB_PUBLIC_URL")]
pub public_url: Option<String>,
/// Base URL for the web view of repos on this node (e.g. https://gitlawb.com).
/// When set, `GET /` advertises it as `web_url` so the CLI can print a
/// working `View:` link. Omit for nodes that have no web front-end.
#[arg(long, env = "GITLAWB_WEB_URL")]
pub web_url: Option<String>,
/// Comma-separated list of bootstrap peer URLs to announce to on startup
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>,
@@ -152,17 +146,6 @@ pub struct Config {
#[arg(long, env = "GITLAWB_AUTO_SYNC", default_value_t = false)]
pub auto_sync: bool,
/// Enable the periodic reconciliation sweep that re-derives pin/seal sets
/// and fills durability gaps. Defaults to true; set to false to disable
/// the sweep even when a pin backend (IPFS/Pinata) is configured.
#[arg(
long,
env = "GITLAWB_RECONCILIATION_SWEEP",
default_value_t = true,
action = clap::ArgAction::Set
)]
pub reconciliation_sweep: bool,
/// Irys URL for Arweave permanent anchoring.
/// Leave empty to disable. Use https://devnet.irys.xyz for free devnet.
#[arg(long, env = "GITLAWB_IRYS_URL", default_value = "")]
@@ -820,13 +803,7 @@ impl Config {
/// Cross-field boot validation. Single-field ranges are enforced by clap; this
/// catches combinations that ship a denial-of-service under otherwise-valid
/// values. Call once at startup and fail fast on `Err`.
///
/// `&mut self` is required so the validator can normalize `web_url` to
/// its canonical form after validation: the configured value is
/// advertised on public `GET /`, and any consumer reading
/// `config.web_url` later must see the same string that was actually
/// validated.
pub fn validate(&mut self) -> Result<(), String> {
pub fn validate(&self) -> Result<(), String> {
// Concurrent git writes pin the dedicated lock pool for their whole
// duration (connection-affine advisory lock in acquire_write). The main
// pool no longer carries that occupancy.
@@ -943,101 +920,10 @@ impl Config {
.map_err(|e| e.to_string())?;
}
// GITLAWB_WEB_URL is advertised in GET / so the CLI can print a working
// View: link. Clap maps "" to Some("") which would produce a broken URL;
// reject early instead of serving a malformed link. Non-blank values must
// additionally parse as an absolute http(s) URL — the CLI appends
// `/{owner}/{repo}` to it, so scheme-less hosts ("gitlawb.com") and other
// garbage would render links no browser can follow.
if let Some(raw) = &self.web_url {
match validate_web_url(raw) {
Ok(()) => {}
Err(reason) => {
return Err(format!(
"GITLAWB_WEB_URL {reason} — \
set it to an absolute URL like https://gitlawb.com or leave it unset."
));
}
}
// Normalize to one canonical form so every consumer (the regular
// and degraded `GET /` handlers, anyone who reads
// `config.web_url` directly) sees the same string that was
// actually validated. The CLI's own defensive trim would paper
// over the divergence on the `View:` line, but that hides the
// bug from any other consumer that parses the value verbatim.
let canonical = raw
.trim()
.parse::<url::Url>()
.map(|u| u.to_string())
.map_err(|_| "is not a valid absolute URL".to_string())?;
if canonical != *raw {
self.web_url = Some(canonical);
}
}
Ok(())
}
}
/// Validate a `web_url` value for use as a browser-reachable base URL.
/// Blank values are rejected (clap maps `--web-url ""` to `Some("")`), and
/// non-blank values must parse as absolute `http`/`https` URLs with no
/// query, fragment, or userinfo — the CLI treats this as a string prefix
/// to append paths to, so anything else produces links no browser can
/// follow (`?a=1/owner/repo` puts the repo path inside the query string);
/// and userinfo is rejected outright because the configured value is
/// published on public `GET /` and reprinted on the CLI's `View:` line,
/// so an accidentally configured password would land in public metadata,
/// terminal scrollback, and CI logs.
pub(crate) fn validate_web_url(raw: &str) -> Result<(), String> {
if raw.trim().is_empty() {
return Err("must not be empty or whitespace-only".into());
}
let parsed: url::Url = raw
.trim()
.parse()
.map_err(|_| "is not a valid absolute URL".to_string())?;
match parsed.scheme() {
"http" | "https" => {}
other => return Err(format!("must use http or https, got '{other}:'")),
}
// url::Url cannot represent a URL without a host for http/https schemes, so
// reaching here guarantees an absolute browser-usable base.
if parsed.query().is_some() {
return Err(
"must not contain a query string — it is used as a path prefix for View links".into(),
);
}
if parsed.fragment().is_some() {
return Err(
"must not contain a fragment — it is used as a path prefix for View links".into(),
);
}
if !parsed.username().is_empty() || parsed.password().is_some() {
return Err(
"must not contain a username or password — it is advertised publicly and printed on the View: line"
.into(),
);
}
// Bidi/control bytes can hide behind percent-encoding: a literal scan of
// the wire form misses `.../%E2%80%AE` even though a browser decodes it
// into a reordering character on the rendered View: link. Check the
// decoded host and path before the value is advertised on GET / (INV-6).
let host = parsed.host_str().unwrap_or("");
let decoded_path = percent_encoding::percent_decode_str(parsed.path()).decode_utf8_lossy();
if host
.chars()
.chain(decoded_path.chars())
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c))
{
return Err(
"must not contain control or bidi characters, even percent-encoded — the View: line is terminal output"
.into(),
);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -1680,7 +1566,7 @@ mod tests {
.expect("default config must validate");
// An under-sized lock pool relative to the push cap is rejected.
let mut under_lock = Config::parse_from([
let under_lock = Config::parse_from([
"gitlawb-node",
"--db-lock-pool-max-connections",
"16",
@@ -1693,7 +1579,7 @@ mod tests {
);
// Exactly at the push cap without non-push headroom is rejected.
let mut push_only = Config::parse_from([
let push_only = Config::parse_from([
"gitlawb-node",
"--db-lock-pool-max-connections",
"32",
@@ -1706,7 +1592,7 @@ mod tests {
);
// Main pool can be smaller than pushes + headroom when the lock pool carries writes.
let mut split = Config::parse_from([
let split = Config::parse_from([
"gitlawb-node",
"--db-max-connections",
"16",
@@ -1721,153 +1607,13 @@ mod tests {
);
// Main pool still needs the app headroom floor.
let mut under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
let under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
assert!(
under_main.validate().is_err(),
"db_max_connections below DB_POOL_APP_HEADROOM must be rejected"
);
}
#[test]
fn web_url_rejects_empty_and_whitespace() {
// Unset is fine.
Config::parse_from(["gitlawb-node"])
.validate()
.expect("no web_url must validate");
// A real URL validates.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://gitlawb.com".into());
assert!(cfg.validate().is_ok());
// Empty string is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("".into());
assert!(cfg.validate().is_err(), "empty web_url must be rejected");
// Whitespace-only is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" ".into());
assert!(
cfg.validate().is_err(),
"whitespace-only web_url must be rejected"
);
}
#[test]
fn web_url_rejects_non_absolute_or_non_browser_urls() {
// Scheme-less host: parses nowhere, renders a broken View: link.
for bad in [
"gitlawb.com",
"not a url",
"ftp://files.example.com",
"javascript:alert(1)",
"https://", // no host
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(cfg.validate().is_err(), "web_url {bad:?} must be rejected");
}
// Query strings and fragments corrupt the appended repo path, since
// web_url is used as a raw string prefix (`?a=1` would swallow
// `/{owner}/{repo}` into the query).
for bad in [
"https://gitlawb.com?a=1",
"https://gitlawb.com/?utm_source=docs",
"https://gitlawb.com#section",
"https://gitlawb.com/#faq",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (query or fragment) must be rejected"
);
}
// Surrounding whitespace around a valid URL is tolerated.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://gitlawb.com ".into());
assert!(
cfg.validate().is_ok(),
"padded-but-valid web_url must validate"
);
// Percent-encoded bidi must not boot: the wire form carries no Cf
// code point, but a browser decodes %E2%80%AE into a reordering
// character on the rendered View: link (INV-6). Literal bidi and
// control bytes are rejected the same way.
for bad in [
"https://example.com/%E2%80%AE",
"https://example.com/\u{202e}spoof",
"https://example.com/%1b[31m",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (control or bidi, even encoded) must be rejected"
);
}
// Non-default ports and paths are fine.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("http://localhost:8080/ui".into());
assert!(cfg.validate().is_ok(), "port+path web_url must validate");
// Userinfo must be rejected: `web_url` is published on public
// `GET /` and reprinted on the CLI's `View:` line, so an
// accidentally configured password would land in public metadata,
// terminal scrollback, and CI logs. Reject username-only and
// user+password forms.
for bad in [
"https://[email protected]",
"https://user:[email protected]",
"https://:[email protected]",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (userinfo) must be rejected"
);
}
}
/// `validate()` accepts whitespace-padded input but the stored value
/// must match the canonical form actually validated, so every consumer
/// (the regular and degraded `GET /` handlers, anyone who reads
/// `config.web_url` directly) sees the same string. Without this, an
/// operator that sets `GITLAWB_WEB_URL=" https://git.example "` boots
/// a node that advertises `" https://git.example "` on public
/// metadata — different from what was validated, different from what
/// any other consumer would parse.
#[test]
fn web_url_is_normalized_to_canonical_form_after_validation() {
// Padded input is rewritten to the canonical form. `url::Url`'s
// own `to_string` is what we use for the canonicalization, and
// it always appends a path-separator slash to a bare-host URL,
// so the stored value is `https://git.example/` (which is also
// what every browser would normalize the operator-typed form to).
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://git.example ".into());
cfg.validate().unwrap();
assert_eq!(
cfg.web_url.as_deref(),
Some("https://git.example/"),
"stored web_url must be the canonical form, not the operator-typed one"
);
// Already-canonical input is left alone after a no-op
// re-normalization — same form comes back out, so any consumer
// reading the stored value sees a stable string across calls.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://git.example/".into());
cfg.validate().unwrap();
assert_eq!(cfg.web_url.as_deref(), Some("https://git.example/"));
}
/// The DECLARED default, read off the parser rather than out of a parse.
///
/// `Config::parse_from` consults the process environment, so on a host that
@@ -2132,7 +1878,7 @@ mod tests {
std::os::unix::fs::symlink(&target, &link).unwrap();
let key_path = link.join("p2p.key");
let mut config = Config::parse_from([
let config = Config::parse_from([
"gitlawb-node",
"--p2p-port",
"0",
@@ -2227,7 +1973,7 @@ mod tests {
let key_path = build(base.path());
let before = snapshot(base.path());
let mut config = Config::parse_from([
let config = Config::parse_from([
"gitlawb-node",
"--p2p-port",
"7546",
File diff suppressed because it is too large. Load diff
+2 -387
View File
@@ -6,7 +6,6 @@
use std::collections::{BTreeSet, HashMap};
use std::path::Path;
use std::str::FromStr;
use std::time::Duration;
use ed25519_dalek::VerifyingKey;
use gitlawb_core::did::Did;
@@ -128,60 +127,17 @@ fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&
/// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob
/// actually sealed and recorded this call (the per-push delta), used by Option B3
/// to anchor a manifest. Recipient identities are never stored or returned.
///
/// Nine args (the fence joins the seal's eight) but grouping them would churn
/// both callers and the race/hung-git tests for no behavioral gain.
#[allow(clippy::too_many_arguments)]
pub async fn encrypt_and_pin(
ipfs_api: &str,
repo_path: &Path,
db: &Db,
repo_id: &str,
node_seed: &[u8; 32],
git_bin: &str,
batch_budget: Duration,
recipients: &HashMap<String, BTreeSet<String>>,
fence: Option<&crate::ipfs_pin::PolicyFence>,
) -> Vec<(String, String)> {
let mut sealed = Vec::new();
let mut skipped_unresolvable = 0usize;
// One shared read deadline for the whole batch, like `pin_new_objects`: a
// hung git child is watchdog-reaped at this bound, so the outer
// `PIN_PHASE_DEADLINE` timeout cannot be held open by a blocking read
// (R1-P2). Each read runs under `spawn_blocking` — it is synchronous child
// spawn + pipe drain + watchdog join.
let read_deadline = std::time::Instant::now() + batch_budget;
let total = recipients.len();
for (attempted, (oid, dids)) in recipients.iter().enumerate() {
// Batch budget gate (R2-P3), mirroring the public pin loops: an object
// is never started with a remainder too small to cover a bounded read's
// teardown. This is consistency (the seal is bounded by the outer
// `PIN_PHASE_DEADLINE` either way), but it keeps the three loops from
// drifting apart in how they report a truncated batch.
if crate::ipfs_pin::batch_budget_gate(
"encrypted-seal",
read_deadline,
sealed.len(),
total - attempted,
)
.is_none()
{
break;
}
// Policy fence (R1-P1): the recipients snapshot was derived before the
// long withheld-blob walk; if a visibility rule moved while that walk
// ran (a reader added or removed), stop sealing instead of pinning to a
// stale recipient set. Checked FIRST so a changed policy costs nothing.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
oid = %oid,
"visibility policy changed after the recipients snapshot; stopping the seal loop"
);
break;
}
}
for (oid, dids) in recipients {
// A DB read failure is not a cache miss: re-sealing here would do an
// avoidable IPFS write during a partial outage. Skip and retry next push.
let stored_tag = match db.encrypted_blob_recipients_tag(repo_id, oid).await {
@@ -232,9 +188,7 @@ pub async fn encrypt_and_pin(
}
SealPlan::Seal { keys, tag } => (keys, tag),
};
let data = match read_object_bounded_spawn_blocking(git_bin, repo_path, oid, read_deadline)
.await
{
let data = match crate::git::store::read_object(repo_path, oid) {
Ok(Some((_t, bytes))) => bytes,
Ok(None) => {
tracing::warn!(oid = %oid, "git object not found; skipping encrypted pin");
@@ -252,22 +206,6 @@ pub async fn encrypt_and_pin(
continue;
}
};
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the tag lookup, recipient resolution, git read, or seal — all
// of which can take seconds. Without this, a reader removed during
// preparation can still receive a newly published envelope.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
oid = %oid,
"visibility policy changed during encrypted seal preparation; aborting upload"
);
break;
}
}
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
@@ -299,33 +237,10 @@ pub async fn encrypt_and_pin(
sealed
}
/// Bounded, reaped git object read for the seal loop, run off the async thread:
/// `read_object_bounded` is synchronous child spawn + pipe drain + watchdog
/// join, so blocking the runtime task on it would let a hung git hold a worker
/// thread (R1-P2). The `deadline` is the batch's shared read deadline; a child
/// still alive at it is SIGTERM/SIGKILL group-reaped by the watchdog.
async fn read_object_bounded_spawn_blocking(
git_bin: &str,
repo_path: &Path,
sha256_hex: &str,
deadline: std::time::Instant,
) -> anyhow::Result<Option<(String, Vec<u8>)>> {
let git_bin = git_bin.to_string();
let repo_path = repo_path.to_path_buf();
let sha256_hex = sha256_hex.to_string();
tokio::task::spawn_blocking(move || {
crate::git::store::read_object_bounded(&git_bin, &repo_path, &sha256_hex, deadline)
.map_err(anyhow::Error::from)
})
.await
.map_err(|e| anyhow::anyhow!("read_object spawn_blocking join failed: {e}"))?
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use std::time::Duration;
/// A did:key encoding a small-order point. Well-formed, indistinguishable
/// from a real one by eye, and refused at resolution since the choke-point
@@ -560,304 +475,4 @@ mod tests {
other => panic!("changed recipient set must re-seal; got {other:?}"),
}
}
/// A reader removed mid-seal must stop the seal loop (R1-P1 "race test for
/// reader removal"): `encrypt_and_pin` re-checks the policy fence before
/// each blob, so a `remove_visibility_rule` landing while the first seal is
/// in flight aborts before a later blob is pinned to a stale recipient set.
#[sqlx::test]
async fn encrypt_and_pin_stops_sealing_when_reader_removed_mid_batch(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("seal-race.git");
// Three loose blobs, each withheld (path-scoped deny exists so the sweep
// would have derived recipients for them).
let oids: Vec<String> = {
crate::git::store::init_bare(&repo_path).expect("init bare repo");
(0..3)
.map(|i| {
let mut cmd = std::process::Command::new("git");
cmd.args(["hash-object", "-w", "--stdin"])
.current_dir(&repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped());
let mut child = cmd.spawn().expect("spawn git hash-object");
{
use std::io::Write;
child
.stdin
.as_mut()
.expect("stdin")
.write_all(format!("secret blob {i}\n").as_bytes())
.expect("write stdin");
}
let out = child.wait_with_output().expect("hash-object output");
assert!(out.status.success());
String::from_utf8_lossy(&out.stdout).trim().to_string()
})
.collect()
};
// A real repos row so the fence has an epoch and a reader can be removed.
let now = chrono::Utc::now();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.clone(),
name: "seal-race-repo".into(),
owner_did: "did:key:zSealRaceOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.expect("create repo");
// A rule whose removal is the "reader removed" mutation: one reader per
// blob, all under the same path glob.
let reader = did_key(1);
db.set_visibility_rule(
&repo_id,
"**/secret/*",
crate::db::VisibilityMode::B,
std::slice::from_ref(&reader),
"did:key:zSealRaceOwner",
)
.await
.expect("set rule");
// IPFS endpoint that delays the FIRST add 2s so the removal lands while
// that seal is in flight, then answers immediately.
let endpoint = delaying_cid_endpoint(vec![Duration::from_secs(2)]).await;
let recipients: HashMap<String, BTreeSet<String>> = oids
.iter()
.cloned()
.map(|oid| {
let mut s = BTreeSet::new();
s.insert(reader.clone());
(oid, s)
})
.collect();
let fence = crate::ipfs_pin::PolicyFence::capture(&db, &repo_id)
.await
.expect("fence captures");
let sealed = tokio::time::timeout(Duration::from_secs(30), async {
let seal_db = db.clone();
let seal_repo = repo_path.clone();
let seal_endpoint = endpoint.clone();
let seal_repo_id = repo_id.clone();
let handle = tokio::spawn(async move {
encrypt_and_pin(
&seal_endpoint,
&seal_repo,
&seal_db,
&seal_repo_id,
&SEED,
"git",
Duration::from_secs(60),
&recipients,
Some(&fence),
)
.await
});
// Let the first add start (endpoint sleeps 2s), then remove the
// reader so the fence is stale before the loop checks again.
// The window has to cover blob 1's encrypt phase: if the removal
// lands before the loop reaches dispatch, the fence aborts with
// nothing sealed, so give the in-flight add room to start under
// a loaded box while staying inside the 2s endpoint delay.
tokio::time::sleep(Duration::from_millis(1200)).await;
db.remove_visibility_rule(&repo_id, "**/secret/*")
.await
.expect("remove rule");
handle.await.expect("seal task")
})
.await
.expect("wedge guard: the fence abort must not take 30s");
assert!(
sealed.len() < oids.len(),
"a reader removal landing mid-batch must abort before every blob is sealed: {}",
sealed.len()
);
assert!(
!sealed.is_empty(),
"at least the blob already in flight before the removal completes"
);
}
/// A hung git must not hold the seal loop past its read budget (R1-P2): the
/// git read runs under `spawn_blocking` against `read_object_bounded`, so
/// the watchdog reaps a wedged child at the batch deadline and the loop
/// keeps its shape instead of blocking a runtime worker indefinitely.
#[cfg(unix)]
#[sqlx::test]
async fn encrypt_and_pin_returns_by_budget_with_a_hung_git(pool: sqlx::PgPool) {
use std::os::unix::fs::PermissionsExt;
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("seal-hung.git");
let oids: Vec<String> = {
crate::git::store::init_bare(&repo_path).expect("init bare repo");
(0..2)
.map(|i| {
let mut cmd = std::process::Command::new("git");
cmd.args(["hash-object", "-w", "--stdin"])
.current_dir(&repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped());
let mut child = cmd.spawn().expect("spawn git hash-object");
{
use std::io::Write;
child
.stdin
.as_mut()
.expect("stdin")
.write_all(format!("secret blob {i}\n").as_bytes())
.expect("write stdin");
}
let out = child.wait_with_output().expect("hash-object output");
assert!(out.status.success());
String::from_utf8_lossy(&out.stdout).trim().to_string()
})
.collect()
};
// A git that wedges forever, ignoring SIGTERM, so only the watchdog's
// SIGKILL can reap it.
let fake = tmp.path().join("hanging-git");
std::fs::write(&fake, "#!/bin/sh\ntrap '' TERM\necho $$ > pid\nsleep 30\n").unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let now = chrono::Utc::now();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.clone(),
name: "seal-hung-repo".into(),
owner_did: "did:key:zSealHungOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.expect("create repo");
db.set_visibility_rule(
&repo_id,
"**/secret/*",
crate::db::VisibilityMode::B,
&[did_key(1)],
"did:key:zSealHungOwner",
)
.await
.expect("set rule");
let recipients: HashMap<String, BTreeSet<String>> = oids
.iter()
.cloned()
.map(|oid| {
let mut s = BTreeSet::new();
s.insert(did_key(1));
(oid, s)
})
.collect();
// Unreachable endpoint: even if a read somehow succeeded, the pin would
// fail; the read itself is the thing under test.
let started = std::time::Instant::now();
let sealed = tokio::time::timeout(
Duration::from_secs(60),
encrypt_and_pin(
"http://127.0.0.1:9",
&repo_path,
&db,
&repo_id,
&SEED,
fake.to_str().unwrap(),
Duration::from_secs(2),
&recipients,
None,
),
)
.await
.expect("a hung git must not hold the seal past the outer wedge guard");
let elapsed = started.elapsed();
assert!(
elapsed < Duration::from_secs(10),
"a hung git must be watchdog-reaped inside the read budget, not block the loop for ~10s+ (took {elapsed:?})"
);
assert!(
sealed.is_empty(),
"with a hung git no blob can be read, so nothing may be reported sealed"
);
}
/// Local TCP endpoint that answers `{ "Hash": "QmMock" }` after an optional
/// per-request delay, so a seal can be made to straddle a policy mutation.
async fn delaying_cid_endpoint(delays: Vec<Duration>) -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
let mut seen = 0usize;
while let Ok((mut sock, _)) = listener.accept().await {
let delay = *delays
.get(seen)
.or_else(|| delays.last())
.unwrap_or(&Duration::ZERO);
seen += 1;
tokio::spawn(async move {
let mut acc = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
acc.extend_from_slice(&buf[..n]);
if let Some(hdr_end) =
acc.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4)
{
let headers = String::from_utf8_lossy(&acc[..hdr_end]).to_lowercase();
let len: usize = headers
.lines()
.find_map(|l| l.strip_prefix("content-length:"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0);
if acc.len() >= hdr_end + len {
break;
}
}
}
tokio::time::sleep(delay).await;
let body = br#"{"Hash":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4"}"#;
let _ = sock
.write_all(
format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n", body.len())
.as_bytes(),
)
.await;
let _ = sock.write_all(body).await;
let _ = sock.flush().await;
});
}
});
endpoint
}
}
+1 -1
View File
@@ -101,7 +101,7 @@ pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
pub(crate) fn db_unavailable(e: &sqlx::Error) -> bool {
fn db_unavailable(e: &sqlx::Error) -> bool {
matches!(
e,
sqlx::Error::PoolTimedOut
+8 -15
View File
@@ -185,13 +185,10 @@ fn rev_list_delta(
/// List every object in the repository via
/// `git cat-file --batch-all-objects --batch-check='%(objectname)'`.
///
/// This is the whole-repo enumeration the push path falls back to. It
/// returns *all* objects (including unreachable/dangling ones), which is
/// what the push path needs to catch stragglers — do not swap it for a
/// reachability walk. The reconciliation sweep no longer uses it: sweep
/// candidates come from the windowed walk (window commits plus walked
/// pairs), which are reachable by construction, so dangling objects are
/// absent rather than filtered.
/// This is the whole-repo enumeration the push path falls back to and the
/// reconciliation sweep relies on. It returns *all* objects (including
/// unreachable/dangling ones), which is what the sweep needs to catch
/// stragglers — do not swap it for a reachability walk.
pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<Vec<String>> {
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
@@ -217,7 +214,6 @@ pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> R
/// filter needs to tell blobs (content, withholdable) and trees (content,
/// withholdable since #135/#173) from commits/tags (structural, never withheld)
/// without typing the candidate list itself.
#[allow(dead_code)] // used by tests and all_blob_oids
pub fn list_all_objects_with_type(
repo_path: &Path,
git_bin: &str,
@@ -265,7 +261,6 @@ pub fn all_blob_oids(
/// candidate blob or tree absent from the matching reachable visibility-allowed
/// set; a dangling or withheld object is in this universe but not the allowed
/// set, so it never replicates (#99, #172).
#[allow(dead_code)] // production went to the typed sweep path; tests still use it
pub fn all_blob_and_tree_oids(
repo_path: &Path,
git_bin: &str,
@@ -308,12 +303,10 @@ pub struct PinCandidateSet {
/// Every degraded path is **logged**, not silent: a full-scan fallback, a
/// failed full scan, and a panicked blocking task each emit a warning. On a
/// failed full scan or a task panic the candidate set is empty (pin nothing
/// this push); that is a durability gap the reconciliation sweep backstops
/// when it is enabled and a pin backend is configured (a node running with the
/// sweep disabled or with no IPFS/Pinata backend has no backstop), and it can
/// never leak because the withheld/fail-closed filter still runs on whatever
/// set is returned. `full_scan` rides on the returned set so the caller knows
/// when the dangling-inclusive filter is required.
/// this push); that is a durability gap the reconciliation sweep backstops, and
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
+4 -7
View File
@@ -118,7 +118,7 @@ impl RepoStore {
/// Test-only: every guard from this store parks in `release` right before the
/// `pg_advisory_unlock` await, until `gate` is notified. Dropping the future
/// while it is parked reproduces a client disconnect inside `release`.
#[cfg(all(test, unix))]
#[cfg(test)]
pub fn with_pre_unlock_gate(mut self, gate: Arc<tokio::sync::Notify>) -> Self {
self.pre_unlock_gate = Some(gate);
self
@@ -126,7 +126,7 @@ impl RepoStore {
/// Test-only: how many write guards from this store have reached the Tigris upload
/// site. See [`RepoWriteGuard::release`].
#[cfg(all(test, unix))]
#[cfg(test)]
pub fn tigris_upload_site_reached(&self) -> usize {
self.upload_site_reached
.load(std::sync::atomic::Ordering::SeqCst)
@@ -1717,11 +1717,8 @@ fn validate_repo_name(repo_name: &str) -> Result<()> {
if repo_name.is_empty() {
anyhow::bail!("repo_name is empty");
}
// Byte bound shared with the API-side validator via `MAX_REPO_NAME_LEN`:
// the two validators differ on charset by design, but must not drift on
// length, or an API-admitted name fails here after the proof is spent.
if repo_name.len() > crate::db::MAX_REPO_NAME_LEN {
anyhow::bail!("repo_name exceeds {} chars", crate::db::MAX_REPO_NAME_LEN);
if repo_name.len() > 100 {
anyhow::bail!("repo_name exceeds 100 chars");
}
// Repo names are `[A-Za-z0-9._-]+` minus path-traversal traps.
if repo_name.contains("..") {
-13
View File
@@ -273,9 +273,6 @@ pub struct TreeEntry {
///
/// Get just the object type. Returns `None` if the object doesn't exist; a
/// probe that could not examine the object store is `Err`, never `None`.
// Kept for tests and the bounded variants' docs; the async serve/seal paths use
// the `_bounded` forms.
#[allow(dead_code)]
pub fn object_type(repo_path: &Path, sha256_hex: &str) -> Result<Option<String>> {
let type_output = Command::new("git")
.args(["cat-file", "-t", sha256_hex])
@@ -308,7 +305,6 @@ pub fn object_type(repo_path: &Path, sha256_hex: &str) -> Result<Option<String>>
}
/// Read an object's content if its type is already known.
#[allow(dead_code)]
pub fn read_object_content(repo_path: &Path, sha256_hex: &str, obj_type: &str) -> Result<Vec<u8>> {
let content_output = Command::new("git")
.args(["cat-file", obj_type, sha256_hex])
@@ -741,7 +737,6 @@ pub fn read_object_bounded(
/// `gitlawb_core::cid::Cid::from_git_object_bytes`.
///
/// Returns `None` if the object does not exist in this repo.
#[allow(dead_code)]
pub fn read_object(repo_path: &Path, sha256_hex: &str) -> Result<Option<(String, Vec<u8>)>> {
let obj_type = match object_type(repo_path, sha256_hex)? {
Some(t) => t,
@@ -2108,16 +2103,8 @@ mod tests {
/// reports rather than wedges the suite, and the watchdog must escalate to SIGKILL to
/// reap it. The failure is a spawn/timeout of the reaped child, which is retryable, so
/// the variant is Transient.
///
/// Ignored on CI: pre-existing race unrelated to #218 (this file is
/// untouched by that work) — the test reads the fake's pid file with
/// no readiness handshake, so a slow scheduler yields "No such file
/// or directory" (stable-only red while the identical beta run
/// passes). Re-enable with a readiness signal from the fake instead
/// of the pid-file poll.
#[cfg(unix)]
#[test]
#[ignore]
fn read_object_bounded_returns_by_deadline_with_a_hung_git() {
use std::os::unix::fs::PermissionsExt;
use std::time::{Duration, Instant};
+1 -3
View File
@@ -627,9 +627,7 @@ impl TigrisClient {
}
/// Compress a bare repo directory into a tar.zst byte vector.
/// `pub(crate)` for sweep tests that seed a fake object store with a
/// byte-identical archive (rather than duplicating the format).
pub(crate) fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
let buf = Vec::new();
let encoder = zstd::stream::Encoder::new(buf, 3)?; // level 3 = fast + decent ratio
let mut tar = tar::Builder::new(encoder);
File diff suppressed because it is too large. Load diff
+2 -684
View File
@@ -3,7 +3,7 @@ pub mod query;
pub mod subscription;
pub mod types;
use async_graphql::{Schema, SchemaBuilder};
use async_graphql::Schema;
use std::sync::Arc;
use crate::db::Db;
@@ -14,22 +14,6 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
// Keep ordinary schema discovery and application queries usable while
// bounding validation work and the number of resolver selections per request.
const GRAPHQL_MAX_COMPLEXITY: usize = 400;
// The current public schema is shallow; this leaves headroom for composed
// clients and the canonical getIntrospectionQuery without allowing
// recursively nested documents to grow unchecked.
const GRAPHQL_MAX_DEPTH: usize = 14;
fn apply_query_limits<Query, Mutation, Subscription>(
builder: SchemaBuilder<Query, Mutation, Subscription>,
) -> SchemaBuilder<Query, Mutation, Subscription> {
builder
.limit_complexity(GRAPHQL_MAX_COMPLEXITY)
.limit_depth(GRAPHQL_MAX_DEPTH)
}
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
@@ -97,7 +81,7 @@ pub fn build_schema(
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>,
) -> GitlawbSchema {
apply_query_limits(Schema::build(QueryRoot, MutationRoot, SubscriptionRoot))
Schema::build(QueryRoot, MutationRoot, SubscriptionRoot)
.data(db)
.data(ref_update_tx)
.data(task_event_tx)
@@ -107,10 +91,6 @@ pub fn build_schema(
#[cfg(test)]
mod tests {
use super::*;
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
use crate::db::{MAX_VISIBLE_REPO_PAGE_SIZE, MAX_VISIBLE_TASK_PAGE_SIZE};
use async_graphql::{EmptyMutation, EmptySubscription, Object, Value};
use std::sync::atomic::{AtomicUsize, Ordering};
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
@@ -180,668 +160,6 @@ mod tests {
assert!(!err.message.contains("failed to open"));
}
fn production_test_schema() -> GitlawbSchema {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
production_test_schema_with_db(Arc::new(Db::for_testing(pool)))
}
fn production_test_schema_with_db(db: Arc<Db>) -> GitlawbSchema {
build_schema(
db,
tokio::sync::broadcast::channel(1).0,
tokio::sync::broadcast::channel(1).0,
)
}
#[tokio::test]
async fn ref_update_subscription_alias_budget() {
assert_subscription_alias_budget("refUpdates { repo }").await;
}
#[tokio::test]
async fn task_event_subscription_alias_budget() {
assert_subscription_alias_budget("taskEvents { taskId }").await;
}
async fn assert_subscription_alias_budget(field: &str) {
use futures::StreamExt;
use std::time::Duration;
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(
Arc::new(Db::for_testing(pool)),
ref_tx.clone(),
task_tx.clone(),
);
for count in [7, 8] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let query = format!("subscription {{ {fields} }}");
let mut responses = Box::pin(schema.execute_stream(query));
if count == 8 {
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.expect("over-budget subscriptions must reject before waiting for events")
.expect("the validation error must be returned");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
continue;
}
// Poll the accepted operation far enough to register its receivers.
assert!(
tokio::time::timeout(Duration::from_millis(30), responses.next())
.await
.is_err()
);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 7);
let _ = ref_tx.send(RefUpdateBroadcast {
repo: "owner/repo".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "1".repeat(40),
pusher_did: "did:key:test".into(),
node_did: "did:key:node".into(),
timestamp: "2026-01-01T00:00:00Z".into(),
owner_did: "did:key:test".into(),
});
let _ = task_tx.send(TaskEventBroadcast {
task_id: "task".into(),
old_status: "pending".into(),
new_status: "claimed".into(),
by_did: "did:key:test".into(),
at: "2026-01-01T00:00:00Z".into(),
});
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.unwrap()
.unwrap();
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_ne!(response.data, Value::Null);
drop(responses);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
}
}
#[sqlx::test]
async fn production_mutation_budget_accepts_seven_and_rejects_eight(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let caller = "did:key:reviewer";
let now = chrono::Utc::now().to_rfc3339();
for index in 0..8 {
db.create_task(&crate::db::AgentTask {
id: format!("task-{index}"),
repo_id: None,
kind: "test".into(),
status: "pending".into(),
delegator_did: caller.into(),
assignee_did: None,
capability: "test".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = production_test_schema_with_db(db.clone());
for count in [8, 7] {
let fields = (0..count)
.map(|n| {
format!("r{n}: claimTask(id: \"task-{n}\", assigneeDid: \"{caller}\") {{ id }}")
})
.collect::<Vec<_>>()
.join(" ");
let response = schema
.execute(
async_graphql::Request::new(format!("mutation {{ {fields} }}"))
.data(crate::auth::AuthenticatedDid(caller.into())),
)
.await;
if count == 8 {
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(
db.list_tasks(Some("pending"), None, 8).await.unwrap().len(),
8
);
continue;
}
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
7
);
assert_eq!(
db.list_tasks(Some("claimed"), Some(caller), 8)
.await
.unwrap()
.len(),
7
);
assert_eq!(
db.get_task("task-7").await.unwrap().unwrap().status,
"pending"
);
}
}
#[sqlx::test]
async fn production_list_cost_scales_with_the_same_alias_count(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
// Bounds are per-root named constants so the cases move with the
// documented maximums instead of going stale as literals.
for (field, selection, max) in [
("refUpdates", "repo", MAX_VISIBLE_REF_UPDATES),
("tasks", "id", MAX_VISIBLE_TASK_PAGE_SIZE),
(
"reposPage",
"nodes { name }",
MAX_VISIBLE_REPO_PAGE_SIZE as i64,
),
] {
// Two aliases at the maximum always exceed the budget.
let fields = (0..2)
.map(|n| format!("r{n}: {field}(limit: {max}) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
// At limit 1 two aliases fit for the row-priced roots; reposPage
// prices the full corpus scan, so a single alias is the most that
// fits under the budget.
let aliases = if field == "reposPage" { 1 } else { 2 };
let fields = (0..aliases)
.map(|n| format!("r{n}: {field}(limit: 1) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{field}: {:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
aliases
);
}
}
#[tokio::test]
async fn production_limits_reject_mutation_aliases_and_large_lists() {
let schema = production_test_schema();
// Limits are derived from the MAX_* constants so the boundaries move
// with the documented maximums instead of going stale as literals.
for (prefix, count, field) in [
("mutation", 8, "claimTask(id: \"missing\", assigneeDid: \"did:key:test\") { id }".to_owned()),
("mutation", 8, "createTask(delegatorDid: \"did:key:test\", input: { kind: \"test\", capability: \"test\" }) { id }".to_owned()),
("mutation", 8, "completeTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("mutation", 8, "failTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("query", 8, "task(id: \"missing\") { id }".to_owned()),
("query", 2, format!("refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }}")),
("query", 2, format!("tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id }}")),
("query", 2, format!("reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ nodes {{ name }} }}")),
("query", 8, "reposPage(limit: 1) { nodes { name } }".to_owned()),
] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{prefix} {{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
}
#[sqlx::test]
async fn production_repos_page_serves_the_documented_maximum(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
for selection in [
"nodes { name }",
"nodes { name ownerDid } hasNextPage endCursor",
] {
let response = schema
.execute(format!(
"{{ reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ {selection} }} }}"
))
.await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response.data.into_json().unwrap()["reposPage"]["nodes"],
serde_json::json!([])
);
}
}
#[tokio::test]
async fn seven_root_aliases_are_accepted() {
struct Root(Arc<AtomicUsize>);
#[Object]
impl Root {
#[graphql(complexity = "50 + child_complexity")]
async fn repos(&self) -> Vec<Nested> {
self.0.fetch_add(1, Ordering::Relaxed);
vec![Nested]
}
}
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
Root(calls.clone()),
EmptyMutation,
EmptySubscription,
))
.finish();
let fields = (0..7)
.map(|n| format!("r{n}: repos {{ value }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(calls.load(Ordering::Relaxed), 7);
}
#[tokio::test]
async fn expensive_root_aliases_are_rejected_before_database_access() {
// Use the production builder with a lazy pool: rejection must precede DB access.
let schema = production_test_schema();
let fields = (0..8)
.map(|n| format!("r{n}: repos {{ name }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.data, Value::Null);
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ordinary_schema_introspection_remains_available() {
let schema = production_test_schema();
let canonical_introspection = r#"
query IntrospectionQuery {
__schema {
queryType { name }
mutationType { name }
subscriptionType { name }
types {
...FullType
}
directives {
name
description
locations
args {
...InputValue
}
}
}
}
fragment FullType on __Type {
kind
name
description
fields(includeDeprecated: true) {
name
description
args {
...InputValue
}
type {
...TypeRef
}
isDeprecated
deprecationReason
}
inputFields {
...InputValue
}
interfaces {
...TypeRef
}
enumValues(includeDeprecated: true) {
name
description
isDeprecated
deprecationReason
}
possibleTypes {
...TypeRef
}
}
fragment InputValue on __InputValue {
name
description
type { ...TypeRef }
defaultValue
}
fragment TypeRef on __Type {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
}
}
}
}
}
}
}
}
"#;
let response = schema.execute(canonical_introspection).await;
assert!(
response.errors.is_empty(),
"canonical introspection query failed against production schema: {:?}",
response.errors
);
assert_ne!(response.data, Value::Null);
}
#[tokio::test]
async fn ref_updates_and_tasks_single_alias_max_limit_complexity_contract() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
pool.close().await;
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(Arc::new(Db::for_testing(pool)), ref_tx, task_tx);
// refUpdates(limit: MAX_VISIBLE_REF_UPDATES):
// Single-field selection has child_complexity 1 -> cost 100 + 200 + 1 = 301 <= 400.
// It passes validation and reaches the resolver (which yields db error on lazy pool).
let single_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }} }}"
))
.await;
assert!(
!single_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field refUpdates at max limit must pass complexity validation: {:?}",
single_ref.errors
);
assert!(
single_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field refUpdates at max limit must reach the resolver: {:?}",
single_ref.errors
);
// Two-field selection has child_complexity 2 -> cost 100 + 200 + 2 = 302 <= 400.
// The documented max-200 request stays reachable with multiple fields
// (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo refName }} }}"
))
.await;
assert!(
!multi_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field refUpdates at max limit must pass complexity validation: {:?}",
multi_ref.errors
);
assert!(
multi_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field refUpdates at max limit must reach the resolver: {:?}",
multi_ref.errors
);
// tasks(limit: MAX_VISIBLE_TASK_PAGE_SIZE):
// Single-field selection has child_complexity 1, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 1 <= 400.
// It passes validation and reaches the resolver.
let single_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id }} }}"
))
.await;
assert!(
!single_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field tasks at the maximum page size must pass complexity validation: {:?}",
single_task.errors
);
assert!(
single_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field tasks at the maximum page size must reach the resolver: {:?}",
single_task.errors
);
// Two-field selection has child_complexity 2, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 2 <= 400.
// A request at the documented maximum stays reachable with multiple
// fields (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id status }} }}"
))
.await;
assert!(
!multi_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field tasks at the maximum page size must pass complexity validation: {:?}",
multi_task.errors
);
assert!(
multi_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field tasks at the maximum page size must reach the resolver: {:?}",
multi_task.errors
);
}
#[tokio::test]
async fn two_repos_aliases_exceed_the_budget() {
// `repos` is priced as the full page it scans
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child), so two `repos { name }`
// aliases cost 2 * 251 = 502 > 400 and the document is rejected before
// any resolver runs. If the price reverts to `50 + child_complexity`
// the aliases cost 2 * 51 = 102, validation passes, and the failure
// mode changes (resolver DB errors on the lazy pool): this test goes
// red on revert.
let schema = production_test_schema();
let response = schema
.execute("{ r0: repos { name } r1: repos { name } }")
.await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn seven_repos_page_aliases_exceed_the_budget() {
// `reposPage` is priced as the full corpus scan
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity), so seven
// `reposPage(limit: 1) { nodes { name } }` aliases cost
// 7 * (50 + 200 + 2) = 1764 > 400 and the document is rejected
// before any resolver runs. Under the previous row-priced formula
// the same aliases cost 7 * (50 + 1 + 2) = 371, validation passes,
// and the failure mode changes (resolver DB errors on the lazy
// pool): this test goes red on revert. Eight aliases are rejected
// under both formulas, so seven is the load-bearing boundary.
let schema = production_test_schema();
let fields = (0..7)
.map(|n| format!("r{n}: reposPage(limit: 1) {{ nodes {{ name }} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ref_updates_fixed_base_is_charged() {
// The 100 base in the refUpdates complexity formula charges for the
// collector's fixed work (loading the deduped repo set, the
// quarantined set, and the visibility rules, then walking up to 16
// keyset pages) even at limit: 1. Four aliases at limit: 1 cost
// 4 * (100 + 1 + 1) = 408 > 400 and must be rejected. If the base
// disappears, the cost drops to 4 * (1 + 1) = 8 and this test fails.
let schema = production_test_schema();
let fields = (0..4)
.map(|n| format!("r{n}: refUpdates(limit: 1) {{ repo }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[derive(Clone, Copy)]
struct Nested;
#[Object]
impl Nested {
async fn child(&self) -> Nested {
Nested
}
async fn value(&self) -> i32 {
1
}
}
struct CountingQuery(Arc<AtomicUsize>);
#[Object]
impl CountingQuery {
async fn nested(&self) -> Nested {
self.0.fetch_add(1, Ordering::Relaxed);
Nested
}
}
#[tokio::test]
async fn query_depth_limit_accepts_fourteen_and_rejects_fifteen() {
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
CountingQuery(Arc::clone(&calls)),
EmptyMutation,
EmptySubscription,
))
.finish();
let query_at_depth = |depth: usize| {
let selection = (2..depth).fold("value".to_string(), |selection, _| {
format!("child {{ {selection} }}")
});
format!("{{ nested {{ {selection} }} }}")
};
let accepted = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH)).await;
assert!(
accepted.errors.is_empty(),
"depth {GRAPHQL_MAX_DEPTH} should be accepted: {:?}",
accepted.errors
);
assert_eq!(calls.swap(0, Ordering::Relaxed), 1);
let rejected = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH + 1)).await;
assert_eq!(rejected.data, Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
assert_eq!(calls.load(Ordering::Relaxed), 0);
}
#[tokio::test]
async fn production_schema_enforces_depth_limit() {
let schema = production_test_schema();
// Depth 14 is accepted through the production build_schema path.
let of_types_14 = (0..9).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query_14 =
format!("{{ __schema {{ types {{ fields {{ type {{ {of_types_14} }} }} }} }} }}");
let accepted = schema.execute(&query_14).await;
assert!(accepted.errors.is_empty(), "{:?}", accepted.errors);
// Construct an introspection query of depth 15 using __schema.
// 1: __schema
// 2: types
// 3: fields
// 4: type
// 5..14: ofType (10 times)
// 15: name
// Total depth = 15 > GRAPHQL_MAX_DEPTH (14).
let of_types = (0..10).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query = format!("{{ __schema {{ types {{ fields {{ type {{ {of_types} }} }} }} }} }}");
let rejected = schema.execute(&query).await;
assert_eq!(rejected.data, async_graphql::Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
+7 -419
View File
@@ -22,7 +22,6 @@ pub struct MutationRoot;
#[Object]
impl MutationRoot {
#[graphql(complexity = "50 + child_complexity")]
async fn create_task(
&self,
ctx: &Context<'_>,
@@ -37,33 +36,6 @@ impl MutationRoot {
}
let delegator_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
// #496: same write gate as the REST handler — a repo_id naming a
// hosted, non-quarantined repo admits tasks only from its owner, so a
// stranger cannot file under a foreign repo id. An id naming no hosted
// repo (unknown, or quarantined — hidden as if it did not exist) is
// stored verbatim as an opaque label; the gate does not bind an
// unresolved id to a repo. Task reads are caller-independent on this
// base (the tasks resolver is not repo-scoped), before and after this
// change. As in tasks.rs, the success-vs-denial split between a hosted
// repo id and an unknown one is itself a one-bit oracle — a stranger
// holding a private repo's id learns it is hosted here — but denying
// foreign repos requires drawing exactly that line.
if let Some(repo_id) = input.repo_id.as_deref() {
let record = db
.get_repo_by_id(repo_id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if let Some(record) = record {
let quarantined = db
.is_repo_quarantined(&record.id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if !quarantined {
crate::api::require_repo_owner(&record, caller)
.map_err(crate::graphql::graphql_app_err)?;
}
}
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
@@ -80,17 +52,12 @@ impl MutationRoot {
updated_at: now,
deadline: input.deadline,
};
// Return the stored row, not the pre-insert input: create_task
// normalizes the assignee (whitespace-only becomes NULL/open), and a
// response that shows the raw input lies about the reservation.
let stored = db
.create_task(&task)
db.create_task(&task)
.await
.map_err(crate::graphql::graphql_db_err)?;
Ok(AgentTaskType::from(stored))
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn claim_task(
&self,
ctx: &Context<'_>,
@@ -110,25 +77,16 @@ impl MutationRoot {
.claim_task(&id, &assignee_did)
.await
.map_err(crate::graphql::graphql_db_err)?;
// Report the stored assignee form like REST, so subscribers see the
// same identity whichever API surface ran the transition.
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(assignee_did.as_str())
.to_string();
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did,
by_did: assignee_did,
at: Utc::now().to_rfc3339(),
});
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn complete_task(
&self,
ctx: &Context<'_>,
@@ -152,24 +110,15 @@ impl MutationRoot {
.await
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
"only the task assignee can complete it",
));
}
let task = db
.finish_task(&id, "completed", input.result.as_deref(), caller)
.finish_task(&id, "completed", input.result.as_deref())
.await
.map_err(crate::graphql::graphql_db_err)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(by_did.as_str())
.to_string();
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -180,7 +129,6 @@ impl MutationRoot {
Ok(AgentTaskType::from(task))
}
#[graphql(complexity = "50 + child_complexity")]
async fn fail_task(
&self,
ctx: &Context<'_>,
@@ -203,25 +151,16 @@ impl MutationRoot {
.await
.map_err(crate::graphql::graphql_db_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
"only the task assignee can fail it",
));
}
let reason = input.reason.unwrap_or_default();
let task = db
.finish_task(&id, "failed", Some(&reason), caller)
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(crate::graphql::graphql_db_err)?;
let by_did = task
.assignee_did
.as_deref()
.map(crate::db::trim_assignee_did)
.unwrap_or(by_did.as_str())
.to_string();
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -247,22 +186,6 @@ mod tests {
.join("; ")
}
/// The `id` scalar returned by a successful `createTask` mutation.
fn created_id(resp: &Response) -> String {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::Object(task) = obj.get("createTask").expect("createTask key")
else {
panic!("createTask not an object");
};
let async_graphql::Value::String(id) = task.get("id").expect("id key") else {
panic!("id not a string");
};
id.clone()
}
/// N2: every GraphQL mutation requires a verified signer and binds the acting
/// DID to it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate.
@@ -430,339 +353,4 @@ mod tests {
errors(&resp)
);
}
/// #496 (GraphQL): createTask applies the same repo-ownership gate as the
/// REST handler — a stranger naming a hosted repo is rejected without
/// persisting anything, while the owner files. The quarantined and
/// unknown-id arms are pinned too, including that each accepted write
/// stores the supplied id verbatim, so this copy of the gate cannot regress
/// silently.
#[sqlx::test]
async fn create_task_rejects_foreign_repo_id(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let owner = "did:key:zGQLTASKOWNERAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zGQLTASKSTRANGERBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
)
};
// Stranger signs as themselves (so the signer binding passes) but
// names the victim's repo → rejected by the ownership gate, leaking
// nothing about the repo.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::api::REPO_OWNER_ONLY_MESSAGE),
"a non-owner must not file under a foreign repo id (shared denial text): {errs}"
);
// The denial must not persist: no task row may carry the victim's
// repo id afterwards.
let stored = state
.db
.list_tasks(None, None, 50)
.await
.expect("list tasks");
assert!(
stored
.iter()
.all(|t| t.repo_id.as_deref() != Some(repo.id.as_str())),
"a denied write must not leave a task under the foreign repo id"
);
// The owner files under their own repo id.
let resp = schema
.execute(Request::new(q(owner)).data(AuthenticatedDid(owner.into())))
.await;
assert!(
errors(&resp).is_empty(),
"the owner should file against their repo: {}",
errors(&resp)
);
// Unknown repo ids stay accepted as opaque labels.
let q_unknown = format!(
r#"mutation {{ createTask(delegatorDid: "{stranger}", input: {{ kind: "build", capability: "repo:write", repoId: "no-such-repo-id" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(q_unknown).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"an id naming no hosted repo stays an opaque label: {}",
errors(&resp)
);
// Read back the row: the unresolved id must be stored verbatim, not
// dropped or rewritten to a repo it never named.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some("no-such-repo-id"),
"an unresolved id must be stored verbatim as a label"
);
// A quarantined repo id is treated as unknown (accepted, never a 403
// that would confirm the id names a real repo). This pins the
// `!quarantined` arm on this copy of the gate.
state
.db
.set_repo_quarantine(&repo.id, true)
.await
.expect("quarantine");
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"a quarantined repo id must not 403 (existence oracle): {}",
errors(&resp)
);
// Read back the row: the quarantined id is stored verbatim as a label
// too, so the accepted write is not silently dropped.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some(repo.id.as_str()),
"a quarantined id must be stored verbatim as a label"
);
}
/// #496 (GraphQL): a repo-lookup fault in createTask's gate must stay
/// opaque. Dropping a column `get_repo_by_id` selects faults that lookup;
/// the resolver must return the fixed DB message and leak no schema text
/// (#250), as `create_task_db_error_message_is_opaque` pins for the
/// agent_tasks fault.
#[sqlx::test]
async fn create_task_gate_repo_lookup_fault_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner = "did:key:zGQLTASKFAULTOWNERAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-fault-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-fault-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
sqlx::query("ALTER TABLE repos DROP COLUMN machine_id")
.execute(&pool)
.await
.expect("drop machine_id");
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{owner}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(owner.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"gate sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("machine_id") && !errs.contains("column"),
"schema text leaked through the gate: {errs}"
);
}
/// Pre-assigned GraphQL claimTask must reject a stranger (no UCAN leak)
/// and admit the reserved assignee.
#[sqlx::test]
async fn claim_task_honors_preassigned_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let reserved = "did:key:zGQLRESERVEDAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let thief = "did:key:zGQLTHIEFBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-reserved".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: Some(reserved.into()),
capability: "repo:write".into(),
ucan_token: Some("gql-ucan-secret".into()),
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ claimTask(id: "task-gql-reserved", assigneeDid: "{actor}") {{ id status ucanToken }} }}"#
)
};
let resp = schema
.execute(Request::new(q(thief)).data(AuthenticatedDid(thief.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("reserved"),
"stranger claim must surface reserved deny distinctly: {errs}"
);
assert!(
!errs.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL errors: {errs}"
);
if let Ok(data) = resp.data.into_json() {
let s = data.to_string();
assert!(
!s.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL data on failed claim: {s}"
);
}
// Parity with REST: failed steal must leave the row untouched.
let still = state
.db
.get_task("task-gql-reserved")
.await
.expect("get")
.expect("task exists");
assert_eq!(
still.status, "pending",
"status must stay pending after steal"
);
assert_eq!(
still.assignee_did.as_deref(),
Some(reserved),
"assignee_did must stay reserved after steal"
);
let resp = schema
.execute(Request::new(q(reserved)).data(AuthenticatedDid(reserved.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim: {}",
errors(&resp)
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["claimTask"]["status"], "claimed");
assert_eq!(data["claimTask"]["ucanToken"], "gql-ucan-secret");
}
/// create_task normalizes a whitespace-only assignee to NULL (open task),
/// so the create response must show the stored row, not echo the input: a
/// client trusting the echoed "reservation" would believe a claimable task
/// is reserved. Before the fix the response was built from the pre-insert
/// struct and returned the padded input verbatim.
#[sqlx::test]
async fn create_task_reports_stored_assignee_not_input(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let caller = "did:key:zGQLCREATORAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let schema = state.graphql_schema.as_ref();
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{caller}", input: {{ kind: "build", capability: "repo:write", assigneeDid: " " }}) {{ id assigneeDid }} }}"#
);
let resp = schema
.execute(Request::new(q).data(AuthenticatedDid(caller.into())))
.await;
let data = resp.data.into_json().expect("json data");
assert!(
data["createTask"]["assigneeDid"].is_null(),
"whitespace-only assigneeDid must come back null (stored shape): {data}"
);
}
/// A task seeded with a whitespace-padded assignee must still be
/// completable by that assignee over GraphQL: REST and Db::finish_task
/// trim the stored value before comparing, and the GraphQL gate must do
/// the same or padded reservations finish only through REST.
#[sqlx::test]
async fn graphql_finish_accepts_padded_stored_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let assignee = "did:key:zGQLPADDEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-padded".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORDDDDDDDDDDDDDDDDDDDDDDDDDD".into(),
assignee_did: Some(format!("\t{assignee}\n")),
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let claim = format!(
r#"mutation {{ claimTask(id: "task-gql-padded", assigneeDid: "{assignee}") {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(claim).data(AuthenticatedDid(assignee.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim a padded reservation: {}",
errors(&resp)
);
let finish = format!(
r#"mutation {{ completeTask(id: "task-gql-padded", byDid: "{assignee}", input: {{}}) {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(finish).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.is_empty(),
"assignee must complete a padded reservation: {errs}"
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["completeTask"]["status"], "completed");
}
}
+35 -814
View File
@@ -1,154 +1,57 @@
use async_graphql::{Context, Object, Result};
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
use std::sync::Arc;
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
use crate::db::{Db, RepoRecord, MAX_VISIBLE_REPO_PAGE_SIZE, MAX_VISIBLE_TASK_PAGE_SIZE};
use crate::db::Db;
use super::types::{AgentTaskType, RefUpdateType, RepoPageType, RepoType};
fn repo_type(repo: RepoRecord) -> RepoType {
RepoType {
name: repo.name,
owner_did: repo.owner_did,
description: repo.description,
default_branch: repo.default_branch,
created_at: repo.created_at.to_rfc3339(),
}
}
fn repo_cursor(repo: &RepoRecord) -> String {
URL_SAFE_NO_PAD.encode(
serde_json::to_vec(&(&repo.owner_did, &repo.name))
.expect("a pair of strings is JSON serializable"),
)
}
fn parse_repo_cursor(cursor: &str) -> Result<(String, String)> {
// A position contains only public response fields, not a database id or
// authority. Every page re-evaluates visibility for the current caller.
if cursor.len() > 4096 {
return Err(async_graphql::Error::new("invalid repository cursor"));
}
let bytes = URL_SAFE_NO_PAD
.decode(cursor)
.map_err(|_| async_graphql::Error::new("invalid repository cursor"))?;
let position: (String, String) = serde_json::from_slice(&bytes)
.map_err(|_| async_graphql::Error::new("invalid repository cursor"))?;
// PostgreSQL text cannot contain NUL. Reject it as malformed input before
// binding either cursor component, rather than reporting a database error.
if position.0.contains('\0') || position.1.contains('\0') {
return Err(async_graphql::Error::new("invalid repository cursor"));
}
Ok(position)
}
use super::types::{AgentTaskType, RefUpdateType, RepoType};
pub struct QueryRoot;
#[Object]
impl QueryRoot {
// DB-backed roots carry a base cost so aliases consume the request budget
// even when each alias selects only one inexpensive response field.
// `repos` performs the same full corpus scan as a max-size `reposPage`
// (list_visible_repos_page with MAX_VISIBLE_REPO_PAGE_SIZE + 1), so it
// is priced as the full page it is, not as a cheap alias.
#[graphql(complexity = "50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity")]
/// Complete visible repository list, up to the documented maximum number
/// of entries. Larger lists must use reposPage; this field returns an
/// error instead of truncating silently.
async fn repos(&self, ctx: &Context<'_>) -> Result<Vec<RepoType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
let repos = db
.list_all_repos_deduped()
.await
.map_err(crate::graphql::graphql_db_err)?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
// threaded onto the context by optional_signature; absent = anonymous.
let caller = ctx
.data::<crate::auth::AuthenticatedDid>()
.ok()
.map(|d| d.0.as_str());
let mut repos = db
.list_visible_repos_page(caller, None, MAX_VISIBLE_REPO_PAGE_SIZE + 1)
let ids: Vec<String> = repos.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
.await
.map_err(crate::graphql::graphql_db_err)?;
if repos.len() > MAX_VISIBLE_REPO_PAGE_SIZE {
return Err(async_graphql::Error::new(format!(
"repository list exceeds {} entries; use reposPage with limit and after",
MAX_VISIBLE_REPO_PAGE_SIZE
)));
}
// Preserve the legacy activity ordering for complete, small lists.
repos.sort_by_key(|repo| std::cmp::Reverse(repo.updated_at));
Ok(repos.into_iter().map(repo_type).collect())
Ok(repos
.into_iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, caller)
})
.map(|r| RepoType {
name: r.name,
owner_did: r.owner_did,
description: r.description,
default_branch: r.default_branch,
created_at: r.created_at.to_rfc3339(),
})
.collect())
}
/// Bounded visible repositories ordered by owner and name. Continue with
/// endCursor while hasNextPage is true. Pages are not a database snapshot.
// Every page sorts the full deduplicated corpus under COLLATE "C"
// (list_visible_repos_page documents the accepted O(total repos) bound),
// so the price charges the scan, not the emitted rows: a limit:1 page
// costs the same as a max-size page, the way `repos` is priced.
#[graphql(complexity = "50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity")]
async fn repos_page(
&self,
ctx: &Context<'_>,
#[graphql(
default = 50,
desc = "Page size from 1 to the documented maximum; other values are rejected."
)]
limit: i64,
after: Option<String>,
) -> Result<RepoPageType> {
if !(1..=MAX_VISIBLE_REPO_PAGE_SIZE as i64).contains(&limit) {
return Err(async_graphql::Error::new(format!(
"limit must be between 1 and {}",
MAX_VISIBLE_REPO_PAGE_SIZE
)));
}
let after = after.as_deref().map(parse_repo_cursor).transpose()?;
let caller = ctx
.data::<crate::auth::AuthenticatedDid>()
.ok()
.map(|d| d.0.as_str());
let db = ctx.data_unchecked::<Arc<Db>>();
let mut repos = db
.list_visible_repos_page(
caller,
after
.as_ref()
.map(|(owner, name)| (owner.as_str(), name.as_str())),
limit as usize + 1,
)
.await
.map_err(crate::graphql::graphql_db_err)?;
let has_next_page = repos.len() > limit as usize;
repos.truncate(limit as usize);
let end_cursor = repos.last().map(repo_cursor);
Ok(RepoPageType {
nodes: repos.into_iter().map(repo_type).collect(),
has_next_page,
end_cursor,
})
}
// Complexity is additive with a base reflecting the collector's fixed scan:
// every request loads the full deduped repo set, the quarantined set, and
// the visibility rules, then walks up to max(limit, 2048) event rows past
// withheld ones, regardless of `limit`. A multiplicative price both
// undercharged limit:1 (51 for that fixed work) and made a request at the
// documented maximum unreachable (two fields at max limit cost 450 > 400).
//
// Decision on the collector floor (review #465, 3rd round): the up-to-2048
// walk is bounded constant overhead that does not scale with `limit`, so
// it is priced in the fixed base, not the limit term; the limit term
// prices only the emitted rows. Pricing the walk at face value in the base
// would put every refUpdates query over the 400 budget and make the field
// unusable, so the base prices that fixed scan as a lump sum instead.
#[graphql(
complexity = "100 + (limit.clamp(0, MAX_VISIBLE_REF_UPDATES) as usize) + child_complexity"
)]
async fn ref_updates(
&self,
ctx: &Context<'_>,
repo: Option<String>,
#[graphql(
default = 20,
desc = "Requests above the documented maximum return the newest rows up to that maximum (no continuation cursor)."
desc = "Max 200; larger requests return the newest 200 rows (no continuation cursor)."
)]
limit: i64,
) -> Result<Vec<RefUpdateType>> {
@@ -202,14 +105,6 @@ impl QueryRoot {
Ok(resolved)
}
// Complexity is additive: list_tasks is a single LIMIT-bounded query whose
// cost does not scale with the selected fields, so the price is the base
// plus the clamped limit plus the child fields. A multiplicative price
// made a request at the documented maximum unreachable (two fields at max
// limit cost 450 > 400).
#[graphql(
complexity = "50 + (limit.clamp(0, MAX_VISIBLE_TASK_PAGE_SIZE) as usize) + child_complexity"
)]
async fn tasks(
&self,
ctx: &Context<'_>,
@@ -217,7 +112,7 @@ impl QueryRoot {
assignee_did: Option<String>,
#[graphql(
default = 50,
desc = "Requests above the documented maximum are clamped to it (no error). Negative values clamp to 0."
desc = "Max 200; larger requests are clamped to 200 (no error). Negative values clamp to 0."
)]
limit: i64,
) -> Result<Vec<AgentTaskType>> {
@@ -225,7 +120,7 @@ impl QueryRoot {
// Clamp before SQL: a negative LIMIT is a client fault that Postgres
// rejects with 2201W, which would otherwise trip the opaque DB path
// and write an error-level log on every probe (#250 review).
let limit = limit.clamp(0, MAX_VISIBLE_TASK_PAGE_SIZE);
let limit = limit.clamp(0, 200);
let tasks = db
.list_tasks(status.as_deref(), assignee_did.as_deref(), limit)
.await
@@ -233,7 +128,6 @@ impl QueryRoot {
Ok(tasks.into_iter().map(AgentTaskType::from).collect())
}
#[graphql(complexity = "50 + child_complexity")]
async fn task(&self, ctx: &Context<'_>, id: String) -> Result<Option<AgentTaskType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
let t = db
@@ -247,642 +141,12 @@ impl QueryRoot {
#[cfg(test)]
mod tests {
use crate::db::{Db, ReceivedRefUpdate, RepoRecord};
use base64::Engine;
use chrono::Utc;
use sqlx::PgPool;
use std::sync::Arc;
const OWNER: &str = "did:key:z6MkOwner";
#[sqlx::test]
async fn repos_legacy_rejects_overflow_and_pages_reach_every_visible_repo(pool: PgPool) {
let db = db(pool).await;
let total = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE + 5;
for index in 0..total {
let name = format!("repo-{index:03}");
db.create_repo(&repo(&name, OWNER, &name, true))
.await
.unwrap();
}
// The SQL helper itself must bound materialization, even for a caller
// accidentally requesting an unlimited page.
assert_eq!(
db.list_visible_repos_page(None, None, usize::MAX)
.await
.unwrap()
.len(),
crate::db::MAX_VISIBLE_REPO_PAGE_SIZE + 1
);
let schema = schema(db);
let legacy = anon(&schema, "{ repos { name } }").await;
assert_eq!(legacy.errors.len(), 1);
assert!(legacy.errors[0].message.contains("use reposPage"));
assert_eq!(legacy.data, async_graphql::Value::Null);
let mut cursor = None;
let mut names = Vec::new();
loop {
let response = schema.execute(
async_graphql::Request::new(
"query($after: String) { reposPage(limit: 50, after: $after) { nodes { name } hasNextPage endCursor } }",
).variables(async_graphql::Variables::from_json(serde_json::json!({"after": cursor}))),
).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
let json = response.data.into_json().unwrap();
let page = &json["reposPage"];
let rows = page["nodes"].as_array().unwrap();
assert!(rows.len() <= 50);
names.extend(
rows.iter()
.map(|row| row["name"].as_str().unwrap().to_owned()),
);
if !page["hasNextPage"].as_bool().unwrap() {
break;
}
let next = page["endCursor"].as_str().unwrap().to_owned();
assert_ne!(cursor.as_ref(), Some(&next));
cursor = Some(next);
assert!(names.len() <= crate::db::MAX_VISIBLE_REPO_PAGE_SIZE);
}
assert_eq!(
names,
(0..total)
.map(|index| format!("repo-{index:03}"))
.collect::<Vec<_>>()
);
}
#[sqlx::test]
async fn repos_legacy_accepts_exactly_the_visible_bound(pool: PgPool) {
let db = db(pool).await;
let base_time = Utc::now();
let total = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE;
for index in 0..total {
let name = format!("repo-{index:03}");
let mut r = repo(&name, OWNER, &name, true);
// Anti-correlate updated_at with name, creation order, and index
// so only ordering by updated_at DESC can satisfy the expectation.
let offset = (index * 37) % total;
r.updated_at = base_time + chrono::Duration::seconds(offset as i64);
db.create_repo(&r).await.unwrap();
}
db.create_repo(&repo("hidden", OWNER, "hidden", false))
.await
.unwrap();
let response = anon(&schema(db), "{ repos { name } }").await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
let repos = response.data.into_json().unwrap()["repos"]
.as_array()
.unwrap()
.clone();
assert_eq!(repos.len(), total);
let names: Vec<_> = repos.iter().filter_map(|r| r["name"].as_str()).collect();
assert!(
!names.contains(&"hidden"),
"hidden repo must be excluded from legacy repos response"
);
let mut expected_indices = (0..total).collect::<Vec<_>>();
expected_indices.sort_by_key(|&idx| std::cmp::Reverse((idx * 37) % total));
let expected_names: Vec<String> = expected_indices
.into_iter()
.map(|index| format!("repo-{index:03}"))
.collect();
assert_eq!(
names,
expected_names
.iter()
.map(|s| s.as_str())
.collect::<Vec<_>>(),
"legacy repos must retain activity ordering (most recently updated first)"
);
}
#[sqlx::test]
async fn repos_page_pagination_crosses_owner_boundary(pool: PgPool) {
let db = db(pool).await;
let owner1 = "did:key:z6MkaOwner1";
let owner2 = "did:key:z6MkbOwner2";
let earlier_owner = "did:web:example.com";
// This owner sorts first only after did:key normalization. Its name
// sorts past the first cursor, so an unqualified OR name predicate
// would incorrectly serve it again on the next page.
//
// Rows are inserted out of order and not in reverse asserted order.
// Assigned IDs do not track name order, and owner2 contains both
// "B-repo" and "a-repo" whose byte order (B < a) differs from their
// case-folded order (a < b), ensuring lower(d.name) cannot satisfy
// the keyset query. Sized to 6 entries so the last page is full-sized
// (limit 2), proving hasNextPage distinguishes a full terminal page.
db.create_repo(&repo("r5", owner2, "c-repo", true))
.await
.unwrap();
db.create_repo(&repo("r9", earlier_owner, "zz-repo", true))
.await
.unwrap();
db.create_repo(&repo("r3", owner2, "B-repo", true))
.await
.unwrap();
db.create_repo(&repo("r8", owner1, "z-repo", true))
.await
.unwrap();
db.create_repo(&repo("r2", owner1, "b-repo", true))
.await
.unwrap();
db.create_repo(&repo("r1", owner2, "a-repo", true))
.await
.unwrap();
let schema = schema(db);
let page1_resp = anon(
&schema,
"{ reposPage(limit: 2) { nodes { name ownerDid } hasNextPage endCursor } }",
)
.await;
assert!(page1_resp.errors.is_empty(), "{:?}", page1_resp.errors);
let p1 = page1_resp.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p1["hasNextPage"], true);
assert_eq!(
p1["nodes"],
serde_json::json!([
{"name": "zz-repo", "ownerDid": earlier_owner},
{"name": "b-repo", "ownerDid": owner1},
])
);
let cursor = p1["endCursor"].as_str().unwrap();
let page2_query = format!(
"{{ reposPage(limit: 2, after: \"{cursor}\") {{ nodes {{ name ownerDid }} hasNextPage endCursor }} }}"
);
let page2_resp = anon(&schema, &page2_query).await;
assert!(page2_resp.errors.is_empty(), "{:?}", page2_resp.errors);
let p2 = page2_resp.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p2["hasNextPage"], true);
assert_eq!(
p2["nodes"],
serde_json::json!([
{"name": "z-repo", "ownerDid": owner1},
{"name": "B-repo", "ownerDid": owner2},
])
);
let cursor = p2["endCursor"].as_str().unwrap();
let page3_query = format!(
"{{ reposPage(limit: 2, after: \"{cursor}\") {{ nodes {{ name ownerDid }} hasNextPage endCursor }} }}"
);
let page3_resp = anon(&schema, &page3_query).await;
assert!(page3_resp.errors.is_empty(), "{:?}", page3_resp.errors);
let p3 = page3_resp.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p3["hasNextPage"], false);
assert_eq!(
p3["nodes"],
serde_json::json!([
{"name": "a-repo", "ownerDid": owner2},
{"name": "c-repo", "ownerDid": owner2},
])
);
}
#[sqlx::test]
async fn repos_page_at_documented_maximum_distinguishes_terminal_page(pool: PgPool) {
let db = db(pool).await;
let total = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE;
for index in 0..total {
let name = format!("repo-{index:03}");
db.create_repo(&repo(&name, OWNER, &name, true))
.await
.unwrap();
}
let schema = schema(db.clone());
let query =
format!("{{ reposPage(limit: {total}) {{ nodes {{ name }} hasNextPage endCursor }} }}");
let resp200 = anon(&schema, &query).await;
assert!(resp200.errors.is_empty(), "{:?}", resp200.errors);
let p200 = resp200.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p200["hasNextPage"], false);
let nodes200 = p200["nodes"].as_array().unwrap().clone();
assert_eq!(nodes200.len(), total);
// Adding row 201 keeps page 1 content identical while flipping hasNextPage to true.
let name201 = format!("repo-{total:03}");
db.create_repo(&repo(&name201, OWNER, &name201, true))
.await
.unwrap();
let resp201 = anon(&schema, &query).await;
assert!(resp201.errors.is_empty(), "{:?}", resp201.errors);
let p201 = resp201.data.into_json().unwrap()["reposPage"].clone();
assert_eq!(p201["hasNextPage"], true);
assert_eq!(p201["nodes"], p200["nodes"]);
assert_eq!(p201["endCursor"], p200["endCursor"]);
}
#[sqlx::test]
async fn repos_page_visibility_matches_the_shared_gate(pool: PgPool) {
use crate::db::VisibilityMode;
let db = db(pool).await;
let reader = "did:key:zReader";
for (id, public) in [
("open", true),
("private", false),
("root-deny", true),
("root-reader", false),
("subtree", true),
("root-tie", true),
("odd-star", true),
("malformed-readers", true),
("non-array-readers", true),
("non-string-reader", true),
("quarantined", true),
] {
db.create_repo(&repo(id, OWNER, id, public)).await.unwrap();
}
// Canonical and mirror copies must still collapse before pagination.
db.create_repo(&repo("z6MkOwner/open", "z6MkOwner", "open", true))
.await
.unwrap();
db.create_repo(&repo(
"other-method",
"did:web:z6MkOwner",
"other-method",
false,
))
.await
.unwrap();
db.set_repo_quarantine("quarantined", true).await.unwrap();
for (id, glob, readers) in [
("root-deny", "/", vec![]),
("root-reader", "/**", vec![reader.to_owned()]),
("subtree", "/secret/**", vec![]),
("root-tie", "/", vec![reader.to_owned()]),
("root-tie", "/**", vec![]),
("odd-star", "/*", vec![]),
("malformed-readers", "/", vec![]),
("non-array-readers", "/", vec![]),
("non-string-reader", "/", vec![]),
] {
db.set_visibility_rule(id, glob, VisibilityMode::B, &readers, OWNER)
.await
.unwrap();
}
// The typed setter cannot create malformed JSON, but existing TEXT rows
// can contain it. Such a rule must deny this repo, not abort the page.
// All three deny arms of the reader_dids predicate are pinned here:
// (a) invalid JSON, (b) valid JSON that is not an array, (c) an array
// containing a non-string member. The Rust gate parses reader_dids as
// Vec<String> with unwrap_or_default(), so (b) and (c) also fail to
// parse and deny; the SQL CASE must agree on every shape.
let rows = sqlx::query("UPDATE visibility_rules SET reader_dids = $1 WHERE repo_id = $2")
.bind("not JSON")
.bind("malformed-readers")
.execute(db.pool())
.await
.unwrap();
assert_eq!(rows.rows_affected(), 1);
let rows = sqlx::query("UPDATE visibility_rules SET reader_dids = $1 WHERE repo_id = $2")
.bind("\"just-a-string\"")
.bind("non-array-readers")
.execute(db.pool())
.await
.unwrap();
assert_eq!(rows.rows_affected(), 1);
// The named reader is listed, but the non-string member poisons the
// whole list: both sides must still deny them.
let rows = sqlx::query("UPDATE visibility_rules SET reader_dids = $1 WHERE repo_id = $2")
.bind("[\"did:key:zReader\", 42]")
.bind("non-string-reader")
.execute(db.pool())
.await
.unwrap();
assert_eq!(rows.rows_affected(), 1);
let all = db.list_all_repos_deduped().await.unwrap();
for caller in [
None,
Some(OWNER),
Some("z6MkOwner"),
Some(reader),
Some("zReader"),
Some("did:web:z6MkOwner"),
] {
let mut expected = Vec::new();
for record in &all {
let rules = db.list_visibility_rules(&record.id).await.unwrap();
if crate::visibility::listable_at_root(
&rules,
record.is_public,
&record.owner_did,
caller,
) {
expected.push(record.id.clone());
}
}
expected.sort();
let mut actual = db
.list_visible_repos_page(caller, None, usize::MAX)
.await
.unwrap()
.into_iter()
.map(|record| record.id)
.collect::<Vec<_>>();
actual.sort();
assert_eq!(actual, expected, "caller {caller:?}");
}
let schema = schema(db);
for caller in [None, Some("did:key:zUnauthorized")] {
let query = "{ repos { name } }";
let response = match caller {
None => anon(&schema, query).await,
Some(caller) => authed(&schema, query, caller).await,
};
assert!(response.errors.is_empty(), "{:?}", response.errors);
let mut names: Vec<String> = response.data.into_json().unwrap()["repos"]
.as_array()
.unwrap()
.iter()
.map(|repo| repo["name"].as_str().unwrap().to_owned())
.collect();
names.sort();
assert_eq!(names, ["odd-star", "open", "subtree"]);
}
let query = "{ reposPage(limit: 1) { nodes { name ownerDid } hasNextPage endCursor } }";
let response = anon(&schema, query).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
let json = response.data.into_json().unwrap();
let page = &json["reposPage"];
assert_eq!(
page["nodes"],
serde_json::json!([{"name": "odd-star", "ownerDid": OWNER}])
);
assert_eq!(page["hasNextPage"], true);
let cursor = super::parse_repo_cursor(page["endCursor"].as_str().unwrap()).unwrap();
assert_eq!(cursor, (OWNER.to_owned(), "odd-star".to_owned()));
assert!(!json.to_string().contains("private"));
assert!(!json.to_string().contains("quarantined"));
// Route-level check with unauthorized authenticated caller:
// Excludes private, quarantined, and root-deny repos, matching anonymous behavior.
let all_query = "{ reposPage(limit: 50) { nodes { name } hasNextPage } }";
let unauth_response = authed(&schema, all_query, "did:key:zUnauthorized").await;
assert!(
unauth_response.errors.is_empty(),
"{:?}",
unauth_response.errors
);
let unauth_json = unauth_response.data.into_json().unwrap();
let unauth_names: Vec<&str> = unauth_json["reposPage"]["nodes"]
.as_array()
.unwrap()
.iter()
.filter_map(|n| n["name"].as_str())
.collect();
assert_eq!(unauth_names, vec!["odd-star", "open", "subtree"]);
let unauth_str = unauth_json.to_string();
assert!(!unauth_str.contains("private"));
assert!(!unauth_str.contains("quarantined"));
assert!(!unauth_str.contains("root-deny"));
assert!(!unauth_str.contains("root-reader"));
assert!(!unauth_str.contains("root-tie"));
// Reader caller gets root-reader and root-tie, but still excludes private, quarantined, root-deny.
let reader_response = authed(&schema, all_query, reader).await;
assert!(
reader_response.errors.is_empty(),
"{:?}",
reader_response.errors
);
let reader_json = reader_response.data.into_json().unwrap();
let reader_names: Vec<&str> = reader_json["reposPage"]["nodes"]
.as_array()
.unwrap()
.iter()
.filter_map(|n| n["name"].as_str())
.collect();
assert_eq!(
reader_names,
vec!["odd-star", "open", "root-reader", "subtree"]
);
let reader_str = reader_json.to_string();
assert!(!reader_str.contains("private"));
assert!(!reader_str.contains("quarantined"));
assert!(!reader_str.contains("root-deny"));
assert!(!reader_str.contains("root-tie"));
}
#[sqlx::test]
async fn repos_page_rechecks_cursor_authority_and_exact_boundary(pool: PgPool) {
let db = db(pool).await;
let visible = repo("visible", OWNER, "a-visible", true);
db.create_repo(&visible).await.unwrap();
db.create_repo(&repo("hidden", OWNER, "z-private", false))
.await
.unwrap();
let schema = schema(db);
let first = anon(
&schema,
"{ reposPage(limit: 1) { nodes { name } hasNextPage endCursor } }",
)
.await;
assert!(first.errors.is_empty());
let first = first.data.into_json().unwrap();
assert_eq!(first["reposPage"]["hasNextPage"], false);
let query = format!(
"{{ reposPage(limit: 1, after: \"{}\") {{ nodes {{ name }} hasNextPage endCursor }} }}",
super::repo_cursor(&visible)
);
let owner = authed(&schema, &query, OWNER).await;
assert!(owner.errors.is_empty());
assert_eq!(
owner.data.into_json().unwrap()["reposPage"]["nodes"][0]["name"],
"z-private"
);
let anon = anon(&schema, &query).await;
assert!(anon.errors.is_empty());
assert_eq!(
anon.data.into_json().unwrap()["reposPage"],
serde_json::json!({
"nodes": [], "hasNextPage": false, "endCursor": null
})
);
let unauth = authed(&schema, &query, "did:key:zUnauthorized").await;
assert!(unauth.errors.is_empty());
assert_eq!(
unauth.data.into_json().unwrap()["reposPage"],
serde_json::json!({
"nodes": [], "hasNextPage": false, "endCursor": null
})
);
}
#[tokio::test]
async fn repos_page_rejects_nul_cursors_before_database_access() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
for position in [("did:key:zOwner\0", "repo"), ("did:key:zOwner", "repo\0")] {
let cursor = super::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&position).unwrap());
let query = format!("{{ reposPage(after: \"{cursor}\") {{ hasNextPage }} }}");
let response =
tokio::time::timeout(std::time::Duration::from_secs(1), anon(&schema, &query))
.await
.expect("invalid cursors must be rejected without accessing the database");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "invalid repository cursor");
assert_eq!(response.data, async_graphql::Value::Null);
}
let position = ("did:key:zOwner", "repo");
let cursor = super::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&position).unwrap());
assert_eq!(
super::parse_repo_cursor(&cursor).unwrap(),
(position.0.to_owned(), position.1.to_owned())
);
}
#[tokio::test]
async fn repos_page_rejects_invalid_inputs_before_database_access() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
let over_max = crate::db::MAX_VISIBLE_REPO_PAGE_SIZE + 1;
let expected_limit_message = format!(
"limit must be between 1 and {}",
crate::db::MAX_VISIBLE_REPO_PAGE_SIZE
);
// Each input pins its own rejection message: the loop must not accept
// either message for every query, or a misattributed message (e.g. the
// limit arm returning "invalid repository cursor") stays invisible.
for (query, expected_message) in [
(
"{ reposPage(limit: 0) { hasNextPage } }".to_owned(),
expected_limit_message.clone(),
),
(
"{ reposPage(limit: -1) { hasNextPage } }".to_owned(),
expected_limit_message.clone(),
),
(
format!("{{ reposPage(limit: {over_max}) {{ hasNextPage }} }}"),
expected_limit_message.clone(),
),
(
"{ reposPage(after: \"invalid!\") { hasNextPage } }".to_owned(),
"invalid repository cursor".to_owned(),
),
] {
let response =
tokio::time::timeout(std::time::Duration::from_secs(1), anon(&schema, &query))
.await
.unwrap();
assert_eq!(response.errors.len(), 1);
assert_eq!(
response.errors[0].message, expected_message,
"wrong rejection message for query: {query}"
);
}
// Valid base64 and valid cursor JSON: only the length guard rejects it.
let oversized = super::URL_SAFE_NO_PAD
.encode(serde_json::to_vec(&("did:key:reader", "a".repeat(3100))).unwrap());
assert!(oversized.len() > 4096);
assert!(serde_json::from_slice::<(String, String)>(
&super::URL_SAFE_NO_PAD.decode(&oversized).unwrap()
)
.is_ok());
assert!(super::parse_repo_cursor(&oversized).is_err());
}
#[tokio::test]
async fn parse_repo_cursor_rejects_wrong_shape_json() {
// Valid base64 and valid JSON, but not a (String, String) pair: the
// wrong-shape arm must report "invalid repository cursor", exactly,
// before any database access.
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
for payload in [
serde_json::json!(["only-one"]),
serde_json::json!(["a", "b", "c"]),
serde_json::json!({"owner": "x", "name": "y"}),
serde_json::json!(42),
serde_json::json!(["a", 1]),
] {
let cursor = super::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&payload).unwrap());
let err = super::parse_repo_cursor(&cursor).unwrap_err();
assert_eq!(err.message, "invalid repository cursor");
let query = format!("{{ reposPage(after: \"{cursor}\") {{ hasNextPage }} }}");
let response =
tokio::time::timeout(std::time::Duration::from_secs(1), anon(&schema, &query))
.await
.expect("wrong-shape cursors must be rejected without accessing the database");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "invalid repository cursor");
assert_eq!(response.data, async_graphql::Value::Null);
}
}
#[tokio::test]
async fn longest_repo_name_cursor_round_trips() {
// #465 4th round: a cursor the server emits must always parse back.
// Repo names are bounded at the create/fork/mirror write paths, so the
// longest admissible name must produce a cursor under the 4096-byte
// parse limit.
let name = "a".repeat(crate::db::MAX_REPO_NAME_LEN);
let record = repo("long", OWNER, &name, true);
let cursor = super::repo_cursor(&record);
assert!(
cursor.len() <= 4096,
"emitted cursor must stay under the parse limit: {}",
cursor.len()
);
assert_eq!(
super::parse_repo_cursor(&cursor).unwrap(),
(OWNER.to_owned(), name)
);
}
/// A negative refUpdates(limit) must be accepted and clamped to zero rows,
/// like tasks(limit: -1): the complexity formula's `limit.clamp(0, ...)`
/// prices it at the floor and the shared collector returns before any
/// database access. Mirrors tasks_negative_limit_clamped.
#[tokio::test]
async fn ref_updates_negative_limit_clamped() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let schema = schema(Arc::new(Db::for_testing(pool)));
let resp = tokio::time::timeout(
std::time::Duration::from_secs(1),
anon(&schema, "{ refUpdates(limit: -1) { repo } }"),
)
.await
.expect("negative limit must clamp without accessing the database");
assert!(
resp.errors.is_empty(),
"negative limit must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(count(&resp), 0);
}
/// The mirror write path binds peer-supplied names verbatim, so it must
/// enforce the same bound as the local write paths: an overlong peer name
/// would otherwise make reposPage emit an unparseable cursor. The
/// rejection happens before any database access.
#[tokio::test]
async fn mirror_repo_rejects_overlong_name() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let db = Db::for_testing(pool);
let name = "b".repeat(crate::db::MAX_REPO_NAME_LEN + 1);
let err = db
.upsert_mirror_repo("z6MkLong", &name, "/tmp/long", None, false)
.await
.unwrap_err();
assert!(
err.to_string().contains("exceeds"),
"unexpected error: {err}"
);
}
async fn db(pool: PgPool) -> Arc<Db> {
let db = Db::for_testing(pool);
db.run_migrations().await.unwrap();
@@ -1211,13 +475,12 @@ mod tests {
assert_eq!(count_tasks(&resp), 0);
}
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises the
/// documented maximum).
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises Max 200).
#[sqlx::test]
async fn tasks_limit_ceiling_clamped_to_max(pool: PgPool) {
async fn tasks_limit_ceiling_clamped_to_200(pool: PgPool) {
let db = db(pool).await;
let now = Utc::now().to_rfc3339();
for i in 0..crate::db::MAX_VISIBLE_TASK_PAGE_SIZE + 1 {
for i in 0..201 {
db.create_task(&crate::db::AgentTask {
id: format!("task-ceil-{i}"),
repo_id: None,
@@ -1238,49 +501,7 @@ mod tests {
}
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: 5000) { id } }").await;
assert_eq!(
count_tasks(&resp),
crate::db::MAX_VISIBLE_TASK_PAGE_SIZE as usize,
"limit above the maximum must clamp to MAX_VISIBLE_TASK_PAGE_SIZE"
);
}
/// A limit above MAX_VISIBLE_REF_UPDATES must reach the resolver clamped,
/// not be rejected: the complexity formula clamps the priced limit (so the
/// document passes validation) and the shared collector clamps the rows
/// (so at most MAX_VISIBLE_REF_UPDATES come back). Removing the formula
/// clamp prices the raw limit and rejects the document; removing the
/// collector clamp returns unbounded rows. Mirrors
/// tasks_limit_ceiling_clamped_to_max.
#[sqlx::test]
async fn ref_updates_limit_ceiling_clamped_to_max(pool: PgPool) {
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
let db = db(pool).await;
db.create_repo(&repo("r1", OWNER, "widget", true))
.await
.unwrap();
let total = MAX_VISIBLE_REF_UPDATES as usize + 1;
for i in 0..total {
let mut row = ref_row(&format!("ceil{i}"), "z6MkOwner/widget");
row.timestamp = format!("2026-07-01T10:{:02}:{:02}+00:00", i / 60, i % 60);
db.insert_ref_update(&row).await.unwrap();
}
let schema = schema(db);
let query = format!(
"{{ refUpdates(limit: {}) {{ repo }} }}",
MAX_VISIBLE_REF_UPDATES + 4800
);
let resp = anon(&schema, &query).await;
assert!(
resp.errors.is_empty(),
"limit above the maximum must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(
count(&resp),
MAX_VISIBLE_REF_UPDATES as usize,
"limit above the maximum must clamp to MAX_VISIBLE_REF_UPDATES"
);
assert_eq!(count_tasks(&resp), 200, "limit above 200 must clamp to 200");
}
fn count_tasks(resp: &async_graphql::Response) -> usize {
@@ -20,7 +20,6 @@ impl SubscriptionRoot {
/// `if announce` block, `api/repos.rs`). This is a single-point invariant —
/// any new sender to `ref_update_tx` MUST be `announce`-gated, or private-repo
/// ref metadata leaks here to unauthenticated subscribers (#112/#114 class).
#[graphql(complexity = "50 + child_complexity")]
async fn ref_updates(
&self,
ctx: &Context<'_>,
@@ -47,7 +46,6 @@ impl SubscriptionRoot {
})
}
#[graphql(complexity = "50 + child_complexity")]
async fn task_events(
&self,
ctx: &Context<'_>,
-8
View File
@@ -11,14 +11,6 @@ pub struct RepoType {
pub created_at: String,
}
#[derive(SimpleObject)]
pub struct RepoPageType {
pub nodes: Vec<RepoType>,
pub has_next_page: bool,
/// Position after the last returned visible repository; null for an empty page.
pub end_cursor: Option<String>,
}
#[derive(SimpleObject, Clone)]
pub struct AgentTaskType {
pub id: String,
+49 -407
View File
@@ -922,12 +922,6 @@ async fn sweep_pass(
// Advance FIRST: every path below this line may skip the row, and none of them
// may wedge the walk (scenario 7).
last = sha.clone();
// Round-3 P1: skip Pinata-only rows whose local cid is NULL.
// The legacy repair walk re-keys `cid` from a provider CID to
// the raw resolver CID; a row with no local cid has no string
// to re-key, and skipping is the natural behavior. The cursor
// still advances so the walk does not loop on this row.
let Some(stored) = stored else { continue };
// Same cost gate as the skip-path repair: a canonical raw CIDv1 key is already
// the resolver key, so it reads no bytes and resolves no repo.
if gitlawb_core::cid::is_raw_cidv1(&stored) {
@@ -1463,97 +1457,6 @@ fn note_legacy_repair_read() {
/// have to be documented, validated, and kept meaningful.
pub const PIN_BATCH_BUDGET: Duration = Duration::from_secs(120);
/// A captured per-repo visibility-policy epoch that fences a pin batch.
///
/// The reconciliation sweep reads the epoch immediately before dispatching a
/// pin loop and passes a fence in; the loop re-reads the epoch before every
/// upload and aborts the batch the moment it moves. A visibility narrow that
/// lands mid-batch (a rule made private, a repo quarantined) must not let the
/// remaining pre-authorized objects still go to a public content-addressed
/// backend — the narrow is a policy change, and dispatching against the stale
/// snapshot is the exact irreversible-publication class this fence exists for
/// (R1-P1). `None` (the push path) means "no fence": the push derives its own
/// object list at admission and holds a write lease, so no sweep-style batch
/// snapshot crosses the dispatch boundary.
///
/// The fence deliberately does NOT hold any lock across the batch: a
/// visibility narrow (rule insert/remove, quarantine set) must commit
/// immediately, and the batch aborts on its next `is_current` check. Holding
/// a per-repo mutex from capture through drop would invert this — the narrow
/// would block behind the background sweep and every object in the batch
/// would still be sealed/posted to the reader being removed. The accepted
/// residual is a single in-flight object: a narrow that commits between the
/// pre-POST `is_current` check and the HTTP POST cannot be recalled, but the
/// next iteration aborts and the fenced DB record (row-locked against the
/// narrow's epoch bump) refuses to land the raced row as durable.
/// Multi-process / multi-node narrows are ordered by the same epoch column;
/// no in-process registry is involved.
///
/// Product decision (revocation model): prompt revocation wins over
/// publication atomicity. The suite intentionally allows one provider
/// upload already in flight when a rule narrows and requires every
/// LATER object to stop (`encrypt_and_pin_stops_sealing_when_reader_
/// removed_mid_batch`). A stronger contract — revocation commit
/// linearizing against every concurrent publication, or compensation
/// (unpin/delete) for an envelope that finishes after removal — is
/// explicitly OUT of scope until decided and documented here. Do not
/// reintroduce batch-spanning mutual exclusion to close the
/// single-in-flight window without that decision: it trades one
/// possibly-raced object for sealing the whole batch to a removed
/// reader.
#[derive(Clone)]
pub struct PolicyFence {
db: crate::db::Db,
repo_id: String,
epoch: i64,
}
impl PolicyFence {
/// Capture the current policy epoch for `repo_id`. A read failure is a
/// skip, not a retry-with-zero: the caller must not dispatch a batch it
/// cannot fence (fail closed on a stale allow).
pub async fn capture(db: &crate::db::Db, repo_id: &str) -> Option<Self> {
match db.repo_policy_epoch(repo_id).await {
Ok(epoch) => Some(PolicyFence {
db: db.clone(),
repo_id: repo_id.to_string(),
epoch,
}),
Err(e) => {
tracing::warn!(repo = %repo_id, err = %e, "policy-epoch read failed; not fencing pin batch");
None
}
}
}
/// Whether the repo's policy epoch is unchanged since capture. A read
/// failure is treated as "changed": never dispatch on a policy we cannot
/// prove current.
pub async fn is_current(&self) -> bool {
match self.db.repo_policy_epoch(&self.repo_id).await {
Ok(epoch) => epoch == self.epoch,
Err(_) => false,
}
}
/// The epoch value captured at `capture` time. Exposed so the
/// pinner can pass it to
/// `Db::record_pinned_cid_with_source_fenced` — the third
/// fence in the same transaction as the row insert.
/// Returning the field directly (rather than a `Option`)
/// matches the contract: a `PolicyFence` always has a
/// captured epoch; `is_current()` reports whether it still
/// matches.
pub fn captured_epoch(&self) -> i64 {
self.epoch
}
/// The repo this fence guards, for log correlation.
pub fn repo_id(&self) -> &str {
&self.repo_id
}
}
/// The smallest remainder worth starting a bounded git read (or an add) with.
///
/// A 1ms remainder otherwise buys a child spawned already past its deadline, which
@@ -1605,7 +1508,6 @@ pub async fn pin_git_object(
return Ok(String::new());
}
let expected_cid = Cid::from_git_object_bytes(data).to_string();
let url = format!(
"{}/api/v0/add?cid-version=1&raw-leaves=true&pin=true",
ipfs_api.trim_end_matches('/')
@@ -1648,15 +1550,6 @@ pub async fn pin_git_object(
// Kubo returns newline-delimited JSON; we only care about the last object
// (there's typically just one for a single-file add).
//
// The response MUST carry a real `Hash`: a misconfigured `GITLAWB_IPFS_API`
// (proxy returning HTML, health check on the wrong port, truncated gateway)
// can otherwise answer 2xx with no JSON, and falling back to the locally
// computed `expected_cid` would record a row for bytes the backend never
// stored. The reconciliation sweep trusts `pinned_cids` rows as durability
// evidence, so a silent false positive at pin time becomes a permanent blind
// spot for the backstop. A missing `Hash` fails the pin rather than recording
// a phantom row (mirrors Pinata's `data.cid` check).
let body = resp
.text()
.await
@@ -1672,26 +1565,13 @@ pub async fn pin_git_object(
let v: serde_json::Value = serde_json::from_str(line).ok()?;
v["Hash"].as_str().map(|s| s.to_string())
})
.next_back();
let cid = match cid {
Some(cid) => {
if cid != expected_cid {
tracing::warn!(
sha256 = %sha256_hex,
returned = %cid,
expected = %expected_cid,
"IPFS returned a different CID than computed locally (Kubo chunking may differ); recording the backend's answer"
);
}
cid
}
None => {
return Err(anyhow::anyhow!(
"IPFS /api/v0/add returned 2xx without a Hash field; refusing to record \
a CID the backend never acknowledged (misconfigured GITLAWB_IPFS_API?)"
));
}
};
.next_back()
.ok_or_else(|| {
anyhow::anyhow!(
"IPFS /api/v0/add returned 200 without a usable Hash: {:.200}",
body
)
})?;
// The string still has to be a CID. A garbage Hash would be recorded and
// later cat-ed the same way a missing one was, so reject it here.
Cid::from_str(&cid).map_err(|e| {
@@ -1774,8 +1654,8 @@ pub(crate) fn batch_budget_gate(
/// than [`PIN_READ_FLOOR`] left. It is a gate, not a hard ceiling, since a started
/// iteration still runs to completion;
/// - the git read: `store::read_object_bounded` runs under `spawn_blocking` against the
/// ABSOLUTE batch deadline (not the loop-top remainder, which the `has_ipfs_cid`
/// round-trip sitting between the two would push past it), with SIGTERM-then-SIGKILL
/// ABSOLUTE batch deadline (not the loop-top remainder, which the `is_pinned` round-trip
/// sitting between the two would push past it), with SIGTERM-then-SIGKILL
/// process-group teardown, so a hung `git cat-file` costs this batch its remaining
/// budget plus one watchdog teardown instead of holding the permit for the child's
/// whole lifetime and blocking a runtime worker while it does;
@@ -1819,11 +1699,10 @@ pub(crate) fn batch_budget_gate(
/// # Truncation semantics
///
/// A batch stopped at the deadline leaves its remaining objects unpinned, and
/// nothing sweeps them up afterwards on the push path; recovery is opportunistic
/// (a later full-scan push re-offers the skipped OIDs). The reconciliation
/// sweep is the systematic backstop: when it is enabled and a pin backend is
/// configured, it re-derives the public object set each pass and fills any
/// remaining gap.
/// nothing sweeps them up afterwards. There is no reconciliation pass over
/// `pinned_cids`; recovery is opportunistic, happening only if some later push
/// on the repo takes the full-scan fallback (`push_delta::list_all_objects`) and
/// re-derives the whole object set, which then re-offers the skipped OIDs.
///
/// The twin in `pinata.rs` is back at parity on everything that bounds or repairs an
/// object: it runs the same shared budget gate at the top of every iteration, the same
@@ -1840,29 +1719,6 @@ pub(crate) fn batch_budget_gate(
///
/// Returns a list of `(sha256_hex, cid)` pairs pinned AND durably recorded this
/// call.
/// What one `pin_new_objects` call (IPFS or Pinata twin) did, with the
/// three backend states kept apart instead of inferred from each other:
///
/// - `confirmed`: `(sha, provider_cid)` pairs whose DB record durably
/// landed. ONLY these may advance `gaps_filled`, branch/gossip CID
/// state, or any "repaired" bookkeeping. A provider upload whose
/// record timed out, failed, or was refused by the fence is absent
/// here even though the bytes may sit on the provider.
/// - `last_attempted`: the last OID whose loop body was entered — i.e.
/// the backend did real work for it (reads, an upload attempt, or a
/// skip-branch decision), whether that work succeeded, failed, or hit
/// an unknown outcome. `None` when nothing was entered (empty input,
/// immediate fence abort, or immediate budget gate). Durable
/// continuation cursors advance to this, never to the tail of the
/// planned vector: the tail may never have been visited, and
/// promoting it would rotate an untouched suffix behind the backlog
/// forever.
#[derive(Debug)]
pub struct PinBatchOutcome {
pub confirmed: Vec<(String, String)>,
pub last_attempted: Option<String>,
}
// Eight because #173's git seam (`git_bin`, `git_timeout`) and pin provenance
// (`repo_id`) sit alongside #174's batch budget. All four callers pass every one, and
// grouping them into a context struct would add a type whose only job is to be
@@ -1877,34 +1733,16 @@ pub async fn pin_new_objects(
db: &crate::db::Db,
repo_id: &str,
batch_budget: Duration,
fence: Option<&PolicyFence>,
) -> PinBatchOutcome {
) -> Vec<(String, String)> {
if ipfs_api.is_empty() {
return PinBatchOutcome {
confirmed: Vec::new(),
last_attempted: None,
};
return vec![];
}
let deadline = Instant::now() + batch_budget;
let total = object_list.len();
let mut pinned = Vec::new();
let mut last_attempted: Option<String> = None;
for (attempted, sha) in object_list.into_iter().enumerate() {
// Policy fence (R1-P1): a visibility narrow that lands after the caller
// built this batch must abort it before the next irreversible upload.
// Checked FIRST so a changed policy costs nothing beyond the read.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id,
unattempted = total - attempted,
"visibility policy changed mid-batch; stopping the pin loop"
);
break;
}
}
// Top of the iteration, before any of this object's work: an object is
// never started with a remainder too small to cover a bounded read's
// teardown. Consumed as a guard only: the read below runs against the
@@ -1913,40 +1751,19 @@ pub async fn pin_new_objects(
if batch_budget_gate("IPFS", deadline, pinned.len(), total - attempted).is_none() {
break;
}
// Attempt progress (not a durability claim): from here the loop does
// real work for this OID — skip-branch reads, airgapped git reads,
// an upload attempt — whatever the outcome. The caller persists
// this as the continuation, never the planned vector's tail.
last_attempted = Some(sha.clone());
// Skip if the object is ALREADY a real local IPFS pin, but first
// backfill provenance if the existing pin has none. A legacy pin
// (recorded before repo_id existed, #173, jatmn) is skipped here
// before record_pinned_cid ever runs, so its NULL provenance would
// never resolve to one repo and known CIDs keep hitting the scan.
// The backfill only sets repo_id (AND repo_id IS NULL guard
// preserves first-pinner-owns) and never re-pins the bytes: the
// object is already on IPFS.
//
// #218 review P1a: this check keys on `has_ipfs_cid` (writer-owned
// `local_ipfs_provenance = TRUE`), NOT on row existence
// (`is_pinned`). A Pinata-only row is `is_pinned = true` but
// `has_ipfs_cid = false`: the bytes never reached the local IPFS
// daemon, only Pinata, and we MUST fall through to the local
// writer path so a real local pin lands. Using `is_pinned` here
// was the gap that made the Pinata-only → local-IPFS repair
// path inert: every sweep pass re-entered this arm, recorded
// the source, and continued without ever calling
// `pin_git_object` or `record_pinned_cid_with_source`. The flag
// stayed FALSE forever.
//
// Every DB call from here to the end of the iteration is bounded
// by the ABSOLUTE batch deadline (F3, #173): the loop runs under
// a global pin permit and a bare await parked it for the whole
// stall. The elapsed arm is mapped per site below, never as a
// blanket "existing error arm": a timeout cancels the client
// future but not the statement Postgres is running, so it
// reports an UNKNOWN outcome, not a failed write.
match db_bounded(deadline, db.has_ipfs_cid(&sha)).await {
// Skip if already pinned, but first backfill provenance if the existing
// pin has none. A legacy pin (recorded before repo_id existed, #173, jatmn)
// is skipped here before record_pinned_cid ever runs, so its NULL provenance
// would never resolve to one repo and known CIDs keep hitting the scan. The
// backfill only sets repo_id (AND repo_id IS NULL guard preserves
// first-pinner-owns) and never re-pins the bytes: the object is already on IPFS.
// Every DB call from here to the end of the iteration is bounded by the
// ABSOLUTE batch deadline (F3, #173): the loop runs under a global pin permit
// and a bare await parked it for the whole stall. The elapsed arm is mapped per
// site below, never as a blanket "existing error arm": a timeout cancels the
// client future but not the statement Postgres is running, so it reports an
// UNKNOWN outcome, not a failed write.
match db_bounded(deadline, db.is_pinned(&sha)).await {
Ok(true) => {
// Elapsed here is free to skip: these are reads, so a late server-side
// completion costs nothing, and the backfill's own `AND repo_id IS NULL`
@@ -2056,7 +1873,7 @@ pub async fn pin_new_objects(
}
Ok(false) => {}
Err(e) => {
tracing::warn!(sha = %sha, err = %e, "DB error checking IPFS pinned status");
tracing::warn!(sha = %sha, err = %e, "DB error checking pinned status");
continue;
}
}
@@ -2070,7 +1887,7 @@ pub async fn pin_new_objects(
// own deadline regardless.
//
// The read runs against the ABSOLUTE batch deadline, not against the remainder
// measured at the top of the iteration: the `has_ipfs_cid` round-trip above sits
// measured at the top of the iteration: the `is_pinned` round-trip above sits
// between the two, so `Instant::now() + budget_left` would land past `deadline`
// by however long the DB took, and under a saturated pool that is the dominant
// term. A slow DB check must not push the read's own bound out.
@@ -2155,24 +1972,6 @@ pub async fn pin_new_objects(
break;
};
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the has_ipfs_cid round-trip or the bounded Git read — both of
// which can take seconds and during which a quarantine or rule change may
// have committed. Without this, stale plaintext can start uploading
// under authorization that is no longer current.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id,
unattempted = total - attempted,
"visibility policy changed during preparation; aborting IPFS upload"
);
break;
}
}
// Pin to IPFS
match pin_git_object(ipfs_api, &sha, &data, Some(add_timeout)).await {
Ok(cid) if !cid.is_empty() => {
@@ -2222,31 +2021,7 @@ pub async fn pin_new_objects(
// per-object failure.
match db_bounded(
db_record_deadline(deadline),
retry_db_record(|| {
// #218 review round 9 (guidance #3 —
// linearization): always go through the
// fenced form. The fence is either
// captured (sweep / public-pin path: the
// third fence is the linearization point
// that closes the rule-write /
// record-write race) or absent
// (push-side admission where the
// decision is made at request time —
// we pass `i64::MAX` as a sentinel that
// the fenced form treats as "no fence
// check"). The 3-arg
// `record_pinned_cid_with_source` is
// still available for tests that don't
// own a fence, but the production
// pinner routes through here.
let fence_epoch = fence.map(|f| f.captured_epoch()).unwrap_or(i64::MAX);
db.record_pinned_cid_with_source_fenced(
&sha,
&raw_cid,
repo_id,
fence_epoch,
)
}),
retry_db_record(|| db.record_pinned_cid_with_source(&sha, &raw_cid, repo_id)),
)
.await
{
@@ -2263,10 +2038,7 @@ pub async fn pin_new_objects(
}
}
PinBatchOutcome {
confirmed: pinned,
last_attempted,
}
pinned
}
#[cfg(test)]
@@ -2602,74 +2374,6 @@ mod tests {
);
}
/// The misconfigured-`GITLAWB_IPFS_API` false positive (P3): a 2xx response
/// that carries no `Hash` field (proxy returning HTML, health check on the
/// wrong port, truncated gateway) must FAIL the pin, not fall back to the
/// locally computed `expected_cid`. Falling back records a `pinned_cids`
/// row for bytes the backend never stored, and the reconciliation sweep
/// trusts rows as durability evidence — so the false positive becomes a
/// permanent blind spot for the backstop. A missing `Hash` must surface as
/// an explicit error, never a successful pin.
#[tokio::test]
async fn pin_git_object_rejects_a_2xx_without_a_hash_field() {
let endpoint = empty_ok_endpoint().await;
let inner = tokio::time::timeout(
Duration::from_secs(30),
pin_git_object(&endpoint, "deadbeef", b"some object bytes\n", None),
)
.await
.expect("wedge guard: an immediate empty 200 cannot take 30s");
let err = inner.expect_err(
"a 2xx without a Hash field must not surface as a successful pin \
(would record a phantom pinned_cids row the sweep then trusts)",
);
assert!(
err.to_string().contains("without a Hash field"),
"the error must name the missing Hash so operators diagnose the endpoint: {err:#}"
);
}
/// A 200 that answers with an empty body and no `Hash` — the exact shape of
/// a proxy or health-check endpoint mistaken for a Kubo API.
async fn empty_ok_endpoint() -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
while let Ok((mut sock, _)) = listener.accept().await {
tokio::spawn(async move {
let mut acc = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
acc.extend_from_slice(&buf[..n]);
if let Some(hdr_end) =
acc.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4)
{
let headers = String::from_utf8_lossy(&acc[..hdr_end]).to_lowercase();
let len: usize = headers
.lines()
.find_map(|l| l.strip_prefix("content-length:"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0);
if acc.len() >= hdr_end + len {
break;
}
}
}
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n")
.await;
let _ = sock.flush().await;
});
}
});
endpoint
}
/// The second unhardened sink, reached from `sync.rs`. Same shape as above.
#[tokio::test]
async fn cat_against_silent_endpoint_errors_within_its_own_timeout() {
@@ -2782,16 +2486,15 @@ mod tests {
&db,
"repo-batch-budget",
Duration::from_millis(5500),
None,
),
)
.await
.expect("wedge guard: a 5.5s budget cannot take 30s");
assert!(
(1..=3).contains(&pinned.confirmed.len()),
(1..=3).contains(&pinned.len()),
"the batch must stop partway, not pin all five and not stall on the first: pinned {}",
pinned.confirmed.len()
pinned.len()
);
let text = logs.text();
let warns: Vec<&str> = text
@@ -2813,52 +2516,9 @@ mod tests {
})
.unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}"));
assert!(
unattempted >= 1 && unattempted + pinned.confirmed.len() <= 5,
unattempted >= 1 && unattempted + pinned.len() <= 5,
"unattempted={unattempted} with {} pinned is not a partial batch of five",
pinned.confirmed.len()
);
}
/// The outcome reports the last object actually ENTERED, not the planned
/// tail: the first upload hangs 6s against a ~2s per-request timeout, so
/// it fails, and the loop-top gate breaks the batch before the second
/// object starts. `last_attempted` must be the first OID — the only one
/// the loop body entered — even though nothing was confirmed and two
/// OIDs were never visited. A cursor persisted from `to_pin.last()`
/// would rotate the untouched suffix behind the backlog forever.
#[sqlx::test]
async fn pin_new_objects_reports_last_attempted_not_planned_tail(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("attempted.git");
let oids = seed_loose_blobs(&repo_path, 3);
let endpoint = delaying_endpoint(vec![Duration::from_secs(6)]).await;
let outcome = tokio::time::timeout(
Duration::from_secs(30),
pin_new_objects(
&endpoint,
&repo_path,
"git",
Duration::from_secs(30),
oids.clone(),
&db,
"repo-attempted",
Duration::from_secs(2),
None,
),
)
.await
.expect("a 2s budget cannot take 30s");
assert_eq!(
outcome.last_attempted,
Some(oids[0].clone()),
"only the first object was entered; the planned tail was never visited"
);
assert!(
outcome.confirmed.is_empty(),
"the hung upload timed out, so nothing was confirmed"
pinned.len()
);
}
@@ -2893,13 +2553,12 @@ mod tests {
&db,
"repo-batch-continues",
Duration::from_secs(90),
None,
),
)
.await
.expect("wedge guard: a 13s add plus an immediate one cannot take 60s");
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
2,
"a slow but progressing endpoint must pin both objects: an upload past the client's \
10s default is not a dead endpoint"
@@ -2932,12 +2591,11 @@ mod tests {
&db,
"repo-batch-rejects",
Duration::from_secs(60),
None,
),
)
.await
.expect("a rejecting endpoint answers immediately, so this cannot take 30s");
assert!(pinned.confirmed.is_empty(), "every add was rejected");
assert!(pinned.is_empty(), "every add was rejected");
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
4,
@@ -2980,7 +2638,6 @@ mod tests {
/// type probe carries its oid on stdin rather than in argv, so an oid appears in
/// the log only once an object has already got past its probe, and a healthy
/// object costs two invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log)
.unwrap_or_default()
@@ -3031,7 +2688,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(2),
None,
),
)
.await
@@ -3042,7 +2698,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -3119,7 +2775,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_millis(1500),
None,
),
)
.await
@@ -3134,7 +2789,7 @@ mod tests {
that can only be spawned and reaped"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"the first object is inside the budget and must pin: {pinned:?}"
);
@@ -3193,7 +2848,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -3203,7 +2857,7 @@ mod tests {
if genuinely_unreadable {
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"nothing can be pinned through a store that cannot be read: {pinned:?}"
);
assert_eq!(
@@ -3268,7 +2922,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -3284,7 +2937,7 @@ mod tests {
must still be attempted, got {attempted} of {}",
oids.len()
);
let pinned_shas: Vec<&String> = pinned.confirmed.iter().map(|(sha, _)| sha).collect();
let pinned_shas: Vec<&String> = pinned.iter().map(|(sha, _)| sha).collect();
let expected: Vec<&String> = oids.iter().filter(|o| !tainted.contains(o)).collect();
assert_eq!(
pinned_shas, expected,
@@ -3337,7 +2990,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -3349,7 +3001,7 @@ mod tests {
"an object-scoped fault must not stop the batch: every object must be read"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
4,
"one corrupt object must cost only itself: the other four must still pin"
);
@@ -3572,7 +3224,6 @@ mod tests {
&db,
"repo-stalled-db",
Duration::from_millis(1500),
None,
),
)
.await
@@ -3583,7 +3234,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a stalled pinned-status check cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -3643,7 +3294,6 @@ mod tests {
&db,
"repo-skip-stalled",
Duration::from_millis(1500),
None,
),
)
.await
@@ -3654,7 +3304,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"an already-pinned object is skipped, never re-pinned: {pinned:?}"
);
assert!(
@@ -3710,17 +3360,13 @@ mod tests {
&db,
"repo-multi-stalled",
Duration::from_millis(1500),
None,
),
)
.await
.expect("three stalled objects must still cost one budget, not three");
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
"nothing can pin against a stalled DB"
);
assert!(pinned.is_empty(), "nothing can pin against a stalled DB");
assert!(
elapsed < Duration::from_secs(3),
"three stalled objects must charge ONE budget (1.5s), not one each; got {elapsed:?}"
@@ -3743,7 +3389,7 @@ mod tests {
///
/// The lock time is a MARGIN, not a boundary: taking it at 100ms left `is_pinned`
/// racing it on a loaded box, and losing that race makes the read block, time out,
/// and break the batch, which fails on `pinned.confirmed.len() == 1` for a reason that has
/// and break the batch, which fails on `pinned.len() == 1` for a reason that has
/// nothing to do with the floor. Any time between the `is_pinned` round trip and
/// the add's 1.7s return proves the same thing.
#[sqlx::test]
@@ -3781,7 +3427,6 @@ mod tests {
&db,
"repo-spent-budget",
Duration::from_millis(2000),
None,
),
);
let (pinned, ()) = tokio::join!(pin, locker);
@@ -3794,7 +3439,7 @@ mod tests {
nothing can resolve the CID"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"the durably recorded pin must be returned: {pinned:?}"
);
@@ -3855,7 +3500,6 @@ mod tests {
&db,
"repo-definite-error",
Duration::from_millis(1200),
None,
),
);
let (pinned, ()) = tokio::join!(pin, commit);
@@ -3937,7 +3581,6 @@ mod tests {
&db,
"repo-marker-floor",
Duration::from_millis(1500),
None,
),
);
let (pinned, ()) = tokio::join!(pin, controller);
@@ -3948,7 +3591,7 @@ mod tests {
drop(sources_lock);
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"an already-pinned object is skipped, never re-pinned: {pinned:?}"
);
assert!(
@@ -4042,7 +3685,6 @@ mod tests {
&db,
"repo-repair-stalled",
Duration::from_millis(2200),
None,
),
);
let (pinned, mut cids_lock) = tokio::join!(pin, controller);
+2 -43
View File
@@ -18,7 +18,6 @@ mod operator;
mod p2p;
mod pinata;
mod rate_limit;
mod reconciliation;
mod server;
mod state;
mod sync;
@@ -52,7 +51,6 @@ use state::AppState;
struct DegradedState {
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
}
/// Two independent counters with no cross-field invariant — atomics, not a
@@ -161,7 +159,6 @@ async fn main() -> Result<()> {
degraded_listener,
node_did.to_string(),
Arc::clone(&db_startup),
config.web_url.clone(),
db_ready_rx,
shutdown_tx.subscribe(),
));
@@ -432,8 +429,6 @@ async fn main() -> Result<()> {
std::time::Duration::from_secs(3600),
200_000,
);
let federated_rate_limiter =
rate_limit::RateLimiter::new_bounded(12, std::time::Duration::from_secs(60), 10_000);
if config.sync_trigger_rate_limit == 0 {
tracing::warn!(
"GITLAWB_SYNC_TRIGGER_RATE_LIMIT=0 — /sync/trigger IP rate limiting disabled"
@@ -472,7 +467,6 @@ async fn main() -> Result<()> {
push_limiter_trust,
sync_trigger_rate_limiter,
peer_write_rate_limiter,
federated_rate_limiter,
shutdown_tx: shutdown_tx.clone(),
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(config.max_concurrent_git_ops)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(
@@ -667,30 +661,6 @@ async fn main() -> Result<()> {
info!("auto-sync worker started");
}
// Periodic reconciliation sweep: re-derives pin/seal sets and fills gaps
// so a dropped replication job never means data loss.
{
let db = state.db.clone();
let config = Arc::clone(&state.config);
let http_client = Arc::clone(&state.http_client);
let node_keypair = Arc::clone(&state.node_keypair);
let node_did = state.node_did.clone();
let pin_sem = Arc::clone(&state.pin_semaphore);
let shutdown_rx = state.subscribe_shutdown();
if reconciliation::spawn(
db,
config,
http_client,
node_keypair,
node_did,
pin_sem,
shutdown_rx,
Some(state.repo_store.clone()),
) {
info!("reconciliation sweep worker started");
}
}
// On-chain operator setup: verify stake + spawn heartbeat loop
if !state.config.contract_node_staking.is_empty()
&& !state.config.operator_private_key.is_empty()
@@ -899,12 +869,11 @@ async fn run_degraded_server(
listener: TcpListener,
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
mut db_ready_rx: watch::Receiver<bool>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<()> {
let addr = listener.local_addr().ok();
let router = build_degraded_router(node_did, db_startup, web_url);
let router = build_degraded_router(node_did, db_startup);
info!(?addr, "degraded HTTP server ready");
axum::serve(listener, router)
@@ -921,15 +890,10 @@ async fn run_degraded_server(
Ok(())
}
fn build_degraded_router(
node_did: String,
db_startup: Arc<DbStartupStatus>,
web_url: Option<String>,
) -> Router {
fn build_degraded_router(node_did: String, db_startup: Arc<DbStartupStatus>) -> Router {
let state = DegradedState {
node_did,
db_startup,
web_url,
};
// Everything answers 503 with the same body — including /health and
// /ready, so peer readiness probes and uptime monitors correctly see a
@@ -961,9 +925,6 @@ async fn degraded_node_info(State(state): State<DegradedState>) -> impl IntoResp
obj.insert("name".into(), "gitlawb-node".into());
obj.insert("version".into(), env!("CARGO_PKG_VERSION").into());
obj.insert("did".into(), state.node_did.clone().into());
if let Some(web_url) = &state.web_url {
obj.insert("web_url".into(), web_url.clone().into());
}
}
(StatusCode::SERVICE_UNAVAILABLE, Json(body))
}
@@ -1210,7 +1171,6 @@ mod rate_limiter_sweep_tests {
state.close_issue_rate_limiter = RateLimiter::new(10, window);
state.sync_trigger_rate_limiter = RateLimiter::new(10, window);
state.peer_write_rate_limiter = RateLimiter::new(10, window);
state.federated_rate_limiter = RateLimiter::new(10, window);
state.ipfs_rate_limiter = RateLimiter::new(10, window);
let limiters = |s: &crate::state::AppState| {
@@ -1221,7 +1181,6 @@ mod rate_limiter_sweep_tests {
s.close_issue_rate_limiter.clone(),
s.sync_trigger_rate_limiter.clone(),
s.peer_write_rate_limiter.clone(),
s.federated_rate_limiter.clone(),
s.ipfs_rate_limiter.clone(),
]
};
-52
View File
@@ -20,9 +20,6 @@
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
//! * currently-connected peer count — `gitlawb_peers_connected`
//! * reconciliation sweep gaps found and filled —
//! `gitlawb_reconciliation_gaps_found_total` /
//! `gitlawb_reconciliation_gaps_filled_total`
//!
//! All metrics live in a single process-wide registry initialized by
//! [`init`]. Increment helpers (`record_push`, `record_auth_failure`, ...)
@@ -61,8 +58,6 @@ static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
static RECONCILIATION_GAPS_FOUND: OnceLock<IntCounter> = OnceLock::new();
static RECONCILIATION_GAPS_FILLED: OnceLock<IntCounter> = OnceLock::new();
/// 1 when the node started without its p2p identity because the key storage
/// could not be used. The node keeps serving HTTP in that state and /health
/// stays green, so without this the outage is visible only to whoever reads
@@ -246,29 +241,6 @@ fn init_inner(version: &str, node_did: &str) {
.set(peers_connected)
.expect("set PEERS_CONNECTED once");
let gaps_found = IntCounter::with_opts(Opts::new(
"gitlawb_reconciliation_gaps_found_total",
"Total reconciliation sweep gaps detected (objects that should be pinned but are not)",
))
.expect("gitlawb_reconciliation_gaps_found_total definition");
registry
.register(Box::new(gaps_found.clone()))
.expect("register gitlawb_reconciliation_gaps_found_total");
RECONCILIATION_GAPS_FOUND
.set(gaps_found)
.expect("set RECONCILIATION_GAPS_FOUND once");
let gaps_filled = IntCounter::with_opts(Opts::new(
"gitlawb_reconciliation_gaps_filled_total",
"Total reconciliation sweep gaps successfully filled (objects pinned by the sweep)",
))
.expect("gitlawb_reconciliation_gaps_filled_total definition");
registry
.register(Box::new(gaps_filled.clone()))
.expect("register gitlawb_reconciliation_gaps_filled_total");
RECONCILIATION_GAPS_FILLED
.set(gaps_filled)
.expect("set RECONCILIATION_GAPS_FILLED once");
let p2p_key_load_failed = IntGauge::with_opts(Opts::new(
"gitlawb_p2p_identity_key_load_failed",
"1 if the node is running without p2p because its identity key could not be loaded",
@@ -425,20 +397,6 @@ pub fn set_peers_connected(count: i64) {
}
}
/// Record reconciliation sweep gaps found (objects that should be pinned but are not).
pub fn record_reconciliation_gaps_found(count: u64) {
if let Some(c) = RECONCILIATION_GAPS_FOUND.get() {
c.inc_by(count);
}
}
/// Record reconciliation sweep gaps filled (objects successfully pinned by the sweep).
pub fn record_reconciliation_gaps_filled(count: u64) {
if let Some(c) = RECONCILIATION_GAPS_FILLED.get() {
c.inc_by(count);
}
}
/// Record whether the node is running without its p2p identity.
pub fn set_p2p_key_load_failed(failed: bool) {
if let Some(g) = P2P_KEY_LOAD_FAILED.get() {
@@ -483,8 +441,6 @@ mod tests {
.expect("PUSHES set after init")
.with_label_values(&["alice/repo"])
.inc();
record_reconciliation_gaps_found(7);
record_reconciliation_gaps_filled(3);
let body = encode().expect("encode should succeed after init");
assert!(
@@ -499,14 +455,6 @@ mod tests {
body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"),
"expected the incremented counter to be visible in: {body}"
);
assert!(
body.contains("gitlawb_reconciliation_gaps_found_total 7"),
"expected the reconciliation gaps-found counter to be visible in: {body}"
);
assert!(
body.contains("gitlawb_reconciliation_gaps_filled_total 3"),
"expected the reconciliation gaps-filled counter to be visible in: {body}"
);
}
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
+3 -692
View File
@@ -7,11 +7,11 @@
//! The node's PeerId comes from an Ed25519 keypair loaded from a persistent
//! key file, so the PeerId is stable across restarts.
use std::collections::{hash_map::DefaultHasher, HashMap, HashSet, VecDeque};
use std::collections::{hash_map::DefaultHasher, HashMap};
use std::hash::{Hash, Hasher};
use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant};
use std::time::Duration;
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
@@ -315,17 +315,6 @@ fn is_zero(v: &u32) -> bool {
*v == 0
}
// Identify is an untrusted address source. Keep enough entries for normal
// multi-homed peers while bounding both one peer and the whole routing table.
const IDENTIFY_ADDRESS_LIMIT: usize = 8;
// Bound canonicalization and sorting too, including rejected/duplicate entries.
const IDENTIFY_REPORT_ADDRESS_LIMIT: usize = 64;
const IDENTIFY_NEW_ADDRESS_LIMIT: usize = 8;
const IDENTIFY_GLOBAL_ADDRESS_LIMIT: usize = 1024;
const IDENTIFY_ADDRESS_WINDOW: Duration = Duration::from_secs(60);
const IDENTIFY_ADDRESS_TTL: Duration = Duration::from_secs(30 * 60);
const IDENTIFY_ADDRESS_CLEANUP_INTERVAL: Duration = Duration::from_secs(60);
/// A ref-update event published to Gossipsub when a push lands.
///
/// The signing bytes are this struct serialized with `sig` set to None (see
@@ -1642,200 +1631,6 @@ fn did_to_kad_key(did: &str) -> kad::RecordKey {
kad::RecordKey::new(&format!("/gitlawb/did/{did}").as_bytes())
}
/// Returns true when `address` was explicitly configured for `peer` through
/// AddKnownPeer. Explicit addresses are retained in Kademlia even when an
/// overlapping Identify lease expires or is evicted.
fn is_explicit_address(
explicit_addresses: &HashMap<PeerId, HashSet<Multiaddr>>,
peer: &PeerId,
address: &Multiaddr,
) -> bool {
explicit_addresses
.get(peer)
.is_some_and(|addresses| addresses.contains(address))
}
#[derive(Debug)]
struct IdentifyAddress {
address: Multiaddr,
expires_at: Instant,
}
#[derive(Debug)]
struct IdentifyPeerAddresses {
addresses: VecDeque<IdentifyAddress>,
window_started: Instant,
new_addresses: usize,
}
#[derive(Debug, Default, PartialEq, Eq)]
struct IdentifyAddressChanges {
added: Vec<Multiaddr>,
refreshed: Vec<Multiaddr>,
removed: Vec<(PeerId, Multiaddr)>,
}
#[derive(Debug, Default)]
struct IdentifyAddressBook {
peers: HashMap<PeerId, IdentifyPeerAddresses>,
address_count: usize,
}
impl IdentifyAddressBook {
fn update(
&mut self,
peer_id: PeerId,
now: Instant,
addresses: &[Multiaddr],
) -> IdentifyAddressChanges {
let mut changes = IdentifyAddressChanges::default();
if self.peers.contains_key(&peer_id) {
changes.removed.extend(self.expire_peer(peer_id, now));
}
let reported = addresses
.iter()
.take(IDENTIFY_REPORT_ADDRESS_LIMIT)
.filter_map(|address| address.clone().with_p2p(peer_id).ok())
.collect::<HashSet<_>>();
if reported.is_empty() && !self.peers.contains_key(&peer_id) {
return changes;
}
let state = self
.peers
.entry(peer_id)
.or_insert_with(|| IdentifyPeerAddresses {
addresses: VecDeque::new(),
window_started: now,
new_addresses: 0,
});
if now.saturating_duration_since(state.window_started) >= IDENTIFY_ADDRESS_WINDOW {
state.window_started = now;
state.new_addresses = 0;
}
// Refresh every retained address in the bounded report before admission.
// Refreshed addresses are reported so the Identify handler can re-admit
// them to Kademlia. Kademlia may have dropped a retained address
// after a failed dial or never kept it for a full k-bucket, and a
// TTL refresh alone would never re-add it.
for existing in &mut state.addresses {
if reported.contains(&existing.address) {
existing.expires_at = now + IDENTIFY_ADDRESS_TTL;
changes.refreshed.push(existing.address.clone());
}
}
let mut new_addresses: Vec<Multiaddr> = reported
.iter()
.filter(|address| {
!state
.addresses
.iter()
.any(|existing| &existing.address == *address)
})
.cloned()
.collect();
// HashSet iteration is deliberately unordered. Sort new candidates so
// an oversized Identify report has deterministic admission behavior.
new_addresses.sort();
for address in new_addresses {
if state.new_addresses >= IDENTIFY_NEW_ADDRESS_LIMIT {
break;
}
let evicted = if state.addresses.len() < IDENTIFY_ADDRESS_LIMIT {
None
} else {
state
.addresses
.iter()
.position(|existing| !reported.contains(&existing.address))
.and_then(|index| state.addresses.remove(index))
};
if let Some(evicted) = evicted {
self.address_count -= 1;
changes.removed.push((peer_id, evicted.address));
} else if state.addresses.len() >= IDENTIFY_ADDRESS_LIMIT {
// Every retained address was refreshed in this report. Keep
// that stable subset and drop surplus new candidates.
break;
}
if self.address_count >= IDENTIFY_GLOBAL_ADDRESS_LIMIT {
// Reject the admission instead of evicting another peer's
// entry: cross-peer eviction lets a flood of new identities
// push honest peers' addresses out of the book and, when the
// evicted address is their last one, out of the Kademlia
// routing table. Per-peer eviction above still applies to the
// reporting peer's own addresses.
break;
}
state.new_addresses += 1;
state.addresses.push_back(IdentifyAddress {
address: address.clone(),
expires_at: now + IDENTIFY_ADDRESS_TTL,
});
self.address_count += 1;
changes.added.push(address);
}
// Do not keep a row when every candidate was rejected: a zero-address
// row is invisible to address_count and would otherwise linger until
// the expiry tick. This matches expire_peer, which drops empty rows.
if state.addresses.is_empty() {
self.peers.remove(&peer_id);
}
changes
}
fn expire(&mut self, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let mut removed = Vec::new();
let peers = self.peers.keys().copied().collect::<Vec<_>>();
for peer_id in peers {
removed.extend(self.expire_peer(peer_id, now));
}
removed
}
fn expire_peer(&mut self, peer_id: PeerId, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let expired = self
.peers
.get_mut(&peer_id)
.map(|state| Self::expire_state(state, now))
.unwrap_or_default();
self.address_count -= expired.len();
if self
.peers
.get(&peer_id)
.is_some_and(|state| state.addresses.is_empty())
{
self.peers.remove(&peer_id);
}
expired
.into_iter()
.map(|address| (peer_id, address))
.collect()
}
fn expire_state(state: &mut IdentifyPeerAddresses, now: Instant) -> Vec<Multiaddr> {
let mut expired = Vec::new();
let mut retained = VecDeque::with_capacity(state.addresses.len());
while let Some(address) = state.addresses.pop_front() {
if address.expires_at <= now {
expired.push(address.address);
} else {
retained.push_back(address);
}
}
state.addresses = retained;
expired
}
}
/// Combined libp2p behaviour.
#[derive(NetworkBehaviour)]
#[behaviour(prelude = "libp2p_swarm::derive_prelude")]
@@ -4165,14 +3960,6 @@ pub async fn start(
// Track in-flight GetRecord queries → reply channels
let mut pending_get_did: HashMap<kad::QueryId, oneshot::Sender<Option<DidRecord>>> =
HashMap::new();
// Keep Identify-derived entries across disconnects so a bootstrap address
// learned through Identify remains usable for redial. The TTL cleanup below
// removes entries that are no longer refreshed without touching explicit
// AddKnownPeer addresses.
let mut identify_addresses = IdentifyAddressBook::default();
let mut explicit_addresses: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
let mut identify_cleanup = tokio::time::interval(IDENTIFY_ADDRESS_CLEANUP_INTERVAL);
identify_cleanup.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
// Start the event loop as a background task
tokio::spawn(async move {
@@ -4192,13 +3979,6 @@ pub async fn start(
_ = ingest_sweep.tick() => {
run_ingest_sweep(&ingest_limiters, &replay_guard, ingest_now()).await;
}
_ = identify_cleanup.tick() => {
for (peer_id, address) in identify_addresses.expire(Instant::now()) {
if !is_explicit_address(&explicit_addresses, &peer_id, &address) {
swarm.behaviour_mut().kademlia.remove_address(&peer_id, &address);
}
}
}
// Graceful shutdown: exit the swarm loop when the
// process-wide signal flips. This drops the Swarm
// which closes all libp2p connections cleanly.
@@ -4333,21 +4113,7 @@ pub async fn start(
identify::Event::Received { peer_id, info, .. }
)) => {
debug!(peer = %peer_id, "identify received");
let changes = identify_addresses.update(
peer_id,
Instant::now(),
&info.listen_addrs,
);
for (removed_peer, addr) in changes.removed {
if !is_explicit_address(&explicit_addresses, &removed_peer, &addr)
{
swarm
.behaviour_mut()
.kademlia
.remove_address(&removed_peer, &addr);
}
}
for addr in changes.refreshed.into_iter().chain(changes.added) {
for addr in info.listen_addrs {
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
}
}
@@ -4384,14 +4150,6 @@ pub async fn start(
}
}
P2pCommand::AddKnownPeer { peer_id, addr } => {
let Ok(addr) = addr.clone().with_p2p(peer_id) else {
warn!(%peer_id, %addr, "dropping known-peer address with a foreign peer suffix");
continue;
};
explicit_addresses
.entry(peer_id)
.or_default()
.insert(addr.clone());
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
}
P2pCommand::Dial(addr) => {
@@ -4443,453 +4201,6 @@ pub async fn start(
mod tests {
use super::*;
fn identify_address(peer_id: PeerId, octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
.with_p2p(peer_id)
.unwrap()
}
fn identify_address_without_peer(octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
}
#[test]
fn identify_empty_reports_do_not_retain_peer_entries() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let foreign = identify_address(PeerId::random(), 1, 10_000);
for _ in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT + 1 {
let peer = PeerId::random();
assert_eq!(
book.update(peer, now, &[]),
IdentifyAddressChanges::default()
);
assert_eq!(
book.update(peer, now, std::slice::from_ref(&foreign)),
IdentifyAddressChanges::default()
);
}
assert!(book.peers.is_empty());
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_fully_rejected_peer_leaves_no_row_at_full_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// Fill the global budget with one-address peers.
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// A fresh peer reporting at saturation has every candidate rejected
// by the global-budget check. The row created by or_insert_with must
// be dropped: a zero-address row is invisible to address_count and
// would otherwise linger until the expiry tick.
let fresh = PeerId::random();
let fresh_address = identify_address(fresh, 2, 30_000);
let changes = book.update(fresh, now, std::slice::from_ref(&fresh_address));
assert!(changes.added.is_empty());
assert!(!book.peers.contains_key(&fresh));
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_update_reports_expired_addresses_as_removed() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
// An update past the TTL expires the stale address through
// expire_peer and reports it in changes.removed so the Identify
// handler can drop it from Kademlia.
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1),
std::slice::from_ref(&address),
);
assert_eq!(changes.removed, vec![(peer, address.clone())]);
assert_eq!(changes.added, vec![address]);
}
#[test]
fn identify_expire_drops_rows_emptied_by_expiry() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
assert!(book.peers.contains_key(&peer));
// Once every address in the row has expired, expire() must drop the
// row itself, matching the empty-row drop in update().
let removed = book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1));
assert_eq!(removed, vec![(peer, address)]);
assert!(!book.peers.contains_key(&peer));
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_report_processing_stops_at_the_input_limit() {
let peer = PeerId::random();
let now = Instant::now();
let foreign = identify_address(PeerId::random(), 1, 10_000);
let accepted = identify_address(peer, 2, 10_001);
let ignored = identify_address(peer, 3, 10_002);
let mut report = vec![foreign; IDENTIFY_REPORT_ADDRESS_LIMIT - 1];
report.push(accepted.clone());
report.push(ignored);
let mut book = IdentifyAddressBook::default();
let changes = book.update(peer, now, &report);
assert_eq!(changes.added, vec![accepted]);
assert!(changes.removed.is_empty());
assert_eq!(book.address_count, 1);
}
#[test]
fn identify_addresses_grow_without_eviction_below_the_peer_cap() {
let peer = PeerId::random();
let now = Instant::now();
let first = identify_address(peer, 1, 10_000);
let second = identify_address(peer, 2, 10_001);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&first));
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&second),
);
assert_eq!(changes.added, vec![second]);
assert!(changes.removed.is_empty());
assert_eq!(book.peers[&peer].addresses.len(), 2);
assert_eq!(book.address_count, 2);
}
#[test]
fn identify_global_budget_rejects_new_admissions() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let peers = (0..IDENTIFY_GLOBAL_ADDRESS_LIMIT)
.map(|index| {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
(peer, address)
})
.collect::<Vec<_>>();
let (peer, refreshed) = &peers[0];
let new_address = identify_address(*peer, 2, 30_000);
let changes = book.update(
*peer,
now + IDENTIFY_ADDRESS_WINDOW,
&[new_address.clone(), refreshed.clone()],
);
// The budget is full: the new address is rejected rather than
// evicting another peer's entry, and the refreshed address keeps its
// slot and is reported for re-admission to Kademlia.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![(*refreshed).clone()]);
assert_eq!(book.peers.len(), IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(
book.peers[peer]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![refreshed]
);
}
#[test]
fn identify_per_peer_eviction_applies_at_full_global_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// One filler peer goes in before the victim so the evicted address
// is not also the globally-oldest insertion. If saturation evicted
// the globally-oldest entry instead of the victim's own oldest,
// the sentinel's address would be the one removed.
let sentinel = PeerId::random();
let sentinel_address = identify_address(sentinel, 200, 19_999);
book.update(sentinel, now, std::slice::from_ref(&sentinel_address));
// The victim peer holds a full per-peer row while the rest of the
// global budget is filled by one-address peers.
let victim = PeerId::random();
let victim_addresses: Vec<Multiaddr> = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(victim, index as u8 + 1, 10_000 + index as u16))
.collect();
book.update(victim, now, &victim_addresses);
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT - IDENTIFY_ADDRESS_LIMIT - 1 {
let peer = PeerId::random();
let address = identify_address(peer, 200, 20_000 + index as u16);
book.update(peer, now, &[address]);
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// The victim reports a ninth address after the rate window rolls over.
// Per-peer eviction must still apply at global saturation: the oldest
// non-reported address is evicted to make room, and the global count
// is unchanged. This ordering only holds because eviction runs before
// the global-budget check; hoisting that check above the eviction
// block makes this test fail.
let ninth = identify_address(victim, 100, 30_000);
let changes = book.update(
victim,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&ninth),
);
assert_eq!(changes.added, vec![ninth]);
assert_eq!(changes.removed, vec![(victim, victim_addresses[0].clone())]);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.peers[&victim].addresses.len(), IDENTIFY_ADDRESS_LIMIT);
// The sentinel was inserted first; per-peer eviction must leave it
// alone. A global-oldest eviction policy would have removed it.
assert_eq!(
book.peers[&sentinel]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&sentinel_address]
);
}
#[test]
fn explicit_address_helper_detects_retained_addresses() {
let peer = PeerId::random();
let other = PeerId::random();
let address = identify_address(peer, 1, 10_000);
let mut explicit: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
// Unknown peer: nothing is explicit.
assert!(!is_explicit_address(&explicit, &peer, &address));
explicit.entry(peer).or_default().insert(address.clone());
assert!(is_explicit_address(&explicit, &peer, &address));
// The same address under a different peer is not retained.
assert!(!is_explicit_address(&explicit, &other, &address));
// A different address for the same peer is not retained.
assert!(!is_explicit_address(
&explicit,
&peer,
&identify_address(peer, 2, 10_001)
));
}
#[test]
fn identify_addresses_are_capped_with_fifo_eviction() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &initial);
assert_eq!(first.added, initial);
assert!(first.removed.is_empty());
let second = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
);
assert_eq!(second.added, vec![replacement]);
assert_eq!(second.removed, vec![(peer_id, initial[0].clone())]);
}
#[test]
fn identify_oversized_reports_keep_the_same_refreshed_subset() {
let peer_id = PeerId::random();
let now = Instant::now();
let mut report = (0..IDENTIFY_ADDRESS_LIMIT + 2)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &report);
assert_eq!(first.added.len(), IDENTIFY_ADDRESS_LIMIT);
assert_eq!(first.removed.len(), 0);
let retained = first.added.clone();
report.reverse();
let second = book.update(peer_id, now + IDENTIFY_ADDRESS_WINDOW, &report);
assert!(second.added.is_empty());
assert!(second.removed.is_empty());
let current = book
.peers
.get(&peer_id)
.unwrap()
.addresses
.iter()
.map(|entry| entry.address.clone())
.collect::<Vec<_>>();
assert_eq!(current, retained);
}
#[test]
fn identify_addresses_rate_limit_new_entries_across_updates() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_NEW_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(book.update(peer_id, now, &initial).added, initial);
assert!(book
.update(peer_id, now, std::slice::from_ref(&replacement))
.added
.is_empty());
assert_eq!(
book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
)
.added,
vec![replacement]
);
}
#[test]
fn identify_addresses_have_a_global_budget_across_peers() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let mut first = None;
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer_id = PeerId::random();
let address = identify_address(peer_id, (index % 254) as u8 + 1, 10_000 + index as u16);
if first.is_none() {
first = Some((peer_id, address.clone()));
}
assert!(book.update(peer_id, now, &[address]).removed.is_empty());
}
let (peer_id, first_address) = first.unwrap();
let address = identify_address(peer_id, 250, 30_000);
let changes = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&address),
);
// The global budget is full, so the admission is rejected and the
// peer's existing entry is left alone.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(
book.peers[&peer_id]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&first_address]
);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_addresses_refresh_and_expire() {
let peer_id = PeerId::random();
let now = Instant::now();
let address = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, now, std::slice::from_ref(&address))
.added,
vec![address.clone()]
);
let refreshed = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let refresh = book.update(peer_id, refreshed, std::slice::from_ref(&address));
assert!(refresh.added.is_empty());
assert!(refresh.removed.is_empty());
// Refreshed addresses are reported so the Identify handler re-adds
// them to Kademlia even though they were not newly admitted.
assert_eq!(refresh.refreshed, vec![address.clone()]);
assert!(book
.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1))
.is_empty());
assert_eq!(
book.expire(refreshed + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, address)]
);
}
#[test]
fn identify_refresh_reports_only_addresses_in_the_report() {
let peer_id = PeerId::random();
let now = Instant::now();
let reported = identify_address(peer_id, 1, 10_000);
let unreported = identify_address(peer_id, 2, 20_000);
let mut book = IdentifyAddressBook::default();
// Admit both addresses up front so the peer holds two.
assert_eq!(
book.update(peer_id, now, &[reported.clone(), unreported.clone()])
.added
.len(),
2
);
// A later report carrying only one of the two addresses must refresh
// just that one: pins the negative direction of the refresh
// predicate, so an address the peer stopped advertising keeps its
// original TTL instead of being refreshed back onto the Kademlia
// re-admit path on every report.
let report_at = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let changes = book.update(peer_id, report_at, std::slice::from_ref(&reported));
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![reported.clone()]);
// The unreported address still expires at its original TTL, while
// the refreshed address survives until its refreshed TTL.
assert_eq!(
book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, unreported)]
);
assert!(book.expire(report_at + Duration::from_secs(1)).is_empty());
assert_eq!(
book.expire(report_at + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, reported)]
);
}
#[test]
fn identify_addresses_reject_foreign_peer_suffixes() {
let peer_id = PeerId::random();
let other_peer_id = PeerId::random();
let address = identify_address(other_peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert!(book
.update(peer_id, Instant::now(), &[address])
.added
.is_empty());
}
#[test]
fn identify_addresses_canonicalize_the_peer_suffix_once() {
let peer_id = PeerId::random();
let raw = identify_address_without_peer(1, 10_000);
let canonical = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, Instant::now(), &[raw]).added,
vec![canonical]
);
}
#[test]
fn dir_mode_replace_authority_is_the_write_bits() {
assert!(!dir_mode_allows_untrusted_replace(0o755));
+41 -490
View File
@@ -153,26 +153,6 @@ pub async fn pin_object(
/// `upsert_branch_cid` and the p2p `publish_ref_update` gossip CID. The twin's return is
/// log-only, so it omits a record-failed pin rather than logging a pin the resolver
/// cannot serve. Moving this side to match would need that consumer moved first.
/// Whether the committed `pinned_cids` row already names `repo_id` as
/// first pinner, making a failed redundant `record_pin_source` insert
/// irrelevant to resolvability (`pin_sources_for_oid` unions the
/// primary row). Bounded read; any failure (including its own
/// timeout) answers false — an unproven row never confirms.
/// Used only after the primary `record_pinata_cid` for this call
/// succeeded or verified: callers must not promote a row this call
/// did not establish.
async fn primary_covers_repo(
db: &crate::db::Db,
sha: &str,
repo_id: &str,
deadline: std::time::Instant,
) -> bool {
matches!(
crate::ipfs_pin::db_bounded(deadline, db.provenance_for_oid(sha)).await,
Ok(Some(owner)) if owner.as_str() == repo_id
)
}
// Ten arguments, over clippy's threshold: the three the budget and the git seam add
// (`git_bin`, `git_timeout`, `batch_budget`) plus #173's `repo_id` are what put the read
// under test injection and under a deadline, and grouping them into a struct would only
@@ -190,33 +170,16 @@ pub async fn pin_new_objects(
db: &crate::db::Db,
repo_id: &str,
batch_budget: Duration,
fence: Option<&crate::ipfs_pin::PolicyFence>,
) -> crate::ipfs_pin::PinBatchOutcome {
) -> Vec<(String, String)> {
if jwt.is_empty() {
return crate::ipfs_pin::PinBatchOutcome {
confirmed: Vec::new(),
last_attempted: None,
};
return vec![];
}
let deadline = Instant::now() + batch_budget;
let total = object_list.len();
let mut pinned = Vec::new();
let mut last_attempted: Option<String> = None;
for (attempted, sha) in object_list.into_iter().enumerate() {
// Policy fence (R1-P1): a visibility narrow that lands after the caller
// built this batch must abort it before the next irreversible upload.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
unattempted = total - attempted,
"visibility policy changed mid-batch; stopping the Pinata pin loop"
);
break;
}
}
// Top of the iteration, before any of this object's work: an object is never
// started with a remainder too small to cover a bounded read's teardown. The
// gate is shared with the IPFS loop so the two cannot drift apart in how they
@@ -228,10 +191,6 @@ pub async fn pin_new_objects(
{
break;
}
// Attempt progress (not a durability claim): same contract as the
// IPFS twin — the continuation persists this, never the planned
// vector's tail.
last_attempted = Some(sha.clone());
// Every DB call from here to the end of the iteration is bounded by the
// ABSOLUTE batch deadline (F3, #173), through the same `db_bounded` helper the
@@ -436,21 +395,6 @@ pub async fn pin_new_objects(
}
};
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the has_pinata_cid round-trip or the bounded Git read.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
unattempted = total - attempted,
"visibility policy changed during preparation; aborting Pinata upload"
);
break;
}
}
match pin_object(client, upload_url, jwt, &sha, &data).await {
Ok(cid) if !cid.is_empty() => {
// The resolver key (`pinned_cids.cid`) must be the locally-computed
@@ -466,109 +410,30 @@ pub async fn pin_new_objects(
// runs under the shared client's own ceiling, so a successful one can
// return with ~0 of the batch budget left, and an unfloored bound would
// fail a write that today completes in milliseconds. That costs more on
// this side than on the twin: `pinned.push` below feeds
// `api/repos.rs`, which builds its `cid_map` from the pair and drives
// this side than on the twin: `pinned.push` below is UNCONDITIONAL, so
// `api/repos.rs` builds its `cid_map` from the pair either way and drives
// `upsert_branch_cid` plus the p2p `publish_ref_update` gossip from it. A
// dropped record therefore makes the node ADVERTISE a CID whose `/ipfs`
// read 404s. Only durably recorded pairs are pushed, so the map never
// advertises an unconfirmed row.
//
// Round 10 P2: a closed/failed DB write means the (sha, cid)
// pair is not durable; we suppress the `pinned.push` for this
// sha so the reconcile cannot count a Pinata gap as filled
// when no row exists. The source-record failure arms are also
// hard failures (multi-statement transaction never committed)
// and suppress the push in the same way.
//
// `record_pinata_cid` is an explicit multi-statement
// transaction (row lock + insert + commit), NOT a single
// autocommit upsert, so a timed-out future proves nothing:
// cancellation before COMMIT wrote nothing, during COMMIT is
// unknown. The Elapsed arm therefore verifies by content
// (`verify_pinata_record`: exact row + unchanged fence epoch)
// under its own bound and pushes only on proof; anything
// else stays suppressed and the gap is re-offered.
let fence_epoch = fence.map(|f| f.captured_epoch()).unwrap_or(i64::MAX);
let mut db_record_durable = false;
match crate::ipfs_pin::db_bounded(
// read 404s. If the floored bound still fires, THIS site's outcome really
// is unknown, and unlike the source record below that is a property of
// the operation: `record_pinata_cid` is a single autocommit upsert, so
// the statement Postgres already started can still land after the client
// future is cancelled. The warn names the arm through the error's own
// Display and the site keeps its existing behavior: the pair is still
// returned, and the row may or may not exist.
if let Err(e) = crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
crate::ipfs_pin::retry_db_record(|| {
// P2 (reviewer round 9): Pinata's POST is
// irreversible exactly like IPFS's, so
// route the record through the fenced
// variant when a fence is in scope. The
// `i64::MAX` sentinel tells the helper
// to skip the lock + comparison (the
// unfenced caller path).
db.record_pinata_cid(&sha, &raw_cid, &cid, Some(repo_id), fence_epoch)
db.record_pinata_cid(&sha, &raw_cid, &cid, Some(repo_id))
}),
)
.await
{
Ok(()) => db_record_durable = true,
Err(crate::ipfs_pin::BoundedDbError::Elapsed) => {
tracing::warn!(
sha = %sha,
"record_pinata_cid deadline elapsed on a multi-statement transaction; \
verifying the exact row before treating it as durable"
);
match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
db.verify_pinata_record(&sha, &raw_cid, &cid, repo_id, fence_epoch),
)
.await
{
Ok(true) => {
tracing::warn!(
sha = %sha,
"timed-out pinata record verified present with unchanged epoch; \
treating as durable"
);
db_record_durable = true;
}
Ok(false) => {
tracing::warn!(
sha = %sha,
"timed-out pinata record has no matching row or the epoch moved; \
suppressing the push so the gap is re-offered"
);
}
Err(e) => {
tracing::warn!(
sha = %sha,
err = %e,
"pinata record verification did not complete; \
suppressing the push so the gap is re-offered"
);
}
}
}
Err(e) => {
tracing::warn!(
sha = %sha,
err = %e,
"failed to record pinata_cid in DB; suppressing the (sha, cid) push \
so the reconcile cannot count this gap as filled"
);
// db_record_durable stays false → push suppressed
}
tracing::warn!(sha = %sha, err = %e, "failed to record pinata_cid in DB");
}
// F1 (#173 round 8): also record the first pinner in pin_repo_sources.
// U3: an exhausted retry marks the set incomplete so the resolver keeps
// the scan fallback rather than 404ing a copy it could serve.
//
// Whether the pair stays confirmed depends on what the
// PRIMARY `record_pinata_cid` above established: that row
// already names this repo as first pinner
// (`pin_sources_for_oid` unions it), so the resolver
// reaches this copy with or without the redundant
// `pin_repo_sources` insert. A source-write failure then
// keeps the pair confirmed (but still marks the set
// incomplete, preserving the U3 compensation). If the
// primary row names ANOTHER repo — or is absent — the
// source write was load-bearing and the push stays
// suppressed. (The IPFS twin needs no equivalent: its
// pin and source land in ONE transaction.)
match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
crate::ipfs_pin::retry_db_record(|| db.record_pin_source(&sha, repo_id)),
@@ -580,7 +445,9 @@ pub async fn pin_new_objects(
// wraps `record_pin_source`, an explicit transaction, so a timed-out
// call definitely never committed and the source is definitely
// missing. Mark the set incomplete rather than leaving it incomplete
// and unmarked.
// and unmarked. Note the contrast with `record_pinata_cid` a few
// lines up: that one is a single autocommit statement, so its
// timeout genuinely is an unknown outcome and it is warn-only.
Err(e @ crate::ipfs_pin::BoundedDbError::Elapsed) => {
tracing::warn!(
sha = %sha,
@@ -597,16 +464,6 @@ pub async fn pin_new_objects(
{
tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete");
}
if !primary_covers_repo(
db,
&sha,
repo_id,
crate::ipfs_pin::db_record_deadline(deadline),
)
.await
{
db_record_durable = false;
}
}
Err(e) => {
tracing::warn!(sha = %sha, err = %e, "failed to record pin source");
@@ -618,21 +475,9 @@ pub async fn pin_new_objects(
{
tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete");
}
if !primary_covers_repo(
db,
&sha,
repo_id,
crate::ipfs_pin::db_record_deadline(deadline),
)
.await
{
db_record_durable = false;
}
}
}
if db_record_durable {
pinned.push((sha, cid));
}
pinned.push((sha, cid));
}
Ok(_) => {}
Err(e) => {
@@ -641,10 +486,7 @@ pub async fn pin_new_objects(
}
}
crate::ipfs_pin::PinBatchOutcome {
confirmed: pinned,
last_attempted,
}
pinned
}
// ── Tests ─────────────────────────────────────────────────────────────────────
@@ -846,20 +688,19 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_millis(5500),
None,
),
)
.await
.expect("wedge guard: a 5.5s budget cannot take 30s");
assert!(
(1..=3).contains(&pinned.confirmed.len()),
(1..=3).contains(&pinned.len()),
"the batch must stop partway, not pin all five and not stall on the first: pinned {}",
pinned.confirmed.len()
pinned.len()
);
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
pinned.confirmed.len(),
pinned.len(),
"no upload may be issued for an object the budget stopped short of"
);
let text = logs.text();
@@ -887,9 +728,9 @@ mod tests {
})
.unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}"));
assert!(
unattempted >= 1 && unattempted + pinned.confirmed.len() <= 5,
unattempted >= 1 && unattempted + pinned.len() <= 5,
"unattempted={unattempted} with {} pinned is not a partial batch of five",
pinned.confirmed.len()
pinned.len()
);
}
@@ -938,7 +779,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(2),
None,
),
)
.await
@@ -949,7 +789,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -1028,7 +868,6 @@ mod tests {
"repo-git-timeout",
// Generous, so a call that ends on time ended on `git_timeout`.
Duration::from_secs(60),
None,
),
)
.await
@@ -1039,7 +878,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -1073,7 +912,6 @@ mod tests {
/// probe carries its oid on stdin rather than in argv, so an oid appears in the log only
/// once an object has already got past its probe, and a healthy object costs two
/// invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log)
.unwrap_or_default()
@@ -1131,7 +969,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -1141,7 +978,7 @@ mod tests {
if genuinely_unreadable {
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"nothing can be pinned through a store that cannot be read: {pinned:?}"
);
assert_eq!(
@@ -1206,7 +1043,6 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
@@ -1218,7 +1054,7 @@ mod tests {
"an object-scoped fault must not stop the batch: every object must be read"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
4,
"one corrupt object must cost only itself: the other four must still pin"
);
@@ -1258,18 +1094,13 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
.expect("an immediate endpoint and three healthy objects cannot take 30s");
assert_eq!(
pinned.confirmed.len(),
3,
"every healthy object must pin: {pinned:?}"
);
for (i, (sha, cid)) in pinned.confirmed.iter().enumerate() {
assert_eq!(pinned.len(), 3, "every healthy object must pin: {pinned:?}");
for (i, (sha, cid)) in pinned.iter().enumerate() {
assert_eq!(sha, &oids[i], "the pairs must carry the objects' own oids");
assert_eq!(cid, "QmPinataBatchTestCid");
assert!(
@@ -1292,15 +1123,11 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
.expect("a fully deduped batch cannot take 30s");
assert!(
again.confirmed.is_empty(),
"already-recorded objects must be skipped"
);
assert!(again.is_empty(), "already-recorded objects must be skipped");
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
3,
@@ -1308,63 +1135,6 @@ mod tests {
);
}
/// The outcome reports the last object actually ENTERED, not the planned
/// tail (Pinata twin of the IPFS test above): the first upload takes 6s
/// against a 2s batch budget, so it still completes — the Pinata loop
/// has no per-request timeout — but the loop-top gate then breaks the
/// batch before the second object starts. `last_attempted` must be the
/// first OID even though two OIDs were never visited; the 6s server
/// sleep makes the break deterministic on any box.
#[sqlx::test]
async fn pin_new_objects_reports_last_attempted_not_planned_tail(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("attempted.git");
let oids = seed_loose_blobs(&repo_path, 3);
let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let endpoint = delaying_pinata_endpoint(
vec![Duration::from_secs(6)],
std::sync::Arc::clone(&requests),
)
.await;
let client = reqwest::Client::new();
let outcome = tokio::time::timeout(
Duration::from_secs(60),
pin_new_objects(
&client,
&endpoint,
"test-jwt",
&repo_path,
"git",
Duration::from_secs(60),
oids.clone(),
&db,
"repo-attempted",
Duration::from_secs(2),
None,
),
)
.await
.expect("one 6s upload plus bounded gates cannot take 60s");
assert_eq!(
outcome.last_attempted,
Some(oids[0].clone()),
"only the first object was entered; the planned tail was never visited"
);
assert_eq!(
outcome.confirmed.len(),
1,
"the slow first upload still completed and recorded"
);
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
1,
"exactly one upload was attempted"
);
}
/// The no-op configuration still short-circuits with the budgeted signature: an
/// empty JWT must return before any git child is spawned and before any request
/// is issued. The `git_bin` here records every invocation, so "git was never
@@ -1406,13 +1176,12 @@ mod tests {
&db,
"repo-merge-test",
Duration::from_secs(60),
None,
),
)
.await
.expect("an unconfigured sink returns immediately");
assert!(pinned.confirmed.is_empty(), "an empty JWT pins nothing");
assert!(pinned.is_empty(), "an empty JWT pins nothing");
assert!(
!log.exists(),
"no git child may be spawned when the sink is not configured"
@@ -1512,7 +1281,6 @@ mod tests {
&db,
"repo-pinata-stalled",
Duration::from_millis(1500),
None,
),
)
.await
@@ -1523,7 +1291,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"a stalled pinata-status check cannot produce a pinned object: {pinned:?}"
);
assert!(
@@ -1569,15 +1337,9 @@ mod tests {
// bytes.
let raw_cid =
gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata skip seed").to_string();
db.record_pinata_cid(
&sha,
&raw_cid,
"QmSeedProviderCid",
Some("repo-seed"),
i64::MAX,
)
.await
.unwrap();
db.record_pinata_cid(&sha, &raw_cid, "QmSeedProviderCid", Some("repo-seed"))
.await
.unwrap();
db.record_pin_source(&sha, "repo-seed").await.unwrap();
let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let endpoint = delaying_pinata_endpoint(
@@ -1609,7 +1371,6 @@ mod tests {
&db,
"repo-pinata-skip-stalled",
Duration::from_millis(1500),
None,
),
)
.await
@@ -1620,7 +1381,7 @@ mod tests {
let elapsed = started.elapsed();
assert!(
pinned.confirmed.is_empty(),
pinned.is_empty(),
"an already-pinned object is skipped, never re-uploaded: {pinned:?}"
);
assert_eq!(
@@ -1664,7 +1425,7 @@ mod tests {
///
/// The lock time is a MARGIN, not a boundary: taking it at 100ms left
/// `has_pinata_cid` racing it on a loaded box, and losing that race makes the read
/// block, time out, and break the batch, which fails on `pinned.confirmed.len() == 1` for a
/// block, time out, and break the batch, which fails on `pinned.len() == 1` for a
/// reason that has nothing to do with the floor. Any time between the
/// `has_pinata_cid` round trip and the upload's 1.7s return proves the same thing.
#[sqlx::test]
@@ -1710,7 +1471,6 @@ mod tests {
&db,
"repo-pinata-spent-budget",
Duration::from_millis(2000),
None,
),
);
let (pinned, ()) = tokio::join!(pin, locker);
@@ -1728,7 +1488,7 @@ mod tests {
record makes api/repos.rs advertise a CID the resolver cannot serve"
);
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"the uploaded pin must still be returned: {pinned:?}"
);
@@ -1796,7 +1556,6 @@ mod tests {
&db,
"repo-pinata-post-upload",
Duration::from_millis(1500),
None,
),
)
.await
@@ -1810,21 +1569,10 @@ mod tests {
drop(lock);
upload.assert_async().await;
// The primary `record_pinata_cid` committed above naming this
// repo as first pinner, so `pin_sources_for_oid` resolves the
// copy with or without the redundant `pin_repo_sources`
// insert: the pair stays confirmed even though the source
// record timed out. The incomplete marker below is still set
// (compensation preserved); only the suppression is gone.
// Prior to the round 10 fix, the push fired regardless of
// every post-upload outcome; the correction since is that
// confirmation follows the durable primary row, not the
// redundant source write.
assert_eq!(
pinned.confirmed.len(),
pinned.len(),
1,
"record_pin_source timed out (pin_repo_sources locked) but the primary row \
covers this repo, so the pair stays confirmed: {pinned:?}"
"the upload succeeded, so this lane still returns the pair: {pinned:?}"
);
assert!(
elapsed < Duration::from_secs(8),
@@ -1841,203 +1589,6 @@ mod tests {
);
}
/// The other half of the source-write contract: when the committed
/// primary row names ANOTHER repo as first pinner, the failed
/// `record_pin_source` was load-bearing (not redundant) and the
/// push stays suppressed. Fixture: a local-only row owned by
/// repo-first (no Pinata CID yet), then a Pinata loop as
/// repo-second with `pin_repo_sources` locked. The upload lands,
/// the primary upsert keeps repo-first, the source insert stalls,
/// and coverage fails — so no pair, but the incomplete marker
/// still lands and the provider CID is still recorded.
#[sqlx::test]
async fn pinata_post_upload_stalled_source_without_coverage_suppresses_push(
pool: sqlx::PgPool,
) {
let db = crate::db::Db::for_testing(pool.clone());
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("pinata_second_source.git");
let oids = seed_loose_blobs(&repo_path, 1);
let sha = oids[0].clone();
// A local-only row owned by repo-first: the writer path below
// (repo-second, Pinata-only so far) must not claim it.
let raw =
gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata loop object 0\n").to_string();
db.record_pinned_cid_with_source(&sha, &raw, "repo-first")
.await
.unwrap();
let mut server = mockito::Server::new_async().await;
let upload = server
.mock("POST", mockito::Matcher::Any)
.with_status(200)
.with_body(r#"{"data":{"cid":"QmSecondSourceProviderCid"}}"#)
.expect(1)
.create_async()
.await;
let (_logs, _log_guard) = capture_logs();
let mut lock = lock_table(&pool, "pin_repo_sources").await;
let client = reqwest::Client::new();
let pinned = tokio::time::timeout(
Duration::from_secs(20),
pin_new_objects(
&client,
&server.url(),
"test-jwt",
&repo_path,
"git",
Duration::from_secs(30),
oids,
&db,
"repo-second",
Duration::from_millis(1500),
None,
),
)
.await
.expect("record + coverage ladders are each floored");
rollback(&mut lock).await;
drop(lock);
upload.assert_async().await;
assert!(
pinned.confirmed.is_empty(),
"repo-second is not covered by the repo-first primary row, so the \
failed source write stays suppressing: {pinned:?}"
);
assert!(
db.pin_sources_incomplete(&sha).await.unwrap(),
"the compensation marker still lands"
);
// The primary upsert itself landed (provider CID recorded under
// the first-pinner row); only the pair was suppressed.
assert!(
db.has_pinata_cid(&sha).await.unwrap(),
"the committed provider row survives the suppressed push"
);
}
/// Hold a row-level lock on one repos row in an open transaction: plain
/// `SELECT`s (fence reads, skip checks) proceed, but any `SELECT ...
/// FOR UPDATE` on the row — exactly what the fenced record takes —
/// blocks until rollback. A table-level lock would also stall the
/// fence's own epoch reads and deadlock the loop under test, proving
/// nothing about the record arm.
async fn hold_repo_row_lock(
pool: &sqlx::PgPool,
repo_id: &str,
) -> sqlx::pool::PoolConnection<sqlx::Postgres> {
let mut conn = pool.acquire().await.unwrap();
sqlx::raw_sql("BEGIN").execute(&mut *conn).await.unwrap();
sqlx::query("SELECT policy_epoch FROM repos WHERE id = $1 FOR UPDATE")
.bind(repo_id)
.fetch_one(&mut *conn)
.await
.unwrap();
conn
}
/// A timed-out `record_pinata_cid` with no verifiable row suppresses the
/// push: `record_pinata_cid` is an explicit transaction, so Elapsed proves
/// nothing, and the verify read finds no row, so nothing is proven that
/// way either. The upload mock still expects the POST — Pinata accepted
/// the bytes — but without a durable row the pair must not be returned,
/// or the reconcile counts a fill that never happened and the push path
/// advertises an unresolvable CID.
#[sqlx::test]
async fn pinata_post_upload_elapsed_record_without_row_suppresses_push(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool.clone());
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("pinata_elapsed_verify.git");
let oids = seed_loose_blobs(&repo_path, 1);
let sha = oids[0].clone();
// A real repo row: the fenced record's `FOR UPDATE` must have a row
// to block on. Against a missing row the predicate takes no lock and
// the record would sail through instead of stalling.
let repo_id = "repo-pinata-elapsed-verify";
let now = chrono::Utc::now();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.to_string(),
name: "elapsed-verify".into(),
owner_did: "did:key:zElapsedVerifyOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
// Capture the fence BEFORE locking: the capture itself is a plain
// epoch read and must succeed for the fenced record path to run.
let fence = crate::ipfs_pin::PolicyFence::capture(&db, repo_id)
.await
.expect("fence captures");
let mut server = mockito::Server::new_async().await;
let upload = server
.mock("POST", mockito::Matcher::Any)
.with_status(200)
.with_body(r#"{"data":{"cid":"QmElapsedVerifyProviderCid"}}"#)
.expect(1)
.create_async()
.await;
let (_logs, _log_guard) = capture_logs();
// Stall only the fenced record's `FOR UPDATE`: fence reads and the
// skip check are plain SELECTs and proceed, the upload lands, then
// the record elapses. The verify read proceeds too and finds no row.
let mut lock = hold_repo_row_lock(&pool, repo_id).await;
let client = reqwest::Client::new();
let pinned = tokio::time::timeout(
Duration::from_secs(30),
pin_new_objects(
&client,
&server.url(),
"test-jwt",
&repo_path,
"git",
Duration::from_secs(30),
oids,
&db,
repo_id,
Duration::from_millis(1500),
Some(&fence),
),
)
.await
.expect(
"record + verify ladders are each floored, so the call returns in seconds, \
never at the lock's lifetime",
);
rollback(&mut lock).await;
drop(lock);
upload.assert_async().await;
assert!(
pinned.confirmed.is_empty(),
"an unverified timed-out record must not return the pair: {pinned:?}"
);
assert!(
!db.has_pinata_cid(&sha).await.unwrap(),
"the cancelled record transaction must not have landed a row"
);
}
#[tokio::test]
async fn test_pin_skipped_when_jwt_empty() {
let client = reqwest::Client::new();
File diff suppressed because it is too large. Load diff
+3 -15
View File
@@ -412,15 +412,7 @@ pub fn build_router(state: AppState) -> Router {
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
.route("/api/v1/repos", get(repos::list_repos))
.route(
"/api/v1/repos/federated",
get(repos::list_federated_repos)
.route_layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.route_layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.federated_rate_limiter.clone(),
trust: state.push_limiter_trust,
})),
)
.route("/api/v1/repos/federated", get(repos::list_federated_repos))
.route("/api/v1/repos/{owner}/{repo}", get(repos::get_repo))
.route(
"/api/v1/repos/{owner}/{repo}/commits",
@@ -619,7 +611,7 @@ async fn ready(State(state): State<AppState>) -> axum::response::Response {
async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
let p2p_peer_id = state.p2p.as_ref().map(|h| h.local_peer_id.to_string());
let mut body = json!({
Json(json!({
"name": "gitlawb-node",
"version": env!("CARGO_PKG_VERSION"),
"did": state.node_did.to_string(),
@@ -628,11 +620,7 @@ async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
"auth": "http-signature-rfc9421",
"identity": "ed25519",
"p2p_peer_id": p2p_peer_id,
});
if let Some(web_url) = &state.config.web_url {
body["web_url"] = json!(web_url);
}
Json(body)
}))
}
pub(crate) async fn stats(State(state): State<AppState>) -> Json<serde_json::Value> {
-4
View File
@@ -190,9 +190,6 @@ pub struct AppState {
/// sink as trigger and accepts unsigned requests from known peers, so it is
/// braked too; each peer's distinct IP gets its own bucket.
pub peer_write_rate_limiter: RateLimiter,
/// Per-client-IP federation budget: 12 requests per minute, with at most
/// 10,000 tracked source keys. Shared by router instances and swept below.
pub federated_rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
@@ -354,7 +351,6 @@ impl AppState {
self.ipfs_work_rate_limiter.cleanup().await;
self.sync_trigger_rate_limiter.cleanup().await;
self.peer_write_rate_limiter.cleanup().await;
self.federated_rate_limiter.cleanup().await;
}
/// Trigger graceful shutdown. Idempotent — calling more than once
+4 -20
View File
@@ -1330,18 +1330,6 @@ mod tests {
.await;
}
/// Use a rooted path without a Windows drive prefix in the remote fixture.
/// The destination still resolves on the temp directory's current drive,
/// while the remote's relative path contains no illegal colon component.
fn absolute_slug_path(path: &Path) -> String {
assert!(path.is_absolute());
path.components()
.filter(|part| !matches!(part, std::path::Component::Prefix(_)))
.collect::<std::path::PathBuf>()
.to_string_lossy()
.replace('\\', "/")
}
#[sqlx::test]
async fn process_batch_rejects_slug_escaping_repos_dir(pool: PgPool) {
// The verified escape from #272: `PathBuf::join` discards everything
@@ -1357,13 +1345,12 @@ mod tests {
// fails on but leaves the parent it created, so assert on both.
let outside = TempDir::new().unwrap();
let escape_dir = outside.path().join("nest");
let escape_path = absolute_slug_path(&escape_dir);
let slug = format!("a/{escape_path}/escape");
let slug = format!("a/{}/escape", escape_dir.display());
let escape_target = escape_dir.join("escape.git");
// Serve the composed URL for real, so a run without the guard genuinely
// clones outside the root rather than merely failing at git.
let rel = format!("a{escape_path}/escape");
let rel = format!("a{}/escape", escape_dir.display());
let (_remote, peer_url) = rooted_remote(&[&rel]);
let did = "did:key:z6MkAttacker";
@@ -1483,7 +1470,6 @@ mod tests {
// ── canonical containment before the git call (issue #272) ───────────────
/// Every ref in `repo`, as one string, for a before/after comparison.
#[cfg(unix)]
fn refs_of(repo: &Path) -> String {
let out = Command::new("git")
.args(["-C", repo.to_str().unwrap(), "for-each-ref"])
@@ -1816,7 +1802,6 @@ mod tests {
/// starvation tests is fixed rather than dependent on how fast the loop
/// runs. Two rows enqueued in the same microsecond would otherwise order
/// arbitrarily.
#[cfg(unix)]
async fn enqueue_at(db: &Db, pool: &PgPool, repo: &str, did: &str, enqueued_at: &str) {
enqueue(db, repo, did).await;
sqlx::query("UPDATE sync_queue SET enqueued_at = $1 WHERE repo = $2")
@@ -2010,8 +1995,7 @@ mod tests {
let outside = TempDir::new().unwrap();
let escape_dir = outside.path().join("nest");
let escape_path = absolute_slug_path(&escape_dir);
let slug = format!("a/{escape_path}/gitlawb-probe");
let slug = format!("a/{}/gitlawb-probe", escape_dir.display());
let escape_target = escape_dir.join("gitlawb-probe.git");
// Two things this fixture must get right or the test is green for the
@@ -2022,7 +2006,7 @@ mod tests {
// run without the guard genuinely clones outside the root instead of
// just failing at git. Db::upsert_peer cannot seed this row: it gates on
// is_public_http_url, which rejects file://.
let rel = format!("a{escape_path}/gitlawb-probe");
let rel = format!("a{}/gitlawb-probe", escape_dir.display());
let (_remote, peer_url) = rooted_remote(&[&rel]);
let did = "did:key:z6MkAttacker";
seed_local_peer(&pool, did, &peer_url).await;
-75
View File
@@ -1,75 +0,0 @@
//! Native Git fixtures shared by timing tests on Unix and Windows.
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// The fixture delays or hangs in its own process, so the production watchdog
/// can terminate it without depending on a shell or descendant process support.
pub(super) enum Behavior<'a> {
Delay(u64),
Hang,
HangOid(&'a str),
HangRepo(&'a str),
LogTypes(&'a Path),
}
pub(super) struct GitShim {
_directory: tempfile::TempDir,
executable: PathBuf,
}
impl Deref for GitShim {
type Target = Path;
fn deref(&self) -> &Path {
&self.executable
}
}
pub(super) fn create(name: &str, behavior: Behavior<'_>) -> GitShim {
static COMPILED: OnceLock<tempfile::TempDir> = OnceLock::new();
let compiled = COMPILED.get_or_init(|| {
let directory = tempfile::tempdir().expect("native Git fixture directory");
let source = directory.path().join("git_shim.rs");
std::fs::write(&source, include_str!("../tests/fixtures/git_shim.rs"))
.expect("write native Git fixture source");
let executable = directory
.path()
.join(format!("git-shim{}", std::env::consts::EXE_SUFFIX));
let output =
std::process::Command::new(std::env::var_os("RUSTC").unwrap_or_else(|| "rustc".into()))
.arg("--edition=2021")
.arg("-Dwarnings")
.arg(&source)
.arg("-o")
.arg(executable)
.output()
.expect("compile native Git fixture with the installed Rust toolchain");
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
directory
});
let directory = tempfile::Builder::new()
.prefix(name)
.tempdir()
.expect("Git fixture instance");
let file_name = format!("git-shim{}", std::env::consts::EXE_SUFFIX);
let executable = directory.path().join(&file_name);
std::fs::copy(compiled.path().join(file_name), &executable).expect("copy native Git fixture");
let config = match behavior {
Behavior::Delay(milliseconds) => format!("delay\n{milliseconds}"),
Behavior::Hang => "hang\n".to_owned(),
Behavior::HangOid(oid) => format!("hang-oid\n{oid}"),
Behavior::HangRepo(repo) => format!("hang-repo\n{repo}"),
Behavior::LogTypes(path) => format!("log-types\n{}", path.display()),
};
std::fs::write(directory.path().join("config"), config).expect("configure native Git fixture");
GitShim {
_directory: directory,
executable,
}
}
File diff suppressed because it is too large. Load diff
-59
View File
@@ -1,59 +0,0 @@
use std::io::{Read, Write};
use std::process::{Command, Stdio};
use std::time::Duration;
fn main() -> std::io::Result<()> {
let executable = std::env::current_exe()?;
let config = std::fs::read_to_string(executable.parent().unwrap().join("config"))?;
let (mode, parameter) = config.split_once('\n').unwrap_or((&config, ""));
let args: Vec<_> = std::env::args_os().skip(1).collect();
if mode == "delay" {
std::thread::sleep(Duration::from_millis(parameter.parse().unwrap()));
}
let batch = args.get(1).is_some_and(|arg| arg == "--batch-check");
let input = if batch {
let mut bytes = Vec::new();
std::io::stdin().read_to_end(&mut bytes)?;
Some(bytes)
} else {
None
};
let cwd = std::env::current_dir()?;
let hang = mode == "hang"
|| (mode == "hang-repo" && cwd.file_name().is_some_and(|name| name == parameter))
|| (mode == "hang-oid"
&& (args.iter().any(|arg| arg == parameter)
|| input
.as_ref()
.is_some_and(|bytes| String::from_utf8_lossy(bytes).trim() == parameter)));
if hang {
std::thread::sleep(Duration::from_secs(30));
std::process::exit(1);
}
if mode == "log-types" && batch {
let mut log = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(parameter)?;
writeln!(
log,
"{} {}",
String::from_utf8_lossy(input.as_ref().unwrap()).trim(),
cwd.file_name().unwrap().to_string_lossy()
)?;
}
let mut child = Command::new("git")
.args(&args)
.stdin(if input.is_some() {
Stdio::piped()
} else {
Stdio::inherit()
})
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()?;
if let Some(bytes) = input {
child.stdin.take().unwrap().write_all(&bytes)?;
}
std::process::exit(child.wait()?.code().unwrap_or(1));
}
-3
View File
@@ -26,10 +26,7 @@ clap = { version = "4", features = ["derive", "env"] }
dirs = "5"
reqwest = { workspace = true }
uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
urlencoding = "2"
libc = "0.2"
alloy = { version = "1", default-features = false, features = [
"contract",
"provider-http",
+21 -541
View File
@@ -4,7 +4,6 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -35,12 +34,7 @@ pub enum CertCmd {
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show a specific ref certificate and verify its signature.
///
/// `--verify` requires `--expect-node <did>`: the queried node's
/// self-reported DID is not a trust anchor, because a node serving a
/// forged certificate also controls its own self-report. Callers
/// must name the issuer they trust.
/// Show a specific ref certificate and verify its signature
Show {
/// Repository in <owner>/<repo> or <repo> format
repo: String,
@@ -55,17 +49,13 @@ pub enum CertCmd {
#[arg(long)]
dir: Option<PathBuf>,
/// Exit non-zero unless the Ed25519 signature verifies AND the
/// issuing node matches --expect-node (which --verify requires:
/// without a caller-supplied anchor there is no trusted issuer to
/// verify against)
/// issuing node matches the queried node (or --expect-node)
#[arg(long)]
verify: bool,
/// Expected issuing node DID for --verify. A valid signature alone
/// only proves the cert is internally consistent — signed by whatever
/// key it names — so --verify also anchors the issuer to a DID you
/// trust. The queried node's self-reported DID is deliberately NOT
/// accepted as a fallback anchor: the node that served the cert can
/// serve a matching self-report.
/// trust: this value when given, else the queried node's DID.
#[arg(long, requires = "verify")]
expect_node: Option<String>,
},
@@ -162,24 +152,6 @@ async fn cmd_show(
let node_did = cert["node_did"].as_str().unwrap_or("?");
let signature = cert["signature"].as_str().unwrap_or("?");
let issued_at = cert["issued_at"].as_str().unwrap_or("?");
// #26 Split PR 3: read the wire-format version. A MISSING key
// selects legacy v1 (an old server that predates the field).
// A PRESENT value must be a JSON integer that fits in u32 and
// names a version this client supports — otherwise the response
// declares a format we cannot represent and we refuse to verify
// rather than guess the payload shape. Reviewer 2 finding:
// collapsing `null`, a string, a float, or an overflow integer
// onto v1 would let a well-signed v1 signature pass `--verify`
// against a server that explicitly said "version 2 (or 99, or
// 4294967297)" — that is the exact mismatch the field exists
// to prevent. parse_cert_version is the load-bearing parser
// that distinguishes missing-key (legacy v1) from invalid-value
// (unsupported).
let parsed_version = parse_cert_version(cert.get("version"));
let version_display: String = match &parsed_version {
Ok(v) => v.to_string(),
Err(reason) => format!("unsupported ({reason})"),
};
println!("Ref Certificate: {cert_id}");
println!(" Ref: {ref_name}");
@@ -188,7 +160,6 @@ async fn cmd_show(
println!(" Pusher: {pusher}");
println!(" Node DID: {node_did}");
println!(" Issued at: {issued_at}");
println!(" Version: {version_display}");
println!(" Signature: {signature}");
println!();
@@ -198,31 +169,9 @@ async fn cmd_show(
// This proves the cert is internally authentic — signed by the key it
// names; the node-DID comparison below covers *which* node that is.
let repo_id = cert["repo_id"].as_str().unwrap_or("");
let verdict = match parsed_version {
// v1 is the only version this client verifies. The v1
// signed payload is the 7-field canonical form with no
// `version` key — see gitlawb-node/src/cert.rs. A future
// v2+ cert has a different signed payload shape (the
// version field becomes part of the signed JSON per
// `v2_signing_payload`), so a v2+ cert from a server this
// client does not know about must NOT be silently verified
// as v1.
// Reviewer 2: refuse rather than guess; the client and
// server must agree on the version.
// parse_cert_version admits exactly one Ok value: 1 (missing key and
// integer 1 both land there; 2, floats, strings, overflow and every
// other spelling come back Err with their own reason). A separate
// `Ok(v != 1)` arm existed here and was DEAD CODE — a test aimed at it
// could never catch drift in this match (review round 2) — so the
// match now mirrors the parser's real contract, which the
// `parse_cert_version_truth_table` test pins.
Ok(_) => verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
),
Err(ref reason) => Err(format!(
"cert declared a version this client cannot represent ({reason}); refusing to verify"
)),
};
let verdict = verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
);
println!("Signature verification:");
match &verdict {
@@ -232,11 +181,7 @@ async fn cmd_show(
);
}
Err(reason) => {
// A version refusal means no cryptographic check ran, so it must
// not print as an INVALID signature: operators and log scrapers
// would read that as a failed Ed25519 verification. INVALID is
// reserved for failures out of verify_signature itself.
println!(" {} — {reason}", signature_verdict_label(&parsed_version));
println!(" INVALID — {reason}");
}
}
@@ -253,39 +198,29 @@ async fn cmd_show(
},
Err(e) => Err(e.to_string()),
};
let current_node_did = current.as_ref().ok().cloned();
for line in did_check_report(&current, node_did) {
println!("{line}");
}
if require_valid {
if let Err(reason) = verdict {
// Round 3: do not claim the signature failed when it was
// never checked. A version refusal means the payload
// shape is unknown, so no signature verification ran.
anyhow::bail!("certificate could not be verified: {reason}");
anyhow::bail!("certificate signature did not verify: {reason}");
}
// A valid signature proves internal consistency only: the payload was
// signed by whatever key the certificate itself names. A hostile
// source can mint a keypair, put its DID in node_did, and self-sign.
// --verify therefore also anchors the issuer — and the anchor must be
// a DID the CALLER trusts, which is exactly what --expect-node is.
// The DID the queried node reports at `/` is NOT an anchor: the same
// node that served a forged certificate can serve a matching
// self-report, and with that fallback a wholly forged cert printed
// VALID and exited 0 (review round 2, demonstrated by execution). So
// --verify without --expect-node refuses to claim a trusted issuer at
// all, rather than laundering the server's self-assertion into one.
match expect_node.as_deref() {
// --verify therefore also anchors the issuer to a trusted DID:
// --expect-node when given, else the DID of the node being queried.
let expected = expect_node.as_deref().or(current_node_did.as_deref());
match expected {
Some(expected) if expected == node_did => {}
Some(expected) => anyhow::bail!(
"certificate is signed by {node_did}, but the expected issuer is {expected} — \
a valid signature alone proves internal consistency, not a trusted issuer"
),
None => anyhow::bail!(
"--verify needs --expect-node <did> to anchor the issuer: the signature is \
valid, but it only proves the cert is self-consistent with the DID it \
names ({node_did}); the queried node's self-reported DID is not a trust \
anchor"
"cannot anchor the issuer: node info is unreachable and no --expect-node was given"
),
}
}
@@ -333,18 +268,6 @@ fn did_check_report(current: &std::result::Result<String, String>, node_did: &st
}
}
/// The stdout label for a failed verification verdict. A `parsed_version`
/// error means `verify_signature` never ran, so the refusal must print as
/// UNSUPPORTED VERSION rather than INVALID, which operators and log
/// scrapers would read as a failed Ed25519 check.
fn signature_verdict_label(parsed_version: &std::result::Result<u32, String>) -> &'static str {
if parsed_version.is_err() {
"UNSUPPORTED VERSION"
} else {
"INVALID"
}
}
/// Rebuild the node's canonical signing payload (field order must match
/// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the
/// certificate's Ed25519 signature against the key embedded in `node_did`.
@@ -391,65 +314,6 @@ fn verify_signature(
.map_err(|_| "Ed25519 signature does not match the signed payload".to_string())
}
/// Parse the wire-format `version` field of a certificate response.
///
/// Semantics (Reviewer 2):
/// - MISSING key (`None`) → `Ok(1)`. An old server predates the field, so
/// the cert is by definition the v1 (pre-versioning) shape; the v1
/// verify path is the only path that can possibly match the bytes.
/// - PRESENT key must be a JSON integer that fits losslessly in `u32`
/// and equals a version this client supports. `null`, a string,
/// a float, an overflow integer, or any version other than 1 all
/// yield `Err`. Collapsing any of these onto v1 would let a
/// well-signed v1 signature pass `--verify` against a response
/// that explicitly declared a format this client cannot represent
/// — which is exactly the mismatch the version field exists to
/// prevent.
///
/// The supported-version set is hard-coded to `{1}` because PR 3 ships
/// only v1 verification; bump this set (and the verdict branch) when
/// v2 verification lands.
fn parse_cert_version(value: Option<&Value>) -> Result<u32, String> {
let v = match value {
None => return Ok(1),
Some(v) => v,
};
// serde_json::Value::as_u64 rejects strings, floats, booleans,
// nulls, arrays, and objects — but it silently truncates floats
// that are whole numbers (e.g. 2.0 → 2). We must reject floats
// explicitly so a server cannot smuggle a v2 cert through the
// float shape.
if v.is_f64() {
return Err(format!(
"version is a JSON number with a fractional part ({v}); expected an integer"
));
}
let n = v.as_u64().ok_or_else(|| {
// null / string / bool / array / object — anything that is
// not a JSON integer.
format!("version is {v}; expected a JSON integer")
})?;
// Lossy narrowing: refuse anything that does not fit in u32.
// u32::MAX is 4_294_967_295; serde_json only goes up to u64.
if n > u32::MAX as u64 {
return Err(format!(
"version {n} does not fit in u32 (max {})",
u32::MAX
));
}
let n = n as u32;
if n != 1 {
return Err(format!(
"this client supports cert version 1 only; server returned {n}"
));
}
Ok(1)
}
async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str) -> Result<String> {
if id.len() >= 36 {
return Ok(id.to_string());
@@ -947,15 +811,7 @@ mod tests {
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let got = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(kp.did().as_str().to_string()),
)
.await;
let got = cmd_show("alice/r".to_string(), id, server.url(), None, true, None).await;
assert!(got.is_ok(), "valid sig + matching issuer must pass");
cert.assert_async().await;
}
@@ -971,17 +827,10 @@ mod tests {
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let err = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(other.did().as_str().to_string()),
)
.await
.unwrap_err()
.to_string();
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("expected issuer"), "got: {err}");
cert.assert_async().await;
}
@@ -999,7 +848,7 @@ mod tests {
.await
.unwrap_err()
.to_string();
assert!(err.contains("--expect-node"), "got: {err}");
assert!(err.contains("cannot anchor the issuer"), "got: {err}");
cert.assert_async().await;
}
@@ -1040,7 +889,7 @@ mod tests {
.await
.unwrap_err()
.to_string();
assert!(err.contains("does not match"), "got: {err}");
assert!(err.contains("signature did not verify"), "got: {err}");
cert.assert_async().await;
}
@@ -1124,373 +973,4 @@ mod tests {
assert!(report.contains("verify offline"), "got: {report}");
assert!(!report.contains("WARNING"), "got: {report}");
}
/// #26 Split PR 3: a missing `version` field on a cert defaults
/// to 1, so an old server's response is forward-compatible with
/// a new client. The v1 verify path is taken, and a
/// well-signed v1 cert round-trips through `verify_signature`.
///
/// Reviewer 1 finding: the previous shape only parsed the field
/// and never called `verify_signature`, so the test could not
/// fail if the verdict branch was wired to the wrong path. This
/// version signs and verifies the full round-trip.
#[test]
fn missing_version_defaults_to_1_and_verifies() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
// Simulate the JSON a v1 server returns: no `version` key.
let cert_json = serde_json::json!({
"id": "test-id",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig.clone(),
"issued_at": "2026-07-22T00:00:00+00:00",
// no `version` field
});
// Forward-compat: the parse yields v1 (the only version this
// client verifies), and the verdict arm matching Ok(1) feeds
// the v1 payload through verify_signature end to end.
assert_eq!(parse_cert_version(cert_json.get("version")).unwrap(), 1);
let verdict = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(
verdict.is_ok(),
"missing version must take the v1 verify path: {verdict:?}"
);
}
/// #26 Split PR 3: an explicit `version: 1` on a cert is the v1
/// verify path. A v1 cert with an explicit version verifies the
/// same as a v1 cert without one — round-trip through
/// `verify_signature`.
#[test]
fn explicit_version_1_takes_v1_path() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
// Simulate a v1 cert with an explicit `version: 1` field.
let cert_json = serde_json::json!({
"id": "test-id",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig.clone(),
"issued_at": "2026-07-22T00:00:00+00:00",
"version": 1,
});
// The parse arms the Ok(1) verdict branch, which calls
// verify_signature on the v1 payload. A signature mismatch
// here would mean the verdict branch was wired to the
// wrong path or the payload drifted.
assert_eq!(parse_cert_version(cert_json.get("version")).unwrap(), 1);
let verdict = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(
verdict.is_ok(),
"explicit version: 1 must take the v1 verify path: {verdict:?}"
);
}
/// #26 Split PR 3 + Reviewer 2: parse_cert_version is the
/// load-bearing gate that distinguishes a missing-key (legacy
/// server) from a present-but-malformed value. The four cases
/// below pin every cell of that truth table; collapsing any
/// pair would let a server advertise "version 2" (or 99, or
/// 4294967297, or "two") while the client runs the v1
/// signature path on a different signed payload — a hostile
/// misconfiguration that `--verify` must not silently accept.
#[test]
fn parse_cert_version_truth_table() {
// MISSING key → legacy v1 (the v1 path is the only one that
// could match pre-versioning bytes).
assert_eq!(
parse_cert_version(None).unwrap(),
1,
"a missing version field is the legacy v1 path"
);
// Explicit 1 → v1.
assert_eq!(
parse_cert_version(Some(&serde_json::json!(1))).unwrap(),
1,
"explicit version 1 is the v1 path"
);
// Explicit 2 → Err, NOT Ok(1). Reviewer 2: a v1 signature
// must not verify against a server that said "version 2".
assert!(
parse_cert_version(Some(&serde_json::json!(2))).is_err(),
"explicit version 2 must not collapse to v1"
);
// u32 overflow (2^32 + 1) → Err, NOT Ok(1). The previous
// `.map(|v| v as u32)` silently truncated this to 1.
let overflow = serde_json::json!(u64::from(u32::MAX) + 1);
assert!(
parse_cert_version(Some(&overflow)).is_err(),
"a version that overflows u32 must not truncate to 1"
);
// Non-numeric: a string, a float, null, bool — every shape
// that is not a JSON integer is rejected. None of these
// may collapse to v1.
for bad in [
serde_json::json!("two"),
serde_json::json!(2.0), // serde_json::Value::is_f64 is true for this
serde_json::json!(2.5), // obviously fractional
serde_json::json!(true),
serde_json::json!(null),
serde_json::json!([2]),
serde_json::json!({"v": 2}),
] {
assert!(
parse_cert_version(Some(&bad)).is_err(),
"non-integer version {bad} must not collapse to v1"
);
}
}
/// A version refusal must not print as INVALID — no cryptographic
/// check ran, so labelling it INVALID would tell operators an Ed25519
/// verification failed when the payload shape was the disqualifier.
/// Removing the label branch fails this test.
#[test]
fn signature_verdict_label_distinguishes_refusal_from_bad_signature() {
assert_eq!(
signature_verdict_label(&Err("version 2 not supported".to_string())),
"UNSUPPORTED VERSION"
);
assert_eq!(signature_verdict_label(&Ok(1)), "INVALID");
}
/// #26 Split PR 3, round 2: a `version: 2` certificate must fail
/// `--verify` THROUGH `cmd_show` itself, not through a copy of its
/// match. The prior form of this test duplicated the verdict match in
/// the test body and aimed at an `Ok(v != 1)` arm the real parser can
/// never produce, so gutting `cmd_show`'s rejection kept every test
/// green (demonstrated by the reviewer by execution). Driving the
/// command end to end is what makes this a guard: the mock serves a
/// cert whose SIGNATURE IS VALID for v1 bytes but which declares
/// version 2, and the command must exit Err with the version named —
/// the v1 verify path must not launder it through.
#[tokio::test]
async fn cmd_show_verify_rejects_v2_even_with_valid_v1_signature() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
// Sign a v1-shaped payload so the ONLY disqualifier is the version.
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let cert_json = serde_json::json!({
"id": "0123456789abcdef0123456789abcdef0123",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
"version": 2,
});
let mut server = mockito::Server::new_async().await;
// 36-char id short-circuits resolve_cert_id; "owner/repo" input
// short-circuits resolve_repo — the cert GET is the only required
// route. `/` may be probed for the contextual DID line; serving it
// keeps the test about the verdict, not about network noise.
let _cert = server
.mock(
"GET",
"/api/v1/repos/o/r/certs/0123456789abcdef0123456789abcdef0123",
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_json.to_string())
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(serde_json::json!({ "did": node_did }).to_string())
.create_async()
.await;
let err = cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
Some(node_did.clone()),
)
.await
.expect_err("a version-2 cert must fail --verify even with a valid v1 signature");
let msg = format!("{err:#}");
assert!(
msg.contains("version"),
"the failure must name the version mismatch, not a generic invalid signature: {msg}"
);
}
/// Round 2's second demonstrated escape: with no --expect-node, the old
/// fallback anchored the issuer to the DID the QUERIED NODE reports at
/// `/` — which a forger controls along with the cert, so a wholly forged
/// self-signed certificate passed --verify with exit 0. `cmd_show` with
/// --verify and no --expect-node must now refuse to claim a trusted
/// issuer, even when the served cert's signature is valid and the node's
/// self-report matches it.
#[tokio::test]
async fn cmd_show_verify_without_expect_node_refuses_self_asserted_anchor() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let cert_json = serde_json::json!({
"id": "0123456789abcdef0123456789abcdef0123",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
});
let mut server = mockito::Server::new_async().await;
// P3 (reviewer round 2): both mocks are required to be
// hit for the test to mean anything. `mockito 1.7`'s
// `assert_on_drop` is off by default, so the previous
// `_cert` and `_root` bindings would silently let a
// regression that skipped the root fetch pass.
// `.expect(2)` pins each mock's call count (the cert mock
// is hit twice because the test runs `cmd_show` twice) and
// `.assert_async()` at the end fires if any are unmet.
let cert_mock = server
.mock(
"GET",
"/api/v1/repos/o/r/certs/0123456789abcdef0123456789abcdef0123",
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_json.to_string())
.expect(2)
.create_async()
.await;
// The forger's matching self-report — the exact shape
// that used to convert into a trust anchor. The guard's
// whole point is to make the test FAIL if a future
// regression stops calling `/` here: `.expect(2)` +
// `.assert_async()` makes that dependency visible.
let root_mock = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(serde_json::json!({ "did": node_did }).to_string())
.expect(2)
.create_async()
.await;
let err = cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
None,
)
.await
.expect_err("--verify without --expect-node must not mint a trust anchor from the node's self-report");
let msg = format!("{err:#}");
assert!(
msg.contains("--expect-node"),
"the refusal must tell the caller how to anchor trust: {msg}"
);
// Positive control: the same cert with the DID explicitly trusted
// passes, so the guard is about the anchor, not the signature.
cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
Some(node_did),
)
.await
.expect("explicitly anchored --verify of a valid v1 cert must pass");
// P3 (reviewer round 2): confirm the mocks were actually
// hit. Without this, a regression that skipped the cert or
// root fetches would still produce a passing test (the
// `expect_err` arm succeeds, the positive control succeeds)
// and the test would mean nothing.
cert_mock.assert_async().await;
root_mock.assert_async().await;
}
}
+36 -177
View File
@@ -3,7 +3,7 @@
//! Checks:
//! 1. Identity — ~/.gitlawb/identity.pem exists, DID parseable
//! 2. Registration — ~/.gitlawb/ucan.json exists (registered with a node)
//! 3. GITLAWB_NODE — env var points at a public node (unset is advisory)
//! 3. GITLAWB_NODE — env var is set to a non-localhost URL
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.twigpine.com)
//! 5. git helper — git-remote-gitlawb is in PATH
//! 6. git — git is installed
@@ -80,6 +80,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
});
let mut checks = Vec::new();
let mut all_ok = true;
// ── 1. Identity ───────────────────────────────────────────────────────
let pem_path = dir.join("identity.pem");
@@ -140,16 +141,32 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
checks.push(gitlawb_node_env_check(
std::env::var("GITLAWB_NODE").ok().as_deref(),
));
match std::env::var("GITLAWB_NODE") {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Ok(v) if is_loopback_url(&v) => {
checks.push(Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
));
}
Ok(v) if !v.is_empty() => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
_ => {
checks.push(Check::fail(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
"export GITLAWB_NODE=https://node.twigpine.com",
));
}
}
// ── 4. Node connectivity ──────────────────────────────────────────────
// GITLAWB_NODE="" binds args.node to "" through clap, and check 3 groups
// empty with unset. The probe must apply the same normalization or it
// would dial "" and Fail while the env row only Warns.
let node_url = effective_node_url(&args.node);
let client = NodeClient::new(node_url, None);
let client = NodeClient::new(&args.node, None);
match client.get("/").await {
Ok(resp) if resp.status().is_success() => {
let info = resp.json::<serde_json::Value>().await.unwrap_or_default();
@@ -158,7 +175,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
let short_did = did.chars().take(30).collect::<String>();
checks.push(Check::pass(
"node",
format!("{} — v{version} ({short_did}…)", node_url),
format!("{} — v{version} ({short_did}…)", args.node),
));
// Capability drift: a node newer than this CLI may require features
// (RFC 9421 signing, iCaptcha) the CLI doesn't speak.
@@ -177,14 +194,14 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
Ok(resp) => {
checks.push(Check::fail(
"node",
format!("{} returned HTTP {}", node_url, resp.status()),
format!("{} returned HTTP {}", args.node, resp.status()),
"check GITLAWB_NODE env var or try gl register --node <url>",
));
}
Err(e) => {
checks.push(Check::fail(
"node",
format!("{} unreachable: {e}", node_url),
format!("{} unreachable: {e}", args.node),
"check your internet connection or set GITLAWB_NODE",
));
}
@@ -268,21 +285,23 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
CheckState::Fail => "✗",
};
println!(" {icon} {:<24} {}", check.label, check.detail);
if matches!(check.state, CheckState::Fail) {
all_ok = false;
}
}
println!();
let has_failures_now = has_failures(&checks);
let has_issues = checks
.iter()
.any(|c| matches!(c.state, CheckState::Fail | CheckState::Warn));
if !has_issues {
println!("Everything looks good. Run `gl quickstart` to create your first repo.");
} else {
if has_failures_now {
println!("Some checks failed. Suggested fixes:");
} else {
if all_ok {
println!("Setup looks good with some warnings:");
} else {
println!("Some checks failed. Suggested fixes:");
}
for check in &checks {
if matches!(check.state, CheckState::Fail | CheckState::Warn) {
@@ -292,75 +311,14 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
}
println!();
if has_failures_now {
if !all_ok {
println!("Run `gl quickstart` for a guided setup.");
}
}
// #357: a non-zero exit is the only signal scripting and CI have. Warn-only
// rows stay exit 0 (iCaptcha offline, version drift, shell-alias shadowing
// are all non-fatal); Fail-class rows (identity, registration, node
// reachable, git-remote-gitlawb present, git present) flip to exit 1.
// Direct `process::exit` rather than returning Err: the prose above is the
// user-facing summary, and an anyhow Error frame would just duplicate it.
if has_failures(&checks) {
std::process::exit(1);
}
Ok(())
}
/// Classify the `GITLAWB_NODE` env var for `gl doctor`. Extracted so unit
/// tests can pin the Warn tier directly (#357): an unset env is advisory, not
/// a failure, because `--node` defaults to `https://node.twigpine.com` and the
/// CLI works without it. Flipping this back to `Check::fail` must break a
/// test, not just change prose.
fn gitlawb_node_env_check(env_val: Option<&str>) -> Check {
match env_val {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Some(v) if is_loopback_url(v) => Check::pass(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
),
),
Some(v) if !v.is_empty() => Check::pass("GITLAWB_NODE", v.to_string()),
// `--node` defaults to `https://node.twigpine.com` and the CLI works
// fine without the variable, so an unset env is advisory, not a
// failure. Warn-class keeps the exit code 0 for a stock working
// install; this used to be Fail, which the obvious "return non-zero
// on any failure" change would have flipped to exit 1 on every
// default-config install (#357).
_ => Check::warn(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
format!("export GITLAWB_NODE={PUBLIC_NODE}"),
),
}
}
/// The URL the connectivity probe should dial. `GITLAWB_NODE` set but empty
/// (or whitespace-only) binds `args.node` to `""` through clap rather than
/// to the `default_value`, and check 3 already groups empty with unset, so
/// the probe applies the same normalization: an empty effective URL means
/// the public node, same as unset.
fn effective_node_url(node: &str) -> &str {
if node.trim().is_empty() {
PUBLIC_NODE
} else {
node
}
}
/// True when at least one check is Fail-class. Warn-only runs (iCaptcha offline,
/// version drift, shell-alias shadowing, GITLAWB_NODE unset) keep exit 0;
/// this is the gating predicate for #357's exit-status fix.
fn has_failures(checks: &[Check]) -> bool {
checks.iter().any(|c| matches!(c.state, CheckState::Fail))
}
/// Check if a binary name exists anywhere on PATH.
/// True when the rc file contains a real `unalias` command naming `gl` —
/// not a comment, and not a longer word like `unalias global`. Ordering
@@ -686,103 +644,4 @@ mod tests {
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("GitHub API returned HTTP 403"));
}
/// #357: an all-Ok run does not flip the exit code to non-zero — a healthy
/// install must keep returning 0 so scripts like `gl doctor && gl push`
/// keep working.
#[test]
fn exit_predicate_is_false_for_all_ok() {
let checks = vec![
Check::pass("identity", "ok"),
Check::pass("registration", "ok"),
Check::pass("git", "ok"),
];
assert!(!has_failures(&checks));
}
/// #357: Warn-class rows (iCaptcha offline, version drift, shell-alias
/// shadowing, GITLAWB_NODE unset on a default-config install) are NOT
/// exit-status failures — they are still printed to the user but the
/// process exits 0. This is the rule that the previous "any
/// Fail-or-Warn row → exit 1" change would have broken for stock
/// installs that simply have not set the env var.
#[test]
fn exit_predicate_is_false_for_warn_only() {
let checks = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export GITLAWB_NODE=..."),
Check::warn("iCaptcha", "offline", "check connectivity"),
Check::warn("version", "drift", "upgrade"),
Check::warn("shell alias", "omz git plugin", "unalias gl"),
];
assert!(!has_failures(&checks));
}
/// #357: a single Fail-class row trips the exit code. The set of Fail
/// rows is whatever `run` writes through `Check::fail`: identity missing
/// or unparseable, registration missing/malformed, node unreachable,
/// git-remote-gitlawb absent, git absent.
#[test]
fn exit_predicate_is_true_when_a_fail_row_is_present() {
let checks = vec![
Check::pass("registration", "ok"),
Check::fail("identity", "missing", "gl identity new"),
Check::pass("git", "ok"),
];
assert!(has_failures(&checks));
let mixed_warns = vec![
Check::pass("identity", "ok"),
Check::warn("GITLAWB_NODE", "unset", "export"),
Check::fail("node", "unreachable", "check network"),
Check::warn("iCaptcha", "offline", "check"),
];
assert!(has_failures(&mixed_warns));
}
/// #357: `GITLAWB_NODE` unset must be Warn, not Fail. This pins the
/// re-tier so flipping it back to `Check::fail` breaks the suite — the
/// previous `has_failures`-only tests built their own `Check::warn` rows
/// and could not see the re-tier at all.
#[test]
fn gitlawb_node_env_unset_is_warn() {
let check = gitlawb_node_env_check(None);
assert!(matches!(check.state, CheckState::Warn));
assert!(check.detail.contains("not set"));
let empty = gitlawb_node_env_check(Some(""));
assert!(matches!(empty.state, CheckState::Warn));
}
/// `GITLAWB_NODE=""` binds `args.node` to `""` through clap, and check 3
/// groups empty with unset. The probe must dial the same URL unset
/// would — removing the normalization makes check 4 probe `""` and Fail
/// while the env row only Warns, so an empty env install exits 1.
#[test]
fn effective_node_url_treats_empty_and_blank_as_unset() {
assert_eq!(effective_node_url(""), PUBLIC_NODE);
assert_eq!(effective_node_url(" "), PUBLIC_NODE);
assert_eq!(effective_node_url("\t\n"), PUBLIC_NODE);
assert_eq!(
effective_node_url("http://127.0.0.1:7545"),
"http://127.0.0.1:7545"
);
}
#[test]
fn gitlawb_node_env_set_is_pass() {
let check = gitlawb_node_env_check(Some("https://node.twigpine.com"));
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("https://node.twigpine.com"));
}
#[test]
fn gitlawb_node_env_loopback_is_pass() {
for url in ["http://127.0.0.1:7545", "http://localhost:7545"] {
let check = gitlawb_node_env_check(Some(url));
assert!(
matches!(check.state, CheckState::Ok),
"loopback {url} must be Ok, not Warn/Fail"
);
}
}
}
+33 -5
View File
@@ -131,13 +131,23 @@ async fn cmd_new_with_reader(
}
}
crate::secret_file::create_dir(&dir)
fs::create_dir_all(&dir)
.with_context(|| format!("failed to create directory {}", dir.display()))?;
let keypair = Keypair::generate();
let pem = keypair.to_pem()?;
crate::secret_file::write(&path, pem.as_bytes())?;
// Write with restricted permissions
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::write(&path, pem.as_bytes())?;
fs::set_permissions(&path, fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
fs::write(&path, pem.as_bytes())?;
}
let did = keypair.did();
println!("✓ Generated new identity");
@@ -192,7 +202,16 @@ async fn cmd_backup(out: Option<PathBuf>, dir: Option<PathBuf>) -> Result<()> {
.join("identity.pem.bak")
});
crate::secret_file::write(&dest, pem.as_bytes())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::write(&dest, pem.as_bytes())?;
fs::set_permissions(&dest, fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
fs::write(&dest, pem.as_bytes())?;
}
println!("✓ Identity backed up");
println!(" DID: {}", keypair.did());
@@ -243,10 +262,19 @@ async fn cmd_restore_with_reader(
}
}
crate::secret_file::create_dir(&base)
fs::create_dir_all(&base)
.with_context(|| format!("failed to create directory {}", base.display()))?;
crate::secret_file::write(&dest, pem.as_bytes())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::write(&dest, pem.as_bytes())?;
fs::set_permissions(&dest, fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
fs::write(&dest, pem.as_bytes())?;
}
println!("✓ Identity restored");
println!(" DID: {}", keypair.did());
+15 -6
View File
@@ -109,16 +109,16 @@ pub async fn run(args: InitArgs) -> Result<()> {
.dir
.clone()
.unwrap_or_else(|| dirs::home_dir().unwrap_or_default().join(".gitlawb"));
crate::secret_file::create_dir(&ucan_dir)?;
std::fs::create_dir_all(&ucan_dir)?;
let record = json!({
"ucan": ucan,
"node": args.node,
"did": did.to_string(),
"saved_at": chrono::Utc::now().to_rfc3339(),
});
crate::secret_file::write(
&ucan_dir.join("ucan.json"),
serde_json::to_string_pretty(&record)?.as_bytes(),
std::fs::write(
ucan_dir.join("ucan.json"),
serde_json::to_string_pretty(&record)?,
)?;
}
}
@@ -232,13 +232,22 @@ fn generate_identity(dir: Option<&std::path::Path>) -> Result<gitlawb_core::iden
.context("could not determine home directory")?
.join(".gitlawb")
};
crate::secret_file::create_dir(&base)?;
std::fs::create_dir_all(&base)?;
let keypair = gitlawb_core::identity::Keypair::generate();
let pem = keypair.to_pem()?;
let path = base.join("identity.pem");
crate::secret_file::write(&path, pem.as_bytes())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::write(&path, pem.as_bytes())?;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
std::fs::write(&path, pem.as_bytes())?;
}
Ok(keypair)
}
+8 -178
View File
@@ -127,93 +127,19 @@ async fn cmd_list(node: String, dir: Option<PathBuf>) -> Result<()> {
println!("IPFS pins ({count}) on {node}");
println!();
for pin in &pins {
for line in render_pin(pin) {
println!("{line}");
}
let cid = pin["cid"].as_str().unwrap_or("?");
let sha = pin["sha256_hex"].as_str().unwrap_or("?");
let pinned_at = pin["pinned_at"].as_str().unwrap_or("?");
// Trim pinned_at to date+time without subseconds
let ts = crate::text::truncate(pinned_at, 19);
println!(" {cid}");
println!(" sha256: {sha}");
println!(" pinned: {ts}");
println!();
}
Ok(())
}
/// The rendered lines for one pins-listing entry.
///
/// #218 review round 8 P2 — why this is not just `pin["cid"]`: the wire format
/// gained a provenance split, and `cid` is now NULLABLE. The node stopped
/// aliasing a Pinata provider CID into `cid` (round 3) because `cid` is the
/// key `GET /ipfs/{cid}` resolves against, and a Pinata provider CID is not
/// resolvable there — advertising one would send clients to a guaranteed 404.
/// The truthful representation of a Pinata-only row is therefore `cid: null`
/// plus a `pinata_cid`. This function was reading only `cid`, so such a row
/// printed as a bare `?` — the CLI showed nothing usable for an object that is
/// in fact durably stored, and nothing in the client read `pinata_cid` at all.
///
/// So: the heading is the node-resolvable CID when there is one, and otherwise
/// the Pinata provider CID, labelled as such so nobody feeds it back to this
/// node's resolver. A `backends:` line names where the bytes actually are,
/// taken from the writer-owned `local_pinned` / `pinata_pinned` booleans rather
/// than re-inferred from CID shape (which is what the node's own comment warns
/// against). Older nodes send neither flag; their `cid`-only rows fall back to
/// CID presence and render exactly as they did before.
fn render_pin(pin: &Value) -> Vec<String> {
let sha = pin["sha256_hex"].as_str().unwrap_or("?");
let pinned_at = pin["pinned_at"].as_str().unwrap_or("?");
// Trim pinned_at to date+time without subseconds; truncate() is
// boundary-safe (a raw &s[..19] panics on a split code point).
let ts = if pinned_at.len() >= 19 {
crate::text::truncate(pinned_at, 19)
} else {
pinned_at
};
// `cid` and `local_cid` carry the same raw CID; read both so a node that
// ships only one of them still renders.
let local_cid = pin["cid"]
.as_str()
.or_else(|| pin["local_cid"].as_str())
.filter(|s| !s.is_empty());
let pinata_cid = pin["pinata_cid"].as_str().filter(|s| !s.is_empty());
// Writer-owned flags when present; CID presence is the pre-split fallback.
let local_pinned = pin["local_pinned"]
.as_bool()
.unwrap_or_else(|| local_cid.is_some());
let pinata_pinned = pin["pinata_pinned"]
.as_bool()
.unwrap_or_else(|| pinata_cid.is_some());
let mut lines = Vec::new();
match (local_cid, pinata_cid) {
// Node-resolvable CID present: it leads, as it always has.
(Some(cid), _) => lines.push(format!(" {cid}")),
// Pinata-only: show the provider CID rather than "?", and say plainly
// that this node's resolver will not serve it.
(None, Some(p)) => lines.push(format!(" {p} (pinata provider CID)")),
(None, None) => lines.push(" ?".to_string()),
}
lines.push(format!(" sha256: {sha}"));
lines.push(format!(" pinned: {ts}"));
// A dual row keeps the provider CID visible too; it differs from the local
// CID and is the only key that resolves on Pinata's gateway.
if local_cid.is_some() {
if let Some(p) = pinata_cid {
lines.push(format!(" pinata: {p}"));
}
}
let backends = match (local_pinned, pinata_pinned) {
(true, true) => "local ipfs, pinata",
(true, false) => "local ipfs",
(false, true) => "pinata",
// Neither flag set: the node filters such rows out, so this is only
// reachable from a hand-rolled response. Say so rather than imply
// durability the row does not claim.
(false, false) => "none recorded",
};
lines.push(format!(" backends: {backends}"));
lines
}
/// Automatic resumes attempted after the initial request when the node reports a
/// truncated legacy scan, so at most `MAX_SCAN_RESUMES + 1` node calls per invocation.
const MAX_SCAN_RESUMES: usize = 8;
@@ -707,102 +633,6 @@ mod tests {
dir
}
/// #218 review round 8 P2: a Pinata-only row must render its provider CID,
/// not a bare `?`. The node stopped aliasing that CID into `cid` because
/// `cid` is the key `GET /ipfs/{cid}` resolves on and a provider CID 404s
/// there — so `cid` is now legitimately null for such a row, and a client
/// that reads only `cid` shows the user nothing for an object that IS
/// durably stored. The heading must carry the provider CID and label it,
/// and the row must not claim a local pin.
#[test]
fn render_pin_shows_provider_cid_for_a_pinata_only_row() {
let pin: Value = serde_json::from_str(
r#"{"sha256_hex":"abc123","cid":null,"local_cid":null,
"pinata_cid":"QmProviderOnly","local_pinned":false,
"pinata_pinned":true,"pinned_at":"2026-07-02T12:00:00.123456Z"}"#,
)
.unwrap();
let out = render_pin(&pin).join("\n");
assert!(
out.contains("QmProviderOnly"),
"the provider CID must be shown; got:\n{out}"
);
assert!(
!out.contains(" ?"),
"a Pinata-only row must not render as a bare `?`; got:\n{out}"
);
assert!(
out.contains("backends: pinata"),
"the row must say where the bytes are; got:\n{out}"
);
assert!(
out.contains("pinata provider CID"),
"the heading must be labelled so it is not fed back to this node's \
resolver, which cannot serve it; got:\n{out}"
);
}
/// A dual row leads with the node-resolvable CID (unchanged behavior) and
/// additionally surfaces the provider CID, which is a different key and the
/// only one Pinata's gateway answers for.
#[test]
fn render_pin_shows_both_cids_for_a_dual_row() {
let pin: Value = serde_json::from_str(
r#"{"sha256_hex":"abc123","cid":"bafyLocal","local_cid":"bafyLocal",
"pinata_cid":"QmProvider","local_pinned":true,
"pinata_pinned":true,"pinned_at":"2026-07-02T12:00:00.123456Z"}"#,
)
.unwrap();
let out = render_pin(&pin).join("\n");
assert!(
out.starts_with(" bafyLocal"),
"the resolver key leads; got:\n{out}"
);
assert!(
out.contains("pinata: QmProvider"),
"the provider CID must still be reachable from the listing; got:\n{out}"
);
assert!(out.contains("backends: local ipfs, pinata"), "got:\n{out}");
}
/// Backward compatibility: a pre-provenance node sends `cid` alone, with no
/// `local_pinned` / `pinata_pinned` flags. That row must render as it always
/// did, with the backends line inferred from CID presence.
#[test]
fn render_pin_handles_a_legacy_cid_only_row() {
let pin: Value = serde_json::from_str(
r#"{"sha256_hex":"abc123","cid":"bafyone",
"pinned_at":"2026-07-02T12:00:00.123456Z"}"#,
)
.unwrap();
let out = render_pin(&pin).join("\n");
assert!(out.starts_with(" bafyone"), "got:\n{out}");
assert!(out.contains("sha256: abc123"), "got:\n{out}");
assert!(
out.contains("pinned: 2026-07-02T12:00:00"),
"the timestamp is still trimmed to seconds; got:\n{out}"
);
assert!(out.contains("backends: local ipfs"), "got:\n{out}");
}
/// A node-supplied timestamp whose 19th byte splits a multi-byte code
/// point must not panic the CLI — `&s[..19]` panics there; `truncate`
/// returns the boundary-safe prefix.
#[test]
fn render_pin_does_not_panic_on_a_non_boundary_timestamp() {
let mut ts = "2026-07-02T12:00:0".to_string();
ts.push('é'); // 2-byte char starting at byte 18: byte 19 is its tail
assert_eq!(ts.len(), 20);
assert!(std::panic::catch_unwind(|| &ts[..19]).is_err());
let pin: Value = serde_json::json!({
"sha256_hex":"abc123","cid":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4",
"local_cid":null,"pinata_cid":null,"local_pinned":true,
"pinata_pinned":false,"pinned_at":ts,
});
let out = render_pin(&pin).join("\n");
assert!(out.contains("pinned: 2026-07-02T12:00:0"), "got:\n{out}");
}
#[tokio::test]
async fn test_cmd_list_signs_request_and_renders_pins() {
let mut server = mockito::Server::new_async().await;
-1
View File
@@ -26,7 +26,6 @@ mod protect;
mod quickstart;
mod register;
mod repo;
mod secret_file;
mod star;
mod status;
mod sync;
+1 -5
View File
@@ -142,11 +142,7 @@ pub async fn run(args: MirrorArgs) -> Result<()> {
println!();
println!("✓ Mirror complete: {name}");
println!(" Clone: git clone {gitlawb_url}");
// Only print View: when the node advertises a web_url — self-hosted nodes
// without a web front-end would otherwise produce a 404 link (#370).
if let Some(web_url) = crate::repo::fetch_node_web_url(&args.node).await {
println!(" View: {web_url}/{owner_short}/{name}");
}
println!(" View: https://twigpine.com/{owner_short}/{name}");
Ok(())
}
+18 -20
View File
@@ -100,29 +100,19 @@ pub async fn run(args: QuickstartArgs) -> Result<()> {
let payload: Value = resp.json().await.unwrap_or_default();
let ucan = payload["ucan"].as_str().unwrap_or("");
if !ucan.is_empty() {
crate::secret_file::create_dir(&dir)?;
std::fs::create_dir_all(&dir)?;
let record = json!({
"ucan": ucan,
"node": args.node,
"did": did,
"saved_at": chrono::Utc::now().to_rfc3339(),
});
crate::secret_file::write(
&ucan_path,
serde_json::to_string_pretty(&record)?.as_bytes(),
)?;
let trust = payload["trust_score"].as_f64().unwrap_or(0.0);
println!(" ✓ Registered successfully");
println!(" Trust score: {trust:.2}");
println!(" UCAN saved to {}", ucan_path.display());
} else {
// A 2xx without a usable ucan is not a registration; the
// response may not even be JSON. Saying success here would
// also leave a stale ucan.json from another node in place.
println!(" ✗ Registration returned no UCAN (unexpected response body)");
// No retry hint here: a `gl register` instruction without
// this invocation's --dir would load a different identity.
std::fs::write(&ucan_path, serde_json::to_string_pretty(&record)?)?;
}
let trust = payload["trust_score"].as_f64().unwrap_or(0.0);
println!(" ✓ Registered successfully");
println!(" Trust score: {trust:.2}");
println!(" UCAN saved to {}", ucan_path.display());
println!();
}
Ok(resp) => {
@@ -236,15 +226,23 @@ pub async fn run(args: QuickstartArgs) -> Result<()> {
// ── Helpers ───────────────────────────────────────────────────────────────
fn generate_identity(dir: &std::path::Path) -> Result<gitlawb_core::identity::Keypair> {
crate::secret_file::create_dir(dir)
.with_context(|| format!("failed to create {}", dir.display()))?;
fn generate_identity(dir: &PathBuf) -> Result<gitlawb_core::identity::Keypair> {
std::fs::create_dir_all(dir).with_context(|| format!("failed to create {}", dir.display()))?;
let keypair = gitlawb_core::identity::Keypair::generate();
let pem = keypair.to_pem()?;
let path = dir.join("identity.pem");
crate::secret_file::write(&path, pem.as_bytes())?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::write(&path, pem.as_bytes())?;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?;
}
#[cfg(not(unix))]
{
std::fs::write(&path, pem.as_bytes())?;
}
let did = keypair.did();
println!(" ✓ Generated new identity");
+2 -5
View File
@@ -72,10 +72,7 @@ pub async fn run(args: RegisterArgs) -> Result<()> {
"did": did.to_string(),
"saved_at": chrono::Utc::now().to_rfc3339(),
});
crate::secret_file::write(
&ucan_path,
serde_json::to_string_pretty(&record)?.as_bytes(),
)?;
std::fs::write(&ucan_path, serde_json::to_string_pretty(&record)?)?;
tracing::debug!("saved UCAN to {}", ucan_path.display());
}
@@ -111,7 +108,7 @@ fn ucan_path(dir: Option<&std::path::Path>) -> Result<PathBuf> {
.context("could not determine home directory")?
.join(".gitlawb")
};
crate::secret_file::create_dir(&base)?;
std::fs::create_dir_all(&base)?;
Ok(base.join("ucan.json"))
}
+2 -405
View File
@@ -5,7 +5,7 @@ use clap::{Args, Subcommand};
use serde_json::{json, Value};
use std::path::PathBuf;
use crate::http::{sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -274,126 +274,6 @@ async fn resolve_owner_did(_node: &str, dir: Option<&std::path::Path>) -> Result
Ok(did.split(':').next_back().unwrap_or(&did).to_string())
}
/// Fetch `GET /` from the node and return the `web_url` if the node advertises one.
///
/// The View: link is a nice-to-have, so every failure mode degrades to `None`
/// rather than failing the enclosing command — but failures are surfaced as
/// stderr warnings (request error, non-success HTTP status with the status,
/// malformed advertised value), since they all mean the node itself is
/// misbehaving or misconfigured, which the user would otherwise never learn.
/// A successful response that lacks a usable `web_url` (field missing or
/// blank) omits the link silently: self-hosted nodes without a web front-end
/// are expected to not advertise one (#370).
///
/// The body is treated like every other caller-chosen node reply (INV-6): the
/// read is capped and an accepted `web_url` must be free of control/bidi bytes
/// — it reaches the terminal verbatim through the View: line.
pub(crate) async fn fetch_node_web_url(node: &str) -> Option<String> {
let info_client = NodeClient::new(node, None);
let info_resp = match info_client.get("/").await {
Ok(resp) => resp,
Err(err) => {
eprintln!("warning: node info request failed ({err}); skipping View link");
return None;
}
};
let status = info_resp.status();
if !status.is_success() {
eprintln!("warning: node info request returned {status}; skipping View link");
return None;
}
// An info reply is a DID, a few URLs and counts — 8 KiB is well past what
// the shape needs; anything longer is hostile or broken (peer.rs precedent).
let raw_body = crate::http::read_body_capped(info_resp, 8 * 1024).await;
match web_url_from_info_body(&raw_body) {
Ok(url) => url,
Err(reason) => {
eprintln!("warning: {reason}; skipping View link");
None
}
}
}
/// Classify a capped `GET /` body for the `View:` advertisement. Split out of
/// `fetch_node_web_url` so tests can pin the classification contract directly:
/// a truncated or half-read body is indistinguishable from a node that
/// advertised nothing, so it must warn rather than silently omit the link.
fn web_url_from_info_body(
raw_body: &crate::http::CappedBody,
) -> std::result::Result<Option<String>, String> {
if raw_body.truncated || raw_body.read_failed {
return Err("node info body was oversized or could not be read in full".to_string());
}
let info: Value = match serde_json::from_str(&raw_body.text) {
Ok(json) => json,
Err(_) => return Err("node info body was not valid JSON".to_string()),
};
// Missing field / non-string degrade to None without warning — same contract
// as before; only transport- and advertisement-level failures warn there.
let raw = match info["web_url"].as_str() {
Some(v) => v,
None => return Ok(None),
};
let trimmed = raw.trim().trim_end_matches('/');
if trimmed.is_empty() {
return Ok(None);
}
// A present-but-malformed value means the node is misconfigured; say so
// instead of quietly dropping the link. Mirrors the node-side boot
// validation: must be an absolute http(s) URL with no query or fragment —
// the link is built by string-appending `/{owner}/{repo}` to it. Control
// and bidi-format bytes are rejected outright (not stripped): they have no
// legitimate place in a base URL and would reach the terminal through the
// View: line — including when they arrive percent-encoded (INV-6).
let malformed_reason: Option<String> = if trimmed
.chars()
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c))
{
Some("contains control or bidi characters".to_string())
} else {
match trimmed.parse::<url::Url>() {
Err(_) => Some("not an absolute URL".to_string()),
Ok(parsed) if !matches!(parsed.scheme(), "http" | "https") => {
Some("not http(s)".to_string())
}
Ok(parsed) if parsed.query().is_some() || parsed.fragment().is_some() => {
Some("contains a query or fragment".to_string())
}
// userinfo is rejected because the `View:` line prints the
// value verbatim; a remote advertisement carrying
// `user:pass@host` would land that credential in terminal
// scrollback and CI logs.
Ok(parsed) if !parsed.username().is_empty() || parsed.password().is_some() => {
Some("contains a username or password".to_string())
}
Ok(parsed)
if parsed
.host_str()
.unwrap_or("")
.chars()
.chain(
percent_encoding::percent_decode_str(parsed.path())
.decode_utf8_lossy()
.chars(),
)
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c)) =>
{
Some("contains control or bidi characters, even percent-encoded".to_string())
}
Ok(_) => None,
}
};
match malformed_reason {
// The reason string is ours; the advertised value is not — defang it
// exactly as it would have been defanged had it been accepted.
Some(reason) => Err(format!(
"node advertised a malformed web_url ({:?}, {reason})",
sanitize_node_msg(trimmed)
)),
None => Ok(Some(trimmed.to_string())),
}
}
async fn cmd_create(
name: String,
description: Option<String>,
@@ -431,11 +311,7 @@ async fn cmd_create(
println!("✓ Created repository: {name}");
println!(" Clone: git clone {gitlawb_url}");
println!(" HTTP: {clone_url}");
// Only print View: when the node advertises a web_url — self-hosted nodes
// without a web front-end would otherwise produce a 404 link (#370).
if let Some(web_url) = fetch_node_web_url(&node).await {
println!(" View: {web_url}/{owner_short}/{name}");
}
println!(" View: https://twigpine.com/{owner_short}/{name}");
if let Some(desc) = payload["description"].as_str().filter(|s| !s.is_empty()) {
println!(" Desc: {desc}");
}
@@ -972,285 +848,6 @@ mod tests {
.unwrap();
}
#[tokio::test]
async fn test_fetch_node_web_url_with_web_url() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"https://example.com","did":"did:key:z6Mk"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url.as_deref(), Some("https://example.com"));
}
#[tokio::test]
async fn test_fetch_node_web_url_trims_trailing_slash() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"https://example.com/"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url.as_deref(), Some("https://example.com"));
}
#[tokio::test]
async fn test_fetch_node_web_url_without_web_url() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:z6Mk"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
#[tokio::test]
async fn test_fetch_node_web_url_empty_string() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":""}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
#[tokio::test]
async fn test_fetch_node_web_url_whitespace_only() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":" "}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
#[tokio::test]
async fn test_fetch_node_web_url_server_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(500)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
}
/// A node advertising a present-but-malformed web_url (not an absolute
/// http(s) URL) must not yield a View link — and must warn, since a
/// malformed advertisement means the node is misconfigured (#370).
#[tokio::test]
async fn test_fetch_node_web_url_malformed_value_is_rejected() {
for bad in [
"gitlawb.com",
"not a url",
"ftp://files.example.com",
// Query/fragment corrupt the appended /{owner}/{repo} path.
"https://example.com?a=1",
"https://example.com/#faq",
] {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(format!(r#"{{"web_url":"{bad}"}}"#))
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(
web_url, None,
"malformed web_url {bad:?} must not render a View link"
);
}
}
/// A transport-level failure (connection refused) degrades to None but is
/// no longer silent: the request error is surfaced on stderr so an
/// unreachable node isn't indistinguishable from one without a web_url.
#[tokio::test]
async fn test_fetch_node_web_url_connection_refused_warns() {
// Port 1 on localhost is reserved (tcpmux) and refuses connections.
let web_url = fetch_node_web_url("http://127.0.0.1:1").await;
assert_eq!(web_url, None);
}
/// A hostile node can smuggle ANSI/bell/bidi controls inside a web_url that
/// still passes http(s) URL parsing — those bytes would reach the terminal
/// verbatim through the View: line. The advertised value must be rejected
/// outright: no control byte may survive into the returned string.
#[tokio::test]
async fn test_fetch_node_web_url_rejects_control_bytes() {
for bad in [
"https://example.com/\x1b[31mred", // ANSI CSI escape in path
"https://example.com\x07", // bell
"https://ex\u{202e}ample.com", // bidi override (RLO)
"https://example.com/\u{200f}", // RLM format char
"\x1b]0;title\x07https://evil.example", // OSC title-set prefix
] {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
// serde_json escapes the control bytes so the body is VALID
// JSON — a raw literal would make parse fail and the test
// would pass without reaching the control check at all.
.with_body(serde_json::json!({"web_url": bad}).to_string())
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(
web_url, None,
"web_url with control bytes {bad:?} must be rejected"
);
}
}
/// An oversized or half-read `GET /` body must warn, not silently
/// classify as "no web front-end": a cut body fails JSON parse, which is
/// indistinguishable from a node that advertised nothing without the
/// truncated flag. Removing the flag check makes this test pass
/// vacuously through the parse-fail arm.
#[test]
fn web_url_from_info_body_warns_on_truncated_reads() {
for (body, label) in [
(
crate::http::CappedBody {
text: r#"{"web_url":"https://x"#.repeat(2000),
truncated: true,
read_failed: false,
},
"truncated",
),
(
crate::http::CappedBody {
text: r#"{"web_url":"https://x"#.to_string(),
truncated: false,
read_failed: true,
},
"read_failed",
),
] {
let err = web_url_from_info_body(&body)
.expect_err(&format!("{label} body must warn, not silently classify"));
assert!(
err.contains("oversized or could not be read"),
"{label}: got {err}"
);
}
// A clean body missing web_url still classifies as a silent None.
let clean = crate::http::CappedBody {
text: r#"{"did":"did:key:x"}"#.to_string(),
truncated: false,
read_failed: false,
};
assert_eq!(web_url_from_info_body(&clean).unwrap(), None);
}
/// Percent-encoded bidi must not slip past the literal scan: a node
/// advertising `https://example.com/%E2%80%AE` serves wire bytes with no
/// Cf code point, but a browser decodes it into a reordering character
/// on the rendered View: link (INV-6).
#[test]
fn web_url_from_info_body_rejects_percent_encoded_bidi() {
let body = crate::http::CappedBody {
text: r#"{"web_url":"https://example.com/%E2%80%AE"}"#.to_string(),
truncated: false,
read_failed: false,
};
let err = web_url_from_info_body(&body).expect_err("percent-encoded bidi must be rejected");
assert!(err.contains("bidi"), "got {err}");
}
#[tokio::test]
async fn test_fetch_node_web_url_validates_after_trimming() {
// Trailing slash is trimmed before validation; result stays usable.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"http://localhost:8080/ui/"}"#)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url.as_deref(), Some("http://localhost:8080/ui"));
}
/// A remote node that advertises `web_url` with userinfo (`user@`,
/// `user:pass@`) must be dropped: the value is reprinted on the `View:`
/// line, and a credential embedded in the URL would land in terminal
/// scrollback and CI logs. Mirrors the node-side boot validation.
#[tokio::test]
async fn test_fetch_node_web_url_rejects_userinfo() {
for bad in [
"https://[email protected]",
"https://user:[email protected]",
] {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(format!(r#"{{"web_url":"{}"}}"#, bad))
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(
web_url, None,
"web_url with userinfo {bad:?} must be rejected"
);
}
}
/// A non-success status must still return None (View link is cosmetic) but
/// the status surfaces as a user-visible stderr warning instead of being
/// swallowed silently (#370 review).
#[tokio::test]
async fn test_fetch_node_web_url_non_success_warns_with_status() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/")
.with_status(503)
.create_async()
.await;
let web_url = fetch_node_web_url(&server.url()).await;
assert_eq!(web_url, None);
// The warning itself goes to stderr; mockito can't capture it here, so
// this asserts only the None contract. Manual check: run against a
// 503-ing node and observe "node info request returned 503" on stderr.
}
#[tokio::test]
async fn test_cmd_create_server_error() {
let dir = TempDir::new().unwrap();
-180
View File
@@ -1,180 +0,0 @@
//! Writes and directories that hold key material or bearer tokens.
use std::io::Write;
use std::path::Path;
/// Write `contents` to `path` with owner-only permissions.
///
/// On unix the mode is pinned at creation, so the file never exists with a
/// permissive mode between open and chmod. `O_NOFOLLOW` refuses to write
/// through a pre-planted symlink. `mode()` applies only when the file is
/// created, so the mode is pinned on the descriptor before the file is
/// truncated: a pre-existing loose file is tightened before, not after, the
/// new contents land in it.
pub(crate) fn write(path: &Path, contents: &[u8]) -> std::io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::{OpenOptionsExt, PermissionsExt};
let mut f = std::fs::OpenOptions::new()
.write(true)
.create(true)
.mode(0o600)
.custom_flags(libc::O_NOFOLLOW)
.open(path)?;
f.set_permissions(std::fs::Permissions::from_mode(0o600))?;
f.set_len(0)?;
f.write_all(contents)?;
}
#[cfg(not(unix))]
std::fs::write(path, contents)?;
Ok(())
}
/// Create `path` (and missing parents) as an owner-only directory.
///
/// An existing directory is also re-pinned: a `~/.gitlawb` created before
/// this helper stays group- and world-listable otherwise. A symlinked
/// `path` is left alone rather than chmodded through, which would strip
/// group and world access from whatever the link points at.
pub(crate) fn create_dir(path: &Path) -> std::io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::{DirBuilderExt, OpenOptionsExt};
std::fs::DirBuilder::new()
.recursive(true)
.mode(0o700)
.create(path)?;
// Re-pin through the opened directory, not the path: O_NOFOLLOW
// refuses a swapped-in symlink and fchmod acts on the checked inode,
// so a final-component swap between check and chmod cannot retarget
// the mode change at whatever the link points at.
match std::fs::OpenOptions::new()
.read(true)
.custom_flags(libc::O_NOFOLLOW)
.open(path)
{
Ok(dir) => {
use std::os::unix::io::AsRawFd;
let fd = dir.as_raw_fd();
// Re-pin only a leaf this process owns. A --dir the caller
// can write but cannot chmod (foreign-owned 0775 leaf, ACL
// grant) is adopted unchanged: create_dir_all allowed the
// write before, and aborting turns a usable directory into
// a command failure. An owned directory that still refuses
// chmod (EPERM) is adopted the same way.
let owned = unsafe {
let mut st: libc::stat = std::mem::zeroed();
libc::fstat(fd, &mut st) == 0 && st.st_uid == libc::geteuid()
};
if owned && unsafe { libc::fchmod(fd, 0o700) } != 0 {
let e = std::io::Error::last_os_error();
if e.raw_os_error() != Some(libc::EPERM) {
return Err(e);
}
}
}
// ELOOP: the path is a symlink and is left alone per contract.
Err(e) if e.raw_os_error() == Some(libc::ELOOP) => {}
Err(e) => return Err(e),
}
}
#[cfg(not(unix))]
std::fs::create_dir_all(path)?;
Ok(())
}
#[cfg(test)]
mod tests {
use tempfile::TempDir;
#[test]
#[cfg(unix)]
fn create_dir_adopts_writable_leaf_it_cannot_chmod() {
use std::os::unix::fs::PermissionsExt;
// /tmp is root-owned 1777: writable by this euid, not chmod-able.
// The pre-helper create_dir_all allowed the write; the re-pin must
// adopt the leaf unchanged rather than abort on EPERM.
let before = std::fs::metadata("/tmp").unwrap().permissions().mode();
super::create_dir(std::path::Path::new("/tmp")).unwrap();
let after = std::fs::metadata("/tmp").unwrap().permissions().mode();
assert_eq!(before, after);
}
#[test]
#[cfg(unix)]
fn write_creates_file_with_0600() {
use std::os::unix::fs::PermissionsExt;
let dir = TempDir::new().unwrap();
let path = dir.path().join("key.pem");
super::write(&path, b"secret").unwrap();
assert_eq!(std::fs::read(&path).unwrap(), b"secret");
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o600
);
}
#[test]
#[cfg(unix)]
fn write_tightens_preexisting_loose_file() {
use std::os::unix::fs::PermissionsExt;
let dir = TempDir::new().unwrap();
let path = dir.path().join("key.pem");
std::fs::write(&path, b"old").unwrap();
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o644)).unwrap();
super::write(&path, b"new").unwrap();
assert_eq!(std::fs::read(&path).unwrap(), b"new");
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o600
);
}
#[test]
#[cfg(unix)]
fn write_refuses_symlink() {
let dir = TempDir::new().unwrap();
let target = dir.path().join("target.pem");
std::fs::write(&target, b"planted").unwrap();
let link = dir.path().join("link.pem");
std::os::unix::fs::symlink(&target, &link).unwrap();
assert!(super::write(&link, b"secret").is_err());
assert_eq!(std::fs::read(&target).unwrap(), b"planted");
}
#[test]
#[cfg(unix)]
fn create_dir_modes_0700_and_tightens_existing() {
use std::os::unix::fs::PermissionsExt;
let dir = TempDir::new().unwrap();
let path = dir.path().join("a").join("b");
super::create_dir(&path).unwrap();
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o700
);
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap();
super::create_dir(&path).unwrap();
assert_eq!(
std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
0o700
);
}
#[test]
#[cfg(unix)]
fn create_dir_does_not_chmod_through_symlink() {
use std::os::unix::fs::PermissionsExt;
let dir = TempDir::new().unwrap();
let target = dir.path().join("shared");
std::fs::create_dir(&target).unwrap();
std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755)).unwrap();
let link = dir.path().join("link");
std::os::unix::fs::symlink(&target, &link).unwrap();
super::create_dir(&link).unwrap();
assert_eq!(
std::fs::metadata(&target).unwrap().permissions().mode() & 0o777,
0o755
);
}
}
+1 -1
View File
@@ -90,7 +90,7 @@ async fn cmd_delegate(
let encoded = ucan.encode()?;
if let Some(path) = out {
crate::secret_file::write(&path, encoded.as_bytes())?;
std::fs::write(&path, &encoded)?;
println!("UCAN saved to {}", path.display());
return Ok(());
}
-138
View File
@@ -1,138 +0,0 @@
//! End-to-end test for the `View:` line printed by `gl repo create` (#370).
//!
//! `cmd_create` is the load-bearing #370 path: it is the one place the CLI
//! prints a `View:` line, and the only thing keeping it from being a
//! hardcoded `https://gitlawb.com/...` 404 is `fetch_node_web_url`. The
//! helper-level tests in `repo.rs` cover the helper, not the command path —
//! if `cmd_create` went back to a hardcoded constant, every helper test
//! would still pass.
//!
//! This integration test drives the real `gl` binary against a mockito
//! server that answers both `POST /api/v1/repos` and `GET /`. Asserting on
//! the subprocess's stdout is straightforward: bytes are bytes, with no
//! libtest / gag / in-process capture race.
//!
//! CARGO_BIN_EXE_gl is set by Cargo for integration tests of a `[[bin]]`
//! in the same package, so the test always picks up the in-tree build.
use std::process::Stdio;
use tempfile::TempDir;
use tokio::io::AsyncWriteExt;
use tokio::process::Command;
fn gl_bin() -> String {
std::env::var("CARGO_BIN_EXE_gl").expect("CARGO_BIN_EXE_gl is unset for this test")
}
async fn write_identity(dir: &TempDir) {
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
let path = dir.path().join("identity.pem");
let mut f = tokio::fs::File::create(&path).await.unwrap();
f.write_all(pem.as_bytes()).await.unwrap();
}
/// The trailing key segment of the freshly-generated identity. Mirrors the
/// in-source `resolve_owner_did` test helper but lives in this integration
/// file because that helper is `pub(crate)`.
async fn owner_short(dir: &TempDir) -> String {
let pem = tokio::fs::read_to_string(dir.path().join("identity.pem"))
.await
.unwrap();
let kp = gitlawb_core::identity::Keypair::from_pem(&pem).unwrap();
let did = kp.did().to_string();
did.split(':').next_back().unwrap_or(&did).to_string()
}
async fn run_gl_create(node_url: &str, dir: &TempDir) -> String {
let output = Command::new(gl_bin())
.arg("repo")
.arg("create")
.arg("myrepo")
.arg("--private")
.arg("--branch")
.arg("main")
.arg("--node")
.arg(node_url)
.arg("--dir")
.arg(dir.path())
.env("NO_COLOR", "1")
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.output()
.await
.expect("failed to spawn `gl repo create`");
assert!(
output.status.success(),
"`gl repo create` failed: stderr:\n{}\nstdout:\n{}",
String::from_utf8_lossy(&output.stderr),
String::from_utf8_lossy(&output.stdout),
);
String::from_utf8(output.stdout).expect("`gl` wrote non-UTF-8 to stdout")
}
#[tokio::test]
async fn cmd_create_view_url_tracks_node_advertisement() {
let dir = TempDir::new().unwrap();
write_identity(&dir).await;
let owner = owner_short(&dir).await;
// Case 1: node advertises a web_url. The `View:` line must use the
// advertised origin and the resolved owner/short name.
{
let mut server = mockito::Server::new_async().await;
let _m_create = server
.mock("POST", "/api/v1/repos")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"name":"myrepo","clone_url":"gitlawb://did:key:z6Mk/myrepo"}"#)
.create_async()
.await;
let _m_info = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"web_url":"https://git.example","did":"did:key:z6Mk"}"#)
.create_async()
.await;
let server_url = server.url();
let out = run_gl_create(&server_url, &dir).await;
let view_line = out
.lines()
.find(|l| l.trim_start().starts_with("View:"))
.unwrap_or_else(|| panic!("View: line missing from stdout:\n{out}"));
assert!(
view_line.contains(&format!("https://git.example/{owner}/myrepo")),
"View: line must use the advertised origin, got: {view_line:?}"
);
}
// Case 2: node does NOT advertise a web_url. The `View:` line must be
// absent, so a self-hosted node without a web front-end never produces
// a 404 link (#370).
{
let mut server = mockito::Server::new_async().await;
let _m_create = server
.mock("POST", "/api/v1/repos")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"name":"myrepo","clone_url":"gitlawb://did:key:z6Mk/myrepo"}"#)
.create_async()
.await;
let _m_info = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:z6Mk"}"#)
.create_async()
.await;
let server_url = server.url();
let out = run_gl_create(&server_url, &dir).await;
assert!(
!out.lines().any(|l| l.trim_start().starts_with("View:")),
"View: line must be absent when node omits web_url; got:\n{out}"
);
}
}
-48
View File
@@ -1,48 +0,0 @@
//! #357: pin the `gl doctor` exit-code wiring, not just the predicate.
//!
//! The unit tests for `has_failures` never touch `std::process::exit(1)` or
//! the `GITLAWB_NODE` Warn tier. Deleting the exit gate outright kept
//! `cargo test -p gl doctor::` green, so this binary probe is the only thing
//! that breaks when the wiring is gutted.
use std::process::Command;
#[test]
fn doctor_exits_nonzero_when_fail_rows_present() {
let dir = tempfile::tempdir().expect("temp dir");
// Use an unroutable node so the "node unreachable" check is Fail-class.
// The temp dir has no identity.pem / ucan.json, so identity + registration
// are also Fail — at least one Fail must flip the process to exit 1.
let bin = env!("CARGO_BIN_EXE_gl");
let output = Command::new(bin)
.args([
"doctor",
"--dir",
dir.path().to_str().unwrap(),
"--node",
"http://127.0.0.1:1",
])
// Inherit PATH but neutralize GITLAWB_NODE so the test is deterministic
// across machines that have it set (otherwise it would be Pass).
.env_remove("GITLAWB_NODE")
// iCaptcha probes a real URL by default; keep it unreachable too so
// it stays Warn and does not affect the Fail gating.
.env("GITLAWB_ICAPTCHA_URL", "http://127.0.0.1:1")
.output()
.expect("run gl doctor");
// The command should have exited 1 because at least one Fail row exists.
// If the `std::process::exit(1)` gate is removed, this becomes 0 and the
// test fails — that is the regression it pins.
assert!(
!output.status.success(),
"gl doctor must exit non-zero when Fail rows are present; stdout: {}\nstderr: {}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
assert_eq!(
output.status.code(),
Some(1),
"exit code must be 1, not some other non-zero"
);
}
-2
View File
@@ -8,7 +8,6 @@ Step-by-step guide to staking $GITLAWB, registering your node on-chain, and earn
- A wallet with at least **10,000 $GITLAWB** (minimum stake) plus a small amount of ETH on Base for gas
- Docker or Rust 1.91+ (for running the node process)
- PostgreSQL 16+ — the node uses `pg_input_is_valid`, which exists only on PostgreSQL 16 and later. The bundled `docker-compose.yml` pins `postgres:16-alpine`; if you supply your own `DATABASE_URL`, the server must be 16+.
- A public HTTP URL (your-host.com) — can be a VPS, Fly.io app, or anything reachable. A Fly.io config is provided at `infra/fly/fly.toml` (deploy from the repo root with `fly deploy -c infra/fly/fly.toml`)
---
@@ -91,7 +90,6 @@ Required env for on-chain PoS mode:
Optional:
- `GITLAWB_OPERATOR_STRICT_MODE=true` — refuse to start if not registered or not currently active
- `GITLAWB_HEARTBEAT_INTERVAL_HOURS=20` — how often to post heartbeats (must be < 24)
- `GITLAWB_RECONCILIATION_SWEEP=true` — enable the hourly durability sweep that re-pins/backstops missing objects (default `true`; disabled when no IPFS/Pinata backend is configured). Public pin repair runs against any configured backend. Encrypted recovery repair requires local IPFS (`GITLAWB_IPFS_API`); Pinata-only nodes reconcile public pins only. Set `=false` to disable.
## 5. Verify
-76
View File
@@ -1,76 +0,0 @@
# GraphQL repository pagination
`repos` returns the complete visible repository list when it contains at most
200 entries. Larger lists return a GraphQL error directing the caller to
`reposPage`; they are never silently truncated.
Use `reposPage` for larger lists:
```graphql
query Repositories($after: String) {
reposPage(limit: 50, after: $after) {
nodes {
name
ownerDid
}
hasNextPage
endCursor
}
}
```
Start with `after: null`. When `hasNextPage` is true, pass the returned
`endCursor` as `after` on the next request. Stop when `hasNextPage` is false.
An empty page has a null `endCursor`. The default limit is 50 and the maximum
page size is 200; limits below 1 or above 200 are rejected (see Query budgets
below). Malformed cursors, including decoded owner or name strings containing
NUL, return `invalid repository cursor` before database access.
Pages are ordered by normalized owner DID and repository name. Visibility and
mirror deduplication are applied in the database before limiting the page;
hidden and quarantined repositories do not occupy page slots or create a
continuation signal. Cursors contain a position from the last returned visible
repository, not a permission grant. Each request checks the current caller's
visibility independently. Keep the same caller while traversing a list.
Malformed reader lists in a repository's root visibility rule deny access to
non-owners for that repository while other visible repositories remain listable.
Pagination is not a snapshot: concurrent renames, ownership changes, or visibility
changes can alter subsequent pages. Treat cursors as opaque and restart from the
first page when a fresh complete traversal is needed. Small legacy `repos`
queries retain their activity ordering.
## Query budgets
Every GraphQL document is validated against two budgets before any resolver
runs. Both apply per document (per request), not per client: `/graphql` has
no per-caller rate limiting, so a rejected document can always be retried
smaller. The constants live in `crates/gitlawb-node/src/graphql/mod.rs`.
- **Complexity budget: 400 per document** (`GRAPHQL_MAX_COMPLEXITY`). Each
root field carries a base cost plus the clamped page size plus the selected
child fields:
- `repos`: 50 + `MAX_VISIBLE_REPO_PAGE_SIZE` + child fields. `repos`
always scans the full visible corpus, so it is priced as the full page
it is.
- `reposPage`: 50 + `MAX_VISIBLE_REPO_PAGE_SIZE` + child fields. Every
page request sorts the full deduplicated corpus, so the price does not
depend on `limit`: seven `reposPage(limit: 1)` aliases cost the same as
seven max-size pages.
- `refUpdates`: 100 + clamped `limit` (`MAX_VISIBLE_REF_UPDATES`) + child
fields. The 100 base covers the collector's fixed scan (the deduplicated
repo set, the quarantined set, the visibility rules, and the bounded
up-to-2048-row walk past withheld rows); the limit term prices only the
rows returned.
- `tasks`: 50 + clamped `limit` (`MAX_VISIBLE_TASK_PAGE_SIZE`) + child
fields.
- `task`, mutations, subscriptions: 50 + child fields.
A document whose total exceeds 400 is rejected with
`Query is too complex.` before any resolver runs, so an over-budget
document needs fewer aliases or fewer child fields. A smaller `limit` does
not help `reposPage` or `repos`: both are priced at the full
`MAX_VISIBLE_REPO_PAGE_SIZE` scan regardless of `limit`.
- **Depth cap: 14 per document** (`GRAPHQL_MAX_DEPTH`). Documents nested
deeper than 14 selections are rejected with `Query is nested too deep.`