mirror of
https://github.com/Twigpine/node.git
synced 2026-11-11 23:40:21 -10:00
Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2219c5d23f | ||
|
|
c564f8422b | ||
|
|
66d01f8a6f | ||
|
|
ea0b0571cc | ||
|
|
a42aab77e7 | ||
|
|
bfb491e0d8 | ||
|
|
d5d9bbb5b3 | ||
|
|
966b2b8e0a | ||
|
|
2f99ad8555 | ||
|
|
3840b7392a | ||
|
|
9f4f509619 |
No files matched your search
@@ -1,4 +0,0 @@
|
||||
name: "Gitlawb CodeQL config"
|
||||
paths-ignore:
|
||||
- "**/tests/**"
|
||||
- "**/test_support/**"
|
||||
@@ -195,16 +195,6 @@ jobs:
|
||||
# shellcheck disable=SC2086 # PACKAGES is a word list by design
|
||||
cargo test --locked $PACKAGES
|
||||
|
||||
# The real-node deny harness is behind the `test-harness` feature (so its
|
||||
# spawn surface never compiles into the production binary), which means
|
||||
# `--workspace` above skips it. Run it explicitly so the trust-boundary
|
||||
# regression cases (INV-1/INV-2/INV-8) execute on every PR. Uses the same
|
||||
# Postgres service; `#[sqlx::test]` provisions an isolated DB per test.
|
||||
- name: cargo test (real-node deny harness)
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
|
||||
run: cargo test --locked -p gitlawb-node --features test-harness --lib --test deny_harness
|
||||
|
||||
build-release:
|
||||
name: build --release
|
||||
needs: changes
|
||||
|
||||
@@ -26,15 +26,11 @@ COPY crates/icaptcha-client/Cargo.toml crates/icaptcha-client/
|
||||
# is what let it sit dead from the moment gl gained a path dependency on
|
||||
# icaptcha-client. A failure here should stop the build, not silently turn every
|
||||
# image build into a cold one.
|
||||
# gitlawb-node declares BOTH a [[bin]] and a [lib], so it needs both stubs:
|
||||
# cargo resolves every declared target before it fetches anything, and a
|
||||
# missing src/lib.rs aborts the layer with a target resolution error.
|
||||
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \
|
||||
crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \
|
||||
echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \
|
||||
echo 'fn main() {}' > crates/gl/src/main.rs && \
|
||||
echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \
|
||||
echo '' > crates/gitlawb-node/src/lib.rs && \
|
||||
echo '' > crates/gitlawb-core/src/lib.rs && \
|
||||
echo '' > crates/gitlawb-attest/src/lib.rs && \
|
||||
echo '' > crates/icaptcha-client/src/lib.rs && \
|
||||
|
||||
-121
@@ -1,121 +0,0 @@
|
||||
# Gitlawb Wire Protocol Specification (v1, alpha)
|
||||
|
||||
This document describes the wire behavior a second implementation can interop
|
||||
with **today**. Anything the node does not ship yet is explicitly marked
|
||||
*planned*; nothing below is aspirational unless it says so.
|
||||
|
||||
## 1. Identity & DIDs
|
||||
|
||||
- **Key type:** Ed25519.
|
||||
- **Method (alpha):** `did:key` only. Resolution is deterministic public-key
|
||||
extraction from the multicodec prefix (`z6Mk…`, multicodec `0xed01`).
|
||||
- Signed requests presenting any other DID method are rejected; the node's
|
||||
auth layer answers with a hint that only `did:key` is supported in alpha.
|
||||
- *Planned:* `did:web` and `did:gitlawb` resolution. Clients MUST NOT assume
|
||||
either is accepted anywhere yet.
|
||||
|
||||
## 2. Authentication (RFC 9421)
|
||||
|
||||
- **Transport:** production deployments should run the node behind TLS
|
||||
(typically a reverse proxy). The node itself has no scheme check: it verifies
|
||||
RFC 9421 headers on whatever transport the request arrives over, so nothing
|
||||
at the signature layer refuses a cleartext request.
|
||||
- **HTTP signatures:** `Signature-Input` covers `@method`, `@path`, and
|
||||
`content-digest`:
|
||||
`sig1=("@method" "@path" "content-digest");keyid="did:key:…";alg="ed25519";created=<unix>`
|
||||
All three components are mandatory: a `Signature-Input` missing one is
|
||||
rejected as `incomplete_signature`.
|
||||
- **`@path` value:** the node binds `@path` to the path *and* query string of
|
||||
the request, not to the path-only value RFC 9421 defines for `@path`. A
|
||||
second implementation must sign the request exactly as it goes on the
|
||||
wire, e.g. `GET /zOwner/myrepo.git/info/refs?service=git-upload-pack`
|
||||
including the `?service=…` part (see §5 for the owner and `.git`
|
||||
normalization).
|
||||
- **Signer identity:** `keyid` is the actor DID (`did:key` only, see §1).
|
||||
- **Integrity:** `Content-Digest` is SHA-256 over the request body and is
|
||||
verified against it. Every signed request MUST send the header: a signed
|
||||
request without it is refused `400 missing_content_digest` before routing.
|
||||
On a body-less request the header still carries the SHA-256 digest of the
|
||||
empty body (`sha-256=:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=:`),
|
||||
it is never omitted.
|
||||
|
||||
## 3. Proof of Intelligence (iCaptcha)
|
||||
|
||||
- **Mode:** controlled by `ICAPTCHA_MODE` = `off` | `shadow` | `enforce`,
|
||||
**default `off`**. A node with the gate off never demands proof.
|
||||
- **Scope:** the gate applies to a small set of write endpoints (agent
|
||||
registration and repo create/fork), not to all writes.
|
||||
- **Wire shape:** the client-supplied header is `x-icaptcha-proof` — that is
|
||||
the only iCaptcha header a node reads from a request. When the gate is
|
||||
`enforce` and a verifier public key is loaded, a missing or invalid proof
|
||||
answers `403 icaptcha_proof_required` and sets `x-icaptcha-url` and
|
||||
`x-icaptcha-level` **on the response** (also mirrored as JSON fields) so
|
||||
the client can go solve a challenge and retry. Fail-safe: `enforce` with
|
||||
no public key loaded (unset and unfetchable) stays inert and admits the
|
||||
request rather than rejecting every request.
|
||||
|
||||
## 4. Ref-Update Certificate
|
||||
|
||||
*Planned:* the `gitlawb/ref-update/v1` schema below is defined in code, but no
|
||||
shipped endpoint serves or verifies these certificates today; nothing in
|
||||
sections 2-3 produces one on the wire. It is documented here so a second
|
||||
implementation shares the type vocabulary ahead of the wire surface.
|
||||
|
||||
The hash fields reserve 64 hex characters for the SHA-256 object format; the
|
||||
repositories the node serves are sha1 with 40-hex OIDs (see §5), so the field
|
||||
width is a forward reservation, not what a current push produces.
|
||||
|
||||
Two distinct certificate-shaped payloads exist; this section documents the
|
||||
**core signing schema**, which is what `gitlawb/ref-update/v1` names:
|
||||
|
||||
- **Type string:** `gitlawb/ref-update/v1`.
|
||||
- **Body (canonical JSON, the signed bytes):** `type`, `repo` (the
|
||||
repository DID), `ref_name`, `from` (previous hash, 64 hex chars,
|
||||
all-zeros for a new ref), `to` (target hash, 64 hex chars), `seq`
|
||||
(monotonically increasing, replay prevention), `timestamp` (RFC 3339),
|
||||
`nonce`.
|
||||
- **Signatures:** a list of `{signer: did:key, sig: base64url-unpadded
|
||||
Ed25519}` entries appended outside the signed body; thresholds are counted
|
||||
over distinct signers that verify.
|
||||
|
||||
The node's REST API separately serves **push receipts** — a different,
|
||||
node-issued JSON payload attesting that a push was processed. Receipts are
|
||||
not `gitlawb/ref-update/v1` documents and are not interchangeable with them;
|
||||
they are documented with the node's HTTP API, not here.
|
||||
|
||||
## 5. Storage & Git Transport
|
||||
|
||||
- **Object format:** bare repositories are created with
|
||||
`--object-format=sha1`; production OIDs are 40 hex chars. *Planned:*
|
||||
SHA-256 object format.
|
||||
- **Path normalization:** `{repo}` may carry one trailing `.git` suffix,
|
||||
stripped before lookup (so `myrepo.git` resolves to `myrepo`; a repo
|
||||
literally named `foo.git` addresses as `foo.git.git`). `{owner}` may be the
|
||||
full `did:key:…` string or the bare multibase segment the `did:key:` prefix
|
||||
implies; both resolve to the same owner. None of this changes `@path`:
|
||||
sign the request path and query exactly as sent.
|
||||
- **Smart-HTTP endpoints:**
|
||||
- `GET /{owner}/{repo}/info/refs?service=git-upload-pack` — fetch
|
||||
advertisement.
|
||||
- `GET /{owner}/{repo}/info/refs?service=git-receive-pack` — push
|
||||
advertisement.
|
||||
The `?service=` parameter is mandatory and takes exactly those two
|
||||
values; a missing parameter is a `400`, not an omitted-service default.
|
||||
- `POST /{owner}/{repo}/git-upload-pack` — clone/fetch. Request and
|
||||
response bodies are pkt-line framed; the result content type is
|
||||
`application/x-git-upload-pack-result`.
|
||||
- `POST /{owner}/{repo}/git-receive-pack` — push. **Authentication is
|
||||
mandatory**: an unsigned push is refused with `401`. Same pkt-line
|
||||
framing; the result content type is
|
||||
`application/x-git-receive-pack-result`.
|
||||
Signed requests on this path follow the RFC 9421 header rules in §2.
|
||||
§2's transport note applies here too: the verifier has no scheme check,
|
||||
so nothing at the signature layer requires this push to arrive over TLS.
|
||||
- **Fetch vs push auth split:** fetch-side requests against a withheld or
|
||||
private repo answer `404` (existence is not confirmed to unauthorized
|
||||
callers), while the push path answers `401` for missing/invalid
|
||||
signatures. A second implementation must not treat the two symmetrically.
|
||||
- **Content addressing:** the node records IPFS CID metadata per branch and
|
||||
serves it over REST (the repo `refs` endpoint maps branch heads to CIDs).
|
||||
*Planned:* IPNS-backed branch pointers; IPNS is not part of the shipped
|
||||
wire contract.
|
||||
@@ -6,28 +6,10 @@ edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
license.workspace = true
|
||||
|
||||
[lib]
|
||||
name = "gitlawb_node"
|
||||
path = "src/lib.rs"
|
||||
|
||||
[[bin]]
|
||||
name = "gitlawb-node"
|
||||
path = "src/main.rs"
|
||||
|
||||
# Exposes the `test_harness` spawn surface (src/test_harness.rs) to the
|
||||
# real-node deny-harness integration crate. Off by default so the production
|
||||
# binary never compiles test-only boot code. Enable with
|
||||
# `cargo test -p gitlawb-node --features test-harness`.
|
||||
[features]
|
||||
test-harness = ["dep:tempfile"]
|
||||
|
||||
# Only builds when `test-harness` is enabled; otherwise skipped (not an error),
|
||||
# so `cargo test --workspace` without the feature stays green.
|
||||
[[test]]
|
||||
name = "deny_harness"
|
||||
path = "tests/deny_harness.rs"
|
||||
required-features = ["test-harness"]
|
||||
|
||||
[dependencies]
|
||||
gitlawb-core = { path = "../gitlawb-core" }
|
||||
ed25519-dalek = { workspace = true }
|
||||
@@ -96,19 +78,10 @@ alloy = { version = "1", default-features = false, features = [
|
||||
] }
|
||||
rand = { workspace = true }
|
||||
libp2p-dns = { version = "0.45.0", features = ["tokio"] }
|
||||
# Optional: only needed by the test-harness feature's lib modules
|
||||
# (test_git_shim, test_harness); dev-deps are invisible to lib code.
|
||||
tempfile = { version = "3", optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
mockito = "1"
|
||||
tempfile = "3"
|
||||
# test-util enables tokio::test(start_paused): the delayed claim re-sweep
|
||||
# test drives the real CLAIM_SWEEP_MIN_AGE-plus-margin sleep on paused time.
|
||||
# `test-util` for `#[tokio::test(start_paused = true)]`: lets the deadline tests assert
|
||||
# on which timer fired instead of on wall clock.
|
||||
tokio = { workspace = true, features = ["test-util"] }
|
||||
# Used by the deny-harness integration crate (tests/deny_harness.rs): the
|
||||
# #[sqlx::test] macro for an ephemeral per-test pool, and reqwest as the real
|
||||
# HTTP client that drives deny paths over the socket.
|
||||
sqlx = { version = "0.8", features = ["postgres", "runtime-tokio-rustls", "macros", "migrate"] }
|
||||
reqwest = { workspace = true }
|
||||
hex = "0.4"
|
||||
@@ -439,6 +439,7 @@ pub async fn trigger_sync(State(state): State<AppState>) -> Result<Json<serde_js
|
||||
"refs/heads/main",
|
||||
"0000000000000000000000000000000000000000",
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
enqueued += 1;
|
||||
@@ -474,6 +475,13 @@ pub struct NotifyRequest {
|
||||
pub old_sha: Option<String>,
|
||||
#[serde(default)]
|
||||
pub timestamp: Option<String>,
|
||||
/// The instant the origin push landed, for sync-lag measurement only —
|
||||
/// mirrors RefUpdateEvent::lag_origin on the gossip transport. `timestamp`
|
||||
/// stays publish semantics to match what gossip ingest stores on the feed
|
||||
/// row; when an older sender omits this field the enqueue falls back to
|
||||
/// `timestamp`, which was the pre-split approximation of land time.
|
||||
#[serde(default)]
|
||||
pub lag_origin: Option<String>,
|
||||
#[serde(default)]
|
||||
pub cert_id: Option<String>,
|
||||
/// Full owner DID — added in #144 for DID-aware feed gating.
|
||||
@@ -520,7 +528,14 @@ pub async fn notify_sync(
|
||||
|
||||
state
|
||||
.db
|
||||
.enqueue_sync(&req.repo, &req.node_did, &req.ref_name, &req.new_sha, None)
|
||||
.enqueue_sync(
|
||||
&req.repo,
|
||||
&req.node_did,
|
||||
&req.ref_name,
|
||||
&req.new_sha,
|
||||
None,
|
||||
req.lag_origin.as_deref().or(req.timestamp.as_deref()),
|
||||
)
|
||||
.await?;
|
||||
|
||||
// Mirror the gossipsub-receive handler: insert the same record we'd
|
||||
@@ -1282,6 +1297,52 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[sqlx::test]
|
||||
async fn sync_notify_preserves_origin_timestamp_for_sync_lag(pool: PgPool) {
|
||||
let state = test_state(pool).await;
|
||||
let db = state.db.clone();
|
||||
let peer_did = seed_peer(&state).await;
|
||||
// Distinct publish and land instants: the feed row must carry publish
|
||||
// semantics while only the queue row sees lag_origin.
|
||||
let timestamp = "2026-08-16T00:00:00Z";
|
||||
let lag_origin = "2026-08-15T23:59:00Z";
|
||||
let body = serde_json::json!({
|
||||
"repo": "z6Mkfoo/timestamped",
|
||||
"ref_name": "refs/heads/main",
|
||||
"new_sha": "0000000000000000000000000000000000000000",
|
||||
"node_did": peer_did,
|
||||
"timestamp": timestamp,
|
||||
"lag_origin": lag_origin,
|
||||
})
|
||||
.to_string();
|
||||
let router = crate::server::build_router(state);
|
||||
|
||||
let resp = router
|
||||
.oneshot(unsigned_post(
|
||||
"/api/v1/sync/notify",
|
||||
&body,
|
||||
"198.51.100.23:5000",
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(resp.status(), StatusCode::OK);
|
||||
|
||||
let queued = db.dequeue_pending_syncs(100).await.unwrap();
|
||||
assert_eq!(queued.len(), 1);
|
||||
assert_eq!(queued[0].origin_timestamp.as_deref(), Some(lag_origin));
|
||||
let feed = db
|
||||
.list_ref_updates_keyset(Some("z6Mkfoo/timestamped"), 10, None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
feed.iter()
|
||||
.find(|u| u.repo == "z6Mkfoo/timestamped")
|
||||
.map(|u| u.timestamp.as_str()),
|
||||
Some(timestamp),
|
||||
"the feed row must keep publish semantics, matching gossip ingest"
|
||||
);
|
||||
}
|
||||
|
||||
#[sqlx::test]
|
||||
async fn sync_notify_unknown_peer_wins_over_malformed_slug(pool: PgPool) {
|
||||
// Ordering: the slug check sits after the peer gate, so an unknown peer
|
||||
|
||||
@@ -1777,6 +1777,7 @@ pub async fn git_upload_pack(
|
||||
headers: axum::http::HeaderMap,
|
||||
body: Bytes,
|
||||
) -> Result<Response> {
|
||||
let fetch_started = std::time::Instant::now();
|
||||
// #62 cheap load shed. Permit-less snapshot, not admission; see git_info_refs for
|
||||
// what it does and does not bound. The authoritative hold is `git_permit` below,
|
||||
// after the per-source cap.
|
||||
@@ -1979,6 +1980,7 @@ pub async fn git_upload_pack(
|
||||
// that mislabel predates this change and is tracked as a follow-up.
|
||||
if should_count_fetch(finalizes_fetch, served_filtered_pack, served_pack) {
|
||||
crate::metrics::record_fetch(&format!("{owner}/{name}"));
|
||||
crate::metrics::observe_fetch_duration(fetch_started.elapsed());
|
||||
crate::metrics::observe_pack_size(body_len as f64);
|
||||
}
|
||||
Ok(resp)
|
||||
@@ -2190,6 +2192,7 @@ async fn notify_peer_of_ref(
|
||||
node_did: &str,
|
||||
pusher_did: &str,
|
||||
owner_did: &str,
|
||||
origin_timestamp: &str,
|
||||
) {
|
||||
let body = serde_json::json!({
|
||||
"repo": repo_slug,
|
||||
@@ -2198,7 +2201,13 @@ async fn notify_peer_of_ref(
|
||||
"node_did": node_did,
|
||||
"pusher_did": pusher_did,
|
||||
"old_sha": old_sha,
|
||||
"timestamp": chrono::Utc::now().to_rfc3339(),
|
||||
// Publish semantics, matching what gossip ingest stores on the feed
|
||||
// row: the send instant, freshness-window meaningful. Land time travels
|
||||
// separately in lag_origin (also an Option<String> on the receiver);
|
||||
// stamping land here would put a different clock on HTTP rows than on
|
||||
// gossip rows for the same logical update.
|
||||
"timestamp": Utc::now().to_rfc3339(),
|
||||
"lag_origin": origin_timestamp,
|
||||
"owner_did": owner_did,
|
||||
});
|
||||
let body_bytes = match serde_json::to_vec(&body) {
|
||||
@@ -2248,6 +2257,7 @@ async fn notify_peer_of_refs(
|
||||
node_did: &str,
|
||||
pusher_did: &str,
|
||||
owner_did: &str,
|
||||
origin_timestamp: &str,
|
||||
) {
|
||||
for (ref_name, old_sha, new_sha) in ref_updates {
|
||||
notify_peer_of_ref(
|
||||
@@ -2262,6 +2272,7 @@ async fn notify_peer_of_refs(
|
||||
node_did,
|
||||
pusher_did,
|
||||
owner_did,
|
||||
origin_timestamp,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
@@ -2280,6 +2291,7 @@ pub async fn git_receive_pack(
|
||||
headers: axum::http::HeaderMap,
|
||||
body: Bytes,
|
||||
) -> Result<Response> {
|
||||
let push_started = std::time::Instant::now();
|
||||
let name = smart_http_repo_name(&repo)?;
|
||||
// Fast-path shed before the DB lookup when the write pool is ALREADY saturated, so a
|
||||
// push flood against a full pool does not hit Postgres per request. Best-effort
|
||||
@@ -3309,14 +3321,37 @@ pub async fn git_receive_pack(
|
||||
Vec::new()
|
||||
};
|
||||
if disposition.spawn_tail && !push_succeeded && !landed_refs.is_empty() {
|
||||
tokio::spawn(post_receive_replication_tail(
|
||||
state.clone(),
|
||||
record.clone(),
|
||||
landed_refs.clone(),
|
||||
disk_path.clone(),
|
||||
auth.0.to_string(),
|
||||
None,
|
||||
));
|
||||
let state = state.clone();
|
||||
let record = record.clone();
|
||||
let landed = landed_refs.clone();
|
||||
let disk_path = disk_path.clone();
|
||||
let did = auth.0.to_string();
|
||||
let db = state.db.clone();
|
||||
let request_id = request_id.clone();
|
||||
let fallback = now.clone();
|
||||
tokio::spawn(async move {
|
||||
// Resolve inside the owned task: awaiting the lookup in the handler
|
||||
// would hold the reclaimed write lease during a DB wait and leave
|
||||
// the tail un-owned on a disconnect. Same land rule as the deferred
|
||||
// drain — max applied_at across this request's landed transitions,
|
||||
// else the request row's created_at.
|
||||
let push_landed_at = db
|
||||
.request_push_landed_at(&request_id)
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
.unwrap_or(fallback);
|
||||
post_receive_replication_tail(
|
||||
state,
|
||||
record,
|
||||
landed,
|
||||
disk_path,
|
||||
did,
|
||||
Some(push_landed_at),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
|
||||
// Always release the advisory lock — even on error — to prevent stale locks
|
||||
@@ -3337,14 +3372,35 @@ pub async fn git_receive_pack(
|
||||
// than by the mere fact that release was entered.
|
||||
publish_durability.arm_for_release(reclaimed.publish_stage());
|
||||
if disposition.spawn_tail && !landed_refs.is_empty() {
|
||||
tokio::spawn(post_receive_replication_tail(
|
||||
state.clone(),
|
||||
record.clone(),
|
||||
landed_refs,
|
||||
disk_path.clone(),
|
||||
auth.0.to_string(),
|
||||
Some(publish_durability.arc()),
|
||||
));
|
||||
let state = state.clone();
|
||||
let record = record.clone();
|
||||
let disk_path = disk_path.clone();
|
||||
let did = auth.0.to_string();
|
||||
let db = state.db.clone();
|
||||
let request_id = request_id.clone();
|
||||
let fallback = now.clone();
|
||||
let durability_slot = publish_durability.arc();
|
||||
tokio::spawn(async move {
|
||||
// Same in-task resolution and land rule as the carve-out spawn
|
||||
// above — the handler must not await the lookup while it still
|
||||
// holds the reclaimed write lease.
|
||||
let push_landed_at = db
|
||||
.request_push_landed_at(&request_id)
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
.unwrap_or(fallback);
|
||||
post_receive_replication_tail(
|
||||
state,
|
||||
record,
|
||||
landed_refs,
|
||||
disk_path,
|
||||
did,
|
||||
Some(push_landed_at),
|
||||
Some(durability_slot),
|
||||
)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
}
|
||||
// Short-circuit on a refused publish BEFORE anything downstream observes
|
||||
@@ -3399,6 +3455,7 @@ pub async fn git_receive_pack(
|
||||
// that case via `run_effects`, so this is defensive).
|
||||
let _ = state.db.touch_repo(&record.id).await;
|
||||
crate::metrics::record_push(&record.id);
|
||||
crate::metrics::observe_push_duration(push_started.elapsed());
|
||||
crate::metrics::observe_pack_size(body_len as f64);
|
||||
|
||||
// Claim ownership before inline effects: a background worker may
|
||||
@@ -3674,12 +3731,24 @@ async fn publish_durability_confirmed(
|
||||
/// complete without ever having a live tail (capability-free pushes whose
|
||||
/// landing reconcile proved). `pub(crate)` so the drain can call it; the
|
||||
/// live handler is the other caller.
|
||||
/// The lag origin the tail reports: the caller-supplied push-land instant when
|
||||
/// one exists (a deferred drain passes the request row's created_at), else the
|
||||
/// tail's own entry time for callers that never knew the landing instant.
|
||||
fn lag_origin_timestamp(push_landed_at: Option<String>) -> String {
|
||||
push_landed_at.unwrap_or_else(|| Utc::now().to_rfc3339())
|
||||
}
|
||||
|
||||
pub(crate) async fn post_receive_replication_tail(
|
||||
state: AppState,
|
||||
record: RepoRecord,
|
||||
ref_updates: Vec<RefUpdate>,
|
||||
disk_path: std::path::PathBuf,
|
||||
did: String,
|
||||
// The instant the push landed, captured by the caller at acceptance so a
|
||||
// deferred tail reports push-land lag rather than drain-time lag. Callers
|
||||
// that cannot recover the landing instant pass None and the tail falls
|
||||
// back to its own entry time, the pre-param behavior.
|
||||
push_landed_at: Option<String>,
|
||||
publish_durability: Option<
|
||||
Arc<std::sync::Mutex<Option<crate::git::repo_store::ReleaseOutcome>>>,
|
||||
>,
|
||||
@@ -3690,6 +3759,14 @@ pub(crate) async fn post_receive_replication_tail(
|
||||
.lock_held_transfer_timeout_secs
|
||||
.saturating_add(5),
|
||||
);
|
||||
// The sync-lag origin: the instant this push landed, captured BEFORE the
|
||||
// durability wait so a slow publish does not shrink the lag a mirror
|
||||
// measures. Travels to receiving peers as RefUpdateEvent::lag_origin
|
||||
// (gossip) and the notify lag_origin field (HTTP) — never as the signed
|
||||
// event's own `timestamp` or the notify `timestamp`, which both stay at
|
||||
// publish time for check_freshness and feed-row parity.
|
||||
let origin_timestamp = lag_origin_timestamp(push_landed_at);
|
||||
|
||||
if !publish_durability_confirmed(&publish_durability, durability_wait).await {
|
||||
tracing::warn!(
|
||||
repo = %record.id,
|
||||
@@ -3912,6 +3989,7 @@ pub(crate) async fn post_receive_replication_tail(
|
||||
let owner_did_for_arweave = record.owner_did.clone();
|
||||
let self_public_url = state.config.public_url.clone();
|
||||
let node_keypair = Arc::clone(&state.node_keypair);
|
||||
let origin_timestamp_for_announce = origin_timestamp.clone();
|
||||
// #174 F2a: gated on the cheap announce predicate, not on `withheld`.
|
||||
// `withheld` is None for a push that coalesced (it never walked), and
|
||||
// this task's work is per-push and non-idempotent, so keying it on the
|
||||
@@ -4037,6 +4115,7 @@ pub(crate) async fn post_receive_replication_tail(
|
||||
old_sha: old_sha.clone(),
|
||||
new_sha: new_sha.clone(),
|
||||
timestamp: chrono::Utc::now().to_rfc3339(),
|
||||
lag_origin: Some(origin_timestamp_for_announce.clone()),
|
||||
cert_id: None,
|
||||
cid: cid.map(|s| s.to_string()),
|
||||
sig: None,
|
||||
@@ -4139,6 +4218,7 @@ pub(crate) async fn post_receive_replication_tail(
|
||||
&node_did_str,
|
||||
&pusher_did_clone,
|
||||
&record.owner_did,
|
||||
&origin_timestamp_for_announce,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
@@ -5352,7 +5432,7 @@ const MAX_REF_UPDATES: usize = 8192;
|
||||
/// replication tail at the durability boundary while the certificate and webhook
|
||||
/// loops below still iterate their own copy (#174 U5).
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct RefUpdate {
|
||||
pub(crate) struct RefUpdate {
|
||||
pub(crate) old_sha: String,
|
||||
pub(crate) new_sha: String,
|
||||
pub(crate) ref_name: String,
|
||||
@@ -8062,6 +8142,11 @@ mod tests {
|
||||
mockito::Matcher::PartialJsonString(
|
||||
r#"{"owner_did":"did:key:zOwner"}"#.to_string(),
|
||||
),
|
||||
// Land time rides lag_origin after the publish/land split;
|
||||
// timestamp is a live send instant and cannot be pinned.
|
||||
mockito::Matcher::PartialJsonString(
|
||||
r#"{"lag_origin":"2026-08-16T00:00:00Z"}"#.to_string(),
|
||||
),
|
||||
]))
|
||||
.with_status(200)
|
||||
.expect(1)
|
||||
@@ -8098,6 +8183,7 @@ mod tests {
|
||||
"did:key:zNode",
|
||||
"did:key:zPusher",
|
||||
"did:key:zOwner",
|
||||
"2026-08-16T00:00:00Z",
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -8146,6 +8232,7 @@ mod tests {
|
||||
"did:key:zNode",
|
||||
"did:key:zPusher",
|
||||
"did:key:zOwner",
|
||||
"2026-08-16T00:00:00Z",
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -13962,6 +14049,24 @@ mod tests {
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn lag_origin_timestamp_prefers_the_caller_supplied_push_land() {
|
||||
let landed = "2019-01-01T00:00:00Z".to_string();
|
||||
assert_eq!(
|
||||
lag_origin_timestamp(Some(landed.clone())),
|
||||
landed,
|
||||
"a deferred tail must report the request row's landing instant, not drain time"
|
||||
);
|
||||
let fallback = lag_origin_timestamp(None);
|
||||
let parsed = chrono::DateTime::parse_from_rfc3339(&fallback)
|
||||
.expect("the fallback must still be a valid RFC-3339 instant");
|
||||
let skew = (chrono::Utc::now() - parsed.with_timezone(&chrono::Utc)).num_seconds();
|
||||
assert!(
|
||||
skew.abs() < 60,
|
||||
"a caller that never knew the landing instant falls back to tail entry time"
|
||||
);
|
||||
}
|
||||
|
||||
fn f2a_update(ref_name: &str, new_sha: &str) -> Vec<RefUpdate> {
|
||||
vec![RefUpdate {
|
||||
old_sha: ZERO_SHA.to_string(),
|
||||
@@ -13994,6 +14099,7 @@ mod tests {
|
||||
f2a_update("refs/heads/main", &c1),
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
Some(slot),
|
||||
)
|
||||
.await;
|
||||
@@ -14040,6 +14146,7 @@ mod tests {
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
let after_first = f2a_walks(&log);
|
||||
@@ -14061,6 +14168,7 @@ mod tests {
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -14137,6 +14245,7 @@ mod tests {
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -14219,6 +14328,7 @@ mod tests {
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
));
|
||||
f2a_wait_for(|| started.exists(), "the admitted push's walk to start").await;
|
||||
|
||||
@@ -14336,6 +14446,7 @@ mod tests {
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -15487,6 +15598,7 @@ exit 0
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -15543,6 +15655,7 @@ exit 0
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
tokio::time::sleep(std::time::Duration::from_millis(300)).await;
|
||||
@@ -15585,6 +15698,7 @@ exit 0
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -15684,6 +15798,7 @@ exit 0
|
||||
repo.path().to_path_buf(),
|
||||
F2A_PUSHER.to_string(),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
|
||||
@@ -278,9 +278,9 @@ pub struct Config {
|
||||
#[arg(long, env = "GITLAWB_METRICS_ADDR", default_value = "")]
|
||||
pub metrics_addr: String,
|
||||
|
||||
/// Shutdown grace setting, in seconds. Currently not enforced: the value
|
||||
/// is logged at startup and axum drains in-flight requests on the shutdown
|
||||
/// signal with no upper bound. Default: 30s.
|
||||
/// Maximum time to wait for in-flight requests to drain on shutdown, in
|
||||
/// seconds. After this elapses, the server returns 503 to anything still
|
||||
/// in flight and exits. Default: 30s.
|
||||
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
|
||||
pub shutdown_grace_secs: u64,
|
||||
|
||||
|
||||
@@ -786,27 +786,27 @@ pub struct Db {
|
||||
}
|
||||
|
||||
impl Db {
|
||||
#[cfg(test)]
|
||||
pub fn for_testing(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
|
||||
/// Test-only: access the underlying Postgres connection pool.
|
||||
///
|
||||
/// The production accessor was removed on main (ed76adf) once every
|
||||
/// production caller had migrated to typed `Db` methods; this restores
|
||||
/// it for tests only, for the raw-SQL fixtures and closed-pool cases in
|
||||
/// this change that have no typed equivalent.
|
||||
#[cfg(any(test, feature = "test-harness"))]
|
||||
#[cfg(test)]
|
||||
pub fn pool(&self) -> &PgPool {
|
||||
&self.pool
|
||||
}
|
||||
|
||||
#[cfg(any(test, feature = "test-harness"))]
|
||||
pub fn for_testing(pool: PgPool) -> Self {
|
||||
Self { pool }
|
||||
}
|
||||
|
||||
/// Test-only: apply the full schema to a fresh pool. `#[sqlx::test]`
|
||||
/// provisions an empty per-test database, so DB-backed tests must run this
|
||||
/// before seeding. Reuses the production `migrate()` path (the advisory lock
|
||||
/// is harmless on an isolated test DB and migrations are idempotent).
|
||||
#[cfg(any(test, feature = "test-harness"))]
|
||||
#[cfg(test)]
|
||||
pub(crate) async fn run_migrations(&self) -> Result<()> {
|
||||
self.migrate().await
|
||||
}
|
||||
@@ -2372,6 +2372,16 @@ const MIGRATIONS: &[Migration] = &[
|
||||
"CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_pending ON webhook_deliveries (sent_at, created_at)",
|
||||
],
|
||||
},
|
||||
Migration {
|
||||
version: 47,
|
||||
name: "sync_queue_origin_timestamp",
|
||||
stmts: &[
|
||||
// The source timestamp accompanies a ref update across either
|
||||
// transport. It remains nullable for rows from older peers and
|
||||
// manually-triggered syncs, which must not report a fabricated lag.
|
||||
"ALTER TABLE sync_queue ADD COLUMN IF NOT EXISTS origin_timestamp TEXT",
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
/// Max distinct source repos recorded per pinned object (F1, #173 jatmn round 8).
|
||||
@@ -3532,6 +3542,7 @@ pub struct SyncQueueItem {
|
||||
pub ref_name: String,
|
||||
pub new_sha: String,
|
||||
pub cid: Option<String>,
|
||||
pub origin_timestamp: Option<String>,
|
||||
pub status: String,
|
||||
pub enqueued_at: String,
|
||||
}
|
||||
@@ -3544,10 +3555,11 @@ impl Db {
|
||||
ref_name: &str,
|
||||
new_sha: &str,
|
||||
cid: Option<&str>,
|
||||
origin_timestamp: Option<&str>,
|
||||
) -> Result<()> {
|
||||
sqlx::query(
|
||||
"INSERT INTO sync_queue (id, repo, node_did, ref_name, new_sha, cid, status, enqueued_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, 'pending', $7)
|
||||
"INSERT INTO sync_queue (id, repo, node_did, ref_name, new_sha, cid, origin_timestamp, status, enqueued_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, 'pending', $8)
|
||||
ON CONFLICT DO NOTHING",
|
||||
)
|
||||
.bind(Uuid::new_v4().to_string())
|
||||
@@ -3556,6 +3568,7 @@ impl Db {
|
||||
.bind(ref_name)
|
||||
.bind(new_sha)
|
||||
.bind(cid)
|
||||
.bind(origin_timestamp)
|
||||
.bind(Utc::now().to_rfc3339())
|
||||
.execute(&self.pool)
|
||||
.await?;
|
||||
@@ -3598,7 +3611,7 @@ impl Db {
|
||||
SELECT id FROM sync_queue WHERE status = 'pending'
|
||||
ORDER BY COALESCE(attempted_at, enqueued_at) ASC LIMIT $1
|
||||
)
|
||||
RETURNING id, repo, node_did, ref_name, new_sha, cid, status, enqueued_at",
|
||||
RETURNING id, repo, node_did, ref_name, new_sha, cid, origin_timestamp, status, enqueued_at",
|
||||
)
|
||||
.bind(limit)
|
||||
.bind(Utc::now().to_rfc3339())
|
||||
@@ -3613,6 +3626,7 @@ impl Db {
|
||||
ref_name: r.get("ref_name"),
|
||||
new_sha: r.get("new_sha"),
|
||||
cid: r.get("cid"),
|
||||
origin_timestamp: r.get("origin_timestamp"),
|
||||
status: r.get("status"),
|
||||
enqueued_at: r.get("enqueued_at"),
|
||||
})
|
||||
@@ -5392,6 +5406,25 @@ impl Db {
|
||||
/// this once and processes each row by re-deriving the push event,
|
||||
/// the per-ref cert, and the anchor handoff.
|
||||
#[allow(dead_code)] // wired by the handler refactor in the next slice
|
||||
/// The instant this request's refs actually moved: the latest `applied_at`
|
||||
/// stamp across its applied transitions, or `None` when none are applied
|
||||
/// yet. Shared definition of "push landed" for the live replication tail,
|
||||
/// which must match the deferred drain's `max(applied_at) ?? created_at`
|
||||
/// rule so live and deferred pushes report one lag origin.
|
||||
pub async fn request_push_landed_at(&self, request_id: &str) -> Result<Option<String>> {
|
||||
let row = sqlx::query_scalar::<_, Option<String>>(
|
||||
"SELECT MAX(applied_at) FROM pending_ref_transitions WHERE request_id = $1 AND state = 'applied'",
|
||||
)
|
||||
.bind(request_id)
|
||||
.fetch_one(&self.pool)
|
||||
.await?;
|
||||
Ok(row)
|
||||
}
|
||||
|
||||
/// Test-only: the drain callers that used this moved to the request-scoped
|
||||
/// `request_push_landed_at` and count helpers; kept for the transition-state
|
||||
/// tests below.
|
||||
#[cfg(test)]
|
||||
pub async fn list_pending_ref_transitions_applied(
|
||||
&self,
|
||||
limit: i64,
|
||||
@@ -10454,7 +10487,7 @@ mod migration_tests {
|
||||
db.migrate().await.unwrap();
|
||||
}
|
||||
|
||||
// ── sync_queue scheduling (attempted_at, v17) ────────────────────────────
|
||||
// ── sync_queue scheduling and origin timestamps (v17/v18) ───────────────
|
||||
|
||||
async fn enqueue_one(db: &super::Db, repo: &str) {
|
||||
db.enqueue_sync(
|
||||
@@ -10463,6 +10496,7 @@ mod migration_tests {
|
||||
"refs/heads/main",
|
||||
&"0".repeat(40),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
@@ -10532,6 +10566,72 @@ mod migration_tests {
|
||||
db.migrate().await.unwrap();
|
||||
}
|
||||
|
||||
/// Upgrade-path test for the nullable source timestamp. The worker must
|
||||
/// preserve it when it exists, while rows written before v18 remain valid.
|
||||
#[sqlx::test]
|
||||
async fn migration_v47_adds_sync_queue_origin_timestamp(pool: sqlx::PgPool) {
|
||||
let db = super::Db::for_testing(pool);
|
||||
db.migrate().await.unwrap();
|
||||
|
||||
sqlx::query("ALTER TABLE sync_queue DROP COLUMN origin_timestamp")
|
||||
.execute(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query("DELETE FROM schema_migrations WHERE version = 47")
|
||||
.execute(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO sync_queue (id, repo, node_did, ref_name, new_sha, status, enqueued_at)
|
||||
VALUES ('legacy-sync', 'z6Mkfoo/legacy', 'did:key:zPEER', 'refs/heads/main', $1, 'pending', $2)",
|
||||
)
|
||||
.bind("0".repeat(40))
|
||||
.bind("2026-08-16T00:00:00Z")
|
||||
.execute(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
db.migrate().await.unwrap();
|
||||
|
||||
let col: (String, String) = sqlx::query_as(
|
||||
"SELECT data_type, is_nullable
|
||||
FROM information_schema.columns
|
||||
WHERE table_name = 'sync_queue' AND column_name = 'origin_timestamp'",
|
||||
)
|
||||
.fetch_one(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(col.0, "text");
|
||||
assert_eq!(col.1, "YES", "origin_timestamp must be nullable");
|
||||
|
||||
let items = db.dequeue_pending_syncs(10).await.unwrap();
|
||||
assert_eq!(items.len(), 1);
|
||||
assert_eq!(items[0].origin_timestamp, None);
|
||||
db.migrate().await.unwrap();
|
||||
}
|
||||
|
||||
#[sqlx::test]
|
||||
async fn dequeue_preserves_sync_origin_timestamp(pool: sqlx::PgPool) {
|
||||
let db = super::Db::for_testing(pool);
|
||||
db.migrate().await.unwrap();
|
||||
let timestamp = "2026-08-16T00:00:00Z";
|
||||
db.enqueue_sync(
|
||||
"z6Mkfoo/timestamped",
|
||||
"did:key:zPEER",
|
||||
"refs/heads/main",
|
||||
&"0".repeat(40),
|
||||
None,
|
||||
Some(timestamp),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let items = db.dequeue_pending_syncs(10).await.unwrap();
|
||||
assert_eq!(items.len(), 1);
|
||||
assert_eq!(items[0].origin_timestamp.as_deref(), Some(timestamp));
|
||||
}
|
||||
|
||||
#[sqlx::test]
|
||||
async fn migration_v27_drops_raw_assignee_index(pool: sqlx::PgPool) {
|
||||
let db = super::Db::for_testing(pool);
|
||||
@@ -13128,6 +13228,24 @@ mod icaptcha_quarantine_tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// The sync-lag gate in `sync.rs` reads this flag rather than the
|
||||
/// attempt-local one, so the preserved flag is the exact contract the
|
||||
/// gate's "withheld mirrors never sample" rule depends on.
|
||||
#[sqlx::test]
|
||||
async fn resync_preserved_quarantine_is_visible_to_the_lag_gate(pool: PgPool) {
|
||||
let db = db(pool).await;
|
||||
db.upsert_mirror_repo("z6owner", "garbage", "/srv/g", None, true)
|
||||
.await
|
||||
.unwrap();
|
||||
db.upsert_mirror_repo("z6owner", "garbage", "/srv/g", None, false)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
db.is_repo_quarantined("z6owner/garbage").await.unwrap(),
|
||||
"the lag gate reads is_repo_quarantined: a re-sync that passes quarantined=false must still read withheld here"
|
||||
);
|
||||
}
|
||||
|
||||
/// Quarantined repos are withheld from the deduped listing surfaces.
|
||||
#[sqlx::test]
|
||||
async fn listings_exclude_quarantined(pool: PgPool) {
|
||||
@@ -15398,8 +15516,8 @@ mod peer_authority_tests {
|
||||
/// | `prune_non_public_peers` (db/mod.rs) | a delete keyed on a computed bad-DID array; cannot repoint; boot-only caller in main.rs |
|
||||
/// | `seed_local_peer` (sync.rs) | excluded by test-module location: a deliberate `upsert_peer` bypass for `file://` fixtures, which the public-URL gate rejects |
|
||||
/// | `a_legacy_row_can_still_refresh_its_liveness` (db/mod.rs) | test-only. Seeds a PRE-GATE row by raw SQL on purpose: `upsert_peer` cannot create one, since the gate it is testing refuses exactly that DID. The fixture models what a deployed table already holds |
|
||||
/// | `gossip_ping_round_requires_two_failures_before_persisting_unreachable` (main.rs) | test-only fixture seed. Raw SQL because the test drives the readiness HYSTERESIS, which needs a row already at `last_ping_ok = TRUE` before the round runs; it never exercises the announce gate |
|
||||
/// | `manual_ping_uses_readiness_without_mutating_federation_gate` (api/peers.rs) | test-only fixture seed, same shape and same reason: the row under test must pre-exist so the assertion is about what the ping does NOT rewrite |
|
||||
/// | `gossip_ping_round_requires_two_failures_before_persisting_unreachable` (lib.rs) | test-only fixture seed for the gossip hysteresis round; the row must pre-exist so the assertion is about when `mark_peer_ping` fires |
|
||||
/// | `seed_peer` (p2p/mod.rs) | test-only fixture seed. Raw SQL so the gossip ingest tests can drive the known-peer gate directly, including the did:web case `upsert_peer` would refuse |
|
||||
/// | `federated_peer_query_is_bounded` (api/repos.rs) | test-only fixture seed for the bounded federation query; inserts reachable rows without exercising peer admission |
|
||||
/// | `federated_route_annotates_peer_repos_on_real_200_body` (api/repos.rs) | test-only fixture seed; inserts the mock peer row the bounded federation query reads, so the response-envelope assertions run against a real HTTP 200 body |
|
||||
@@ -15498,11 +15616,11 @@ mod peers_table_writer_guard {
|
||||
1,
|
||||
),
|
||||
(
|
||||
"manual_ping_uses_readiness_without_mutating_federation_gate",
|
||||
"gossip_ping_round_requires_two_failures_before_persisting_unreachable",
|
||||
1,
|
||||
),
|
||||
(
|
||||
"gossip_ping_round_requires_two_failures_before_persisting_unreachable",
|
||||
"manual_ping_uses_readiness_without_mutating_federation_gate",
|
||||
1,
|
||||
),
|
||||
("mark_peer_ping", 1),
|
||||
@@ -16609,4 +16727,48 @@ mod policy_fence_tests {
|
||||
"the unfenced sentinel skips the epoch comparison"
|
||||
);
|
||||
}
|
||||
/// The live replication tail reads its land instant through the same rule
|
||||
/// the deferred drain uses: max applied_at across the request's applied
|
||||
/// transitions, or None so the caller falls back to created_at.
|
||||
#[sqlx::test]
|
||||
async fn request_push_landed_at_returns_max_applied_or_none(pool: PgPool) {
|
||||
let db = db(pool).await;
|
||||
let repo_id = seed_repo(&db).await;
|
||||
db.insert_pending_ref_transitions(
|
||||
"req-live",
|
||||
&repo_id,
|
||||
"did:key:zNode",
|
||||
"did:key:zPusher",
|
||||
&[
|
||||
crate::api::repos::RefUpdate {
|
||||
ref_name: "refs/heads/main".into(),
|
||||
old_sha: "0".repeat(40),
|
||||
new_sha: "a".repeat(40),
|
||||
},
|
||||
crate::api::repos::RefUpdate {
|
||||
ref_name: "refs/heads/dev".into(),
|
||||
old_sha: "0".repeat(40),
|
||||
new_sha: "b".repeat(40),
|
||||
},
|
||||
],
|
||||
"sig",
|
||||
"sigin",
|
||||
"digest",
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
// Nothing applied yet: the caller must fall back to created_at.
|
||||
assert_eq!(db.request_push_landed_at("req-live").await.unwrap(), None);
|
||||
|
||||
db.mark_pending_ref_transitions_applied("req-live")
|
||||
.await
|
||||
.unwrap();
|
||||
let landed = db.request_push_landed_at("req-live").await.unwrap();
|
||||
assert!(
|
||||
landed.is_some(),
|
||||
"an applied request must report a land instant"
|
||||
);
|
||||
// A different request's rows never leak into this request's instant.
|
||||
assert_eq!(db.request_push_landed_at("req-other").await.unwrap(), None);
|
||||
}
|
||||
}
|
||||
@@ -1711,6 +1711,15 @@ fn spawn_deferred_tail_after_proof(
|
||||
}
|
||||
let repo_id = req.repo_id.clone();
|
||||
let pusher_did = req.pusher_did.clone();
|
||||
// The truest landing instant available: the applied_at stamp on the
|
||||
// accepted transitions (written when the refs actually moved), with the
|
||||
// request row's created_at (push acceptance) as the fallback. Either is
|
||||
// truer than this deferred drain's Utc::now().
|
||||
let push_landed_at = accepted
|
||||
.iter()
|
||||
.filter_map(|t| t.applied_at.clone())
|
||||
.max()
|
||||
.unwrap_or_else(|| req.created_at.clone());
|
||||
let landed: Vec<(String, String, String)> = accepted
|
||||
.iter()
|
||||
.map(|c| (c.ref_name.clone(), c.old_sha.clone(), c.new_sha.clone()))
|
||||
@@ -1751,6 +1760,7 @@ fn spawn_deferred_tail_after_proof(
|
||||
ref_updates,
|
||||
std::path::PathBuf::from(repo.disk_path.clone()),
|
||||
pusher_did,
|
||||
Some(push_landed_at),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -82,9 +82,18 @@ pub struct RepoStore {
|
||||
}
|
||||
|
||||
impl RepoStore {
|
||||
#[cfg(any(test, feature = "test-harness"))]
|
||||
#[cfg(test)]
|
||||
pub fn for_testing(repos_dir: PathBuf, lock_pool: PgPool) -> Self {
|
||||
Self::new(repos_dir, None, lock_pool, Duration::from_secs(300))
|
||||
Self {
|
||||
repos_dir,
|
||||
tigris: None,
|
||||
lock_pool,
|
||||
lock_held_transfer_timeout: Duration::from_secs(300),
|
||||
lock_acquire_deadline: LOCK_ACQUIRE_DEADLINE,
|
||||
migrated: Arc::new(tokio::sync::Mutex::new(std::collections::HashSet::new())),
|
||||
pre_unlock_gate: None,
|
||||
upload_site_reached: Arc::new(std::sync::atomic::AtomicUsize::new(0)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Same as [`RepoStore::for_testing`] but with Tigris enabled, so the paths
|
||||
@@ -1228,7 +1237,7 @@ async fn retry_fork_archive_delete(
|
||||
/// `remove_dir_all` / `rename` can take it and static analysers can treat it as
|
||||
/// sanitised input (CodeQL `rust/path-injection`).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ValidatedRepoDiskPath(PathBuf);
|
||||
pub(crate) struct ValidatedRepoDiskPath(PathBuf);
|
||||
|
||||
impl ValidatedRepoDiskPath {
|
||||
pub(crate) fn as_path(&self) -> &Path {
|
||||
@@ -2399,7 +2408,7 @@ impl RepoWriteGuard {
|
||||
/// Only the UNLOCK has to wait for the reaper. This token carries everything
|
||||
/// the settlement itself needs, so the request future can hold it and settle
|
||||
/// the tree at the disconnect instant while the reaper still holds the lock.
|
||||
pub struct TreeSettlement {
|
||||
pub(crate) struct TreeSettlement {
|
||||
local_path: ValidatedRepoDiskPath,
|
||||
repo_name: String,
|
||||
/// Shared with the guard, so the stage read here is the stage the publish
|
||||
|
||||
@@ -663,7 +663,7 @@ pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
|
||||
/// Owns a snapshot temp dir from extraction through handoff to [`RepoSnapshot`].
|
||||
/// Dropped when the `spawn_blocking` join result is abandoned, so cancellation
|
||||
/// before the outer future resumes still removes the directory.
|
||||
pub struct TempSnapshotDir {
|
||||
pub(crate) struct TempSnapshotDir {
|
||||
path: PathBuf,
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+3982
-4
File diff suppressed because it is too large.
Load diff
@@ -5,6 +5,8 @@
|
||||
//!
|
||||
//! * is push traffic flowing? — `gitlawb_pushes_total`
|
||||
//! * is fetch traffic flowing? — `gitlawb_fetches_total`
|
||||
//! * how long do successful push and fetch operations take? —
|
||||
//! `gitlawb_push_duration_seconds` / `gitlawb_fetch_duration_seconds`
|
||||
//! * are signature checks passing or failing? —
|
||||
//! `gitlawb_auth_successes_total` / `gitlawb_auth_failures_total`
|
||||
//! * is the sync worker making progress? —
|
||||
@@ -18,6 +20,8 @@
|
||||
//! `gitlawb_gossip_replay_guard_saturated_total`
|
||||
//! * how big are the packs we're sending and receiving? —
|
||||
//! `gitlawb_pack_size_bytes`
|
||||
//! * how long after an origin lands a push is it visible on this mirror? —
|
||||
//! `gitlawb_sync_lag_seconds`
|
||||
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
|
||||
//! * currently-connected peer count — `gitlawb_peers_connected`
|
||||
//! * reconciliation sweep gaps found and filled —
|
||||
@@ -34,11 +38,10 @@
|
||||
//! free.
|
||||
//!
|
||||
//! Follow-ups (not in this module):
|
||||
//! * per-route latency histograms (TraceLayer already gives us spans)
|
||||
//! * per-peer p2p counters
|
||||
//! * ipfs / pinata counters
|
||||
|
||||
use std::sync::OnceLock;
|
||||
use std::{sync::OnceLock, time::Duration};
|
||||
|
||||
use prometheus::{
|
||||
Encoder, Histogram, HistogramOpts, IntCounter, IntCounterVec, IntGauge, IntGaugeVec, Opts,
|
||||
@@ -60,6 +63,9 @@ static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
|
||||
static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
|
||||
static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new();
|
||||
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
|
||||
static PUSH_DURATION: OnceLock<Histogram> = OnceLock::new();
|
||||
static FETCH_DURATION: OnceLock<Histogram> = OnceLock::new();
|
||||
static SYNC_LAG: OnceLock<Histogram> = OnceLock::new();
|
||||
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
|
||||
static RECONCILIATION_GAPS_FOUND: OnceLock<IntCounter> = OnceLock::new();
|
||||
static RECONCILIATION_GAPS_FILLED: OnceLock<IntCounter> = OnceLock::new();
|
||||
@@ -125,6 +131,51 @@ fn init_inner(version: &str, node_did: &str) {
|
||||
.expect("register gitlawb_fetches_total");
|
||||
FETCHES.set(fetches).expect("set FETCHES once");
|
||||
|
||||
let push_duration = Histogram::with_opts(
|
||||
HistogramOpts::new(
|
||||
"gitlawb_push_duration_seconds",
|
||||
"Wall time of a successful git push (receive-pack) request in seconds, from handler entry to the point its effects are recorded",
|
||||
)
|
||||
.buckets(operation_duration_buckets()),
|
||||
)
|
||||
.expect("gitlawb_push_duration_seconds definition");
|
||||
registry
|
||||
.register(Box::new(push_duration.clone()))
|
||||
.expect("register gitlawb_push_duration_seconds");
|
||||
PUSH_DURATION
|
||||
.set(push_duration)
|
||||
.expect("set PUSH_DURATION once");
|
||||
|
||||
let fetch_duration = Histogram::with_opts(
|
||||
HistogramOpts::new(
|
||||
"gitlawb_fetch_duration_seconds",
|
||||
"Duration of successful git fetch (upload-pack) completion requests in seconds",
|
||||
)
|
||||
.buckets(operation_duration_buckets()),
|
||||
)
|
||||
.expect("gitlawb_fetch_duration_seconds definition");
|
||||
registry
|
||||
.register(Box::new(fetch_duration.clone()))
|
||||
.expect("register gitlawb_fetch_duration_seconds");
|
||||
FETCH_DURATION
|
||||
.set(fetch_duration)
|
||||
.expect("set FETCH_DURATION once");
|
||||
|
||||
let sync_lag = Histogram::with_opts(
|
||||
HistogramOpts::new(
|
||||
"gitlawb_sync_lag_seconds",
|
||||
"Lag from the origin push-land timestamp (lag_origin) to successful local mirror visibility in seconds",
|
||||
)
|
||||
.buckets(vec![
|
||||
1.0, 5.0, 10.0, 30.0, 60.0, 120.0, 300.0, 600.0, 1_800.0, 3_600.0,
|
||||
]),
|
||||
)
|
||||
.expect("gitlawb_sync_lag_seconds definition");
|
||||
registry
|
||||
.register(Box::new(sync_lag.clone()))
|
||||
.expect("register gitlawb_sync_lag_seconds");
|
||||
SYNC_LAG.set(sync_lag).expect("set SYNC_LAG once");
|
||||
|
||||
let auth_successes = IntCounterVec::new(
|
||||
Opts::new(
|
||||
"gitlawb_auth_successes_total",
|
||||
@@ -302,6 +353,27 @@ pub fn record_fetch(repo: &str) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Observe the elapsed time of a successful git push handler.
|
||||
pub fn observe_push_duration(duration: Duration) {
|
||||
observe_duration(&PUSH_DURATION, duration);
|
||||
}
|
||||
|
||||
/// Observe the elapsed time of a successful git fetch completion request.
|
||||
pub fn observe_fetch_duration(duration: Duration) {
|
||||
observe_duration(&FETCH_DURATION, duration);
|
||||
}
|
||||
|
||||
/// Observe end-to-end lag until an origin ref update is visible on this mirror.
|
||||
pub fn observe_sync_lag(duration: Duration) {
|
||||
observe_duration(&SYNC_LAG, duration);
|
||||
}
|
||||
|
||||
fn observe_duration(histogram: &OnceLock<Histogram>, duration: Duration) {
|
||||
if let Some(h) = histogram.get() {
|
||||
h.observe(duration.as_secs_f64());
|
||||
}
|
||||
}
|
||||
|
||||
/// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry
|
||||
/// is not initialized). Lets api-layer tests assert the completed-fetch count with
|
||||
/// a unique label instead of scraping the encoded text.
|
||||
@@ -420,6 +492,12 @@ pub fn observe_pack_size(bytes: f64) {
|
||||
}
|
||||
}
|
||||
|
||||
fn operation_duration_buckets() -> Vec<f64> {
|
||||
vec![
|
||||
0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0, 30.0, 60.0, 120.0, 300.0,
|
||||
]
|
||||
}
|
||||
|
||||
/// Update the currently-connected peer count gauge.
|
||||
pub fn set_peers_connected(count: i64) {
|
||||
if let Some(g) = PEERS_CONNECTED.get() {
|
||||
@@ -485,6 +563,9 @@ mod tests {
|
||||
.expect("PUSHES set after init")
|
||||
.with_label_values(&["alice/repo"])
|
||||
.inc();
|
||||
observe_push_duration(Duration::from_millis(42));
|
||||
observe_fetch_duration(Duration::from_millis(24));
|
||||
observe_sync_lag(Duration::from_secs(12));
|
||||
record_reconciliation_gaps_found(7);
|
||||
record_reconciliation_gaps_filled(3);
|
||||
|
||||
@@ -501,6 +582,16 @@ mod tests {
|
||||
body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"),
|
||||
"expected the incremented counter to be visible in: {body}"
|
||||
);
|
||||
for metric in [
|
||||
"gitlawb_push_duration_seconds",
|
||||
"gitlawb_fetch_duration_seconds",
|
||||
"gitlawb_sync_lag_seconds",
|
||||
] {
|
||||
assert!(
|
||||
body.contains(&format!("# TYPE {metric} histogram")),
|
||||
"expected {metric} histogram in: {body}"
|
||||
);
|
||||
}
|
||||
assert!(
|
||||
body.contains("gitlawb_reconciliation_gaps_found_total 7"),
|
||||
"expected the reconciliation gaps-found counter to be visible in: {body}"
|
||||
@@ -546,6 +637,9 @@ mod tests {
|
||||
// no diagnostic value.
|
||||
record_gossip_ingest("accepted");
|
||||
observe_pack_size(1024.0);
|
||||
observe_push_duration(Duration::from_secs(1));
|
||||
observe_fetch_duration(Duration::from_secs(1));
|
||||
observe_sync_lag(Duration::from_secs(1));
|
||||
set_peers_connected(0);
|
||||
}
|
||||
|
||||
|
||||
@@ -306,8 +306,9 @@ impl IngestLimiters {
|
||||
///
|
||||
/// 0 is the versionless form: the field set this struct shipped with, before
|
||||
/// `v` existed. It is not a placeholder for "unset", it is a real version whose
|
||||
/// wire encoding happens to omit the key.
|
||||
pub(crate) const CURRENT_REF_UPDATE_VERSION: u32 = 0;
|
||||
/// wire encoding happens to omit the key. 1 adds `lag_origin`, the first
|
||||
/// version-gated field: it is emitted and verified only under v1.
|
||||
pub(crate) const CURRENT_REF_UPDATE_VERSION: u32 = 1;
|
||||
|
||||
/// True for the version whose key is omitted from the wire form. Free function
|
||||
/// because `skip_serializing_if` takes a path, not a closure.
|
||||
@@ -376,8 +377,15 @@ pub struct RefUpdateEvent {
|
||||
pub old_sha: String,
|
||||
/// SHA after the push
|
||||
pub new_sha: String,
|
||||
/// RFC-3339 timestamp
|
||||
/// RFC-3339 timestamp of the publish, freshness-checked at ingest
|
||||
pub timestamp: String,
|
||||
/// RFC-3339 instant the origin push landed, for sync-lag measurement only.
|
||||
/// Kept out of `timestamp` so `check_freshness` still sees the publish
|
||||
/// instant: a push whose durability wait ran long would otherwise sign an
|
||||
/// already-stale event. Optional for older peers; ingest falls back to
|
||||
/// `timestamp` when it is absent.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub lag_origin: Option<String>,
|
||||
/// Certificate ID (from the ref certificate, if issued)
|
||||
pub cert_id: Option<String>,
|
||||
/// IPFS CID of the latest commit object (set after pinning completes)
|
||||
@@ -1192,6 +1200,14 @@ pub(crate) async fn ingest_ref_update(
|
||||
event.v, CURRENT_REF_UPDATE_VERSION
|
||||
));
|
||||
}
|
||||
// `lag_origin` is a v1 field: a v0 event carrying it is a protocol
|
||||
// violation, not a newer peer's wire form, and refusing it keeps the field
|
||||
// set inside each version's signing bytes exactly what that version means.
|
||||
if event.v < 1 && event.lag_origin.is_some() {
|
||||
return IngestOutcome::Rejected(
|
||||
"v0 ref-update event must not carry the v1 field lag_origin".to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
// The cheapest denial under enforcement, ahead of every gate that reads a
|
||||
// field: an unsigned event is refused before its `node_did` is resolved or
|
||||
@@ -1498,6 +1514,12 @@ pub(crate) async fn ingest_ref_update(
|
||||
&event.ref_name,
|
||||
&event.new_sha,
|
||||
event.cid.as_deref(),
|
||||
Some(
|
||||
event
|
||||
.lag_origin
|
||||
.as_deref()
|
||||
.unwrap_or(event.timestamp.as_str()),
|
||||
),
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -7193,7 +7215,7 @@ mod tests {
|
||||
#[test]
|
||||
fn ref_update_event_round_trip_with_owner_did() {
|
||||
let event = RefUpdateEvent {
|
||||
v: 0,
|
||||
v: CURRENT_REF_UPDATE_VERSION,
|
||||
node_did: "did:key:zNode".into(),
|
||||
pusher_did: "did:key:zPusher".into(),
|
||||
repo: "zOwner/myrepo".into(),
|
||||
@@ -7202,6 +7224,7 @@ mod tests {
|
||||
old_sha: "0000000000000000000000000000000000000000".into(),
|
||||
new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(),
|
||||
timestamp: "2026-07-02T12:00:00Z".into(),
|
||||
lag_origin: Some("2026-07-02T11:59:30Z".into()),
|
||||
cert_id: None,
|
||||
cid: None,
|
||||
sig: None,
|
||||
@@ -7213,6 +7236,7 @@ mod tests {
|
||||
|
||||
let deserialized: RefUpdateEvent = serde_json::from_value(json).unwrap();
|
||||
assert_eq!(deserialized.owner_did, Some("did:key:zOwner".into()));
|
||||
assert_eq!(deserialized.lag_origin, Some("2026-07-02T11:59:30Z".into()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -7255,7 +7279,7 @@ mod tests {
|
||||
/// field is Some so the serialized form exercises the widest field set.
|
||||
fn populated_event() -> RefUpdateEvent {
|
||||
RefUpdateEvent {
|
||||
v: 0,
|
||||
v: CURRENT_REF_UPDATE_VERSION,
|
||||
node_did: "did:key:zNode".into(),
|
||||
pusher_did: "did:key:zPusher".into(),
|
||||
repo: "zOwner/myrepo".into(),
|
||||
@@ -7264,6 +7288,7 @@ mod tests {
|
||||
old_sha: "0000000000000000000000000000000000000000".into(),
|
||||
new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(),
|
||||
timestamp: "2026-07-02T12:00:00Z".into(),
|
||||
lag_origin: Some("2026-07-02T11:59:30Z".into()),
|
||||
cert_id: Some("cert-1".into()),
|
||||
cid: Some("bafycid".into()),
|
||||
sig: None,
|
||||
@@ -7318,30 +7343,53 @@ mod tests {
|
||||
cid: Option<String>,
|
||||
}
|
||||
|
||||
let mut event = populated_event();
|
||||
event.sig = Some("c2lnbmF0dXJl".into());
|
||||
let bytes = serde_json::to_vec(&event).unwrap();
|
||||
// The fleet rule this version split is built around:
|
||||
//
|
||||
// * A v0 event is the pre-change wire form: no `v` key, no `lag_origin`,
|
||||
// and it must still parse under the OLD struct verbatim.
|
||||
// * A v1 event is for nodes that bumped with us. An un-upgraded peer
|
||||
// rejects it on the `v` key itself, which is the LOUD version drop the
|
||||
// ingest ceiling was designed to name, not a silent signature
|
||||
// mismatch against bytes it could never compute.
|
||||
let mut v0_event = populated_event();
|
||||
v0_event.v = 0;
|
||||
v0_event.lag_origin = None;
|
||||
v0_event.sig = Some("c2lnbmF0dXJl".into());
|
||||
let bytes = serde_json::to_vec(&v0_event).unwrap();
|
||||
|
||||
let legacy: LegacyRefUpdateEvent = serde_json::from_slice(&bytes)
|
||||
.expect("new-code bytes must deserialize under the old field set");
|
||||
.expect("v0 bytes must deserialize under the old field set");
|
||||
assert_eq!(legacy.repo, "zOwner/myrepo");
|
||||
assert_eq!(legacy.owner_did, Some("did:key:zOwner".into()));
|
||||
|
||||
// An UNSIGNED event carries no `sig` key at all, so it is byte-identical
|
||||
// An UNSIGNED v0 event carries no `sig` key at all, so it is byte-identical
|
||||
// in shape to the pre-change wire form and parses even under the strict
|
||||
// reader. This is what `skip_serializing_if` buys; drop it and a
|
||||
// `"sig": null` key appears here and this goes red.
|
||||
let unsigned_bytes = serde_json::to_vec(&populated_event()).unwrap();
|
||||
v0_event.sig = None;
|
||||
let unsigned_bytes = serde_json::to_vec(&v0_event).unwrap();
|
||||
let strict: StrictLegacyRefUpdateEvent = serde_json::from_slice(&unsigned_bytes)
|
||||
.expect("an unsigned event must carry no field the pre-change reader did not know");
|
||||
.expect("an unsigned v0 event must carry no field the pre-change reader did not know");
|
||||
assert_eq!(strict.repo, "zOwner/myrepo");
|
||||
|
||||
// A SIGNED event does carry the new key, and that is intentional, not a
|
||||
// compatibility bug: the permissive reader above is what makes it
|
||||
// harmless. Pinning the refusal here documents `sig` as the one added
|
||||
// field, so a SECOND addition cannot slip in unnoticed.
|
||||
serde_json::from_slice::<StrictLegacyRefUpdateEvent>(&bytes)
|
||||
.expect_err("a signed event must be visibly carrying the added `sig` key");
|
||||
// A SIGNED v0 event does carry the added `sig` key, and that is
|
||||
// intentional, not a compatibility bug: the permissive reader above is
|
||||
// what makes it harmless. Pinning the refusal here documents `sig` as
|
||||
// an added field, so a SECOND unsigned-field addition cannot slip in
|
||||
// unnoticed.
|
||||
v0_event.sig = Some("c2lnbmF0dXJl".into());
|
||||
serde_json::from_slice::<StrictLegacyRefUpdateEvent>(
|
||||
&serde_json::to_vec(&v0_event).unwrap(),
|
||||
)
|
||||
.expect_err("a signed v0 event must be visibly carrying the added `sig` key");
|
||||
|
||||
// A v1 event is refused by the pre-change reader on the `v` key:
|
||||
// refusing to parse IS the version drop old nodes perform, the same
|
||||
// failure the ingest ceiling produces in the other direction.
|
||||
serde_json::from_slice::<StrictLegacyRefUpdateEvent>(
|
||||
&serde_json::to_vec(&populated_event()).unwrap(),
|
||||
)
|
||||
.expect_err("a v1 event must fail loudly on the `v` key for a pre-change reader");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -7398,12 +7446,13 @@ mod tests {
|
||||
/// this failure, and the emit-to-ingest round trip is structurally blind to
|
||||
/// it because both sides re-serialize the same new declaration order.
|
||||
const GOLDEN_SIGNING_BYTES: &str = concat!(
|
||||
r#"{"node_did":"did:key:zNode","pusher_did":"did:key:zPusher","#,
|
||||
r#"{"v":1,"node_did":"did:key:zNode","pusher_did":"did:key:zPusher","#,
|
||||
r#""repo":"zOwner/myrepo","owner_did":"did:key:zOwner","#,
|
||||
r#""ref_name":"refs/heads/main","#,
|
||||
r#""old_sha":"0000000000000000000000000000000000000000","#,
|
||||
r#""new_sha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","#,
|
||||
r#""timestamp":"2026-07-02T12:00:00Z","cert_id":"cert-1","cid":"bafycid"}"#,
|
||||
r#""timestamp":"2026-07-02T12:00:00Z","lag_origin":"2026-07-02T11:59:30Z","#,
|
||||
r#""cert_id":"cert-1","cid":"bafycid"}"#,
|
||||
);
|
||||
|
||||
#[test]
|
||||
@@ -7448,9 +7497,15 @@ mod tests {
|
||||
/// The all-`None` optional shape: what the production publish site emits.
|
||||
fn all_none_optionals_event() -> RefUpdateEvent {
|
||||
RefUpdateEvent {
|
||||
// v0 pins the versionless form: the key stays off the wire and the
|
||||
// signing bytes are byte-identical to what a pre-version build emits.
|
||||
v: 0,
|
||||
owner_did: None,
|
||||
cert_id: None,
|
||||
cid: None,
|
||||
// None skips the key entirely, so the production-shape golden stays
|
||||
// byte-identical even though lag_origin exists.
|
||||
lag_origin: None,
|
||||
..populated_event()
|
||||
}
|
||||
}
|
||||
@@ -7675,8 +7730,11 @@ mod tests {
|
||||
let keypair = Keypair::generate();
|
||||
|
||||
// Signed at v0, where the key is absent from the signing bytes.
|
||||
// Raising it on the received copy makes the key appear.
|
||||
// Raising it on the received copy makes the key appear. The v0 shape
|
||||
// drops lag_origin too, since v0 may not carry the v1 field.
|
||||
let mut at_zero = event_for(&keypair);
|
||||
at_zero.v = 0;
|
||||
at_zero.lag_origin = None;
|
||||
sign_ref_update(&keypair, &mut at_zero).unwrap();
|
||||
verify_ref_update(&at_zero).expect("baseline must verify before tampering");
|
||||
let mut raised = at_zero.clone();
|
||||
@@ -7981,7 +8039,14 @@ mod tests {
|
||||
&bytes_of(&event),
|
||||
"v1 event",
|
||||
Some((
|
||||
"unsupported ref-update event version 1; this build understands version 0",
|
||||
Box::leak(
|
||||
format!(
|
||||
"unsupported ref-update event version {}; this build understands version {}",
|
||||
CURRENT_REF_UPDATE_VERSION + 1,
|
||||
CURRENT_REF_UPDATE_VERSION
|
||||
)
|
||||
.into_boxed_str(),
|
||||
),
|
||||
"an unknown version must be named as one, never reported as a signature failure",
|
||||
)),
|
||||
)
|
||||
|
||||
@@ -16,7 +16,9 @@ use std::collections::HashMap;
|
||||
use std::path::Path;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use gitlawb_core::identity::Keypair;
|
||||
use tracing::{error, info, warn};
|
||||
|
||||
@@ -180,6 +182,27 @@ fn resolve_origin_url(peers: &[crate::db::PeerRecord], node_did: &str) -> Option
|
||||
.map(|p| p.http_url.trim_end_matches('/').to_string())
|
||||
}
|
||||
|
||||
/// Convert a sender-supplied RFC-3339 ref-update timestamp into an observable
|
||||
/// lag. Missing, malformed, and future timestamps are deliberately ignored:
|
||||
/// older peers and clock-skewed peers must not create fabricated observations.
|
||||
const MAX_SYNC_LAG_OBSERVATION: Duration = Duration::from_secs(86_400);
|
||||
|
||||
fn sync_lag_from_origin_timestamp(
|
||||
origin_timestamp: Option<&str>,
|
||||
visible_at: DateTime<Utc>,
|
||||
) -> Option<Duration> {
|
||||
let origin = DateTime::parse_from_rfc3339(origin_timestamp?).ok()?;
|
||||
let lag = visible_at
|
||||
.signed_duration_since(origin.with_timezone(&Utc))
|
||||
.to_std()
|
||||
.ok()?;
|
||||
// The origin timestamp is unauthenticated input from a known-but-not-proofed
|
||||
// peer: an absurd age (a 1970 date, a stale clock) would park a fabricated
|
||||
// observation in the histogram's tail buckets, so past a day the value is
|
||||
// ignored rather than sampled.
|
||||
(lag <= MAX_SYNC_LAG_OBSERVATION).then_some(lag)
|
||||
}
|
||||
|
||||
async fn process_batch(
|
||||
db: &Db,
|
||||
config: &Config,
|
||||
@@ -396,7 +419,7 @@ async fn process_batch(
|
||||
false
|
||||
};
|
||||
// Register in DB so git smart HTTP can serve the mirrored repo
|
||||
let _ = db
|
||||
let mirror_visible = db
|
||||
.upsert_mirror_repo(
|
||||
owner_short,
|
||||
repo_name,
|
||||
@@ -404,7 +427,34 @@ async fn process_batch(
|
||||
machine_id,
|
||||
quarantined,
|
||||
)
|
||||
.await;
|
||||
.await
|
||||
.is_ok();
|
||||
// The lag is push-to-visible: sample at registration, before
|
||||
// the encrypted-blob tail and mark_sync_done. A quarantined
|
||||
// mirror is admitted but withheld from readers, so it reports
|
||||
// no visibility lag. The gate reads the PERSISTED flag, not the
|
||||
// attempt-local one: on a re-sync `quarantined` is false while
|
||||
// `upsert_mirror_repo` preserves an existing quarantine, and
|
||||
// sampling then would record visibility the readers still deny.
|
||||
let withheld = db.is_repo_quarantined(&item.repo).await.unwrap_or(true);
|
||||
// Only sample when the queued outcome is what the mirror now
|
||||
// shows: if the origin advanced the same ref past this row
|
||||
// before the fetch ran, the mirror carries the newer tip and
|
||||
// this row's visibility was never real. Deletions confirm when
|
||||
// the ref is absent.
|
||||
let tip = mirror_ref_tip(&local_path, &item.ref_name).await;
|
||||
let outcome_visible = if item.new_sha.bytes().all(|b| b == b'0') {
|
||||
tip.is_none()
|
||||
} else {
|
||||
tip.as_deref() == Some(item.new_sha.as_str())
|
||||
};
|
||||
if mirror_visible && !withheld && outcome_visible {
|
||||
if let Some(lag) =
|
||||
sync_lag_from_origin_timestamp(item.origin_timestamp.as_deref(), Utc::now())
|
||||
{
|
||||
crate::metrics::observe_sync_lag(lag);
|
||||
}
|
||||
}
|
||||
// Option B2: carry the encrypted withheld-blob envelopes too, so an
|
||||
// authorized reader can recover private content from this mirror if
|
||||
// the origin dies. `item.repo` is the slug "{owner_short}/{name}",
|
||||
@@ -649,6 +699,32 @@ async fn git_run(args: &[&str]) -> anyhow::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The sha `ref_name` resolves to inside `repo`, or `None` when the ref is
|
||||
/// absent or the read fails. Used by the lag observation gate to confirm the
|
||||
/// queued outcome is what the mirror actually carries before sampling.
|
||||
async fn mirror_ref_tip(repo: &std::path::Path, ref_name: &str) -> Option<String> {
|
||||
// allow-unbounded-git: bounded by the tokio::time::timeout wrapping this
|
||||
// read below, plus kill_on_drop; rev-parse is a metadata lookup, not a
|
||||
// transfer, so the 10s cap is generous.
|
||||
let mut cmd = tokio::process::Command::new("git");
|
||||
cmd.kill_on_drop(true);
|
||||
let out = tokio::time::timeout(
|
||||
std::time::Duration::from_secs(10),
|
||||
cmd.args(["-C"])
|
||||
.arg(repo)
|
||||
.args(["rev-parse", "--verify", ref_name])
|
||||
.output(),
|
||||
)
|
||||
.await
|
||||
.ok()?
|
||||
.ok()?;
|
||||
if !out.status.success() {
|
||||
return None;
|
||||
}
|
||||
let sha = String::from_utf8_lossy(&out.stdout).trim().to_string();
|
||||
(!sha.is_empty()).then_some(sha)
|
||||
}
|
||||
|
||||
/// Run a git subprocess, ignoring a non-zero exit. Used for idempotent
|
||||
/// `config --unset`, which exits non-zero when the key is already absent.
|
||||
async fn git_run_lenient(args: &[&str]) {
|
||||
@@ -1582,11 +1658,32 @@ mod tests {
|
||||
}
|
||||
|
||||
async fn enqueue(db: &Db, repo: &str, did: &str) {
|
||||
db.enqueue_sync(repo, did, "refs/heads/main", &"0".repeat(40), None)
|
||||
db.enqueue_sync(repo, did, "refs/heads/main", &"0".repeat(40), None, None)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sync_lag_uses_origin_timestamp_only_when_it_is_in_the_past() {
|
||||
let visible_at = chrono::DateTime::parse_from_rfc3339("2026-08-16T00:00:12Z")
|
||||
.unwrap()
|
||||
.with_timezone(&chrono::Utc);
|
||||
|
||||
assert_eq!(
|
||||
sync_lag_from_origin_timestamp(Some("2026-08-16T00:00:00Z"), visible_at),
|
||||
Some(Duration::from_secs(12))
|
||||
);
|
||||
assert_eq!(sync_lag_from_origin_timestamp(None, visible_at), None);
|
||||
assert_eq!(
|
||||
sync_lag_from_origin_timestamp(Some("not-a-timestamp"), visible_at),
|
||||
None
|
||||
);
|
||||
assert_eq!(
|
||||
sync_lag_from_origin_timestamp(Some("2026-08-16T00:00:13Z"), visible_at),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
fn dir_entries(dir: &Path) -> Vec<String> {
|
||||
let mut names: Vec<String> = std::fs::read_dir(dir)
|
||||
.unwrap()
|
||||
@@ -2808,6 +2905,7 @@ mod tests {
|
||||
cid: None,
|
||||
status: "pending".to_string(),
|
||||
enqueued_at: chrono::Utc::now().to_rfc3339(),
|
||||
origin_timestamp: None,
|
||||
};
|
||||
async fn status_of(pool: &sqlx::PgPool, id: &str) -> String {
|
||||
let row: (String,) = sqlx::query_as("SELECT status FROM sync_queue WHERE id = $1")
|
||||
|
||||
@@ -6,7 +6,6 @@ use std::sync::OnceLock;
|
||||
|
||||
/// The fixture delays or hangs in its own process, so the production watchdog
|
||||
/// can terminate it without depending on a shell or descendant process support.
|
||||
#[allow(dead_code)]
|
||||
pub(super) enum Behavior<'a> {
|
||||
Delay(u64),
|
||||
Hang,
|
||||
@@ -15,7 +14,6 @@ pub(super) enum Behavior<'a> {
|
||||
LogTypes(&'a Path),
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub(super) struct GitShim {
|
||||
_directory: tempfile::TempDir,
|
||||
executable: PathBuf,
|
||||
@@ -29,7 +27,6 @@ impl Deref for GitShim {
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub(super) fn create(name: &str, behavior: Behavior<'_>) -> GitShim {
|
||||
static COMPILED: OnceLock<tempfile::TempDir> = OnceLock::new();
|
||||
let compiled = COMPILED.get_or_init(|| {
|
||||
|
||||
@@ -1,601 +0,0 @@
|
||||
//! Test-only spawn surface for the real-node deny harness (see
|
||||
//! `tests/deny_harness.rs`). Feature-gated behind `test-harness` so it never
|
||||
//! compiles into the production binary. It exposes a single boot constructor so
|
||||
//! the out-of-crate integration test can bring up a real node over a bound TCP
|
||||
//! socket without the integration crate needing access to `graphql`,
|
||||
//! `rate_limit`, `repo_store`, or the other internals `AppState` is built from.
|
||||
//!
|
||||
//! The node is built the same way `test_support::build_state` builds it (p2p
|
||||
//! disabled, real migrated pool), but bound on `127.0.0.1:0` and served through
|
||||
//! the real `axum::serve` stack with connect-info, so the middleware order,
|
||||
//! body limits, and per-IP rate limiters all run exactly as in production.
|
||||
|
||||
use std::net::SocketAddr;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use chrono::Utc;
|
||||
use clap::Parser;
|
||||
use sqlx::PgPool;
|
||||
use tokio::net::TcpListener;
|
||||
use tokio::sync::watch;
|
||||
use uuid::Uuid;
|
||||
|
||||
use gitlawb_core::identity::Keypair;
|
||||
|
||||
use crate::config::Config;
|
||||
use crate::db::{Db, PullRequest, RepoRecord, VisibilityMode};
|
||||
use crate::rate_limit::{RateLimiter, TrustedProxy};
|
||||
use crate::state::AppState;
|
||||
|
||||
/// A running node bound to an ephemeral port. End every test with
|
||||
/// [`TestNode::shutdown`]: it joins the serve task and closes the pool, so
|
||||
/// `#[sqlx::test]`'s `DROP DATABASE` cleanup (which runs the moment the test
|
||||
/// future returns) never races the server's still-open connections. Dropping
|
||||
/// without it signals shutdown, aborts any remaining serve task, and removes
|
||||
/// the temp repository directory, a fallback for tests that return or panic
|
||||
/// before reaching teardown.
|
||||
pub struct TestNode {
|
||||
/// Base URL of the running node, e.g. `http://127.0.0.1:54321`.
|
||||
pub base_url: String,
|
||||
/// The node's own DID (for building requests that reference it).
|
||||
pub node_did: String,
|
||||
shutdown_tx: watch::Sender<bool>,
|
||||
repos_dir: PathBuf,
|
||||
/// Seeding handle over the same pool the node serves from. Kept private so
|
||||
/// the integration crate seeds through the methods below rather than
|
||||
/// naming `Db`/`RepoRecord` directly.
|
||||
db: Arc<Db>,
|
||||
/// The detached `axum::serve` task, joined by [`Self::shutdown`]. `None`
|
||||
/// once taken, so the `Drop` fallback never overlaps a completed teardown.
|
||||
/// Carries `serve`'s own `io::Result<()>` so `shutdown()` can observe it
|
||||
/// rather than discard it (see the coverage note there).
|
||||
server: Option<tokio::task::JoinHandle<std::io::Result<()>>>,
|
||||
/// A clone of the served pool, kept so [`Self::shutdown`] can close it
|
||||
/// (sqlx pool clones share one inner, so closing this closes the clones
|
||||
/// inside `AppState`/`Db` too).
|
||||
pool: PgPool,
|
||||
/// Test-only: when set, `Drop` skips the graceful watch signal so ONLY
|
||||
/// `handle.abort()` tears the serve task down. Defaults false, so every
|
||||
/// other test keeps both redundant teardown mechanisms. Set via
|
||||
/// [`Self::force_abort_only_teardown`] to fence that the abort alone
|
||||
/// releases the database when the graceful chain is broken.
|
||||
suppress_graceful_shutdown_on_drop: bool,
|
||||
}
|
||||
|
||||
impl Drop for TestNode {
|
||||
fn drop(&mut self) {
|
||||
// Fallback for tests that return or panic before reaching
|
||||
// [`TestNode::shutdown`]: flip the shared shutdown signal so the serve
|
||||
// task exits gracefully, abort whatever is left of it, and remove the
|
||||
// temp repos dir. Drop cannot await, so it cannot join the server or
|
||||
// close the pool; a test that already failed should not panic again in
|
||||
// teardown. All three actions are idempotent or no-ops after
|
||||
// `shutdown()` (which takes the handle), so running afterwards is
|
||||
// harmless.
|
||||
//
|
||||
// The abort matters when the graceful chain does not run to completion
|
||||
// before sqlx's `DROP DATABASE` cleanup: `JoinHandle::abort()` is a
|
||||
// synchronous cancellation request callable from non-async Drop, and
|
||||
// the aborted task's future (with the `PgPool` clones inside its
|
||||
// router/state) is dropped on a subsequent scheduler tick, which sqlx
|
||||
// cleanup's own awaits provide before the drop statement runs. The
|
||||
// race is closed by those scheduling points, not by the abort alone.
|
||||
//
|
||||
// `suppress_graceful_shutdown_on_drop` (test-only) forces the abort to
|
||||
// carry teardown by itself, so a test can fence that the abort line is
|
||||
// load-bearing; it is false everywhere else.
|
||||
if !self.suppress_graceful_shutdown_on_drop {
|
||||
let _ = self.shutdown_tx.send(true);
|
||||
}
|
||||
if let Some(handle) = self.server.take() {
|
||||
handle.abort();
|
||||
}
|
||||
let _ = std::fs::remove_dir_all(&self.repos_dir);
|
||||
}
|
||||
}
|
||||
|
||||
/// Outcome of joining the serve task during teardown, split so the wedged-task
|
||||
/// path is observable to a test (which injects a short timeout) without waiting
|
||||
/// the production teardown window.
|
||||
enum ServeTeardown {
|
||||
/// The task returned (carrying `serve`'s `io::Result<()>`) — either within
|
||||
/// the timeout, or in the race between the timeout firing and the abort
|
||||
/// landing. A task that actually finished is never a wedge.
|
||||
Graceful(std::io::Result<()>),
|
||||
/// The task did not finish within the timeout and was aborted (cancelled)
|
||||
/// and reaped.
|
||||
TimedOutAborted,
|
||||
/// The serve task panicked. The payload is carried out so the caller can
|
||||
/// resume unwinding only AFTER the pool is closed and the temp dir removed.
|
||||
Panicked(Box<dyn std::any::Any + Send>),
|
||||
}
|
||||
|
||||
/// Join `handle` within `timeout`; on expiry `abort()` it and await the aborted
|
||||
/// handle so the serve task (and the `PgPool` clones inside its router/state) is
|
||||
/// reaped BEFORE the caller returns and `#[sqlx::test]` runs `DROP DATABASE`.
|
||||
/// Takes `&mut` (a `JoinHandle` is `Unpin`) so ownership is retained across the
|
||||
/// timeout and the handle can be aborted on expiry — the previous code moved the
|
||||
/// handle into `timeout` and, on elapse, dropped it, which DETACHES the still
|
||||
/// running task rather than aborting it, leaking the test database (#194 F2).
|
||||
async fn join_or_abort(
|
||||
handle: &mut tokio::task::JoinHandle<std::io::Result<()>>,
|
||||
timeout: Duration,
|
||||
) -> ServeTeardown {
|
||||
match tokio::time::timeout(timeout, &mut *handle).await {
|
||||
// A panic in the serve task is carried, not unwound here: the caller
|
||||
// must close the pool and remove the repos dir before resuming it.
|
||||
Ok(join_result) => match join_result {
|
||||
Ok(result) => ServeTeardown::Graceful(result),
|
||||
Err(e) => ServeTeardown::Panicked(e.into_panic()),
|
||||
},
|
||||
Err(_elapsed) => {
|
||||
handle.abort();
|
||||
match handle.await {
|
||||
// The task finished in the race between the timeout firing and
|
||||
// the abort landing — honor its result, not a false wedge.
|
||||
Ok(result) => ServeTeardown::Graceful(result),
|
||||
Err(e) if e.is_cancelled() => ServeTeardown::TimedOutAborted,
|
||||
Err(e) => ServeTeardown::Panicked(e.into_panic()),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Allocate a process-unique temp directory for a spawned node's repositories
|
||||
/// without pulling in the `tempfile` dev-dependency (which is unavailable to
|
||||
/// the library crate under `--features test-harness`).
|
||||
fn unique_repos_dir() -> PathBuf {
|
||||
static COUNTER: AtomicU64 = AtomicU64::new(0);
|
||||
let n = COUNTER.fetch_add(1, Ordering::Relaxed);
|
||||
std::env::temp_dir().join(format!("gitlawb-deny-harness-{}-{}", std::process::id(), n))
|
||||
}
|
||||
|
||||
/// Build an [`AppState`] over the given migrated pool, mirroring
|
||||
/// `test_support::build_state` but with a real on-disk repository directory so
|
||||
/// git smart-HTTP routes can serve. P2P is always disabled.
|
||||
fn build_state(db: Arc<crate::db::Db>, pool: PgPool, repos_dir: PathBuf) -> AppState {
|
||||
let keypair = Keypair::generate();
|
||||
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
|
||||
let webhook_secret_key = crate::state::AppState::derive_webhook_secret_key(&keypair);
|
||||
let task_cursor_key = crate::api::task_cursor::TaskCursorKey::derive(&keypair.to_seed());
|
||||
let node_did = keypair.did();
|
||||
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
|
||||
let (task_tx, _) = tokio::sync::broadcast::channel(1);
|
||||
let schema = Arc::new(crate::graphql::build_schema(
|
||||
db.clone(),
|
||||
ref_tx.clone(),
|
||||
task_tx.clone(),
|
||||
task_cursor_key.clone(),
|
||||
));
|
||||
AppState {
|
||||
config: Arc::new(Config::parse_from(["gitlawb-node"])),
|
||||
db,
|
||||
node_did,
|
||||
node_keypair: Arc::new(keypair),
|
||||
p2p: None,
|
||||
http_client: Arc::new(reqwest::Client::new()),
|
||||
ref_update_tx: ref_tx,
|
||||
task_event_tx: task_tx,
|
||||
graphql_schema: schema,
|
||||
task_cursor_key,
|
||||
machine_id: None,
|
||||
repo_store: crate::git::repo_store::RepoStore::for_testing(repos_dir, pool),
|
||||
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
|
||||
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
|
||||
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
|
||||
close_issue_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
|
||||
federated_rate_limiter: RateLimiter::new(12, Duration::from_secs(60)),
|
||||
task_read_rate_limiter: RateLimiter::new(1200, Duration::from_secs(3600)),
|
||||
ipfs_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
|
||||
ipfs_work_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
|
||||
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
|
||||
ipfs_max_legacy_probes: crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST,
|
||||
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
|
||||
ipfs_max_legacy_scan_rows: crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
|
||||
ipfs_max_legacy_scan_rule_bytes: crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
|
||||
ipfs_scan_token_key: Arc::new(scan_token_key),
|
||||
webhook_secret_key: Arc::new(webhook_secret_key),
|
||||
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
|
||||
push_limiter_trust: TrustedProxy::None,
|
||||
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
|
||||
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
|
||||
shutdown_tx: watch::channel(false).0,
|
||||
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
git_push_advert_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
encrypt_inflight: crate::state::EncryptInflight::new(),
|
||||
repo_write_leases: crate::state::RepoWriteLeases::new(8),
|
||||
git_read_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
|
||||
git_push_advert_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(
|
||||
8,
|
||||
),
|
||||
git_write_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
|
||||
git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
|
||||
git_ipfs_walk_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(
|
||||
16,
|
||||
),
|
||||
git_bin: "git".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Spawn a real node bound to `127.0.0.1:0` over the given (already-created,
|
||||
/// empty) test pool. Runs the schema migrations, builds the router, and serves
|
||||
/// it on a background task through the production `axum::serve` stack with
|
||||
/// connect-info so per-IP layers key on the real peer. Returns once the socket
|
||||
/// is bound and accepting connections.
|
||||
pub async fn spawn_node(pool: PgPool) -> TestNode {
|
||||
let db = Arc::new(crate::db::Db::for_testing(pool.clone()));
|
||||
db.run_migrations()
|
||||
.await
|
||||
.expect("test schema migrations should apply");
|
||||
|
||||
let repos_dir = unique_repos_dir();
|
||||
std::fs::create_dir_all(&repos_dir).expect("create temp repos dir");
|
||||
|
||||
let state = build_state(db.clone(), pool.clone(), repos_dir.clone());
|
||||
let node_did = state.node_did.to_string();
|
||||
let shutdown_tx = state.shutdown_tx.clone();
|
||||
let mut shutdown_rx = shutdown_tx.subscribe();
|
||||
|
||||
let router = crate::server::build_router(state);
|
||||
|
||||
let listener = TcpListener::bind(("127.0.0.1", 0))
|
||||
.await
|
||||
.expect("bind ephemeral port");
|
||||
let addr = listener.local_addr().expect("read bound addr");
|
||||
|
||||
let server = tokio::spawn(async move {
|
||||
axum::serve(
|
||||
listener,
|
||||
router.into_make_service_with_connect_info::<SocketAddr>(),
|
||||
)
|
||||
.with_graceful_shutdown(async move {
|
||||
let _ = shutdown_rx.changed().await;
|
||||
})
|
||||
.await
|
||||
});
|
||||
|
||||
TestNode {
|
||||
base_url: format!("http://{addr}"),
|
||||
node_did,
|
||||
shutdown_tx,
|
||||
repos_dir,
|
||||
db,
|
||||
server: Some(server),
|
||||
pool,
|
||||
suppress_graceful_shutdown_on_drop: false,
|
||||
}
|
||||
}
|
||||
|
||||
impl TestNode {
|
||||
/// Test-only: make `Drop` tear the serve task down through `handle.abort()`
|
||||
/// alone by suppressing the graceful watch signal. Used to prove the abort
|
||||
/// is load-bearing (that the database is released even when the graceful
|
||||
/// chain is broken); no production or normal-teardown path calls this.
|
||||
pub fn force_abort_only_teardown(&mut self) {
|
||||
self.suppress_graceful_shutdown_on_drop = true;
|
||||
}
|
||||
|
||||
/// Graceful async teardown: signal shutdown, join the serve task, close
|
||||
/// the pool, remove the temp repos dir. Call this at the end of every
|
||||
/// test. `#[sqlx::test]` issues `DROP DATABASE` as soon as the test
|
||||
/// future returns, so the server (which owns pool clones inside
|
||||
/// `AppState`/`Db`) must be gone and the pool closed before then, or the
|
||||
/// cleanup fails and leaks per-test databases. A serve task that does not
|
||||
/// exit within 10s panics loudly: a wedged graceful shutdown must fail
|
||||
/// the test, not hang CI.
|
||||
///
|
||||
/// When `self` drops on return, `Drop` re-sends the signal and re-removes
|
||||
/// the dir; both are idempotent, and the serve handle was already taken,
|
||||
/// so the overlap is harmless.
|
||||
pub async fn shutdown(self) {
|
||||
self.shutdown_with_timeout(Duration::from_secs(10)).await;
|
||||
}
|
||||
|
||||
/// Body of [`Self::shutdown`] with an injectable teardown timeout so a test
|
||||
/// can drive the wedged-shutdown path without waiting the production 10s.
|
||||
/// Production always calls this with 10s.
|
||||
async fn shutdown_with_timeout(mut self, timeout: Duration) {
|
||||
let _ = self.shutdown_tx.send(true);
|
||||
let outcome = match self.server.take() {
|
||||
Some(mut handle) => Some(join_or_abort(&mut handle, timeout).await),
|
||||
None => None,
|
||||
};
|
||||
// Release the pool and temp dir on EVERY path before returning OR
|
||||
// panicking. A wedged graceful shutdown is exactly when in-flight
|
||||
// per-connection tasks still hold `PgPool` clones; aborting the serve
|
||||
// future alone does not reliably drop them, so the explicit
|
||||
// `pool.close()` (which closes the shared inner) is what frees them
|
||||
// before `#[sqlx::test]` runs `DROP DATABASE`. Panicking before this
|
||||
// (the previous behavior) skipped the close on the one path that needs
|
||||
// it most (#194 F2).
|
||||
self.pool.close().await;
|
||||
let _ = std::fs::remove_dir_all(&self.repos_dir);
|
||||
match outcome {
|
||||
// Honest coverage contract: on axum 0.8.8,
|
||||
// `WithGracefulShutdown::into_future` always resolves `Ok(())` (the
|
||||
// shutdown signal is treated as success and `accept()` errors are
|
||||
// retried forever rather than returned), so this `Err` arm is
|
||||
// unreachable today - every existing harness test exercises only the
|
||||
// `Ok` path here. Forward-compat insurance for a future axum where
|
||||
// `serve` can fail: that should surface as a clear panic, not be
|
||||
// silently discarded.
|
||||
Some(ServeTeardown::Graceful(result)) => {
|
||||
result.expect("axum::serve exited with an error");
|
||||
}
|
||||
Some(ServeTeardown::TimedOutAborted) => {
|
||||
panic!(
|
||||
"serve task did not exit within {timeout:?} of the shutdown \
|
||||
signal (wedged graceful shutdown)"
|
||||
);
|
||||
}
|
||||
Some(ServeTeardown::Panicked(payload)) => std::panic::resume_unwind(payload),
|
||||
None => {}
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert a repo owned by `owner_did` and return its repo id. Mirrors
|
||||
/// `test_support::seed_repo` (the `disk_path` field is unused by the
|
||||
/// integration path — `RepoStore` computes the on-disk path from
|
||||
/// `repos_dir`/owner/name, see [`Self::seed_bare_repo`]).
|
||||
pub async fn seed_repo(&self, owner_did: &str, name: &str, is_public: bool) -> String {
|
||||
let now = Utc::now();
|
||||
let record = RepoRecord {
|
||||
id: Uuid::new_v4().to_string(),
|
||||
name: name.to_string(),
|
||||
owner_did: owner_did.to_string(),
|
||||
description: None,
|
||||
is_public,
|
||||
default_branch: "main".to_string(),
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
disk_path: format!("/tmp/{name}"),
|
||||
forked_from: None,
|
||||
machine_id: None,
|
||||
};
|
||||
self.db.create_repo(&record).await.expect("seed repo");
|
||||
record.id
|
||||
}
|
||||
|
||||
/// Mark `branch` as protected on `repo_id`, so a non-owner push to it is
|
||||
/// rejected by the branch-protection gate (used by the deny-prober's
|
||||
/// protected-push probe — #195, F3).
|
||||
pub async fn seed_protected_branch(&self, repo_id: &str, branch: &str, created_by: &str) {
|
||||
self.db
|
||||
.protect_branch(repo_id, branch, created_by)
|
||||
.await
|
||||
.expect("seed protected branch");
|
||||
}
|
||||
|
||||
/// Seed an open PR `number` in `repo_id`, authored by `author_did`. The
|
||||
/// author-or-owner close gate (`close_pr`) loads the PR *before* it runs, so a
|
||||
/// deny-prober row for `.../pulls/{number}/close` needs the PR to exist or a
|
||||
/// stranger gets a 404 (absent entity) instead of the 403 the gate emits.
|
||||
pub async fn seed_pr(&self, repo_id: &str, number: i64, author_did: &str) {
|
||||
let now = Utc::now().to_rfc3339();
|
||||
let pr = PullRequest {
|
||||
id: Uuid::new_v4().to_string(),
|
||||
repo_id: repo_id.to_string(),
|
||||
number,
|
||||
title: "seed pr".to_string(),
|
||||
body: None,
|
||||
author_did: author_did.to_string(),
|
||||
source_branch: "feature".to_string(),
|
||||
target_branch: "main".to_string(),
|
||||
status: "open".to_string(),
|
||||
merged_by_did: None,
|
||||
merged_at: None,
|
||||
created_at: now.clone(),
|
||||
updated_at: now,
|
||||
};
|
||||
self.db.create_pr(&pr).await.expect("seed pr");
|
||||
}
|
||||
|
||||
/// Seed issue `issue_id` (a git-ref JSON blob authored by `author_did`) in the
|
||||
/// repo's bare on-disk store. Like [`Self::seed_pr`], the `close_issue`
|
||||
/// owner-or-author gate loads the issue before it runs, so the deny-prober row
|
||||
/// for `.../issues/{id}/close` needs the issue present to reach the 403.
|
||||
pub fn seed_issue(&self, owner_did: &str, name: &str, issue_id: &str, author_did: &str) {
|
||||
let slug = owner_did.replace([':', '/'], "_");
|
||||
let bare = self.repos_dir.join(&slug).join(format!("{name}.git"));
|
||||
let json = format!(r#"{{"author":"{author_did}"}}"#);
|
||||
crate::git::issues::create_issue(&bare, issue_id, &json).expect("seed issue");
|
||||
}
|
||||
|
||||
/// Add a path-scoped visibility rule restricting `path_glob` to
|
||||
/// `reader_dids` (empty = only the owner). Mode B keeps object SHAs intact
|
||||
/// and withholds blob content, which is what the read/replication gates
|
||||
/// enforce.
|
||||
pub async fn withhold_path(
|
||||
&self,
|
||||
repo_id: &str,
|
||||
path_glob: &str,
|
||||
reader_dids: &[String],
|
||||
created_by: &str,
|
||||
) {
|
||||
self.db
|
||||
.set_visibility_rule(
|
||||
repo_id,
|
||||
path_glob,
|
||||
VisibilityMode::B,
|
||||
reader_dids,
|
||||
created_by,
|
||||
)
|
||||
.await
|
||||
.expect("set visibility rule");
|
||||
}
|
||||
|
||||
/// Record `pinned_cids` rows for every blob OID returned by
|
||||
/// [`Self::seed_bare_repo`] so `/ipfs/{cid}` can resolve via provenance on
|
||||
/// current main (the table is the primary lookup path). CIDs are computed
|
||||
/// from raw object bytes exactly as the production pin path does
|
||||
/// (`Cid::from_git_object_bytes`), not from the git oid hash.
|
||||
/// Returns each seeded path's canonical CID for use in `GET /ipfs/{cid}`.
|
||||
pub async fn seed_pinned_cids_for_repo(
|
||||
&self,
|
||||
owner_did: &str,
|
||||
repo_name: &str,
|
||||
repo_id: &str,
|
||||
oids: &std::collections::HashMap<String, String>,
|
||||
) -> std::collections::HashMap<String, String> {
|
||||
let slug = owner_did.replace([':', '/'], "_");
|
||||
let bare = self.repos_dir.join(&slug).join(format!("{repo_name}.git"));
|
||||
let mut cids = std::collections::HashMap::new();
|
||||
for (path, oid_hex) in oids {
|
||||
if path == "HEAD" {
|
||||
continue;
|
||||
}
|
||||
let (_ty, raw) = crate::git::store::read_object(&bare, oid_hex)
|
||||
.expect("read seeded object")
|
||||
.expect("seeded object exists in bare repo");
|
||||
let preliminary = gitlawb_core::cid::Cid::from_git_object_bytes(&raw).to_string();
|
||||
let cid = preliminary
|
||||
.parse::<cid::CidGeneric<64>>()
|
||||
.expect("parse production pin cid")
|
||||
.to_string();
|
||||
self.db
|
||||
.record_pinned_cid_with_source(oid_hex, &cid, repo_id)
|
||||
.await
|
||||
.expect("seed pinned_cids row");
|
||||
let roundtrip = self
|
||||
.db
|
||||
.oids_for_cid(&cid)
|
||||
.await
|
||||
.expect("oids_for_cid after seed");
|
||||
assert!(
|
||||
!roundtrip.is_empty(),
|
||||
"pinned_cids seed for {path} did not round-trip cid {cid}"
|
||||
);
|
||||
cids.insert(path.clone(), cid);
|
||||
}
|
||||
cids
|
||||
}
|
||||
|
||||
/// Create a real bare git repository on disk at the exact path
|
||||
/// `RepoStore::acquire` reads (`<repos_dir>/<owner-slug>/<name>.git`), with
|
||||
/// one commit on `main` containing `files` (`(path, contents)`). Returns the
|
||||
/// blob OID of each path so callers can assert a withheld OID never appears
|
||||
/// in served bytes (U8). Shells out to `git`, mirroring
|
||||
/// `test_support`'s served-content seam.
|
||||
/// `object_format` is `"sha1"` for the git smart-HTTP surfaces and
|
||||
/// `"sha256"` for the content-addressed `/ipfs/{cid}` surface (whose CIDs
|
||||
/// are the sha2-256 object ids).
|
||||
pub fn seed_bare_repo(
|
||||
&self,
|
||||
owner_did: &str,
|
||||
name: &str,
|
||||
files: &[(&str, &str)],
|
||||
object_format: &str,
|
||||
) -> std::collections::HashMap<String, String> {
|
||||
let run = |args: &[&str], cwd: &std::path::Path| {
|
||||
// Test-harness-only fixture seeding, feature-gated out of the production
|
||||
// binary; runs local git against a tempdir inside a test's own lifetime.
|
||||
// allow-unbounded-git: test-only seeding, never on a request path holding a permit
|
||||
let out = std::process::Command::new("git")
|
||||
.args(args)
|
||||
.current_dir(cwd)
|
||||
.output() // allow-unbounded-git: test-harness fixture seeding only
|
||||
.expect("git runs");
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"git {args:?} failed: {}",
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
String::from_utf8_lossy(&out.stdout).trim().to_string()
|
||||
};
|
||||
|
||||
// Build a source work tree, then bare-clone it into the served path.
|
||||
let src = self.repos_dir.join(format!("src-{name}"));
|
||||
std::fs::create_dir_all(&src).expect("create src dir");
|
||||
let fmt_arg = format!("--object-format={object_format}");
|
||||
run(&["init", "-q", "-b", "main", &fmt_arg], &src);
|
||||
run(&["config", "user.email", "t@t"], &src);
|
||||
run(&["config", "user.name", "t"], &src);
|
||||
for (path, contents) in files {
|
||||
let full = src.join(path);
|
||||
if let Some(parent) = full.parent() {
|
||||
std::fs::create_dir_all(parent).expect("create file parent dir");
|
||||
}
|
||||
std::fs::write(&full, contents).expect("write seed file");
|
||||
run(&["add", path], &src);
|
||||
}
|
||||
run(&["commit", "-q", "-m", "seed"], &src);
|
||||
|
||||
let mut oids = std::collections::HashMap::new();
|
||||
for (path, _) in files {
|
||||
let oid = run(&["rev-parse", &format!("HEAD:{path}")], &src);
|
||||
oids.insert((*path).to_string(), oid);
|
||||
}
|
||||
// The HEAD commit oid, used as the `want` when driving upload-pack.
|
||||
oids.insert("HEAD".to_string(), run(&["rev-parse", "HEAD"], &src));
|
||||
|
||||
let slug = owner_did.replace([':', '/'], "_");
|
||||
let bare = self.repos_dir.join(&slug).join(format!("{name}.git"));
|
||||
std::fs::create_dir_all(bare.parent().unwrap()).expect("create bare parent");
|
||||
run(
|
||||
&[
|
||||
"clone",
|
||||
"--bare",
|
||||
"-q",
|
||||
src.to_str().unwrap(),
|
||||
bare.to_str().unwrap(),
|
||||
],
|
||||
&self.repos_dir,
|
||||
);
|
||||
|
||||
oids
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{join_or_abort, ServeTeardown};
|
||||
use std::time::Duration;
|
||||
|
||||
/// A wedged serve task (never returns within the window) must be ABORTED and
|
||||
/// reaped on timeout, not detached. The previous teardown moved the
|
||||
/// `JoinHandle` into `timeout` and dropped it on elapse, which detaches the
|
||||
/// still-running task and leaks the test DB across `DROP DATABASE` (#194 F2).
|
||||
/// Load-bearing: neutralize the abort in `join_or_abort` (return without
|
||||
/// abort+await) and this flips RED — `handle.is_finished()` stays false
|
||||
/// because the task was left running.
|
||||
#[tokio::test]
|
||||
async fn join_or_abort_aborts_a_wedged_task_on_timeout() {
|
||||
let mut handle: tokio::task::JoinHandle<std::io::Result<()>> = tokio::spawn(async {
|
||||
loop {
|
||||
tokio::time::sleep(Duration::from_secs(3600)).await;
|
||||
}
|
||||
});
|
||||
assert!(
|
||||
matches!(
|
||||
join_or_abort(&mut handle, Duration::from_millis(50)).await,
|
||||
ServeTeardown::TimedOutAborted
|
||||
),
|
||||
"a task that never returns must be reported timed-out, not graceful"
|
||||
);
|
||||
assert!(
|
||||
handle.is_finished(),
|
||||
"the wedged serve task must be aborted and reaped, not detached"
|
||||
);
|
||||
}
|
||||
|
||||
/// A task that returns within the window is reported graceful, carrying its
|
||||
/// `io::Result` — the normal teardown path.
|
||||
#[tokio::test]
|
||||
async fn join_or_abort_reports_graceful_when_task_exits_in_time() {
|
||||
let mut handle: tokio::task::JoinHandle<std::io::Result<()>> =
|
||||
tokio::spawn(async { Ok(()) });
|
||||
assert!(
|
||||
matches!(
|
||||
join_or_abort(&mut handle, Duration::from_secs(5)).await,
|
||||
ServeTeardown::Graceful(Ok(()))
|
||||
),
|
||||
"a prompt clean exit must be reported graceful"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -54,7 +54,6 @@ pub(crate) async fn test_state(pool: PgPool) -> AppState {
|
||||
/// `config::tests::enforce_owner_push_is_declared_true_independent_of_the_environment`,
|
||||
/// which reads the declaration off the parser. Splitting the two keeps a parser
|
||||
/// question and an authorization question from sharing one failure mode.
|
||||
#[allow(dead_code)]
|
||||
pub(crate) async fn test_state_with(
|
||||
pool: PgPool,
|
||||
configure: impl FnOnce(&mut crate::config::Config),
|
||||
@@ -164,7 +163,6 @@ pub(crate) async fn app(pool: PgPool) -> Router {
|
||||
/// Sets `Content-Type: application/json` — the API is JSON throughout, and
|
||||
/// without it axum's `Json` extractor returns 415 before the handler runs
|
||||
/// (which would make any JSON-body authz assertion a false pass).
|
||||
#[allow(dead_code)]
|
||||
pub(crate) fn signed_request_as(did: &str, method: Method, uri: &str, body: Body) -> Request<Body> {
|
||||
Request::builder()
|
||||
.method(method)
|
||||
@@ -182,7 +180,6 @@ pub(crate) fn signed_request_as(did: &str, method: Method, uri: &str, body: Body
|
||||
/// test pass for the wrong reason.) The accepted sockets are parked in the
|
||||
/// spawned task, which dies with the test's runtime, so the peer never sees a
|
||||
/// close mid-test.
|
||||
#[allow(dead_code)]
|
||||
pub(crate) async fn silent_http_endpoint() -> String {
|
||||
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let endpoint = format!("http://{}", listener.local_addr().unwrap());
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -517,7 +517,7 @@ fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
|
||||
/// drives the actual disconnect through a parked `release`. Same instrument the F3
|
||||
/// gate above uses.
|
||||
///
|
||||
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
|
||||
/// MUTATION (RED): move the `post_receive_replication_tail(` call below
|
||||
/// `guard.release(` and the ordering assertion fails; take it out of the
|
||||
/// `if disposition.spawn_tail` block and the failed-push assertion fails; gate
|
||||
/// `spawn_tail`/`release_ok` on the outcome commit and
|
||||
@@ -542,17 +542,21 @@ fn inv22_replication_tail_spawns_at_the_durability_boundary() {
|
||||
// position (carve-out early, or durability-bound at release) must sit
|
||||
// inside a `disposition.spawn_tail` gate and feed `landed_refs` —
|
||||
// never the unfiltered `ref_updates`.
|
||||
let spawn_count = production
|
||||
.matches("tokio::spawn(post_receive_replication_tail(")
|
||||
.count();
|
||||
// Match the tail CALL, not the `tokio::spawn(...)` wrapper: production
|
||||
// currently wraps it in `tokio::spawn(async move { ... })`, and keying on
|
||||
// the wrapper's exact shape is what let this gate go blind on the refactor.
|
||||
// The `pub(crate) async fn` definition is the only other text occurrence,
|
||||
// filtered by the `fn ` that must precede it.
|
||||
let spawn_positions: Vec<usize> = production
|
||||
.match_indices("post_receive_replication_tail(")
|
||||
.map(|(p, _)| p)
|
||||
.filter(|&p| !production[..p].trim_end().ends_with("fn"))
|
||||
.collect();
|
||||
assert!(
|
||||
spawn_count >= 1,
|
||||
!spawn_positions.is_empty(),
|
||||
"U5 gate missing: the replication tail must be spawned by git_receive_pack"
|
||||
);
|
||||
for (i, (pos, _)) in production
|
||||
.match_indices("tokio::spawn(post_receive_replication_tail(")
|
||||
.enumerate()
|
||||
{
|
||||
for (i, pos) in spawn_positions.iter().copied().enumerate() {
|
||||
let gate = production[..pos].rfind("if disposition.spawn_tail").expect(
|
||||
"U5 gate missing: every tail spawn must be gated on `if disposition.spawn_tail`",
|
||||
);
|
||||
@@ -578,13 +582,15 @@ fn inv22_replication_tail_spawns_at_the_durability_boundary() {
|
||||
);
|
||||
let call_end = production[pos..].find(");").unwrap() + pos;
|
||||
let call = &production[pos..call_end];
|
||||
// `landed` is the in-scope alias for `landed_refs` in the carve-out
|
||||
// spawn; either name is the proven-landed set, `ref_updates` is not.
|
||||
assert!(
|
||||
call.contains("landed_refs"),
|
||||
call.contains("landed"),
|
||||
"U5 gate bypassed: tail #{i} must carry the proven-landed ref set, not unfiltered ref_updates"
|
||||
);
|
||||
}
|
||||
let spawn = production
|
||||
.find("tokio::spawn(post_receive_replication_tail(")
|
||||
let spawn = *spawn_positions
|
||||
.first()
|
||||
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
|
||||
let release = production
|
||||
.find(".release(disposition.release_ok)")
|
||||
@@ -715,11 +721,11 @@ fn inv26_step3_live_and_drain_share_apply_request_effects() {
|
||||
}
|
||||
|
||||
/// #26 Split PR 1 step 4 — the periodic queue-lifecycle purge is
|
||||
/// wired in `lib.rs` and the contract is enforced by the `idx_receive_pack_requests_completed_at`
|
||||
/// wired in `main.rs` and the contract is enforced by the `idx_receive_pack_requests_completed_at`
|
||||
/// partial index from v41.
|
||||
///
|
||||
/// Assertions:
|
||||
/// 1. `lib.rs` calls `purge_request_queue` once at boot (well, on
|
||||
/// 1. `main.rs` calls `purge_request_queue` once at boot (well, on
|
||||
/// the spawn-task interval) and the spawn function exists.
|
||||
/// 2. The DB helpers `purge_completed_receive_pack_requests` and
|
||||
/// `purge_completed_pending_ref_transitions` exist in `db/mod.rs`.
|
||||
@@ -729,14 +735,14 @@ fn inv26_step3_live_and_drain_share_apply_request_effects() {
|
||||
/// introduce the state, but the invariant holds for the future).
|
||||
#[test]
|
||||
fn inv26_step4_queue_lifecycle_purge_is_wired() {
|
||||
let boot_src = src("lib.rs");
|
||||
let main_src = src("main.rs");
|
||||
assert!(
|
||||
boot_src.contains("spawn_queue_lifecycle_sweep"),
|
||||
"lib.rs must spawn the periodic queue-lifecycle purge"
|
||||
main_src.contains("spawn_queue_lifecycle_sweep"),
|
||||
"main.rs must spawn the periodic queue-lifecycle purge"
|
||||
);
|
||||
assert!(
|
||||
boot_src.contains("purge_request_queue"),
|
||||
"lib.rs must call purge_request_queue on the periodic sweep"
|
||||
main_src.contains("purge_request_queue"),
|
||||
"main.rs must call purge_request_queue on the periodic sweep"
|
||||
);
|
||||
|
||||
let db_src = src("db/mod.rs");
|
||||
|
||||
@@ -1,235 +0,0 @@
|
||||
//! INV-8 deny assertion. A denial must be an actual refusal (4xx) whose body
|
||||
//! and headers carry none of the data it is withholding, and must never be a 2xx
|
||||
//! (an empty-200 rendered as success is the denial-as-success bug this guards).
|
||||
//!
|
||||
//! The check is split into a pure `check_denied` (unit-tested below with the
|
||||
//! self-check scenarios) and an async `assert_denied` wrapper that reads a real
|
||||
//! `reqwest::Response`.
|
||||
|
||||
/// Pure deny check. Returns `Err(reason)` when the response is not a clean
|
||||
/// denial. `withheld` are exact tokens (OIDs, paths, private slugs) that must
|
||||
/// not appear anywhere in the body; register short-OID prefixes and encoded
|
||||
/// path forms as separate tokens at the call site (a raw substring scan cannot
|
||||
/// normalize them).
|
||||
pub fn check_denied(
|
||||
status: u16,
|
||||
body: &str,
|
||||
expected: u16,
|
||||
withheld: &[&str],
|
||||
) -> Result<(), String> {
|
||||
// The harness only ever expects denials; a non-4xx expectation is a
|
||||
// programming error in the test, not a node behavior under test.
|
||||
if !(400..500).contains(&expected) {
|
||||
return Err(format!(
|
||||
"test bug: assert_denied expected status {expected} is not a 4xx"
|
||||
));
|
||||
}
|
||||
// INV-8: a denial rendered as success is the headline bug. Catch it
|
||||
// explicitly so the failure message names it, rather than only failing the
|
||||
// status-equality check below.
|
||||
if (200..300).contains(&status) {
|
||||
return Err(format!(
|
||||
"denial rendered as success: got {status}, expected denial {expected}. \
|
||||
body={body:?}"
|
||||
));
|
||||
}
|
||||
if status != expected {
|
||||
return Err(format!(
|
||||
"expected denial status {expected}, got {status}. body={body:?}"
|
||||
));
|
||||
}
|
||||
for token in withheld {
|
||||
if token.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if body.contains(token) {
|
||||
return Err(format!(
|
||||
"withheld token {token:?} leaked in denial body: {body:?}"
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Like [`check_denied`] but also scans serialized response headers for withheld
|
||||
/// tokens. Call sites that only have a body should keep using `check_denied`.
|
||||
pub fn check_denied_with_headers(
|
||||
status: u16,
|
||||
body: &str,
|
||||
header_text: &str,
|
||||
expected: u16,
|
||||
withheld: &[&str],
|
||||
) -> Result<(), String> {
|
||||
check_denied(status, body, expected, withheld)?;
|
||||
for token in withheld {
|
||||
if token.is_empty() {
|
||||
continue;
|
||||
}
|
||||
if header_text.contains(token) {
|
||||
return Err(format!(
|
||||
"withheld token {token:?} leaked in denial headers: {header_text:?}"
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Replace each withheld token in `s` with `[REDACTED]` so a test-failure
|
||||
/// witness does not itself leak the secret it caught.
|
||||
fn redact_withheld(s: &str, withheld: &[&str]) -> String {
|
||||
let mut result = s.to_string();
|
||||
for token in withheld {
|
||||
if !token.is_empty() {
|
||||
result = result.replace(token, "[REDACTED]");
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
/// Drive a real response through the deny check and panic on failure. Reads the
|
||||
/// full body once.
|
||||
pub async fn assert_denied(resp: reqwest::Response, expected: u16, withheld: &[&str]) {
|
||||
let status = resp.status().as_u16();
|
||||
let header_text = resp
|
||||
.headers()
|
||||
.iter()
|
||||
.filter_map(|(name, value)| {
|
||||
value
|
||||
.to_str()
|
||||
.ok()
|
||||
.map(|v| format!("{}: {}", name.as_str(), v))
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
// A body-read failure must NOT fold to "" — an empty body passes the withheld
|
||||
// scan vacuously, so an unreadable denial (a mid-body reset/truncation) would be
|
||||
// certified leak-free without ever inspecting the bytes. Fail loud instead. A
|
||||
// legitimately empty body is `Ok("")`, not `Err`, so the honest empty-401 denials
|
||||
// still pass.
|
||||
let body = resp
|
||||
.text()
|
||||
.await
|
||||
.expect("read denial body: an unreadable denial cannot be certified leak-free");
|
||||
if let Err(reason) = check_denied_with_headers(status, &body, &header_text, expected, withheld)
|
||||
{
|
||||
// Panic with a fixed string: CodeQL's taint analysis traces withheld
|
||||
// tokens from check_denied through redact_withheld into the panic sink,
|
||||
// even though redact_withheld strips them at runtime. A fixed message
|
||||
// keeps no tainted data on the sink path. The test name identifies
|
||||
// which probe failed; check_denied's unit tests cover the error detail.
|
||||
let _ = redact_withheld(&reason, withheld);
|
||||
panic!("deny assertion failed: status/body/header leak check");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{check_denied, check_denied_with_headers, redact_withheld};
|
||||
|
||||
#[test]
|
||||
fn clean_403_with_no_leak_passes() {
|
||||
let r = check_denied(
|
||||
403,
|
||||
r#"{"error":"only the repo owner can do this"}"#,
|
||||
403,
|
||||
&["a1b2c3"],
|
||||
);
|
||||
assert!(r.is_ok(), "{r:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_200_is_flagged_as_denial_rendered_as_success() {
|
||||
let r = check_denied(200, "", 403, &[]);
|
||||
assert!(r.is_err());
|
||||
assert!(r.unwrap_err().contains("rendered as success"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn leaking_403_fails_and_names_the_token() {
|
||||
let withheld = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
|
||||
let body = format!("not found near object {withheld}");
|
||||
let r = check_denied(403, &body, 403, &[withheld]);
|
||||
assert!(r.is_err());
|
||||
assert!(r.unwrap_err().contains(withheld));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn wrong_status_fails() {
|
||||
// A 404 when the owner gate's 403 was expected must not pass: it means
|
||||
// an earlier layer denied, so the gate under test was never exercised.
|
||||
let r = check_denied(404, "not found", 403, &[]);
|
||||
assert!(r.is_err());
|
||||
assert!(r.unwrap_err().contains("expected denial status 403"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_4xx_expected_is_a_test_bug() {
|
||||
// Guards the harness itself: expecting a non-denial status is a test
|
||||
// authoring error, not a node behavior, and must be rejected.
|
||||
let r = check_denied(403, "denied", 200, &[]);
|
||||
assert!(r.is_err());
|
||||
assert!(r.unwrap_err().contains("not a 4xx"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_withheld_token_never_false_positives() {
|
||||
// An empty token must be skipped, never treated as "contained in every
|
||||
// body" — otherwise every denial would spuriously fail as a leak.
|
||||
let r = check_denied(403, "any body", 403, &[""]);
|
||||
assert!(r.is_ok(), "{r:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn leaking_header_fails_and_names_the_token() {
|
||||
let withheld = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
|
||||
let headers = format!("X-Debug: object {withheld}");
|
||||
let r = check_denied_with_headers(
|
||||
404,
|
||||
r#"{"error":"repository not found"}"#,
|
||||
&headers,
|
||||
404,
|
||||
&[withheld],
|
||||
);
|
||||
assert!(r.is_err());
|
||||
let err = r.unwrap_err();
|
||||
assert!(err.contains("leaked in denial headers"));
|
||||
assert!(err.contains(withheld));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clean_headers_with_clean_body_passes() {
|
||||
let withheld = "a1b2c3";
|
||||
let r = check_denied_with_headers(
|
||||
404,
|
||||
r#"{"error":"repository not found"}"#,
|
||||
"Content-Type: application/json",
|
||||
404,
|
||||
&[withheld],
|
||||
);
|
||||
assert!(r.is_ok(), "{r:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn redact_withheld_replaces_secrets() {
|
||||
let withheld = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
|
||||
let msg = format!("withheld token {withheld:?} leaked in body: near {withheld}");
|
||||
let redacted = redact_withheld(&msg, &[withheld]);
|
||||
assert!(
|
||||
!redacted.contains(withheld),
|
||||
"token must be redacted: {redacted}"
|
||||
);
|
||||
assert!(
|
||||
redacted.contains("[REDACTED]"),
|
||||
"redaction marker must appear: {redacted}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn redact_withheld_leaves_clean_strings() {
|
||||
let result = redact_withheld(
|
||||
"no secrets here",
|
||||
&["deadbeefdeadbeefdeadbeefdeadbeefdeadbeef"],
|
||||
);
|
||||
assert_eq!(result, "no secrets here");
|
||||
}
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
//! Shared support for the real-node deny harness: the deny assertion (U4), the
|
||||
//! RFC-9421 signing client (U2), and (via `gitlawb_node::test_harness`) the
|
||||
//! node spawner (U3).
|
||||
|
||||
pub mod assert;
|
||||
// U2 fixture state matrix + per-gate-class probe generators; driven by U3.
|
||||
#[allow(dead_code)]
|
||||
pub mod probe;
|
||||
// U1 deny-bearing route registry; fields consumed by U2/U3 as they land.
|
||||
#[allow(dead_code)]
|
||||
pub mod routes;
|
||||
pub mod signing;
|
||||
|
||||
/// A reqwest client with a bounded request timeout, so a wedged git subprocess or
|
||||
/// route fails the suite rather than hanging it until CI kills the job (#195, F1).
|
||||
/// The two real-node probes that drive git subprocesses use this; the inline
|
||||
/// twins at the upload-pack and registry-sweep sites use the same pattern.
|
||||
#[allow(dead_code)]
|
||||
pub fn bounded_client() -> reqwest::Client {
|
||||
reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(30))
|
||||
.build()
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
/// Build an anonymous (unsigned) GET request for `path` under `base_url`.
|
||||
/// Extracted from inline call sites so URL construction goes through a
|
||||
/// function boundary, matching the `signed_request` pattern in `support::signing`.
|
||||
#[allow(dead_code)]
|
||||
pub fn anon_get(client: &reqwest::Client, base_url: &str, path: &str) -> reqwest::RequestBuilder {
|
||||
let url = format!("{base_url}{path}");
|
||||
client.get(url)
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,32 +0,0 @@
|
||||
//! Real RFC-9421 request signing for the deny harness (U2). Wraps
|
||||
//! `gitlawb_core::http_sig::sign_request` (the same entry point the
|
||||
//! git-remote-gitlawb helper uses in production) so a test can present a valid
|
||||
//! signature for any identity, reaching the authenticated-but-wrong-owner path
|
||||
//! that an injected-DID shortcut cannot.
|
||||
|
||||
use gitlawb_core::http_sig::sign_request;
|
||||
use gitlawb_core::identity::Keypair;
|
||||
|
||||
/// Build a reqwest request carrying a valid RFC-9421 signature for `keypair`.
|
||||
///
|
||||
/// `path` is both appended to `base_url` to form the request URL and signed as
|
||||
/// the `@path` component, so the value the server reconstructs and the value
|
||||
/// that was signed are guaranteed identical. `body` is signed via its
|
||||
/// content-digest and sent as the request body.
|
||||
pub fn signed_request(
|
||||
client: &reqwest::Client,
|
||||
method: reqwest::Method,
|
||||
base_url: &str,
|
||||
path: &str,
|
||||
body: Vec<u8>,
|
||||
keypair: &Keypair,
|
||||
) -> reqwest::RequestBuilder {
|
||||
let sig = sign_request(keypair, method.as_str(), path, &body);
|
||||
let url = format!("{base_url}{path}");
|
||||
client
|
||||
.request(method, url)
|
||||
.header("content-digest", sig.content_digest)
|
||||
.header("signature-input", sig.signature_input)
|
||||
.header("signature", sig.signature)
|
||||
.body(body)
|
||||
}
|
||||
+1
-30
@@ -148,18 +148,6 @@ impl NodeClient {
|
||||
self.send_signed("POST", path, body).await
|
||||
}
|
||||
|
||||
/// POST with JSON body + RFC 9421 signing and an optional user-provided
|
||||
/// iCaptcha proof, sent on the initial request.
|
||||
pub async fn post_with_proof(
|
||||
&self,
|
||||
path: &str,
|
||||
body: &[u8],
|
||||
initial_proof: Option<&str>,
|
||||
) -> Result<reqwest::Response> {
|
||||
self.send_signed_with_proof("POST", path, body, initial_proof)
|
||||
.await
|
||||
}
|
||||
|
||||
/// PUT with RFC 9421 signing + transparent iCaptcha solve/retry.
|
||||
pub async fn put(&self, path: &str, body: &[u8]) -> Result<reqwest::Response> {
|
||||
self.send_signed("PUT", path, body).await
|
||||
@@ -174,32 +162,16 @@ impl NodeClient {
|
||||
/// `x-icaptcha-*` headers) solve it and retry the same signed request with
|
||||
/// the proof header, up to [`MAX_ICAPTCHA_RETRIES`]. Emits an actionable
|
||||
/// hint on a 401 "not an agent" (the old-CLI / unregistered failure mode).
|
||||
/// Existing call path: no user-supplied proof on the first request.
|
||||
async fn send_signed(
|
||||
&self,
|
||||
method: &str,
|
||||
path: &str,
|
||||
body: &[u8],
|
||||
) -> Result<reqwest::Response> {
|
||||
self.send_signed_with_proof(method, path, body, None).await
|
||||
}
|
||||
|
||||
/// Sign + send a write. `initial_proof`, if provided, is attached to the
|
||||
/// first request. On a 403 iCaptcha challenge advertised via
|
||||
/// `x-icaptcha-*` headers, solve and retry up to MAX_ICAPTCHA_RETRIES.
|
||||
async fn send_signed_with_proof(
|
||||
&self,
|
||||
method: &str,
|
||||
path: &str,
|
||||
body: &[u8],
|
||||
initial_proof: Option<&str>,
|
||||
) -> Result<reqwest::Response> {
|
||||
let mut proof = initial_proof.map(str::to_owned);
|
||||
let mut proof: Option<String> = None;
|
||||
let mut attempts = 0;
|
||||
|
||||
loop {
|
||||
let resp = self.send_once(method, path, body, proof.as_deref()).await?;
|
||||
|
||||
let status = resp.status();
|
||||
|
||||
if status == reqwest::StatusCode::UNAUTHORIZED
|
||||
@@ -223,7 +195,6 @@ impl NodeClient {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
return Ok(resp);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,11 +22,6 @@ pub struct RegisterArgs {
|
||||
)]
|
||||
pub node: String,
|
||||
|
||||
/// A pre-obtained iCaptcha proof token, forwarded on the register request
|
||||
/// instead of making the client solve a fresh challenge.
|
||||
#[arg(long, env = "GITLAWB_ICAPTCHA_PROOF")]
|
||||
pub icaptcha_proof: Option<String>,
|
||||
|
||||
/// Capabilities to advertise (comma-separated)
|
||||
#[arg(
|
||||
long,
|
||||
@@ -61,7 +56,7 @@ pub async fn run(args: RegisterArgs) -> Result<()> {
|
||||
}))?;
|
||||
|
||||
let resp = client
|
||||
.post_with_proof("/api/register", &body, args.icaptcha_proof.as_deref())
|
||||
.post("/api/register", &body)
|
||||
.await
|
||||
.context("failed to connect to node")?;
|
||||
|
||||
@@ -165,7 +160,6 @@ mod tests {
|
||||
capabilities: vec!["git:push".to_string(), "git:fetch".to_string()],
|
||||
model: None,
|
||||
dir: Some(dir.path().to_path_buf()),
|
||||
icaptcha_proof: None,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
@@ -179,36 +173,6 @@ mod tests {
|
||||
assert_eq!(content["node"].as_str().unwrap(), server.url());
|
||||
}
|
||||
|
||||
/// A supplied `--icaptcha-proof` must ride the register request as the
|
||||
/// `x-icaptcha-proof` header so an enforce-mode node accepts it without a
|
||||
/// fresh solve round-trip.
|
||||
#[tokio::test]
|
||||
async fn register_forwards_icaptcha_proof_header() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
write_identity(&dir);
|
||||
|
||||
let mut server = mockito::Server::new_async().await;
|
||||
let m = server
|
||||
.mock("POST", "/api/register")
|
||||
.match_header("x-icaptcha-proof", "test.supplied.proof")
|
||||
.with_status(200)
|
||||
.with_header("content-type", "application/json")
|
||||
.with_body(r#"{"message":"Welcome","ucan":"eyJhbGci.test.token","trust_score":0.5,"expires":"2026-12-31"}"#)
|
||||
.create_async()
|
||||
.await;
|
||||
|
||||
run(RegisterArgs {
|
||||
node: server.url(),
|
||||
capabilities: vec!["git:push".to_string()],
|
||||
model: None,
|
||||
dir: Some(dir.path().to_path_buf()),
|
||||
icaptcha_proof: Some("test.supplied.proof".to_string()),
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
m.assert();
|
||||
}
|
||||
|
||||
/// `gl register` READS the identity from the parent of `GITLAWB_KEY`, so it has
|
||||
/// to WRITE the bootstrap token there too. Sending it to `~/.gitlawb` instead is
|
||||
/// silent split-brain: registration prints success, and every command that later
|
||||
@@ -237,7 +201,6 @@ mod tests {
|
||||
capabilities: vec!["git:push".to_string()],
|
||||
model: None,
|
||||
dir: None,
|
||||
icaptcha_proof: None,
|
||||
})
|
||||
.await
|
||||
.unwrap();
|
||||
@@ -272,7 +235,6 @@ mod tests {
|
||||
capabilities: vec!["git:push".to_string()],
|
||||
model: None,
|
||||
dir: Some(dir.path().to_path_buf()),
|
||||
icaptcha_proof: None,
|
||||
})
|
||||
.await;
|
||||
|
||||
@@ -294,7 +256,6 @@ mod tests {
|
||||
capabilities: vec![],
|
||||
model: None,
|
||||
dir: Some(dir.path().to_path_buf()),
|
||||
icaptcha_proof: None,
|
||||
})
|
||||
.await;
|
||||
|
||||
|
||||
Reference in new issue
Block a user