Compare commits

..
Author SHA1 Message Date
beardthelion 5da280b6ff fix(node): propagate for-each-ref failures in resolve_issue_id 2026-09-10 23:08:58 -05:00
beardthelion f2b6cebdc1 fix(node): propagate for-each-ref failures in list_issues 2026-09-10 22:56:25 -05:00
beardthelion b5048f749f fix(node): read issue refs via cat-file --batch on stdin, resolve absence by enumeration 2026-09-10 22:00:08 -05:00
beardthelion 326f5e55ac fix(node): bound issue ref names and propagate rev-parse probe failures 2026-09-10 21:30:04 -05:00
beardthelion 4fa108160a fix(node): split absent ref from unreadable object in issue reads
get_issue folded every cat-file failure into Ok(None), so a corrupt or
unreadable object store looked identical to a missing issue. close_issue
then hit that None right after resolve_issue_id had proven the ref
exists and panicked on the expect.

A shared read_issue_blob helper now re-resolves the ref with rev-parse
(which does not read the object) when cat-file fails: unresolved means
genuinely absent (Ok(None)), resolved means a real read failure
(propagated as Err). list_issues applies the same split so a degraded
store errors the listing instead of silently under-reporting, and
close_issue treats a ref deleted between resolve and read as not-found
instead of panicking.

Closes #426.
2026-09-10 19:59:12 -05:00
19 changed files with 1059 additions and 4427 deletions

No files matched your search

+15 -19
View File
@@ -8,9 +8,24 @@
# The audit job stays hard-fail: only the advisories explicitly listed below are
# suppressed, so any NEW advisory still breaks CI. Each entry below has no
# available upstream fix; everything fixable is fixed in Cargo.lock instead.
#
# Tracking: the hickory ignores are reachable accepted risk. A scheduled re-check
# (.github/workflows/audit-schedule.yml) runs without these ignores and fails if
# the lockfile moves off the pinned vulnerable versions while the ignores remain.
[advisories]
ignore = [
# hickory-proto 0.25.2 (pulled by libp2p-dns 0.44 -> hickory-resolver 0.25).
# REACHABLE: the node wraps its QUIC transport in the system DNS resolver
# (crates/gitlawb-node/src/p2p/mod.rs:238), so a malicious DNS response can
# trigger these DoS paths. No fix is available: the latest libp2p-dns on
# crates.io is 0.44 and pins hickory 0.25; the fix requires hickory >=0.26.1.
# REMOVE both once libp2p-dns ships a release using hickory 0.26 (verified
# 2026-06-21: cargo update -p hickory-proto locks 0 packages; still 0.25.2).
# Tracking: https://github.com/Gitlawb/node/issues/76
"RUSTSEC-2026-0118", # hickory NSEC3 closest-encloser unbounded loop
"RUSTSEC-2026-0119", # hickory O(n^2) name-compression CPU exhaustion
# rsa 0.9.10 (Marvin timing sidechannel). Present in Cargo.lock because sqlx
# resolves sqlx-mysql unconditionally, and sqlx-mysql depends on rsa. cargo
# audit scans the lockfile, so it flags rsa even though it is NOT linked into
@@ -20,23 +35,4 @@ ignore = [
# ever built with the `mysql` feature, or any other consumer of rsa enters
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
# crypto-bigint 0.5.5 (secret-dependent branches in operations meant
# to be constant-time). REACHABLE accepted risk, on two paths:
# - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256
# PrivateKeySigner that signs transactions in `gl name`
# (crates/gl/src/name.rs) and the node operator
# (crates/gitlawb-node/src/operator.rs).
# - aws-sigv4 (`sigv4a` feature, crates/gitlawb-node/Cargo.toml) ->
# crypto-bigint 0.5 (directly and via p256 0.13): SigV4a key
# derivation, used only for requests to endpoints that require SigV4a
# (e.g. S3 Multi-Region Access Points).
# Exploiting it needs local timing observation of those signing operations.
# The fix is crypto-bigint >=0.7.6, and no compatible dependent uses it:
# every alloy-signer release (through 2.5.0) requires elliptic-curve ^0.13,
# and aws-sigv4 1.6.4 is the first release on crypto-bigint 0.7 (1.5.x -
# 1.6.3 still require ^0.5.4), but it needs Rust 1.94.1, above our 1.91 MSRV
# (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint
# below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`.
"RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix)
]
+2 -6
View File
@@ -88,12 +88,8 @@ GITLAWB_P2P_PORT=7546
# GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true for isolated dev/test networks.
GITLAWB_BOOTSTRAP_PEERS=
GITLAWB_BOOTSTRAP_DISABLE_SEEDS=false
# Require a peer write to prove its DID on every transport: RFC 9421 signatures
# on the announce/sync routes, and an Ed25519 payload signature on gossip
# ref-update events. Keep false during rolling upgrades so existing live nodes
# can still communicate. Upgrade every gossip-publishing peer to a build that
# signs events before setting this true, or their updates are dropped on
# arrival with no error visible on their side.
# Require signatures on peer announce/sync write routes. Keep false during
# rolling upgrades so existing live nodes can still communicate.
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Require the authenticated pusher to be the repo owner on git-receive-pack.
-100
View File
@@ -1,100 +0,0 @@
name: CodeQL
# Advanced-setup replacement for CodeQL default setup. Default setup analyzes
# every detected language on every PR, including Swift (macos-app/), which
# autobuilds on a macOS runner for ~13 minutes per PR and has hung for 55 on
# main; that single job was the longest check on most PRs. Here a PR analyzes
# only the languages it touches (scripts/ci-surfaces.sh), while push to main
# and the weekly schedule analyze every language, so main's code-scanning
# baseline stays complete.
#
# Requires CodeQL default setup to be DISABLED (Settings -> Code security ->
# Code scanning): GitHub rejects advanced-setup uploads while it is enabled.
#
# A PR that skips a language may show "configuration not found" on the CodeQL
# check for that language; that is informational and expected.
on:
pull_request:
branches: [main]
push:
branches: [main]
schedule:
- cron: "17 4 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: codeql-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
changes:
name: changed languages
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
codeql: ${{ steps.surfaces.outputs.codeql }}
matrix: ${{ steps.surfaces.outputs.codeql_matrix }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
fetch-depth: 2
- name: Map changed paths to languages
id: surfaces
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "pull_request" ]; then
# -z: literal paths; plain --name-only C-quotes unusual names.
git diff -z --name-only --no-renames HEAD^1 HEAD > changed-paths
tr '\0' '\n' < changed-paths >&2
scripts/ci-surfaces.sh < changed-paths > surfaces.out
else
scripts/ci-surfaces.sh --all > surfaces.out
fi
cat surfaces.out
grep -E '^(codeql|codeql_matrix)=' surfaces.out >> "$GITHUB_OUTPUT"
analyze:
name: Analyze (${{ matrix.language }})
needs: changes
if: needs.changes.outputs.codeql == 'true'
runs-on: ${{ matrix.runner }}
# Normal runs take <=14 min (Swift autobuild is the longest); cap a hung
# autobuild instead of letting it hold a runner for an hour.
timeout-minutes: 30
permissions:
contents: read
security-events: write
actions: read
packages: read
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.changes.outputs.matrix) }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Initialize CodeQL
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- name: Autobuild
if: matrix.build-mode == 'autobuild'
uses: github/codeql-action/autobuild@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
category: "/language:${{ matrix.language }}"
+3 -98
View File
@@ -30,62 +30,8 @@ env:
RUST_BACKTRACE: 1
jobs:
# Decide which checks this event needs. On pull_request, each job below runs
# only when the PR touches the surface it validates, and the cargo test /
# release-build jobs are scoped to the touched crates plus their reverse
# dependencies (scripts/ci-surfaces.sh). Every other event (push to main,
# merge_group, the release flow's workflow_dispatch) runs the full suite, so
# main and the merge queue keep whole-workspace coverage and main-red-alert
# still sees a complete run. A skipped job reports as "skipped", not green.
changes:
name: changed surfaces
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
rust: ${{ steps.surfaces.outputs.rust }}
rust_packages: ${{ steps.surfaces.outputs.rust_packages }}
windows: ${{ steps.surfaces.outputs.windows }}
windows_packages: ${{ steps.surfaces.outputs.windows_packages }}
docker: ${{ steps.surfaces.outputs.docker }}
audit: ${{ steps.surfaces.outputs.audit }}
core_deps: ${{ steps.surfaces.outputs.core_deps }}
release_versioning: ${{ steps.surfaces.outputs.release_versioning }}
release_tag: ${{ steps.surfaces.outputs.release_tag }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
# The pull_request checkout is the test-merge commit; depth 2 brings
# its first parent (the base tip) so the diff is exactly this PR.
fetch-depth: 2
# Every job's gate comes from the mapper, so pin its mapping first; a
# failure here fails this job and the run is red, not silently narrowed.
- name: Test the surface mapper
run: scripts/test-ci-surfaces.sh
- name: Map changed paths to surfaces
id: surfaces
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "pull_request" ]; then
# -z: literal paths; plain --name-only C-quotes unusual names.
git diff -z --name-only --no-renames HEAD^1 HEAD > changed-paths
tr '\0' '\n' < changed-paths >&2
scripts/ci-surfaces.sh < changed-paths > surfaces.out
else
scripts/ci-surfaces.sh --all > surfaces.out
fi
cat surfaces.out
cat surfaces.out >> "$GITHUB_OUTPUT"
fmt-clippy:
name: fmt + clippy
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -111,8 +57,6 @@ jobs:
release-tag-validation:
name: release tag validation
needs: changes
if: needs.changes.outputs.release_tag == 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
@@ -126,8 +70,6 @@ jobs:
test:
name: test (${{ matrix.toolchain }})
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
@@ -172,17 +114,10 @@ jobs:
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
# `--workspace`, or `-p <crate>...` for the crates a PR touches plus
# every crate that depends on them.
PACKAGES: ${{ needs.changes.outputs.rust_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo test --locked $PACKAGES
run: cargo test --locked --workspace
build-release:
name: build --release
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
@@ -202,20 +137,10 @@ jobs:
key: release
- name: cargo build --release
env:
PACKAGES: ${{ needs.changes.outputs.rust_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo build --locked --release $PACKAGES
run: cargo build --locked --release --workspace
audit:
name: cargo audit
# cargo audit reads only Cargo.lock (+ .cargo/audit.toml), so a PR that
# leaves both unchanged cannot change its verdict; a newly published
# advisory against main is caught by the push-to-main run and the weekly
# Scheduled Audit instead of turning every unrelated PR red.
needs: changes
if: needs.changes.outputs.audit == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -275,8 +200,6 @@ jobs:
msrv:
name: MSRV (Rust 1.91)
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -300,12 +223,6 @@ jobs:
docker-build:
name: Docker build smoke test
# Runs when the image's own inputs change: the Dockerfile, the manifests
# its dependency layer copies, the lockfile, bootstrap-peers.json. A
# source-only Rust change is already compiled by `build --release` (stable)
# and `MSRV` (1.91, the image's toolchain); push to main still runs this.
needs: changes
if: needs.changes.outputs.docker == 'true'
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -322,8 +239,6 @@ jobs:
test-windows:
name: test (windows, non-blocking)
needs: changes
if: needs.changes.outputs.windows == 'true'
runs-on: windows-latest
timeout-minutes: 45
# Non-blocking, mirroring the informational `beta` lane above. We ship `gl`
@@ -351,13 +266,7 @@ jobs:
key: windows
- name: cargo test (shipped Windows crates)
shell: bash
env:
# `-p gl -p git-remote-gitlawb`, or whichever of the two a PR affects.
PACKAGES: ${{ needs.changes.outputs.windows_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo test --locked $PACKAGES
run: cargo test --locked -p gl -p git-remote-gitlawb
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
@@ -367,8 +276,6 @@ jobs:
# cannot do. Regen instructions live in the allowlist header.
core-deps-purity:
name: gitlawb-core dependency purity
needs: changes
if: needs.changes.outputs.core_deps == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
@@ -401,8 +308,6 @@ jobs:
# not its author knew this gate existed.
release-versioning:
name: release versioning coverage
needs: changes
if: needs.changes.outputs.release_versioning == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
+1 -1
View File
@@ -40,5 +40,5 @@ When a node denies a request, surface the denial to the user. Never render a den
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow's jobs are `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. On a PR, `scripts/ci-surfaces.sh` runs each job only when the PR touches its inputs, and scopes `cargo test` and the release build to the touched crates plus every crate that depends on them. Root `Cargo.toml`/`Cargo.lock`/`.cargo/` changes run the Rust jobs (fmt, clippy, tests, release build, MSRV) on the whole workspace; every other job still runs only when its own inputs change. Edits to the CI wiring itself (`pr-checks.yml`, `codeql.yml`, `scripts/ci-surfaces.sh`), push to `main`, and the merge queue run every job on the whole workspace. Edits to any other workflow run none of these jobs on a PR; CodeQL only analyzes them as Actions code. Run `cargo test --workspace` locally regardless. The same script limits CodeQL (`codeql.yml`) to the languages a PR touches. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
Generated
+544 -923
View File
File diff suppressed because it is too large. Load diff
+1 -7
View File
@@ -369,12 +369,6 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
`POST /api/v1/sync/trigger` is not part of the staged rollout: it always requires a signature in both config modes and returns 401 without one, because each call drives an O(peers) outbound fan-out.
The flag is not HTTP-only. It also gates inbound gossip ref-update events on the libp2p mesh, which carry an Ed25519 payload signature rather than RFC 9421 headers because there are no HTTP headers to sign. Set to `true`, an unsigned gossip event is dropped. Set to `false`, an unsigned event is accepted with a warning during the rolling-upgrade window, and the write budget it consumes is charged to the forwarding peer rather than to the DID it claims, since an unsigned event's claimed DID is asserted and not proven. A present-but-invalid signature is refused in both modes; that is forgery, not an un-upgraded peer.
Because the flag now spans both transports, the rollout order matters in one direction: **upgrade every gossip-publishing peer to a build that signs events before you set this to `true`.** Enabling it while an old publisher is still live drops that publisher's ref-updates on arrival, and the publisher sees no error, because gossip has no response to carry one. Upgrading the HTTP peers alone is not sufficient.
There is a second precondition on gossip ingest that the flag does not control. A publisher's `node_did` must already exist in the receiving node's peers table, or the event is dropped as an unknown peer DID. This holds in both modes, signed and unsigned alike: a valid signature proves key possession, not membership. Rows reach that table over HTTP, either from `GITLAWB_BOOTSTRAP_PEERS` when this node contacts a bootstrap peer and records the DID it reports, or from a prior `POST /api/v1/peers/announce`. So a peer that only ever joined the libp2p mesh, with no HTTP announce and no bootstrap contact in either direction, will have its ref-updates dropped even with a good signature and the flag off. If gossip is silently not landing from a peer you can see on the mesh, check that its DID is in `GET /api/v1/peers` on the receiving side first.
---
## Configuration
@@ -398,7 +392,7 @@ Important node settings:
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require peer writes to prove their DID on every transport: RFC 9421 signatures on the announce/sync routes, and an Ed25519 payload signature on gossip ref-update events. Defaults to `false` during the staged rollout below. Upgrade every gossip publisher before enabling, or their updates are dropped with no error on their side. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. Defaults to `false` during the staged rollout below. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
+9 -9
View File
@@ -44,14 +44,14 @@ dirs-next = "2"
# withholds a canonically-equivalent path regardless of NFC/NFD form (#101).
unicode-normalization = "0.1"
reqwest = { workspace = true }
libp2p-core = "0.44"
libp2p-gossipsub = "0.50.1"
libp2p-identify = "0.48"
libp2p-identity = { version = "0.3", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.49"
libp2p-quic = { version = "0.14", features = ["tokio"] }
libp2p-swarm = { version = "0.48", features = ["macros", "tokio"] }
multiaddr = "0.19"
libp2p-core = "0.43"
libp2p-gossipsub = "0.49.4"
libp2p-identify = "0.47"
libp2p-identity = { version = "0.2", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.48"
libp2p-quic = { version = "0.13", features = ["tokio"] }
libp2p-swarm = { version = "0.47", features = ["macros", "tokio"] }
multiaddr = "0.18"
futures = "0.3"
aws-sdk-s3 = { version = "1", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
aws-config = { version = "1", features = ["behavior-version-latest"] }
@@ -72,7 +72,7 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
libp2p-dns = { version = "0.45.0", features = ["tokio"] }
libp2p-dns = { version = "0.44.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
+195
View File
@@ -449,3 +449,198 @@ mod lock_pool_shed_tests {
);
}
}
/// #426: end-to-end at the handler seam. A corrupt object store behind a
/// resolving issue ref used to look identical to an absent issue: get returned
/// a fake 404, close returned a fake 404 after releasing the write guard, and
/// list silently under-reported. All three must now surface a git error.
#[cfg(test)]
mod corrupt_store_tests {
use super::*;
use axum::response::IntoResponse;
use sqlx::PgPool;
use tempfile::TempDir;
fn seed_repo(owner_did: &str, name: &str) -> crate::db::RepoRecord {
let now = Utc::now();
crate::db::RepoRecord {
id: Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
}
}
/// A state whose repo store roots in `repos_dir`, so the test can lay down
/// a real repo and then corrupt it without touching /tmp.
async fn state_with_repos_dir(pool: &PgPool, repos_dir: &std::path::Path) -> AppState {
let mut state = crate::test_support::test_state(pool.clone()).await;
state.repo_store =
crate::git::repo_store::RepoStore::for_testing(repos_dir.to_path_buf(), pool.clone());
state
}
/// Bare repo at the store's expected path, holding one issue ref.
fn bare_repo_with_issue(
repos_dir: &std::path::Path,
owner: &str,
name: &str,
) -> (std::path::PathBuf, String) {
let repo_path = repos_dir
.join(owner.replace([':', '/'], "_"))
.join(format!("{name}.git"));
std::fs::create_dir_all(&repo_path).unwrap();
let out = std::process::Command::new("git")
.args(["init", "--bare"])
.arg(&repo_path)
.output()
.unwrap();
assert!(out.status.success());
let issue_id = "aaaabbbb-0000-0000-0000-000000000000".to_string();
git_issues::create_issue(
&repo_path,
&issue_id,
&serde_json::json!({
"id": issue_id,
"title": "t",
"status": "open",
})
.to_string(),
)
.unwrap();
(repo_path, issue_id)
}
/// Delete the blob object behind the issue ref and put garbage in its
/// place: `cat-file` fails while `rev-parse` still resolves the ref.
fn corrupt_issue_object(repo_path: &std::path::Path, issue_id: &str) {
let out = std::process::Command::new("git")
.args(["rev-parse", &format!("refs/gitlawb/issues/{issue_id}")])
.current_dir(repo_path)
.output()
.unwrap();
let oid = String::from_utf8(out.stdout).unwrap().trim().to_string();
let obj = repo_path.join("objects").join(&oid[..2]).join(&oid[2..]);
std::fs::remove_file(&obj).unwrap();
std::fs::write(&obj, b"garbage").unwrap();
}
/// The regression: closing over a corrupt store used to return a fake 404
/// (or panic deeper in). It must reach the caller as a git 500, and the
/// handler must return rather than unwind.
#[sqlx::test]
async fn close_issue_on_corrupt_store_returns_git_error_not_panic(pool: PgPool) {
let owner = "did:key:zISSUECORRUPTCCCCCCCCCCCCCCCCCCCCCCCCC";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "corrupt-close"))
.await
.expect("seed repo");
let (repo_path, issue_id) = bare_repo_with_issue(dir.path(), owner, "corrupt-close");
corrupt_issue_object(&repo_path, &issue_id);
let err = close_issue(
State(state),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), "corrupt-close".to_string(), issue_id)),
)
.await
.expect_err("a corrupt store must error, not panic or 404");
assert_eq!(
err.into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR,
"a read failure is a git 500, not a fake not-found"
);
}
/// get on a corrupt store: 500, not the 404 the fold used to produce.
#[sqlx::test]
async fn get_issue_on_corrupt_store_returns_500_not_404(pool: PgPool) {
let owner = "did:key:zISSUECORRUPTGETDDDDDDDDDDDDDDDDDDDDDDD";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "corrupt-get"))
.await
.expect("seed repo");
let (repo_path, issue_id) = bare_repo_with_issue(dir.path(), owner, "corrupt-get");
corrupt_issue_object(&repo_path, &issue_id);
let err = get_issue(
State(state),
Path((owner.to_string(), "corrupt-get".to_string(), issue_id)),
None,
)
.await
.expect_err("a corrupt store must error, not 404");
assert_eq!(
err.into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR,
);
}
/// list on a corrupt store must fail outright; a 200 that silently drops
/// the unreadable issue is the other half of the fold.
#[sqlx::test]
async fn list_issues_on_corrupt_store_returns_git_error(pool: PgPool) {
let owner = "did:key:zISSUECORRUPTLISTEEEEEEEEEEEEEEEEEEEEEE";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "corrupt-list"))
.await
.expect("seed repo");
let (repo_path, issue_id) = bare_repo_with_issue(dir.path(), owner, "corrupt-list");
corrupt_issue_object(&repo_path, &issue_id);
let err = list_issues(
State(state),
Path((owner.to_string(), "corrupt-list".to_string())),
None,
)
.await
.expect_err("a corrupt store must error, not under-report");
assert_eq!(
err.into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR,
);
}
/// Must-not direction: a genuinely absent issue still gets the 404.
#[sqlx::test]
async fn get_issue_on_absent_ref_still_returns_404(pool: PgPool) {
let owner = "did:key:zISSUEABSENTFFFFFFFFFFFFFFFFFFFFFFFFFFF";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "absent-get"))
.await
.expect("seed repo");
bare_repo_with_issue(dir.path(), owner, "absent-get");
let err = get_issue(
State(state),
Path((
owner.to_string(),
"absent-get".to_string(),
"ffffffff-0000-0000-0000-000000000000".to_string(),
)),
None,
)
.await
.expect_err("a missing issue is still not-found");
assert_eq!(err.into_response().status(), StatusCode::NOT_FOUND);
}
}
-14
View File
@@ -2758,11 +2758,6 @@ async fn post_receive_replication_tail(
if announce {
if let Some(p2p) = &p2p_handle {
p2p.publish_ref_update(crate::p2p::RefUpdateEvent {
// Named, not literal: the ingest gate compares `v`
// against this same constant, so a bump has to move
// both ends together rather than leaving the emitter
// on a version the gate no longer accepts.
v: crate::p2p::CURRENT_REF_UPDATE_VERSION,
node_did: node_did_str.clone(),
pusher_did: pusher_did_clone.clone(),
repo: repo_slug.clone(),
@@ -2773,15 +2768,6 @@ async fn post_receive_replication_tail(
timestamp: chrono::Utc::now().to_rfc3339(),
cert_id: None,
cid: cid.map(|s| s.to_string()),
// Left unsigned here on purpose. The swarm loop
// signs the event with the node keypair via
// `p2p::signed_publish_bytes` immediately before it
// publishes, and skips the publish outright if
// signing fails, so the wire always carries a
// signature even though this construction site does
// not. Setting one here would be signed over a
// payload the publisher has not finished building.
sig: None,
})
.await;
}
+1 -3
View File
@@ -67,9 +67,7 @@ pub struct Config {
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>,
/// Require a peer write to prove its DID on any transport: RFC 9421
/// signatures on the peer announce/sync write routes, and a payload
/// signature on gossip ref-update events.
/// Require RFC 9421 signatures on peer announce/sync write routes.
/// Keep false during rolling upgrades so existing live nodes can still gossip.
#[arg(
long,
+9 -53
View File
@@ -2571,7 +2571,15 @@ impl Db {
// The read-then-write race is benign: if a row appears in between,
// this write becomes an UPDATE and the UnprovenRepoint guard below
// still refuses any http_url change.
PeerWriteAuthority::Unproven if !self.peer_exists(did).await.unwrap_or(false) => {
PeerWriteAuthority::Unproven
if !sqlx::query_scalar::<_, bool>(
"SELECT EXISTS(SELECT 1 FROM peers WHERE did = $1)",
)
.bind(did)
.fetch_one(&self.pool)
.await
.unwrap_or(false) =>
{
match did.parse::<gitlawb_core::did::Did>() {
// Only reachable from the bootstrap announce-back in main.rs,
// which passes the contacted peer's raw JSON string; the
@@ -2701,21 +2709,6 @@ impl Db {
Ok(())
}
/// Whether a DID has a row in `peers`, by keyed lookup.
///
/// For callers that only need the membership answer. `list_peers` fetches
/// and materializes every row, so using it as a membership test on a hot
/// path (the gossip ingest gate) makes the cost of one event grow with the
/// size of the table.
pub async fn peer_exists(&self, did: &str) -> Result<bool> {
Ok(
sqlx::query_scalar::<_, bool>("SELECT EXISTS(SELECT 1 FROM peers WHERE did = $1)")
.bind(did)
.fetch_one(&self.pool)
.await?,
)
}
pub async fn list_peers(&self) -> Result<Vec<PeerRecord>> {
let rows = sqlx::query(
"SELECT did, http_url, last_seen, last_ping_ok, announced_at
@@ -7871,41 +7864,6 @@ mod peer_authority_tests {
db
}
/// `peer_exists` answers the membership question the gossip ingest gate
/// asks on every event, so both answers are pinned here: a registered DID
/// is true, and an unregistered one is false rather than an error or a
/// prefix match on a registered DID.
#[sqlx::test]
async fn peer_exists_answers_both_ways(pool: sqlx::PgPool) {
let db = db(pool).await;
let did = VICTIM_DID;
assert!(
!db.peer_exists(did).await.unwrap(),
"an empty peers table must answer false, not error"
);
db.upsert_peer(
did,
"https://peer.example.com",
PeerWriteAuthority::Proven(did),
)
.await
.unwrap();
assert!(db.peer_exists(did).await.unwrap());
assert!(
!db.peer_exists(OTHER_DID).await.unwrap(),
"a different DID must not match"
);
assert!(
!db.peer_exists(&VICTIM_DID[..VICTIM_DID.len() - 1])
.await
.unwrap(),
"the lookup is an equality test, not a prefix test"
);
}
/// The whole row, read back through `list_peers` rather than raw SQL, so a
/// case that claims "unchanged" is comparing every column a consumer sees.
async fn row(db: &Db, did: &str) -> Option<(String, String, Option<String>, bool, String)> {
@@ -8322,7 +8280,6 @@ mod peer_authority_tests {
/// | `a_legacy_row_can_still_refresh_its_liveness` (db/mod.rs) | test-only. Seeds a PRE-GATE row by raw SQL on purpose: `upsert_peer` cannot create one, since the gate it is testing refuses exactly that DID. The fixture models what a deployed table already holds |
/// | `gossip_ping_round_requires_two_failures_before_persisting_unreachable` (main.rs) | test-only fixture seed. Raw SQL because the test drives the readiness HYSTERESIS, which needs a row already at `last_ping_ok = TRUE` before the round runs; it never exercises the announce gate |
/// | `manual_ping_uses_readiness_without_mutating_federation_gate` (api/peers.rs) | test-only fixture seed, same shape and same reason: the row under test must pre-exist so the assertion is about what the ping does NOT rewrite |
/// | `seed_peer` (p2p/mod.rs) | test-only fixture seed. Raw SQL so the gossip ingest tests can drive the known-peer gate directly, including the did:web case `upsert_peer` would refuse |
///
/// And the `upsert_peer` CALL-SITE authority table, which the ledger above
/// structurally cannot hold, because the bootstrap site issues no SQL of its own
@@ -8420,7 +8377,6 @@ mod peers_table_writer_guard {
("prune_non_public_peers", 1),
("prune_self_peers", 1),
("seed_local_peer", 1),
("seed_peer", 1),
("upsert_peer", 2),
];
+240 -27
View File
@@ -61,6 +61,119 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
Ok(())
}
/// Issue ids are UUIDs; refs under `refs/gitlawb/issues/` only ever carry
/// `[0-9a-zA-Z_-]` names. Anything else is not a value we pass to git.
fn issue_ref_name_is_safe(ref_name: &str) -> bool {
let Some(id) = ref_name.strip_prefix("refs/gitlawb/issues/") else {
return false;
};
!id.is_empty()
&& id
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_')
}
/// Resolve a ref name without putting it on a command line: enumerate refs
/// (constant argv) and match in memory. A nonzero exit propagates, so "cannot
/// determine" is an error rather than a quiet `false`.
fn ref_name_resolves(repo_path: &Path, ref_name: &str) -> Result<bool> {
// allow-unbounded-git: failure-path existence recheck, module convention.
let out = Command::new("git")
.args(["for-each-ref", "--format=%(refname)"])
.current_dir(repo_path)
.output()
.context("failed to run git for-each-ref")?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr);
anyhow::bail!("git for-each-ref failed: {}", stderr.trim());
}
let stdout = String::from_utf8_lossy(&out.stdout);
Ok(stdout.lines().any(|line| line.trim() == ref_name))
}
/// Read the blob behind an issue ref.
///
/// `Ok(None)` means the ref itself is absent. A ref that resolves but whose
/// object cannot be read (corrupt object, unreadable store, a mid-read gc) is
/// an error, not an absence (#426).
///
/// The ref name reaches git on stdin, never argv: `cat-file --batch` answers
/// `<oid> <type> <size>\n<body>\n` for a readable object and
/// `<spec> missing\n` for an absent ref or an unreadable object alike, so the
/// `missing` case is re-resolved by name enumeration to tell them apart.
fn read_issue_blob(repo_path: &Path, ref_name: &str) -> Result<Option<String>> {
if !issue_ref_name_is_safe(ref_name) {
anyhow::bail!("refusing issue ref outside the issues namespace: {ref_name}");
}
// allow-unbounded-git: this module's spawns predate the bounded runner and
// its callers run inside issue handlers that already hold the repo guard.
let mut child = Command::new("git")
.args(["cat-file", "--batch"])
.current_dir(repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.context("failed to spawn git cat-file --batch")?;
use std::io::Write;
// Reap the child even when the write fails so a stdin error cannot drop an
// unwaited Child and leak a zombie (#53).
let write_result = match child.stdin.take() {
Some(mut stdin) => stdin
.write_all(ref_name.as_bytes())
.and_then(|()| stdin.write_all(b"\n")),
None => Err(std::io::Error::new(
std::io::ErrorKind::BrokenPipe,
"git cat-file --batch stdin unavailable",
)),
};
let output = child
.wait_with_output()
.context("git cat-file --batch failed")?;
write_result.context("failed to write ref name to git cat-file stdin")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("git cat-file --batch failed: {}", stderr.trim());
}
// "<spec> missing\n" covers an absent ref and an unreadable object; only a
// ref that still resolves is the error case.
let missing = format!("{ref_name} missing\n");
if output.stdout.as_slice() == missing.as_bytes() {
return match ref_name_resolves(repo_path, ref_name)? {
true => anyhow::bail!("issue ref resolves but its object is unreadable: {ref_name}"),
false => Ok(None),
};
}
// Header: "<oid> <type> <size>\n" then exactly <size> bytes plus a newline.
let header_end = output
.stdout
.iter()
.position(|b| *b == b'\n')
.context("unexpected git cat-file --batch output")?;
let header = String::from_utf8_lossy(&output.stdout[..header_end]);
let mut parts = header.split(' ');
parts.next();
let obj_type = parts.next().unwrap_or("");
if obj_type != "blob" {
anyhow::bail!("issue ref {ref_name} resolves to a non-blob object");
}
let size: usize = parts
.next()
.and_then(|s| s.parse().ok())
.context("unexpected git cat-file --batch header")?;
let body = &output.stdout[header_end + 1..];
anyhow::ensure!(
body.len() >= size,
"truncated git cat-file --batch body for {ref_name}"
);
Ok(Some(String::from_utf8_lossy(&body[..size]).to_string()))
}
/// List all issue refs and return their JSON content.
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
@@ -74,9 +187,11 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
.output()
.context("failed to run git for-each-ref")?;
// for-each-ref exits 0 with empty output when no refs match, so a nonzero
// exit is a real enumeration failure, not "no issues yet".
if !list_output.status.success() {
// No issues yet
return Ok(vec![]);
let stderr = String::from_utf8_lossy(&list_output.stderr);
anyhow::bail!("git for-each-ref failed: {}", stderr.trim());
}
let refs_str = String::from_utf8_lossy(&list_output.stdout);
@@ -88,16 +203,12 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
continue;
}
// Read the blob content
let cat_output = Command::new("git")
.args(["cat-file", "blob", ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if cat_output.status.success() {
let content = String::from_utf8_lossy(&cat_output.stdout).to_string();
issues.push(content);
// Read the blob content. A ref that vanished between for-each-ref and
// here is skipped; a ref that resolves but fails to read propagates so
// a degraded object store errors the listing instead of under-reporting.
match read_issue_blob(repo_path, ref_name)? {
Some(content) => issues.push(content),
None => continue,
}
}
@@ -127,8 +238,12 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
.output()
.context("failed to run git for-each-ref")?;
// for-each-ref exits 0 with empty output when nothing matches, so a
// nonzero exit is a real enumeration failure (e.g. not a repo), not
// "issue not found".
if !list.status.success() {
return Ok(None);
let stderr = String::from_utf8_lossy(&list.stderr);
anyhow::bail!("git for-each-ref failed: {}", stderr.trim());
}
let output = String::from_utf8_lossy(&list.stdout);
@@ -161,8 +276,12 @@ pub fn close_issue(repo_path: &Path, issue_id: &str) -> Result<Option<String>> {
None => return Ok(None),
};
let raw = get_issue(repo_path, &full_id)?
.expect("ref existed in resolve but not in get — should be impossible");
// A ref that resolved and is now absent was deleted between the two calls;
// any other read failure propagates as an error rather than panic (#426).
let raw = match get_issue(repo_path, &full_id)? {
Some(raw) => raw,
None => return Ok(None),
};
let mut issue: serde_json::Value =
serde_json::from_str(&raw).context("invalid issue JSON in git ref")?;
@@ -181,18 +300,7 @@ pub fn get_issue(repo_path: &Path, issue_id: &str) -> Result<Option<String>> {
};
let ref_name = format!("refs/gitlawb/issues/{full_id}");
let cat_output = Command::new("git")
.args(["cat-file", "blob", &ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !cat_output.status.success() {
return Ok(None);
}
let content = String::from_utf8_lossy(&cat_output.stdout).to_string();
Ok(Some(content))
read_issue_blob(repo_path, &ref_name)
}
#[cfg(test)]
@@ -294,4 +402,109 @@ mod tests {
let v: serde_json::Value = serde_json::from_str(&updated).unwrap();
assert_eq!(v["status"], "closed");
}
/// Overwrite the loose object behind an issue ref with garbage, so
/// `cat-file` fails while `rev-parse` still resolves the ref (#426).
fn corrupt_issue_object(dir: &TempDir, full_id: &str) {
let oid_out = Command::new("git")
.args(["rev-parse", &format!("refs/gitlawb/issues/{full_id}")])
.current_dir(dir.path())
.output()
.unwrap();
let oid = String::from_utf8(oid_out.stdout)
.unwrap()
.trim()
.to_string();
let obj = dir
.path()
.join(".git/objects")
.join(&oid[..2])
.join(&oid[2..]);
std::fs::remove_file(&obj).unwrap();
std::fs::write(&obj, b"garbage").unwrap();
}
#[test]
fn test_get_and_close_missing_issue_return_none() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
assert_eq!(get_issue(dir.path(), "nosuch00").unwrap(), None);
assert_eq!(close_issue(dir.path(), "nosuch00").unwrap(), None);
}
// #426: a directory that is not a repo makes `cat-file --batch` exit
// nonzero; that failure is an error, not a quiet None.
#[test]
fn test_issue_read_on_non_repo_errors_instead_of_none() {
let dir = TempDir::new().unwrap();
let not_a_repo = dir.path().join("not-a-repo");
std::fs::create_dir_all(&not_a_repo).unwrap();
assert!(read_issue_blob(&not_a_repo, "refs/gitlawb/issues/deadbeef").is_err());
}
// #426: the ref name is constrained to the issues namespace and a safe
// charset before it is fed to git, even on stdin (a newline would smuggle
// a second `--batch` spec).
#[test]
fn test_issue_read_rejects_ref_outside_issues_namespace() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
assert!(read_issue_blob(dir.path(), "refs/heads/main").is_err());
assert!(read_issue_blob(dir.path(), "refs/gitlawb/issues/../x").is_err());
}
// #426: a failed for-each-ref enumeration is an error, not an empty list.
#[test]
fn test_list_issues_on_non_repo_errors_instead_of_empty() {
let dir = TempDir::new().unwrap();
let not_a_repo = dir.path().join("not-a-repo");
std::fs::create_dir_all(&not_a_repo).unwrap();
assert!(list_issues(&not_a_repo).is_err());
}
// #426: the same enumeration failure inside resolve_issue_id must reach
// get_issue/close_issue as an error, not a "not found" None.
#[test]
fn test_get_and_close_on_non_repo_error_instead_of_none() {
let dir = TempDir::new().unwrap();
let not_a_repo = dir.path().join("not-a-repo");
std::fs::create_dir_all(&not_a_repo).unwrap();
assert!(get_issue(&not_a_repo, "nosuch00").is_err());
assert!(close_issue(&not_a_repo, "nosuch00").is_err());
}
// #426: a ref that resolves but whose object is corrupt is a read error,
// not an absent issue. `close_issue` used to `expect()` on that None and
// panic inside a request handler holding the write guard.
#[test]
fn test_corrupt_issue_object_errors_instead_of_panicking() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "bad00000-0000-0000-0000-000000000000";
create_issue(dir.path(), full_id, r#"{"status":"open"}"#).unwrap();
corrupt_issue_object(&dir, full_id);
assert!(get_issue(dir.path(), full_id).is_err());
assert!(close_issue(dir.path(), full_id).is_err());
assert!(list_issues(dir.path()).is_err());
}
#[test]
fn test_list_issues_returns_all() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(
dir.path(),
"aaa00000-0000-0000-0000-000000000000",
r#"{"status":"open"}"#,
)
.unwrap();
create_issue(
dir.path(),
"bbb00000-0000-0000-0000-000000000000",
r#"{"status":"open"}"#,
)
.unwrap();
assert_eq!(list_issues(dir.path()).unwrap().len(), 2);
}
}
+2 -4
View File
@@ -115,7 +115,7 @@ async fn main() -> Result<()> {
}
// Load or generate the node's identity keypair
let keypair = Arc::new(load_or_create_keypair(&config)?);
let keypair = load_or_create_keypair(&config)?;
// Sealing key for the legacy-scan continuation tokens, DERIVED from the identity
// just loaded so it is the same key after a restart (see `derive_scan_token_key`).
let scan_token_key = AppState::derive_scan_token_key(&keypair);
@@ -265,8 +265,6 @@ async fn main() -> Result<()> {
Arc::clone(&db),
config.auto_sync,
shutdown_rx,
Arc::clone(&keypair),
config.require_signed_peer_writes,
)
.await
{
@@ -414,7 +412,7 @@ async fn main() -> Result<()> {
config: Arc::new(config.clone()),
db,
node_did: node_did.clone(),
node_keypair: keypair,
node_keypair: Arc::new(keypair),
p2p: p2p_handle,
http_client,
ref_update_tx,
-46
View File
@@ -11,8 +11,6 @@
//! `gitlawb_sync_queue_processed_total{status}`
//! * are webhooks reaching their endpoints? —
//! `gitlawb_webhook_deliveries_total{result}`
//! * is inbound gossip being admitted or shed, and for which reason?
//! `gitlawb_gossip_ingest_events_total{outcome}`
//! * how big are the packs we're sending and receiving? —
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
@@ -51,7 +49,6 @@ static AUTH_SUCCESSES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_FAILURES: OnceLock<IntCounterVec> = OnceLock::new();
static SYNC_PROCESSED: OnceLock<IntCounterVec> = OnceLock::new();
static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
@@ -171,21 +168,6 @@ fn init_inner(version: &str, node_did: &str) {
.set(webhook_deliveries)
.expect("set WEBHOOK_DELIVERIES once");
let gossip_ingest = IntCounterVec::new(
Opts::new(
"gitlawb_gossip_ingest_events_total",
"Total inbound gossip ref-update events and what the ingest path decided about each",
),
&["outcome"],
)
.expect("gitlawb_gossip_ingest_events_total definition");
registry
.register(Box::new(gossip_ingest.clone()))
.expect("register gitlawb_gossip_ingest_events_total");
GOSSIP_INGEST
.set(gossip_ingest)
.expect("set GOSSIP_INGEST once");
let pack_size = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_pack_size_bytes",
@@ -288,33 +270,6 @@ pub fn record_webhook_delivery(result: &str) {
}
}
/// Record what the gossip ingest path decided about one inbound ref-update.
/// `outcome` ∈ {accepted, unsigned_admitted, write_failed, rejected,
/// source_rate_limited, author_rate_limited, unsigned_source_rate_limited}.
///
/// `accepted` is reserved for signature-verified events. An unsigned event that
/// survives the rolling-upgrade window is `unsigned_admitted`, so the
/// authenticated-admission rate is not silently padded by legacy compatibility
/// traffic; an operator can tell whether the fleet still relies on the
/// compatibility allowance.
///
/// The three shed reasons stay separate labels rather than one `rate_limited`
/// because they answer different operator questions: `source_rate_limited` is a
/// forwarding peer over the pre-parse brake, `unsigned_source_rate_limited` is
/// that same forwarder relaying unproven traffic, and `author_rate_limited`
/// names a principal a signature actually proved. Collapsing them would make a
/// flood of unauthenticated garbage indistinguishable from one registered peer
/// pushing too hard, which is exactly the distinction an alert needs.
///
/// The label is the variant, never the free-form reason string that
/// `Rejected`/`WriteFailed` carry: those are attacker-influenced and would blow
/// up the label cardinality of a process-wide registry.
pub fn record_gossip_ingest(outcome: &str) {
if let Some(c) = GOSSIP_INGEST.get() {
c.with_label_values(&[outcome]).inc();
}
}
/// Record a pack body size observation (bytes).
pub fn observe_pack_size(bytes: f64) {
if let Some(h) = PACK_SIZE.get() {
@@ -410,7 +365,6 @@ mod tests {
record_auth_failure("test/route", "test_reason");
record_sync_processed("done");
record_webhook_delivery("ok");
record_gossip_ingest("accepted");
observe_pack_size(1024.0);
set_peers_connected(0);
}
File diff suppressed because it is too large. Load diff
-17
View File
@@ -143,23 +143,6 @@ impl RateLimiter {
false
}
/// The configured key ceiling. Tests use it to assert that a limiter a
/// production factory builds is actually bounded, which flooding the real
/// ceiling in distinct keys would be far too slow to show.
#[cfg(test)]
pub fn max_keys(&self) -> usize {
self.max_keys
}
/// The configured per-key request cap. Tests use it to assert that a
/// limiter a production factory builds carries the cap it is documented
/// with; re-asserting the constant on its own cannot tell whether the
/// factory passed that constant or a different one.
#[cfg(test)]
pub fn max_requests(&self) -> usize {
self.max_requests
}
pub async fn cleanup(&self) {
let now = Instant::now();
let mut state = self.state.lock().await;
-221
View File
@@ -1,221 +0,0 @@
#!/usr/bin/env bash
#
# Map a pull request's changed paths to the CI surfaces they touch, so each
# check in pr-checks.yml and codeql.yml runs only when its inputs changed and
# Rust jobs test only the affected crates.
#
# Usage:
# git diff -z --name-only --no-renames BASE HEAD | scripts/ci-surfaces.sh
# scripts/ci-surfaces.sh --all # every surface, whole workspace
#
# Input is NUL-terminated paths (`git diff -z`). Without -z, git C-quotes any
# path with non-ASCII, tab, quote or newline characters ("crates/gl/\303\251.rs"),
# which would match no surface and skip every job; newline-delimited input
# also cannot carry a path that contains a newline. Input that does not end in
# NUL (e.g. a plain `git diff --name-only`) is rejected rather than misread.
# scripts/test-ci-surfaces.sh pins the mapping.
#
# Prints GITHUB_OUTPUT lines (key=value). Each surface runs only for the paths
# listed for it below, so a path no surface lists (docs, README, infra/, any
# other workflow) runs no PR Checks job. Within the Rust surface, a change this
# script cannot attribute to specific crates widens rather than narrows: a
# global Rust input or an unknown crate directory runs the Rust jobs on the
# whole workspace. The CI wiring itself (pr-checks.yml, codeql.yml, this
# script) runs every surface.
# Non-PR events (push to main, merge_group, workflow_dispatch, schedule) are
# expected to pass --all, so main and the merge queue keep full coverage.
#
# Runnable from anywhere in the repo.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
all=false
if [ "${1:-}" = "--all" ]; then
all=true
elif [ "$#" -ne 0 ]; then
echo "usage: $0 [--all] < NUL-terminated-changed-paths" >&2
exit 2
fi
changed=()
if [ "$all" = false ]; then
path=""
while IFS= read -r -d '' path; do
[ -n "$path" ] && changed+=("$path")
done
# read leaves any unterminated tail in $path.
if [ -n "$path" ]; then
echo "ci-surfaces.sh: input is not NUL-terminated; pipe \`git diff -z --name-only\`" >&2
exit 2
fi
fi
# Workspace crates, derived from crates/*/Cargo.toml so a new crate is picked
# up without editing this script. dir -> package name, and dir -> the crate
# dirs it depends on through `path = "../<dir>"` (normal, dev, or build deps).
declare -A pkg_of=() deps_of=()
for manifest in "$ROOT"/crates/*/Cargo.toml; do
dir="$(basename "$(dirname "$manifest")")"
pkg_of[$dir]="$(sed -n 's/^name *= *"\([^"]*\)".*/\1/p' "$manifest" | head -n1)"
deps_of[$dir]="$(sed -n 's#.*path *= *"\.\./\([^"/]*\)".*#\1#p' "$manifest" | tr '\n' ' ')"
done
# Files outside crates/ that a crate compiles in (include_str!/include_bytes!),
# path -> crate dir. A change to one is a change to that crate, so its tests
# run. Candidates (keep those that resolve outside crates/<dir>/):
# git grep -nE 'include_(str|bytes)!\("(\.\./){2,}' crates
declare -A embedded_in=(
[bootstrap-peers.json]=gitlawb-node
)
# Mark a crate dir as touched; a dir this checkout does not know (removed or
# renamed crate) widens the Rust jobs to the whole workspace.
touch_crate() {
if [ -n "${pkg_of[$1]:-}" ]; then
touched[$1]=1
else
full_rust=true
fi
}
full_rust=false
rust=false
docker=false
audit=false
core_deps=false
release_versioning=false
release_tag=false
cq_actions=false
cq_js=false
cq_rust=false
cq_swift=false
declare -A touched=()
for path in "${changed[@]}"; do
case "$path" in
# The CI wiring itself: validate everything it gates.
.github/workflows/pr-checks.yml | .github/workflows/codeql.yml | scripts/ci-surfaces.sh)
all=true ;;
# Workspace-wide Rust inputs.
Cargo.toml | Cargo.lock | .cargo/* | rust-toolchain | rust-toolchain.toml \
| rustfmt.toml | .rustfmt.toml | clippy.toml | .clippy.toml)
full_rust=true ;;
crates/*/*)
dir="${path#crates/}"
touch_crate "${dir%%/*}" ;;
*)
if [ -n "${embedded_in[$path]:-}" ]; then
touch_crate "${embedded_in[$path]}"
fi ;;
esac
case "$path" in
Dockerfile | .dockerignore | bootstrap-peers.json | Cargo.toml | Cargo.lock \
| .cargo/* | rust-toolchain | rust-toolchain.toml | crates/*/Cargo.toml)
docker=true ;;
esac
case "$path" in
Cargo.lock | .cargo/audit.toml) audit=true ;;
esac
case "$path" in
Cargo.toml | Cargo.lock | crates/gitlawb-core/Cargo.toml \
| ci/gitlawb-core-allowed-deps.txt | scripts/check-gitlawb-core-deps.sh)
core_deps=true ;;
esac
case "$path" in
Cargo.toml | crates/*/Cargo.toml | release-please-config.json \
| .release-please-manifest.json | scripts/check-release-versioning.sh)
release_versioning=true ;;
esac
case "$path" in
scripts/resolve-release-tag.sh | scripts/test-resolve-release-tag.sh)
release_tag=true ;;
esac
case "$path" in
.github/workflows/* | .github/actions/*) cq_actions=true ;;
esac
case "$path" in
npm/* | *.js | *.mjs | *.cjs | *.jsx | *.ts | *.mts | *.cts | *.tsx) cq_js=true ;;
esac
case "$path" in
macos-app/* | *.swift) cq_swift=true ;;
esac
done
if [ "$all" = true ]; then
full_rust=true
docker=true
audit=true
core_deps=true
release_versioning=true
release_tag=true
cq_actions=true
cq_js=true
cq_swift=true
fi
# Affected crates: the touched ones plus everything that depends on them,
# transitively, so a gitlawb-core change still tests every consumer.
declare -A affected=()
for dir in "${!touched[@]}"; do affected[$dir]=1; done
grew=true
while [ "$grew" = true ]; do
grew=false
for dir in "${!deps_of[@]}"; do
[ -n "${affected[$dir]:-}" ] && continue
for dep in ${deps_of[$dir]}; do
if [ -n "${affected[$dep]:-}" ]; then
affected[$dir]=1
grew=true
break
fi
done
done
done
if [ "$full_rust" = true ] || [ "${#affected[@]}" -gt 0 ]; then
rust=true
cq_rust=true
fi
rust_packages=""
windows_packages=""
if [ "$full_rust" = true ]; then
rust_packages="--workspace"
windows_packages="-p gl -p git-remote-gitlawb"
elif [ "$rust" = true ]; then
for dir in $(printf '%s\n' "${!affected[@]}" | sort); do
rust_packages+="${rust_packages:+ }-p ${pkg_of[$dir]}"
case "${pkg_of[$dir]}" in
gl | git-remote-gitlawb) windows_packages+="${windows_packages:+ }-p ${pkg_of[$dir]}" ;;
esac
done
fi
windows=false
[ -n "$windows_packages" ] && windows=true
languages=()
[ "$cq_actions" = true ] && languages+=('{"language":"actions","build-mode":"none","runner":"ubuntu-latest"}')
[ "$cq_js" = true ] && languages+=('{"language":"javascript-typescript","build-mode":"none","runner":"ubuntu-latest"}')
[ "$cq_rust" = true ] && languages+=('{"language":"rust","build-mode":"none","runner":"ubuntu-latest"}')
[ "$cq_swift" = true ] && languages+=('{"language":"swift","build-mode":"autobuild","runner":"macos-latest"}')
codeql_matrix="{\"include\":[$(IFS=,; echo "${languages[*]}")]}"
codeql=false
[ "${#languages[@]}" -gt 0 ] && codeql=true
cat <<EOF
rust=$rust
rust_packages=$rust_packages
windows=$windows
windows_packages=$windows_packages
docker=$docker
audit=$audit
core_deps=$core_deps
release_versioning=$release_versioning
release_tag=$release_tag
codeql=$codeql
codeql_matrix=$codeql_matrix
EOF
-121
View File
@@ -1,121 +0,0 @@
#!/usr/bin/env bash
# Fixture test for scripts/ci-surfaces.sh, which alone decides which PR Checks
# jobs and CodeQL languages run. Pins at least one must-not-skip case per
# surface row, the crate fan-out and widening rules, and paths that git would
# C-quote without -z. Run from pr-checks.yml before the mapper is used.
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
mapper="$repo_root/scripts/ci-surfaces.sh"
failures=0
# map PATH... : run the mapper on the paths as `git diff -z` would emit them.
map() {
if [ "$#" -gt 0 ]; then printf '%s\0' "$@"; fi | "$mapper"
}
# check CASE OUTPUT KEY=VALUE... : each KEY must have exactly VALUE.
check() {
local name="$1" out="$2" want key
shift 2
for want in "$@"; do
key="${want%%=*}"
if ! grep -qxF -- "$want" <<<"$out"; then
printf 'FAIL %s: want %s, got %s\n' "$name" "$want" \
"$(grep -E "^${key}=" <<<"$out" || echo '<missing>')" >&2
failures=$((failures + 1))
fi
done
}
# has_packages CASE OUTPUT PKG... : rust_packages must include each -p PKG.
has_packages() {
local name="$1" out="$2" pkgs pkg
shift 2
pkgs=" $(sed -n 's/^rust_packages=//p' <<<"$out") "
for pkg in "$@"; do
if [[ "$pkgs" != *" -p $pkg "* ]]; then
printf 'FAIL %s: rust_packages lacks -p %s (got%s)\n' "$name" "$pkg" "$pkgs" >&2
failures=$((failures + 1))
fi
done
}
nothing=(rust=false rust_packages= windows=false windows_packages= docker=false
audit=false core_deps=false release_versioning=false release_tag=false
codeql=false 'codeql_matrix={"include":[]}')
everything=(rust=true rust_packages=--workspace windows=true
'windows_packages=-p gl -p git-remote-gitlawb' docker=true audit=true
core_deps=true release_versioning=true release_tag=true codeql=true)
all_languages='codeql_matrix={"include":[{"language":"actions","build-mode":"none","runner":"ubuntu-latest"},{"language":"javascript-typescript","build-mode":"none","runner":"ubuntu-latest"},{"language":"rust","build-mode":"none","runner":"ubuntu-latest"},{"language":"swift","build-mode":"autobuild","runner":"macos-latest"}]}'
rust_only='codeql_matrix={"include":[{"language":"rust","build-mode":"none","runner":"ubuntu-latest"}]}'
# Docs-only and empty input run nothing.
check "docs only" "$(map README.md docs/x.md infra/x.tf)" "${nothing[@]}"
check "empty input" "$(map)" "${nothing[@]}"
# A crate change tests that crate and its reverse deps, nothing else.
check "gl source" "$(map crates/gl/src/main.rs)" \
rust=true 'rust_packages=-p gl' windows=true 'windows_packages=-p gl' \
docker=false audit=false "$rust_only"
out="$(map crates/gitlawb-core/src/lib.rs)"
check "gitlawb-core source" "$out" rust=true windows=true \
'windows_packages=-p git-remote-gitlawb -p gl' "$rust_only"
has_packages "gitlawb-core fan-out" "$out" gitlawb-core icaptcha-client \
gitlawb-attest git-remote-gitlawb gl gitlawb-node
check "icaptcha-client source" "$(map crates/icaptcha-client/src/lib.rs)" \
'rust_packages=-p gl -p icaptcha-client' 'windows_packages=-p gl'
# Anything the Rust surface cannot attribute to known crates widens.
check "unknown crate dir" "$(map crates/no-such-crate/src/lib.rs)" \
rust=true rust_packages=--workspace docker=false
check "root Cargo.lock" "$(map Cargo.lock)" \
rust=true rust_packages=--workspace docker=true audit=true core_deps=true \
release_versioning=false
check "toolchain file" "$(map rust-toolchain.toml)" rust=true rust_packages=--workspace
# Files compiled into a crate from outside crates/.
check "bootstrap-peers.json" "$(map bootstrap-peers.json)" \
rust=true 'rust_packages=-p gitlawb-node' windows=false docker=true
# One must-not-skip trigger per non-Rust row.
check "Dockerfile" "$(map Dockerfile)" docker=true rust=false
check "crate manifest" "$(map crates/gl/Cargo.toml)" \
docker=true release_versioning=true 'rust_packages=-p gl'
check "audit config" "$(map .cargo/audit.toml)" audit=true rust_packages=--workspace
check "core allowlist" "$(map ci/gitlawb-core-allowed-deps.txt)" core_deps=true rust=false
check "release-please config" "$(map release-please-config.json)" \
release_versioning=true rust=false
check "release tag resolver" "$(map scripts/resolve-release-tag.sh)" release_tag=true rust=false
check "other workflow" "$(map .github/workflows/release.yml)" "${nothing[@]:0:9}" \
codeql=true 'codeql_matrix={"include":[{"language":"actions","build-mode":"none","runner":"ubuntu-latest"}]}'
check "npm" "$(map npm/package.json)" codeql=true \
'codeql_matrix={"include":[{"language":"javascript-typescript","build-mode":"none","runner":"ubuntu-latest"}]}'
check "swift" "$(map macos-app/Sources/App.swift)" rust=false codeql=true \
'codeql_matrix={"include":[{"language":"swift","build-mode":"autobuild","runner":"macos-latest"}]}'
# Editing the CI wiring runs every surface, exactly like --all.
for wiring in .github/workflows/pr-checks.yml .github/workflows/codeql.yml scripts/ci-surfaces.sh; do
check "wiring $wiring" "$(map "$wiring")" "${everything[@]}" "$all_languages"
done
check "--all" "$("$mapper" --all </dev/null)" "${everything[@]}" "$all_languages"
# Names git C-quotes without -z (non-ASCII, tab, quote, newline) still map.
for name in $'\xc3\xa9.rs' $'a\tb.rs' 'a"b.rs' $'a\nb.rs'; do
check "quoted name $(printf %q "$name")" "$(map "crates/gl/src/$name")" \
rust=true 'rust_packages=-p gl'
done
check "non-ASCII swift" "$(map $'macos-app/Sources/Caf\xc3\xa9.swift')" codeql=true \
'codeql_matrix={"include":[{"language":"swift","build-mode":"autobuild","runner":"macos-latest"}]}'
# Newline-delimited input (git diff without -z) is rejected, not misread.
if printf 'crates/gl/src/main.rs\n' | "$mapper" >/dev/null 2>&1; then
echo "FAIL newline-delimited input: mapper accepted it" >&2
failures=$((failures + 1))
fi
if [ "$failures" -ne 0 ]; then
echo "$failures ci-surfaces mapping check(s) failed" >&2
exit 1
fi
echo "ci-surfaces mapping: all checks passed"