Compare commits

...
Author SHA1 Message Date
Kevin Codex 15e0f4f629 feat(node): add --bulk mode to purge-spam for fast set-based cleanup
The per-repo cascade (one transaction per repo) stalls and drops connections
on small DB instances with large match sets (node2/node3, ~70k keep* mirrors).

--bulk deletes all matching repos and their child rows with a handful of
set-based statements in one transaction, then removes the on-disk bare repos
(collected before the delete). Skips Tigris (mirror rows have no archive).
Slug-keyed child tables match both the repo id and the {owner_short}/{name}
slug so canonical and mirror rows are both cleaned.

Tested: bulk_delete_by_name_regex removes both canonical + mirror matches and
their id-/slug-keyed children while leaving non-matching repos intact.
2026-07-01 09:17:41 +08:00
Kevin Codex f83c6689ef feat(node): admin purge-spam subcommand for bulk spam-repo cleanup
Adds `gitlawb-node purge-spam --regex <re> [--execute] [--limit N]
[--skip-tigris]`, a one-off maintenance command to remove repos whose name
matches a Postgres regex (e.g. '^keep[0-9]' for the keepNxNxN spam flood).

- Defaults to a dry run (reports match count + a sample); --execute deletes.
- Per repo: cascade-deletes DB rows (repos + all child tables, matching both
  the repo_id and the {owner_short}/{name} slug used by branch_cids/sync_queue/
  received_ref_updates/arweave_anchors), removes the on-disk bare repo, and
  deletes the Tigris archive (unless --skip-tigris). Batched, re-querying after
  each batch so no offset bookkeeping is needed.
- Runs instead of the daemon and exits (Config gains an optional subcommand).

DB layer: count_repos_by_name_regex, list_repos_by_name_regex,
delete_repo_cascade. Tested: regex precision (^keep[0-9] matches keepNxNxN but
not "keeper-tools") and cascade removal of child rows.

Cargo.lock: reconcile workspace crate versions to 0.4.0.
2026-07-01 09:17:41 +08:00
Kevin Codex 8d96cab4f7 chore(node): seed Gitlawb nodes in bootstrap-peers; ignore node1 fly config (#132)
- bootstrap-peers.json: add node2, node3, and manila (manila.gitlawb.com)
  to the canonical seed list alongside node.gitlawb.com, and normalize the
  Gitlawb operator label to "Gitlawb". DIDs left null — bootstrap merges
  peers on http_url and the parser treats did as optional.
- .gitignore: ignore infra/fly/node1.fly.toml. node1's production Fly
  config is pulled from Fly (fly config save), not tracked in the repo.
2026-06-30 21:59:56 +08:00
github-actions[bot] 0190fbbbf6 chore(main): release 0.4.0 (#131)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-06-30 21:09:16 +08:00
beardthelion 86341d195d test(node): guard that every repo-scoped handler is visibility-gated (#122)
* test(node): guard that every repo-scoped handler is visibility-gated

Mirrors the existing every_in_scope_mutation_has_its_gate source-string CI test on
the egress side: scrapes every repo-scoped handler (any pub/pub(crate) async fn
taking Path<(String,String..)>) and asserts it carries an authz marker, either a
read gate (authorize_repo_read / visibility_check) or a write/owner/self gate
(require_repo_owner / require_owner / did_matches / &auth.0), or is listed in
KNOWN_UNGATED with its tracking issue. Verb-agnostic on purpose: a repo-scoped read
named outside list_/get_ (git_info_refs, replicate_encrypted_blobs, withheld_paths)
is still checked, so a new ungated egress of any name fails CI instead of shipping
as the next leak.

A completeness scan over src/api/ asserts every module declaring a repo-scoped
handler is wired into the guard, so a whole new file cannot add an ungated handler
the scrape never looks at. KNOWN_UNGATED is the live gap list (certs/issues/labels/
bounties/stars per #120, plus events/webhooks/replicas/protected pending PR #113);
the staleness assert forces each entry out the moment its gate lands. A count floor
trips if the scrape silently breaks, and unit tests pin handler_names and
is_repo_scoped.

Verified by execution: removing an allowlist entry trips the missing-gate assert;
listing a gated handler trips the staleness assert; dropping a source file trips the
completeness scan; breaking the parser trips the count floor; and a non-list/get
handler (git_info_refs) is confirmed scraped and gate-checked.

* test(node): don't let a service-conditional gate satisfy the egress guard (#122)

The guard counted a handler as gated on bare substring presence of a gate
marker, so git_info_refs false-passed on visibility_check( even though that
check only runs under `if service == "git-upload-pack"` and leaves the
git-receive-pack advertisement ungated. Add gate_runs_unconditionally, which
ignores markers that sit only inside an `if service ==` block, and track
git_info_refs in KNOWN_UNGATED until #119 makes the gate unconditional. When
that lands, the staleness assert forces the entry's removal.

* fix(review): harden gate_runs_unconditionally against unclosed blocks (#122)

Code review of the egress-guard hardening:
- Clamp the span-advance index so an unclosed `if service ==` block can't
  slice past the body end and panic (latent: real source is balanced, but the
  scraper should not crash on a pathological future body). Keep span-to-EOF as
  the fail-safe direction.
- Document that only `if service ==` is detected, not `match service`, so a
  future match-arm gate isn't silently treated as full.
- Add unit cases: gate inside both service blocks (conditional), gate inside
  and outside (full), and the unclosed-block no-panic regression.
2026-06-30 20:48:03 +08:00
Kevin Codex 8b9ceec25e feat(node): iCaptcha-aware repo propagation gate with quarantine (#125)
Phase 2 of the iCaptcha work: extend the proof-of-intelligence gate from
direct writes to federated propagation, so a node that enforces iCaptcha
can defend against spam repos that originate on a node which does not.

The proof now travels with the repo and is re-verified at mirror time:

- Capture & persist: create_repo/fork_repo store the verified proof
  (raw token + claims) against the new repo. New repo_icaptcha_proofs table.

- Serve: GET /api/v1/repos/{owner}/{repo}/icaptcha-proof returns the stored
  token, gated on whole-repo "/" read like the other replication endpoints.

- Admit at mirror time: the sync worker fetches the origin's proof and
  icaptcha::admit_mirror re-verifies it offline — signature + level +
  owner-DID binding, but NOT expiry — and consumes the jti in the local
  ledger so one solved challenge admits at most one mirror per node.

- Quarantine: in enforce mode a mirror that fails the gate is kept on disk
  but hidden from clone/serve/push (authorize_repo_read + the git serve and
  receive-pack handlers) and from listings (dedup CTE + federated list), via
  a new repos.quarantined column. Shadow logs only; off admits normally.
  Fail-safe: off/shadow/no-key always admit.

Hardening from review:
- git-receive-pack also rejects quarantined mirrors (no writes while hidden).
- mirror admission keys "first-time" on DB-row absence, not on-disk presence,
  so a clone that crashed before the DB insert is still gated on retry.
- mirror jti is spent against a fixed forward retention window, not the
  proof's already-past exp, so the replay record isn't swept early.

Known limitation: an iCaptcha proof binds to the owner DID, not the repo, so
a malicious origin can get one spam repo per harvested public proof admitted
(per-node jti single-use bounds the amplification). Fully binding a proof to
a repo needs a repo-scoped iCaptcha challenge (protocol change, future work).
The operator release HTTP endpoint is deferred (no admin-auth surface yet);
DB methods are in place.

Tested: verify_token expiry-skip + signature/level/DID enforcement on the
propagation path; proof round-trip; quarantine set/release, preservation
across re-sync, exclusion from listings; mirror jti retention survives a
sweep at the proof's exp. Full workspace green against Postgres.
2026-06-30 20:46:54 +08:00
Kevin Codex 12bfb1b3c8 fix(release): use simple release-type + generic Cargo.toml updaters (#130)
release-type: rust at path . is incompatible with this virtual
workspace root. release-please's Rust strategy unconditionally appends a
CargoToml update for the root Cargo.toml, and that updater throws
"is not a package manifest (might be a cargo workspace)" whenever the
manifest has no [package] section — so every run died on the final step.

Switch to release-type: simple (touches only a version file + CHANGELOG,
never a manifest) and bump each crate version through the generic updater
via extra-files plus an inline x-release-please-version annotation.

Preserves the single vX.Y.Z tag, all action outputs, and the entire
downstream release.yml unchanged.
2026-06-30 20:01:42 +08:00
Kevin Codex 788a8686c1 fix(release): give crates explicit versions for release-please (#129)
release-please's Rust workspace strategy versions each member crate in its own
Cargo.toml and cannot rewrite an inherited `version.workspace = true`, so it
failed with 'value at path package.version is not tagged'. Give each of the five
crates an explicit version = "0.3.9" (kept in lockstep by release-please) and
drop the now-unused [workspace.package].version. Inter-crate deps are path-only,
so no version requirements need syncing.
2026-06-30 18:24:38 +08:00
beardthelion 174f25a206 fix(node): gate GET /ipfs/{cid} on per-caller path-scoped visibility (#110) (#128)
* fix(node): gate GET /ipfs/{cid} on per-caller path-scoped visibility (#110)

get_by_cid served any git object by raw SHA-256 with no auth and no
visibility check. For a public repo with a path-scoped deny rule, an
unauthenticated caller could recover a withheld blob's oid from the
served trees, derive its CID, and fetch the cleartext here, bypassing
the pack-path withholding.

Bind optional caller identity (optional_signature on a dedicated
/ipfs/{cid} sub-router; the pins route stays unsigned, tracked by #121)
and gate each iterated repo row against its OWN rules: visibility_check
at "/" denies the repo, and withheld_blob_oids skips a blob withheld
from the caller when the row carries path-scoped rules. The row is gated
directly rather than via authorize_repo_read, whose fuzzy get_repo
re-resolve could authorize a different physical row than the one read.
The per-repo withheld set is memoized on repo.id within the request.

Adds get_by_cid to the authz_guard drift table (visibility_check marker).

Scope: this closes the direct unauthenticated scan. A stale-public
mirror row still serves withheld content on both this path and the pack
path; that pre-existing leak is tracked separately in #124.

* fix(review): batch visibility-rule lookup and fail-closed walk panic in get_by_cid

Code-review follow-ups for #110:
- Replace the per-repo list_visibility_rules call with a single
  list_visibility_rules_for_repos batch query (the existing #97 pattern),
  removing the N+1 on an unauthenticated scan and the "one repo's DB error
  500s the whole request" semantics.
- Fail closed on a spawn_blocking JoinError (walk task panic) by skipping the
  repo, matching the walk-error arm, instead of 500-ing the whole scan.
- Add a regression test for the repo-level "/" deny branch (private repo,
  anon denied / owner served) and note the access gate in the module doc.

* test(node): cover the fail-closed walk-error arm in get_by_cid

Induce a withheld_blob_oids walk error (a ref pointing at a non-tree-ish
blob, same technique as visibility_pack::fails_closed_when_a_ref_cannot_be_traversed)
and assert the handler skips the whole repo: the withheld blob 404s with no
leak, and the public blob in the same repo also 404s (proving fail-closed-skip
rather than serve-on-error). Verified load-bearing: mutating the Ok(Err) arm to
treat the error as no-withholding makes it RED (secret served 200).
2026-06-30 18:16:33 +08:00
Kevin Codex 3f08d9936b Release automation (#127)
* ci(release): add Windows target and cross-platform packaging

Add x86_64-pc-windows-msvc to the release matrix. Windows ships a .zip
(7z) while unix keeps .tar.gz; a portable sha256 helper handles
sha256sum/shasum/certutil. Smoke test and packaging now resolve the
.exe suffix on Windows. Release attach uses globs to cover both formats.

* feat(release): vendor npm wrapper and auto-publish to npm

Vendor the @gitlawb/gl npm wrapper (optionalDependencies platform-package
pattern) into npm/, replacing the standalone gl-npm repo. A new npm-publish
job downloads the four unix release tarballs from the GitHub release, lays in
the binaries, stamps every package.json with the release version, and runs
npm publish --provenance. Guarded on NPM_TOKEN so it no-ops until the secret
is configured.

* feat(release): auto-bump Homebrew tap on release

Add a homebrew-bump job that regenerates Formula/gl.rb in Gitlawb/homebrew-tap
from the release tarballs' sha256 checksums (macOS+Linux, arm64+x64) and pushes
the bump. Guarded on HOMEBREW_TAP_PAT so it no-ops until the secret and tap repo
exist. Enables 'brew install gitlawb/tap/gl'.

* feat(release): Windows installer and web auto-sync on release

Add install.ps1 (Windows PowerShell installer, source of truth in node).
Add a web-sync job that opens a PR to Gitlawb/web on each release syncing
install.sh + install.ps1 + public/version.json, plus idempotent drift fixes
(remove orphaned public/bin, fix stale skill.md releases link). Guarded on
WEB_SYNC_PAT. Vercel deploys on merge.

* docs(readme): document npm, Homebrew, and Windows install methods

* ci(release): build only CLI binaries on Windows

The gitlawb-node daemon targets Linux/macOS; building it on Windows risks
a compile failure that would block the whole release (npm/brew/web jobs all
depend on release-binaries). Windows now builds only gl + git-remote-gitlawb;
the smoke-test and package loops follow the per-target binary list.

* fix(release): reconcile release-please manifest with v0.3.9 and bump action to v5

The manifest still said 0.3.8 while the only tag is a manual v0.3.9 and
Cargo.toml is 0.3.9, so release-please had no valid baseline (no v0.3.8 tag)
and never cut a release. Set the manifest to 0.3.9 so release-please diffs
v0.3.9..main and opens a proper release PR for the next version. Also pin
release-please-action to v5.0.0 to match the openclaude pipeline.

* ci(release): set explicit minimal permissions on homebrew-bump and web-sync

CodeQL flagged both jobs for lacking a permissions block, leaving GITHUB_TOKEN
at broad default scopes. Both only need contents: read (cross-repo writes use
dedicated PATs).

* fix(release): harden installers and workflow per review

- install.ps1: fail closed if the .sha256 cannot be fetched; pure ASCII (no BOM)
- npm install.js: fail (exit 1 / throw) when the platform payload or a binary is
  missing instead of reporting a successful but broken install
- workflow: top-level permissions {} + persist-credentials: false on read-only
  checkouts
- docs: PowerShell-only Windows note; text language tag on npm/README fenced block

* fix(installer): require core CLI binaries on Windows

install.ps1 only failed when zero binaries were present, so a partial archive
missing gl.exe could still report success. Require gl.exe + git-remote-gitlawb.exe
(gitlawb-node.exe stays optional, matching the Windows build) and fail early if a
required binary is absent.

* ci(release): make Windows non-blocking and stop persisting cross-repo PATs

- release-binaries: continue-on-error for the Windows leg so a failure there
  no longer fails the job or blocks npm/Homebrew (matches the best-effort intent)
- homebrew-bump/web-sync: persist-credentials: false on the PAT-backed checkouts;
  supply the token only on the git push via an x-access-token URL

* fix(release): gate web-sync on binaries and detect native arch in installer

- web-sync now needs release-binaries so the site never advertises a release
  whose artifacts failed to build
- install.ps1 reads PROCESSOR_ARCHITEW6432 before PROCESSOR_ARCHITECTURE so a
  32-bit PowerShell host on x64 Windows is not wrongly rejected

* ci(release): make npm publish idempotent across reruns

Skip any @gitlawb package version already on the registry so rerunning the
job after a partial publish no longer errors on the first existing package
and leaves the wrapper/platform packages out of sync.

* fix(release): harden web-sync and tap commit per review

- web-sync: verify the Windows ZIP + .sha256 exist on the release before
  syncing install.ps1/version.json (Windows is best-effort, so a failed leg
  must not make the site advertise a missing installer)
- stage generated files before the no-op check so brand-new untracked files
  (formula, version.json) are committed instead of skipped
- gh pr create: only swallow the 'PR already exists' case; surface auth/API errors
- npm README: scope npm/yarn/pnpm/bun install to macOS/Linux/WSL, point native
  Windows at the PowerShell installer

* ci(release): guard npm-publish before doing any work

Check NPM_TOKEN in an up-front guard step (like homebrew-bump/web-sync) and
skip checkout, release downloads, extraction, version rewrites, and publish
when it is absent, instead of doing all that work and bailing in Publish.

* fix(release): robust web PR error handling and Windows installer hardening

- web-sync: capture gh pr list output so a real API/auth failure surfaces
  instead of being misread as 'PR already exists'
- install.ps1: enforce TLS 1.2 and use -UseBasicParsing so downloads work on
  stock Windows PowerShell 5.1; remove binaries no longer shipped on upgrade so
  an in-place update can't leave a mixed-version install; handle an empty user
  PATH without a leading separator
2026-06-30 16:56:58 +08:00
Kevin CodexandClaude Opus 4.8 adc20f9eff feat(node): iCaptcha proof-of-intelligence gate on create_repo + register (#108)
* feat(node): iCaptcha proof-of-intelligence gate on create_repo + register

Add an optional gate that requires callers to present an iCaptcha proof (an
Ed25519-signed token from icaptcha.gitlawb.com, earned by solving an escalating
challenge) on the two spam-prone endpoints. Proofs are verified OFFLINE with
ed25519-dalek against the published public key, and bound to the authenticated
agent DID so they cannot be shared.

Backward-compatible by default: ICAPTCHA_MODE is `off` unless set, so deploying
changes nothing. `shadow` verifies and logs would-be rejections but always
allows; `enforce` rejects requests lacking a valid, strong-enough proof. If the
public key cannot be loaded at startup the gate stays inert (fail safe) so an
iCaptcha outage can never break repo creation or registration.

- New src/icaptcha.rs: env-driven verifier (OnceLock), /v1/pubkey fetch at
  startup or ICAPTCHA_PUBKEY override, proof parse + signature + expiry + level
  + DID-binding checks; mode-aware decide() split out for testing.
- Wired into create_repo and register (X-ICaptcha-Proof header). git push
  (receive-pack) is intentionally NOT gated.
- 11 unit tests incl. verification of a real captured proof (JS-signed,
  Rust-verified), tamper/expiry/wrong-DID/level/missing-header rejections, and
  the off/shadow/enforce/fail-safe mode matrix.

Author: Kevin Codex <[email protected]>

* fix(node): address review on icaptcha gate (validate config, bound key fetch)

Addresses CodeRabbit and @jatmn review feedback on #108:

- parse_mode now returns Option and rejects unrecognized ICAPTCHA_MODE values;
  init() warns and falls back to off instead of silently disabling the gate on
  a typo like "enforced". [P2]
- ICAPTCHA_REQUIRED_LEVEL: warn on a non-empty unparseable value instead of
  silently lowering the threshold to the default. [P2]
- fetch_key uses a reqwest client with a 5s timeout so a hung /v1/pubkey can
  never block node startup; still fails inert on timeout/error. [P2]
- .env.example: reorder ICAPTCHA_* keys (MODE, PUBKEY, REQUIRED_LEVEL, URL) to
  satisfy dotenv-linter. [P3]
- add parse_mode unit test (rejects junk, accepts documented values).

cargo test -p gitlawb-node icaptcha: 12 passed. clippy + fmt clean.

Author: Kevin Codex <[email protected]>

* fix(node): consume iCaptcha proof jti so a proof can't be replayed

Addresses @jatmn P2 on #108: the verifier checked exp/level/sub but never
consumed the proof's jti, so one solved challenge was a reusable bearer proof
for that DID until expiry, covering every gated create_repo/register in the
window. Now, in enforce mode, each verified proof's jti is recorded once and
replays are rejected.

- db: migration v7 adds icaptcha_consumed_proofs(jti PK, expires_at); new
  consume_proof_jti (INSERT ON CONFLICT DO NOTHING -> bool) and
  sweep_expired_proofs. DB-backed so it holds across restarts and instances.
- icaptcha: ProofClaims carries jti; verify returns the claims; decide() stays
  pure and returns Allow | Reject | Consume{jti,exp}; check() is now async and
  spends the jti via the DB, rejecting replays (enforce only; shadow observes
  without consuming).
- handlers: create_repo/register await check(&state.db, ...).
- main: periodic sweep of expired proof rows alongside rate-limit cleanup.
- tests: 12 pass incl. decide() yields Consume with the proof's jti.

Author: Kevin Codex <[email protected]>

* fix(node): consume iCaptcha proof only after request passes validation

Addresses CodeRabbit (Major) on #108: check() spent the jti as soon as the proof
verified, but create_repo/register ran it before name/DID/existence checks, so a
rejected request (bad name, already-exists, wrong DID) permanently burned a
valid proof without the action succeeding.

Split verification from consumption:
- icaptcha: replace check() with verify_request() -> ProofGuard (verifies and
  rejects invalid/missing proofs early; off/shadow/inert yield a no-op guard)
  plus ProofGuard::consume(db) which spends the jti right before the write and
  rejects replays. Guard is #[must_use].
- create_repo/register: verify_request() up front, then consume() immediately
  before the first write, after cheap validation passes.

Replay protection unchanged (consume-before-write). 12 tests pass; clippy clean.

Author: Kevin Codex <[email protected]>

* fix(node): give iCaptcha rejections a dedicated error code

Addresses CodeRabbit on #108: iCaptcha failure paths returned
AppError::Unauthorized, which maps to the "not_an_agent" API code and conflates
a valid-but-unproven agent with a failed signature. Add a dedicated
AppError::IcaptchaProofRequired variant mapped to the "icaptcha_proof_required"
code (HTTP 401), and use it for both the missing/invalid-proof and replay paths
so clients can distinguish "solve a challenge" from "your signature is invalid".

12 tests pass; clippy + fmt clean.

Author: Kevin Codex <[email protected]>

* fix(node): gate fork_repo with iCaptcha and cover the proof ledger

Address remaining PR #108 review comments:

- fork_repo was the third repo-creation entrypoint and bypassed the
  iCaptcha gate. Verify the proof up front (fail fast on missing/invalid)
  and consume it only just before the git clone, so cheap rejections
  (bad name, conflict, withheld subtree) never burn a valid proof —
  matching create_repo's verify/consume split.

- Add #[sqlx::test] coverage for the single-use ledger that the gate's
  anti-replay value depends on: consume_proof_jti (first-use vs replay vs
  distinct jti) and sweep_expired_proofs (only expired rows removed,
  unexpired spent proofs still block replays).

- Return 403 Forbidden (was 401) for IcaptchaProofRequired: the caller is
  authenticated but forbidden without a valid proof. The distinct
  icaptcha_proof_required code is unchanged.

- Recommend pinning ICAPTCHA_PUBKEY in production in .env.example to
  avoid the startup fetch dependency.

The two shadow-mode review comments were already resolved by the earlier
verify/consume refactor (decide/verify are IO-free; check_proof_jti gone).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-30 10:58:08 +08:00
beardthelion 828dd279a2 fix(node): gate repo-listing and stats surfaces on visibility (#97, #99, #101, #104) (#111)
* fix(node): NFC-normalize visibility matcher to close NFC/NFD under-withholding (#101)

A deny rule authored in NFC byte-compared unequal to a path committed in NFD
(or vice versa), so blob_paths/withheld_blob_oids failed to classify the blob
and it shipped in cleartext on the replication/pack path. Normalize both the
glob prefix and the candidate path to NFC at the single matcher seam
(glob_matches + specificity); NFC not NFKC so compatibility forms are not
folded into distinct paths. Live serve is unaffected (git resolves byte-exact;
the gate only becomes more restrictive). Adds a matcher unit test and a
replication-path integration test with the variant byte inside the
rule-covered directory name.

* fix(node): fail closed on the full-scan pin path to stop dangling-blob leak (#99)

The withheld set is computed over reachable objects (rev-list --all + HEAD),
but the full-scan pin fallback (cat-file --batch-all-objects) includes dangling
objects the reachable walk never classified. Subtracting the reachable-only
withheld set therefore could not catch a private blob orphaned by a
force-push/amend, which pinned to IPFS/Pinata in cleartext.

Invert the default for blobs on the full-scan path: a candidate replicates only
if it is a non-blob (commits/trees are structural) or a reachable,
visibility-allowed blob; anything else (a dangling blob, or a withheld one) is
dropped. The delta path keeps the cheaper reachable-only subtraction since delta
candidates are always reachable, so no extra walk on the hot path.

Adds list_all_objects_with_type/all_blob_oids (typed enumeration),
replicable_blob_set + replicable_objects_fail_closed, and threads a full_scan
flag through resolve_candidates_for_push. Pin functions now take a pre-filtered
object list. Regression tests: dangling blob excluded end-to-end, fail-closed
unit semantics, and the all-blob universe includes dangling + excludes
non-blobs.

* fix(node): gate repo-listing surfaces on visibility to stop private-repo enumeration (#97)

list_repos, list_federated_repos, and the GraphQL repos query returned every
repo row with no visibility check, so an unauthenticated caller could enumerate
private repos' name, owner, description, and clone URL — and X-Total-Count
leaked the private-repo count even when rows were filtered.

Each surface now keeps a row only when visibility_check at "/" allows the
caller, the same decision the per-repo content endpoints make — not a bare
is_public filter, which would diverge for is_public=false-with-root-reader and
is_public=true-with-root-deny. The "/" decision depends on owner short/full-DID
matching and JSON reader-DID membership, so it is applied in Rust over an
over-fetched set rather than pushed into SQL; the paged path derives
X-Total-Count from the filtered set (dropping the SQL window count) and
paginates after filtering, so neither the page nor the count leaks. Adds a
batch root-rule query (list_visibility_rules_for_repos) to avoid N+1, and
converts list_all_repos_paged into list_all_repos_deduped (mirror dedup
preserved, pagination moved to Rust).

Tests: anonymous hides private repo + count, owner sees own private repo,
authorized root-reader sees an is_public=false repo (proving the visibility_check
path, not is_public), and federated listing hides private from anonymous.

* fix(review): share the listing visibility seam, extract fail-closed pin helper, add listing tests

Code-review follow-up on the VIS-subtree fixes (no behavior change to the three
shipped fixes; all confirmed sound by review):

- DRY the '/' listing gate into one shared visibility::listable_at_root used by
  list_repos, list_federated_repos, and the GraphQL repos query, so the three
  surfaces enforce one rule instead of three drifting copies
  (enforce-shared-rule-on-every-reader-path).
- Extract the full-scan fail-closed pin block into fail_closed_full_scan_objects,
  mirroring replication_withheld_set's degraded-path shape; drop the duplicated
  comment and the per-site no_rules sentinel (use &[]).
- Add the three listing tests the review flagged as missing: anonymous GraphQL
  repos hides a private repo, the paged X-Total-Count path excludes the private
  count (the KTD2 exploit shape), and an is_public=true repo under a root deny is
  not listed (proves the gate is visibility_check, not a bare is_public filter).

* fix(review): route the last inline root-visibility check through listable_at_root

Round-2 review follow-up. replication_withheld_set still made the announce
decision with an inline visibility_check(.., "/") == Allow — the fourth copy of
the listing gate the seam refactor was meant to collapse. Route it through
crate::visibility::listable_at_root (caller is always None, the anonymous
replication audience), so every reader of the '/' decision now shares one seam.
Also drop an em dash from the fail_closed_full_scan_objects doc comment.

* fix(node): gate the /api/v1/stats repos count through visibility to close the count oracle (#104)

The stats endpoint returned count_repos_deduped(): an unfiltered, unauthenticated
count of every logical repo including is_public=false and mode-A hide-existence
repos. A polling observer could detect that a hidden repo was created (N to N+1),
the exact existence signal mode-A suppresses everywhere else.

Count only the repos an anonymous caller could list: over-fetch the deduped set,
batch-load visibility rules, and keep rows that pass listable_at_root(.., None),
mirroring the listing seam in api/repos.rs. The caller is always None (meta_routes
carries no auth layer). Fail closed: any DB error collapses the count to 0, since
an under-count cannot leak existence.

count_repos_deduped is retained as the tested reference for the DEDUP_CTE dedup
count (its tests pin the CASE the live list paths share); it has no production
caller after this change and is allowed dead outside tests.

Tests: bare-private (no-rule branch), hide-existence mode-A with both repos
is_public=true (the Some(rule) branch), public-under-root-deny, list-parity,
sibling-fields-preserved, and empty-DB.

* test(review): add listing visibility regression guards (#97)

Three gaps the review flagged as uncovered:
- owner-filtered listing (?owner=) still hides a private repo and its count
  from anonymous (a distinct SQL bind branch from the unfiltered path).
- offset past the visible set returns an empty page while X-Total-Count stays
  the full visible total (guards against deriving the total from the cut page).
- a logical repo present as both a canonical row (with a root-deny rule) and a
  gossip mirror row stays hidden: pins that the DEDUP_CTE picks the canonical
  survivor so the batch rule lookup finds the deny. If dedup ever picked the
  mirror (slash-form id, no rule) the gate would fall back to is_public and leak.

* fix(review): address CodeRabbit feedback on #111

- visibility_pack NFC/NFD test: accept a SHA-1 or SHA-256 object id
  (40 | 64) instead of hardcoding 40, matching the fixture guard later in
  the file, so the test is not SHA-1-only.
- federated listing test: also assert the response `count` reflects only
  the visible repos, guarding against a regression to the pre-filter total.
- add an authenticated-caller GraphQL test: an owner sees their own private
  repo via `repos`, covering the resolver's caller-DID path that the
  anonymous-only test did not exercise.
- stats: count via the no-stars `list_all_repos_deduped()` (same DEDUP_CTE)
  instead of the stars-joined variant, since the count never needs stars.

* fix(review): match owner filter to did_matches for bare-owner mirror rows (#111)

Addresses jatmn's P2. The shared listing CTE filtered owners with
`owner_did = $1 OR owner_did LIKE '%:' || $1`, so a full `did:key:z...`
query did not match a mirror-only row stored with the bare owner `z...`.
#97 routed the no-limit owner-filtered path (used by `gl repo list`)
through this SQL predicate, regressing the full-DID-against-bare-mirror
direction that `crate::api::did_matches` handled in the old Rust path.

Normalize the bound owner to its did:key short form and compare it against
the row's owner key using the same CASE the dedup grouping already applies,
so full and short forms match symmetrically and a non-key DID still matches
only exactly. The no-arg `list_all_repos_deduped()` binds NULL and is
unaffected.

Regression test: a bare-owner mirror-only row is returned when queried by
both the full `did🔑` form and the bare short form. Confirmed RED
(test fails) before the fix.
2026-06-30 08:30:32 +08:00
beardthelion 6ae316cc88 fix(node): gate fork_repo on per-caller path-scoped visibility (#98) (#109)
* fix(node): gate fork_repo on per-caller path-scoped visibility (#98)

fork_repo authorized the caller only at the repo root "/" and then
git clone --mirror'd the full source, discarding the visibility rules.
A non-owner with root read but denied a path-scoped subtree could fork
the full mirror and obtain blobs the filtered git_upload_pack path
withholds.

Refuse the fork when the caller has any withheld glob (per-caller
withheld_globs non-empty), the same decision the read path makes;
owners and authorized subtree readers are unaffected. Fail closed with
a not-found response, before repo_store.acquire, so no withheld object
is materialized. Add unit coverage for the policy and a direct-mount
integration test pinning the gate wiring.

* fix(review): note fork-gate glob-grammar dependency, import withheld_globs

Address code-review findings on the #98 fork gate:
- Document that fork_withheld_blocks's prefix-probe equivalence with the
  serve path depends on validate_path_glob keeping "/" the only whole-repo
  scope, mirroring has_path_scoped_rule's caveat (latent-coupling guard).
- Import withheld_globs and call it unqualified, consistent with the
  sibling visibility_check import in the same module.
2026-06-28 09:01:13 +08:00
beardthelion 6809201daa fix(node): anchor the real old_sha and issue a per-ref certificate (#72)
* fix(node): anchor the real old_sha and issue a per-ref certificate (#26)

The push handler discarded the real old->new transition before anchoring
and only certified the first ref of a multi-ref push, so the permanent
Arweave record was a weaker, incomplete copy of facts the node had
already parsed.

- Thread the parsed old_sha through the clone moved into the anchor task
  instead of hardcoding a 64-zero placeholder. This also fixes the SHA
  width mismatch: object ids are 40-hex SHA-1, not 64-hex.
- Issue a signed RefCertificate for every ref the push advanced, each
  carrying that ref's real old/new, rather than one cert for the first
  ref (which, on an empty push, signed a timestamp string as new_sha).

Lock the anchor-body contract with a test asserting the upload carries
the real old_sha and new_sha, never a placeholder.

Scope is the integrity subset only. Carrying the signed certificate on
Turbo, capturing the pusher signature, chaining certificates, and
gl-side verification remain open under #26.

Refs #26

* fix(node): thread real old_sha through every ref-update fan-out path

PR #72 made the Arweave/DB anchor path per-ref with the real old_sha but
left three sibling fan-out paths in git_receive_pack flattening multi-ref
pushes and/or emitting a placeholder old_sha. Bring them in line:

- gossip publish_ref_update now forwards the real old_sha (was ""), matching
  the RefUpdateEvent contract that documents all-zeros for a created ref
- HTTP /sync/notify sends one signed request per ref with its real old_sha;
  the receiver is single-ref, so a multi-ref push no longer drops refs 2..N
- GraphQL RefUpdateBroadcast emits one broadcast per ref with the real
  old_sha instead of only the first ref + a zero placeholder

No receiver, struct, or wire-schema change; the all-zeros-for-created-ref
convention is preserved on the wire. Addresses jatmn's CHANGES_REQUESTED and
the two CodeRabbit findings.

* test(node): regression-guard the per-ref sync-notify fan-out

Extract the HTTP /sync/notify send into notify_peer_of_ref (one ref) and
notify_peer_of_refs (one peer, looping refs), collapsing the deeply-nested
inline block in git_receive_pack and giving the fan-out a unit-testable seam.
The ref_name is now also logged on every warn arm.

Add mockito regression tests: a two-ref push must emit one request per ref
each carrying its real old_sha (guards against re-flattening to .first() with
a zero placeholder, the bug PR #72 fixed), and a created ref must forward the
all-zeros hash verbatim. Behavior is unchanged; this only adds the seam, the
tests, and ref_name in the logs.

* style(node): match sibling test conventions in repos tests

Prefix the two new sync-notify tests with test_ and use the _mock binding
convention (matching arweave.rs/pinata.rs). No behavior change.

* style(node): fix rustfmt import order and dedupe the zero-sha literal

- Reorder the repos test-module imports so `use crate::...` precedes the
  external `gitlawb_core` import (unblocks cargo fmt --check / CI).
- Webhook `created` check reuses the existing ZERO_SHA constant instead of
  repeating the 40-hex literal.
- Arweave test fixtures use the canonical 40-hex zero old_sha, matching the
  SHA-1 width the per-ref anchoring contract now carries.

* perf(node): run peer notification after local broadcast and anchoring

The post-push background task awaited the HTTP peer-notify fan-out before
the GraphQL subscription broadcast and Arweave anchoring. With per-ref
notification a slow or unreachable peer could delay both local subscribers
and permanent anchoring by the sum of the peer round-trips. Move the
peer-notify loop to run last; peers already learned of the push via the
gossip publish earlier in the task, so this only deprioritizes the
best-effort HTTP fallback.

* style(node): correct notify-ordering comment and reuse ZERO_SHA in test

Review follow-ups on the two prior commits:
- The peer-notify ordering comment claimed peers 'already learned via the
  gossip publish', but that publish is gated on a present p2p handle; with
  no handle (the very case the HTTP fallback exists for) it does not run.
  State the actual rationale: peer-notify is the lowest-priority best-effort
  step, ordered last so it cannot delay local broadcast or anchoring.
- The all-zeros notify test still inlined the 40-hex literal; point it at the
  ZERO_SHA constant like the production path.
2026-06-27 23:16:23 +08:00
beardthelion 3e1e9045e4 fix(node): close under-withholding via full ref scope and full-history classification (#42) (#84)
* fix(node): close under-withholding via full ref scope and reachable-only pin set

blob_paths walked only refs/heads/* and refs/tags/* and skipped silently on a
failed git ls-tree, so a blob reachable only through another namespace, or a ref
that failed to traverse, could fall out of the withheld set and ship in cleartext.
Walk every ref and fail closed on traversal error.

The pin enumerators (ipfs_pin, pinata) used git cat-file --batch-all-objects,
which includes unreachable/dangling objects that have no path and cannot be
classified for withholding. Switch them to git rev-list --objects --all so the
pin set matches the reachable graph blob_paths evaluates.

* fix(node): include HEAD in the withheld-blob walk to close pin/serve asymmetry

blob_paths classified objects from for-each-ref only, but the pin set
(git rev-list --objects --all) and git upload-pack both also expose
objects reachable from HEAD. A detached HEAD (or any HEAD pointing
outside the ref-covered set) therefore put a blob into the pin/serve
path that the withholding walk never classified — a latent cleartext
leak of a withheld blob. Walk HEAD too (guarded for unborn HEAD on an
empty repo) so the withheld set is a superset of every exposure path.

Adds a regression test: a blob reachable only via a detached HEAD must
still be withheld (fails without this change).

* fix(node): classify blobs across full history, not just ref tips

blob_paths walked only ref-tip trees (git ls-tree -r <ref>), so a blob
that existed in an older commit and was deleted or rotated at the tip
was never classified, while the pin set (git rev-list --objects --all)
and upload-pack still expose it from history. That under-withheld the
blob into the IPFS/Pinata replication paths in cleartext.

Enumerate all reachable commits (git rev-list --all + HEAD for the
detached case) and walk each commit tree, deduping (oid, path) pairs.
ls-tree per commit is kept over rev-list --objects so every path a blob
lives at is still represented for the per-path visibility check. A ref
that resolves to a non-commit now fails closed explicitly, since
rev-list --all silently skips it where the old per-ref ls-tree errored.

Regression test: a secret blob deleted at the tip but reachable in
history is withheld.

* fix(node): share one history walk and fully peel ref types in withheld classifier

Addresses two review findings on the under-withholding classifier:

- withheld_blob_recipients walked blob_paths twice (once via
  withheld_blob_oids, once directly). Extract withheld_from_pairs so a
  single history walk feeds both the withheld set and the recipient map.

- The ref-type fail-closed guard used for-each-ref %(*objecttype), which
  peels only one tag level and so misclassified a nested annotated tag of
  a commit as a non-commit, refusing the whole walk (over-withhold). List
  refnames and peel each fully with <ref>^{} through one cat-file
  --batch-check; reap the child even if the stdin write fails. Fail closed
  on a short read or any non-commit/missing target.

Adds tests: annotated + nested-tag of a commit does not fail closed,
annotated tag of a blob still fails closed, and a blob at both an allowed
and a denied path is not withheld.

* fix(node): drain cat-file stdout concurrently to close the peel deadlock

The ref-type guard peeled every <ref>^{} through git cat-file --batch-check,
writing all queries to stdin before draining stdout. On a repo with enough
output the child's stdout pipe fills, it stops reading stdin, and write_all
blocks forever — hanging every serve/pin that classifies the repo. Batching
the queries does NOT bound this: cat-file echoes the full query on a
'<query> missing' line, so output scales with refname length, and a few
hundred long-named dangling refs already exceed the pipe buffer.

Run the stdin write on a separate thread so the main thread drains stdout
concurrently via wait_with_output, which reaps the child unconditionally
before any error is surfaced; the writer is joined after the drain so it
cannot deadlock, and a vanished stdin pipe becomes a broken-pipe write error.
One pass, no batch bound, deadlock-free regardless of ref count or refname
length.

Extracted as assert_all_refs_are_commits with corrected doc attribution.
Detect the missing-object line by its trailing status word. Adds a
timeout-bounded regression test with 500 long-named dangling refs (>64 KiB of
cat-file output) that fails rather than hangs if the deadlock returns.

* fix(node): parse ls-tree -z raw paths and fail closed on non-UTF-8 to stop under-withholding

Plain `git ls-tree -r` C-quotes any path with non-ASCII or special bytes
(café.txt becomes "secret/caf\303\251.txt"), so the stored path no longer
matched a visibility rule like /secret/** and the blob was served/pinned in
cleartext. Switch blob_paths to `git ls-tree -rz` and split the NUL-delimited
records, which emit paths raw; the TAB field separator survives -z so the
per-record parse is otherwise unchanged.

Parse strictly with str::from_utf8 and fail closed on invalid UTF-8 rather than
from_utf8_lossy: a lossy decode would rewrite an invalid byte in a denied
directory name to U+FFFD, and the mangled path would again miss its deny rule
(the same under-withholding class one layer down).

Tests: a non-ASCII path stays withheld (and the public blob still replicates),
and a non-UTF-8 path fails closed.
2026-06-27 23:15:38 +08:00
beardthelion 1960f6b28f ci(triage): label fork PRs and stop auto-closing PRs that only lack an issue link (#107)
* ci(triage): run PR triage on pull_request_target so fork PRs can be labeled

Fork PRs run the triage job with a read-only GITHUB_TOKEN under the
pull_request event regardless of the workflow's declared write
permissions, so addLabels/createComment fail with 403 'Resource not
accessible by integration' for every external contributor (e.g. #103).

Switch the trigger to pull_request_target, which runs in the base-repo
context with the declared write token. The job never checks out or runs
PR head code (it only reads context.payload and the pulls/files API and
manages labels), so it carries none of the untrusted-code-execution risk
that makes pull_request_target dangerous.

* ci(stale): drop needs-issue from auto-close trigger, keep it advisory

needs-issue flags PRs that link no issue, but linking one is a SHOULD in
CONTRIBUTING (required only for protocol-level changes), and many
legitimate small PRs (docs, obvious fixes) have none. Coupling it to the
14/21-day auto-close was overreach: a substantive, well-described PR
could be closed purely for missing an issue reference.

Keep needs-issue as an advisory label and guidance nudge; leave only
needs-description (an empty/thin body, a far stronger low-effort signal)
as the auto-close trigger.

* ci(stale): align stale message with description-only close condition

The auto-close trigger narrowed to needs-description, but the stale
message still told contributors a PR could close for a missing linked
issue. Drop that clause so the warning matches what actually closes a PR.

* style(ci): replace em dashes in triage/stale contributor messages

* docs(contributing): match stale close policy to description-only trigger

Auto-close is driven by needs-description only; a missing issue link stays
advisory. Align the contributor docs with the workflow so the OR wording no
longer implies a missing issue link can close a PR.
2026-06-27 23:14:41 +08:00
beardthelion abc9ad03ac fix(node): fail closed when a recipient DID can't be resolved (#47) (#67)
* fix(node): fail closed when a recipient DID can't be resolved (#47)

encrypt_and_pin resolved recipient DIDs with filter_map, silently
dropping any that failed to resolve and sealing the envelope to the
remaining subset while still recording the full intended recipient set
as covered. A reader whose DID could not be resolved at seal time (any
did:web/did:gitlawb, or a malformed did:key) was then permanently locked
out: the dedup compares the recorded full set against the desired set,
matches, and never re-seals.

Resolve all recipient DIDs up front via resolve_all_recipients; if any
fail, log and skip the blob rather than seal to a partial set. Nothing
partial is recorded, so the dedup stays honest and the blob re-seals on
a later push once resolution is available. Add a bounded per-blob warn
plus one aggregate coverage warn so a fully-migrated did:gitlawb org
(zero encrypted-pin coverage) is one greppable line, not a scrape, and
give the previously silent read_object and pin_git_object failure arms
warn logs.

* test(node): extract plan_seal so the #47 fail-closed gate is unit-tested

The fail-closed invariant lived only in resolve_all_recipients, tested in
isolation; nothing proved encrypt_and_pin acts on it (a refactor falling
through to a partial seal would pass every test). encrypt_and_pin needs a
live Postgres Db, which CI does not provide, so it can't be tested directly.

Extract the per-blob seal/skip decision into a pure plan_seal() -> SealPlan
(no DB, no IO), behaviour-preserving, and unit-test its branches: seals only
when all recipients resolve, fails closed on any unresolvable DID, skips the
empty set, skips an unchanged tag, re-seals a changed set.

* style(node): rustfmt encrypted_pin (Seal variant + assert_eq wrap)
2026-06-26 10:33:13 +08:00
beardthelion 338ff83584 fix(node): skip withheld-walk when no path-scoped rule can withhold (#60)
* feat(node): add has_path_scoped_rule predicate for withheld-walk short-circuit

Shared predicate (rules.iter().any(path_glob != "/")) both serve and
replication paths will use to skip the per-blob withheld walk when no rule
can withhold. Doc records the "/"-gate precondition and the validate_path_glob
dependency. Adds unit tests plus a gate-aware safety invariant test.

* fix(node): skip withheld-walk when no path-scoped rule can withhold (#50)

The serve path (git_upload_pack) ran the full per-ref withheld_blob_oids
walk on every clone/fetch before falling back to a plain upload_pack, even
for repos with no path-scoped rules where the walk can never withhold
anything. Cost grew with #42 (per-ref git ls-tree -r).

Guard both call sites on has_path_scoped_rule: the serve path skips the
spawn_blocking walk entirely and serves upload_pack directly; the
replication path generalizes its is_empty() short-circuit to also cover
root-only rule sets. Safe because the whole-repo "/" gate runs before both
sites, so a denying root rule 404s/withholds upstream and never reaches the
predicate. No change to served-pack contents on the success path.

* fix(node): gate encrypt-then-pin walk on has_path_scoped_rule (#50)

The post-push encrypt-then-pin block in git_receive_pack gated the
withheld_blob_recipients call on `!r.is_empty()`, which is true for a
root-only `/` rule set. withheld_blob_recipients then runs the same
per-ref `git ls-tree` walk (withheld_blob_oids) the short-circuit is
meant to avoid, and returns an empty recipient map anyway. Gate it on
has_path_scoped_rule to match the serve and replication sites; the skip
is behavior-preserving (empty map -> encrypt_and_pin is a no-op).

Also align the two remaining call sites to the visibility_pack import
alias for consistency.

* test(node): cover the serve decision across both has_path_scoped_rule branches

Drives the git_upload_pack serve decision over a real bare repo for the
INV-2 caller (a reader allowed at whole-repo "/" but denied a path-scoped
subtree), exercising both branches the predicate selects: root-only rules
skip the walk and serve the full pack (asserting the walk would withhold
nothing, so the skip is sound), and a /secret/** rule runs the walk and
excludes the secret blob from the served set while keeping the public one.
2026-06-26 10:32:43 +08:00
beardthelion 636d3abe17 chore(deps): bump tar to 0.4.46 and rand to 0.9.3 to clear advisories (#105)
tar 0.4.45 had a PAX header desynchronization issue (GHSA-3pv8-6f4r-ffg2,
fixed in 0.4.46); rand 0.9.2 was unsound with a custom logger using
rand::rng() (GHSA-cq8v-f236-94qc, fixed in 0.9.3). Both resolve as
lockfile-only bumps with no manifest change. Build and cargo audit pass.

lru 0.12.5 (GHSA-rhfx-m35p-ff5j) stays pinned by aws-sdk-s3 (lru ^0.12.2)
and the hickory advisories remain accepted risk per .cargo/audit.toml (#76).
2026-06-26 10:32:09 +08:00
beardthelion 2f6611a30f fix(core): zeroize the derived X25519 secret (#65) (#91)
x25519_secret_from_seed derived the X25519 private scalar from the Ed25519
seed and returned it as a bare [u8; 32]. Because [u8; 32] is Copy with no
Drop (and sha2's digest output likewise has no zeroize), the scalar, the
SHA-512 digest, and the returned temporary were released without being
scrubbed, leaving secret-derived material in freed memory. This is the same
Copy-no-Drop class fixed for the raw seed in #41, on the derived secret.

Return Zeroizing<[u8; 32]>, build the scalar directly into a zeroizing
buffer so no bare secret local persists, and explicitly wipe the SHA-512
digest before it drops. Callers deref the result into crypto_box::SecretKey,
which already scrubs its own copy. No behavior change: the derived scalar and
all decryption output are byte-identical, covered by the existing
ed25519_to_x25519_keypair_agrees and seal_open_round_trip_for_recipients tests.
2026-06-25 08:31:32 +08:00
beardthelion 3e8e333aa0 fix(node): dedupe mirror and canonical repo rows on list surfaces (#6) (#73)
* fix(node): dedupe mirror and canonical repo rows on list surfaces (#6)

The non-paged GET /api/v1/repos legacy path and list_federated_repos
returned both the short-owner peer mirror row and the canonical did:key
row for the same logical repo, so profiles rendered the repo twice. Only
the paged path collapsed them, in SQL.

Add a dedupe_canonical_repos helper that groups by (normalized owner,
name), keeps the canonical non-mirror row (tie broken by earliest
created_at), and carries the group's latest updated_at onto the
survivor, matching the paged SQL dedup. Apply it at both non-paged
surfaces and cover it with unit tests.

* fix(node): dedupe GraphQL + stats repo surfaces and key mirror detection on the repo id

Addresses jatmn's review on #73 (dedup not applied on every reader path; mirror
detection keyed on a user-settable string).

- GraphQL repos and /api/v1/stats now collapse mirror+canonical rows, via new
  Db::list_all_repos_deduped and count_repos_deduped that share the DISTINCT ON
  CTE with list_all_repos_paged so the dedup rule cannot drift.
- Mirror detection keys on the structural slash-form id (written only by
  upsert_mirror_repo) instead of the description == 'mirrored from peer' string,
  in both the SQL paths and dedupe_canonical_repos.
- Deterministic survivor on a full created_at tie (id ASC) in both
  implementations.
- Legacy REST list and federated keep their method-scoped did_matches owner
  filter in Rust; it does not compose with the method-blind SQL group key, so
  those paths intentionally stay on the Rust helper.
- Adds sqlx and unit tests for the new surfaces, the structural marker, and the
  tiebreak.

* test(node): cover mirror-only, empty, and count/list-consistency dedup cases

Follow-ups from code review on the dedup change:
- list_all_repos_deduped/count_repos_deduped: mirror-only group survives, empty
  table returns empty/0, and count_repos_deduped equals the deduped list length
  (guards the two independent SQL queries against grouping-key drift).
- Document list_all_repos as the raw, non-deduped enumeration path (object
  lookup only), so it is not mistaken for a listing-surface method.

* style(node): apply cargo fmt to dedup test code

* fix(node): make the repo dedup key did:key-aware so distinct DID methods don't collapse

The dedup grouping key took the last ':' segment of owner_did
(split_part / rsplit), so two repos owned by did:key:X and
did:gitlawb:X with the same name collapsed into one logical repo on
the list, paged, count, stats, and GraphQL surfaces. That is the exact
cross-method collision the codebase already guards against in
did_matches.

Replace it with a did:key-aware key that strips a did🔑 prefix only
when the remainder is a bare id (no ':'), otherwise keeps the full DID,
reproducing did_matches/key_id as an equivalence relation: did:key:X
and a bare mirror X still collapse, while distinct methods never merge.
Applied byte-identically across DEDUP_CTE (DISTINCT ON / PARTITION BY /
ORDER BY), count_repos_deduped, the empty-page count fallback, and the
in-memory dedupe_canonical_repos, so the SQL and Rust paths agree.

The backing index lived in the already-released migration v1, so v1 is
left untouched and a new migration v7 drops idx_repos_owner_short_name
and builds idx_repos_owner_key_name on the matching expression; the
CASE must stay byte-identical to the queries or Postgres stops using it.

Tests cover both the in-memory and SQL paths: distinct methods stay
separate, bare-id and did:key forms collapse, and the residual-colon
guard keeps a malformed did:key:did:gitlawb:X distinct from the bare
method DID. 216 pass.
2026-06-25 08:25:42 +08:00
beardthelion f28fa02236 fix(security): gate webhook creation through the public-host validator (#81) (#92)
Webhook creation only scheme-checked the URL, so an authenticated repo
owner could register a hook pointed at loopback, private, or metadata
endpoints and make the node POST to them on repo events. Route the URL
through is_public_http_url, the same validator the peer announce path
already uses.

Wiring it up surfaced a bypass in that shared validator: it folded
IPv4-mapped and compatible IPv6 but not 6to4 or NAT64, so [2002:7f00:1::]
(127.0.0.1) slipped through. Fold 6to4 and the NAT64 well-known prefix,
reject the NAT64 local-use prefix and the rest of 64:ff9b::/32 outright
(the embedded v4 offset varies by prefix length, so a decode would be
error-prone), and reject 0.0.0.0/8. The change only ever rejects more,
so the peer announce handler, the DB-boundary writer, and the boot-time
prune inherit the same hardening.

Closes #81.
2026-06-25 08:24:34 +08:00
beardthelion 1af4fdf485 feat(node): pin the per-push object delta instead of re-enumerating the whole repo (#90)
* feat(node): add push-delta object enumeration module

Compute the per-push object delta (git rev-list --objects <new> --not <old>)
for the pin path instead of enumerating the whole repo, with a fully-peeled
ref-type guard (cat-file -t '<sha>^{}') that fails closed to a full scan on a
non-commit tip, any git error, or the GITLAWB_PIN_FORCE_FULLSCAN kill-switch.
The whole-repo lister moves here for the fallback and the reconciliation sweep.

Pin-enumeration scope only (N16); the withheld filter is unchanged.

* feat(node): pin the per-push object delta instead of the whole repo

Wire the push-delta enumeration into the IPFS and Pinata pin paths so pin work
scales with push size, not repo size (N16).

- U2: both pin_new_objects twins take a candidate OID set instead of scanning
  the whole repo internally; repo_path is retained for reading object bytes.
- U4: extract the anonymous replication withheld gate (caller=None, no caller
  param) into a shared helper returning (announce, withheld), so the push path
  and the future reconciliation sweep cannot drift. The per-caller read-serve
  withheld call in git_upload_pack is deliberately left separate.
- U3: the post-receive handler resolves the push delta once (off-thread),
  falls back to a full scan on a non-commit tip / git error / kill-switch, and
  passes the candidate set to both pin paths. Logs delta size and fallbacks.

The reconciliation sweep (U5) is deferred: it depends on the history-deep
blob_paths fix (#84) landing on main first.

* fix(review): log degraded pin-candidate paths, extract wiring to push_delta

Code review found the push candidate computation collapsed both a failed
full-scan fallback and a panicked spawn_blocking task to a silent empty set,
diverging from the sibling withheld path that logs and fails closed.

Extract resolve_candidates_for_push into git::push_delta (it owns the
PinCandidates dispatch and full-scan fallback), logging every degraded path
(fallback taken, fallback failed, task panicked) instead of failing silently.
Pinning nothing on a hard failure is still safe (the withheld filter always
runs; the reconciliation sweep backstops durability) but is now observable.
Moving the dispatch out of the HTTP handler also makes it unit-testable.

Add tests: tag-of-tag-of-commit peels to a Delta, and resolve_candidates_for_push
delta + non-commit-tip full-scan-fallback paths.

* fix(node): fail announce closed when withheld walk fails (#90)

A failed or panicked withheld_blob_oids walk left announce=true while
withheld=None, so a push we could not vet would still gossip ref updates,
notify peers, and write a permanent Arweave anchor. Force announce false
in that arm so the gate fails closed on both axes.
2026-06-24 11:05:19 +08:00
beardthelion 3d880cd56f fix(node): reject malformed path globs with non-trailing or empty-segment wildcards (#74) (#75)
validate_path_glob checked `ends_with("/**")` without first stripping it, so any
glob that still ended in `/**` slipped past the wildcard guard even when it held a
`*` elsewhere in the prefix (e.g. `/a*b/**`, `/a/**/**`). The prefix is fed to
glob_prefix/glob_matches as a literal, so such a rule matched no real path and
silently granted nobody, the exact silent-misconfiguration this validator exists
to prevent.

Strip the one permitted trailing `/**` first, then reject any remaining `*`.

While here, close a related hole in the same validator: `glob_prefix` greedily
trims all trailing `**` and `/`, so `//**` collapsed to `/` (whole repo) and
bypassed the explicit `/**` guard, and `//secret/**` became a dead rule whose
prefix never matches a real single-slash path. Reject empty path segments so the
validator and glob_prefix agree on the canonical form.

Adds `/a*b/**`, `/a/**/**`, `/**/**`, `//**`, `//secret/**` to the rejection test.
2026-06-24 11:04:16 +08:00
0a15e763d2 feat(node): owner-only push enforcement behind GITLAWB_ENFORCE_OWNER_PUSH (#31) (#68)
* feat(node): owner-only push enforcement behind GITLAWB_ENFORCE_OWNER_PUSH (#31)

A valid did:key signature on git-receive-pack is authentication, not
authorization: any party can generate a key, derive its DID, sign, and push to
an unprotected branch. Owner checks only ran on protected branches, so a
non-owner could write to unprotected branches of an arbitrary repo.

Add an opt-in flag GITLAWB_ENFORCE_OWNER_PUSH (default false, mirroring
GITLAWB_REQUIRE_SIGNED_PEER_WRITES) that requires the authenticated pusher to be
the repo owner on every branch. Default-off keeps live nodes unaffected.

Design follows the maintainer's review on #31:
- Authorize on the canonical AuthenticatedDid injected by require_signature, via
  Extension<AuthenticatedDid> on the handler — not a re-parse of the headers.
  The legacy branch-protection check is left as-is.
- Fail closed: an absent identity or a non-owner caller is rejected. The policy
  lives in a pure fn (owner_push_rejection) so it is unit-tested without a DB.
- The gate runs before branch protection, so one rejection yields one error body.
- Factor the duplicated owner-match into auth::is_repo_owner, reused by
  receive-pack, protect.rs, and visibility.rs (each keeps its own message).
  events.rs / peers.rs are left out — they aren't owner checks.
- Name the push gate by intent (caller_authorized_to_push) so the planned
  Phase 2 UCAN git/push capability is a pure addition, not a rewrite.

Tests: owner (full + short DID) allowed, non-owner rejected, missing-DID
rejected, flag-off legacy. Docs: .env.example + docs/RUN-A-NODE.md.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* fix(node): address review on #68 — 403 status, finish owner-match consolidation

Per @beardthelion's review:

- Return 403 Forbidden (not 400) for authorization denials: the owner-push gate,
  the branch-protection denial, and the owner checks in protect.rs / visibility.rs.
  400 reads as a malformed request and some git/CI clients retry it.
- Finish the is_repo_owner consolidation: route the branch-protection check and
  the repos list filter through the helper. The branch-protection check now uses
  the verified AuthenticatedDid (auth.0) instead of re-parsing the headers.
- Drop the second identity source on the push path entirely: the post-push event
  recording also uses auth.0, so extract_did_from_auth (and the handler's
  HeaderMap param + import) are removed.
- Tests assert the Forbidden variant (so a future status regression fails loudly)
  and add direct coverage for is_repo_owner / caller_authorized_to_push.
- Docs: warn that enabling the flag blocks delegated and CI agents until the
  Phase 2 UCAN git/push capability is honored.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-24 11:03:28 +08:00
beardthelion 2153b0b7a6 fix(gl): paginate gl-clone Arweave recovery and make /encrypted-blobs parsing schema-strict (#49) (#70)
* fix(gl): make /encrypted-blobs recovery parsing schema-strict

Deserialize the /encrypted-blobs 200 body into a typed EncryptedBlobsResponse
instead of poking at a serde_json::Value and defaulting blobs to [] via
unwrap_or_default(). A server schema regression now fails closed (a hard parse
error) rather than silently yielding nothing to recover, mirroring the existing
WithheldPathsResponse convention. The sole caller still does not abort the clone
but now surfaces the error instead of swallowing it into an empty result.

Part of #49.

* fix(gl): paginate Arweave manifest discovery and rank ties by block height

Discovery hard-coded a single first:100 GraphQL query with no pagination, so a
repo with more than 100 encrypted-manifest anchors (one per push) silently lost
recovery coverage for any blob whose most recent seal fell outside that window.
Follow cursor pagination (after + pageInfo) until coverage is complete, deduping
transaction ids and bounding the loop (1000 pages / 10000 ids, degenerate-cursor
guard) so a hostile or buggy gateway cannot grow the downstream per-id fetch loop
without limit. A full page returned without pageInfo, a cap hit, or a failure
after discovery began now warns; a page-1 failure (unconfigured/unreachable
gateway) stays silent as before.

Pagination concatenates pages in gateway order, which made merge_manifests'
equal-timestamp tie-break (previously insertion-order, i.e. accidentally newest
on single-page HEIGHT_DESC) gateway-dependent. Rank candidates by the composite
key (timestamp, Arweave block height, cid): timestamp primary keeps the existing
distinct-timestamp latest-wins; block height breaks equal-timestamp ties by
anchor recency (a not-yet-mined anchor counts as newest), preserving the
pre-pagination behavior; cid is a final deterministic tiebreak for same-block
anchors. Height is parsed defensively so a present-but-unreadable height (e.g. a
gateway stringifying large ints) cannot masquerade as a pending/newest anchor.

Part of #49.

* fix(review): bound edges per Arweave page and cover degenerate-cursor/dedup

Code-review follow-ups on the #49 recovery change:
- Cap edges parsed per GraphQL page at ARWEAVE_PAGE_SIZE so a hostile gateway
  returning one oversized page cannot defeat the MAX_TX_IDS budget or flood the
  downstream per-id manifest fetch loop (adversarial review).
- Collapse the two identical gateway-failure arms in the discovery loop into a
  single warn site, and hoist PAGE_SIZE to a named const tied to the query's
  first:100 (maintainability).
- Add tests: per-page edge bound, non-advancing-cursor termination (infinite-loop
  guard), and cross-page tx-id dedup fetched exactly once (testing).

Part of #49.

* fix(review): make the degenerate-cursor test assert, and pin the page-size invariant

Round-2 review follow-ups:
- The non-advancing-cursor test bound its mock to a _-prefixed handle with
  expect_at_most(3) and never asserted it; mockito defaults assert_on_drop=false,
  so the test passed even if the guard were removed. Bind it, tighten to expect(2)
  (the guard breaks on the second page, so discovery makes exactly two POSTs), and
  assert_async.
- Correct the parse_tx_page docstring: the per-page take() bounds the TxRef vec and
  fetch loop, not the response-body deserialization (an unbounded-body concern
  shared by every gateway read here).
- Hoist the discovery query to a const and add query_first_matches_page_size so the
  query first:N literal and ARWEAVE_PAGE_SIZE are kept equal by a test rather than
  a comment.

Part of #49.
2026-06-23 22:09:17 +08:00
beardthelion 96fcfdb1ca fix(node): preserve promisor mirror mode on unknown withheld-paths lookup (#48) (#69)
Peer sync decided mirror mode from the origin's withheld-paths answer via
classify_mirror, whose _ => Plain arm collapsed both Some(empty) (genuinely
public) and None (lookup 404'd / 5xx / network / parse error) into Plain. For an
existing promisor mirror that meant fetch_repo(.., Plain) unset
remote.origin.promisor + partialclonefilter and ran git fetch --refetch, so a
transient withheld-paths outage downgraded a still-withheld mirror to a full
clone and broke syncs until the endpoint recovered.

Replace classify_mirror with resolve_mirror_mode(withheld, exists, promisor):
only Some(empty) downgrades an existing mirror; None preserves the on-disk
promisor mode. A genuine public transition always returns Some(empty), never
None, so preserving on None cannot mask one; it only suppresses the
transient-error false positive.

Add a three-valued PromisorProbe read from git config --get exit codes so a
transient probe failure (Unknown) is not mistaken for a definitive NotPromisor
and cannot itself trigger the downgrade. One warn marks the preserve path,
derived from the resolved mode so it cannot drift.

Tests cover every resolve_mirror_mode arm (incl. the regression and the
indeterminate-probe case) and the probe's Promisor / NotPromisor / Unknown
outcomes against real git repos.
2026-06-23 22:07:53 +08:00
beardthelion c9f43b0105 fix(core): route seed access through the zeroizing wrapper (#41) (#64)
* fix(core): route seed access through the zeroizing wrapper (#41)

Remove Keypair::seed_bytes(), which returned a bare [u8; 32] copy of the
Ed25519 private seed. Because [u8; 32] is Copy with no Drop, each caller's
copy was released without being scrubbed, leaving secret key material in
freed memory. The zeroizing accessor to_seed() -> Zeroizing<[u8; 32]>
already existed beside it; this makes it the sole seed accessor and migrates
the two in-crate callers (envelope open path and a crypto test) to it via
deref coercion. No behavior change to envelope encryption: the derived
X25519 secret is byte-identical, covered by the existing round-trip tests.

Adds an identity test pinning to_seed() as the only seed accessor.

* test(core): pin to_seed's Zeroizing return type, drop duplicate round-trip

The to_seed_is_sole_seed_accessor test was named for a compile-time
property its body couldn't assert, and its from_seed -> DID check
duplicated from_seed_round_trip. Replace it with a type-annotated
binding that fails to compile if to_seed ever returns a bare [u8; 32]
again, plus the determinism check, which is the only behavior it
uniquely covered.

* fix(node): migrate node_seed pin caller to to_seed()

main grew a third seed_bytes() caller (repos.rs, from #40) after this
branch was cut. Route it through to_seed() too: node_seed is moved into
a long-lived tokio::spawn pin task, so the Zeroizing wrapper scrubs the
copy when that task ends. encrypt_and_pin takes &[u8; 32]; the wrapper
deref-coerces, so the call site is unchanged.
2026-06-23 22:05:02 +08:00
Kevin CodexandClaude Opus 4.8 97d37fd541 Perf/alloc and copy cleanups (#80)
* perf: trim allocations and redundant copies on hot paths

- Cargo.toml: enable thin LTO + symbol stripping for release builds
- git-remote-gitlawb: attach pack body after signing to move (not clone)
  potentially large pack bytes, halving peak push memory
- gitlawb-attest: deserialize payload by reference instead of cloning it
- core/http_sig: compare components by &str to avoid per-item String alloc
- node/tigris: drop redundant to_vec() copy of the response body
- node/rate_limit: only allocate the key String on first insert, not per request
- node/sync: resolve the peer table once per batch instead of per item

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* perf(sync): skip peer lookup on empty sync batch

Return early from process_batch when dequeue_pending_syncs yields nothing
— the common case on the 30s idle poll — so an empty batch no longer fires
an unnecessary list_peers() DB query (CodeRabbit, PR #80).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* test(sync): cover per-batch peer origin-URL resolution

Extract the peer -> origin-URL lookup from process_batch into a pure
resolve_origin_url() helper and unit-test it (match + trailing-slash trim,
unknown DID, empty peer list). This is the per-batch resolution PR #80
introduced; pulling it out of the DB-bound process_batch lets it be tested
without Postgres. No behavior change. Addresses the needs-tests label.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(tigris): use a unique temp dir per archive extraction

The fixed `.{file_name}.tmp-extract` path let two concurrent extractions of
the same repo share one temp dir, so one's remove_dir_all/unpack could clobber
the other's in-progress work. Suffix the temp dir with a fresh UUID so each
extraction is isolated; this also removes the need to pre-clear a leftover dir
(a unique name can't collide with an interrupted run). The on-failure cleanup
still removes our own dir, and success consumes it via rename (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(tigris): serialize repo publish per path to close the swap race

Unique temp dirs let extractions unpack in parallel, but the final
remove_dir_all + rename swap could still interleave for the same local_path,
causing a nondeterministic overwrite or a rename-onto-non-empty failure. Guard
just the swap with a process-wide mutex keyed by local_path so only one
extraction publishes per repo at a time; the expensive unpack stays parallel
(CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* docs(tigris): restore decompress_repo doc comment ordering

The earlier publish_lock insertion landed between decompress_repo's doc
comment and its fn, gluing the two doc blocks onto publish_lock and leaving
decompress_repo undocumented. Reorder so each fn keeps its own doc. No
behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* docs(tigris): note publish_lock map is never evicted

Document the known unbounded-growth limitation of the per-path LOCKS map:
bounded by the repo count so negligible for normal use, but high-volume
deployments may want LRU/weak-ref eviction. No behavior change (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-23 15:38:18 +08:00
a8cc33a185 fix(security): reject non-public peer URLs + prune poisoned peers (#78)
* fix(security): reject non-public peer URLs + prune poisoned peers

The unauthenticated POST /api/v1/peers/announce only validated the
http(s) scheme, so anyone could register loopback/private 'peers'
(localhost:5432, localhost:22, internal admin URLs). On every push the
node fans out signed sync-notify POSTs to all peers — turning this into
an SSRF probe and burying the real peers under junk so node-origin repos
stopped replicating. Observed on node.gitlawb.com: 126/194 peer rows were
injected junk (fake DIDs z6MkScan…/z6MkAdmin…, localhost targets).

- is_public_http_url(): reject non-http(s), loopback, unspecified, private
  (RFC1918 + ULA), link-local, and localhost/.local/.internal hosts.
  Applied at announce time.
- db.prune_non_public_peers(): boot-time sweep removing already-injected
  rows, run from main() alongside prune_self_peers — self-heals poisoned
  tables on deploy.
- tests for the validator (accept public, reject loopback/private/internal,
  incl. IPv6 [::1]).

Co-Authored-By: OpenClaude <[email protected]>

* fix(security): address PR #78 review — build, redirect SSRF, validation gaps

Resolves the P1/P2 review feedback on the peer-announce SSRF fix:

- main.rs: revert the unrelated, incomplete `storage::build()` refactor back
  to the tigris init from main (the `storage` module never existed, breaking
  the build). Keeps only the prune_non_public_peers boot call.
- Redirect bypass: set redirect::Policy::none() on all peer-facing reqwest
  clients (shared http_client, sync worker, ping_peer) so a peer cannot answer
  `302 -> http://127.0.0.1/` and bypass the public-URL check. ping_peer now
  reuses the shared no-redirect client instead of bare reqwest::get.
- Move the public-URL guard into db::upsert_peer so the bootstrap announce-back
  path (main.rs) can no longer re-poison the table after the prune.
- is_public_http_url: strip trailing-dot FQDNs (`localhost.`), fold
  IPv4-mapped/compatible IPv6 (`::ffff:127.0.0.1`, `::127.0.0.1`) to IPv4 before
  range checks, and reject CGNAT (100.64.0.0/10). Added tests for each.
- prune_non_public_peers: collect bad DIDs then DELETE ... WHERE did = ANY($1)
  in one statement, so a transient error no longer abandons remaining rows.

Follow-up (separate issue): DNS-resolution rebinding guard rejecting peer
hostnames that resolve to private IPs.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(security): satisfy rustfmt + don't drop redirect guard on builder failure

- peers.rs: collapse the CGNAT/private check onto one line (cargo fmt --check).
- sync.rs: replace the unwrap_or_else(reqwest::Client::new()) fallback with
  .expect(). The default client follows redirects, so the fallback silently
  reintroduced the SSRF vector Policy::none() exists to close (CodeRabbit).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-23 14:07:14 +08:00
beardthelion 2202b0097f feat(node): enforce per-route authorization across the REST and GraphQL surface (#87)
* test(node): add HTTP-API integration harness with CI Postgres

Adds a #[cfg(test)] test-support module: a migrated AppState over a real
#[sqlx::test] pool (test_state), a DB-free variant (test_state_lazy), the
assembled router (app), and signed_request_as for injecting an authenticated
DID without RFC-9421 signing. A test-callable run_migrations reuses the
production migrate() path, and sqlx gains the macros/migrate features so
#[sqlx::test] is available. CI's test job gets a postgres service behind a
pg_isready health gate so the DB-backed tests can run.

The proving test exercises the owner gate on PUT /visibility end to end:
non-owner is rejected with 400, owner succeeds. signed_request_as sets a JSON
content-type so the Json extractor does not 415 before the handler runs.

* feat(node): enforce per-route authorization on REST mutations

Closes the missing-authorization cluster on the signed REST surface, where
'authenticated' meant 'any participant' because identities are permissionless.
Adds two shared helpers in api/mod.rs: require_repo_owner (403) and did_matches,
a method-safe DID comparison that collapses the full did:key vs bare short form
without matching across methods (did:web/did:gitlawb share the base58 space).

Per-route gates:
- merge_pr is owner-only (subsumes branch protection); close_pr and close_issue
  allow the repo owner or the PR/issue author (the issue author is read from the
  git-JSON blob, with a None author falling back to owner-only).
- create_review, create_comment, create_issue_comment, and create_bounty are
  read-gated (participants need read access, not ownership; this also closes the
  private-repo bounty leak).
- the task handlers bind the acting DID to the authenticated signer.
- dispute_bounty, previously ungated, now requires the creator or claimant; the
  other bounty and replica identity checks switch from raw equality to did_matches.

A source-level guard test asserts every in-scope mutation handler still carries
its expected gate marker, so a removed gate or an unclassified new route fails
CI. DB-backed tests (via the integration harness) cover merge owner-only and the
task signer-binding; did_matches has unit coverage for the cross-method case.

* feat(node): authenticate GraphQL mutations

GraphQL mutations were a fully unauthenticated parallel write path to the task
system (N2): the /graphql route had no auth layer and the resolvers took the
acting DID as plain arguments. Apply optional_signature to the /graphql POST so
a verified DID is attached when a signature is present (queries stay open), and
thread it into request-scoped data. Each mutation now reads the verified signer
via require_signer and binds the acting DID to it (rejecting an unsigned request
or one whose claimed delegator/assignee/by_did differs from the signer), reusing
did_matches for the comparison. Read-only subscriptions (/graphql/ws) are left
open. A resolver-level test covers the unsigned, mismatched, and matching cases.

* fix(node): path-scope the get_tree visibility gate

get_tree gated on the repo root ("/") regardless of the requested subtree, so
a caller denied a withheld subtree could still enumerate its filenames and blob
SHAs (N3). Gate on the requested path instead, mirroring get_blob, and reject
traversal segments. A cross-caller test shows the rejection is path-scoped: a
non-reader is denied the withheld subtree but passes the gate on a non-withheld
path.

* fix(node): authorize task completion and read-gate PR/issue creation

Two authorization gaps surfaced by an adversarial review of the auth stack,
both necessary-but-insufficient siblings of fixes already in it.

complete_task/fail_task bound the acting DID to the signer but never checked
the caller against the task's assignee, and finish_task updated by id alone, so
any authenticated did:key could finish or fail any task in any state. Load the
task and require the caller to be its assignee (REST and GraphQL), and add an
'AND status=claimed' predicate so only a claimed task transitions. The
now-unused by_did request fields are removed.

create_pr/create_issue resolved the repo with get_repo but skipped
authorize_repo_read, so a non-reader could open a PR (firing the owner's
webhooks) or file an issue against a private repo they cannot read. Gate both
on read access like their create_review/create_comment/create_bounty siblings.

Adds DB-backed tests covering non-assignee rejection (including the empty-body
bypass), the claimed-state predicate, and private-repo PR/issue denial.

* test(node): harden the authz drift-guard against false passes

The guard asserted a marker STRING appeared in each handler body, which a bare
identifier in a comment or log line could satisfy even after the real gate was
deleted, and its body slice over-ran on any handler not declared exactly
'pub async fn'.

- Markers are now gate-shaped: a call (require_repo_owner(, did_matches(,
  authorize_repo_read() or a binding/comparison expression
  (caller != &record.owner_did, let owner_did = auth.0), never a field name.
- Full-line comments are stripped before matching, so a gate that survives only
  as a comment no longer counts as enforced.
- The body slice bounds at the next top-level fn item across pub async,
  pub(crate) async, async, pub, and bare fn forms.
- Adds rows for the now-read-gated create_pr/create_issue and for fork_repo,
  create_repo, set_profile, claim_bounty; register is excluded with a note (it
  trusts the body did, tracked as P3 D3-1).
- Adds a self-test proving a comment-only marker does not satisfy a row.

* fix(node): P3 consistency batch from the adversarial review

Five low-severity items from the stack review. The sixth (a did_matches
false-negative on non-canonical multibase encodings of the same key) is left
as-is: it is a self-inflicted owner lockout unreachable with the conforming
client, never an impersonation vector, so decoding keys in the auth path is not
warranted.

- register bound the registered DID to the request body, not the signer, so a
  signed caller could create or refresh a trust row under a victim DID. Bind it
  to auth.0 (403 on mismatch) and add a drift-guard row now that it is gated.
- Bounty submit/approve/cancel returned 400 for an identity-mismatch denial
  where dispute returned 403. Return Forbidden, matching the rest of the surface.
- The visibility/protect owner gates (require_owner and the protect.rs inline
  checks) returned 400 for a non-owner; return 403 like require_repo_owner, and
  assert the exact code in the harness.
- get_tree rejected ./.. segments but not empty interior segments, so a path
  like secret//x could reach git while the gate saw a different string. Reject
  empty interior segments too (the empty path remains the root listing).
- Tighten the get_tree path-scope test to assert an exact 200 on the
  non-withheld branch so a future upstream 4xx/5xx cannot masquerade as
  gate-pass.

Adds a register-binding regression test.

* fix(node): unify protect/visibility owner checks on did_matches

CodeRabbit flagged that protect_branch/unprotect_branch and visibility's
require_owner used a trailing-segment owner compare (split(':').next_back())
distinct from the hardened did_matches, so the two owner-match idioms could
drift. It is not exploitable (the authenticated caller is always the full did:
form while the short form is a bare suffix, so no wrong caller passes), but it
is a real inconsistency, and the trailing-segment form is a false-negative for a
bare-stored owner. Replace both with did_matches (collapses did:key full vs bare
on both sides, never across methods), and extend the drift guard to assert
require_owner itself uses did_matches, not just that it is called.
2026-06-23 13:16:35 +08:00
beardthelion ff492b4521 feat(node): signature-gated agent self-deregister (#29) (#63)
* feat(node): signature-gated agent self-deregister (#29)

Agents whose keys are rotated or compromised could never be removed from
the registry, so an orphaned DID persisted and could win ?capability=
routing. Give the DID's key holder a way to retire their own agent.

A v5 migration adds an agent lifecycle status (default 'active') plus a
deactivated_at timestamp. list_agents drops non-active agents via a pure
filter_discoverable helper, so a revoked DID no longer appears in
discovery or capability routing; get_agent stays unfiltered so a revoked
DID still resolves and surfaces its status.

DELETE /api/v1/agents/{did} lets an rfc9421-authenticated caller mark its
own agent revoked. Authorization is on the verified AuthenticatedDid via a
pure caller_matches_did gate (a DID can only retire itself), unit-tested
including the cross-DID rejection. Revocation is terminal: register_agent's
ON CONFLICT leaves status untouched, so re-registering a leaked DID cannot
resurrect it.

Authority is team-owned (the key holder); no external contract, chain, or
operator power. Lost-key retirement and any successor pointer are out of
scope.

* fix(node): preserve original deactivated_at on repeat revoke

A retried DELETE /api/v1/agents/{did} re-ran the revoke UPDATE and
overwrote deactivated_at with a fresh timestamp, losing the true
first-retired time. Use COALESCE(deactivated_at, $2) so the original
timestamp is kept; the row still matches so the idempotent 204 is
unchanged.
2026-06-23 11:41:59 +08:00
beardthelion 2d4105e479 chore(deps): bump quinn-proto 0.11.14 -> 0.11.15 (RUSTSEC-2026-0185) (#83)
quinn-proto 0.11.14 (pulled transitively via libp2p's QUIC transport) is
flagged by RUSTSEC-2026-0185. 0.11.15 is a semver-compatible patch that
clears it; cargo audit passes clean after the bump. Lockfile-only, no
manifest or code change, so it carries no ignore entry. Repo-wide:
unblocks cargo audit on main and every open branch.
2026-06-23 11:39:18 +08:00
beardthelion 803d83efdb fix(node): reap leaked git child processes (#53) (#61)
* fix(docker): reap orphan processes via tini + init, cap pids

The node runs as PID 1 in the container and never reaps reparented
orphans, so git grandchildren (e.g. pack-objects left behind when a
served clone disconnects) accumulate as zombies until fork() fails with
EAGAIN. Run under tini and set compose init: true so reparented orphans
are reaped, and add pids_limit as a blast-radius cap.

Refs #53

* fix(node): reap leaked git child processes (#53)

Serving git over smart-HTTP could leak zombie processes that, over weeks,
exhaust the PID table until fork() fails with EAGAIN and all git
operations break until restart.

The serve path (run_git_service) now runs git in its own process group and
SIGKILLs that group when the request future is dropped (client disconnect)
or returns early, so git and its pack-objects child are torn down together
instead of orphaning the grandchild to PID 1 as a zombie. A drop guard,
disarmed once the request completes cleanly, owns this teardown.

create_issue and build_filtered_pack spawned a std Child and dropped it
unwaited if the stdin write failed (and build_filtered_pack could panic on
a missing stdin handle). Both now always wait the child before surfacing
any error, so the failure path cannot leak.

Adds libc for the group-kill syscall and three regression tests covering
direct-child kill, grandchild reaping via the group, and the disarmed
no-kill path.

Refs #53

* fix(review): reap-then-disarm on serve-path errors; tini/comment nits

Address code-review findings on the #53 fix:

- run_git_service no longer early-returns on a stdin-write error. It captures
  the write result, always reaps the child, then disarms the group guard
  before surfacing the error. Previously a write/wait error path could drop
  the still-armed guard after the child was already reaped, so the group
  SIGKILL could in theory land on a reused pgid. (correctness, reliability,
  adversarial all flagged this region.)
- Drop tini's -g flag: the node manages its own children and graceful
  shutdown, so group-wide signalling would only disturb in-flight git helpers
  on shutdown.
- Correct the SAFETY comment (kill(2) is not relevant to async-signal-safety
  here) and the alive() test-helper comment (EPERM does not read as alive).

* fix(node): SIGTERM (not SIGKILL) the served-git group on disconnect

SIGKILL gave git no chance to run its signal cleanup, so an aborted
receive-pack could leave a stale .git/*.lock that blocks the next push.
SIGTERM lets git remove its lock files before exiting. Signalling the
whole process group still reaches pack-objects directly, so the grandchild
is terminated rather than orphaned, and a single immediate signal avoids
any delayed-kill pid-reuse window.

Refs #53

* docs(node): correct stale SIGKILL comments to SIGTERM in served-git teardown
2026-06-23 10:10:17 +08:00
beardthelion 7533b74f79 ci: harden PR pipeline (audit, MSRV, beta matrix, release + docker smoke) (#57)
* ci: harden PR pipeline (audit, MSRV, release build, beta matrix, docker smoke)

Restructure the single stable-only fmt+clippy+test job into separate jobs:
fmt+clippy (stable), a test matrix on stable+beta (beta non-blocking),
build --release, cargo audit, an MSRV check pinned to Rust 1.91, and a
Docker build smoke test mirroring the release workflow. Add concurrency
cancellation for superseded PR runs and per-job timeouts.

Declare rust-version = "1.91" in [workspace.package] and inherit it in
every member crate so the stated MSRV is a real, CI-enforced contract.

* ci: disable credential persistence on checkout steps

Set persist-credentials: false on all six actions/checkout steps. These
jobs run untrusted PR code via cargo and docker build, and checkout's
default leaves the GITHUB_TOKEN in the local git config where that code
could read it. None of the jobs push or otherwise need git credentials.

* fix(deps): bump rustls-webpki to 0.103.13

Clears three advisories flagged by the new cargo audit job: a reachable
panic in CRL parsing and two name-constraint bypasses (wildcard and URI).
Semver-compatible patch, lockfile only, no manifest or code change.

cargo also corrects a stale lockfile reference while rewriting Cargo.lock:
data-encoding-macro-internal 0.1.17 (a build-time macro crate) requires
syn 1.x, and its dependency entry is re-pinned from syn 2.0.117 to the
1.0.109 it actually resolves to. Unrelated to the bump, unavoidable when
cargo touches the lock, and the more-correct state.

* ci(audit): suppress no-fix advisories with justification and a re-check

The cargo audit job stays hard-fail. The remaining three advisories have
no available upstream fix, so they are suppressed in .cargo/audit.toml
(the location cargo-audit actually reads; a repo-root audit.toml is not
read), each with an inline rationale and removal condition:

- RUSTSEC-2026-0118/0119 (hickory-proto 0.25.2, via libp2p-dns 0.44):
  reachable, since the node wraps QUIC in the system DNS resolver. No fix
  until libp2p-dns ships hickory 0.26. Accepted, reachable risk.
- RUSTSEC-2023-0071 (rsa Marvin): in the lockfile via sqlx's unconditional
  sqlx-mysql, but not linked (sqlx is postgres-only), so no RSA oracle
  exists; no upstream fix regardless.

Two controls keep the suppression honest. The PR audit job echoes the
active ignore list to the job summary, so a green check is not read as
advisory-clean. A weekly scheduled job re-runs cargo audit without the
ignores for visibility and hard-fails if hickory-proto moves off 0.25.x
while the ignores remain, so the suppression cannot silently outlive the
upstream fix.

* ci(audit): reference tracking issue #76 in the hickory ignores

* ci(audit): treat hickory-proto removal as drift too

The scheduled drift guard only failed when hickory-proto was present and
off 0.25.x, so removing it from Cargo.lock entirely left the RUSTSEC-2026-0118
/0119 ignores in place with the guard still passing. Treat an absent version
as drift as well, so dead ignore entries are caught whether the dependency is
bumped past 0.25.x or removed.
2026-06-22 19:02:38 +08:00
beardthelion abdc775770 feat(node): blind recipient identities at rest and gate B1 by repo readability (#40)
* feat(node): blind recipient identities at rest and gate B1 by repo readability

The origin no longer stores recipient DIDs. Migration v5 replaces the
encrypted_blobs.recipients column with an opaque, node-keyed recipients_tag
used only to detect a recipient-set change for re-seal. B1 discovery and fetch
are now gated by the same repo-readability check the git read path uses, not by
per-recipient matching; decryption is gated by the envelope crypto, so a
non-recipient who can read the repo sees a blob's {oid, cid} but cannot open it.
encrypt_and_pin keys the tag from the node seed and returns {oid, cid}; the
Arweave manifest tuple drops the now-unused recipient vec.

A DB compromise no longer reveals the reader set; recovering it would require
brute-forcing candidate DID sets against the keyed tag with the node key.

* chore: remove redundant .gitignore entry (covered by local exclude)

* fix(node): gate /replicate by repo readability; harden reseal on DB error

Address review on the at-rest blinding change:
- The encrypted-blobs/replicate listing returned {oid, cid} with no visibility
  check, so a non-readable repo's blob index was reachable by an unauthenticated
  caller who guessed {owner}/{repo}. Gate it by the same repo-readability check
  discovery and fetch use. For the intended case (a public repo with withheld
  subtrees) the public root keeps this open to peers; only fully non-readable
  repos are withheld, which is the desired behavior.
- encrypt_and_pin treated a recipients_tag DB read error as a cache miss and
  resealed, causing avoidable IPFS writes during a partial outage; skip and retry
  on the next push instead.
- Correct the get_encrypted_blob doc comment to describe repo-readability access.
2026-06-22 19:01:19 +08:00
beardthelion e37ea7fec6 fix(node): enforce path-scoped visibility on the REST read API (#52)
* fix(node): enforce path-scoped visibility on the REST read API

The repo-scoped REST read endpoints (blob, tree, commits, refs, changelog,
pulls, pr diff, repo metadata) read straight from disk after a bare repo
lookup, with no visibility check, so private repositories and private
subtrees of public repositories were served to unauthenticated callers,
bypassing the withholding the git and replication paths enforce.

Add a shared authorize_repo_read gate (resolve repo, load rules,
visibility_check, 404 on deny) and call it from every repo-scoped read
handler. get_blob gates at the requested file path; get_pr_diff withholds
the whole diff when it touches a path the caller cannot read. Attach
optional_signature to the read routes so authorized readers authenticate.
Listings stay gated at the repo root, consistent with mode B exposing tree
and commit SHAs over git.

Closes #51

* fix(node): gate PR reviews/comments and fork on repo read visibility

The REST visibility gate added in this PR missed three read surfaces:
list_reviews and list_comments served a private repo's PR reviews and
comments to unauthenticated callers, and fork_repo cloned a full mirror of
the source without checking the caller could read it. Route all three through
authorize_repo_read at the repo root, matching the git-protocol gate.

Also make branch_diff_names fail closed: it ignored git's exit status and
returned an empty path list on error, which would let get_pr_diff serve a diff
it should have withheld. Fix changelog.rs import order to satisfy cargo fmt.

* fix(node): NUL-delimit branch_diff_names to keep path-gate enforcement exact

git diff --name-only quotes and escapes paths containing newlines, so they
no longer match the visibility globs in get_pr_diff. On a public repo where /
is allowed but a subtree is denied, a touched restricted path could be missed
by the deny check and the full PR diff disclosed. Use -z and split on NUL to
preserve exact path bytes.
2026-06-19 22:53:18 +08:00
beardthelion 197c4e1cf1 Add contributor templates, issue forms, and PR triage automation (#58)
* docs: add pull request template

* docs: add structured issue forms and template chooser config

* chore: add CODEOWNERS for CI, release, and security paths

* ci: add quality-signal PR triage workflow

* ci: add label-gated auto-stale workflow for ignored PRs

* docs: document PR quality bar and triage/stale policy in CONTRIBUTING

* ci: recognize inline #[cfg(test)] tests and fix triage comment grammar
2026-06-19 22:52:32 +08:00
a5d2264cc0 ci(release): smoke-test packaged binaries' --version before attaching (#59)
The release-binaries job packaged gl, git-remote-gitlawb, and gitlawb-node
into the release tarball but never ran them, so a broken artifact could ship.
This is the follow-up promised in #30 and the gap noted in OSS-READINESS-AUDIT
("release binaries are not smoke-tested after packaging"), now unblocked by
git-remote-gitlawb gaining --version.

Run each binary's --version after build and assert the released version string
is present. Skipped for aarch64-unknown-linux-musl, which is cross-compiled and
can't execute on the x86_64 runner.

Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-19 22:50:35 +08:00
beardthelionandCommandCodeBot 5ff7af84fa feat: encrypted replication for private subtrees (B1/B2/B3) for #18 (#36)
* feat(core): ed25519 to x25519 key conversion for envelope crypto

* feat(core): seal/open per-blob encryption envelopes

* feat(node): resolve per-blob encryption recipients (least-privilege)

* feat(node): encrypted_blobs table and recipient-scoped queries

* feat(node): encrypt-then-pin withheld blobs at push (IPFS)

At the push chokepoint, after pinning withheld objects, resolve each
withheld blob's recipient DIDs and seal it to their Ed25519 keys with
seal_blob, pinning the ciphertext to IPFS and recording it in
encrypted_blobs. Best-effort per blob: failures are logged and skipped,
never pinned in plaintext. Pinata replication is unchanged; B1
encrypts to IPFS only.

Adds ed25519-dalek as a direct dependency of gitlawb-node (it was only
declared in the workspace Cargo.toml).

* feat(node): authenticated discovery and fetch for encrypted blobs

* feat(gl): transparent recovery of authorized encrypted blobs on clone

* style: cargo fmt

* fix(node): re-seal encrypted blob when recipient set changes

Skip re-pinning only when an existing envelope already covers exactly the
current recipients. A reader added to a rule after the first pin now gets
a re-seal on the next push instead of being permanently locked out.
Reader removal stays non-retroactive (the old envelope is already public).

* feat(node): unscoped encrypted-blob listing for replication

* feat(node): encrypted-blobs/replicate endpoint for peer mirrors

* feat(node): route encrypted-blobs/replicate under git_read_routes

* feat(node): peer mirrors replicate encrypted withheld blobs (Option B2)

* feat(node): anchor_encrypted_manifest for Option B3 Arweave manifests

* feat(node): encrypt_and_pin returns the per-push sealed delta

* feat(node): anchor encrypted-blob manifest on push (Option B3)

* feat(gl): Arweave/IPFS gateway recovery for encrypted blobs (Option B3)

* feat(gl): transparent Arweave fallback recovery on clone (Option B3)

* style: cargo fmt

* fix(gl): bound Arweave recovery gateway requests with a 30s timeout

* test(gl): read-path recovery tests for Arweave fallback; quiet promisor present-check

Add two hermetic integration tests for recover_from_arweave that drive the
full read path over mocked Arweave GraphQL + IPFS gateways: discover the
manifest, fetch it, fetch the envelope, decrypt, and install the withheld
blob. One covers an authorized recipient (blob installed), the other a
non-recipient (nothing recovered). Both simulate origin death by removing
the promisor remote and enable uploadpack.allowFilter so the blob is truly
withheld over file://.

Also harden the local presence check in recover_from_arweave with
GIT_NO_LAZY_FETCH=1 and .output() so the expected 'missing object' case does
not trigger a wasted promisor fetch or leak git stderr to the user.

* fix: surface corrupt recipients JSON and silent recovery I/O failures

db: parse_recipients now propagates a descriptive error instead of
defaulting corrupt recipients JSON to an empty list, which would have
denied authorized readers and handed peers incomplete metadata.

gl: clone recovery now warns when the sparse-checkout file cannot be read
or written, or when the post-recovery checkout fails, instead of silently
discarding those errors and claiming files were recovered.

* feat(core): blind recipient identities in withheld-blob envelopes

Envelope v2 drops the cleartext recipient public key (kid) from each
wrapped-key header entry, so a party holding the public IPFS/Arweave copy
can no longer enumerate who is authorized to decrypt a withheld blob. The
version is bumped to 2 and v1 envelopes are rejected. open_blob now selects
the reader's entry by trial decryption (the AEAD tag authenticates exactly
one entry) instead of matching on the public key.

Recipient count is still visible; identity is not. Scope is envelope-only,
the node DB recipients column and peer replication metadata are unchanged.

Co-authored-by: CommandCodeBot <[email protected]>

* fix(core): reject malformed envelope nonces instead of panicking

open_blob fed attacker-controlled envelopes to GenericArray::from_slice and
XNonce::from_slice, which panic on a wrong-length input. Validate both the
per-recipient box nonce and the body nonce to 24 bytes before use: skip a
recipient entry whose nonce is malformed, and return an error for a malformed
body nonce, so the public recovery path surfaces an error rather than panicking.

* feat(node): blind recipient identities on the B2 replication surface

The encrypted-blobs/replicate endpoint shipped every blob's full recipient
DID list to mirrors, which persisted it, so any mirroring peer learned the
reader set. The v2 envelope blinding already removed recipient public keys
from the pinned envelopes, so the comment justifying this (DIDs are already
public) was no longer true.

/replicate now returns {oid, cid} only. Mirrors detect a re-seal by the CID
changing (the OID is stable across re-seals) instead of comparing recipient
sets, and store no recipient identities. Origin-side authz and the at-rest
recipients column are unchanged; this blinds only the peer-facing surface.

* feat(node): stop anchoring recipient identities to Arweave

The B3 encrypted-blob manifest anchored {oid, cid, recipients} to Arweave, a
permanent public record of every blob's reader set. The v2 envelope blinding
already removed recipient keys from the pinned envelopes, so the comment
justifying this (recipient DIDs are already public) was no longer true. The
manifest now anchors {oid, cid} only. The gl reader already ignores the
recipients field (it recovers by trial decryption), so no reader changes.

---------

Co-authored-by: CommandCodeBot <[email protected]>
2026-06-19 22:48:33 +08:00
beardthelionandKevin Codex 8680d0f9d6 feat(node): replication enforcement (Phase 2) for #18 (#34)
* docs(node): Phase 3 subtree-withholding plan with Task 0 spike findings

* feat(node): resolve withheld blob OIDs for path-scoped visibility

* feat(node): filtered upload-pack serve that omits withheld blobs

* feat(node): serve filtered pack when caller has withheld subtree blobs

* test(node): end-to-end assert served pack omits withheld blob

* docs(node): note why info/refs is not gated on subtree visibility

* fix(node): frame filtered serve as protocol v0 and prove it with a real clone

upload_pack_excluding emitted a v2 packfile section, but info_refs
advertises v0, so real clients negotiated v0 and rejected the response
with 'expected ACK/NAK, got packfile'. Frame the v0 stateless-rpc shape
instead (NAK, then the pack via side-band-64k when offered).

Add an end-to-end test that stands up info_refs + upload_pack_excluding
and runs a real git partial clone, asserting the withheld blob's bytes
never reach the client while its tree entry and SHA stay visible. A stock
full clone cannot consume the pack (it is not closed under reachability,
so fetch fails the connectivity check); a partial clone is required.

* docs(node): correct Phase 3 caveat (full clone refused at fetch, not checkout)

* test(node): prove incremental fetch still withholds; document negotiation choice

Add a real-git test that partial-clones, pushes a new commit server-side,
then fetches: the new object arrives and the withheld blob stays absent.
This pins down that ignoring have/want negotiation (always sending a
self-contained pack of all refs minus withheld, with NAK) is correct for
both clone and fetch; the only cost is a fetch re-sends the full object
set. Refactor the real-git tests onto a shared server harness and document
the negotiation decision in code and in the plan's follow-ups.

* refactor(node): address CodeRabbit review on PR #28

Move the two blocking git shell-outs in the filtered upload-pack path off
the async worker thread, matching the tokio::process / spawn_blocking usage
already in this file: build_filtered_pack (rev-list + pack-objects) and
withheld_blob_oids (per-ref ls-tree) now run inside spawn_blocking so a large
repo cannot stall the tokio runtime. Behavior is unchanged.

Also fix the Task 0 findings block in the Phase 3 plan: it still recorded v2
packfile framing, which is the exact path that failed against a real client
and was corrected to v0. The block now documents the shipped v0 contract.
Drop a stray trailing code fence flagged by markdownlint (MD040).

The speculative ls-tree timeout and the public/no-rules fast-path from the
review are intentionally left out: the timeout guards against adversarial
repos we do not yet host, and the fast-path is a micro-optimization not worth
the extra branch right now.

* chore(node): drop planning doc from PR per maintainer request

kevincodex1 asked to keep the superpowers planning docs out of the repo. The
Phase 3 plan was scaffolding for this change, not something the project needs
to carry. Removing it leaves only the code and tests in the PR.

* chore: gitignore local planning docs (docs/superpowers/)

* feat(node): replicable_objects filter for replication enforcement

* test(node): pin anonymous-caller contract of withheld_blob_oids

* feat(node): IPFS pinning skips withheld blob OIDs

* feat(node): Pinata pinning skips withheld blob OIDs

* feat(node): enforce visibility on push replication (IPFS/Pinata/gossip/Arweave)

* test(node): announce gate matches anonymous repo readability

* fix(node): run withheld_blob_oids off the async worker on push

The receive-pack replication chokepoint called withheld_blob_oids
directly on the tokio worker, where its blocking git ls-tree walk can
stall the runtime for repos with many refs. Wrap it in spawn_blocking
to match the upload-pack serve path.

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-06-18 22:51:36 +08:00
Kevin CodexandOpenClaude c03c9af8fa feat(sync): auto-register as replica with origin after successful mirror (#56)
The sync worker mirrored repos from peers but never told the origin, so
replica_count (repo_replicas table, GET /api/v1/repos/:owner/:repo/replicas)
stayed at 0 network-wide — the dashboard showed 0 replication even for
repos that were actually mirrored.

After each successful clone/fetch the worker now sends a signed
PUT /api/v1/repos/{owner}/{repo}/replicas to the origin with our
public_url, using the same HTTP-Signature scheme as push notifies.
Best-effort by design: skipped when public_url is unset, failures are
logged without failing the sync, and the PUT is idempotent on the origin
(201 first, 200 after) so re-registering every sync is self-healing.

Nodes need this build before their mirrors are counted: a new node's
repos show replication once its peers upgrade.

Co-authored-by: OpenClaude <[email protected]>
2026-06-18 22:31:29 +08:00
Kevin CodexandOpenClaude 7a8b6c4974 Aws ecr image (#55)
* feat(infra): optional private ECR repo for the node image

create_ecr_repo = true provisions an ECR repository in-account and
switches the instance to pull from it: the instance role gets
AmazonEC2ContainerRegistryReadOnly and user-data configures
amazon-ecr-credential-helper, so first-boot pulls and SSM upgrades
authenticate via the role (no expiring login tokens). README documents
the local build-and-push flow.

Motivation: ghcr.io/gitlawb/node is not anonymously pullable (403),
which left first boot unable to start the stack.

Co-Authored-By: OpenClaude <[email protected]>

* feat(infra): HTTPS via Caddy + production tier (RDS, alarms, S3 state)

- domain_name: Caddy sidecar terminates TLS on 80/443 with auto-renewed
  Let's Encrypt certs (persisted on the data volume); public_url
  defaults to https://<domain>; SG opens 80/443 — matches the Fly nodes
- use_rds: managed Postgres (db.t4g.micro, encrypted, 7-day PITR,
  deletion protection) replaces the in-container postgres; DB security
  group only admits the node instance
- alert_email: SNS email alerts wired to CloudWatch alarms — external
  HTTPS /health via Route53 health check, EC2 status checks, CPU,
  RDS free storage
- user-data grows the filesystem when the EBS volume is enlarged
- terraform state moved to S3 (versioned, KMS-encrypted, public-blocked
  bucket) with lockfile locking (requires terraform >= 1.10)

Co-Authored-By: OpenClaude <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
2026-06-18 22:05:50 +08:00
Kevin CodexandOpenClaude 7a99d0f27c fix(node): make Tigris repo hydration resilient to corrupt archives & failed writes (#54)
Repo pushes 500'd whenever a repo's Tigris archive was unreadable (corrupt
or truncated) or the node was under disk pressure, because hydration was
treated as fatal and a failed extraction could clobber the good local copy.
Three hardening changes:

- tigris: extract archives atomically (unpack into a temp dir, swap into
  place only on success) so a corrupt/truncated archive can never clobber a
  good existing local copy.
- repo_store: acquire_fresh / acquire_write fall back to the valid local
  copy when a Tigris download fails, instead of hard-failing the operation;
  the post-write upload then re-syncs (self-heals) Tigris.
- repo_store: RepoWriteGuard::release(success) uploads to Tigris only when
  the write succeeded, so a half-applied / inconsistent repo is never
  propagated to Tigris or to other nodes.

Co-authored-by: OpenClaude <[email protected]>
2026-06-18 21:38:47 +08:00
Kevin CodexandClaude Fable 5 c7df6cad20 Fix/backport prod fixes (#39)
* fix: backport production fixes from monorepo

Three fixes that landed on the production node but never made it here:

- trigger_sync: 30s timeout with the body read inside it (was 5s
  covering headers only — large unpaginated repo lists from canonical
  nodes and transpacific peers aborted mid-body), plus distinct logging
  for non-2xx vs timeout
- announce: reject self-announcements by public URL or own DID;
  complements the existing boot-time prune_self_peers so self-loop
  rows never enter the peers table in the first place
- list_all_repos_paged: report MAX(updated_at) across each logical
  repo group instead of the canonical row's own value — gossip pushes
  touch only the mirror row, so canonical timestamps go stale

Monorepo refs: 8c2fe1b, c1dc33e, ef21c75.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(p2p): dual-stack QUIC listen + DNS multiaddr dialing for Fly 6PN

Backports the monorepo TCP-era p2p fixes (9f875b6, 267d371) adapted to
the QUIC transport:

- listen on /ip6/:: as well as /ip4/0.0.0.0 — Fly's 6PN inter-app
  network is IPv6-only (<app>.internal resolves to AAAA records), so
  without an IPv6 socket peers on the private network can't reach us
- wrap the QUIC transport in libp2p-dns so bootstrap multiaddrs can use
  /dns6/<app>.internal — dialing peers through the public anycast edge
  breaks the handshake when the proxy closes the connection mid-stream

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(sync): HTTP notify records received_ref_updates like the gossip path

A push propagated via the /api/v1/sync/notify HTTP fallback replicated
the repo but left no trace in received_ref_updates — only the libp2p
gossip path recorded it, so /api/v1/events/ref-updates went blind
whenever the mesh wasn't delivering. Insert the same record from the
notify handler, with optional provenance fields (pusher_did, old_sha,
timestamp, cert_id) that new senders already include.

Monorepo ref: 087eeac.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-06-18 21:36:11 +08:00
Kevin CodexandClaude Fable 5 b61a1bd46f fix(api): blob endpoint returns 400/404 instead of 500 on bad paths (#37)
Unnormalized "../.." paths from crawler-exploded relative links made
`git show` fail and surfaced as 500s. Reject traversal segments with
400 up front, and map paths absent from the tree to 404 so only real
git failures count as server errors.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-06-18 21:35:36 +08:00
beardthelion e365a57b51 feat(node): peer partial-mirrors for repos with private subtrees (#35)
* feat(node): withheld-paths endpoint reporting a caller's denied globs

* feat(gl): gl clone with promisor + sparse-exclude for private subtrees

* test(gl): gl clone repo-argument parsing

* fix(visibility): gate withheld-paths and handle nested/exact globs

Three fixes from the PR #33 review:

- withheld_paths now applies the whole-repo "/" read gate (returns
  repo-not-found when the caller cannot read the root), matching the git read
  endpoints. Without it the endpoint disclosed a private repo's existence and
  path layout to unauthorized callers. The withheld_globs doc already assumed
  this gate existed; now it does.

- A nested allow under a denied parent (e.g. "/secret/public/**" allowed,
  "/secret/**" denied) was over-withheld: the client sparse-excluded the whole
  parent and hid paths the caller may read. The endpoint now also returns a
  "reinclude" list (allowed globs strictly under a denied one) and gl clone
  re-includes them in the sparse spec after the excludes.

- Wildcard-free globs like "/docs/private" match both the exact path and a
  subtree (per glob_matches), but the client only emitted the subtree exclude.
  sparse_patterns now emits both "/docs/private" and "/docs/private/".

Verified the exclude-then-reinclude sparse ordering checks out cleanly with
real git, plus unit tests for reincluded_globs, the nested re-include, the
exact-path exclude, and sparse_patterns.

* fix(gl): reject multi-slash repo input and stop failing open on withheld-path errors

split_once('/') accepted owner/name/extra, smuggling a path segment into
the repo name that then flowed into the API path and clone URL; reject it.

fetch_withheld swallowed every network/auth/5xx/JSON error into an empty
result, dropping to a stock clone that the node refuses once blobs are
withheld. Now only 404/501 (endpoint unsupported) fall back to empty; the
rest propagate so the real cause surfaces.

* test(gl): guard three-level nested visibility in sparse clone

Add a regression test for deny /secret, allow /secret/public, deny
/secret/public/admin and clarify the sparse-checkout comment. git does
not re-traverse an explicitly excluded directory, so emitting all
excludes before re-includes keeps the deepest deny in force; the broader
parent re-include does not resurrect it.

* fix(gl): robust default-branch detection and strict withheld-paths schema

Read the default branch from the local origin/HEAD symref clone already
set, instead of parsing the localized, network-dependent output of
git remote show origin. Deserialize the withheld-paths body into a typed
struct so a missing or mistyped withheld/reinclude field is a hard error
rather than silently becoming an empty list, which would mask a server
regression behind a later clone failure.

* feat(node): classify mirror mode from origin withheld-paths

* feat(node): promisor-aware mirror clone for withheld repos

* feat(node): promisor-aware mirror fetch via origin remote

* feat(node): classify and mirror peers per withheld-paths

* style: cargo fmt

* fix(node): bound sync withheld lookup and rehydrate on promisor->plain

Give the sync worker's HTTP client a 30s timeout so a stalled peer
withheld-paths lookup cannot hang the worker loop. When a repo that was
mirrored as a promisor (mode B) becomes fully public, fetch_repo now
clears remote.origin.promisor and partialclonefilter and refetches, so
the once-withheld blobs are backfilled instead of the mirror staying
permanently partial.
2026-06-18 21:33:31 +08:00
beardthelion 4a59056a90 gl clone: clean partial clone of repos with private subtrees (#33)
* feat(node): withheld-paths endpoint reporting a caller's denied globs

* feat(gl): gl clone with promisor + sparse-exclude for private subtrees

* test(gl): gl clone repo-argument parsing

* fix(visibility): gate withheld-paths and handle nested/exact globs

Three fixes from the PR #33 review:

- withheld_paths now applies the whole-repo "/" read gate (returns
  repo-not-found when the caller cannot read the root), matching the git read
  endpoints. Without it the endpoint disclosed a private repo's existence and
  path layout to unauthorized callers. The withheld_globs doc already assumed
  this gate existed; now it does.

- A nested allow under a denied parent (e.g. "/secret/public/**" allowed,
  "/secret/**" denied) was over-withheld: the client sparse-excluded the whole
  parent and hid paths the caller may read. The endpoint now also returns a
  "reinclude" list (allowed globs strictly under a denied one) and gl clone
  re-includes them in the sparse spec after the excludes.

- Wildcard-free globs like "/docs/private" match both the exact path and a
  subtree (per glob_matches), but the client only emitted the subtree exclude.
  sparse_patterns now emits both "/docs/private" and "/docs/private/".

Verified the exclude-then-reinclude sparse ordering checks out cleanly with
real git, plus unit tests for reincluded_globs, the nested re-include, the
exact-path exclude, and sparse_patterns.

* fix(gl): reject multi-slash repo input and stop failing open on withheld-path errors

split_once('/') accepted owner/name/extra, smuggling a path segment into
the repo name that then flowed into the API path and clone URL; reject it.

fetch_withheld swallowed every network/auth/5xx/JSON error into an empty
result, dropping to a stock clone that the node refuses once blobs are
withheld. Now only 404/501 (endpoint unsupported) fall back to empty; the
rest propagate so the real cause surfaces.

* test(gl): guard three-level nested visibility in sparse clone

Add a regression test for deny /secret, allow /secret/public, deny
/secret/public/admin and clarify the sparse-checkout comment. git does
not re-traverse an explicitly excluded directory, so emitting all
excludes before re-includes keeps the deepest deny in force; the broader
parent re-include does not resurrect it.

* fix(gl): robust default-branch detection and strict withheld-paths schema

Read the default branch from the local origin/HEAD symref clone already
set, instead of parsing the localized, network-dependent output of
git remote show origin. Deserialize the withheld-paths body into a typed
struct so a missing or mistyped withheld/reinclude field is a hard error
rather than silently becoming an empty list, which would mask a server
regression behind a later clone failure.
2026-06-18 21:25:13 +08:00
3a401eb079 feat(git-remote-gitlawb): add --version and --help flags (#30)
git-remote-gitlawb was the only workspace binary without `--version`, so
release smoke tests could not validate the packaged helper directly. This
is called out in docs/MAINTAINER-ROADMAP.md ("Add --version support to
git-remote-gitlawb so release smoke tests can validate it directly") and
docs/OSS-READINESS-AUDIT.md.

- `--version`/`-V` prints `git-remote-gitlawb <version>`, matching the
  `<bin> <version>` format the clap-based `gl` and `gitlawb-node` emit so
  release tooling can treat all three binaries uniformly. The version
  comes from CARGO_PKG_VERSION.
- `--help`/`-h` prints concise usage (URL form, env vars, flags).
- Flags are only recognized as the first argument, so git's
  `<remote-name> <url>` remote-helper invocation path is untouched; a
  remote literally named "--version" is not mistaken for the flag.
- Informational flags are handled before tracing init so output stays
  clean for scripts.

Adds unit tests for argument classification (including the no-args and
misplaced-flag edge cases) and the version/help strings.

Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-18 21:13:01 +08:00
beardthelion 61b383019f feat(node): subtree content withholding (Phase 3) for #18 (#28)
* docs(node): Phase 3 subtree-withholding plan with Task 0 spike findings

* feat(node): resolve withheld blob OIDs for path-scoped visibility

* feat(node): filtered upload-pack serve that omits withheld blobs

* feat(node): serve filtered pack when caller has withheld subtree blobs

* test(node): end-to-end assert served pack omits withheld blob

* docs(node): note why info/refs is not gated on subtree visibility

* fix(node): frame filtered serve as protocol v0 and prove it with a real clone

upload_pack_excluding emitted a v2 packfile section, but info_refs
advertises v0, so real clients negotiated v0 and rejected the response
with 'expected ACK/NAK, got packfile'. Frame the v0 stateless-rpc shape
instead (NAK, then the pack via side-band-64k when offered).

Add an end-to-end test that stands up info_refs + upload_pack_excluding
and runs a real git partial clone, asserting the withheld blob's bytes
never reach the client while its tree entry and SHA stay visible. A stock
full clone cannot consume the pack (it is not closed under reachability,
so fetch fails the connectivity check); a partial clone is required.

* docs(node): correct Phase 3 caveat (full clone refused at fetch, not checkout)

* test(node): prove incremental fetch still withholds; document negotiation choice

Add a real-git test that partial-clones, pushes a new commit server-side,
then fetches: the new object arrives and the withheld blob stays absent.
This pins down that ignoring have/want negotiation (always sending a
self-contained pack of all refs minus withheld, with NAK) is correct for
both clone and fetch; the only cost is a fetch re-sends the full object
set. Refactor the real-git tests onto a shared server harness and document
the negotiation decision in code and in the plan's follow-ups.

* refactor(node): address CodeRabbit review on PR #28

Move the two blocking git shell-outs in the filtered upload-pack path off
the async worker thread, matching the tokio::process / spawn_blocking usage
already in this file: build_filtered_pack (rev-list + pack-objects) and
withheld_blob_oids (per-ref ls-tree) now run inside spawn_blocking so a large
repo cannot stall the tokio runtime. Behavior is unchanged.

Also fix the Task 0 findings block in the Phase 3 plan: it still recorded v2
packfile framing, which is the exact path that failed against a real client
and was corrected to v0. The block now documents the shipped v0 contract.
Drop a stray trailing code fence flagged by markdownlint (MD040).

The speculative ls-tree timeout and the public/no-rules fast-path from the
review are intentionally left out: the timeout guards against adversarial
repos we do not yet host, and the fast-path is a micro-optimization not worth
the extra branch right now.

* chore(node): drop planning doc from PR per maintainer request

kevincodex1 asked to keep the superpowers planning docs out of the repo. The
Phase 3 plan was scaffolding for this change, not something the project needs
to carry. Removing it leaves only the code and tests in the PR.

* chore: gitignore local planning docs (docs/superpowers/)
2026-06-18 21:11:46 +08:00
Kevin CodexandClaude Fable 5 a2217bff27 fix(infra): drop fly idle_timeout 600 -> 120 (#38)
600s idle windows let hung clients pin Fly proxy connection slots up to
hard_limit, which amplified the 2026-06-12 production outage. 120s was
deployed there as the fix; carry it into the reference config.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-06-12 21:55:57 +08:00
beardthelion 6abaf1d7ed feat: path-scoped repository visibility (Phase 1) for #18 (#25)
* feat: path-scoped repository visibility (Phase 1)

Implements per-path read visibility for repositories, addressing #18.
A repo's private status becomes a property of a path subtree rather
than a whole-repo boolean, enforced on the git read path via the node's
existing DID identity and RFC 9421 HTTP signatures.

Phase 1 scope (whole-repo enforcement, data model for the rest):
- visibility_rules data model (path_glob, mode, reader_dids) + DB access
- visibility_check pure decision function (owner-always-allow, most-
  specific rule wins, reader-DID allow-list, is_public fallback)
- enforcement on git_info_refs and git_upload_pack: unauthorized reads
  return 404 byte-identical to a missing repo (no existence leak)
- owner-only management API: PUT/DELETE/GET /api/v1/repos/{o}/{r}/visibility
- gl visibility set/remove/list CLI

mode A (hide) is whole-repo ("/") only by construction: hiding a
subtree's existence would rewrite ancestor tree hashes and diverge
history. Subtrees use mode B (content withheld, SHAs intact); B-subtree
clone filtering is deferred to a later phase and the CLI/API say so.

260 tests (69 node + 191 gl); fmt + clippy clean.

* fix: address review on path-scoped visibility

- migrations: move visibility_rules DDL out of the v1 migration into a
  new v3 (CodeRabbit, critical). Appending to v1 meant nodes that had
  already recorded v1 would skip the statements forever, so the table
  would be missing after upgrade. v3 matches the convention #23 set with
  v2 (agent_profiles) and the documented 'never append to v1' rule.
- api: validate path_glob on set (must start with '/', no trailing '/',
  only a trailing '/**' wildcard, '/' for whole-repo not '/**'). Empty
  or slash-less globs would otherwise silently misconfigure access.
  Added unit tests for the accepted/rejected forms.

281 -> 283 tests pass (88 node + 195 gl); fmt + clippy clean.
2026-06-05 22:42:26 +08:00
Kevin CodexandOpenClaude 9e6e832ec5 Infra reorg (#27)
* refactor: move fly.toml into infra/fly/

Deployment configs now live under infra/, organized per target.
Dockerfiles and docker-compose.yml intentionally stay at the repo
root (shared by CI, build scripts, and the macOS app).

Deploy with: fly deploy -c infra/fly/fly.toml

Co-Authored-By: OpenClaude <[email protected]>

* feat(infra): Terraform for single-EC2 AWS deployment

infra/aws/ provisions a production-leaning single-node deployment:
- t4g.small (AL2023 arm64) running the published ghcr.io/gitlawb/node
  image + postgres:16 via docker compose
- separate encrypted EBS data volume (prevent_destroy) with daily DLM
  snapshots; survives instance replacement
- postgres password via random_password -> SSM SecureString, fetched at
  boot by instance profile (never in user-data); optional secrets
  (operator key, Pinata JWT, S3 secret) follow the same path
- SSM Session Manager access (no SSH by default), IMDSv2 required,
  metrics port closed unless explicitly opened
- SSM command document for image upgrades (user-data runs once)

Verified: terraform fmt/init/validate clean; rendered compose passes
docker compose config; rendered user-data passes bash -n.

Co-Authored-By: OpenClaude <[email protected]>

* fix(infra): address CodeRabbit review feedback

- api_url output now honors the public_url override (local.public_url)
- disable xtrace around SSM secret fetch + .env write so secret values
  never reach the bootstrap log
- add retry/timeout flags to the docker compose download
- validate snapshot_retain_count (1-1000) at plan time
- add language tags to fenced tree blocks (MD040)

Co-Authored-By: OpenClaude <[email protected]>

* fix(infra): address PR #27 review feedback

- security group now derives its VPC from the selected subnet, so
  subnet_id overrides into non-default VPCs work
- s3_access_key_id moves to SSM SecureString (same path as the secret
  key) instead of being embedded in user-data and state
- ignore user_data drift on the instance: it only runs at first boot,
  so re-rendering caused pointless stop/starts; README documents the
  -replace workflow for config changes
- DLM snapshot targeting is stack-specific (Snapshot + Name tags), not
  any Snapshot=true volume in the account
- comment in fly.toml that it targets the shared test instance
- user-data compose up gains --remove-orphans (consistent with the
  upgrade SSM document)
- node service gets an HTTP /health healthcheck in compose

Co-Authored-By: OpenClaude <[email protected]>

* feat(infra): optional customer-managed KMS key for SSM secrets

Add ssm_kms_key_id (default null = AWS-managed aws/ssm key). When set,
all five SecureString parameters are encrypted with the CMK and the
instance role gains kms:Decrypt scoped to that key — without it the
boot-time secret fetch would fail.

Co-Authored-By: OpenClaude <[email protected]>

* fix(infra): scope CMK kms:Decrypt to this stack's SSM parameters

The grant on the customer-managed key was unconditioned, allowing the
instance role to decrypt any ciphertext under the same CMK. Restrict it
with kms:ViaService = ssm.<region>.amazonaws.com and
kms:EncryptionContext:PARAMETER_ARN limited to the stack's parameter
ARNs.

Co-Authored-By: OpenClaude <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
2026-06-05 22:38:53 +08:00
09a339745e feat: agent profiles (display name, bio, avatar, social links) (#23)
* feat: agent profiles — display name, bio, avatar, social links

Adds a profile system so agents and users can set metadata that the
frontend can render on profile pages.

CLI:
  gl profile set --name "Axiom" --bio "AI builder" --twitter AxiomBot
  gl profile show
  gl profile get <did>
  gl profile import profile.json
  gl profile export

API:
  PUT  /api/v1/profile              (authenticated — upsert own profile)
  GET  /api/v1/agents/{did}/profile (public — read any profile)

Database:
  Migration v2 adds `agent_profiles` table with display_name, bio,
  avatar_url, website, socials (JSON), profile_cid, timestamps.

Profile updates merge fields — setting --bio doesn't clear --name.
Bio capped at 280 chars, display_name at 50. Social links stored as
JSON (twitter, github, farcaster, telegram). IPFS pinning scaffolded
but deferred to a follow-up PR when the node gains a shared Pinata
client on AppState.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: apply cargo fmt to pass CI formatting check

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: suppress dead_code warnings for pin_to_ipfs field and set_profile_cid method

Both are placeholder implementations for planned IPFS pinning — not yet wired
into the handler but intentionally part of the public API surface.

Co-Authored-By: Axiom Bot <[email protected]>

* feat: wire up IPFS pinning for agent profiles via Pinata

When pin_to_ipfs: true is sent in the profile update request and the
node has GITLAWB_PINATA_JWT configured, the profile JSON is pinned to
IPFS via Pinata and the CID is stored in the database.

- Removed #[allow(dead_code)] from pin_to_ipfs field and set_profile_cid
- Profiles serialize to JSON and pin via existing pinata::pin_object
- Graceful fallback: if Pinata JWT is empty or pin fails, profile is
  saved normally without CID
- CID returned in response when available

---------

Co-authored-by: Axiom Bot <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-06-04 16:17:22 +08:00
Vasanth TandClaude Opus 4.8 2ce4da9cc5 feat(node): graceful shutdown + Prometheus metrics endpoint (#22)
* feat(db): versioned schema migrations with idempotent backfill

Replace the inline &str SQL array that re-runs on every node startup with a
versioned migration system. Each migration is recorded in a new
schema_migrations table and applied at most once per node, inside a single
transaction.

For nodes upgrading from a pre-migration-versioning build, the canonical
`repos` table is used as a signal to mark v1 as already applied without
re-running its ~140 statements — so existing operators see zero behavior
change on first restart after this commit.

Closes the gap called out in docs/OSS-READINESS-AUDIT.md:78.

- Adds Db::migration_status() returning applied (version, name, applied_at)
- Adds 6 unit tests validating the static migration catalogue
  (versions strictly increasing, names distinct, bodies non-empty, v1
  name locked to initial_schema)
- cargo fmt + clippy -D warnings + cargo test --workspace all clean

* fix(db): make legacy migration backfill safe and lock-guarded

Address two robustness gaps in the v1 migration runner on the live network:

- Remove the "repos table present => v1 complete" backfill short-circuit.
  It assumed every existing node already had the full current schema; a node
  that was behind would be marked v1-applied while still missing newer tables
  or columns. Every v1 statement is idempotent (CREATE TABLE/INDEX IF NOT
  EXISTS, ADD COLUMN IF NOT EXISTS), so legacy installs now simply run v1 once:
  existing objects are no-ops, missing ones get created.

- Guard the whole runner with a Postgres advisory lock so two instances
  against the same database (blue/green or rolling deploy) can't race to apply
  the same migration and trip the schema_migrations primary key. The lock is
  released explicitly, and automatically if the connection drops.

Also document that per-migration transactions preclude CREATE INDEX
CONCURRENTLY in future migrations.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* feat(node): graceful shutdown + Prometheus metrics endpoint

Two operational-safety changes that the live network needs:

1) Graceful shutdown. A process-wide tokio::sync::watch::Sender<bool>
   in AppState is flipped on SIGINT (all platforms) or SIGTERM (Unix).
   Every long-lived loop now selects on the shutdown arm:
     * axum server (with_graceful_shutdown — drains in-flight)
     * libp2p swarm (drops Swarm, closes connections)
     * gossip task (exits between peer announces)
     * sync worker (exits between batches)
     * operator heartbeat loop
     * rate-limit cleanup loop
     * peer-count metrics poller
     * the deferred DID-record publish task

   New GITLAWB_SHUTDOWN_GRACE_SECS (default 30s) controls how long
   axum waits for in-flight requests before returning.

2) Prometheus /metrics endpoint. New metrics module exposes:
     * gitlawb_info{version, did}                 (constant gauge = 1)
     * gitlawb_pushes_total{repo}                 (counter)
     * gitlawb_fetches_total{repo}                (counter)
     * gitlawb_auth_successes_total{route}        (counter, helper)
     * gitlawb_auth_failures_total{route,reason}  (counter, helper)
     * gitlawb_sync_queue_processed_total{status} (counter)
     * gitlawb_webhook_deliveries_total{result}   (counter)
     * gitlawb_pack_size_bytes                    (histogram)
     * gitlawb_peers_connected                    (gauge)

   Opt-in via GITLAWB_METRICS_ADDR (e.g. 127.0.0.1:9091). The endpoint
   is on a separate listener so the public port stays unexposed;
   bind to localhost or a private interface.

3) Metric instrumentation hooks in api/repos.rs (push/fetch),
   webhooks.rs (delivery outcomes), and sync.rs (per-item status).
   The auth helpers are defined but not yet wired — that's a
   focused follow-up.

Tests:
  * 3 new metrics::tests::* — encode returns valid prometheus text,
    record_* helpers are no-ops before init(), encode() errors if
    init() was never called.
  * cargo fmt + clippy -D warnings + cargo test --workspace all clean.

Closes the metrics and graceful-shutdown gaps in
docs/OSS-READINESS-AUDIT.md:131 and docs/MAINTAINER-ROADMAP.md:35.

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-02 08:58:05 +08:00
Vasanth TandClaude Opus 4.8 927e4d0cfb feat(db): versioned schema migrations with idempotent backfill (#21)
* feat(db): versioned schema migrations with idempotent backfill

Replace the inline &str SQL array that re-runs on every node startup with a
versioned migration system. Each migration is recorded in a new
schema_migrations table and applied at most once per node, inside a single
transaction.

For nodes upgrading from a pre-migration-versioning build, the canonical
`repos` table is used as a signal to mark v1 as already applied without
re-running its ~140 statements — so existing operators see zero behavior
change on first restart after this commit.

Closes the gap called out in docs/OSS-READINESS-AUDIT.md:78.

- Adds Db::migration_status() returning applied (version, name, applied_at)
- Adds 6 unit tests validating the static migration catalogue
  (versions strictly increasing, names distinct, bodies non-empty, v1
  name locked to initial_schema)
- cargo fmt + clippy -D warnings + cargo test --workspace all clean

* fix(db): make legacy migration backfill safe and lock-guarded

Address two robustness gaps in the v1 migration runner on the live network:

- Remove the "repos table present => v1 complete" backfill short-circuit.
  It assumed every existing node already had the full current schema; a node
  that was behind would be marked v1-applied while still missing newer tables
  or columns. Every v1 statement is idempotent (CREATE TABLE/INDEX IF NOT
  EXISTS, ADD COLUMN IF NOT EXISTS), so legacy installs now simply run v1 once:
  existing objects are no-ops, missing ones get created.

- Guard the whole runner with a Postgres advisory lock so two instances
  against the same database (blue/green or rolling deploy) can't race to apply
  the same migration and trip the schema_migrations primary key. The lock is
  released explicitly, and automatically if the connection drops.

Also document that per-migration transactions preclude CREATE INDEX
CONCURRENTLY in future migrations.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-01 14:42:50 +08:00
Achille Wasque 924bccd8e5 feat(gitlawb-attest): External Attestation v1 for ref-update certs (#20)
Adds an optional `attestations` field on ref-update certs plus a small
crate to sign and verify them. An envelope with no attestations
serializes to identical bytes as a bare cert, so nothing on the wire
changes for nodes that don't opt in.

Each attestation carries a `type` discriminator (`covenant/exec/v1`,
`slsa/v1.0`, `sigstore/dsse/v1`, ...), an opaque payload, a `cert_hash`
that binds it to one specific cert, a `did:key` signer, and a base64url
ed25519 signature over a domain-separated JCS-encoded input.

## Wire shape

  cert_hash = SHA-256(JCS(cert_body \ {signatures, attestations}))
              encoded as lowercase hex.

  signing_input = b"gitlawb-attest-sig/v1\n"
                  || JCS({type, payload, cert_hash})

  sig = base64url-no-pad(Ed25519(signing_input))

The cert hash is reproducible across implementations because JCS pins
key order, number formatting, and string escaping (RFC 8785). Lowercase
hex is required on the wire: the JCS signing input includes `cert_hash`
verbatim, so accepting mixed case would force every signer and verifier
to normalize before computing the signing bytes — an easy spec rule to
miss across language ports.

The domain tag lives in the byte stream, not in the canonical JSON, so
an attestation cannot be confused with a same-shape JCS document signed
for a different protocol that uses the same Ed25519 key.

Signers are `did:key` over Ed25519 with the multibase base58btc (`z`)
prefix. Non-base58btc encodings of the same public key are rejected on
verify so allowlists and signer-pinning policies stay consistent across
peers.

## Verification

`Registry` maps type to verifier. `Policy::AcceptKnown` (default) lets
unknown types pass without trust so adoption stays incremental.
`Policy::RequireAll` enforces a per-repo allowlist but stays lenient on
unknown types in the batch — anyone can attach an attestation, so
short-circuiting on an unrelated `attacker/spam/v1` would be a DoS
vector. `Policy::RejectUnknown` is strict.

`AttestedRefUpdateCert::verify_attestations(&Registry)` is the one-call
convenience over `registry.verify_all(&env.attestations, env.cert_hash()?)`.

## Canonical test vectors

Ed25519, JCS, and SHA-256 are all deterministic, so a fixed seed plus a
fixed cert body produce exactly the same `cert_hash`, `signer` DID, and
`sig` across implementations. `tests/canonical_vectors.rs` pins these
values so a port to Go, JS, Python, or any other language has a concrete
oracle to check against without reading Rust. If a vector drifts, that
is a wire-protocol break.

## Tests

42 pass: 33 unit + 4 cross-crate integration against the real
`gitlawb_core::cert::RefUpdateCert` + 4 canonical vector pins + 1
rustdoc quick-start doctest.

`cargo fmt --all -- --check` clean.
`cargo clippy --workspace --all-targets -- -D warnings` clean.
`cargo test --workspace` green; the new crate breaks no existing suite.

## Not in this PR

  - Node integration: storage, GraphQL, register flow.
  - UI surfacing.
  - Trust-score weighting policy for typed attestations.
  - Rollout ordering: old nodes silently drop the new `attestations`
    field (`RefUpdateBody` does not set `deny_unknown_fields`); signers
    and verifiers roll out independently.
  - wasm / no_std: not targeted; ed25519-dalek and serde_json are
    std-only.

## Reference implementation

End-to-end demo at open-covenant/covenant@64ec780,
`agent-os/examples/gitlawb-attest-demo/` (`covenant/exec/v1`).

Closes #7.
2026-05-31 10:05:28 +08:00
R Sundareswaran a9f208a378 Feat/ucan chain validation (#19)
* feat(ucan): validate proof chains and attenuation

* feat(auth): validate UCAN chain in middleware

* feat(server): enforce UCAN chain on write routes

* docs(ucan): clarify wildcard attenuation semantics in doc comment

* test(node): add test-only constructors for Db and RepoStore

* fix(auth): harden require_ucan_chain middleware, lower log level, and add tests

* refactor(server): centralize and deduplicate auth middleware layering
2026-05-30 14:44:40 +08:00
R Sundareswaran 491978a809 Test/core crypto primitives (#16)
* test(core/cert): add RefUpdateCert validation coverage

* test(core/cid): add CID parsing and hash tests

* test(core/did): add DID validation and predicate tests

* test(core/identity): add keypair and signed payload tests

* style(did): wrap long assert! messages to satisfy rustfmt

* style(identity): wrap long assert! messages to satisfy rustfmt
2026-05-29 16:42:32 +08:00
Vasanth T 0b056f3ad0 [codex] fix Dependabot Rust alerts (#15)
* fix(deps): resolve dependabot rust alerts

* docs: clarify dependabot validation
2026-05-28 16:44:05 +08:00
Vasanth T d366fc6b97 [codex] chore: add live-safe OSS readiness pass (#14)
* chore: add live-safe OSS readiness pass

* fix(node): verify optional peer signatures during rollout

* docs: rewrite README for node roadmap
2026-05-28 15:34:47 +08:00
Kevin CodexandClaude Opus 4.7 b12c6bc328 feat: per-DID rate limiting on creation endpoints (10/hour) (#13)
Add sliding-window rate limiter applied to resource creation endpoints:
- POST /api/v1/repos (repo creation)
- POST /api/register (agent registration)
- POST /api/v1/repos/:owner/:repo/fork
- POST /api/v1/repos/:owner/:repo/issues
- POST /api/v1/repos/:owner/:repo/pulls

Returns 429 Too Many Requests with Retry-After header when exceeded.
Other write operations (merge, close, comment, star) are not limited
since they're bounded by existing resources.

Includes 5 unit tests covering: within-limit, over-limit, per-key
isolation, window expiry, and cleanup of stale entries.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-27 12:29:02 +08:00
Kevin CodexandClaude Opus 4.7 14a1cdd656 chore: bump v0.3.9, fix all clippy warnings, enforce strict CI lint (#12)
* chore: bump v0.3.9, fix all clippy warnings, enforce strict CI lint

- Bump workspace version 0.3.8 → 0.3.9
- Fix 22 clippy warnings across 17 files:
  - Replace .last() with .next_back() on DoubleEndedIterators (8 instances)
  - Remove useless .into() conversions on anyhow::Error (3)
  - Remove needless borrows in gl init (3)
  - Replace map_or with is_some_and/is_none_or (3)
  - Collapse consecutive str::replace into char array pattern (2)
  - Change &PathBuf to &Path in function signatures (2)
  - Remove dead code: unused init_bare_repo function
  - Collapse if-let into outer match arm
  - Replace vec![...] with array literal
  - Introduce RecordAnchorInput struct (10 params → 1 struct)
  - Fix always-true boolean condition in gossip task guard
- Promote clippy from advisory to strict (-D warnings) in CI

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-27 10:28:14 +08:00
rapybus be099386e5 Update bootstrap-peers.json (#5) 2026-05-27 09:38:12 +08:00
b09714af0b fix(doctor): detect git-remote-gitlawb.exe on Windows (#10)
which_in_path() now also checks for `{name}.exe` on Windows targets,
so `gl doctor` correctly finds git-remote-gitlawb on that platform.

Co-authored-by: kelvinzer0 <[email protected]>
Co-authored-by: OpenClaude (mimo-v2.5-pro) <[email protected]>
2026-05-27 09:37:58 +08:00
Rimantas (Rimas) MoceviciusandRimas Mocevicius d09c23da86 feat: macOS menu bar app for managing gitlawb-node Docker Compose stack (#8)
- Menu bar icon with colored status dot (green/yellow/red)
- Start/Stop node via Docker Compose
- Settings UI: ports, public URL, postgres password, auto-sync, operator config
- View Logs window with live docker compose logs
- Auto-start on launch and login (SMAppService)
- Docker detection with install prompt
- Repo root detection via compile-time path for local builds
- Port variables wired through docker-compose.yml for custom port support
- GITLAWB_AUTO_SYNC passthrough for peer replication toggle
- Build script: scripts/build-macos-app.sh (codesign + DMG)

Co-authored-by: Rimas Mocevicius <[email protected]>
2026-05-27 09:37:36 +08:00
Esky33 f3df4a4dd4 Resolve bare DID aliases for trust and cert lookups (#9)
* Resolve bare DID aliases for trust and cert lookups

Public profile and CLI surfaces accept bare z6Mk keys, but trust and certificate data are stored against full did:key identities. Normalize agent profile lookups and make cert commands resolve owner-less repos against the caller DID so valid pushes are not hidden behind node-owned paths.

Constraint: Public URLs and CLI examples commonly use short z6Mk identifiers

Constraint: Ref certificates are issued after push and stored under the owner repo id

Rejected: Register duplicate bare-key identities | would split trust and certificate history across aliases

Rejected: Require full certificate UUIDs in show output | list already displays short IDs as the human-facing handle

Confidence: high

Scope-risk: narrow

Directive: Keep bare key inputs as aliases for did:key, not separate identities

Tested: cargo check -p gl

Tested: cargo run -q -p gl -- cert list agentbot-opensource

Tested: cargo run -q -p gl -- cert show agentbot-opensource 3686b1fe

Tested: cargo check -p gitlawb-node

Tested: cargo test -p gitlawb-node normalize_agent_did

* Preserve agent rank on short profile URLs

The public profile route can use compact handles like z6MkpUq1, while trust history is stored under the full did:key identity. Resolve exact aliases first, then fall back to matching registered agent key prefixes so older high-level identities are not displayed as fresh newcomers.

Constraint: Public GitLawb profile URLs expose short z6Mk handles.

Constraint: Trust and push history remain keyed by canonical did:key identities.

Rejected: Mint duplicate short-handle agent rows | would split rank and push history across aliases.

Confidence: high

Scope-risk: narrow

Directive: Keep short profile handles as aliases for canonical did:key identities, never as separate trust principals.

Tested: cargo fmt --check -p gitlawb-node

Tested: cargo test -p gitlawb-node agent

Tested: cargo check -p gitlawb-node

Tested: cargo check -p gl
2026-05-27 09:36:37 +08:00
Kevin CodexandOpenClaude 034cb3c80c feat: replica registration protocol (Phase 2 v1) (#4)
Phase 2 of the network-resilience plan: any node can register itself as a
replica of someone else's repo, and the origin exposes the replica list
publicly as a "your repo is mirrored by N nodes" trust signal.

What ships
- New repo_replicas table keyed by (repo_id, replica_did) with replica URL
  and registration timestamp. UNIQUE on (repo_id, replica_did); duplicate
  registrations update the URL.
- API on origin:
    PUT  /api/v1/repos/:owner/:repo/replicas   (auth) register
    DELETE /api/v1/repos/:owner/:repo/replicas (auth) unregister
    GET  /api/v1/repos/:owner/:repo/replicas   (public) list
  Origin rejects self-registration and validates the replica URL (http(s),
  no whitespace/control chars, length-bounded).
- CLI:
    gl repo replica-register <owner>/<repo> --url <my-public-url>
    gl repo replica-unregister <owner>/<repo>
    gl repo replicas <owner>/<repo>
- gl repo info now shows "Replicas: N" when the origin supports it.
- Tests: 5 new unit tests for the URL validator. Full node suite at 51 pass,
  full gl suite at 188 pass.

Deferred to Phase 2.5
- Replica auto-pulls the repo on registration (today: operator runs
  git clone gitlawb://... manually after registering).
- Periodic freshness sync worker driven by the replica list.
- Gossipsub subscription to push ref-update events to replicas live.

Co-authored-by: OpenClaude <[email protected]>
2026-05-27 09:36:12 +08:00
d2a440e521 Feat/backport perf fixes (#11)
* feat: replica registration protocol (Phase 2 v1)

Phase 2 of the network-resilience plan: any node can register itself as a
replica of someone else's repo, and the origin exposes the replica list
publicly as a "your repo is mirrored by N nodes" trust signal.

What ships
- New repo_replicas table keyed by (repo_id, replica_did) with replica URL
  and registration timestamp. UNIQUE on (repo_id, replica_did); duplicate
  registrations update the URL.
- API on origin:
    PUT  /api/v1/repos/:owner/:repo/replicas   (auth) register
    DELETE /api/v1/repos/:owner/:repo/replicas (auth) unregister
    GET  /api/v1/repos/:owner/:repo/replicas   (public) list
  Origin rejects self-registration and validates the replica URL (http(s),
  no whitespace/control chars, length-bounded).
- CLI:
    gl repo replica-register <owner>/<repo> --url <my-public-url>
    gl repo replica-unregister <owner>/<repo>
    gl repo replicas <owner>/<repo>
- gl repo info now shows "Replicas: N" when the origin supports it.
- Tests: 5 new unit tests for the URL validator. Full node suite at 51 pass,
  full gl suite at 188 pass.

Deferred to Phase 2.5
- Replica auto-pulls the repo on registration (today: operator runs
  git clone gitlawb://... manually after registering).
- Periodic freshness sync worker driven by the replica list.
- Gossipsub subscription to push ref-update events to replicas live.

Co-Authored-By: OpenClaude <[email protected]>

* perf: backport query optimizations + add Fly test deployment

- Add expression indexes (owner_short+name, updated_at DESC) for fast repo listing
- Add list_all_repos_with_stars() batch query eliminating N+1 star count calls
- Add list_all_repos_paged() with DISTINCT ON dedup + pagination + X-Total-Count header
- Add prune_self_peers() to remove stale self-referencing peer rows at startup
- Add self-loop prevention in HTTP peer notifications during git-receive-pack
- Add SwarmStatus observability to p2p_info endpoint (connected/mesh/all peers)
- Add fly.toml for gitlawb-node-test deployment in iad region

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-27 09:35:46 +08:00
Kevin CodexandOpenClaude f312955c35 feat: replica registration protocol (Phase 2 v1) (#3)
Phase 2 of the network-resilience plan: any node can register itself as a
replica of someone else's repo, and the origin exposes the replica list
publicly as a "your repo is mirrored by N nodes" trust signal.

What ships
- New repo_replicas table keyed by (repo_id, replica_did) with replica URL
  and registration timestamp. UNIQUE on (repo_id, replica_did); duplicate
  registrations update the URL.
- API on origin:
    PUT  /api/v1/repos/:owner/:repo/replicas   (auth) register
    DELETE /api/v1/repos/:owner/:repo/replicas (auth) unregister
    GET  /api/v1/repos/:owner/:repo/replicas   (public) list
  Origin rejects self-registration and validates the replica URL (http(s),
  no whitespace/control chars, length-bounded).
- CLI:
    gl repo replica-register <owner>/<repo> --url <my-public-url>
    gl repo replica-unregister <owner>/<repo>
    gl repo replicas <owner>/<repo>
- gl repo info now shows "Replicas: N" when the origin supports it.
- Tests: 5 new unit tests for the URL validator. Full node suite at 51 pass,
  full gl suite at 188 pass.

Deferred to Phase 2.5
- Replica auto-pulls the repo on registration (today: operator runs
  git clone gitlawb://... manually after registering).
- Periodic freshness sync worker driven by the replica list.
- Gossipsub subscription to push ref-update events to replicas live.

Co-authored-by: OpenClaude <[email protected]>
2026-05-13 20:00:40 +08:00
Kevin CodexandOpenClaude ec4ef81770 feat: embed bootstrap peers seed list for automatic network discovery (#1)
* feat: embed bootstrap peers seed list for automatic network discovery

A fresh \`docker compose up\` now joins the Gitlawb network with zero
manual peer configuration. The node parses an embedded \`bootstrap-peers.json\`
on startup and merges the entries into both the HTTP gossip task and the
libp2p Kademlia bootstrap.

- Add \`bootstrap-peers.json\` at repo root (versioned schema, PR-friendly)
- New \`bootstrap\` module in the node crate (parse + merge_seeds)
- Wire into \`main\` after \`Config::parse\`
- Operators can opt out via \`GITLAWB_BOOTSTRAP_DISABLE_SEEDS\` for
  isolated dev networks

Also in this commit:
- \`cargo fmt --all\` over the entire workspace (no logic changes)
- Downgrade CI clippy step to advisory (\`continue-on-error: true\`) until
  the existing lint backlog is cleared. fmt + tests stay strict.

Co-Authored-By: OpenClaude <[email protected]>

* test: cover bootstrap merge logic with unit tests

Refactor bootstrap.rs into pure functions (parse_seed_list, merge_into_vecs)
so the parse + merge logic can be tested without constructing a Config or
mutating process-global env vars.

Adds 11 tests covering:

- valid v1 list parses
- unknown version is rejected
- malformed JSON is rejected
- empty / missing peers array
- merge appends new http + p2p entries
- merge dedupes against existing entries
- invalid p2p_multiaddr is skipped (http still added)
- empty strings are skipped
- null optional fields are tolerated
- the canonical bootstrap-peers.json shipped in the repo always parses
  (regression guard against future schema changes)

Co-Authored-By: OpenClaude <[email protected]>

* fix(security): reject path traversal in repo_store local_path

Closes CodeQL alert #4 (Uncontrolled data used in path expression).

Both `owner_did` and `repo_name` come from URL parameters and were used
unsanitized to build a filesystem path, so a request like
`/did:key:foo/../../../etc/passwd.git/info/refs` could escape the repos
directory.

Fix:

- New `validate_owner_did` and `validate_repo_name` enforce a strict
  allowlist before path construction (alphanumeric + `: . _ -` for DIDs;
  alphanumeric + `. _ -` for repo names; rejects empty, `..`, leading
  `.` or `-`, slashes, backslashes, null bytes, overlong inputs).
- `RepoStore::local_path` now returns `Result<…>` and the 5 callers
  propagate the error.
- Defence in depth: even after the allowlist passes, the joined path is
  checked to still be rooted at `repos_dir`.
- 17 unit tests cover normal DIDs/repo names plus every malicious shape
  (`..`, `/`, `\`, leading dot/dash, null byte, overlong, did:web with
  dots).

Co-Authored-By: OpenClaude <[email protected]>

* fix(security): add Path::components barrier for CodeQL rust/path-injection

CodeQL's rust/path-injection query did not recognise the existing
allowlist validator as a sanitiser, so it kept flagging local_path even
though every malicious input was already rejected.

Adds an explicit Path::components() walk after path construction that
rejects any ParentDir or CurDir segment. This is the barrier pattern
CodeQL recognises — and is also genuine defence-in-depth in case a
future change ever weakens the allowlist.

Allowlist + starts_with + components walk = three independent layers
between attacker-controlled HTTP input and filesystem access. All 17
existing tests still pass.

Co-Authored-By: OpenClaude <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
2026-05-13 19:11:54 +08:00
178 changed files with 30887 additions and 2883 deletions

No files matched your search

+38
View File
@@ -0,0 +1,38 @@
# cargo-audit configuration.
#
# cargo-audit reads its config from .cargo/audit.toml (relative to the working
# directory) or ~/.cargo/audit.toml. It does NOT read a repo-root audit.toml,
# and there is no CLI flag to point at a config file (`-f` is the lockfile path).
# This file lives at .cargo/audit.toml for that reason.
#
# The audit job stays hard-fail: only the advisories explicitly listed below are
# suppressed, so any NEW advisory still breaks CI. Each entry below has no
# available upstream fix; everything fixable is fixed in Cargo.lock instead.
#
# Tracking: the hickory ignores are reachable accepted risk. A scheduled re-check
# (.github/workflows/audit-schedule.yml) runs without these ignores and fails if
# the lockfile moves off the pinned vulnerable versions while the ignores remain.
[advisories]
ignore = [
# hickory-proto 0.25.2 (pulled by libp2p-dns 0.44 -> hickory-resolver 0.25).
# REACHABLE: the node wraps its QUIC transport in the system DNS resolver
# (crates/gitlawb-node/src/p2p/mod.rs:238), so a malicious DNS response can
# trigger these DoS paths. No fix is available: the latest libp2p-dns on
# crates.io is 0.44 and pins hickory 0.25; the fix requires hickory >=0.26.1.
# REMOVE both once libp2p-dns ships a release using hickory 0.26 (verified
# 2026-06-21: cargo update -p hickory-proto locks 0 packages; still 0.25.2).
# Tracking: https://github.com/Gitlawb/node/issues/76
"RUSTSEC-2026-0118", # hickory NSEC3 closest-encloser unbounded loop
"RUSTSEC-2026-0119", # hickory O(n^2) name-compression CPU exhaustion
# rsa 0.9.10 (Marvin timing sidechannel). Present in Cargo.lock because sqlx
# resolves sqlx-mysql unconditionally, and sqlx-mysql depends on rsa. cargo
# audit scans the lockfile, so it flags rsa even though it is NOT linked into
# our binary: sqlx is built postgres-only (crates/gitlawb-node/Cargo.toml),
# so the MySQL RSA auth path is never compiled and there is no RSA oracle to
# attack. No upstream fix exists regardless. REMOVE this ignore if sqlx is
# ever built with the `mysql` feature, or any other consumer of rsa enters
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
]
+40 -2
View File
@@ -63,13 +63,51 @@ POSTGRES_PASSWORD=changeme
# ── P2P networking ────────────────────────────────────────────────────────
GITLAWB_P2P_PORT=7546
# Comma-separated multiaddrs of bootstrap peers
# Comma-separated HTTP peer URLs to announce to on startup.
# The binary also ships with bootstrap-peers.json; set
# GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true for isolated dev/test networks.
GITLAWB_BOOTSTRAP_PEERS=
GITLAWB_BOOTSTRAP_DISABLE_SEEDS=false
# Require signatures on peer announce/sync write routes. Keep false during
# rolling upgrades so existing live nodes can still communicate.
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Require the authenticated pusher to be the repo owner on git-receive-pack.
# A valid did:key signature is authentication, not authorization: anyone can
# sign as their own DID. When true, pushes from a non-owner DID are rejected.
# Keep false until the repo owner is ready for owner-only writes.
GITLAWB_ENFORCE_OWNER_PUSH=false
# Comma-separated libp2p multiaddrs.
# Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
GITLAWB_P2P_BOOTSTRAP=
# ── Access control ────────────────────────────────────────────────────────
# Set to false to require auth on all read endpoints
# Reserved for private-read mode. Public/private repo read enforcement is not
# wired in the current live release; do not rely on this for private repositories.
GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
GITLAWB_MAX_PACK_BYTES=2147483648
# ── Sync ─────────────────────────────────────────────────────────────────
# Enable automatic background sync from known peers
GITLAWB_AUTO_SYNC=false
# ── iCaptcha proof-of-intelligence gate ───────────────────────────────────
# Optional gate on create_repo + register: require callers to present an
# iCaptcha proof (X-ICaptcha-Proof header) earned at icaptcha.gitlawb.com.
# Default off = fully inert (no behavior change).
# off - gate disabled (default)
# shadow - verify + log would-be rejections, but always allow
# enforce - reject requests without a valid, strong-enough proof
ICAPTCHA_MODE=off
# Optional base64url Ed25519 public key override; if unset it is fetched from
# ICAPTCHA_URL/v1/pubkey at startup. Recommended in production: pinning the key
# removes the startup dependency on the iCaptcha service being reachable and
# avoids silently running inert if that one-time fetch fails.
ICAPTCHA_PUBKEY=
# Minimum proof difficulty level required to pass.
ICAPTCHA_REQUIRED_LEVEL=3
# Base URL of the iCaptcha service.
ICAPTCHA_URL=https://icaptcha.gitlawb.com
+11
View File
@@ -0,0 +1,11 @@
# Ownership for CI, release, and security-sensitive infrastructure.
# A PR that edits these paths requires review from a code owner once
# branch protection's "Require review from Code Owners" is enabled.
/.github/ @beardthelion
/Dockerfile @beardthelion
/Dockerfile.bins @beardthelion
/docker-compose.yml @beardthelion
/release-please-config.json @beardthelion
/.release-please-manifest.json @beardthelion
/SECURITY.md @beardthelion
+66
View File
@@ -0,0 +1,66 @@
name: Bug report
description: Report a defect in a Gitlawb crate or node behavior
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for the report. Please fill in the fields below so we can reproduce the issue.
For security vulnerabilities, do NOT open an issue — follow SECURITY.md instead.
- type: textarea
id: what-happened
attributes:
label: What happened?
description: A clear description of the bug.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: repro
attributes:
label: Steps to reproduce
placeholder: |
1. ...
2. ...
3. ...
validations:
required: true
- type: dropdown
id: crate
attributes:
label: Affected crate
options:
- gitlawb-node
- gl
- git-remote-gitlawb
- gitlawb-core
- not sure / other
validations:
required: true
- type: input
id: version
attributes:
label: Version or commit
description: Output of the binary's `--version`, or the git commit you built.
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
description: OS, Rust version (`rustc --version`), and anything else relevant.
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste logs or errors. This will be rendered as code, no need for backticks.
render: shell
validations:
required: false
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/Gitlawb/node/blob/main/SECURITY.md
about: Please report security issues privately per our security policy, not as a public issue.
- name: Questions & discussion
url: https://github.com/Gitlawb/node/discussions
about: For usage questions and open-ended discussion.
@@ -0,0 +1,34 @@
name: Feature request
description: Suggest an improvement or new capability
labels: ["enhancement"]
body:
- type: textarea
id: problem
attributes:
label: Problem / use case
description: What are you trying to do that's hard or impossible today?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
validations:
required: false
- type: dropdown
id: protocol
attributes:
label: Does this touch the protocol?
description: Identity, signatures, UCAN, ref certs, or P2P wire formats. Protocol changes need an issue discussion before any PR.
options:
- "No"
- "Yes"
- "Not sure"
validations:
required: true
+63
View File
@@ -0,0 +1,63 @@
<!--
Thanks for contributing to Gitlawb. Keep PRs focused on one change.
Protocol-level changes (identity, signatures, wire format) should start
as an issue before code — see CONTRIBUTING.md.
-->
## Summary
<!-- One or two sentences: what changes and why it matters. -->
## Motivation & context
<!-- What problem does this solve? Link prior discussion if any. -->
Closes #
## Kind of change
- [ ] Bug fix
- [ ] Feature
- [ ] Security fix
- [ ] Docs
- [ ] Tests / CI
- [ ] Refactor (no behavior change)
- [ ] Breaking or protocol change (issue required first)
## What changed
<!-- Bullet the concrete changes. Note the crate(s) touched:
gitlawb-core / gitlawb-node / gl / git-remote-gitlawb -->
-
## How a reviewer can verify
<!-- Commands to run, or repro steps for a bug plus proof it's fixed. -->
```sh
```
## Before you request review
- [ ] Scope is one logical change; no unrelated churn
- [ ] `cargo test --workspace` passes locally
- [ ] New behavior is covered by tests (required for fixes)
- [ ] `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` are clean
- [ ] Commit titles use Conventional Commits (`feat(...)`, `fix(...)`, `docs(...)`)
- [ ] Docs / `.env.example` updated if behavior or config changed (or N/A)
- [ ] Checked existing PRs so this isn't a duplicate
## Protocol & signing impact
<!-- Delete this block if your change doesn't touch the protocol. -->
- [ ] Touches DID / `did:key`, Ed25519 / RFC 9421 signatures, UCAN, ref certs, or P2P wire formats
- [ ] Discussed in an issue before implementation
- [ ] Backward-compatible with existing nodes and previously signed history
## Notes for reviewers
<!-- Anything out of scope, follow-ups, or known limitations. Optional. -->
+77
View File
@@ -0,0 +1,77 @@
name: Scheduled Audit
# Self-expiring suppression check (companion to the PR-blocking audit gate in
# pr-checks.yml). Runs weekly. It re-runs cargo audit WITHOUT the .cargo/audit.toml
# ignore list to surface the full advisory set for visibility, and it hard-fails
# if the lockfile has moved off the pinned vulnerable versions while the ignores
# are still present, i.e. the upstream fix landed but the ignore was not dropped.
on:
schedule:
- cron: "0 6 * * 1" # Mondays 06:00 UTC
workflow_dispatch:
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
jobs:
scheduled-audit:
name: scheduled audit (no ignores)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit-schedule
- name: Install cargo-audit
run: cargo install --locked cargo-audit
# Full advisory report with NO suppressions. Run from a scratch dir that
# has no .cargo/audit.toml so the ignore list does not apply; never fail
# the build on this step, it is visibility only.
- name: Full audit report (ignores not applied)
run: |
mkdir -p "$RUNNER_TEMP/full-audit"
cp Cargo.lock "$RUNNER_TEMP/full-audit/Cargo.lock"
cd "$RUNNER_TEMP/full-audit"
{
echo '### Full cargo audit (no ignores applied)'
echo '```'
cargo audit -f Cargo.lock || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
# Drift guard: the hickory ignores are only valid while hickory-proto is
# pinned at 0.25.x. If the lockfile has moved past that, the upstream fix
# is in and the ignores in .cargo/audit.toml must be removed. Fail loudly.
- name: Fail if hickory moved past 0.25.x while ignores remain
run: |
set -euo pipefail
version="$(grep -A1 'name = "hickory-proto"' Cargo.lock | grep '^version' | head -1 | cut -d'"' -f2)"
echo "hickory-proto in Cargo.lock: ${version:-not present}"
ignores_present=false
if grep -q 'RUSTSEC-2026-011' .cargo/audit.toml; then
ignores_present=true
fi
# Drift = the lockfile no longer matches what the ignores assume:
# hickory moved off 0.25.x (fix shipped) OR was removed entirely
# (dead ignore entries left behind). Both must fail.
if [ "$ignores_present" = true ] && { [ -z "${version}" ] || [[ "${version}" != 0.25.* ]]; }; then
echo "::error::hickory-proto is ${version:-absent from Cargo.lock} but the RUSTSEC-2026-0118/0119 ignores are still in .cargo/audit.toml. The upstream fix appears to be available or the dependency is gone; remove the ignores."
exit 1
fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version."
+156 -3
View File
@@ -9,18 +9,25 @@ on:
permissions:
contents: read
# Cancel superseded runs on the same PR/branch; never cancel runs on main.
concurrency:
group: pr-checks-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
jobs:
fmt-clippy-test:
name: fmt + clippy + test
fmt-clippy:
name: fmt + clippy
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
@@ -37,5 +44,151 @@ jobs:
- name: cargo clippy
run: cargo clippy --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
toolchain: [stable, beta]
# Beta is informational: an upstream beta regression should warn, not block merges.
continue-on-error: ${{ matrix.toolchain == 'beta' }}
# Postgres for the DB-backed integration tests (#[sqlx::test] provisions an
# isolated database per test off this connection). The health-check gate keeps
# `cargo test` from racing a not-yet-ready server.
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gitlawb_test
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: ${{ matrix.toolchain }}
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: ${{ matrix.toolchain }}
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --workspace
build-release:
name: build --release
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release
- name: cargo build --release
run: cargo build --release --workspace
audit:
name: cargo audit
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit
- name: Install cargo-audit
run: cargo install --locked cargo-audit
# Hard-fail gate. Suppressions live in .cargo/audit.toml (read automatically
# from the repo root), each with no available upstream fix. A green check
# here means "no NEW advisories", not "advisory-clean": surface the active
# ignore list so a reviewer sees exactly what is being accepted.
- name: Show active audit ignores
run: |
{
echo '### cargo audit: accepted advisories (.cargo/audit.toml)'
echo '```toml'
sed -n '/^\[advisories\]/,$p' .cargo/audit.toml
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- name: cargo audit
run: cargo audit
msrv:
name: MSRV (Rust 1.91)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust 1.91
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # 1.91
with:
toolchain: "1.91"
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: msrv
- name: cargo check on MSRV
run: cargo check --workspace --all-targets
docker-build:
name: Docker build smoke test
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Build image
run: docker build -t gitlawb-node:ci-test .
- name: Smoke test --version
run: docker run --rm gitlawb-node:ci-test --version
+124
View File
@@ -0,0 +1,124 @@
name: PR Triage
# Runs on pull_request_target, not pull_request: fork PRs get a read-only
# GITHUB_TOKEN under pull_request regardless of the permissions below, so the
# label/comment writes 403. This job never checks out or runs PR head code (it
# only reads context.payload and the files API), so pull_request_target is safe.
on:
pull_request_target:
types: [opened, edited, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: write
concurrency:
group: pr-triage-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
triage:
name: Quality-signal triage
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Label and guide
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const pr = context.payload.pull_request;
const { owner, repo } = context.repo;
const prNumber = pr.number;
// Trusted authors bypass triage entirely.
const trusted = ["OWNER", "MEMBER", "COLLABORATOR"];
if (trusted.includes(pr.author_association)) {
core.info(`Author association ${pr.author_association} is trusted; skipping triage.`);
return;
}
// Labels this workflow owns. Anything here may be added or removed each run.
const MANAGED = ["needs-issue", "needs-description", "needs-tests", "workflow-change"];
const want = new Set();
const body = pr.body || "";
// Linked issue: look for "closes/fixes/resolves #N" or any bare #N reference.
const hasLinkedIssue =
/\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\b[^\n]*#\d+/i.test(body) ||
/#\d+/.test(body);
if (!hasLinkedIssue) want.add("needs-issue");
// Description substance: strip HTML comments and markdown headings, then measure.
const stripped = body
.replace(/<!--[\s\S]*?-->/g, "")
.replace(/^#+.*$/gm, "")
.trim();
if (stripped.length < 50) want.add("needs-description");
// File-based signals.
const files = await github.paginate(github.rest.pulls.listFiles, {
owner, repo, pull_number: prNumber, per_page: 100,
});
const names = files.map(f => f.filename);
const isFork = pr.head.repo && pr.head.repo.full_name !== pr.base.repo.full_name;
const touchesWorkflows = names.some(n => n.startsWith(".github/workflows/"));
if (isFork && touchesWorkflows) want.add("workflow-change");
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or (the common Rust
// pattern) an inline #[test] / #[cfg(test)] block added inside the source file.
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
// Reconcile labels: add wanted managed labels, remove managed labels no longer wanted.
const current = (await github.rest.issues.listLabelsOnIssue({
owner, repo, issue_number: prNumber,
})).data.map(l => l.name);
const toAdd = [...want].filter(l => !current.includes(l));
const toRemove = MANAGED.filter(l => current.includes(l) && !want.has(l));
if (toAdd.length) {
await github.rest.issues.addLabels({ owner, repo, issue_number: prNumber, labels: toAdd });
}
for (const name of toRemove) {
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNumber, name })
.catch(() => {}); // ignore if already gone
}
// Guidance comment: only for advisory signals, and only once.
const advisory = ["needs-issue", "needs-description", "needs-tests"].filter(l => want.has(l));
if (!advisory.length) return;
const MARKER = "<!-- gitlawb-triage -->";
const comments = await github.paginate(github.rest.issues.listComments, {
owner, repo, issue_number: prNumber, per_page: 100,
});
if (comments.some(c => c.body && c.body.includes(MARKER))) {
core.info("Guidance comment already present; not reposting.");
return;
}
const reasons = {
"needs-issue": "Link the issue this addresses (`Closes #123`). For protocol changes, open an issue first.",
"needs-description": "Add a short summary of what changes and why: the template's Summary and Motivation sections.",
"needs-tests": "This changes Rust source but no tests changed. Tests are required for fixes and strongly encouraged for features.",
};
const lines = advisory.map(l => `- ${reasons[l]}`).join("\n");
const comment = [
MARKER,
"Thanks for the contribution. A couple of things will help us review this faster:",
"",
lines,
"",
"See [CONTRIBUTING.md](https://github.com/Gitlawb/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
].join("\n");
await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: comment });
+405 -15
View File
@@ -9,6 +9,9 @@ concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
# Least privilege by default; every job opts in to exactly what it needs.
permissions: {}
jobs:
release-please:
if: ${{ github.repository == 'Gitlawb/node' }}
@@ -24,7 +27,7 @@ jobs:
steps:
- name: Run release-please
id: release
uses: googleapis/release-please-action@16a9c90856f42705d54a6fda1823352bdc62cf38 # v4.3.0
uses: googleapis/release-please-action@45996ed1f6d02564a971a2fa1b5860e934307cf7 # v5.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -41,6 +44,7 @@ jobs:
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Set up QEMU
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
@@ -101,6 +105,9 @@ jobs:
needs: release-please
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ${{ matrix.os }}
# Windows is best-effort: a failure there must not fail the job or block the
# downstream npm/Homebrew jobs that only consume the unix artifacts.
continue-on-error: ${{ startsWith(matrix.target, 'x86_64-pc-windows') }}
permissions:
contents: write
strategy:
@@ -115,11 +122,14 @@ jobs:
os: macos-13
- target: aarch64-apple-darwin
os: macos-14
- target: x86_64-pc-windows-msvc
os: windows-latest
steps:
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
@@ -143,30 +153,410 @@ jobs:
with:
key: ${{ matrix.target }}
- name: Build
run: cargo build --release --target ${{ matrix.target }} -p gl -p git-remote-gitlawb -p gitlawb-node
# The gitlawb-node daemon is a Linux/macOS service; on Windows we ship only
# the two CLI binaries. A Windows-only failure must not block the release.
- name: Determine binaries
shell: bash
run: |
BINS="gl git-remote-gitlawb gitlawb-node"
case "${{ matrix.target }}" in *windows*) BINS="gl git-remote-gitlawb" ;; esac
echo "BINS=$BINS" >> "$GITHUB_ENV"
- name: Package
id: pkg
- name: Build
shell: bash
run: |
set -euo pipefail
NAME="gitlawb-node-${{ needs.release-please.outputs.version }}-${{ matrix.target }}"
args=""
for b in $BINS; do args="$args -p $b"; done
cargo build --release --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Skipped for cross-compiled targets that can't run
# on the host runner (aarch64 linux on an x86_64 runner).
- name: Smoke test binaries
if: ${{ matrix.target != 'aarch64-unknown-linux-musl' }}
shell: bash
run: |
set -euo pipefail
VERSION="${{ needs.release-please.outputs.version }}"
BIN_DIR="target/${{ matrix.target }}/release"
EXE=""
case "${{ matrix.target }}" in *windows*) EXE=".exe" ;; esac
for bin in $BINS; do
echo "== $bin --version =="
out="$("$BIN_DIR/$bin$EXE" --version)"
echo "$out"
# Each binary prints "<name> <version>"; assert the released version is present.
grep -qF "$VERSION" <<<"$out" || {
echo "::error::$bin --version did not report expected version $VERSION (got: $out)"
exit 1
}
done
- name: Package
shell: bash
run: |
set -euo pipefail
TARGET="${{ matrix.target }}"
NAME="gitlawb-node-${{ needs.release-please.outputs.version }}-${TARGET}"
BIN_DIR="target/${TARGET}/release"
EXE=""
case "$TARGET" in *windows*) EXE=".exe" ;; esac
# Portable sha256 → "<hash> <file>" so install scripts and brew can parse it.
sha256_file() {
local f="$1"
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$f"
elif command -v shasum >/dev/null 2>&1; then
shasum -a 256 "$f"
else
local h
h=$(certutil -hashfile "$f" SHA256 | sed -n 2p | tr -d ' \r')
printf '%s %s\n' "$h" "$f"
fi
}
mkdir -p "dist/$NAME"
cp "target/${{ matrix.target }}/release/gl" "dist/$NAME/"
cp "target/${{ matrix.target }}/release/git-remote-gitlawb" "dist/$NAME/"
cp "target/${{ matrix.target }}/release/gitlawb-node" "dist/$NAME/"
for bin in $BINS; do
cp "$BIN_DIR/$bin$EXE" "dist/$NAME/"
done
cp README.md LICENSE-MIT LICENSE-APACHE "dist/$NAME/"
cd dist
tar czf "$NAME.tar.gz" "$NAME"
shasum -a 256 "$NAME.tar.gz" > "$NAME.tar.gz.sha256"
echo "asset=dist/$NAME.tar.gz" >> "$GITHUB_OUTPUT"
echo "checksum=dist/$NAME.tar.gz.sha256" >> "$GITHUB_OUTPUT"
case "$TARGET" in
*windows*)
# 7z is preinstalled on windows-latest runners.
7z a -tzip "$NAME.zip" "$NAME" >/dev/null
sha256_file "$NAME.zip" > "$NAME.zip.sha256"
;;
*)
tar czf "$NAME.tar.gz" "$NAME"
sha256_file "$NAME.tar.gz" > "$NAME.tar.gz.sha256"
;;
esac
- name: Attach to release
uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8 # v2.3.2
with:
tag_name: ${{ needs.release-please.outputs.tag_name }}
files: |
${{ steps.pkg.outputs.asset }}
${{ steps.pkg.outputs.checksum }}
dist/*.tar.gz
dist/*.zip
dist/*.sha256
npm-publish:
name: Publish to npm
needs: [release-please, release-binaries]
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # npm provenance
steps:
- name: Guard on secret
id: guard
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
if [ -z "${NPM_TOKEN:-}" ]; then
echo "::warning::NPM_TOKEN is not set — skipping npm publish."
echo "enabled=false" >> "$GITHUB_OUTPUT"
else
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Checkout release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Lay in release binaries
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# npm platform package -> Rust target triple (unix only; Windows is not
# published to npm).
MAP="
gl-darwin-arm64:aarch64-apple-darwin
gl-darwin-x64:x86_64-apple-darwin
gl-linux-arm64:aarch64-unknown-linux-musl
gl-linux-x64:x86_64-unknown-linux-musl
"
mkdir -p _dl
for entry in $MAP; do
pkg="${entry%%:*}"
target="${entry#*:}"
archive="gitlawb-node-${VERSION}-${target}.tar.gz"
echo "==> $pkg <- $archive"
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern "$archive" --dir _dl --clobber
tar -xzf "_dl/$archive" -C _dl
src="_dl/gitlawb-node-${VERSION}-${target}"
cp "$src/gl" "npm/packages/$pkg/gl"
cp "$src/git-remote-gitlawb" "npm/packages/$pkg/git-remote-gitlawb"
chmod +x "npm/packages/$pkg/gl" "npm/packages/$pkg/git-remote-gitlawb"
done
- name: Set versions
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
node -e '
const fs = require("fs");
const v = process.env.VERSION;
for (const p of fs.readdirSync("npm/packages")) {
const f = `npm/packages/${p}/package.json`;
const pkg = JSON.parse(fs.readFileSync(f, "utf8"));
pkg.version = v;
if (pkg.optionalDependencies) {
for (const k of Object.keys(pkg.optionalDependencies)) {
pkg.optionalDependencies[k] = v;
}
}
fs.writeFileSync(f, JSON.stringify(pkg, null, 2) + "\n");
}
'
- name: Publish
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc
# Platform packages first, then the wrapper (so its optionalDependencies resolve).
# Skip versions already on the registry so a rerun after a partial publish
# is idempotent instead of erroring on the first existing package.
for pkg in gl-darwin-arm64 gl-darwin-x64 gl-linux-arm64 gl-linux-x64 gl; do
name="@gitlawb/$pkg"
if npm view "$name@$VERSION" version >/dev/null 2>&1; then
echo "==> $name@$VERSION already published, skipping"
continue
fi
echo "==> npm publish $name@$VERSION"
npm publish "npm/packages/$pkg" --provenance --access public
done
homebrew-bump:
name: Bump Homebrew tap
needs: [release-please, release-binaries]
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN downloads release assets; the tap push uses HOMEBREW_TAP_PAT
steps:
- name: Guard on secret
id: guard
env:
HOMEBREW_TAP_PAT: ${{ secrets.HOMEBREW_TAP_PAT }}
run: |
if [ -z "${HOMEBREW_TAP_PAT:-}" ]; then
echo "::warning::HOMEBREW_TAP_PAT is not set — skipping Homebrew bump."
echo "enabled=false" >> "$GITHUB_OUTPUT"
else
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Checkout tap repo
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: Gitlawb/homebrew-tap
token: ${{ secrets.HOMEBREW_TAP_PAT }}
path: tap
persist-credentials: false
- name: Regenerate formula
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
base="https://github.com/${GITHUB_REPOSITORY}/releases/download/${TAG}"
mkdir -p _sums
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern '*.tar.gz.sha256' --dir _sums --clobber
sha() { awk '{print $1}' "_sums/gitlawb-node-${VERSION}-$1.tar.gz.sha256"; }
SHA_MAC_ARM="$(sha aarch64-apple-darwin)"
SHA_MAC_X64="$(sha x86_64-apple-darwin)"
SHA_LNX_ARM="$(sha aarch64-unknown-linux-musl)"
SHA_LNX_X64="$(sha x86_64-unknown-linux-musl)"
mkdir -p tap/Formula
cat > tap/Formula/gl.rb <<EOF
class Gl < Formula
desc "Gitlawb CLI — decentralized git for AI agents and developers"
homepage "https://gitlawb.com"
version "${VERSION}"
license "MIT OR Apache-2.0"
on_macos do
on_arm do
url "${base}/gitlawb-node-${VERSION}-aarch64-apple-darwin.tar.gz"
sha256 "${SHA_MAC_ARM}"
end
on_intel do
url "${base}/gitlawb-node-${VERSION}-x86_64-apple-darwin.tar.gz"
sha256 "${SHA_MAC_X64}"
end
end
on_linux do
on_arm do
url "${base}/gitlawb-node-${VERSION}-aarch64-unknown-linux-musl.tar.gz"
sha256 "${SHA_LNX_ARM}"
end
on_intel do
url "${base}/gitlawb-node-${VERSION}-x86_64-unknown-linux-musl.tar.gz"
sha256 "${SHA_LNX_X64}"
end
end
def install
bin.install "gl"
bin.install "git-remote-gitlawb"
end
test do
assert_match version.to_s, shell_output("#{bin}/gl --version")
end
end
EOF
- name: Commit and push
if: ${{ steps.guard.outputs.enabled == 'true' }}
working-directory: tap
env:
HOMEBREW_TAP_PAT: ${{ secrets.HOMEBREW_TAP_PAT }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
# Stage first so a brand-new (untracked) formula is detected by the no-op check.
git add Formula/gl.rb
if git diff --cached --quiet; then
echo "Formula already up to date."
exit 0
fi
git commit -m "gl ${VERSION}"
# Credentials are not persisted in .git/config; supply the token only for the push.
git push "https://x-access-token:${HOMEBREW_TAP_PAT}@github.com/Gitlawb/homebrew-tap.git" HEAD:main
web-sync:
name: Sync web (install scripts + version)
needs: [release-please, release-binaries]
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN checks out node; the web PR uses WEB_SYNC_PAT
steps:
- name: Guard on secret
id: guard
env:
WEB_SYNC_PAT: ${{ secrets.WEB_SYNC_PAT }}
run: |
if [ -z "${WEB_SYNC_PAT:-}" ]; then
echo "::warning::WEB_SYNC_PAT is not set — skipping web sync."
echo "enabled=false" >> "$GITHUB_OUTPUT"
else
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Checkout node (release tag)
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
path: node
persist-credentials: false
- name: Checkout web
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: Gitlawb/web
token: ${{ secrets.WEB_SYNC_PAT }}
path: web
persist-credentials: false
# web-sync publishes install.ps1 + version.json, so the Windows ZIP must exist.
# Windows is best-effort for npm/Homebrew (continue-on-error), so verify the
# asset is present before advertising a release the site can't actually serve.
- name: Verify website release assets
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets \
--jq '.assets[].name' > /tmp/release-assets
for asset in \
"gitlawb-node-${VERSION}-x86_64-pc-windows-msvc.zip" \
"gitlawb-node-${VERSION}-x86_64-pc-windows-msvc.zip.sha256"; do
grep -Fxq "$asset" /tmp/release-assets || {
echo "::error::release asset missing: $asset (Windows build likely failed); not syncing web"
exit 1
}
done
- name: Sync and open PR
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.WEB_SYNC_PAT }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# Canonical install scripts (source of truth lives in the node repo).
cp node/install.sh web/public/install.sh
cp node/install.ps1 web/public/install.ps1
# Single version source the site can read.
printf '{\n "version": "%s",\n "tag": "%s"\n}\n' "$VERSION" "$TAG" > web/public/version.json
# One-time, idempotent drift fixes: drop orphaned binaries, fix stale link.
rm -rf web/public/bin
if [ -f web/public/skill.md ]; then
sed -i 's#github.com/gitlawb/releases#github.com/Gitlawb/node/releases#g' web/public/skill.md
fi
cd web
# Stage first so brand-new (untracked) files are caught by the no-op check.
git add -A
if git diff --cached --quiet; then
echo "web already up to date."
exit 0
fi
branch="release-sync/${TAG}"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "$branch"
git commit -m "chore: sync gitlawb ${TAG} (install scripts + version)"
# Credentials are not persisted in .git/config; supply the token only for the push.
git push --force "https://x-access-token:${GH_TOKEN}@github.com/Gitlawb/web.git" "HEAD:$branch"
gh pr create --repo Gitlawb/web --head "$branch" \
--title "Sync gitlawb ${TAG}" \
--body "Automated sync from Gitlawb/node ${TAG}: install.sh, install.ps1, public/version.json, and one-time drift fixes (removed orphaned public/bin, fixed skill.md releases link). Review and merge to deploy via Vercel." \
|| {
# Only swallow the "PR already exists" case; surface auth/API failures
# (a failed gh pr list must not be misread as "PR exists").
pr_count="$(gh pr list --repo Gitlawb/web --head "$branch" --state open --json number --jq 'length')" || exit 1
if [ "$pr_count" != "0" ]; then
echo "PR for $branch already exists; branch was updated."
else
exit 1
fi
}
+40
View File
@@ -0,0 +1,40 @@
name: Stale PRs
on:
schedule:
- cron: "30 1 * * *" # daily at 01:30 UTC
workflow_dispatch:
permissions:
contents: read
pull-requests: write
issues: write
jobs:
stale:
name: Flag and close ignored PRs
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9
with:
days-before-stale: 14
days-before-close: 7
# Act on PRs only; never on issues.
days-before-issue-stale: -1
days-before-issue-close: -1
# Only PRs the triage step flagged as missing a real description.
# needs-issue is advisory-only (linking an issue is a SHOULD, and many
# legitimate small PRs have none) so it does not drive auto-close.
only-pr-labels: "needs-description"
stale-pr-label: "stale"
exempt-pr-labels: "workflow-change,security,pinned"
remove-stale-when-updated: true
stale-pr-message: >
This PR has been inactive for 14 days and is still missing a description.
It will be closed in 7 days if there's no update. Push a change or leave a
comment to keep it open. No hard feelings, you can reopen anytime.
close-pr-message: >
Closing due to inactivity. Reopen whenever you're ready to update it. See
CONTRIBUTING.md for what helps us review quickly.
ascending: true
+5
View File
@@ -1,6 +1,7 @@
/target
**/*.rs.bk
Cargo.lock.bak
dist/
# Local node data
/data
@@ -21,3 +22,7 @@ keys/
# Logs
*.log
.openclaude-profile.json
# Per-operator Fly config (node1 prod config is pulled from Fly, not tracked)
infra/fly/node1.fly.toml
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.3.8"
".": "0.4.0"
}
+52
View File
@@ -0,0 +1,52 @@
# Changelog
## [0.4.0](https://github.com/Gitlawb/node/compare/v0.3.9...v0.4.0) (2026-06-30)
### Features
* agent profiles (display name, bio, avatar, social links) ([#23](https://github.com/Gitlawb/node/issues/23)) ([09a3397](https://github.com/Gitlawb/node/commit/09a339745eca40a2567d911e947e3fc7426fc621))
* **db:** versioned schema migrations with idempotent backfill ([#21](https://github.com/Gitlawb/node/issues/21)) ([927e4d0](https://github.com/Gitlawb/node/commit/927e4d0cfb4ea11dca6930780939afaa067797ba))
* encrypted replication for private subtrees (B1/B2/B3) for [#18](https://github.com/Gitlawb/node/issues/18) ([#36](https://github.com/Gitlawb/node/issues/36)) ([5ff7af8](https://github.com/Gitlawb/node/commit/5ff7af84fa21fab53a12dfa04a0e7fb7e7d672e6))
* **git-remote-gitlawb:** add --version and --help flags ([#30](https://github.com/Gitlawb/node/issues/30)) ([3a401eb](https://github.com/Gitlawb/node/commit/3a401eb0792f9c5f5a10de6c16861655ab3836e0))
* **gitlawb-attest:** External Attestation v1 for ref-update certs ([#20](https://github.com/Gitlawb/node/issues/20)) ([924bccd](https://github.com/Gitlawb/node/commit/924bccd8e53e9be2dc9d6d4e4f1376952d6462bb))
* **node:** blind recipient identities at rest and gate B1 by repo readability ([#40](https://github.com/Gitlawb/node/issues/40)) ([abdc775](https://github.com/Gitlawb/node/commit/abdc7757708d2bbb2bfda99bb65d50756142a42e))
* **node:** enforce per-route authorization across the REST and GraphQL surface ([#87](https://github.com/Gitlawb/node/issues/87)) ([2202b00](https://github.com/Gitlawb/node/commit/2202b0097fab6976ab366a2cdc385f1146a72f86))
* **node:** graceful shutdown + Prometheus metrics endpoint ([#22](https://github.com/Gitlawb/node/issues/22)) ([2ce4da9](https://github.com/Gitlawb/node/commit/2ce4da9cc5a4791e8ae5a1cd90270b67f60d9ec3))
* **node:** iCaptcha proof-of-intelligence gate on create_repo + register ([#108](https://github.com/Gitlawb/node/issues/108)) ([adc20f9](https://github.com/Gitlawb/node/commit/adc20f9effad7b42fab55002875d209c4ed79518))
* **node:** iCaptcha-aware repo propagation gate with quarantine ([#125](https://github.com/Gitlawb/node/issues/125)) ([8b9ceec](https://github.com/Gitlawb/node/commit/8b9ceec25ef338965d1db72d39a7f2adb5300cc9))
* **node:** owner-only push enforcement behind GITLAWB_ENFORCE_OWNER_PUSH ([#31](https://github.com/Gitlawb/node/issues/31)) ([#68](https://github.com/Gitlawb/node/issues/68)) ([0a15e76](https://github.com/Gitlawb/node/commit/0a15e763d2ab46737a3831715facbe51045b33ba))
* **node:** peer partial-mirrors for repos with private subtrees ([#35](https://github.com/Gitlawb/node/issues/35)) ([e365a57](https://github.com/Gitlawb/node/commit/e365a57b51c167abf06e87bfeb0565c80ed1b849))
* **node:** pin the per-push object delta instead of re-enumerating the whole repo ([#90](https://github.com/Gitlawb/node/issues/90)) ([1af4fdf](https://github.com/Gitlawb/node/commit/1af4fdf485c7084cd160551b757b1ad1eed65cc6))
* **node:** replication enforcement (Phase 2) for [#18](https://github.com/Gitlawb/node/issues/18) ([#34](https://github.com/Gitlawb/node/issues/34)) ([8680d0f](https://github.com/Gitlawb/node/commit/8680d0f9d6600bba1a52d15624f8a2802a169511))
* **node:** signature-gated agent self-deregister ([#29](https://github.com/Gitlawb/node/issues/29)) ([#63](https://github.com/Gitlawb/node/issues/63)) ([ff492b4](https://github.com/Gitlawb/node/commit/ff492b452126f5568dac4286a7249f0cadb8b380))
* **node:** subtree content withholding (Phase 3) for [#18](https://github.com/Gitlawb/node/issues/18) ([#28](https://github.com/Gitlawb/node/issues/28)) ([61b3830](https://github.com/Gitlawb/node/commit/61b383019fd895a1a6adfad934ad6c626e0f095e))
* path-scoped repository visibility (Phase 1) for [#18](https://github.com/Gitlawb/node/issues/18) ([#25](https://github.com/Gitlawb/node/issues/25)) ([6abaf1d](https://github.com/Gitlawb/node/commit/6abaf1d7ed8fc55c6547568ae7247131311bde98))
* per-DID rate limiting on creation endpoints (10/hour) ([#13](https://github.com/Gitlawb/node/issues/13)) ([b12c6bc](https://github.com/Gitlawb/node/commit/b12c6bc3283c2647224a62fb520a3cb7acf4a747))
* **sync:** auto-register as replica with origin after successful mirror ([#56](https://github.com/Gitlawb/node/issues/56)) ([c03c9af](https://github.com/Gitlawb/node/commit/c03c9af8fadafe262a2bf3cf25e19edf7160d376))
### Bug Fixes
* **api:** blob endpoint returns 400/404 instead of 500 on bad paths ([#37](https://github.com/Gitlawb/node/issues/37)) ([b61a1bd](https://github.com/Gitlawb/node/commit/b61a1bd46ffe78b39c0967e97b0ad349eba0b046))
* **core:** route seed access through the zeroizing wrapper ([#41](https://github.com/Gitlawb/node/issues/41)) ([#64](https://github.com/Gitlawb/node/issues/64)) ([c9f43b0](https://github.com/Gitlawb/node/commit/c9f43b010576edb3c92a3be3d935cad232250344))
* **core:** zeroize the derived X25519 secret ([#65](https://github.com/Gitlawb/node/issues/65)) ([#91](https://github.com/Gitlawb/node/issues/91)) ([2f6611a](https://github.com/Gitlawb/node/commit/2f6611a30fbb66cef9991cf4c6e507548acc5038))
* **gl:** paginate gl-clone Arweave recovery and make /encrypted-blobs parsing schema-strict ([#49](https://github.com/Gitlawb/node/issues/49)) ([#70](https://github.com/Gitlawb/node/issues/70)) ([2153b0b](https://github.com/Gitlawb/node/commit/2153b0b7a67a48c29a33cd70b80cfbb69760805d))
* **infra:** drop fly idle_timeout 600 -&gt; 120 ([#38](https://github.com/Gitlawb/node/issues/38)) ([a2217bf](https://github.com/Gitlawb/node/commit/a2217bff27ff3d6cae02bd8a12f8a0af7ce2b0a1))
* **node:** anchor the real old_sha and issue a per-ref certificate ([#72](https://github.com/Gitlawb/node/issues/72)) ([6809201](https://github.com/Gitlawb/node/commit/6809201daa223d6f833be6e95876a7b4e1f2b0b5))
* **node:** close under-withholding via full ref scope and full-history classification ([#42](https://github.com/Gitlawb/node/issues/42)) ([#84](https://github.com/Gitlawb/node/issues/84)) ([3e1e904](https://github.com/Gitlawb/node/commit/3e1e9045e4e3aa5ea0aee69767ceb01637920d2a))
* **node:** dedupe mirror and canonical repo rows on list surfaces ([#6](https://github.com/Gitlawb/node/issues/6)) ([#73](https://github.com/Gitlawb/node/issues/73)) ([3e8e333](https://github.com/Gitlawb/node/commit/3e8e333aa03d7a2fe455d5d83f23089d58feb8c9))
* **node:** enforce path-scoped visibility on the REST read API ([#52](https://github.com/Gitlawb/node/issues/52)) ([e37ea7f](https://github.com/Gitlawb/node/commit/e37ea7fec6d5a3171b526c84f884670bbbd258fb))
* **node:** fail closed when a recipient DID can't be resolved ([#47](https://github.com/Gitlawb/node/issues/47)) ([#67](https://github.com/Gitlawb/node/issues/67)) ([abc9ad0](https://github.com/Gitlawb/node/commit/abc9ad03acb48e608234650a25049622673fa53a))
* **node:** gate fork_repo on per-caller path-scoped visibility ([#98](https://github.com/Gitlawb/node/issues/98)) ([#109](https://github.com/Gitlawb/node/issues/109)) ([6ae316c](https://github.com/Gitlawb/node/commit/6ae316cc88521747cbacb2a612b9433897d2e490))
* **node:** gate GET /ipfs/{cid} on per-caller path-scoped visibility ([#110](https://github.com/Gitlawb/node/issues/110)) ([#128](https://github.com/Gitlawb/node/issues/128)) ([174f25a](https://github.com/Gitlawb/node/commit/174f25a206380b26796b8782e1bd860b0a409fc9))
* **node:** gate repo-listing and stats surfaces on visibility ([#97](https://github.com/Gitlawb/node/issues/97), [#99](https://github.com/Gitlawb/node/issues/99), [#101](https://github.com/Gitlawb/node/issues/101), [#104](https://github.com/Gitlawb/node/issues/104)) ([#111](https://github.com/Gitlawb/node/issues/111)) ([828dd27](https://github.com/Gitlawb/node/commit/828dd279a286f58bbb3c73627b2a1e23778b25cf))
* **node:** make Tigris repo hydration resilient to corrupt archives & failed writes ([#54](https://github.com/Gitlawb/node/issues/54)) ([7a99d0f](https://github.com/Gitlawb/node/commit/7a99d0f27cfa869e9f8a803f16cff30191745a51))
* **node:** preserve promisor mirror mode on unknown withheld-paths lookup ([#48](https://github.com/Gitlawb/node/issues/48)) ([#69](https://github.com/Gitlawb/node/issues/69)) ([96fcfdb](https://github.com/Gitlawb/node/commit/96fcfdb1ca6d4cd27226670068702c9f177e283a))
* **node:** reap leaked git child processes ([#53](https://github.com/Gitlawb/node/issues/53)) ([#61](https://github.com/Gitlawb/node/issues/61)) ([803d83e](https://github.com/Gitlawb/node/commit/803d83efdbcfa9d98affac1c5aaf8aacc511ae64))
* **node:** reject malformed path globs with non-trailing or empty-segment wildcards ([#74](https://github.com/Gitlawb/node/issues/74)) ([#75](https://github.com/Gitlawb/node/issues/75)) ([3d880cd](https://github.com/Gitlawb/node/commit/3d880cd56fed10e4c5ae787b52ce411384950f5e))
* **node:** skip withheld-walk when no path-scoped rule can withhold ([#60](https://github.com/Gitlawb/node/issues/60)) ([338ff83](https://github.com/Gitlawb/node/commit/338ff83584f2ab960be7e42ecec64191f5aaeb95))
* **release:** give crates explicit versions for release-please ([#129](https://github.com/Gitlawb/node/issues/129)) ([788a868](https://github.com/Gitlawb/node/commit/788a8686c1deafe03573703d72eb927cde81f54d))
* **release:** use simple release-type + generic Cargo.toml updaters ([#130](https://github.com/Gitlawb/node/issues/130)) ([12bfb1b](https://github.com/Gitlawb/node/commit/12bfb1b3c86b38f0b22d488cd6e85dfd96e0c37f))
* **security:** gate webhook creation through the public-host validator ([#81](https://github.com/Gitlawb/node/issues/81)) ([#92](https://github.com/Gitlawb/node/issues/92)) ([f28fa02](https://github.com/Gitlawb/node/commit/f28fa02236bd65c6a4fb690131ab14640cc53a12))
* **security:** reject non-public peer URLs + prune poisoned peers ([#78](https://github.com/Gitlawb/node/issues/78)) ([a8cc33a](https://github.com/Gitlawb/node/commit/a8cc33a185f2649d3fe100ec271ee5739a55eba7))
+20 -1
View File
@@ -55,10 +55,29 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
## What gets merged, what gets closed
We welcome contributions from humans and agents alike. To keep review sustainable, PRs are
expected to clear a basic quality bar:
- **Link an issue.** Bug fixes and features should reference an issue (`Closes #123`). For
protocol-level changes (identity, signatures, UCAN, ref certs, wire formats), open the
issue *before* writing code.
- **One change per PR.** Unrelated churn slows review and gets sent back.
- **Tests and a green pipeline.** New behavior needs tests; `cargo fmt`, `cargo clippy`,
and the full CI suite must pass.
- **A real description.** Say what changes and why. "Update code" is not a description.
A triage bot labels PRs that are missing these and leaves a short note. Nothing is closed
automatically while you're engaging. A flagged PR still missing a description after 14 days of
inactivity gets a stale warning, and is closed 7 days later if still untouched. A missing issue
link alone never triggers closure; that label stays advisory. Closed PRs can be reopened at any
time once updated.
## Development environment
**Requirements:**
- Rust stable (≥ 1.85) — install via [rustup](https://rustup.rs)
- Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs)
- PostgreSQL — required for the node. Use the bundled `docker-compose.yml` for local dev.
- Docker (optional, for full-stack local testing)
Generated
+212 -585
View File
File diff suppressed because it is too large. Load diff
+15 -2
View File
@@ -5,11 +5,19 @@ members = [
"crates/gitlawb-node",
"crates/gl",
"crates/git-remote-gitlawb",
"crates/gitlawb-attest",
]
[workspace.package]
version = "0.3.8"
# Per-crate versions live in each crate's [package] so release-please can manage
# them (it cannot rewrite an inherited `version.workspace = true`). They are kept
# in lockstep by release-please's generic updater: each crate's `version` line
# carries an `# x-release-please-version` annotation and is listed under
# `extra-files` in release-please-config.json. The release type is `simple`
# (not `rust`) because the rust strategy unconditionally tries to rewrite this
# virtual workspace root as a package manifest and errors out.
edition = "2021"
rust-version = "1.91"
license = "MIT OR Apache-2.0"
authors = ["gitlawb contributors"]
repository = "https://github.com/gitlawb/node"
@@ -20,12 +28,13 @@ tokio = { version = "1", features = ["full"] }
# serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
serde_jcs = "0.2"
# errors
thiserror = "2"
anyhow = "1"
# crypto
ed25519-dalek = { version = "2", features = ["rand_core", "pkcs8", "pem"] }
rand = "0.8"
rand = "0.8.6"
sha2 = "0.10"
# encoding
hex = "0.4.3"
@@ -45,3 +54,7 @@ uuid = { version = "1", features = ["v4"] }
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# HMAC
hmac = "0.12"
[profile.release]
lto = "thin"
strip = "symbols"
+11 -2
View File
@@ -28,6 +28,7 @@ RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src crate
# cargo can keep the dummy `fn main() {}` binaries from the cache layer above
# and the runtime container exits immediately with code 0.
COPY crates/ crates/
COPY bootstrap-peers.json ./
RUN find crates -name "*.rs" -exec touch {} + && \
rm -f target/release/gitlawb-node target/release/gl target/release/git-remote-gitlawb && \
rm -rf target/release/.fingerprint/gitlawb-node-* \
@@ -43,6 +44,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
tini \
&& rm -rf /var/lib/apt/lists/*
# Non-root user for runtime
@@ -64,11 +66,18 @@ ENV GITLAWB_REPOS_DIR=/data/repos \
GITLAWB_PORT=7545 \
GITLAWB_P2P_PORT=7546
EXPOSE 7545 7546
EXPOSE 7545 7546/udp
VOLUME ["/data"]
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
CMD curl -fsS http://127.0.0.1:${GITLAWB_PORT}/health || exit 1
ENTRYPOINT ["gitlawb-node"]
# Run under tini so the node is never PID 1. As PID 1 a process must reap
# reparented orphans itself; the node does not, so git's reparented grandchildren
# (e.g. pack-objects orphaned when a served upload-pack dies on a client
# disconnect) would accumulate as zombies until fork() fails with EAGAIN (#53).
# tini reaps them and forwards SIGTERM/SIGINT to the node, which runs its own
# graceful shutdown. (No `-g`: the node manages its own children, so group-wide
# signalling would only disturb in-flight git helpers during shutdown.)
ENTRYPOINT ["tini", "--", "gitlawb-node"]
+397 -56
View File
@@ -1,49 +1,281 @@
# gitlawb node
# Gitlawb Node
**Decentralized git for AI agents and developers.** Run your own node, host repos under your own DID, and connect to the gitlawb network.
**Decentralized git infrastructure for developers, AI agents, and app delivery.**
This repository contains the open-source node software:
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
- `gitlawb-node` — Axum HTTP server, git smart-HTTP, libp2p gossip, optional S3-compatible storage
- `gl` — CLI for identity, repos, PRs, MCP server
- `git-remote-gitlawb` — `git remote helper` for `gitlawb://` URLs
- `gitlawb-core` — shared crypto primitives (DID, CID, HTTP signatures, UCAN)
Gitlawb is not trying to be only “another git host.” The long-term direction is:
```txt
Decentralized GitHub
+ signed agent-native workflows
+ resilient repo replication
+ CDN-style app/code delivery
```
The mission is simple: once code is pushed to the network, it should not disappear because one server went down.
---
## Quickstart
## What is in this repository?
The fastest way to run a node is via Docker. You'll get a node + Postgres locally with one command.
This is a Rust workspace with four crates:
| Crate | Purpose |
|---|---|
| `gitlawb-node` | The node daemon: Axum HTTP server, git smart-HTTP, Postgres metadata, libp2p gossip, optional S3/Tigris/IPFS/Arweave/Base PoS hooks. |
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `gitlawb-core` | Shared primitives: Ed25519 identities, `did:key`, CIDs, RFC 9421 HTTP signatures, certificates, and UCAN tokens. |
---
## Why Gitlawb Nodes?
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
---
## Current status
Gitlawb Node is live early infrastructure. It is useful today, but some security and reliability features are intentionally staged for compatibility with existing nodes.
Good today:
- Local or Docker node startup.
- Postgres-backed repo metadata.
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- DID identities.
- `gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
- Optional Tigris/S3 storage.
- Optional IPFS/Pinata and Arweave/Irys hooks.
- Optional Base node-operator staking/heartbeat hooks.
Known limitations:
- Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
- Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
See:
- [`SECURITY.md`](SECURITY.md)
- [`docs/OSS-READINESS-AUDIT.md`](docs/OSS-READINESS-AUDIT.md)
- [`docs/MAINTAINER-ROADMAP.md`](docs/MAINTAINER-ROADMAP.md)
---
## Quickstart: run a local node
The fastest path is Docker Compose. It starts a node and Postgres.
```bash
git clone https://github.com/gitlawb/node.git
git clone https://github.com/Gitlawb/node.git
cd node
cp .env.example .env
docker compose up -d
```
Your node is now serving:
Your local node will serve:
- HTTP + git smart-HTTP on `:7545`
- libp2p gossip on `:7546`
| Service | Default |
|---|---|
| HTTP API + git smart-HTTP | `http://localhost:7545` |
| libp2p QUIC/UDP | `7546` |
| Postgres | compose-managed |
Verify:
```bash
curl http://localhost:7545/health
curl http://localhost:7545/api/v1/stats
```
Expected health response:
```json
{ "status": "ok" }
```
Stop it:
```bash
docker compose down
```
---
## How it works
## Install the CLI
- **Identity** — every actor (human or agent) is an Ed25519 keypair → `did:key:z6Mk...`. No accounts, no passwords.
- **Auth** — every write is signed via HTTP Signatures (RFC 9421).
- **Delegation** — UCAN capability tokens.
- **Storage** — local disk + optional S3-compatible bucket (Tigris, MinIO, AWS S3).
- **Networking** — libp2p Gossipsub for ref-update events, Kademlia DHT for node discovery.
```bash
# npm (macOS / Linux)
npm install -g @gitlawb/gl
The node is **stateless beyond Postgres + the repos directory.** You can scale horizontally by pointing multiple nodes at a shared S3 bucket and database.
# Homebrew (macOS / Linux)
brew install gitlawb/tap/gl
# curl (macOS / Linux)
curl -fsSL https://gitlawb.com/install.sh | sh
# PowerShell (Windows)
irm https://gitlawb.com/install.ps1 | iex
```
Or build from source:
```bash
cargo build --release -p gl -p git-remote-gitlawb -p gitlawb-node
```
Put these binaries on your `PATH`:
```txt
target/release/gl
target/release/git-remote-gitlawb
target/release/gitlawb-node
```
Check your setup:
```bash
gl doctor
```
---
## First repo flow
Create an identity:
```bash
gl identity new
gl identity show
```
Register against your local node:
```bash
gl register --node http://localhost:7545
```
Create a repo:
```bash
gl repo create my-repo --description "My first Gitlawb repo" --node http://localhost:7545
```
Use the git remote helper:
```bash
export GITLAWB_NODE=http://localhost:7545
git clone gitlawb://did:key:z6Mk.../my-repo
```
For public-network use, make sure `GITLAWB_NODE` points to the node you want. The helper defaults to localhost for local development.
---
## Architecture
```txt
┌──────────────────────────┐
│ gl CLI / git / AI agents │
└────────────┬─────────────┘
│ signed HTTP writes / git smart-HTTP
↓
┌──────────────────────────┐
│ gitlawb-node │
│ Axum API + git routes │
└────────────┬─────────────┘
│
┌────────┴────────┐
↓ ↓
Postgres Bare git repos
metadata local disk / optional S3
│ │
└────────┬────────┘
↓
libp2p peers
gossip + discovery + sync
↓
optional IPFS / Arweave / Base PoS
```
### Core concepts
| Concept | Meaning |
|---|---|
| DID | A user, agent, or node identity derived from an Ed25519 public key. |
| HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. |
| Ref certificate | Signed record of a ref update. Useful for audit and replication. |
| UCAN | Delegation token for future capability-based workflows. |
| Peer announce | Node-to-node HTTP announcement of DID + public URL. |
| Gossipsub | libp2p topic for ref-update events. |
| Smart HTTP | Standard git protocol over HTTP for clone/fetch/push. |
---
## API surface
The node exposes both git smart-HTTP routes and JSON APIs.
Common public read routes:
```txt
GET /health
GET /
GET /api/v1/stats
GET /api/v1/contracts
GET /api/v1/repos
GET /api/v1/repos/{owner}/{repo}
GET /api/v1/repos/{owner}/{repo}/tree
GET /api/v1/repos/{owner}/{repo}/blob/{path}
GET /api/v1/repos/{owner}/{repo}/issues
GET /api/v1/repos/{owner}/{repo}/pulls
GET /api/v1/peers
GET /{owner}/{repo}/info/refs
POST /{owner}/{repo}/git-upload-pack
```
Signed write routes include:
```txt
POST /api/v1/repos
POST /api/register
POST /api/v1/repos/{owner}/{repo}/fork
POST /api/v1/repos/{owner}/{repo}/issues
POST /api/v1/repos/{owner}/{repo}/pulls
POST /api/v1/repos/{owner}/{repo}/pulls/{number}/merge
POST /api/v1/repos/{owner}/{repo}/hooks
POST /api/v1/bounties/{id}/...
POST /{owner}/{repo}/git-receive-pack
```
Peer write routes support staged rollout:
```txt
POST /api/v1/peers/announce
POST /api/v1/sync/notify
POST /api/v1/sync/trigger
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
```
---
@@ -51,76 +283,185 @@ The node is **stateless beyond Postgres + the repos directory.** You can scale h
All configuration is via environment variables. See [`.env.example`](.env.example) for the full reference.
The minimum required:
Minimum required for a persistent node:
- `DATABASE_URL` — Postgres connection string
```env
DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
```
Everything else has sensible defaults. Optional features:
Important node settings:
- `GITLAWB_TIGRIS_BUCKET` (+ AWS credentials) — S3-compatible shared storage
- `GITLAWB_PINATA_JWT` — IPFS warm-storage pinning
- `GITLAWB_IRYS_URL` — Arweave permanent anchoring
- `GITLAWB_BOOTSTRAP_PEERS` — comma-separated peer URLs to announce to on startup
- `GITLAWB_P2P_BOOTSTRAP` — comma-separated libp2p multiaddrs
| Variable | Purpose |
|---|---|
| `GITLAWB_HOST` / `GITLAWB_PORT` | HTTP bind address and port. |
| `GITLAWB_REPOS_DIR` | Local bare repo storage directory. |
| `GITLAWB_PUBLIC_URL` | Public HTTP URL announced to peers. |
| `GITLAWB_P2P_PORT` | libp2p QUIC/UDP port. Use `0` to disable. |
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
### On-chain Proof-of-Stake (advanced, optional)
The node supports an optional on-chain PoS layer where operators register their DID, stake $GITLAWB on Base L2, and receive a share of protocol fees. Smart contracts are in a separate repository at [github.com/gitlawb/contracts](https://github.com/gitlawb/contracts).
PoS is **disabled by default** and currently paused pending external audit. To enable, set `GITLAWB_CONTRACT_NODE_STAKING` and `GITLAWB_OPERATOR_PRIVATE_KEY`. See [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md) for details.
Production note: change the default Postgres password before exposing a node publicly.
---
## Build from source
## Optional node staking
Requires Rust 1.85+ and a Postgres instance.
Gitlawb Node includes optional Base L2 node-operator hooks. Operators can register a node DID, stake `$GITLAWB`, and post heartbeats.
PoS is disabled unless these are configured:
```env
GITLAWB_CONTRACT_NODE_STAKING=0x...
GITLAWB_OPERATOR_PRIVATE_KEY=0x...
GITLAWB_CHAIN_RPC_URL=https://mainnet.base.org
```
Recommended for operators:
```env
GITLAWB_OPERATOR_STRICT_MODE=true
GITLAWB_HEARTBEAT_INTERVAL_HOURS=20
```
Read:
- [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md)
- [`docs/ECONOMICS.md`](docs/ECONOMICS.md)
Use a dedicated low-balance operator wallet. Do not use a treasury wallet as the heartbeat key.
---
## Building from source
Requires Rust 1.91+.
```bash
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb
```
Binaries are placed in `target/release/`. The `gl` and `git-remote-gitlawb` binaries should be on your `$PATH` for client use.
---
## CLI usage
Run tests:
```bash
gl identity new
gl register --node http://localhost:7545
gl repo create my-repo --description "..."
git clone gitlawb://did:key:z6Mk.../my-repo
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
```
Full CLI reference: `gl --help`.
Run the node from source:
```bash
DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb \
cargo run -p gitlawb-node --release
```
---
## Architecture
## macOS menu bar app
A native Swift/AppKit menu bar app is included for managing a local Docker Compose stack without living in the terminal.
Requirements:
- macOS 26+
- Xcode Command Line Tools
- Docker Desktop, OrbStack, or Colima
Build:
```bash
./scripts/build-macos-app.sh
```
crates/
├── gitlawb-core/ crypto primitives (DID, CID, HTTP sigs, UCAN)
├── gitlawb-node/ node daemon (Axum + git smart-HTTP)
├── gl/ CLI
└── git-remote-gitlawb/ git remote helper
Output:
```txt
dist/Gitlawb Node.app
dist/Gitlawb Node.dmg
```
Features:
- Start/stop local node stack.
- Status indicator.
- Settings GUI.
- Auto-start on login.
- Docker runtime detection.
Unsigned local build:
```bash
xattr -cr "dist/Gitlawb Node.app"
```
---
## Roadmap
The current maintainer focus is live-network stability first.
Short-term priorities:
1. Keep CI green: fmt, clippy, tests, release build.
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Implement repo write authorization: owner checks, protected branches, and UCAN capability checks.
6. Implement private-read enforcement or remove private repo affordances until it exists.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
Product direction:
1. Reliable repo replication.
2. Health-aware peer syncing.
3. CDN-style clone/fetch routing to healthy replicas.
4. App asset/build delivery from nodes.
5. Operator dashboard and desktop UX.
Read the maintainer roadmap:
```txt
docs/MAINTAINER-ROADMAP.md
```
---
## Contributing
See [`CONTRIBUTING.md`](CONTRIBUTING.md). Security issues: see [`SECURITY.md`](SECURITY.md).
Start here:
- [`CONTRIBUTING.md`](CONTRIBUTING.md)
- [`docs/MAINTAINER-ROADMAP.md`](docs/MAINTAINER-ROADMAP.md)
- [`docs/OSS-READINESS-AUDIT.md`](docs/OSS-READINESS-AUDIT.md)
- [`SECURITY.md`](SECURITY.md)
Good first contribution areas:
- docs and install polish
- Docker smoke tests
- CLI error messages
- `gl doctor` checks
- operator dashboard UX
- test coverage for peer sync and signed writes
Security issues should follow [`SECURITY.md`](SECURITY.md).
---
## License
Licensed under either of
Licensed under either of:
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project shall be dual licensed as above, without any additional terms or conditions.
+22 -11
View File
@@ -16,7 +16,7 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
## Current Security Architecture
### What is production-ready (v0.1)
### Current live security controls
**Ed25519 identity and HTTP Signatures**
- Every write operation is signed with RFC 9421 HTTP Signatures
@@ -42,35 +42,46 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
## Dependency Vulnerability Status
| Crate | Advisory | Severity | Status |
|-------|----------|----------|--------|
| `rsa 0.9.x` | RUSTSEC-2023-0071 Marvin Attack | Medium | Unexploitable — bundled via `sqlx-mysql` internals; gitlawb uses PostgreSQL only, MySQL auth code is never executed. No upstream fix available. |
| `lru 0.12.x` | RUSTSEC-2026-0002 unsound IterMut | Warning | In `libp2p-swarm`; the specific unsafe code path is internal to libp2p and not reachable through gitlawb's usage. No upstream fix yet. |
| Area | Status |
|------|--------|
| Dependabot alerts | Current open Rust alerts were addressed by updating vulnerable dependencies, removing libp2p mDNS, and moving P2P transport from TCP/Yamux to QUIC/UDP. |
---
## Known Limitations (Planned for v0.2)
These are **documented, accepted limitations** for the current testnet release. They will be addressed before mainnet.
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
### UCAN chain validation
- The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Nodes can set `GITLAWB_PUBLIC_READ=false` to restrict access. The v0.1 trust score system provides soft rate limiting.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### UCAN revocation
- Issued UCAN tokens cannot be revoked before expiry.
- **Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Nodes will reject the old DID on sight if manually blocklisted.
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth via the `git-remote-gitlawb` remote helper. Direct HTTP push without the helper is currently unauthenticated on v0.1 nodes.
- **Impact:** A node operator accepting direct HTTP git pushes (not via `gitlawb://`) cannot verify the pusher's identity.
- **Mitigation:** Use `gitlawb://` remote URLs. Do not expose `POST /*/git-receive-pack` to untrusted networks.
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
- **Impact:** Direct HTTP pushes without RFC 9421 headers are rejected; users need `gitlawb://` remotes or equivalent signed clients.
- **Mitigation:** Use `gitlawb://` remote URLs and keep `git-remote-gitlawb` on the user's `PATH`.
- **Fix target:** v0.2
### Private repository reads
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
- **Impact:** Do not store private repositories or secrets on public nodes.
- **Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
- **Fix target:** v0.2
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
- **Impact:** Unsigned peer writes are still accepted by default so existing live nodes can keep communicating during rolling upgrades.
- **Mitigation:** After all active peers run signed-node builds, operators can set `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true`.
- **Fix target:** staged rollout
---
## Supported Versions
+47
View File
@@ -0,0 +1,47 @@
{
"$comment": "Canonical seed list for the Gitlawb network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1,
"updated": "2026-06-30",
"peers": [
{
"name": "gitlawb",
"operator": "Gitlawb",
"did": "did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr",
"http_url": "https://node.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-04-29"
},
{
"name": "gitlawb-node2",
"operator": "Gitlawb",
"did": null,
"http_url": "https://node2.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-06-30"
},
{
"name": "gitlawb-node3",
"operator": "Gitlawb",
"did": null,
"http_url": "https://node3.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-06-30"
},
{
"name": "gitlawb-manila",
"operator": "Gitlawb",
"did": null,
"http_url": "https://manila.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-06-30"
},
{
"name": "rapybus-gitlawb",
"operator": "RapyBus S.A.C.",
"did": "did:key:z6MktERLyhFpjUyN7dnkf7CYY5RXKJ6BTZiKohoi37YVBS7R",
"http_url": "https://rapybus-gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-05-16"
}
]
}
+2 -1
View File
@@ -1,8 +1,9 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version.workspace = true
version = "0.4.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
+183 -24
View File
@@ -16,23 +16,39 @@
//! connect git-upload-pack → GET /info/refs | POST /git-upload-pack
//! connect git-receive-pack → GET /info/refs | POST /git-receive-pack (+ auth header)
use std::io::{self, BufRead, Read, Write};
use anyhow::{bail, Context, Result};
use gitlawb_core::identity::Keypair;
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use std::io::{self, BufRead, Read, Write};
fn main() -> Result<()> {
let args: Vec<String> = std::env::args().collect();
// Handle informational flags before anything else. Git always invokes a
// remote helper as `git-remote-gitlawb <remote-name> <url>`, so these flags
// only ever appear on direct user or release-smoke-test invocations. Print
// to stdout and exit before tracing init so the output stays clean.
match classify_args(&args) {
Invocation::Version => {
println!("{}", version_line());
return Ok(());
}
Invocation::Help => {
print!("{}", help_text());
return Ok(());
}
Invocation::Helper => {}
}
// All logging goes to stderr so it doesn't corrupt the git protocol on stdout
tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.with_env_filter(
std::env::var("GITLAWB_LOG").unwrap_or_else(|_| "warn".to_string()),
)
.with_env_filter(std::env::var("GITLAWB_LOG").unwrap_or_else(|_| "warn".to_string()))
.init();
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("usage: git-remote-gitlawb <remote-name> <url>");
eprintln!("try 'git-remote-gitlawb --help' for more information");
std::process::exit(1);
}
@@ -42,8 +58,8 @@ fn main() -> Result<()> {
let (_, short_owner, repo_name) = parse_gitlawb_url(url)?;
// v0.1: default to localhost. Override with GITLAWB_NODE env var.
let node_base = std::env::var("GITLAWB_NODE")
.unwrap_or_else(|_| "http://127.0.0.1:7545".to_string());
let node_base =
std::env::var("GITLAWB_NODE").unwrap_or_else(|_| "http://127.0.0.1:7545".to_string());
let repo_base = format!("{}/{}/{}", node_base, short_owner, repo_name);
tracing::debug!("repo_base: {repo_base}");
@@ -53,6 +69,62 @@ fn main() -> Result<()> {
run_helper(&repo_base, keypair.as_ref())
}
// ── CLI argument handling ──────────────────────────────────────────────────────
/// How the binary was invoked, derived from its CLI arguments.
#[derive(Debug, PartialEq, Eq)]
enum Invocation {
/// `--version` / `-V`: print the version line and exit.
Version,
/// `--help` / `-h`: print usage and exit.
Help,
/// Normal git remote-helper invocation: `<remote-name> <url>`.
Helper,
}
/// Classify the process arguments.
///
/// Git always calls a remote helper as `git-remote-gitlawb <remote-name> <url>`,
/// so the informational flags are only recognized as the first argument; this
/// keeps the remote-helper protocol path untouched for git's own invocations.
fn classify_args(args: &[String]) -> Invocation {
match args.get(1).map(String::as_str) {
Some("--version") | Some("-V") => Invocation::Version,
Some("--help") | Some("-h") => Invocation::Help,
_ => Invocation::Helper,
}
}
/// The version line, matching the `<bin> <version>` format that the clap-based
/// `gl` and `gitlawb-node` binaries emit, so release smoke tests can treat all
/// three binaries uniformly.
fn version_line() -> String {
format!("{} {}", env!("CARGO_PKG_NAME"), env!("CARGO_PKG_VERSION"))
}
/// Usage text for `--help`.
fn help_text() -> String {
format!(
"{}\n\
\n\
Git remote helper for gitlawb:// URLs. Git invokes this automatically\n\
when it encounters a gitlawb:// remote; you normally do not run it directly.\n\
\n\
USAGE:\n\
\x20 git clone gitlawb://did:key:z6Mk.../<repo>\n\
\n\
ENVIRONMENT:\n\
\x20 GITLAWB_NODE Node base URL (default: http://127.0.0.1:7545)\n\
\x20 GITLAWB_KEY Identity PEM path for signed pushes (default: ~/.gitlawb/identity.pem)\n\
\x20 GITLAWB_LOG Log filter (default: warn)\n\
\n\
FLAGS:\n\
\x20 -V, --version Print version and exit\n\
\x20 -h, --help Print this help and exit\n",
version_line()
)
}
// ── Remote helper protocol loop ───────────────────────────────────────────────
fn run_helper(repo_base: &str, keypair: Option<&Keypair>) -> Result<()> {
@@ -62,7 +134,9 @@ fn run_helper(repo_base: &str, keypair: Option<&Keypair>) -> Result<()> {
loop {
let mut line = String::new();
let n = stdin_buf.read_line(&mut line).context("reading command from git")?;
let n = stdin_buf
.read_line(&mut line)
.context("reading command from git")?;
if n == 0 {
break; // EOF
}
@@ -149,7 +223,10 @@ fn handle_connect(
// But git's `connect` protocol expects raw git-upload-pack output (no HTTP wrapper).
// Strip the service-line pkt-line + flush before forwarding.
let advertisement = strip_service_announcement(&refs_bytes);
tracing::debug!("ref advertisement: {} bytes (stripped)", advertisement.len());
tracing::debug!(
"ref advertisement: {} bytes (stripped)",
advertisement.len()
);
let mut stdout = io::stdout();
stdout.write_all(advertisement)?;
@@ -172,7 +249,9 @@ fn handle_connect(
read_upload_pack_request(stdin).context("reading upload-pack request")?
} else {
let mut buf = Vec::new();
stdin.read_to_end(&mut buf).context("reading receive-pack request")?;
stdin
.read_to_end(&mut buf)
.context("reading receive-pack request")?;
buf
};
@@ -192,12 +271,8 @@ fn handle_connect(
let mut req = client
.post(&post_url)
.header(
"Content-Type",
format!("application/x-{}-request", service),
)
.header("User-Agent", "git/2.0 git-remote-gitlawb/0.1.0")
.body(request_body.clone());
.header("Content-Type", format!("application/x-{}-request", service))
.header("User-Agent", "git/2.0 git-remote-gitlawb/0.1.0");
// Add RFC 9421 HTTP Signature auth on push operations
if service == "git-receive-pack" {
@@ -215,7 +290,10 @@ fn handle_connect(
}
}
// Attach the body after signing so the pack bytes are moved, not cloned —
// packs can be large and the clone doubled peak memory on push.
let pack_resp = req
.body(request_body)
.send()
.with_context(|| format!("POST {post_url}"))?;
@@ -296,7 +374,9 @@ fn read_upload_pack_request(stdin: &mut io::BufReader<io::Stdin>) -> Result<Vec<
let data_len = pkt_len - 4;
let mut data = vec![0u8; data_len];
stdin.read_exact(&mut data).context("reading pkt-line data")?;
stdin
.read_exact(&mut data)
.context("reading pkt-line data")?;
buf.extend_from_slice(&data);
// "done\n" signals the end of the want/have negotiation
@@ -380,8 +460,8 @@ fn load_keypair() -> Option<Keypair> {
}
fn resolve_key_path() -> std::path::PathBuf {
let path_str = std::env::var("GITLAWB_KEY")
.unwrap_or_else(|_| "~/.gitlawb/identity.pem".to_string());
let path_str =
std::env::var("GITLAWB_KEY").unwrap_or_else(|_| "~/.gitlawb/identity.pem".to_string());
if let Some(stripped) = path_str.strip_prefix("~/") {
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
@@ -399,8 +479,7 @@ mod tests {
#[test]
fn parse_standard_url() {
let (did, owner, repo) =
parse_gitlawb_url("gitlawb://did:key:z6MkFoo123/my-repo").unwrap();
let (did, owner, repo) = parse_gitlawb_url("gitlawb://did:key:z6MkFoo123/my-repo").unwrap();
assert_eq!(did, "did:key:z6MkFoo123");
assert_eq!(owner, "z6MkFoo123");
assert_eq!(repo, "my-repo");
@@ -408,8 +487,7 @@ mod tests {
#[test]
fn parse_url_strips_dot_git() {
let (_, _, repo) =
parse_gitlawb_url("gitlawb://did:key:z6MkFoo123/my-repo.git").unwrap();
let (_, _, repo) = parse_gitlawb_url("gitlawb://did:key:z6MkFoo123/my-repo.git").unwrap();
assert_eq!(repo, "my-repo");
}
@@ -445,4 +523,85 @@ mod tests {
"/z6Mk/myrepo/git-receive-pack"
);
}
fn args(items: &[&str]) -> Vec<String> {
items.iter().map(|s| s.to_string()).collect()
}
#[test]
fn classify_version_flags() {
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "--version"])),
Invocation::Version
);
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "-V"])),
Invocation::Version
);
}
#[test]
fn classify_help_flags() {
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "--help"])),
Invocation::Help
);
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "-h"])),
Invocation::Help
);
}
#[test]
fn classify_normal_helper_invocation() {
// How git actually calls us: `<remote-name> <url>`.
assert_eq!(
classify_args(&args(&[
"git-remote-gitlawb",
"origin",
"gitlawb://did:key:z6MkFoo123/my-repo",
])),
Invocation::Helper
);
}
#[test]
fn classify_no_args_is_helper() {
// No flag → falls through to the helper path, which reports its own
// usage error. `--version`/`--help` must not be inferred from emptiness.
assert_eq!(
classify_args(&args(&["git-remote-gitlawb"])),
Invocation::Helper
);
}
#[test]
fn classify_flag_only_in_first_position() {
// A remote literally named "--version" must not be treated as the flag.
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "origin", "--version",])),
Invocation::Helper
);
}
#[test]
fn version_line_matches_package_metadata() {
let line = version_line();
assert_eq!(
line,
format!("git-remote-gitlawb {}", env!("CARGO_PKG_VERSION"))
);
// Single line, no trailing newline (println! adds the newline).
assert!(!line.contains('\n'));
}
#[test]
fn help_text_includes_version_and_flags() {
let help = help_text();
assert!(help.starts_with(&version_line()));
assert!(help.contains("--version"));
assert!(help.contains("--help"));
assert!(help.contains("GITLAWB_NODE"));
assert!(help.ends_with('\n'));
}
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.4.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
serde = { workspace = true }
serde_json = { workspace = true }
serde_jcs = { workspace = true }
thiserror = { workspace = true }
ed25519-dalek = { workspace = true }
sha2 = { workspace = true }
hex = { workspace = true }
base64 = { workspace = true }
multibase = { workspace = true }
[dev-dependencies]
gitlawb-core = { path = "../gitlawb-core" }
rand = { workspace = true }
+557
View File
@@ -0,0 +1,557 @@
//! One typed, signed, cert-bound provenance blob.
//!
//! Fields: `type` (discriminator), `payload` (opaque, type-specific JSON),
//! `cert_hash` (binds to one ref-update cert), `signer` (`did:key`), and
//! `sig` (base64url-no-pad ed25519).
//!
//! The signature covers the domain-separated input
//! `b"gitlawb-attest-sig/v1\n"` followed by JCS-encoded `{type, payload,
//! cert_hash}` per RFC 8785. The domain tag is in the byte stream rather
//! than the canonical JSON so the wire shape stays additive: an `Attestation`
//! deserialized today and one constructed by a future `v2` signer differ only
//! by which tag the verifier prepends.
//!
//! Type discriminators are slash-separated namespace + version strings
//! (`covenant/exec/v1`, `slsa/v1.0`, `sigstore/dsse/v1`); verifiers register
//! by exact match.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64U, Engine};
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use crate::error::{Error, Result};
/// Domain-separation tag prepended to the JCS signing input. Bumped to
/// `v2` if the signing-input shape changes.
const SIGNING_DOMAIN: &[u8] = b"gitlawb-attest-sig/v1\n";
/// Multicodec varint prefix for Ed25519 public keys (0xed 0x01).
const ED25519_MULTICODEC: [u8; 2] = [0xed, 0x01];
/// A signed provenance blob attached to a ref-update cert.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Attestation {
/// Type discriminator, e.g. `covenant/exec/v1`.
#[serde(rename = "type")]
pub type_: String,
/// Type-specific payload. The verifier reparses into its concrete shape.
pub payload: serde_json::Value,
/// Lowercase SHA-256 hex of the cert body. Binds this attestation to one
/// cert. The lowercase form is canonical because the JCS signing input
/// includes this field verbatim — accepting mixed case on the wire would
/// require both signer and verifier to normalize before computing the
/// JCS bytes, an easy spec rule to miss across language ports.
pub cert_hash: String,
/// `did:key` of the signer; the verifying key is recoverable from it.
pub signer: String,
/// base64url-no-pad ed25519 signature over the JCS signing input.
pub sig: String,
}
/// Type-specific payload. Implement on a struct that is `Serialize +
/// DeserializeOwned` to participate.
pub trait AttestationPayload: Serialize + DeserializeOwned + Send + Sync {
/// The discriminator string written on the wire.
fn payload_type() -> &'static str;
}
#[derive(Serialize)]
struct SigningInput<'a> {
#[serde(rename = "type")]
type_: &'a str,
payload: &'a serde_json::Value,
cert_hash: &'a str,
}
impl Attestation {
/// Sign a fresh attestation. `cert_hash_bytes` comes from
/// [`crate::cert::cert_hash`].
pub fn sign<P: AttestationPayload>(
signing_key: &SigningKey,
payload: P,
cert_hash_bytes: [u8; 32],
) -> Result<Self> {
let type_ = P::payload_type().to_string();
validate_type(&type_)?;
let payload_value = serde_json::to_value(payload)?;
let cert_hash_hex = hex::encode(cert_hash_bytes);
let bytes = canonical_signing_bytes(&type_, &payload_value, &cert_hash_hex)?;
let sig: Signature = signing_key.sign(&bytes);
Ok(Self {
type_,
payload: payload_value,
cert_hash: cert_hash_hex,
signer: did_key_from_verifying_key(&signing_key.verifying_key()),
sig: B64U.encode(sig.to_bytes()),
})
}
/// Verify the signature and check that `cert_hash` matches
/// `expected_cert_hash`. Returns the recovered verifying key so the caller
/// can check it against an allowlist.
pub fn verify_signature(&self, expected_cert_hash: [u8; 32]) -> Result<VerifyingKey> {
let expected_hex = hex::encode(expected_cert_hash);
if self.cert_hash != expected_hex {
return Err(Error::CertHashMismatch);
}
validate_type(&self.type_)?;
let bytes = canonical_signing_bytes(&self.type_, &self.payload, &self.cert_hash)?;
let vk = verifying_key_from_did_key(&self.signer)?;
let sig_bytes: [u8; 64] = B64U
.decode(&self.sig)
.map_err(|e| Error::Signature(format!("base64url: {e}")))?
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify(&bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(vk)
}
/// Reparse `payload` as `P`. Errors if the type discriminator does not
/// match `P::payload_type()`.
pub fn payload_as<P: AttestationPayload>(&self) -> Result<P> {
if self.type_ != P::payload_type() {
return Err(Error::Type(format!(
"expected '{}', got '{}'",
P::payload_type(),
self.type_
)));
}
Ok(P::deserialize(&self.payload)?)
}
}
/// Discriminator grammar: nonempty, ASCII-graphic only, slash-separated
/// segments of `[a-zA-Z0-9._+-]`, no empty segment, capped at 128 bytes.
/// Matches `covenant/exec/v1`, `slsa/v1.0`, `sigstore/dsse/v1`.
fn validate_type(t: &str) -> Result<()> {
if t.is_empty() {
return Err(Error::Type("empty discriminator".to_string()));
}
if t.len() > 128 {
return Err(Error::Type(format!(
"discriminator longer than 128 bytes ({})",
t.len()
)));
}
if !t.is_ascii() {
return Err(Error::Type(format!("discriminator must be ASCII: '{t}'")));
}
for segment in t.split('/') {
if segment.is_empty() {
return Err(Error::Type(format!(
"discriminator has an empty segment: '{t}'"
)));
}
for c in segment.chars() {
if !(c.is_ascii_alphanumeric() || c == '.' || c == '_' || c == '+' || c == '-') {
return Err(Error::Type(format!(
"discriminator segment '{segment}' has disallowed character '{c}'"
)));
}
}
}
Ok(())
}
fn canonical_signing_bytes(
type_: &str,
payload: &serde_json::Value,
cert_hash_hex: &str,
) -> Result<Vec<u8>> {
let input = SigningInput {
type_,
payload,
cert_hash: cert_hash_hex,
};
let jcs = serde_jcs::to_vec(&input).map_err(|e| Error::Jcs(e.to_string()))?;
let mut out = Vec::with_capacity(SIGNING_DOMAIN.len() + jcs.len());
out.extend_from_slice(SIGNING_DOMAIN);
out.extend_from_slice(&jcs);
Ok(out)
}
fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&key.to_bytes());
let encoded = multibase::encode(multibase::Base::Base58Btc, &buf);
format!("did:key:{encoded}")
}
/// The DID:key spec mandates base58btc (multibase `z`-prefix). Other multibase
/// encodings of the same key bytes are rejected so allowlists, dedup, and
/// signer-pinning policies that compare `signer` strings remain consistent
/// across peers.
fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
let method_id = did
.strip_prefix("did:key:")
.ok_or_else(|| Error::Did(format!("not a did:key: {did}")))?;
if !method_id.starts_with('z') {
return Err(Error::Did(format!(
"did:key must use base58btc (z-prefix): {did}"
)));
}
let (base, bytes) =
multibase::decode(method_id).map_err(|e| Error::Did(format!("multibase: {e}")))?;
if base != multibase::Base::Base58Btc {
return Err(Error::Did(format!(
"did:key must use base58btc (z-prefix): {did}"
)));
}
if bytes.len() != ED25519_MULTICODEC.len() + 32
|| bytes[..ED25519_MULTICODEC.len()] != ED25519_MULTICODEC
{
return Err(Error::Did("not an ed25519 did:key".to_string()));
}
let key_bytes: [u8; 32] = bytes[ED25519_MULTICODEC.len()..]
.try_into()
.expect("length checked above");
VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
fn fresh() -> SigningKey {
SigningKey::generate(&mut OsRng)
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct DummyPayload {
agent: String,
commit: String,
}
impl AttestationPayload for DummyPayload {
fn payload_type() -> &'static str {
"test/dummy/v1"
}
}
fn sample_cert_hash() -> [u8; 32] {
let mut h = [0u8; 32];
for (i, b) in h.iter_mut().enumerate() {
*b = i as u8;
}
h
}
fn dummy_attestation(sk: &SigningKey, cert_hash: [u8; 32]) -> Attestation {
Attestation::sign(
sk,
DummyPayload {
agent: "a".to_string(),
commit: "b".to_string(),
},
cert_hash,
)
.unwrap()
}
#[test]
fn sign_then_verify_roundtrip() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let payload = DummyPayload {
agent: "did:key:z6MkTest".to_string(),
commit: "deadbeef".to_string(),
};
let att = Attestation::sign(&sk, payload.clone(), cert_hash).unwrap();
let vk = att.verify_signature(cert_hash).unwrap();
assert_eq!(vk.to_bytes(), sk.verifying_key().to_bytes());
assert_eq!(att.payload_as::<DummyPayload>().unwrap(), payload);
}
#[test]
fn cross_cert_replay_fails() {
let sk = fresh();
let cert_a = sample_cert_hash();
let mut cert_b = sample_cert_hash();
cert_b[0] ^= 0xff;
let att = dummy_attestation(&sk, cert_a);
let err = att.verify_signature(cert_b).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
#[test]
fn tampered_payload_fails_verify() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&sk, cert_hash);
att.payload = serde_json::json!({ "agent": "evil", "commit": "b" });
let err = att.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
#[test]
fn payload_as_wrong_type_errors() {
let sk = fresh();
let att = dummy_attestation(&sk, sample_cert_hash());
#[derive(Debug, Serialize, Deserialize)]
struct Wrong {
#[allow(dead_code)]
x: String,
}
impl AttestationPayload for Wrong {
fn payload_type() -> &'static str {
"test/other/v1"
}
}
let err = att.payload_as::<Wrong>().unwrap_err();
assert!(matches!(err, Error::Type(_)));
}
#[test]
fn json_roundtrip_preserves_signature() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let att = dummy_attestation(&sk, cert_hash);
let json = serde_json::to_string(&att).unwrap();
let back: Attestation = serde_json::from_str(&json).unwrap();
back.verify_signature(cert_hash).unwrap();
}
/// Wire shape is part of the public protocol — pin field names so an
/// accidental rename surfaces as a test failure, not silent interop drift.
#[test]
fn wire_shape_pins_field_names() {
let sk = fresh();
let att = dummy_attestation(&sk, sample_cert_hash());
let v: serde_json::Value = serde_json::to_value(&att).unwrap();
let obj = v.as_object().expect("attestation must serialize as object");
let keys: std::collections::BTreeSet<&str> = obj.keys().map(String::as_str).collect();
let expected: std::collections::BTreeSet<&str> =
["type", "payload", "cert_hash", "signer", "sig"]
.into_iter()
.collect();
assert_eq!(keys, expected);
assert!(obj["type"].is_string());
assert!(obj["cert_hash"].is_string());
assert!(obj["signer"].is_string());
assert!(obj["sig"].is_string());
}
/// The spec mandates lowercase hex on the wire. An uppercase cert_hash
/// is rejected as `CertHashMismatch` so a non-canonical signer can't
/// produce attestations that verify against some peers but not others.
#[test]
fn uppercase_cert_hash_is_rejected_as_non_canonical() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&sk, cert_hash);
att.cert_hash = att.cert_hash.to_uppercase();
let err = att.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
#[test]
fn empty_type_rejected_at_signing() {
#[derive(Serialize, Deserialize)]
struct Empty {}
impl AttestationPayload for Empty {
fn payload_type() -> &'static str {
""
}
}
let sk = fresh();
let err = Attestation::sign(&sk, Empty {}, sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Type(_)));
}
#[test]
fn validate_type_grammar() {
validate_type("covenant/exec/v1").unwrap();
validate_type("slsa/v1.0").unwrap();
validate_type("sigstore/dsse/v1").unwrap();
validate_type("a").unwrap();
validate_type("a.b-c_d+e/x").unwrap();
for bad in [
"",
" ",
"covenant/exec/v1 ",
"covenant//v1",
"/covenant/v1",
"covenant/v1/",
"covenant\x00exec/v1",
"covenant/exec\nv1",
"сovenant/exec/v1",
"covenant/ex ec/v1",
"covenant/exec/v1#frag",
] {
let err = validate_type(bad).unwrap_err();
assert!(matches!(err, Error::Type(_)), "must reject '{bad}'");
}
let too_long = "a/".repeat(70);
let err = validate_type(&too_long).unwrap_err();
assert!(matches!(err, Error::Type(_)));
}
#[test]
fn did_key_must_be_base58btc() {
// Sign with a real key, then re-encode the public-key bytes in a
// different multibase. Verification must reject.
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
let mut buf = Vec::with_capacity(34);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&sk.verifying_key().to_bytes());
for base in [
multibase::Base::Base64Url,
multibase::Base::Base32Lower,
multibase::Base::Base16Lower,
] {
att.signer = format!("did:key:{}", multibase::encode(base, &buf));
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)), "must reject base {base:?}");
}
}
#[test]
fn verify_rejects_did_without_did_key_prefix() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
att.signer = "ed25519:abc".to_string();
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_did_with_wrong_multicodec() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
// 0x12 0x00 = sha2-256 multicodec — wrong for an ed25519 public key.
let mut buf = vec![0x12, 0x00];
buf.extend_from_slice(&sk.verifying_key().to_bytes());
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_did_with_wrong_key_length() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
let mut buf = Vec::with_capacity(33);
buf.extend_from_slice(&ED25519_MULTICODEC);
// 31 bytes instead of 32.
buf.extend_from_slice(&sk.verifying_key().to_bytes()[..31]);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_bad_signature_base64() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
att.sig = "!!not-base64!!".to_string();
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
#[test]
fn verify_rejects_signature_wrong_length() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
att.sig = B64U.encode([0u8; 32]); // valid base64, wrong size.
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
/// A payload that happens to contain a `cert_hash` field of its own does
/// not interfere with the outer binding: the attestation envelope's
/// `cert_hash` is the only field consulted by `verify_signature`, and the
/// payload is treated as opaque JSON throughout. A reviewer worried about
/// shadowing or smuggling can verify this directly.
#[test]
fn payload_cert_hash_field_does_not_shadow_outer_binding() {
#[derive(Serialize, Deserialize)]
struct Sneaky {
cert_hash: String,
x: String,
}
impl AttestationPayload for Sneaky {
fn payload_type() -> &'static str {
"test/sneaky/v1"
}
}
let sk = fresh();
let outer_hash = sample_cert_hash();
let att = Attestation::sign(
&sk,
Sneaky {
cert_hash: "forged-by-the-attacker".to_string(),
x: "y".to_string(),
},
outer_hash,
)
.unwrap();
// Envelope's cert_hash is the outer (real) hash, lowercase hex.
assert_eq!(att.cert_hash, hex::encode(outer_hash));
// Payload retains its own field — opaque to the protocol.
assert_eq!(
att.payload.get("cert_hash").and_then(|v| v.as_str()),
Some("forged-by-the-attacker"),
);
// Verification binds to the outer envelope's cert_hash.
att.verify_signature(outer_hash).unwrap();
// And rejects when the outer cert hash differs.
let mut wrong = outer_hash;
wrong[0] ^= 0xff;
let err = att.verify_signature(wrong).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
/// A different protocol that also signs `{type, payload, cert_hash}` with
/// the same key would still fail attestation verification because the
/// signing input carries a domain tag the other protocol does not include.
#[test]
fn cross_protocol_replay_is_rejected_by_domain_separation() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let att = dummy_attestation(&sk, cert_hash);
// Sign the same JCS bytes WITHOUT the domain tag and graft the
// signature back onto the attestation.
let cert_hash_hex = hex::encode(cert_hash);
let bare = serde_jcs::to_vec(&SigningInput {
type_: &att.type_,
payload: &att.payload,
cert_hash: &cert_hash_hex,
})
.unwrap();
let other_sig: Signature = sk.sign(&bare);
let mut tampered = att.clone();
tampered.sig = B64U.encode(other_sig.to_bytes());
let err = tampered.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
}
+327
View File
@@ -0,0 +1,327 @@
//! Envelope around a `gitlawb/ref-update/v1` cert plus optional attestations,
//! and the cert-hash helper attestations bind to.
//!
//! An empty envelope serializes to the same bytes as the underlying bare cert.
//! `cert_hash` strips `signatures` and `attestations`, JCS-encodes the rest,
//! and SHA-256s; the result is what every attestation binds to.
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use crate::attestation::Attestation;
use crate::error::{Error, Result};
use crate::verifier::{Registry, VerifiedAttestation};
/// The discriminator the wrapped cert must carry. Matches the constant of the
/// same name in `gitlawb-core::cert`.
pub const CERT_TYPE: &str = "gitlawb/ref-update/v1";
/// A ref-update cert with optional attestations.
///
/// The cert is flattened into the top-level JSON, so an empty envelope
/// produces the same bytes as a bare cert and existing decoders ignore the
/// extra `attestations` field. The cert is held as `serde_json::Value` so
/// this crate has no runtime dependency on `gitlawb-core::cert::RefUpdateCert`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AttestedRefUpdateCert {
/// The cert body and signatures, flattened to the top level.
#[serde(flatten)]
pub cert: serde_json::Value,
/// Attached attestations. Empty when absent on the wire.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub attestations: Vec<Attestation>,
}
impl AttestedRefUpdateCert {
/// Wrap a cert. No attestations are attached yet.
///
/// The serialized cert must be a JSON object whose `type` field equals
/// [`CERT_TYPE`] and which does **not** already carry an `attestations`
/// field. The second rule catches two cases that would otherwise leave
/// the envelope in an inconsistent in-memory state:
///
/// - Double-wrap: `from_cert(&previously_wrapped_envelope)` would put the
/// inner attestations on `self.cert` and leave `self.attestations` empty.
/// - Pre-attested cert from a peer: a malformed or malicious cert with an
/// attached `attestations` field would survive `cert_hash` (which strips
/// the field) but bypass the wrapper's append-and-verify path entirely.
///
/// Both are rejected with [`Error::Payload`]. To re-wrap, deserialize the
/// envelope as `AttestedRefUpdateCert` directly.
pub fn from_cert<T: Serialize>(cert: &T) -> Result<Self> {
let value = serde_json::to_value(cert)?;
let obj = value
.as_object()
.ok_or_else(|| Error::Payload("cert must serialize as a JSON object".to_string()))?;
match obj.get("type").and_then(|v| v.as_str()) {
Some(t) if t == CERT_TYPE => {}
Some(other) => {
return Err(Error::Payload(format!(
"cert type must be '{CERT_TYPE}', got '{other}'"
)));
}
None => {
return Err(Error::Payload(format!(
"cert missing 'type' field (expected '{CERT_TYPE}')"
)));
}
}
if obj.contains_key("attestations") {
return Err(Error::Payload(
"cert already carries an 'attestations' field; deserialize as \
AttestedRefUpdateCert directly instead of re-wrapping"
.to_string(),
));
}
Ok(Self {
cert: value,
attestations: Vec::new(),
})
}
/// The 32-byte cert hash every attached attestation must bind to.
pub fn cert_hash(&self) -> Result<[u8; 32]> {
cert_hash(&self.cert)
}
/// Append an already-signed attestation.
pub fn attach(&mut self, attestation: Attestation) {
self.attestations.push(attestation);
}
/// Verify every attached attestation against this envelope's cert hash,
/// applying the registry's policy. Equivalent to
/// `registry.verify_all(&self.attestations, self.cert_hash()?)` but
/// reads as one operation at call sites.
pub fn verify_attestations(&self, registry: &Registry) -> Result<Vec<VerifiedAttestation>> {
registry.verify_all(&self.attestations, self.cert_hash()?)
}
}
/// SHA-256 of the cert body. The body must be a JSON object; `signatures` and
/// `attestations` are stripped before JCS encoding (RFC 8785), so the hash is
/// reproducible across languages and JSON libraries and is stable across
/// countersignatures.
pub fn cert_hash(cert: &serde_json::Value) -> Result<[u8; 32]> {
let mut body = cert.clone();
let obj = body
.as_object_mut()
.ok_or_else(|| Error::Payload("cert body must be a JSON object".to_string()))?;
obj.remove("signatures");
obj.remove("attestations");
let bytes = serde_jcs::to_vec(&body).map_err(|e| Error::Jcs(e.to_string()))?;
let mut h = Sha256::new();
h.update(&bytes);
let out = h.finalize();
let mut arr = [0u8; 32];
arr.copy_from_slice(&out);
Ok(arr)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn bare_cert() -> serde_json::Value {
json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkRepo",
"ref_name": "refs/heads/main",
"from": "0".repeat(64),
"to": "a".repeat(64),
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "n1",
"signatures": [{"signer": "did:key:z6MkRepo", "sig": "abc"}]
})
}
#[test]
fn empty_envelope_serializes_as_bare_cert() {
let bare = bare_cert();
let env = AttestedRefUpdateCert::from_cert(&bare).unwrap();
let env_json: serde_json::Value =
serde_json::from_str(&serde_json::to_string(&env).unwrap()).unwrap();
let bare_keys: std::collections::BTreeSet<&String> =
bare.as_object().unwrap().keys().collect();
let env_keys: std::collections::BTreeSet<&String> =
env_json.as_object().unwrap().keys().collect();
assert_eq!(bare_keys, env_keys);
}
#[test]
fn bare_cert_parses_as_envelope_with_no_attestations() {
let bare = bare_cert();
let env: AttestedRefUpdateCert = serde_json::from_value(bare.clone()).unwrap();
assert!(env.attestations.is_empty());
assert_eq!(env.cert, bare);
}
#[test]
fn cert_hash_is_stable_across_countersignatures() {
let base_body = json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkRepo",
"ref_name": "refs/heads/main",
"from": "0".repeat(64),
"to": "a".repeat(64),
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "n1"
});
let mut one_sig = base_body.clone();
one_sig["signatures"] = json!([{"signer": "x", "sig": "y"}]);
let mut two_sigs = base_body.clone();
two_sigs["signatures"] = json!([
{"signer": "x", "sig": "y"},
{"signer": "a", "sig": "b"}
]);
assert_eq!(cert_hash(&one_sig).unwrap(), cert_hash(&two_sigs).unwrap());
}
#[test]
fn cert_hash_changes_when_body_changes() {
let body_a = json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkRepo",
"ref_name": "refs/heads/main",
"from": "0".repeat(64),
"to": "a".repeat(64),
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "n1",
"signatures": []
});
let mut body_b = body_a.clone();
body_b["to"] = json!("b".repeat(64));
assert_ne!(cert_hash(&body_a).unwrap(), cert_hash(&body_b).unwrap());
}
#[test]
fn cert_hash_rejects_non_object_root() {
let err = cert_hash(&json!(null)).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
let err = cert_hash(&json!([1, 2, 3])).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
let err = cert_hash(&json!("scalar")).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn from_cert_rejects_wrong_type() {
let wrong = json!({"type": "gitlawb/ref-update/v0", "repo": "did:key:z"});
let err = AttestedRefUpdateCert::from_cert(&wrong).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn from_cert_rejects_missing_type() {
let no_type = json!({"repo": "did:key:z"});
let err = AttestedRefUpdateCert::from_cert(&no_type).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn from_cert_rejects_non_object() {
let arr = json!([1, 2, 3]);
let err = AttestedRefUpdateCert::from_cert(&arr).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn mutating_env_cert_changes_the_hash() {
let mut env = AttestedRefUpdateCert::from_cert(&bare_cert()).unwrap();
let hash_before = env.cert_hash().unwrap();
env.cert.as_object_mut().unwrap().insert(
"nonce".to_string(),
serde_json::Value::String("mutated".to_string()),
);
let hash_after = env.cert_hash().unwrap();
assert_ne!(hash_before, hash_after);
}
/// A cert that already carries an `attestations` field is rejected
/// regardless of the field's contents. Catches double-wrap (where the
/// inner attestations would otherwise be smuggled into `self.cert`) and
/// peer-supplied "pre-attested" certs that would bypass the wrapper.
#[test]
fn from_cert_rejects_preexisting_attestations_field() {
let mut sneaky = bare_cert();
sneaky["attestations"] = json!([{"smuggled": true}]);
let err = AttestedRefUpdateCert::from_cert(&sneaky).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
// Empty array still rejected — the field's mere presence is the signal.
let mut empty = bare_cert();
empty["attestations"] = json!([]);
let err = AttestedRefUpdateCert::from_cert(&empty).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// `from_cert` on an already-wrapped envelope is the same case as
/// `from_cert` on a cert with a pre-existing `attestations` field: serde
/// flattens the envelope back into a JSON object that carries the field.
#[test]
fn from_cert_rejects_a_previously_wrapped_envelope() {
let mut env = AttestedRefUpdateCert::from_cert(&bare_cert()).unwrap();
// Inject an attestation-shaped value so the wire round-trip would
// actually re-emit the field.
env.cert
.as_object_mut()
.unwrap()
.insert("attestations".to_string(), json!([]));
let err = AttestedRefUpdateCert::from_cert(&env).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// `attestations: null` is explicitly not the same as missing. The
/// `#[serde(default)]` attribute only fills in when the key is absent —
/// a present-but-null wire field is rejected, which surfaces malformed
/// payloads early instead of silently treating them as empty.
#[test]
fn attestations_null_on_wire_is_rejected() {
let mut with_null = bare_cert();
with_null["attestations"] = serde_json::Value::Null;
let err = serde_json::from_value::<AttestedRefUpdateCert>(with_null).unwrap_err();
let msg = format!("{err}");
assert!(
msg.contains("expected a sequence") || msg.contains("invalid type"),
"unexpected error: {msg}"
);
}
#[test]
fn attestations_string_on_wire_is_rejected() {
let mut with_str = bare_cert();
with_str["attestations"] = serde_json::Value::String("oops".to_string());
let err = serde_json::from_value::<AttestedRefUpdateCert>(with_str).unwrap_err();
let msg = format!("{err}");
assert!(msg.contains("expected a sequence") || msg.contains("invalid type"));
}
/// `attach` does not deduplicate. Duplicate attestations are a verifier-
/// policy concern (e.g., "at most one SLSA provenance"), not a wrapper
/// concern. Pinning the behavior so a future deduplication change is an
/// intentional break.
#[test]
fn attach_does_not_deduplicate() {
let mut env = AttestedRefUpdateCert::from_cert(&bare_cert()).unwrap();
let att = Attestation {
type_: "demo/v1".to_string(),
payload: serde_json::json!({}),
cert_hash: "deadbeef".to_string(),
signer: "did:key:z6MkDup".to_string(),
sig: "AAAA".to_string(),
};
env.attach(att.clone());
env.attach(att);
assert_eq!(env.attestations.len(), 2);
}
}
+46
View File
@@ -0,0 +1,46 @@
use thiserror::Error;
/// Errors from building or verifying an attestation.
#[derive(Debug, Error)]
pub enum Error {
/// Discriminator failed validation (empty, whitespace, control char,
/// disallowed grammar).
#[error("attestation type: {0}")]
Type(String),
/// Attestation was signed against a different cert.
#[error("cert hash mismatch")]
CertHashMismatch,
/// Ed25519 signature failed verification.
#[error("signature: {0}")]
Signature(String),
/// DID could not be parsed.
#[error("did: {0}")]
Did(String),
/// Payload structure check failed, or the cert body was not a JSON object.
#[error("payload: {0}")]
Payload(String),
/// No verifier registered for the type and policy required one.
#[error("no verifier registered for type '{0}'")]
UnknownType(String),
/// `RequireAll` policy did not find an entry for a required type, or the
/// entry was not fully verified.
#[error("required attestation type missing or unverified: '{0}'")]
RequiredMissing(String),
/// JSON failure.
#[error("json: {0}")]
Json(#[from] serde_json::Error),
/// JCS encoding failure.
#[error("jcs: {0}")]
Jcs(String),
}
/// Crate result alias.
pub type Result<T> = std::result::Result<T, Error>;
+132
View File
@@ -0,0 +1,132 @@
//! Pluggable provenance attachments for gitlawb ref-update certs.
//!
//! A `gitlawb/ref-update/v1` cert proves who pushed and where a ref moved to.
//! It does not say how the commit was produced. This crate adds an optional
//! `attestations` field that lets any provenance system (SLSA, Sigstore,
//! in-toto, agent runtimes) attach a typed, signed blob bound to the cert by
//! its hash.
//!
//! The cert format stays additive. A cert with no attestations serializes the
//! same bytes it does today, and existing decoders (which do not set
//! `deny_unknown_fields`) silently drop the new `attestations` field rather
//! than reject the cert.
//!
//! ## Wire shape
//!
//! ```json
//! {
//! "type": "gitlawb/ref-update/v1",
//! // ...standard cert fields...,
//! "signatures": [...],
//! "attestations": [
//! {
//! "type": "covenant/exec/v1",
//! "payload": { /* opaque, type-specific */ },
//! "cert_hash": "<sha256 hex of JCS-encoded cert body>",
//! "signer": "did:key:z6Mk...",
//! "sig": "<base64url ed25519 over the signing input>"
//! }
//! ]
//! }
//! ```
//!
//! ## Canonical bytes
//!
//! Hashing and signing inputs use JCS (RFC 8785) so they reproduce across
//! implementations regardless of struct field order or JSON library.
//!
//! The cert hash is `SHA-256(JCS(cert_body_without_signatures_or_attestations))`,
//! expressed as lowercase hex on the wire. Uppercase is non-canonical and
//! rejected on verify so the JCS signing input is uniquely determined by the
//! cert bytes alone — there is no implicit normalization rule that
//! cross-language signers might miss.
//!
//! The attestation signing input is the byte string
//! `b"gitlawb-attest-sig/v1\n" || JCS({type, payload, cert_hash})`. The
//! domain tag is in the byte stream, not the canonical JSON, so an attestation
//! cannot be confused with a same-shape JCS document signed for a different
//! protocol that uses the same Ed25519 key.
//!
//! ## DID:key
//!
//! Signers are `did:key` DIDs over Ed25519, encoded with the multibase
//! base58btc (`z`) prefix per the W3C DID:key specification. Non-base58btc
//! encodings of the same public key are rejected on verify so allowlists,
//! deduplication, and signer-pinning policies that compare `signer` strings
//! stay consistent across peers.
//!
//! ## Verification
//!
//! [`Registry`] looks up a verifier by type discriminator. [`Policy`] decides
//! what to do when no verifier matches: `AcceptKnown` (default) lets unknown
//! types pass without trust; `RequireAll` enforces a per-repo allowlist while
//! still letting unrelated attached attestations through unverified (so a
//! third-party attachment cannot DoS the cert); `RejectUnknown` rejects
//! anything unregistered.
//!
//! ## Quick start
//!
//! ```
//! use ed25519_dalek::SigningKey;
//! use gitlawb_attest::{
//! Attestation, AttestationPayload, AttestationVerifier, AttestedRefUpdateCert,
//! Error, Policy, Registry, Result, CERT_TYPE,
//! };
//! use serde::{Deserialize, Serialize};
//!
//! #[derive(Serialize, Deserialize)]
//! struct ExecPayload { agent: String }
//! impl AttestationPayload for ExecPayload {
//! fn payload_type() -> &'static str { "covenant/exec/v1" }
//! }
//!
//! struct ExecVerifier;
//! impl AttestationVerifier for ExecVerifier {
//! fn type_(&self) -> &'static str { "covenant/exec/v1" }
//! fn verify_payload(&self, payload: &serde_json::Value) -> Result<()> {
//! payload.get("agent").and_then(|v| v.as_str())
//! .ok_or_else(|| Error::Payload("missing agent".into()))?;
//! Ok(())
//! }
//! }
//!
//! # fn run() -> Result<()> {
//! let cert = serde_json::json!({
//! "type": CERT_TYPE, "repo": "did:key:z6MkRepo",
//! "ref_name": "refs/heads/main",
//! "from": "0".repeat(64), "to": "a".repeat(64),
//! "seq": 1, "timestamp": "2026-01-01T00:00:00Z", "nonce": "n1",
//! "signatures": []
//! });
//! let mut env = AttestedRefUpdateCert::from_cert(&cert)?;
//!
//! let sk = SigningKey::from_bytes(&[7u8; 32]);
//! let payload = ExecPayload { agent: "did:key:z6MkAgent".into() };
//! let att = Attestation::sign(&sk, payload, env.cert_hash()?)?;
//! env.attach(att);
//!
//! let mut registry = Registry::new().with_policy(Policy::RejectUnknown);
//! registry.register(ExecVerifier);
//! let verified = env.verify_attestations(&registry)?;
//! assert_eq!(verified[0].type_, "covenant/exec/v1");
//! assert!(verified[0].fully_verified);
//! # Ok(()) }
//! # run().unwrap();
//! ```
pub mod attestation;
pub mod cert;
pub mod error;
pub mod verifier;
pub use attestation::{Attestation, AttestationPayload};
pub use cert::{cert_hash, AttestedRefUpdateCert, CERT_TYPE};
pub use error::{Error, Result};
pub use verifier::{AttestationVerifier, Policy, Registry, VerifiedAttestation};
/// Version string for the attestation envelope wire shape.
///
/// Bumped only when the binding rules — JCS input shape, domain-separation
/// tag, hash algorithm — change. Independent of [`CERT_TYPE`], which versions
/// the underlying ref-update cert.
pub const ATTEST_ENVELOPE_VERSION: &str = "v1";
+377
View File
@@ -0,0 +1,377 @@
//! Verifier trait, registry, and policy.
//!
//! A [`Registry`] maps a type discriminator to a verifier. [`Policy`] decides
//! what happens for types nothing is registered for: accept without trust,
//! reject, or require an allowlist.
//!
//! Under [`Policy::RequireAll`] the registry never short-circuits on unknown
//! types — any attestation can be attached by any third party, so blocking
//! the cert because an attacker added an unrelated `attacker/spam/v1` would
//! be a denial-of-service vector. Unknown types are silently treated as
//! `fully_verified = false`; the allowlist is enforced only by the
//! presence-and-verified check on `required_types` after the batch runs.
use std::collections::HashMap;
use crate::attestation::Attestation;
use crate::error::{Error, Result};
/// Verifier for one attestation type. Implementors declare the discriminator
/// they handle and validate the payload's shape. The signature and cert-hash
/// binding are checked by [`Registry::verify`] before `verify_payload` runs.
pub trait AttestationVerifier: Send + Sync {
/// The discriminator this verifier handles. Must be a `'static` literal
/// so the registry can key on it without copying.
fn type_(&self) -> &'static str;
/// Validate payload structure. Signature is already verified.
fn verify_payload(&self, payload: &serde_json::Value) -> Result<()>;
}
/// What to do for attestation types with no registered verifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum Policy {
/// Unknown types pass but are flagged `fully_verified = false`. Default.
#[default]
AcceptKnown,
/// Every entry in `required_types` must be present and fully verified.
/// Unknown types beyond the required set are still accepted unverified,
/// so a malicious extra attestation cannot DoS the cert.
RequireAll,
/// Reject every attestation whose type is not registered.
RejectUnknown,
}
/// Result of verifying one attestation.
#[derive(Debug, Clone)]
pub struct VerifiedAttestation {
/// Type discriminator.
pub type_: String,
/// Signer's `did:key`.
pub signer: String,
/// Lowercase SHA-256 hex of the cert body the attestation bound to.
pub cert_hash: String,
/// `true` when a verifier ran a payload check and accepted it. `false`
/// when the policy let an unknown type through without one.
pub fully_verified: bool,
}
/// Verifiers keyed by type discriminator.
#[derive(Default)]
pub struct Registry {
by_type: HashMap<&'static str, Box<dyn AttestationVerifier>>,
policy: Policy,
required_types: Vec<String>,
}
impl Registry {
/// Empty registry with the default policy.
pub fn new() -> Self {
Self::default()
}
/// Register a verifier by value. Returns the prior verifier for the same
/// type, if any, so callers can detect double-registration.
pub fn register<V: AttestationVerifier + 'static>(
&mut self,
verifier: V,
) -> Option<Box<dyn AttestationVerifier>> {
self.register_boxed(Box::new(verifier))
}
/// Like [`Self::register`] but takes an already-boxed trait object — useful
/// when verifiers are constructed dynamically.
pub fn register_boxed(
&mut self,
verifier: Box<dyn AttestationVerifier>,
) -> Option<Box<dyn AttestationVerifier>> {
let key = verifier.type_();
self.by_type.insert(key, verifier)
}
/// Switch policy.
pub fn with_policy(mut self, policy: Policy) -> Self {
self.policy = policy;
self
}
/// Type discriminators that must be present and verified under
/// `Policy::RequireAll`. Ignored under other policies.
pub fn require_types<I, S>(mut self, types: I) -> Self
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
self.required_types = types.into_iter().map(Into::into).collect();
self
}
/// Active policy.
pub fn policy(&self) -> Policy {
self.policy
}
/// Verify signature, cert-hash binding, and (if a verifier is registered)
/// payload structure. Under `RequireAll` this is lenient on unknown types
/// — see the module docstring.
pub fn verify(
&self,
attestation: &Attestation,
expected_cert_hash: [u8; 32],
) -> Result<VerifiedAttestation> {
attestation.verify_signature(expected_cert_hash)?;
let fully = match self.by_type.get(attestation.type_.as_str()) {
Some(v) => {
v.verify_payload(&attestation.payload)?;
true
}
None => match self.policy {
Policy::AcceptKnown | Policy::RequireAll => false,
Policy::RejectUnknown => {
return Err(Error::UnknownType(attestation.type_.clone()));
}
},
};
Ok(VerifiedAttestation {
type_: attestation.type_.clone(),
signer: attestation.signer.clone(),
cert_hash: attestation.cert_hash.clone(),
fully_verified: fully,
})
}
/// Verify a batch, then enforce `RequireAll`.
pub fn verify_all(
&self,
attestations: &[Attestation],
expected_cert_hash: [u8; 32],
) -> Result<Vec<VerifiedAttestation>> {
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
if self.policy == Policy::RequireAll {
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
}
}
}
Ok(verified)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::attestation::{Attestation, AttestationPayload};
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
fn fresh() -> SigningKey {
SigningKey::generate(&mut OsRng)
}
#[derive(Serialize, Deserialize)]
struct Demo {
a: String,
}
impl AttestationPayload for Demo {
fn payload_type() -> &'static str {
"demo/v1"
}
}
struct DemoVerifier;
impl AttestationVerifier for DemoVerifier {
fn type_(&self) -> &'static str {
"demo/v1"
}
fn verify_payload(&self, payload: &serde_json::Value) -> Result<()> {
payload
.get("a")
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.map(|_| ())
.ok_or_else(|| Error::Payload("missing or empty 'a'".to_string()))
}
}
/// A second verifier that has the same discriminator as `DemoVerifier` —
/// used to test last-wins on `register`.
struct DemoVerifierAlt;
impl AttestationVerifier for DemoVerifierAlt {
fn type_(&self) -> &'static str {
"demo/v1"
}
fn verify_payload(&self, _payload: &serde_json::Value) -> Result<()> {
Err(Error::Payload("alt always rejects".to_string()))
}
}
#[derive(Serialize, Deserialize)]
struct Other {
x: u32,
}
impl AttestationPayload for Other {
fn payload_type() -> &'static str {
"other/v1"
}
}
fn sample_hash() -> [u8; 32] {
let mut h = [0u8; 32];
for (i, b) in h.iter_mut().enumerate() {
*b = (i + 1) as u8;
}
h
}
fn signed_demo(sk: &SigningKey, cert_hash: [u8; 32], a: &str) -> Attestation {
Attestation::sign(sk, Demo { a: a.to_string() }, cert_hash).unwrap()
}
fn signed_other(sk: &SigningKey, cert_hash: [u8; 32], x: u32) -> Attestation {
Attestation::sign(sk, Other { x }, cert_hash).unwrap()
}
#[test]
fn registry_accepts_registered_type() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new();
reg.register(DemoVerifier);
let att = signed_demo(&sk, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
assert!(v.fully_verified);
assert_eq!(v.type_, "demo/v1");
assert_eq!(v.cert_hash, hex::encode(cert_hash));
}
#[test]
fn accept_known_lets_unknown_pass_unverified() {
let sk = fresh();
let cert_hash = sample_hash();
let reg = Registry::new().with_policy(Policy::AcceptKnown);
let att = signed_demo(&sk, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
assert!(!v.fully_verified);
}
#[test]
fn reject_unknown_blocks_unregistered_types() {
let sk = fresh();
let cert_hash = sample_hash();
let reg = Registry::new().with_policy(Policy::RejectUnknown);
let att = signed_demo(&sk, cert_hash, "ok");
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::UnknownType(_)));
}
#[test]
fn require_all_enforces_presence() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
let err = reg.verify_all(&[], cert_hash).unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
let att = signed_demo(&sk, cert_hash, "ok");
let v = reg.verify_all(&[att], cert_hash).unwrap();
assert_eq!(v.len(), 1);
}
/// Even when a junk attestation of an unrelated type is attached first, a
/// `RequireAll` registry must still accept the cert as long as the
/// required type is present and fully verified — the attacker shouldn't
/// be able to DoS the cert by attaching `attacker/spam/v1` before
/// `covenant/exec/v1`.
#[test]
fn require_all_is_lenient_on_unknown_types_in_the_batch() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
let spam = signed_other(&sk, cert_hash, 7);
let real = signed_demo(&sk, cert_hash, "ok");
let verified = reg.verify_all(&[spam, real], cert_hash).unwrap();
assert_eq!(verified.len(), 2);
assert!(verified
.iter()
.any(|v| v.type_ == "demo/v1" && v.fully_verified));
assert!(verified
.iter()
.any(|v| v.type_ == "other/v1" && !v.fully_verified));
}
/// A required type that is present but unverified (no verifier registered
/// even though the type appears in `required_types`) must fail
/// `verify_all` with `RequiredMissing`. Catches a misconfiguration where
/// the operator listed a type but forgot to register a verifier.
#[test]
fn require_all_rejects_present_but_unverified_required_type() {
let sk = fresh();
let cert_hash = sample_hash();
let reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]); // type listed, but no verifier registered.
let att = signed_demo(&sk, cert_hash, "ok");
let err = reg.verify_all(&[att], cert_hash).unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
#[test]
fn payload_check_failure_rejects() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new();
reg.register(DemoVerifier);
// Sign an empty `a` so the signature verifies but the payload check fails.
let att = Attestation::sign(&sk, Demo { a: String::new() }, cert_hash).unwrap();
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// `RequireAll` with an empty allowlist degenerates to "no required types
/// to check" — the loop over `required_types` is a no-op, so an empty
/// batch verifies successfully. Pinned so operators relying on the empty
/// allowlist as a default-permissive RequireAll see consistent behavior.
#[test]
fn require_all_with_empty_allowlist_and_empty_batch_succeeds() {
let reg = Registry::new().with_policy(Policy::RequireAll);
let verified = reg.verify_all(&[], sample_hash()).unwrap();
assert!(verified.is_empty());
}
#[test]
fn register_returns_the_prior_verifier_on_double_register() {
let mut reg = Registry::new();
assert!(reg.register(DemoVerifier).is_none());
let prior = reg.register(DemoVerifierAlt);
let prior = prior.expect("second register must return the previous verifier");
assert_eq!(prior.type_(), "demo/v1");
// Last-wins semantics confirmed by behavior.
let sk = fresh();
let cert_hash = sample_hash();
let att = signed_demo(&sk, cert_hash, "ok");
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
}
@@ -0,0 +1,150 @@
//! Canonical test vectors.
//!
//! Ed25519 signatures are deterministic (RFC 8032). JCS canonicalization is
//! deterministic (RFC 8785). SHA-256 is deterministic. Given fixed inputs the
//! cert hash, signature, signer DID, and full wire JSON are therefore exactly
//! reproducible across implementations. This file pins those values so a
//! port to Go, JS, Python, or any other language has a concrete oracle to
//! check against without reading Rust.
//!
//! If a vector changes, that is a wire-protocol break. The change is either
//! intentional (bump `ATTEST_ENVELOPE_VERSION` and document the migration) or
//! a regression that broke interoperability with every other implementation.
use ed25519_dalek::SigningKey;
use gitlawb_attest::{Attestation, AttestationPayload, AttestedRefUpdateCert, Registry, CERT_TYPE};
use serde::{Deserialize, Serialize};
/// Fixed seed for the attestation signer. Real signers use a CSPRNG; this is a
/// test-vector convention only.
const SIGNER_SEED: [u8; 32] = [
0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef, 0xfe, 0xdc, 0xba, 0x98, 0x76, 0x54, 0x32, 0x10,
0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff,
];
/// Fixed cert body for the vectors. Mirrors the field layout of a
/// `gitlawb-core::cert::RefUpdateBody` plus a `signatures` array. The
/// signatures array is deliberately non-empty to test that it is stripped
/// before hashing.
fn fixture_cert_body() -> serde_json::Value {
serde_json::json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkpYqTAt8jq3SExeUiVdyTrDdocy9j2NjGxFE6S6ZBh1Yt",
"ref_name": "refs/heads/main",
"from": "0000000000000000000000000000000000000000000000000000000000000000",
"to": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "vector-nonce-1",
"signatures": [
{"signer": "did:key:z6MkpYqTAt8jq3SExeUiVdyTrDdocy9j2NjGxFE6S6ZBh1Yt", "sig": "ignored"}
]
})
}
/// Fixed attestation payload. Shape mirrors what Covenant's runtime would
/// produce for a `covenant/exec/v1` attestation.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct ExecPayload {
agent: String,
capability: String,
sandbox_digest: String,
}
impl AttestationPayload for ExecPayload {
fn payload_type() -> &'static str {
"covenant/exec/v1"
}
}
fn fixture_payload() -> ExecPayload {
ExecPayload {
agent: "did:key:z6MkAgentTestVector".to_string(),
capability: "code.write+sandbox.exec".to_string(),
sandbox_digest: "sha256:c0ffee".to_string(),
}
}
/// The expected lowercase-hex SHA-256 of `JCS(cert_body \ {signatures})`.
/// Any port that produces a different value is not JCS-canonicalizing the
/// same way (likely culprits: key ordering, integer or string escaping).
const EXPECTED_CERT_HASH_HEX: &str =
"5898d7c7a7d32a5faaaf9a52bfc92e0eff98027554adeefba0c575816f43ce56";
/// The expected `did:key` of the signer derived from `SIGNER_SEED`. Any port
/// that produces a different value has a multibase or multicodec encoding bug.
const EXPECTED_SIGNER_DID: &str = "did:key:z6MkqkKY69DYo23W4YFDoCNgjr7cMvmJqHtJVoQkEzrbaopZ";
/// The expected base64url-no-pad Ed25519 signature over
/// `b"gitlawb-attest-sig/v1\n" || JCS({type, payload, cert_hash})`. Any port
/// that produces a different value is missing the domain tag, JCS-encoding
/// the input differently, or using a different Ed25519 implementation.
const EXPECTED_SIGNATURE_B64URL: &str =
"4L3ONbhrksB9pn--u86lMeAdePSiZZplAsxuN2Cy6vgI30ndle7gDUVuoGRJr4Ylhz7ipQvUzXaDTvZGFLPXBQ";
#[test]
fn cert_hash_is_deterministic_from_jcs() {
let env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let actual_hex = hex::encode(env.cert_hash().unwrap());
assert_eq!(
actual_hex, EXPECTED_CERT_HASH_HEX,
"cert hash drifted — JCS encoding or hash algorithm changed",
);
}
#[test]
fn signer_did_is_deterministic_from_seed() {
let sk = SigningKey::from_bytes(&SIGNER_SEED);
let env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let cert_hash = env.cert_hash().unwrap();
let att = Attestation::sign(&sk, fixture_payload(), cert_hash).unwrap();
assert_eq!(
att.signer, EXPECTED_SIGNER_DID,
"signer DID drifted — multibase or multicodec encoding changed",
);
}
#[test]
fn attestation_signature_is_deterministic() {
let sk = SigningKey::from_bytes(&SIGNER_SEED);
let env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let cert_hash = env.cert_hash().unwrap();
let att = Attestation::sign(&sk, fixture_payload(), cert_hash).unwrap();
assert_eq!(
att.sig, EXPECTED_SIGNATURE_B64URL,
"signature drifted — domain tag, JCS shape, or Ed25519 implementation changed",
);
}
#[test]
fn vector_attestation_verifies_through_the_public_api() {
// Round-trip the entire pipeline using only the public API on the fixed
// vectors. A non-Rust implementor should be able to:
// 1. Compute cert_hash and check against EXPECTED_CERT_HASH_HEX.
// 2. Derive signer DID and check against EXPECTED_SIGNER_DID.
// 3. Sign a payload identical to fixture_payload() and check against
// EXPECTED_SIGNATURE_B64URL.
// 4. Reconstruct the wire envelope and verify it round-trips.
let sk = SigningKey::from_bytes(&SIGNER_SEED);
let mut env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let cert_hash = env.cert_hash().unwrap();
let att = Attestation::sign(&sk, fixture_payload(), cert_hash).unwrap();
env.attach(att);
let wire = serde_json::to_string(&env).unwrap();
let back: AttestedRefUpdateCert = serde_json::from_str(&wire).unwrap();
assert_eq!(back.attestations.len(), 1);
assert_eq!(back.attestations[0].sig, EXPECTED_SIGNATURE_B64URL);
let recovered = back.attestations[0]
.verify_signature(back.cert_hash().unwrap())
.unwrap();
assert_eq!(recovered.to_bytes(), sk.verifying_key().to_bytes());
// And through a registry, with the convenience method.
let reg = Registry::new();
let verified = back.verify_attestations(&reg).unwrap();
assert_eq!(verified.len(), 1);
assert_eq!(verified[0].cert_hash, EXPECTED_CERT_HASH_HEX);
assert_eq!(verified[0].signer, EXPECTED_SIGNER_DID);
}
@@ -0,0 +1,251 @@
//! End-to-end integration with `gitlawb-core`.
//!
//! Proves the attest envelope composes with the real `RefUpdateCert` from
//! `gitlawb-core`: a cert built and signed through the core API wraps cleanly,
//! survives signing and verifying an attestation, and round-trips back to a
//! `RefUpdateCert` that core still verifies. None of this is exercised by the
//! unit tests in `cert.rs`, which use hand-written JSON literals to keep the
//! crate's runtime deps minimal.
use ed25519_dalek::SigningKey;
use gitlawb_attest::{
Attestation, AttestationPayload, AttestationVerifier, AttestedRefUpdateCert, Error, Policy,
Registry, Result as AttestResult,
};
use gitlawb_core::cert::RefUpdateCert;
use gitlawb_core::identity::Keypair;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
fn zero_hash() -> String {
"0".repeat(64)
}
fn commit_hash(c: char) -> String {
c.to_string().repeat(64)
}
fn fresh_signing_key() -> SigningKey {
SigningKey::generate(&mut OsRng)
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct CovenantExecPayload {
agent: String,
capability: String,
sandbox_digest: String,
}
impl AttestationPayload for CovenantExecPayload {
fn payload_type() -> &'static str {
"covenant/exec/v1"
}
}
struct CovenantExecVerifier;
impl AttestationVerifier for CovenantExecVerifier {
fn type_(&self) -> &'static str {
"covenant/exec/v1"
}
fn verify_payload(&self, payload: &serde_json::Value) -> AttestResult<()> {
for field in ["agent", "capability", "sandbox_digest"] {
payload
.get(field)
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.ok_or_else(|| Error::Payload(format!("missing or empty '{field}'")))?;
}
Ok(())
}
}
#[test]
fn wraps_real_cert_signs_and_verifies_attestation_then_unwraps_cleanly() {
// 1. Build a real cert through gitlawb-core. The pusher is the repo owner
// in this minimal setup; the cert is single-signature.
let pusher = Keypair::generate();
let repo_did = pusher.did();
let cert = RefUpdateCert::new(
repo_did,
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('a'),
1,
&pusher,
)
.unwrap();
cert.validate_structure().unwrap();
let valid_signers = cert.verify_all().unwrap();
assert_eq!(valid_signers.len(), 1);
// 2. Wrap. With no attestations attached the envelope is byte-identical to
// the underlying cert — that's the additive-on-the-wire promise.
let cert_json = serde_json::to_value(&cert).unwrap();
let mut env = AttestedRefUpdateCert::from_cert(&cert).unwrap();
assert_eq!(
serde_json::to_value(&env).unwrap(),
cert_json,
"empty envelope must serialize as the bare cert"
);
// 3. A separate party (here, an agent runtime) attests to how the commit
// was produced. The attestation signer is intentionally not the cert
// pusher: trust in the attestation is evaluated independently of trust
// in the push.
let agent_signer = fresh_signing_key();
let payload = CovenantExecPayload {
agent: "did:key:z6MkAgent".to_string(),
capability: "code.write+sandbox.exec".to_string(),
sandbox_digest: "sha256:c0ffee".to_string(),
};
let cert_hash = env.cert_hash().unwrap();
let attestation = Attestation::sign(&agent_signer, payload.clone(), cert_hash).unwrap();
env.attach(attestation);
// 4. Verify through a registry that knows `covenant/exec/v1`. Strict
// policy so an unknown discriminator would have been rejected.
let mut registry = Registry::new().with_policy(Policy::RejectUnknown);
registry.register(CovenantExecVerifier);
let verified = registry
.verify_all(&env.attestations, env.cert_hash().unwrap())
.unwrap();
assert_eq!(verified.len(), 1);
assert!(verified[0].fully_verified);
assert_eq!(verified[0].type_, "covenant/exec/v1");
assert_eq!(verified[0].cert_hash, hex::encode(cert_hash));
// 5. Round-trip through JSON and back. The wrapped cert with one
// attestation must parse back into the same envelope, and stripping
// the attestations must yield a `RefUpdateCert` core still verifies.
let on_wire = serde_json::to_string(&env).unwrap();
let back: AttestedRefUpdateCert = serde_json::from_str(&on_wire).unwrap();
assert_eq!(back.attestations.len(), 1);
assert_eq!(
back.attestations[0]
.payload_as::<CovenantExecPayload>()
.unwrap(),
payload
);
let mut unwrapped = back.cert.clone();
if let Some(obj) = unwrapped.as_object_mut() {
obj.remove("attestations");
}
let unwrapped_cert: RefUpdateCert = serde_json::from_value(unwrapped).unwrap();
let revalidated = unwrapped_cert.verify_all().unwrap();
assert_eq!(revalidated, valid_signers);
}
#[test]
fn cert_hash_is_stable_when_a_maintainer_countersigns() {
// The cert hash strips `signatures` before hashing, so countersignatures
// added after an attestation is signed must not invalidate the
// attestation's binding.
let pusher = Keypair::generate();
let maintainer = Keypair::generate();
let mut cert = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('b'),
7,
&pusher,
)
.unwrap();
let env_before = AttestedRefUpdateCert::from_cert(&cert).unwrap();
let hash_before = env_before.cert_hash().unwrap();
cert.countersign(&maintainer).unwrap();
let env_after = AttestedRefUpdateCert::from_cert(&cert).unwrap();
let hash_after = env_after.cert_hash().unwrap();
assert_eq!(
hash_before, hash_after,
"cert hash must be stable across countersignatures"
);
// An attestation signed against the pre-countersign hash still verifies
// against the post-countersign cert.
let agent = fresh_signing_key();
let payload = CovenantExecPayload {
agent: "did:key:z6MkAgent2".to_string(),
capability: "code.write".to_string(),
sandbox_digest: "sha256:deadbeef".to_string(),
};
let attestation = Attestation::sign(&agent, payload, hash_before).unwrap();
attestation.verify_signature(hash_after).unwrap();
}
#[test]
fn cross_cert_replay_is_rejected_against_a_real_cert() {
// Negative case: an attestation signed for one cert must not verify on a
// different cert built through the core API.
let pusher = Keypair::generate();
let cert_a = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('a'),
1,
&pusher,
)
.unwrap();
let cert_b = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
commit_hash('a'),
commit_hash('b'),
2,
&pusher,
)
.unwrap();
let env_a = AttestedRefUpdateCert::from_cert(&cert_a).unwrap();
let env_b = AttestedRefUpdateCert::from_cert(&cert_b).unwrap();
let hash_a = env_a.cert_hash().unwrap();
let hash_b = env_b.cert_hash().unwrap();
assert_ne!(hash_a, hash_b);
let agent = fresh_signing_key();
let payload = CovenantExecPayload {
agent: "did:key:z6MkAgent3".to_string(),
capability: "code.write".to_string(),
sandbox_digest: "sha256:cafe".to_string(),
};
let attestation = Attestation::sign(&agent, payload, hash_a).unwrap();
let err = attestation.verify_signature(hash_b).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
/// A cert deserialized from JSON and a cert built through the core API must
/// hash identically. JCS canonicalization is what makes the additive-on-wire
/// promise hold across the serialize/deserialize boundary — without it, the
/// hash would depend on which side built the `serde_json::Value`.
#[test]
fn cert_hash_is_identical_across_the_deserialize_boundary() {
let pusher = Keypair::generate();
let cert = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('c'),
9,
&pusher,
)
.unwrap();
let direct_hash = AttestedRefUpdateCert::from_cert(&cert)
.unwrap()
.cert_hash()
.unwrap();
let wire = serde_json::to_string(&cert).unwrap();
let parsed_value: serde_json::Value = serde_json::from_str(&wire).unwrap();
let parsed_env: AttestedRefUpdateCert = serde_json::from_value(parsed_value).unwrap();
let parsed_hash = parsed_env.cert_hash().unwrap();
assert_eq!(direct_hash, parsed_hash);
}
+5 -1
View File
@@ -1,8 +1,9 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version.workspace = true
version = "0.4.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
@@ -23,6 +24,9 @@ chrono = { workspace = true }
uuid = { workspace = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
crypto_box = { version = "0.9", features = ["std", "chacha20"] }
chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
+117 -14
View File
@@ -10,9 +10,9 @@ use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::{Error, Result};
use crate::did::Did;
use crate::identity::{Keypair, verify};
use crate::identity::{verify, Keypair};
use crate::{Error, Result};
/// The certificate type discriminant. Always `"gitlawb/ref-update/v1"`.
pub const CERT_TYPE: &str = "gitlawb/ref-update/v1";
@@ -110,7 +110,7 @@ impl RefUpdateCert {
///
/// Returns the list of DIDs whose signatures are valid.
pub fn verify_all(&self) -> Result<Vec<Did>> {
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let signing_bytes = self.body.to_signing_bytes()?;
let mut valid = Vec::new();
@@ -118,10 +118,12 @@ impl RefUpdateCert {
// Resolve the verifying key from the DID
let vk = cert_sig.signer.to_verifying_key()?;
let sig_bytes_vec = URL_SAFE_NO_PAD.decode(&cert_sig.sig)
let sig_bytes_vec = URL_SAFE_NO_PAD
.decode(&cert_sig.sig)
.map_err(|e| Error::RefCert(format!("invalid base64 sig: {e}")))?;
let sig_bytes: [u8; 64] = sig_bytes_vec.try_into()
let sig_bytes: [u8; 64] = sig_bytes_vec
.try_into()
.map_err(|_| Error::RefCert("signature must be 64 bytes".to_string()))?;
verify(&vk, &signing_bytes, &sig_bytes)?;
@@ -133,11 +135,7 @@ impl RefUpdateCert {
/// Check if this certificate satisfies a threshold of valid signatures
/// from the provided set of authorized maintainer DIDs.
pub fn satisfies_threshold(
&self,
maintainers: &[Did],
threshold: usize,
) -> Result<bool> {
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
let valid = self.verify_all()?;
let count = valid.iter().filter(|d| maintainers.contains(d)).count();
Ok(count >= threshold)
@@ -187,7 +185,8 @@ mod tests {
dummy_hash('a'),
1,
&kp,
).unwrap();
)
.unwrap();
cert.validate_structure().unwrap();
let valid = cert.verify_all().unwrap();
@@ -208,7 +207,8 @@ mod tests {
dummy_hash('a'),
1,
&kp1,
).unwrap();
)
.unwrap();
cert.countersign(&kp2).unwrap();
let valid = cert.verify_all().unwrap();
@@ -228,7 +228,8 @@ mod tests {
dummy_hash('a'),
1,
&kp1,
).unwrap();
)
.unwrap();
cert.countersign(&kp2).unwrap();
let maintainers = vec![kp1.did(), kp2.did()];
@@ -246,7 +247,8 @@ mod tests {
dummy_hash('b'),
42,
&kp,
).unwrap();
)
.unwrap();
let json = serde_json::to_string_pretty(&cert).unwrap();
assert!(json.contains("gitlawb/ref-update/v1"));
@@ -255,4 +257,105 @@ mod tests {
// Round-trip
let _: RefUpdateCert = serde_json::from_str(&json).unwrap();
}
#[test]
fn validate_structure_rejects_wrong_cert_type() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.body.type_ = "gitlawb/ref-update/v0".to_string();
assert!(cert.validate_structure().is_err());
}
#[test]
fn validate_structure_rejects_invalid_to_hash() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.body.to = "not-a-valid-sha256".to_string();
assert!(cert.validate_structure().is_err());
}
#[test]
fn validate_structure_rejects_empty_signatures() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures.clear();
assert!(cert.validate_structure().is_err());
}
#[test]
fn tampered_signature_fails_verify_all() {
let kp = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
// Replace kp's signature with a signature from kp2 over different bytes.
// The signer DID still points to kp, so verification must fail.
cert.signatures[0].sig = kp2.sign_b64(b"unrelated bytes");
assert!(cert.verify_all().is_err());
}
#[test]
fn threshold_not_met_when_signer_outside_maintainer_list() {
let kp_pusher = Keypair::generate();
let kp_maintainer = Keypair::generate();
let cert = RefUpdateCert::new(
kp_pusher.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp_pusher,
)
.unwrap();
let maintainers = vec![kp_maintainer.did()];
assert!(!cert.satisfies_threshold(&maintainers, 1).unwrap());
}
#[test]
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
assert!(cert.satisfies_threshold(&[], 0).unwrap());
}
}
+65 -5
View File
@@ -11,7 +11,7 @@
use cid::CidGeneric;
use multihash_codetable::{Code, MultihashDigest};
use serde::{Deserialize, Serialize};
use sha2::{Sha256, Digest};
use sha2::{Digest, Sha256};
use std::fmt;
use crate::{Error, Result};
@@ -87,9 +87,9 @@ pub fn sha256_bytes(bytes: &[u8]) -> [u8; 32] {
/// Parse a 64-character hex SHA-256 string into raw bytes.
pub fn sha256_hex_to_bytes(hex_str: &str) -> Result<[u8; 32]> {
let bytes = hex::decode(hex_str)
.map_err(|e| Error::InvalidCid(format!("invalid hex: {e}")))?;
bytes.try_into()
let bytes = hex::decode(hex_str).map_err(|e| Error::InvalidCid(format!("invalid hex: {e}")))?;
bytes
.try_into()
.map_err(|_| Error::InvalidCid("sha256 hash must be 32 bytes (64 hex chars)".to_string()))
}
@@ -110,7 +110,10 @@ mod tests {
// CIDv1 base32 strings start with 'b'
let data = b"blob 13\0hello gitlawb";
let c = Cid::from_git_object_bytes(data);
assert!(c.to_string().starts_with('b'), "CIDv1 should be base32 (starts with 'b')");
assert!(
c.to_string().starts_with('b'),
"CIDv1 should be base32 (starts with 'b')"
);
}
#[test]
@@ -126,4 +129,61 @@ mod tests {
let bytes = sha256_hex_to_bytes(&hex).unwrap();
assert_eq!(sha256_bytes(data), bytes);
}
#[test]
fn from_sha256_bytes_matches_object_bytes() {
// from_sha256_bytes must produce the same CID as from_git_object_bytes
// when given the SHA-256 of those bytes — the two construction paths
// must be equivalent for self-verifying content addressing to hold.
let data = b"blob 5\0hello";
let cid_from_object = Cid::from_git_object_bytes(data);
let hash = sha256_bytes(data);
let cid_from_hash = Cid::from_sha256_bytes(&hash);
assert_eq!(cid_from_object, cid_from_hash);
}
#[test]
fn from_str_parses_valid_cid() {
let data = b"tree content for test";
let cid = Cid::from_git_object_bytes(data);
let parsed = Cid::from_str(cid.as_str()).unwrap();
assert_eq!(cid, parsed);
}
#[test]
fn from_str_rejects_invalid_string() {
let result = Cid::from_str("not-a-valid-cid");
assert!(result.is_err());
}
#[test]
fn different_data_produces_different_cids() {
let c1 = Cid::from_git_object_bytes(b"blob 5\0hello");
let c2 = Cid::from_git_object_bytes(b"blob 5\0world");
assert_ne!(c1, c2);
}
#[test]
fn sha256_hex_to_bytes_rejects_non_hex() {
let non_hex = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz";
assert_eq!(non_hex.len(), 64);
let result = sha256_hex_to_bytes(non_hex);
assert!(result.is_err());
}
#[test]
fn sha256_hex_to_bytes_rejects_wrong_length() {
let result = sha256_hex_to_bytes("deadbeef");
assert!(result.is_err());
}
#[test]
fn sha256_hex_of_empty_input_is_well_known() {
// SHA-256("") is a fixed constant; verifies the hasher is wired correctly.
let h = sha256_hex(b"");
assert_eq!(
h,
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
);
}
}
+67 -7
View File
@@ -75,12 +75,13 @@ impl Did {
pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() {
return Err(Error::InvalidDid(format!(
"expected did:key, got did:{}", self.method()
"expected did:key, got did:{}",
self.method()
)));
}
let (_, bytes) = multibase::decode(self.method_id())
.map_err(|e| Error::InvalidDid(e.to_string()))?;
let (_, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
if !bytes.starts_with(ED25519_MULTICODEC) {
return Err(Error::InvalidDid(
@@ -92,8 +93,7 @@ impl Did {
.try_into()
.map_err(|_| Error::InvalidDid("ed25519 key must be 32 bytes".to_string()))?;
VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::InvalidDid(e.to_string()))
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))
}
/// Return the full DID string as a `&str`.
@@ -105,7 +105,9 @@ impl Did {
pub fn validate(&self) -> Result<()> {
match self.method() {
"key" | "web" | "gitlawb" => Ok(()),
other => Err(Error::InvalidDid(format!("unsupported DID method: {other}"))),
other => Err(Error::InvalidDid(format!(
"unsupported DID method: {other}"
))),
}
}
}
@@ -121,7 +123,9 @@ impl FromStr for Did {
fn from_str(s: &str) -> Result<Self> {
if !s.starts_with("did:") {
return Err(Error::InvalidDid(format!("'{s}' does not start with 'did:'")));
return Err(Error::InvalidDid(format!(
"'{s}' does not start with 'did:'"
)));
}
let did = Self(s.to_string());
did.validate()?;
@@ -248,4 +252,60 @@ mod tests {
let json = serde_json::to_string_pretty(&doc).unwrap();
assert!(json.contains("Ed25519VerificationKey2020"));
}
#[test]
fn to_verifying_key_fails_for_did_web() {
let web = Did::web("example.com");
let result = web.to_verifying_key();
assert!(
result.is_err(),
"did:web cannot resolve to a verifying key locally"
);
}
#[test]
fn to_verifying_key_fails_for_did_gitlawb() {
let gl = Did::gitlawb("z6MkSomeKey");
let result = gl.to_verifying_key();
assert!(
result.is_err(),
"did:gitlawb cannot resolve to a verifying key locally"
);
}
#[test]
fn from_str_rejects_missing_did_prefix() {
let result = "key:z6MkABCD".parse::<Did>();
assert!(result.is_err());
}
#[test]
fn from_str_rejects_unsupported_method() {
let result = "did:ethr:0x1234abcd".parse::<Did>();
assert!(result.is_err());
}
#[test]
fn validate_accepts_all_supported_methods() {
Did::web("example.com").validate().unwrap();
Did::gitlawb("z6MkSomeKey").validate().unwrap();
Keypair::generate().did().validate().unwrap();
}
#[test]
fn is_did_key_and_is_did_gitlawb_predicates() {
let key_did = Keypair::generate().did();
assert!(key_did.is_did_key());
assert!(!key_did.is_did_gitlawb());
let gl = Did::gitlawb("z6MkSomeKey");
assert!(!gl.is_did_key());
assert!(gl.is_did_gitlawb());
}
#[test]
fn method_id_extraction() {
assert_eq!(Did::web("example.com").method_id(), "example.com");
assert_eq!(Did::gitlawb("z6MkSomeKey").method_id(), "z6MkSomeKey");
}
}
+296
View File
@@ -0,0 +1,296 @@
//! Envelope encryption for withheld blobs (Option B). A random content key
//! encrypts the blob (XChaCha20-Poly1305); the content key is wrapped to each
//! recipient via an X25519 box keyed from their Ed25519 `did:key`. The node
//! seals with public keys only; readers open with their own private key.
use crate::identity::Keypair;
use anyhow::{Context, Result};
use ed25519_dalek::VerifyingKey;
use zeroize::Zeroizing;
/// X25519 public key (Montgomery u) for an Ed25519 verifying key.
fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
use curve25519_dalek::edwards::CompressedEdwardsY;
let edwards = CompressedEdwardsY::from_slice(vk.as_bytes())
.ok()
.and_then(|c| c.decompress())
.context("verifying key is not a valid edwards point")?;
Ok(edwards.to_montgomery().to_bytes())
}
/// X25519 secret scalar for an Ed25519 seed (SHA-512 of seed, lower 32, clamped).
/// Returns the scalar wrapped in `Zeroizing`, and scrubs the intermediate
/// SHA-512 digest, so no copy of this secret material lingers in freed memory.
fn x25519_secret_from_seed(seed: &[u8; 32]) -> Zeroizing<[u8; 32]> {
use sha2::{Digest, Sha512};
use zeroize::Zeroize;
let mut h = Sha512::digest(seed);
let mut s = Zeroizing::new([0u8; 32]);
s.copy_from_slice(&h[..32]);
s[0] &= 248;
s[31] &= 127;
s[31] |= 64;
h.as_mut_slice().zeroize();
s
}
use base64::{engine::general_purpose::STANDARD as B64, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit},
XChaCha20Poly1305, XNonce,
};
use crypto_box::{
aead::{AeadCore, OsRng},
ChaChaBox, PublicKey as XPublic, SecretKey as XSecret,
};
use rand::RngCore;
use serde::{Deserialize, Serialize};
const MAGIC: &[u8] = b"GLENC";
const VERSION: u8 = 2;
#[derive(Serialize, Deserialize)]
struct Recipient {
eph: String, // base64 ephemeral x25519 pubkey (32B)
nonce: String, // base64 box nonce (24B)
wrap: String, // base64 wrapped content key
}
#[derive(Serialize, Deserialize)]
struct Header {
alg: String,
nonce: String, // base64 body nonce (24B)
recipients: Vec<Recipient>,
}
/// Encrypt `plaintext` so any of `recipients` (Ed25519 keys) can decrypt.
pub fn seal_blob(plaintext: &[u8], recipients: &[VerifyingKey]) -> Result<Vec<u8>> {
if recipients.is_empty() {
return Err(anyhow::anyhow!("seal_blob: no recipients"));
}
let mut content_key = [0u8; 32];
OsRng.fill_bytes(&mut content_key);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
.map_err(|e| anyhow::anyhow!("content key: {e}"))?;
let mut body_nonce = [0u8; 24];
OsRng.fill_bytes(&mut body_nonce);
let body = body_cipher
.encrypt(XNonce::from_slice(&body_nonce), plaintext)
.map_err(|e| anyhow::anyhow!("body encrypt: {e}"))?;
let mut wrapped = Vec::with_capacity(recipients.len());
for vk in recipients {
let recip_x = XPublic::from(x25519_public(vk)?);
let eph = XSecret::generate(&mut OsRng);
let abox = ChaChaBox::new(&recip_x, &eph);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let ct = abox
.encrypt(&nonce, &content_key[..])
.map_err(|e| anyhow::anyhow!("wrap: {e}"))?;
wrapped.push(Recipient {
eph: B64.encode(eph.public_key().as_bytes()),
nonce: B64.encode(nonce),
wrap: B64.encode(ct),
});
}
let header = Header {
alg: "xchacha20poly1305".into(),
nonce: B64.encode(body_nonce),
recipients: wrapped,
};
let header_json = serde_json::to_vec(&header).context("encode header")?;
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
out.extend_from_slice(&header_json);
out.extend_from_slice(&body);
Ok(out)
}
/// Decrypt an envelope with `keypair`. Errors if not a recipient or on auth fail.
pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
let mut p = 0;
if envelope.len() < MAGIC.len() + 1 + 4 || &envelope[..MAGIC.len()] != MAGIC {
return Err(anyhow::anyhow!("bad envelope magic"));
}
p += MAGIC.len();
if envelope[p] != VERSION {
return Err(anyhow::anyhow!("unsupported envelope version"));
}
p += 1;
let hlen = u32::from_le_bytes(envelope[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header: Header =
serde_json::from_slice(envelope.get(p..p + hlen).context("truncated header")?)
.context("decode header")?;
let body = &envelope[p + hlen..];
let my_x = XSecret::from(*x25519_secret_from_seed(&keypair.to_seed()));
// Identities are blinded: no entry says which recipient it belongs to, so
// try each one. The ChaChaBox AEAD tag authenticates, so exactly the
// reader's own entry unwraps; every other entry fails cleanly.
let mut content_key: Option<Vec<u8>> = None;
for entry in &header.recipients {
let eph = match B64
.decode(&entry.eph)
.ok()
.and_then(|b| <[u8; 32]>::try_from(b.as_slice()).ok())
{
Some(b) => XPublic::from(b),
None => continue,
};
// from_slice panics on a wrong length, and the envelope is attacker
// controlled, so validate the 24-byte box nonce before using it.
let nonce = match B64
.decode(&entry.nonce)
.ok()
.and_then(|n| <[u8; 24]>::try_from(n.as_slice()).ok())
{
Some(n) => n,
None => continue,
};
let wrap = match B64.decode(&entry.wrap) {
Ok(w) => w,
Err(_) => continue,
};
let abox = ChaChaBox::new(&eph, &my_x);
if let Ok(ck) = abox.decrypt(
crypto_box::aead::generic_array::GenericArray::from_slice(&nonce),
wrap.as_slice(),
) {
content_key = Some(ck);
break;
}
}
let content_key = content_key.context("not a recipient of this envelope")?;
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
.map_err(|e| anyhow::anyhow!("content key: {e}"))?;
let body_nonce = B64
.decode(&header.nonce)
.ok()
.and_then(|n| <[u8; 24]>::try_from(n.as_slice()).ok())
.context("invalid body nonce")?;
body_cipher
.decrypt(XNonce::from_slice(&body_nonce), body)
.map_err(|_| anyhow::anyhow!("body decrypt failed"))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::identity::Keypair;
#[test]
fn ed25519_to_x25519_keypair_agrees() {
// The X25519 public derived from the Ed25519 public must equal the
// X25519 public of the X25519 secret derived from the same seed.
let kp = Keypair::generate();
let seed = kp.to_seed();
let xpub_from_public = x25519_public(&kp.verifying_key()).unwrap();
let xsec = x25519_secret_from_seed(&seed);
let xpub_from_secret = crypto_box::SecretKey::from(*xsec).public_key().to_bytes();
assert_eq!(xpub_from_public, xpub_from_secret);
}
#[test]
fn seal_open_round_trip_for_recipients() {
let owner = Keypair::generate();
let reader_a = Keypair::generate();
let reader_b = Keypair::generate();
let msg = b"private blob contents";
let env = seal_blob(msg, &[owner.verifying_key(), reader_a.verifying_key()]).unwrap();
assert_eq!(open_blob(&env, &owner).unwrap(), msg);
assert_eq!(open_blob(&env, &reader_a).unwrap(), msg);
assert!(
open_blob(&env, &reader_b).is_err(),
"non-recipient must fail"
);
}
#[test]
fn tampered_envelope_fails() {
let owner = Keypair::generate();
let mut env = seal_blob(b"hi", &[owner.verifying_key()]).unwrap();
let last = env.len() - 1;
env[last] ^= 0x01;
assert!(open_blob(&env, &owner).is_err());
}
#[test]
fn v2_header_contains_no_recipient_pubkey() {
// The blinded envelope header must not carry any recipient's public key.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Slice out the header bytes using the envelope framing:
// MAGIC | version(1B) | header_len(4B LE) | header_json | body
let mut p = MAGIC.len() + 1; // skip MAGIC + version byte
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header = &env[p..p + hlen];
let header_str = String::from_utf8_lossy(header);
let pubkey_b64 = B64.encode(reader.verifying_key().as_bytes());
assert!(
!header_str.contains(&pubkey_b64),
"recipient public key must not appear in the blinded header"
);
}
#[test]
fn v1_envelope_is_rejected() {
let reader = Keypair::generate();
let mut env = seal_blob(b"hi", &[reader.verifying_key()]).unwrap();
// Flip the version byte (immediately after MAGIC) from 2 to 1.
env[MAGIC.len()] = 1;
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("unsupported envelope version"),
"expected version-rejection error, got: {err}"
);
}
#[test]
fn malformed_nonce_returns_err_not_panic() {
// from_slice panics on wrong-length input; a crafted envelope on the
// public recovery path must surface an error, never panic.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Split the envelope framing into header JSON and body.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
let reframe = |header: &serde_json::Value| -> Vec<u8> {
let hj = serde_json::to_vec(header).unwrap();
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(hj.len() as u32).to_le_bytes());
out.extend_from_slice(&hj);
out.extend_from_slice(body);
out
};
let bad_nonce = serde_json::Value::String(B64.encode([0u8; 12]));
// Corrupted per-recipient nonce: entry is skipped, no match.
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
header["recipients"][0]["nonce"] = bad_nonce.clone();
assert!(open_blob(&reframe(&header), &reader).is_err());
// Corrupted body nonce: unwrap succeeds, body nonce is rejected.
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
header["nonce"] = bad_nonce;
assert!(open_blob(&reframe(&header), &reader).is_err());
}
}
+35 -26
View File
@@ -11,14 +11,14 @@
//! Signature-Input: sig1=("@method" "@path" "content-digest");keyid="did:key:z6Mk...";alg="ed25519";created=<unix>
//! Signature: sig1=:base64signature:
use base64::{Engine, engine::general_purpose::STANDARD};
use base64::{engine::general_purpose::STANDARD, Engine};
use chrono::Utc;
use sha2::{Sha256, Digest};
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use crate::{Error, Result};
use crate::did::Did;
use crate::identity::Keypair;
use crate::{Error, Result};
/// The component identifiers covered by every gitlawb signature.
pub const COVERED_COMPONENTS: &[&str] = &["@method", "@path", "content-digest"];
@@ -53,16 +53,16 @@ impl HttpSignature {
let sig_input = sig_input.trim();
// Expect: sig1=("@method" "@path" "content-digest");keyid="...";alg="...";created=...
let rest = sig_input
.strip_prefix("sig1=")
.ok_or_else(|| Error::HttpSignature("Signature-Input must start with 'sig1='".into()))?;
let rest = sig_input.strip_prefix("sig1=").ok_or_else(|| {
Error::HttpSignature("Signature-Input must start with 'sig1='".into())
})?;
let open = rest.find('(').ok_or_else(|| {
Error::HttpSignature("missing '(' in Signature-Input".into())
})?;
let close = rest.find(')').ok_or_else(|| {
Error::HttpSignature("missing ')' in Signature-Input".into())
})?;
let open = rest
.find('(')
.ok_or_else(|| Error::HttpSignature("missing '(' in Signature-Input".into()))?;
let close = rest
.find(')')
.ok_or_else(|| Error::HttpSignature("missing ')' in Signature-Input".into()))?;
let components_str = &rest[open + 1..close];
let params_str = &rest[close + 1..]; // starts with ';'
@@ -98,15 +98,19 @@ impl HttpSignature {
.trim()
.strip_prefix("sig1=:")
.and_then(|s| s.strip_suffix(':'))
.ok_or_else(|| {
Error::HttpSignature("Signature must be 'sig1=:base64:'".into())
})?;
.ok_or_else(|| Error::HttpSignature("Signature must be 'sig1=:base64:'".into()))?;
let signature_bytes = STANDARD
.decode(sig_b64)
.map_err(|e| Error::HttpSignature(format!("invalid base64 in Signature: {e}")))?;
Ok(Self { key_id, alg, created, components, signature_bytes })
Ok(Self {
key_id,
alg,
created,
components,
signature_bytes,
})
}
/// Reject if the `created` timestamp is more than 5 minutes from now.
@@ -125,7 +129,7 @@ impl HttpSignature {
pub fn missing_components(&self) -> Vec<&str> {
COVERED_COMPONENTS
.iter()
.filter(|c| !self.components.contains(&c.to_string()))
.filter(|c| !self.components.iter().any(|s| s.as_str() == **c))
.copied()
.collect()
}
@@ -178,12 +182,9 @@ pub fn sign_request(
request_values.insert("@path".to_string(), path_and_query.to_string());
request_values.insert("content-digest".to_string(), content_digest.clone());
let signing_string = build_signing_string(
COVERED_COMPONENTS,
sig_params_value,
&request_values,
)
.expect("required components always present when building");
let signing_string =
build_signing_string(COVERED_COMPONENTS, sig_params_value, &request_values)
.expect("required components always present when building");
let sig_bytes = keypair.sign(signing_string.as_bytes());
let sig_b64 = STANDARD.encode(sig_bytes.to_bytes());
@@ -290,10 +291,16 @@ mod tests {
request_values.insert("content-digest".to_string(), headers.content_digest.clone());
let components_ref: Vec<&str> = sig.components.iter().map(String::as_str).collect();
let signing_string = build_signing_string(&components_ref, sig_params_value, &request_values).unwrap();
let signing_string =
build_signing_string(&components_ref, sig_params_value, &request_values).unwrap();
let vk = sig.key_id.to_verifying_key().unwrap();
let sig_b64 = headers.signature.strip_prefix("sig1=:").unwrap().strip_suffix(':').unwrap();
let sig_b64 = headers
.signature
.strip_prefix("sig1=:")
.unwrap()
.strip_suffix(':')
.unwrap();
let sig_bytes: [u8; 64] = STANDARD.decode(sig_b64).unwrap().try_into().unwrap();
assert!(verify(&vk, signing_string.as_bytes(), &sig_bytes).is_ok());
@@ -321,7 +328,9 @@ mod tests {
let kp = Keypair::generate();
let did = kp.did();
// created=1 is way in the past — should fail clock skew check
let sig_input = format!(r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created=1"#);
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created=1"#
);
let sig = HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap();
assert!(sig.check_created().is_err());
}
+92 -15
View File
@@ -3,13 +3,13 @@
//! A gitlawb identity is an Ed25519 keypair. The public key encodes into a
//! `did:key` DID. The private key is stored as PKCS#8 PEM on disk.
use ed25519_dalek::{SigningKey, VerifyingKey, Signer, Signature};
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
use zeroize::Zeroizing;
use crate::{Error, Result};
use crate::did::Did;
use crate::{Error, Result};
/// An Ed25519 keypair that is the root identity for a gitlawb actor.
#[derive(Clone)]
@@ -47,12 +47,14 @@ impl Keypair {
/// Sign and return base64url-encoded signature string.
pub fn sign_b64(&self, msg: &[u8]) -> String {
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig = self.sign(msg);
URL_SAFE_NO_PAD.encode(sig.to_bytes())
}
/// Export the signing key as raw 32-byte seed (wrapped in Zeroizing).
/// The raw 32-byte Ed25519 seed, wrapped in `Zeroizing` so the copy is
/// scrubbed on drop. This is the only seed accessor; it is used to derive
/// the X25519 secret for envelope decryption (see `crate::encrypt`).
pub fn to_seed(&self) -> Zeroizing<[u8; 32]> {
Zeroizing::new(self.signing_key.to_bytes())
}
@@ -69,18 +71,13 @@ impl Keypair {
/// Load from PEM-encoded PKCS#8 private key string.
pub fn from_pem(pem: &str) -> Result<Self> {
use pkcs8::DecodePrivateKey;
let signing_key = SigningKey::from_pkcs8_pem(pem)
.map_err(|e| Error::Key(e.to_string()))?;
let signing_key = SigningKey::from_pkcs8_pem(pem).map_err(|e| Error::Key(e.to_string()))?;
Ok(Self { signing_key })
}
}
/// Verify an Ed25519 signature.
pub fn verify(
verifying_key: &VerifyingKey,
msg: &[u8],
sig_bytes: &[u8; 64],
) -> Result<()> {
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
use ed25519_dalek::Verifier;
let sig = Signature::from_bytes(sig_bytes);
verifying_key
@@ -117,8 +114,8 @@ impl<T: Serialize> Signed<T> {
}
mod sig_b64 {
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use serde::{Deserializer, Serializer, Deserialize};
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
use serde::{Deserialize, Deserializer, Serializer};
pub fn serialize<S: Serializer>(bytes: &[u8; 64], s: S) -> Result<S::Ok, S::Error> {
s.serialize_str(&URL_SAFE_NO_PAD.encode(bytes))
@@ -126,8 +123,12 @@ mod sig_b64 {
pub fn deserialize<'de, D: Deserializer<'de>>(d: D) -> Result<[u8; 64], D::Error> {
let s = String::deserialize(d)?;
let bytes = URL_SAFE_NO_PAD.decode(&s).map_err(serde::de::Error::custom)?;
bytes.try_into().map_err(|_| serde::de::Error::custom("expected 64 bytes"))
let bytes = URL_SAFE_NO_PAD
.decode(&s)
.map_err(serde::de::Error::custom)?;
bytes
.try_into()
.map_err(|_| serde::de::Error::custom("expected 64 bytes"))
}
}
@@ -158,4 +159,80 @@ mod tests {
let did = kp.did();
assert!(did.to_string().starts_with("did:key:z6Mk"));
}
#[test]
fn from_seed_round_trip() {
let kp = Keypair::generate();
let seed = kp.to_seed();
let kp2 = Keypair::from_seed(&seed).unwrap();
assert_eq!(kp.verifying_key(), kp2.verifying_key());
}
// `to_seed()` is the only seed accessor on `Keypair` (the raw, unzeroized
// `seed_bytes()` was removed in #41). Pin its return type so a bare-array
// accessor can't slip back in, and confirm it is deterministic. The
// from_seed -> verifying_key round-trip is already covered by
// `from_seed_round_trip`.
#[test]
fn to_seed_returns_zeroizing_and_is_deterministic() {
let kp = Keypair::generate();
// The type annotation is the regression guard: the seed must be handed
// out wrapped in `Zeroizing`, never as a bare `[u8; 32]`.
let seed: Zeroizing<[u8; 32]> = kp.to_seed();
assert_eq!(*kp.to_seed(), *seed); // stable across calls
}
#[test]
fn sign_b64_decodes_to_valid_signature() {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let kp = Keypair::generate();
let msg = b"test payload for b64 signing";
let b64 = kp.sign_b64(msg);
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&b64)
.expect("sign_b64 must produce valid base64url")
.try_into()
.expect("Ed25519 signature must be 64 bytes");
verify(&kp.verifying_key(), msg, &sig_bytes).unwrap();
}
#[test]
fn verify_rejects_wrong_key() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let msg = b"some message";
let sig = kp1.sign(msg);
let result = verify(&kp2.verifying_key(), msg, &sig.to_bytes());
assert!(
result.is_err(),
"signature from kp1 must not verify under kp2"
);
}
#[test]
fn signed_payload_round_trip() {
let kp = Keypair::generate();
let payload = serde_json::json!({"action": "push", "repo": "my-repo"});
let signed = Signed::new(payload, &kp).unwrap();
signed.verify(&kp.verifying_key()).unwrap();
}
#[test]
fn signed_payload_wrong_key_fails() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let payload = serde_json::json!({"action": "push"});
let signed = Signed::new(payload, &kp1).unwrap();
let result = signed.verify(&kp2.verifying_key());
assert!(
result.is_err(),
"Signed payload must not verify under a different key"
);
}
#[test]
fn from_pem_invalid_input_fails() {
let result = Keypair::from_pem("this is not valid PEM at all");
assert!(result.is_err());
}
}
+6 -5
View File
@@ -1,10 +1,11 @@
pub mod did;
pub mod cid;
pub mod identity;
pub mod http_sig;
pub mod ucan;
pub mod cert;
pub mod cid;
pub mod did;
pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
pub mod ucan;
pub use error::Error;
pub type Result<T> = std::result::Result<T, Error>;
+247 -23
View File
@@ -12,9 +12,9 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use crate::{Error, Result};
use crate::did::Did;
use crate::identity::Keypair;
use crate::{Error, Result};
/// A UCAN capability: what resource the token grants access to.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
@@ -30,13 +30,31 @@ pub struct Capability {
impl Capability {
pub fn new(with: impl Into<String>, can: impl Into<String>) -> Self {
Self { with: with.into(), can: can.into(), constraints: None }
Self {
with: with.into(),
can: can.into(),
constraints: None,
}
}
pub fn with_constraints(mut self, constraints: serde_json::Value) -> Self {
self.constraints = Some(constraints);
self
}
/// Returns `true` if `self` is a valid attenuation of `parent`.
///
/// A delegated capability is only valid if it is at most as permissive as
/// the parent capability backing it. `"*"` on the **parent**'s resource or
/// action field and `repo/admin` in the parent's action position act as
/// wildcards that cover any delegated value; wildcards on `self` carry no
/// special meaning.
pub fn is_attenuated_by(&self, parent: &Capability) -> bool {
let resource_ok = parent.with == self.with || parent.with == "*";
let action_ok =
parent.can == self.can || parent.can == "*" || parent.can == caps::REPO_ADMIN;
resource_ok && action_ok
}
}
/// Well-known gitlawb capability strings.
@@ -128,18 +146,40 @@ impl Ucan {
}
}
/// Check if this UCAN's not-before time is in the future (token not yet valid).
pub fn is_before_valid(&self) -> bool {
if let Some(nbf) = self.payload.nbf {
Utc::now().timestamp() < nbf
} else {
false
}
}
/// Verify this UCAN's audience matches `expected`.
pub fn verify_audience(&self, expected: &Did) -> Result<()> {
if &self.payload.aud != expected {
return Err(Error::Ucan(format!(
"audience mismatch: expected {expected}, got {}",
self.payload.aud
)));
}
Ok(())
}
/// Verify the signature on this UCAN.
pub fn verify_signature(&self) -> Result<()> {
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use crate::identity::verify;
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let vk = self.payload.iss.to_verifying_key()?;
let signing_bytes = serde_json::to_vec(&self.payload)?;
let sig_bytes_vec = URL_SAFE_NO_PAD.decode(&self.s)
let sig_bytes_vec = URL_SAFE_NO_PAD
.decode(&self.s)
.map_err(|e| Error::Ucan(format!("invalid base64 signature: {e}")))?;
let sig_bytes: [u8; 64] = sig_bytes_vec.try_into()
let sig_bytes: [u8; 64] = sig_bytes_vec
.try_into()
.map_err(|_| Error::Ucan("signature must be 64 bytes".to_string()))?;
verify(&vk, &signing_bytes, &sig_bytes)
@@ -148,9 +188,10 @@ impl Ucan {
/// Check if this UCAN grants a specific capability on a resource.
pub fn can(&self, resource: &str, action: &str) -> bool {
self.payload.att.iter().any(|cap| {
cap.with == resource && cap.can == action
})
self.payload
.att
.iter()
.any(|cap| cap.with == resource && cap.can == action)
}
/// Encode to a compact JSON string (the wire format).
@@ -210,6 +251,10 @@ impl Ucan {
return Err(Error::Ucan("token is expired".to_string()));
}
if self.is_before_valid() {
return Err(Error::Ucan("token is not yet valid".to_string()));
}
for proof_token in &self.payload.prf {
let proof = Self::decode(proof_token)
.map_err(|e| Error::Ucan(format!("failed to decode proof: {e}")))?;
@@ -222,6 +267,17 @@ impl Ucan {
)));
}
// Every delegated capability must be covered by the proof (attenuation).
for cap in &self.payload.att {
let covered = proof.payload.att.iter().any(|p| cap.is_attenuated_by(p));
if !covered {
return Err(Error::Ucan(format!(
"capability attenuation violated: '{}' on '{}' not covered by proof",
cap.can, cap.with
)));
}
}
// Verify the proof's signature and chain recursively
proof.verify_chain()?;
}
@@ -245,7 +301,8 @@ mod tests {
audience.clone(),
vec![Capability::new("gitlawb://repos/test/repo", caps::GIT_PUSH)],
None,
).unwrap();
)
.unwrap();
ucan.verify_signature().unwrap();
assert!(!ucan.is_expired());
@@ -291,10 +348,12 @@ mod tests {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer, audience,
&issuer,
audience,
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
None,
).unwrap();
)
.unwrap();
// Root UCAN (no proofs) should verify fine
ucan.verify_chain().unwrap();
}
@@ -307,18 +366,22 @@ mod tests {
// Alice grants Bob push access
let root = Ucan::issue(
&alice, bob.did(),
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
None,
).unwrap();
)
.unwrap();
// Bob delegates to Charlie (with proof from Alice)
let delegated = Ucan::delegate(
&bob, charlie.did(),
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
None,
&root,
).unwrap();
)
.unwrap();
// Chain should verify: Charlie's token → Bob's proof → Alice signed it
delegated.verify_chain().unwrap();
@@ -334,18 +397,22 @@ mod tests {
// Alice grants Bob access
let root = Ucan::issue(
&alice, bob.did(),
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
None,
).unwrap();
)
.unwrap();
// Eve (NOT Bob) tries to delegate using Alice's proof
let bad = Ucan::delegate(
&eve, charlie.did(),
&eve,
charlie.did(),
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
None,
&root,
).unwrap();
)
.unwrap();
// Should fail: proof audience (Bob) != UCAN issuer (Eve)
let err = bad.verify_chain().unwrap_err();
@@ -361,20 +428,177 @@ mod tests {
// Alice grants Bob access with expiry in the past
let exp = chrono::Utc::now() - chrono::Duration::hours(1);
let root = Ucan::issue(
&alice, bob.did(),
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
Some(exp),
).unwrap();
)
.unwrap();
let delegated = Ucan::delegate(
&bob, charlie.did(),
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
None,
&root,
).unwrap();
)
.unwrap();
// Should fail: the proof is expired
let err = delegated.verify_chain().unwrap_err();
assert!(err.to_string().contains("expired"));
}
#[test]
fn is_before_valid_future_nbf() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let nbf_future = chrono::Utc::now() + chrono::Duration::hours(1);
let payload = UcanPayload {
ucan: "1.0.0".to_string(),
iss: issuer.did(),
aud: audience,
att: vec![],
exp: None,
nbf: Some(nbf_future.timestamp()),
prf: vec![],
};
let signing_bytes = serde_json::to_vec(&payload).unwrap();
let sig = issuer.sign_b64(&signing_bytes);
let ucan = Ucan { payload, s: sig };
assert!(ucan.is_before_valid());
let err = ucan.verify_chain().unwrap_err();
assert!(err.to_string().contains("not yet valid"));
}
#[test]
fn is_before_valid_past_nbf() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let nbf_past = chrono::Utc::now() - chrono::Duration::hours(1);
let payload = UcanPayload {
ucan: "1.0.0".to_string(),
iss: issuer.did(),
aud: audience,
att: vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
exp: None,
nbf: Some(nbf_past.timestamp()),
prf: vec![],
};
let signing_bytes = serde_json::to_vec(&payload).unwrap();
let sig = issuer.sign_b64(&signing_bytes);
let ucan = Ucan { payload, s: sig };
assert!(!ucan.is_before_valid());
ucan.verify_chain().unwrap();
}
#[test]
fn verify_audience_matches() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(&issuer, audience.clone(), vec![], None).unwrap();
ucan.verify_audience(&audience).unwrap();
}
#[test]
fn verify_audience_mismatch() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let wrong = Keypair::generate().did();
let ucan = Ucan::issue(&issuer, audience, vec![], None).unwrap();
let err = ucan.verify_audience(&wrong).unwrap_err();
assert!(err.to_string().contains("audience mismatch"));
}
#[test]
fn attenuation_valid_subset() {
let alice = Keypair::generate();
let bob = Keypair::generate();
let charlie = Keypair::generate();
// Alice grants Bob push on a specific repo
let root = Ucan::issue(
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
)
.unwrap();
// Bob delegates the same capability (exact subset) to Charlie
let delegated = Ucan::delegate(
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
&root,
)
.unwrap();
delegated.verify_chain().unwrap();
}
#[test]
fn attenuation_exceeds_parent_is_rejected() {
let alice = Keypair::generate();
let bob = Keypair::generate();
let charlie = Keypair::generate();
// Alice grants Bob push on one repo only
let root = Ucan::issue(
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
)
.unwrap();
// Bob tries to delegate merge (not in the original grant) to Charlie
let delegated = Ucan::delegate(
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::PR_MERGE)],
None,
&root,
)
.unwrap();
let err = delegated.verify_chain().unwrap_err();
assert!(err.to_string().contains("attenuation violated"));
}
#[test]
fn attenuation_repo_admin_covers_all() {
let alice = Keypair::generate();
let bob = Keypair::generate();
let charlie = Keypair::generate();
// Alice grants Bob repo/admin (superpower)
let root = Ucan::issue(
&alice,
bob.did(),
vec![Capability::new(
"gitlawb://repos/org/repo",
caps::REPO_ADMIN,
)],
None,
)
.unwrap();
// Bob delegates a more specific capability — covered by repo/admin
let delegated = Ucan::delegate(
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
&root,
)
.unwrap();
delegated.verify_chain().unwrap();
}
}
+23 -4
View File
@@ -1,8 +1,9 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version.workspace = true
version = "0.4.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
@@ -11,6 +12,8 @@ path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
ed25519-dalek = { workspace = true }
base64 = { workspace = true }
tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
@@ -26,9 +29,10 @@ async-graphql-axum = "7"
tokio-stream = { version = "0.1", features = ["sync"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["cors", "trace", "request-id", "limit"] }
sqlx = { version = "0.8", features = ["postgres", "runtime-tokio-rustls", "chrono", "uuid"] }
sqlx = { version = "0.8", features = ["postgres", "runtime-tokio-rustls", "chrono", "uuid", "macros", "migrate"] }
clap = { version = "4", features = ["derive", "env"] }
bytes = "1"
libc = "0.2"
cid = { workspace = true }
hex = { workspace = true }
sha2 = { workspace = true }
@@ -36,14 +40,28 @@ hmac = { workspace = true }
http-body-util = "0.1"
tokio-util = { version = "0.7", features = ["io"] }
dirs-next = "2"
# NFC-normalize visibility globs/paths at the matcher seam so a deny rule
# withholds a canonically-equivalent path regardless of NFC/NFD form (#101).
unicode-normalization = "0.1"
reqwest = { workspace = true }
libp2p = { version = "0.55", features = ["gossipsub", "kad", "mdns", "noise", "tcp", "tokio", "yamux", "identify", "macros"] }
libp2p-core = "0.43"
libp2p-gossipsub = "0.49.4"
libp2p-identify = "0.47"
libp2p-identity = { version = "0.2", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.48"
libp2p-quic = { version = "0.13", features = ["tokio"] }
libp2p-swarm = { version = "0.47", features = ["macros", "tokio"] }
multiaddr = "0.18"
futures = "0.3"
aws-sdk-s3 = "1"
aws-sdk-s3 = { version = "1", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
aws-config = { version = "1", features = ["behavior-version-latest"] }
async-compression = { version = "0.4", features = ["tokio", "zstd"] }
tar = "0.4"
zstd = "0.13"
# Prometheus metrics. Used to expose a /metrics endpoint for ops/observability
# on the opt-in GITLAWB_METRICS_ADDR listener. The crate is also the de-facto
# exposition format encoder in the Rust ecosystem.
prometheus = { version = "0.13", default-features = false }
alloy = { version = "1", default-features = false, features = [
"contract",
"provider-http",
@@ -54,6 +72,7 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
libp2p-dns = { version = "0.44.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
+130
View File
@@ -0,0 +1,130 @@
//! Admin subcommands (one-off maintenance), invoked as `gitlawb-node <cmd>`.
//! These run instead of the daemon and exit.
use anyhow::Result;
use crate::config::{Config, PurgeSpamArgs};
use crate::db::Db;
use crate::git::tigris::TigrisClient;
/// How many repos to delete per query/commit batch.
const BATCH: i64 = 500;
/// Bulk-delete repos whose name matches a Postgres regex, removing DB rows,
/// the on-disk bare repo, and (unless skipped) the Tigris archive.
///
/// Defaults to a dry run: it reports the match count and a sample and deletes
/// nothing. Pass `--execute` to actually delete.
pub async fn purge_spam(config: &Config, args: &PurgeSpamArgs) -> Result<()> {
let db = Db::connect(&config.database_url).await?;
let total = db.count_repos_by_name_regex(&args.regex).await?;
println!("repos matching /{}/ : {total}", args.regex);
let sample = db.list_repos_by_name_regex(&args.regex, 20).await?;
println!("sample (up to 20):");
for (id, name, owner, _disk) in &sample {
println!(" {name} owner={owner} id={id}");
}
if !args.execute {
println!(
"\nDRY RUN — nothing deleted. Re-run with --execute to delete these {total} repos \
({}).",
if args.bulk {
"bulk set-based DB delete + on-disk .git"
} else if args.skip_tigris {
"DB rows + on-disk .git"
} else {
"DB rows + on-disk .git + Tigris archive"
}
);
return Ok(());
}
// Bulk mode: set-based DB delete (a few statements) + on-disk removal. Far
// faster on large match sets / small DB instances than per-repo cascade.
if args.bulk {
let disk_paths = db.list_disk_paths_by_name_regex(&args.regex).await?;
println!(
"collected {} on-disk paths; running bulk DB delete…",
disk_paths.len()
);
let deleted = db.bulk_delete_by_name_regex(&args.regex).await?;
println!("bulk DB delete done: {deleted} repos removed; removing on-disk repos…");
let mut disk_removed = 0u64;
for p in &disk_paths {
if std::path::Path::new(p).exists() {
match std::fs::remove_dir_all(p) {
Ok(()) => disk_removed += 1,
Err(e) => eprintln!("warning: rm {p} failed: {e}"),
}
}
}
println!("done: {deleted} repos purged (bulk), {disk_removed} on-disk dirs removed");
return Ok(());
}
// Tigris client (best-effort; archive deletion failures are logged, not fatal).
let tigris = if !args.skip_tigris && !config.tigris_bucket.is_empty() {
match TigrisClient::new(&config.tigris_bucket).await {
Ok(t) => Some(t),
Err(e) => {
eprintln!("warning: could not init Tigris ({e}); skipping archive deletion");
None
}
}
} else {
None
};
let mut deleted = 0i64;
let mut disk_removed = 0i64;
let mut tigris_removed = 0i64;
loop {
let take = match args.limit {
n if n > 0 => (n - deleted).min(BATCH),
_ => BATCH,
};
if take <= 0 {
break;
}
let batch = db.list_repos_by_name_regex(&args.regex, take).await?;
if batch.is_empty() {
break;
}
for (id, name, owner_did, disk_path) in &batch {
let short = owner_did.rsplit(':').next().unwrap_or(owner_did);
let slug = format!("{short}/{name}");
// DB first: once the row is gone the repo is invisible to all serve/
// list paths even if disk/Tigris cleanup lags.
db.delete_repo_cascade(id, &slug).await?;
deleted += 1;
if !disk_path.is_empty() && std::path::Path::new(disk_path).exists() {
match std::fs::remove_dir_all(disk_path) {
Ok(()) => disk_removed += 1,
Err(e) => eprintln!("warning: rm {disk_path} failed: {e}"),
}
}
if let Some(t) = &tigris {
let owner_slug = owner_did.replace([':', '/'], "_");
match t.delete(&owner_slug, name).await {
Ok(()) => tigris_removed += 1,
Err(e) => eprintln!("warning: tigris delete {owner_slug}/{name} failed: {e}"),
}
}
}
println!("… deleted {deleted}/{total} (disk {disk_removed}, tigris {tigris_removed})");
}
println!(
"done: {deleted} repos purged (disk dirs removed: {disk_removed}, tigris archives removed: {tigris_removed})"
);
Ok(())
}
+186 -20
View File
@@ -2,12 +2,52 @@
use axum::extract::{Path, Query, State};
use axum::http::StatusCode;
use axum::Json;
use axum::{Extension, Json};
use serde::{Deserialize, Serialize};
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
fn normalize_agent_did(did: &str) -> String {
if did.starts_with("did:") {
did.to_string()
} else {
format!("did:key:{did}")
}
}
fn agent_key_segment(did: &str) -> &str {
did.split(':').next_back().unwrap_or(did)
}
/// Whether `caller` (the verified `AuthenticatedDid`) is the same identity as
/// `target` (the DID being acted on), tolerant of full `did:key:...` vs short
/// key-only forms on either side. Used to gate self-deregistration so a DID
/// can only retire itself.
fn caller_matches_did(caller: &str, target: &str) -> bool {
agent_key_segment(caller) == agent_key_segment(target)
}
async fn resolve_agent_did(state: &AppState, did: &str) -> Result<String> {
let normalized_did = normalize_agent_did(did);
if state.db.get_agent(&normalized_did).await?.is_some() {
return Ok(normalized_did);
}
let requested_key = agent_key_segment(&normalized_did);
let matching_agent = state
.db
.list_agents(None)
.await?
.into_iter()
.find(|agent| agent_key_segment(&agent.did).starts_with(requested_key));
Ok(matching_agent
.map(|agent| agent.did)
.unwrap_or(normalized_did))
}
#[derive(Debug, Serialize)]
pub struct TrustResponse {
pub did: String,
@@ -40,6 +80,8 @@ pub struct AgentResponse {
pub capabilities: Vec<String>,
pub registered_at: String,
pub last_seen: Option<String>,
/// Lifecycle status: `active` or `revoked`.
pub status: String,
}
/// GET /api/v1/agents
@@ -48,13 +90,17 @@ pub async fn list_agents(
Query(params): Query<AgentListQuery>,
) -> Result<Json<serde_json::Value>> {
let agents = state.db.list_agents(params.capability.as_deref()).await?;
let list: Vec<AgentResponse> = agents.into_iter().map(|a| AgentResponse {
did: a.did,
trust_score: a.trust_score,
capabilities: a.capabilities,
registered_at: a.registered_at,
last_seen: a.last_seen,
}).collect();
let list: Vec<AgentResponse> = agents
.into_iter()
.map(|a| AgentResponse {
did: a.did,
trust_score: a.trust_score,
capabilities: a.capabilities,
registered_at: a.registered_at,
last_seen: a.last_seen,
status: a.status,
})
.collect();
Ok(Json(serde_json::json!({ "agents": list })))
}
@@ -63,15 +109,23 @@ pub async fn show_agent(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<(StatusCode, Json<AgentResponse>)> {
let agent = state.db.get_agent(&did).await?
.ok_or_else(|| AppError::NotFound(format!("agent {did} not found")))?;
Ok((StatusCode::OK, Json(AgentResponse {
did: agent.did,
trust_score: agent.trust_score,
capabilities: agent.capabilities,
registered_at: agent.registered_at,
last_seen: agent.last_seen,
})))
let normalized_did = resolve_agent_did(&state, &did).await?;
let agent = state
.db
.get_agent(&normalized_did)
.await?
.ok_or_else(|| AppError::NotFound(format!("agent {normalized_did} not found")))?;
Ok((
StatusCode::OK,
Json(AgentResponse {
did: agent.did,
trust_score: agent.trust_score,
capabilities: agent.capabilities,
registered_at: agent.registered_at,
last_seen: agent.last_seen,
status: agent.status,
}),
))
}
/// GET /api/v1/agents/{did}/trust
@@ -79,14 +133,126 @@ pub async fn get_trust(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<Json<TrustResponse>> {
let trust_score = state.db.get_trust_score(&did).await?;
let push_count = state.db.get_push_count(&did).await?;
let normalized_did = resolve_agent_did(&state, &did).await?;
let trust_score = state.db.get_trust_score(&normalized_did).await?;
let push_count = state.db.get_push_count(&normalized_did).await?;
let level = trust_level(trust_score);
Ok(Json(TrustResponse {
did,
did: normalized_did,
trust_score,
push_count,
level,
}))
}
/// DELETE /api/v1/agents/{did}
///
/// Self-deregistration: the holder of a DID's key marks their own agent
/// `revoked`, removing it from discovery (issue #29). Authenticated by the
/// rfc9421 signature middleware; a caller may only retire its own DID.
pub async fn deregister_agent(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path(did): Path<String>,
) -> Result<StatusCode> {
// Authorize on the verified identity, and act on it too. The path DID is
// only an intent check (it must name the caller); the row we revoke is the
// authenticated DID itself, never a value derived from the untrusted path.
// Revoking `resolve_agent_did(did)` instead would let the fuzzy prefix
// resolver act on a different identity than the one just authorized.
if !caller_matches_did(&auth.0, &did) {
return Err(AppError::BadRequest(
"an agent can only deregister itself".into(),
));
}
if state.db.revoke_agent(&auth.0).await? {
Ok(StatusCode::NO_CONTENT)
} else {
Err(AppError::NotFound(format!("agent {} not found", auth.0)))
}
}
#[cfg(test)]
mod tests {
use super::{agent_key_segment, caller_matches_did, normalize_agent_did, AgentResponse};
#[test]
fn caller_matches_own_did_full_form() {
assert!(caller_matches_did(
"did:key:z6MkExample",
"did:key:z6MkExample"
));
}
#[test]
fn caller_matches_own_did_short_form() {
// Authenticated full DID vs short path form, and vice versa.
assert!(caller_matches_did("did:key:z6MkExample", "z6MkExample"));
assert!(caller_matches_did("z6MkExample", "did:key:z6MkExample"));
}
#[test]
fn caller_cannot_revoke_a_different_did() {
// The core authorization property for issue #29.
assert!(!caller_matches_did(
"did:key:z6MkAttacker",
"did:key:z6MkVictim"
));
assert!(!caller_matches_did("did:key:z6MkAttacker", "z6MkVictim"));
}
#[test]
fn agent_response_surfaces_status() {
// A revoked DID's response must carry its status so callers can see it
// is retired (issue #29).
let resp = AgentResponse {
did: "did:key:orphan".to_string(),
trust_score: 0.1,
capabilities: vec!["reputation:score".to_string()],
registered_at: "2026-06-19T00:00:00Z".to_string(),
last_seen: None,
status: "revoked".to_string(),
};
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["status"], "revoked");
assert!(json.get("replaced_by").is_none());
}
#[test]
fn agent_response_active_status() {
let resp = AgentResponse {
did: "did:key:active".to_string(),
trust_score: 0.5,
capabilities: vec![],
registered_at: "2026-06-19T00:00:00Z".to_string(),
last_seen: None,
status: "active".to_string(),
};
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["status"], "active");
}
#[test]
fn normalize_agent_did_preserves_full_did() {
let did = "did:key:z6MkExample";
assert_eq!(normalize_agent_did(did), did);
}
#[test]
fn normalize_agent_did_expands_bare_key() {
assert_eq!(normalize_agent_did("z6MkExample"), "did:key:z6MkExample");
}
#[test]
fn agent_key_segment_extracts_did_key_material() {
assert_eq!(agent_key_segment("did:key:z6MkExample"), "z6MkExample");
assert_eq!(agent_key_segment("z6MkExample"), "z6MkExample");
}
}
+1 -1
View File
@@ -30,7 +30,7 @@ pub async fn list_anchors(
.db
.list_arweave_anchors(q.repo.as_deref(), limit)
.await
.map_err(|e| crate::error::AppError::Internal(e.into()))?;
.map_err(crate::error::AppError::Internal)?;
Ok(Json(serde_json::json!({
"anchors": anchors,
+139 -51
View File
@@ -79,9 +79,9 @@ pub async fn create_bounty(
return Err(AppError::BadRequest("amount must be positive".into()));
}
// Verify repo exists
let _ = state.db.get_repo(&owner, &repo).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Read-gate: anyone who can read the repo may fund a bounty on it (this closes
// the private-repo bounty leak); ownership is not required.
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let now = Utc::now().to_rfc3339();
let bounty = BountyRecord {
@@ -116,12 +116,15 @@ pub async fn list_repo_bounties(
Path((owner, repo)): Path<(String, String)>,
Query(q): Query<ListBountiesQuery>,
) -> Result<Json<serde_json::Value>> {
let bounties = state.db.list_bounties(
Some(&owner),
Some(&repo),
q.status.as_deref(),
q.limit.unwrap_or(50),
).await?;
let bounties = state
.db
.list_bounties(
Some(&owner),
Some(&repo),
q.status.as_deref(),
q.limit.unwrap_or(50),
)
.await?;
Ok(Json(serde_json::json!({ "bounties": bounties })))
}
@@ -131,12 +134,10 @@ pub async fn list_all_bounties(
State(state): State<AppState>,
Query(q): Query<ListBountiesQuery>,
) -> Result<Json<serde_json::Value>> {
let bounties = state.db.list_bounties(
None,
None,
q.status.as_deref(),
q.limit.unwrap_or(50),
).await?;
let bounties = state
.db
.list_bounties(None, None, q.status.as_deref(), q.limit.unwrap_or(50))
.await?;
Ok(Json(serde_json::json!({ "bounties": bounties })))
}
@@ -146,7 +147,10 @@ pub async fn get_bounty(
State(state): State<AppState>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let bounty = state.db.get_bounty(&id).await?
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
Ok(Json(bounty))
}
@@ -158,19 +162,31 @@ pub async fn claim_bounty(
Path(id): Path<String>,
Json(req): Json<ClaimBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state.db.get_bounty(&id).await?
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "open" {
return Err(AppError::BadRequest(format!("bounty is {}, not open", bounty.status)));
return Err(AppError::BadRequest(format!(
"bounty is {}, not open",
bounty.status
)));
}
let now = Utc::now().to_rfc3339();
state.db.claim_bounty(&id, &auth.0, req.wallet.as_deref(), &now).await?;
state
.db
.claim_bounty(&id, &auth.0, req.wallet.as_deref(), &now)
.await?;
tracing::info!(bounty_id = %id, agent = %auth.0, "bounty claimed");
let updated = state.db.get_bounty(&id).await?
let updated = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
Ok(Json(updated))
}
@@ -182,14 +198,24 @@ pub async fn submit_bounty(
Path(id): Path<String>,
Json(req): Json<SubmitBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state.db.get_bounty(&id).await?
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "claimed" {
return Err(AppError::BadRequest(format!("bounty is {}, not claimed", bounty.status)));
return Err(AppError::BadRequest(format!(
"bounty is {}, not claimed",
bounty.status
)));
}
if bounty.claimant_did.as_deref() != Some(&auth.0) {
return Err(AppError::BadRequest("only the claimant can submit".into()));
let is_claimant = bounty
.claimant_did
.as_deref()
.is_some_and(|c| crate::api::did_matches(&auth.0, c));
if !is_claimant {
return Err(AppError::Forbidden("only the claimant can submit".into()));
}
let now = Utc::now().to_rfc3339();
@@ -197,7 +223,10 @@ pub async fn submit_bounty(
tracing::info!(bounty_id = %id, pr_id = %req.pr_id, "bounty submission");
let updated = state.db.get_bounty(&id).await?
let updated = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
Ok(Json(updated))
}
@@ -209,18 +238,29 @@ pub async fn approve_bounty(
Path(id): Path<String>,
Json(req): Json<ApproveBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state.db.get_bounty(&id).await?
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "submitted" {
return Err(AppError::BadRequest(format!("bounty is {}, not submitted", bounty.status)));
return Err(AppError::BadRequest(format!(
"bounty is {}, not submitted",
bounty.status
)));
}
if bounty.creator_did != auth.0 {
return Err(AppError::BadRequest("only the bounty creator can approve".into()));
if !crate::api::did_matches(&auth.0, &bounty.creator_did) {
return Err(AppError::Forbidden(
"only the bounty creator can approve".into(),
));
}
let now = Utc::now().to_rfc3339();
state.db.approve_bounty(&id, &now, req.tx_hash.as_deref()).await?;
state
.db
.approve_bounty(&id, &now, req.tx_hash.as_deref())
.await?;
// Bump claimant trust score
if let Some(ref agent_did) = bounty.claimant_did {
@@ -231,7 +271,10 @@ pub async fn approve_bounty(
tracing::info!(bounty_id = %id, "bounty approved");
let updated = state.db.get_bounty(&id).await?
let updated = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
Ok(Json(updated))
}
@@ -242,20 +285,31 @@ pub async fn cancel_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let bounty = state.db.get_bounty(&id).await?
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "open" {
return Err(AppError::BadRequest(format!("can only cancel open bounties, status is {}", bounty.status)));
return Err(AppError::BadRequest(format!(
"can only cancel open bounties, status is {}",
bounty.status
)));
}
if bounty.creator_did != auth.0 {
return Err(AppError::BadRequest("only the bounty creator can cancel".into()));
if !crate::api::did_matches(&auth.0, &bounty.creator_did) {
return Err(AppError::Forbidden(
"only the bounty creator can cancel".into(),
));
}
state.db.cancel_bounty(&id).await?;
tracing::info!(bounty_id = %id, "bounty cancelled");
let updated = state.db.get_bounty(&id).await?
let updated = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
Ok(Json(updated))
}
@@ -263,13 +317,33 @@ pub async fn cancel_bounty(
/// POST /api/v1/bounties/{id}/dispute
pub async fn dispute_bounty(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let bounty = state.db.get_bounty(&id).await?
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "claimed" && bounty.status != "submitted" {
return Err(AppError::BadRequest(format!("can only dispute claimed/submitted bounties, status is {}", bounty.status)));
return Err(AppError::BadRequest(format!(
"can only dispute claimed/submitted bounties, status is {}",
bounty.status
)));
}
// Only the bounty creator or the current claimant may dispute (N19 — this
// endpoint had no identity check at all).
let is_creator = crate::api::did_matches(&auth.0, &bounty.creator_did);
let is_claimant = bounty
.claimant_did
.as_deref()
.is_some_and(|c| crate::api::did_matches(&auth.0, c));
if !is_creator && !is_claimant {
return Err(AppError::Forbidden(
"only the bounty creator or claimant can dispute this bounty".into(),
));
}
// Check if deadline exceeded
@@ -277,33 +351,47 @@ pub async fn dispute_bounty(
if let Ok(claimed) = chrono::DateTime::parse_from_rfc3339(claimed_at) {
let deadline = claimed + chrono::Duration::seconds(bounty.deadline_secs);
if Utc::now() < deadline {
return Err(AppError::BadRequest("deadline has not been exceeded yet".into()));
return Err(AppError::BadRequest(
"deadline has not been exceeded yet".into(),
));
}
}
}
state.db.dispute_bounty(&id).await?;
tracing::info!(bounty_id = %id, "bounty disputed — reopened");
tracing::info!(bounty_id = %id, actor = %auth.0, "bounty disputed - reopened");
let updated = state.db.get_bounty(&id).await?
let updated = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
Ok(Json(updated))
}
/// GET /api/v1/bounties/stats
pub async fn bounty_stats(
State(state): State<AppState>,
) -> Result<Json<BountyStatsResponse>> {
pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountyStatsResponse>> {
let open = state.db.count_bounties_by_status("open").await.unwrap_or(0);
let claimed = state.db.count_bounties_by_status("claimed").await.unwrap_or(0);
let completed = state.db.count_bounties_by_status("completed").await.unwrap_or(0);
let claimed = state
.db
.count_bounties_by_status("claimed")
.await
.unwrap_or(0);
let completed = state
.db
.count_bounties_by_status("completed")
.await
.unwrap_or(0);
let leaders = state.db.bounty_leaderboard(10).await.unwrap_or_default();
let leaderboard = leaders.into_iter().map(|(did, cnt, total)| AgentBountyEntry {
did,
completed: cnt,
total_earned: total,
}).collect();
let leaderboard = leaders
.into_iter()
.map(|(did, cnt, total)| AgentBountyEntry {
did,
completed: cnt,
total_earned: total,
})
.collect();
Ok(Json(BountyStatsResponse {
open,
+28 -14
View File
@@ -11,21 +11,29 @@ pub async fn list_certs(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let certs = state.db.list_ref_certificates(&record.id).await?;
let certs_json: Vec<serde_json::Value> = certs.iter().map(|c| serde_json::json!({
"id": c.id,
"repo_id": c.repo_id,
"ref_name": c.ref_name,
"old_sha": c.old_sha,
"new_sha": c.new_sha,
"pusher_did": c.pusher_did,
"node_did": c.node_did,
"signature": c.signature,
"issued_at": c.issued_at,
})).collect();
let certs_json: Vec<serde_json::Value> = certs
.iter()
.map(|c| {
serde_json::json!({
"id": c.id,
"repo_id": c.repo_id,
"ref_name": c.ref_name,
"old_sha": c.old_sha,
"new_sha": c.new_sha,
"pusher_did": c.pusher_did,
"node_did": c.node_did,
"signature": c.signature,
"issued_at": c.issued_at,
})
})
.collect();
Ok(Json(serde_json::json!({ "certificates": certs_json })))
}
@@ -36,10 +44,16 @@ pub async fn get_cert(
Path((owner, name, id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
// Verify the repo exists
let _record = state.db.get_repo(&owner, &name).await?
let _record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let cert = state.db.get_ref_certificate(&id).await?
let cert = state
.db
.get_ref_certificate(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("certificate {id}")))?;
Ok(Json(serde_json::json!({
+25 -14
View File
@@ -1,9 +1,10 @@
//! Changelog endpoint — unified timeline of commits, merged PRs, and closed issues.
use axum::extract::{Path, Query, State};
use axum::extract::{Extension, Path, Query, State};
use axum::Json;
use serde::Deserialize;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::git::store;
use crate::state::AppState;
@@ -14,7 +15,9 @@ pub struct ChangelogQuery {
pub limit: usize,
}
fn default_limit() -> usize { 20 }
fn default_limit() -> usize {
20
}
/// GET /api/v1/repos/:owner/:repo/changelog[?limit=N]
///
@@ -25,28 +28,36 @@ pub async fn get_changelog(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(query): Query<ChangelogQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let limit = query.limit.min(100);
// ── Commits from git log ─────────────────────────────────────────────
let disk_path = state.repo_store.acquire(&record.owner_did, &record.name).await
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let head_ref = store::resolve_head(&disk_path, &record.default_branch);
let commits = store::log(&disk_path, &head_ref, limit).unwrap_or_default();
let mut events: Vec<serde_json::Value> = commits.into_iter().map(|c| {
serde_json::json!({
"type": "commit",
"sha": c.hash,
"message": c.subject,
"author": c.author_name,
"timestamp": c.timestamp,
"branch": record.default_branch,
let mut events: Vec<serde_json::Value> = commits
.into_iter()
.map(|c| {
serde_json::json!({
"type": "commit",
"sha": c.hash,
"message": c.subject,
"author": c.author_name,
"timestamp": c.timestamp,
"branch": record.default_branch,
})
})
}).collect();
.collect();
// ── Merged PRs ───────────────────────────────────────────────────────
let prs = state.db.list_prs(&record.id).await.unwrap_or_default();
+123
View File
@@ -0,0 +1,123 @@
//! Authenticated discovery + fetch for encrypted withheld blobs (Option B1).
use axum::extract::{Extension, Path, State};
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs
/// Returns [{oid, cid}] for every encrypted blob in the repo, to any caller who
/// can read the repo. Not recipient-scoped: recipient identities are not stored,
/// so access control here is repo readability and decryption is gated by the
/// envelope crypto (only a real recipient can open an envelope).
pub async fn list_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
.map(|(oid, cid)| serde_json::json!({ "oid": oid, "cid": cid }))
.collect();
Ok(Json(serde_json::json!({ "blobs": blobs })))
}
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blob/{oid}
/// Returns raw envelope bytes to callers who can read the repo; the envelope
/// crypto still ensures only true recipients can decrypt.
pub async fn get_encrypted_blob(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo, oid)): Path<(String, String, String)>,
) -> Result<Vec<u8>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")));
}
let cid = state
.db
.encrypted_blob_cid(&record.id, &oid)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")))?;
let bytes = crate::ipfs_pin::cat(&state.config.ipfs_api, &cid)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
Ok(bytes)
}
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate
/// Returns [{oid, cid}] for every encrypted blob in the repo, for peer-mirror
/// replication (Option B2). Gated by repo readability, like discovery, so a
/// non-readable repo does not expose its blob index; for the intended case (a
/// public repo with withheld subtrees) the public root keeps this open to peers.
/// Recipient identities are deliberately withheld: the v2 envelopes no longer
/// carry recipient public keys, so peers must not learn the reader set either. A
/// mirror detects a re-seal by the CID changing (the OID is stable across
/// re-seals). Ciphertext metadata only, never plaintext.
pub async fn replicate_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
.map(|(oid, cid)| replicate_blob_json(oid, cid))
.collect();
Ok(Json(serde_json::json!({ "blobs": blobs })))
}
/// Serialize one blob for the replication wire. Recipient identities are
/// intentionally absent so a mirror never learns the reader set.
fn replicate_blob_json(oid: String, cid: String) -> serde_json::Value {
serde_json::json!({ "oid": oid, "cid": cid })
}
#[cfg(test)]
mod tests {
use super::replicate_blob_json;
#[test]
fn replicate_blob_json_omits_recipients() {
let v = replicate_blob_json("oid1".into(), "cidA".into());
assert_eq!(v["oid"], "oid1");
assert_eq!(v["cid"], "cidA");
assert!(
v.get("recipients").is_none(),
"replication wire must not carry recipient identities"
);
}
}
+72 -48
View File
@@ -13,28 +13,36 @@ pub async fn list_ref_updates(
State(state): State<AppState>,
Query(params): Query<HashMap<String, String>>,
) -> Result<Json<serde_json::Value>> {
let limit = params.get("limit")
let limit = params
.get("limit")
.and_then(|v| v.parse::<i64>().ok())
.unwrap_or(50)
.min(200);
let updates = state.db.list_ref_updates(limit).await?;
let events: Vec<serde_json::Value> = updates.iter().map(|u| serde_json::json!({
"id": u.id,
"node_did": u.node_did,
"pusher_did": u.pusher_did,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
})).collect();
let events: Vec<serde_json::Value> = updates
.iter()
.map(|u| {
serde_json::json!({
"id": u.id,
"node_did": u.node_did,
"pusher_did": u.pusher_did,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
})
})
.collect();
let count = events.len();
Ok(Json(serde_json::json!({ "events": events, "count": count })))
Ok(Json(
serde_json::json!({ "events": events, "count": count }),
))
}
/// GET /api/v1/repos/{owner}/{repo}/events
@@ -43,7 +51,8 @@ pub async fn list_repo_events(
Path((owner, repo_name)): Path<(String, String)>,
Query(params): Query<HashMap<String, String>>,
) -> Result<Json<serde_json::Value>> {
let limit = params.get("limit")
let limit = params
.get("limit")
.and_then(|v| v.parse::<i64>().ok())
.unwrap_or(50)
.min(200);
@@ -58,7 +67,11 @@ pub async fn list_repo_events(
let repo_id_str = if let Some(ref record) = repo_record {
format!(
"{}/{}",
record.owner_did.split(':').last().unwrap_or(&record.owner_did),
record
.owner_did
.split(':')
.next_back()
.unwrap_or(&record.owner_did),
repo_name
)
} else {
@@ -67,46 +80,55 @@ pub async fn list_repo_events(
// Fetch local ref certificates for this repo (if the repo exists on this node)
let cert_events: Vec<serde_json::Value> = if let Some(ref record) = repo_record {
state.db.list_ref_certificates(&record.id).await
state
.db
.list_ref_certificates(&record.id)
.await
.unwrap_or_default()
.iter()
.map(|c| serde_json::json!({
"type": "local_cert",
"id": c.id,
"repo": repo_id_str,
"ref_name": c.ref_name,
"old_sha": c.old_sha,
"new_sha": c.new_sha,
"pusher_did": c.pusher_did,
"node_did": c.node_did,
"timestamp": c.issued_at,
"source": "local",
}))
.map(|c| {
serde_json::json!({
"type": "local_cert",
"id": c.id,
"repo": repo_id_str,
"ref_name": c.ref_name,
"old_sha": c.old_sha,
"new_sha": c.new_sha,
"pusher_did": c.pusher_did,
"node_did": c.node_did,
"timestamp": c.issued_at,
"source": "local",
})
})
.collect()
} else {
vec![]
};
// Fetch gossipsub received ref updates for this repo (uses full slug built above)
let gossip_events: Vec<serde_json::Value> = state.db
.list_repo_ref_updates(&repo_id_str, limit).await
let gossip_events: Vec<serde_json::Value> = state
.db
.list_repo_ref_updates(&repo_id_str, limit)
.await
.unwrap_or_default()
.iter()
.map(|u| serde_json::json!({
"type": "gossipsub",
"id": u.id,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"pusher_did": u.pusher_did,
"node_did": u.node_did,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"source": "gossipsub",
}))
.map(|u| {
serde_json::json!({
"type": "gossipsub",
"id": u.id,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"pusher_did": u.pusher_did,
"node_did": u.node_did,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"source": "gossipsub",
})
})
.collect();
// Merge both lists
@@ -124,5 +146,7 @@ pub async fn list_repo_events(
all_events.truncate(limit as usize);
let count = all_events.len();
Ok(Json(serde_json::json!({ "events": all_events, "count": count })))
Ok(Json(
serde_json::json!({ "events": all_events, "count": count }),
))
}
+93 -10
View File
@@ -9,28 +9,47 @@
//! `git cat-file <type> <sha256>` (i.e. without the git framing header).
//! This is consistent with how `gitlawb_core::cid::Cid::from_git_object_bytes`
//! computes CIDs when objects are pushed.
//!
//! Serving is access-controlled: an object is returned only from a repo row the
//! requesting caller is permitted to read (per-caller path-scoped visibility,
//! see `get_by_cid`).
use axum::{
extract::{Path, State},
http::{HeaderMap, HeaderName, HeaderValue, StatusCode},
response::{IntoResponse, Response},
Json,
Extension, Json,
};
use cid::CidGeneric;
use std::collections::{HashMap, HashSet};
use std::str::FromStr;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::git::store;
use crate::git::visibility_pack::{has_path_scoped_rule, withheld_blob_oids};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
/// GET /ipfs/{cid}
///
/// Search all repos on the node for a git object whose SHA-256 hash matches
/// the given CIDv1. Returns the raw object content bytes with appropriate
/// headers if found, or 404 if not found.
/// the given CIDv1, returning its raw content if the caller may read it.
///
/// Visibility (#110): the object is served only from a repo row the caller
/// passes. For each iterated row we gate against that row's OWN rules
/// (`visibility_check` at `"/"`), never re-resolving via `authorize_repo_read`
/// — `get_repo`'s fuzzy match could otherwise authorize a different physical
/// row than the one read (KTD2a). When the row carries path-scoped rules, a
/// blob withheld from the caller (`withheld_blob_oids`) is skipped. Denial and
/// genuine not-found both fall through to an opaque 404.
///
/// Scope: this closes the direct unauthenticated scan. A stale-public mirror
/// row still serves withheld content (tracked separately, #124).
pub async fn get_by_cid(
Path(cid_str): Path<String>,
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Response> {
// 1. Decode the CID and extract the SHA-256 digest
let cid = CidGeneric::<64>::from_str(&cid_str)
@@ -46,19 +65,84 @@ pub async fn get_by_cid(
}
let sha256_hex = hex::encode(mh.digest());
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let caller_owned = caller.map(|c| c.to_string());
// 2. Search all repos for an object with this SHA-256
let repos = state
.db
.list_all_repos().await
.map_err(|e| AppError::Internal(e.into()))?;
.list_all_repos()
.await
.map_err(AppError::Internal)?;
// Fetch every repo's visibility rules in one query rather than one per row
// (the gate runs each row against its OWN rules — KTD2a). A row absent from
// the map has no rules.
let repo_ids: Vec<String> = repos.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = state
.db
.list_visibility_rules_for_repos(&repo_ids)
.await
.map_err(AppError::Internal)?;
// Request-scoped memo of the per-repo withheld set (KTD1). The caller is
// constant for one request, so `repo.id` alone is a safe, sufficient key —
// never a coarse caller "class", which `visibility_check`'s exact full-DID
// reader match would make unsafe.
let mut withheld_memo: HashMap<String, HashSet<String>> = HashMap::new();
for repo in &repos {
// Repo-level read gate against THIS row's own rules (KTD2a).
let rules: &[crate::db::VisibilityRule] = rules_by_repo
.get(&repo.id)
.map(Vec::as_slice)
.unwrap_or(&[]);
if visibility_check(rules, repo.is_public, &repo.owner_did, caller, "/") == Decision::Deny {
continue;
}
let repo_path = match state.repo_store.acquire(&repo.owner_did, &repo.name).await {
Ok(p) => p,
Err(_) => continue,
};
// Per-blob withholding only applies when a path-scoped rule exists (KTD4).
if has_path_scoped_rule(rules) {
if !withheld_memo.contains_key(&repo.id) {
let rp = repo_path.clone();
let r = rules.to_vec();
let is_public = repo.is_public;
let owner = repo.owner_did.clone();
let caller_for_walk = caller_owned.clone();
// Full-history walk shells out to git — keep it off the async runtime.
let walk = tokio::task::spawn_blocking(move || {
withheld_blob_oids(&rp, &r, is_public, &owner, caller_for_walk.as_deref())
})
.await;
// Fail closed on EITHER a task panic (JoinError) or a walk error:
// we cannot prove the caller may read here, so skip this repo and
// let a public copy (if any) serve. Never serve on an unproven gate.
let set = match walk {
Ok(Ok(set)) => set,
Ok(Err(e)) => {
tracing::warn!(repo = %repo.name, err = %e, "withheld walk failed; skipping repo");
continue;
}
Err(e) => {
tracing::warn!(repo = %repo.name, err = %e, "withheld walk task panicked; skipping repo");
continue;
}
};
withheld_memo.insert(repo.id.clone(), set);
}
if withheld_memo
.get(&repo.id)
.is_some_and(|set| set.contains(&sha256_hex))
{
continue;
}
}
match store::read_object(&repo_path, &sha256_hex) {
Ok(Some((_obj_type, content))) => {
// 3. Return the content with IPFS-style headers
@@ -99,13 +183,12 @@ pub async fn get_by_cid(
/// Returns all CIDs that have been pinned to the local IPFS node from git
/// objects received via push. Each entry includes the git SHA-256 hex, the
/// CIDv1 string, and the timestamp when it was pinned.
pub async fn list_pins(
State(state): State<AppState>,
) -> Result<Json<serde_json::Value>> {
pub async fn list_pins(State(state): State<AppState>) -> Result<Json<serde_json::Value>> {
let pins = state
.db
.list_pinned_cids().await
.map_err(|e| AppError::Internal(e.into()))?;
.list_pinned_cids()
.await
.map_err(AppError::Internal)?;
Ok(Json(serde_json::json!({
"pins": pins,
+82 -22
View File
@@ -10,8 +10,8 @@ use uuid::Uuid;
use crate::auth::AuthenticatedDid;
use crate::db::IssueComment;
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::git::issues as git_issues;
use crate::state::AppState;
#[derive(Debug, Deserialize)]
pub struct CreateIssueRequest {
@@ -39,8 +39,11 @@ pub async fn create_issue(
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<CreateIssueRequest>,
) -> Result<(StatusCode, Json<IssueRecord>)> {
let record = state.db.get_repo(&owner, &repo).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Authorize the caller as a reader before accepting an issue: a non-reader
// must not be able to file an issue against a private repo they cannot read.
// Mirrors create_issue_comment / create_review / create_bounty.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let issue_id = Uuid::new_v4().to_string();
let now = Utc::now().to_rfc3339();
@@ -58,14 +61,17 @@ pub async fn create_issue(
let json_str = serde_json::to_string(&issue)
.map_err(|e| AppError::BadRequest(format!("serialization error: {e}")))?;
let guard = state.repo_store.acquire_write(&record.owner_did, &record.name).await
let guard = state
.repo_store
.acquire_write(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let disk_path = guard.path().to_path_buf();
let create_result = git_issues::create_issue(&disk_path, &issue_id, &json_str);
// Always release the advisory lock — even on error
guard.release().await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(create_result.is_ok()).await;
create_result.map_err(|e| AppError::Git(e.to_string()))?;
@@ -85,14 +91,20 @@ pub async fn list_issues(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let disk_path = state.repo_store.acquire(&record.owner_did, &record.name).await
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let raw_issues = git_issues::list_issues(&disk_path)
.map_err(|e| AppError::Git(e.to_string()))?;
let raw_issues =
git_issues::list_issues(&disk_path).map_err(|e| AppError::Git(e.to_string()))?;
let mut issues: Vec<serde_json::Value> = Vec::new();
for raw in raw_issues {
@@ -109,10 +121,16 @@ pub async fn get_issue(
State(state): State<AppState>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let disk_path = state.repo_store.acquire(&record.owner_did, &record.name).await
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let raw = git_issues::get_issue(&disk_path, &issue_id)
@@ -138,13 +156,19 @@ pub async fn create_issue_comment(
Json(req): Json<CreateIssueCommentRequest>,
) -> Result<(StatusCode, Json<IssueComment>)> {
if req.body.trim().is_empty() {
return Err(AppError::BadRequest("comment body must not be empty".into()));
return Err(AppError::BadRequest(
"comment body must not be empty".into(),
));
}
let record = state.db.get_repo(&owner, &repo).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Read-gate: a commenter must be able to read the repo, but need not own it.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let disk_path = state.repo_store.acquire(&record.owner_did, &record.name).await
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
// Verify issue exists
crate::git::issues::get_issue(&disk_path, &issue_id)
@@ -168,7 +192,10 @@ pub async fn list_issue_comments(
State(state): State<AppState>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let _record = state.db.get_repo(&owner, &repo).await?
let _record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let comments = state.db.list_issue_comments(&issue_id).await?;
@@ -178,20 +205,53 @@ pub async fn list_issue_comments(
/// POST /api/v1/repos/{owner}/{repo}/issues/{id}/close
pub async fn close_issue(
State(state): State<AppState>,
Extension(_auth): Extension<AuthenticatedDid>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let guard = state.repo_store.acquire_write(&record.owner_did, &record.name).await
let guard = state
.repo_store
.acquire_write(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let disk_path = guard.path().to_path_buf();
// Owner OR issue author may close. The author lives in the issue's git-JSON
// blob (not a DB column); a None author (legacy issues) falls back to
// owner-only. Read it under the write guard, before mutating.
let author_did: Option<String> = match git_issues::get_issue(&disk_path, &issue_id) {
Ok(Some(raw)) => serde_json::from_str::<IssueRecord>(&raw)
.ok()
.and_then(|i| i.author),
Ok(None) => {
guard.release(false).await;
return Err(AppError::NotFound(format!("issue {issue_id} not found")));
}
Err(e) => {
guard.release(false).await;
return Err(AppError::Git(e.to_string()));
}
};
let is_owner = crate::api::require_repo_owner(&record, &auth.0).is_ok();
let is_author = author_did
.as_deref()
.is_some_and(|a| crate::api::did_matches(&auth.0, a));
if !is_owner && !is_author {
guard.release(false).await;
return Err(AppError::Forbidden(
"only the repo owner or the issue author can close this issue".into(),
));
}
let close_result = git_issues::close_issue(&disk_path, &issue_id);
// Always release the advisory lock — even on error
guard.release().await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(close_result.is_ok()).await;
let updated = close_result
.map_err(|e| AppError::Git(e.to_string()))?
+29 -8
View File
@@ -17,7 +17,7 @@ pub struct LabelRequest {
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(_auth): Extension<AuthenticatedDid>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
@@ -25,28 +25,46 @@ pub async fn add_label(
if label.is_empty() || label.len() > 50 {
return Err(AppError::BadRequest("label must be 1–50 characters".into()));
}
if !label.chars().all(|c| c.is_alphanumeric() || c == '-' || c == ':') {
if !label
.chars()
.all(|c| c.is_alphanumeric() || c == '-' || c == ':')
{
return Err(AppError::BadRequest(
"label must contain only alphanumeric characters, hyphens, and colons".into(),
));
}
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
let added = state.db.add_label(&record.id, &label).await?;
let status = if added { StatusCode::CREATED } else { StatusCode::OK };
Ok((status, Json(serde_json::json!({ "label": label, "added": added }))))
let status = if added {
StatusCode::CREATED
} else {
StatusCode::OK
};
Ok((
status,
Json(serde_json::json!({ "label": label, "added": added })),
))
}
/// DELETE /api/v1/repos/:owner/:repo/labels/:label
pub async fn remove_label(
State(state): State<AppState>,
Extension(_auth): Extension<AuthenticatedDid>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, label)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
state.db.remove_label(&record.id, &label).await?;
Ok(Json(serde_json::json!({ "label": label, "removed": true })))
@@ -57,7 +75,10 @@ pub async fn list_labels(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let labels = state.db.list_labels(&record.id).await?;
+553
View File
@@ -1,18 +1,571 @@
use crate::db::{RepoRecord, VisibilityRule};
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
pub mod agents;
pub mod arweave;
pub mod bounties;
pub mod certs;
pub mod changelog;
pub mod encrypted;
pub mod events;
pub mod ipfs;
pub mod issues;
pub mod labels;
pub mod peers;
pub mod profiles;
pub mod protect;
pub mod pulls;
pub mod register;
pub mod replicas;
pub mod repos;
pub mod resolve;
pub mod stars;
pub mod tasks;
pub mod visibility;
pub mod webhooks;
/// Resolve a repo for a read request and enforce path-scoped visibility.
///
/// Returns 404 (`RepoNotFound`) if the repo does not exist or the caller may not
/// read `path`, using the same opaque response the git serve path returns so
/// existence is not confirmed. Returns the record and its visibility rules so a
/// content handler can apply an extra per-path check without a second DB query.
///
/// Callers pass `"/"` for repo-level reads (listings); content endpoints pass the
/// specific path so a withheld subtree is denied even on an otherwise-public repo.
pub(crate) async fn authorize_repo_read(
state: &AppState,
owner: &str,
name: &str,
caller: Option<&str>,
path: &str,
) -> Result<(RepoRecord, Vec<VisibilityRule>)> {
let record = state
.db
.get_repo(owner, name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// A quarantined mirror (admitted by the iCaptcha propagation gate but not
// validated) is hidden from every reader — serve/clone and fork alike — as if
// it did not exist, until an operator releases it. Checked before the
// visibility gate so its existence is never disclosed.
if state.db.is_repo_quarantined(&record.id).await? {
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, path) == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
Ok((record, rules))
}
/// Match a presented DID against a stored DID that may be the full `did:key:<id>`
/// form or the bare `<id>` short form (mirror rows store the bare key). Collapse
/// representation only within `did:key`; never let a bare id match across methods —
/// `did:web` / `did:gitlawb` share the base58 space with `did:key`, so a
/// trailing-segment compare would treat `did:key:X` and `did:gitlawb:X` as equal.
pub(crate) fn did_matches(a: &str, b: &str) -> bool {
if a == b {
return true;
}
fn key_id(d: &str) -> &str {
d.strip_prefix("did:key:").unwrap_or(d)
}
let (ka, kb) = (key_id(a), key_id(b));
// After stripping `did:key:`, a value still containing ':' is a non-key full
// DID — do not let it match a bare `did:key` id.
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
/// and the author check (close policy) share one normalization.
pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> {
if did_matches(caller, &record.owner_did) {
Ok(())
} else {
Err(AppError::Forbidden(
"only the repo owner can perform this action".into(),
))
}
}
#[cfg(test)]
mod did_tests {
use super::did_matches;
#[test]
fn full_matches_bare_same_key() {
assert!(did_matches("did:key:zABC", "zABC"));
assert!(did_matches("zABC", "did:key:zABC"));
}
#[test]
fn rejects_cross_method_collision() {
assert!(!did_matches("did:key:zABC", "did:gitlawb:zABC"));
assert!(!did_matches("did:key:zABC", "did:web:zABC"));
}
#[test]
fn exact_match_and_distinct_keys() {
assert!(did_matches("did:key:zABC", "did:key:zABC"));
assert!(!did_matches("did:key:zABC", "did:key:zXYZ"));
assert!(!did_matches("zABC", "zXYZ"));
}
}
/// Drift guard (plan 002 §Gate-type table, Step 5). Every in-scope mutation
/// handler must contain its expected gate marker in its own body; removing a
/// gate fails this test. Source-level (no DB), so it runs everywhere. When a new
/// route is added to an in-scope group, add its row here with a deliberate gate
/// type — that forced decision is the point.
///
/// Markers are gate-SHAPED — a call (`require_repo_owner(`, `did_matches(`) or a
/// binding/comparison expression (`caller != &record.owner_did`,
/// `let owner_did = auth.0`) — never a bare identifier that could also appear in
/// a log line. Full-line comments are stripped before matching, so a marker that
/// survives only as a comment above a deleted gate does NOT satisfy a row.
#[cfg(test)]
mod authz_guard {
/// The body of `func` with full-line comments removed. Bounds the slice at the
/// next top-level fn item so a marker in a later handler can't leak in,
/// tolerating `pub async`, `pub(crate) async`, `async`, `pub`, and bare `fn`
/// declarations (the old single-`pub async fn` delimiter over-ran on any other
/// form).
fn fn_body(src: &str, func: &str) -> String {
let needle = format!("fn {func}(");
let start = src
.find(&needle)
.unwrap_or_else(|| panic!("handler `{func}` not found (renamed or removed?)"));
let rest = &src[start..];
let end = [
"\npub async fn ",
"\npub(crate) async fn ",
"\nasync fn ",
"\npub fn ",
"\nfn ",
]
.iter()
.filter_map(|p| rest[1..].find(p).map(|i| i + 1))
.min()
.unwrap_or(rest.len());
rest[..end]
.lines()
.filter(|l| !l.trim_start().starts_with("//"))
.collect::<Vec<_>>()
.join("\n")
}
#[test]
fn every_in_scope_mutation_has_its_gate() {
let pulls = include_str!("pulls.rs");
let webhooks = include_str!("webhooks.rs");
let labels = include_str!("labels.rs");
let issues = include_str!("issues.rs");
let bounties = include_str!("bounties.rs");
let replicas = include_str!("replicas.rs");
let tasks = include_str!("tasks.rs");
let stars = include_str!("stars.rs");
let protect = include_str!("protect.rs");
let visibility = include_str!("visibility.rs");
let profiles = include_str!("profiles.rs");
let repos = include_str!("repos.rs");
let register = include_str!("register.rs");
let ipfs = include_str!("ipfs.rs");
// (source, handler, expected gate marker)
let rows: &[(&str, &str, &str)] = &[
// Bucket A — owner-gate (require_repo_owner -> 403)
(pulls, "merge_pr", "require_repo_owner("),
(webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "require_repo_owner("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "require_repo_owner("),
// Bucket A' — owner OR author (did_matches against the author)
(pulls, "close_pr", "did_matches("),
(issues, "close_issue", "did_matches("),
// Bucket B — read-gate (authorize_repo_read)
(pulls, "create_review", "authorize_repo_read("),
(pulls, "create_comment", "authorize_repo_read("),
(pulls, "create_pr", "authorize_repo_read("),
(issues, "create_issue_comment", "authorize_repo_read("),
(issues, "create_issue", "authorize_repo_read("),
(bounties, "create_bounty", "authorize_repo_read("),
(repos, "fork_repo", "authorize_repo_read("),
// get_by_cid gates each iterated repo row directly via visibility_check
// (KTD2a: it must NOT route through authorize_repo_read's fuzzy re-resolve).
(ipfs, "get_by_cid", "visibility_check("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
(tasks, "claim_task", "did_matches("),
(tasks, "complete_task", "did_matches("),
(tasks, "fail_task", "did_matches("),
(repos, "create_repo", "let owner_did = auth.0"),
(profiles, "set_profile", "let did = auth.0"),
(register, "register", "did_matches("),
(stars, "star_repo", "caller = &auth.0"),
(stars, "unstar_repo", "caller = &auth.0"),
// Bucket D — non-owner-by-design, positive per-route marker
(bounties, "claim_bounty", "claim_bounty(&id, &auth.0"),
(bounties, "submit_bounty", "did_matches("),
(bounties, "approve_bounty", "did_matches("),
(bounties, "cancel_bounty", "did_matches("),
(bounties, "dispute_bounty", "did_matches("),
(replicas, "register_replica", "did_matches("),
(replicas, "unregister_replica", "replica_did = &auth.0"),
// PRE-GATED — already owner-gated, in-scope group; guard the gate itself
(protect, "protect_branch", "did_matches("),
(protect, "unprotect_branch", "did_matches("),
(visibility, "set_visibility", "require_owner("),
(visibility, "remove_visibility", "require_owner("),
(visibility, "list_visibility", "require_owner("),
];
// The visibility rows prove require_owner is CALLED; this proves the helper
// itself does DID-safe matching, not a raw/trailing-segment compare.
assert!(
fn_body(visibility, "require_owner").contains("did_matches("),
"visibility::require_owner must use did_matches for DID-safe owner matching"
);
for (src, func, marker) in rows {
let body = fn_body(src, func);
assert!(
body.contains(marker),
"handler `{func}` is missing its gate marker `{marker}` — gate removed or route reclassified"
);
}
}
/// Proves the comment-stripping that GUARD-1 added: a marker that appears only
/// in a full-line comment (the real `replicas.rs` false-pass shape) must NOT
/// satisfy a row.
#[test]
fn comment_only_marker_does_not_satisfy_a_row() {
let src = "pub async fn demo() {\n // did_matches( handles the owner form\n do_thing();\n}\n";
assert!(
!fn_body(src, "demo").contains("did_matches("),
"a marker present only in a comment must not count as an enforced gate"
);
}
/// All `pub async fn` / `pub(crate) async fn` handler names declared in `src`.
/// Verb-agnostic on purpose: a repo-scoped read of any name is in scope, so a
/// handler named `fetch_*` / `replicate_*` / `info_refs` cannot escape the gate
/// check by not being called `list_*` / `get_*`.
fn handler_names(src: &str) -> Vec<String> {
["pub async fn ", "pub(crate) async fn "]
.iter()
.flat_map(|decl| {
src.match_indices(decl).map(move |(i, _)| {
src[i + decl.len()..]
.chars()
.take_while(|c| c.is_alphanumeric() || *c == '_')
.collect::<String>()
})
})
.collect()
}
/// True when the handler's signature takes an owner+repo path
/// (`Path<(String, String...)>`), i.e. it is a repo-scoped read rather than a
/// node-wide aggregate (`/stats`, `/ipfs/pins`, the global feeds).
fn is_repo_scoped(body: &str) -> bool {
let header = &body[..body.find('{').unwrap_or(body.len())];
header.contains("Path<(String, String")
}
/// True when at least one gate marker runs for EVERY service — i.e. some
/// marker sits outside any `if service == ...` discriminator block. A gate
/// that appears ONLY inside such a block (as the info/refs advertisement gate
/// did before #119: `visibility_check` ran under `if service ==
/// "git-upload-pack"`, leaving `git-receive-pack` ungated) covers a subset of
/// services and must NOT count as a full gate. Other handlers carry no
/// `service ==` discriminator, so for them this matches the plain
/// substring check. NOTE: only `if service ==` is detected — a
/// `match service { .. }` discriminator is NOT tracked and a gate inside one
/// arm would pass as full; avoid that shape, or extend the span loop below.
fn gate_runs_unconditionally(body: &str, markers: &[&str]) -> bool {
// Brace-matched spans of each `if service == ...` block.
let mut cond_spans: Vec<(usize, usize)> = Vec::new();
let mut search = 0;
while let Some(rel) = body[search..].find("if service ==") {
let cond_start = search + rel;
let Some(brace_rel) = body[cond_start..].find('{') else {
break;
};
let open = cond_start + brace_rel;
let mut depth = 0i32;
let mut end = body.len();
for (i, c) in body[open..].char_indices() {
match c {
'{' => depth += 1,
'}' => {
depth -= 1;
if depth == 0 {
end = open + i;
break;
}
}
_ => {}
}
}
cond_spans.push((open, end));
// On an unclosed block `end` stays at body.len() (the fail-safe
// direction: treat the rest as conditional rather than mask a gate);
// clamp so the next slice can't index past the end and panic.
search = (end + 1).min(body.len());
}
markers.iter().any(|m| {
body.match_indices(m)
.any(|(pos, _)| !cond_spans.iter().any(|(s, e)| pos >= *s && pos <= *e))
})
}
/// Egress gate guard: every repo-scoped handler (`Path<(String, String..)>`)
/// must carry an authz marker — a read gate (`authorize_repo_read` /
/// `visibility_check`), or a write gate (`require_repo_owner` / `require_owner`
/// / `did_matches` / a `&auth.0` self-binding) — or be listed in KNOWN_UNGATED
/// (the tracked, ungated reads). A new ungated handler of ANY verb trips this,
/// and a completeness scan over `src/api/` trips it for a whole new module that
/// adds a repo-scoped handler without being wired into `sources`. Mutations are
/// also checked precisely, per handler, by the mutation guard above; here they
/// only need *some* binding so the net stays verb-agnostic.
///
/// Scope and limits (this is a source scrape; the runtime route guard covers
/// behaviour):
/// - It proves a gate is CALLED, not that it runs on the requested path: a
/// content handler gating `"/"` instead of the subtree still passes here.
/// - It sees handlers taking the owner+repo tuple `Path<(String, String..)>`; a
/// repo identified by a struct/custom extractor would be missed (the helper
/// unit tests pin these boundaries).
/// - KNOWN_UNGATED entries need a real open issue and review; the staleness
/// assert removes one only once its gate lands.
#[test]
fn every_repo_scoped_handler_is_gated() {
let sources: &[(&str, &str)] = &[
(include_str!("bounties.rs"), "bounties.rs"),
(include_str!("certs.rs"), "certs.rs"),
(include_str!("changelog.rs"), "changelog.rs"),
(include_str!("encrypted.rs"), "encrypted.rs"),
(include_str!("events.rs"), "events.rs"),
(include_str!("issues.rs"), "issues.rs"),
(include_str!("labels.rs"), "labels.rs"),
(include_str!("protect.rs"), "protect.rs"),
(include_str!("pulls.rs"), "pulls.rs"),
(include_str!("replicas.rs"), "replicas.rs"),
(include_str!("repos.rs"), "repos.rs"),
(include_str!("stars.rs"), "stars.rs"),
(include_str!("visibility.rs"), "visibility.rs"),
(include_str!("webhooks.rs"), "webhooks.rs"),
];
let listed: std::collections::HashSet<&str> = sources.iter().map(|(_, f)| *f).collect();
assert!(!listed.is_empty(), "read-guard `sources` is empty");
// Completeness: every api/*.rs NOT already in `sources` must declare no
// repo-scoped handler, so a brand-new module cannot add an ungated handler
// the scrape never looks at. Reads the directory at test time (so the list
// cannot silently drift from the filesystem) and only inspects unlisted
// files — listed files are covered by the per-handler loop below.
let api_dir = concat!(env!("CARGO_MANIFEST_DIR"), "/src/api");
for (_, f) in sources {
assert!(
std::path::Path::new(api_dir).join(f).exists(),
"read-guard `sources` lists {f} but the file does not exist"
);
}
for entry in std::fs::read_dir(api_dir).expect("read src/api") {
let path = entry.expect("dir entry").path();
let fname = path.file_name().unwrap().to_string_lossy().into_owned();
if !fname.ends_with(".rs") || fname == "mod.rs" || listed.contains(fname.as_str()) {
continue;
}
let src = std::fs::read_to_string(&path).expect("read api file");
let has_repo_handler = handler_names(&src)
.iter()
.any(|n| is_repo_scoped(&fn_body(&src, n)));
assert!(
!has_repo_handler,
"api/{fname} declares a repo-scoped handler but is not in the egress \
guard `sources` list — add it so its handlers are gate-checked"
);
}
// Repo-scoped reads known to be ungated today, each tracked by an issue.
// Remove an entry the moment its gate lands (the staleness assert enforces it).
let known_ungated: &[(&str, &str)] = &[
// info/refs gates only git-upload-pack today; git-receive-pack
// advertisement is ungated until #119 makes the gate unconditional.
("git_info_refs", "#119"),
("list_certs", "#120"),
("get_cert", "#120"),
("list_issues", "#120"),
("get_issue", "#120"),
("list_issue_comments", "#120"),
("list_labels", "#120"),
("list_repo_bounties", "#120"),
("get_star_status", "#120"),
("list_repo_events", "#94 (PR #113)"),
("list_webhooks", "#94 (PR #113)"),
("list_replicas", "PR #113"),
("list_protected_branches", "PR #113"),
];
let is_known = |n: &str| known_ungated.iter().any(|(k, _)| *k == n);
// Any one of these = the handler binds the caller to an authz decision: the
// first two are read gates, the rest are the write/owner/self-binding forms.
// A repo-scoped mutation passes here on its write gate; the mutation guard
// above additionally verifies the exact gate type per handler. NOTE: a local
// rename of `caller`/`replica_did` must be paired with a marker update here.
let markers = [
"authorize_repo_read(",
"visibility_check(",
"require_repo_owner(",
"require_owner(",
"did_matches(",
"caller = &auth.0",
"replica_did = &auth.0",
];
// Every KNOWN_UNGATED name must be a real handler (catch typos / renames).
let all: std::collections::HashSet<String> =
sources.iter().flat_map(|(s, _)| handler_names(s)).collect();
for (n, _) in known_ungated {
assert!(
all.contains(*n),
"KNOWN_UNGATED lists `{n}`, which is not a real handler (renamed or removed?)"
);
}
let mut checked = 0usize;
for (src, file) in sources {
for name in handler_names(src) {
let body = fn_body(src, &name);
if !is_repo_scoped(&body) {
continue; // node-wide aggregate, not a repo-scoped surface
}
checked += 1;
let gated = gate_runs_unconditionally(&body, &markers);
assert!(
gated || is_known(&name),
"repo-scoped handler `{name}` ({file}) has no authz gate and is \
not in KNOWN_UNGATED — add the visibility/owner gate with the \
caller, or track it there"
);
assert!(
!(gated && is_known(&name)),
"handler `{name}` ({file}) is now gated — remove it from \
KNOWN_UNGATED so the allowlist stays an accurate gap list"
);
}
}
// Tripwire: if the scrape silently stopped finding handlers (e.g. a parser
// regression in handler_names/is_repo_scoped), this floor fails loudly
// instead of the guard passing vacuously. Current count is ~54 repo-scoped
// handlers; 20 is a deliberate floor that trips only on a gross regression.
assert!(
checked >= 20,
"egress guard only checked {checked} repo-scoped handlers — the scrape likely broke"
);
}
/// Pins the `handler_names` boundary: it collects every `pub`/`pub(crate)
/// async fn` regardless of verb (so a `fetch_*` read cannot escape), and skips
/// private `async fn` and sync `fn` helpers.
#[test]
fn handler_names_finds_all_pub_async_and_skips_others() {
let src = "pub async fn list_things() {}\n\
pub async fn fetch_thing() {}\n\
pub(crate) async fn get_crate_thing() {}\n\
async fn private_helper() {}\n\
fn sync_helper() {}\n";
let names = handler_names(src);
// Verb-agnostic: a non-list/get read verb is still seen.
assert!(names.contains(&"list_things".to_string()));
assert!(names.contains(&"fetch_thing".to_string()));
// pub(crate) routed handlers are in scope too.
assert!(names.contains(&"get_crate_thing".to_string()));
// Private/sync helpers are not routed handlers and are skipped.
assert!(!names
.iter()
.any(|n| n == "private_helper" || n == "sync_helper"));
}
/// Pins the `is_repo_scoped` boundary: an owner+repo tuple Path is repo-scoped;
/// a single-segment or absent Path is a node-wide aggregate.
#[test]
fn is_repo_scoped_requires_owner_repo_path() {
let repo =
"fn get_x(s: State, Path((owner, name)): Path<(String, String)>) {\n body();\n}";
let three = "fn get_y(Path((o, n, id)): Path<(String, String, String)>) {\n body();\n}";
let node_wide = "fn list_z(s: State<AppState>) {\n body();\n}";
let single = "fn get_w(Path(cid): Path<String>) {\n body();\n}";
assert!(is_repo_scoped(repo), "owner+repo tuple is repo-scoped");
assert!(is_repo_scoped(three), "owner+repo+id tuple is repo-scoped");
assert!(!is_repo_scoped(node_wide), "no Path is node-wide");
assert!(
!is_repo_scoped(single),
"single-segment Path is not repo-scoped"
);
}
/// Pins `gate_runs_unconditionally`: a gate nested only inside an
/// `if service == ...` block is conditional (does NOT count), while the same
/// gate at the top level — or an additional unconditional one — does.
#[test]
fn conditional_service_gate_is_not_a_full_gate() {
let markers = ["visibility_check("];
// Gate runs only for one service: not a full gate.
let conditional = "fn f() {\n \
if service == \"git-upload-pack\" {\n \
visibility_check(rules, caller);\n \
}\n \
if service == \"git-receive-pack\" { acquire_fresh(); }\n}";
assert!(
!gate_runs_unconditionally(conditional, &markers),
"a gate only inside `if service ==` covers a subset of services"
);
// Same gate at top level: full gate.
let unconditional = "fn f() {\n \
visibility_check(rules, caller);\n \
if service == \"git-receive-pack\" { acquire_fresh(); }\n}";
assert!(
gate_runs_unconditionally(unconditional, &markers),
"an unconditional gate runs for every service"
);
// A gate inside EACH of two service blocks, none outside: still a
// subset (no service clears it unconditionally), so not a full gate.
let both_conditional = "fn f() {\n \
if service == \"git-upload-pack\" { visibility_check(a); }\n \
if service == \"git-receive-pack\" { visibility_check(b); }\n}";
assert!(
!gate_runs_unconditionally(both_conditional, &markers),
"a gate inside every service block is still conditional"
);
// A marker inside one block AND again unconditionally: the
// unconditional occurrence makes it a full gate (exercises the
// match_indices scan past the in-block hit).
let inside_and_outside = "fn f() {\n \
if service == \"git-upload-pack\" { visibility_check(a); }\n \
visibility_check(b);\n}";
assert!(
gate_runs_unconditionally(inside_and_outside, &markers),
"an unconditional occurrence counts even when another is conditional"
);
// No marker at all: not gated.
assert!(!gate_runs_unconditionally(
"fn f() { do_thing(); }",
&markers
));
// An unclosed `if service ==` block (e.g. phantom brace from a string
// literal) must not panic on the slice advance; the span runs to EOF, so
// the in-block marker reads as conditional. Real Rust source is balanced,
// so this only guards the scraper against a future pathological body.
let unclosed = "fn f() { if service == \"x\" { visibility_check(a);";
assert!(
!gate_runs_unconditionally(unclosed, &markers),
"an unclosed service block must not panic and stays conditional"
);
}
}
+367 -45
View File
@@ -10,11 +10,13 @@
//! POST /api/v1/sync/notify — receive push notification from a peer node
//! POST /api/v1/sync/trigger — manually pull all repos from known peers
use axum::extract::{Path, State};
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use gitlawb_core::did::Did;
use serde::{Deserialize, Serialize};
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
@@ -38,6 +40,123 @@ pub struct PeerResponse {
pub reachable: bool,
}
/// Extract an IPv4 address embedded in an IPv6 literal across the transition
/// formats that carry one: IPv4-mapped (`::ffff:a.b.c.d`), IPv4-compatible
/// (`::a.b.c.d`), 6to4 (`2002:WWXX:YYZZ::/16`), and the NAT64 well-known prefix
/// (`64:ff9b::/96`). Returns `None` for native IPv6. Callers fold the result
/// back through the v4 range checks so loopback/private addresses smuggled in
/// via any of these encodings are rejected, not just the mapped/compatible pair.
///
/// INCOMPLETE-NAT64 CONTRACT: non-well-known NAT64 prefixes (e.g. the RFC 8215
/// local-use `64:ff9b:1::/48`) are deliberately **not** decoded here — their v4
/// sits at a prefix-length-dependent offset (RFC 6052 §2.2) — so they return
/// `None`. Any caller that needs them blocked must reject the wider
/// `64:ff9b::/32` itself; `is_public_http_url` does this in its native-v6 arm.
fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr> {
if let Some(v4) = v6.to_ipv4_mapped().or_else(|| v6.to_ipv4()) {
return Some(v4);
}
let s = v6.segments();
// 6to4 (2002::/16) embeds W.X.Y.Z in the next two segments.
if s[0] == 0x2002 {
let [a, b] = s[1].to_be_bytes();
let [c, d] = s[2].to_be_bytes();
return Some(std::net::Ipv4Addr::new(a, b, c, d));
}
// NAT64 well-known prefix (64:ff9b::/96) embeds the IPv4 in the low 32 bits.
if s[0] == 0x0064 && s[1] == 0xff9b && s[2..6] == [0, 0, 0, 0] {
let [a, b] = s[6].to_be_bytes();
let [c, d] = s[7].to_be_bytes();
return Some(std::net::Ipv4Addr::new(a, b, c, d));
}
None
}
/// Whether a peer `http_url` is a public http(s) endpoint safe to register.
/// Rejects non-http(s) schemes, loopback/unspecified/private/link-local IPs,
/// and `localhost` / `.local` / `.internal` hostnames. Used at announce time
/// and by the boot-time prune of already-poisoned rows.
pub fn is_public_http_url(raw: &str) -> bool {
let url = match reqwest::Url::parse(raw) {
Ok(u) => u,
Err(_) => return false,
};
if !matches!(url.scheme(), "http" | "https") {
return false;
}
let mut host = match url.host_str() {
Some(h) => h.to_ascii_lowercase(),
None => return false,
};
// Drop a single trailing dot (FQDN root): `localhost.` resolves the same as
// `localhost`, so normalize before the suffix/equality checks below.
if let Some(stripped) = host.strip_suffix('.') {
host = stripped.to_string();
}
if host.is_empty()
|| host == "localhost"
|| host.ends_with(".local")
|| host.ends_with(".internal")
{
return false;
}
// host_str() keeps brackets on IPv6 literals (e.g. "[::1]"); strip them
// before parsing as an IP.
let ip_candidate = host.trim_start_matches('[').trim_end_matches(']');
if let Ok(ip) = ip_candidate.parse::<std::net::IpAddr>() {
// Reject loopback/unspecified on the literal as given — catches `::1`
// and `::` before the IPv4-folding below (`::1`.to_ipv4() would
// otherwise map to a non-loopback `0.0.0.1`).
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
// Fold any IPv6 literal that embeds an IPv4 address (mapped, compatible,
// 6to4, NAT64) down to that IPv4 so the v4 range checks catch
// loopback/private addresses smuggled in via an IPv6 encoding, then
// re-check loopback/unspecified.
let ip = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
v4 => v4,
};
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
match ip {
std::net::IpAddr::V4(v4) => {
let o = v4.octets();
// RFC1918 private, link-local, CGNAT (100.64.0.0/10), or the
// RFC1122 "this host" block 0.0.0.0/8 (never a valid destination;
// 0.0.0.0 itself is already caught by the is_unspecified check).
if v4.is_private()
|| v4.is_link_local()
|| (o[0] == 100 && (o[1] & 0xc0) == 64)
|| o[0] == 0
{
return false;
}
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// fc00::/7 (unique-local) or fe80::/10 (link-local)
if (s[0] & 0xfe00) == 0xfc00 || (s[0] & 0xffc0) == 0xfe80 {
return false;
}
// Any NAT64 address (64:ff9b::/32) that is not the cleanly
// decodable well-known /96 — e.g. the RFC 8215 local-use
// 64:ff9b:1::/48 — carries a translated target whose embedded v4
// sits at a prefix-length-dependent offset (RFC 6052 §2.2).
// Rather than risk a wrong decode across every prefix length we
// reject the whole NAT64 space here. The well-known /96 was
// already folded to its v4 above and never reaches this arm.
if s[0] == 0x0064 && s[1] == 0xff9b {
return false;
}
}
}
}
true
}
/// GET /api/v1/peers
pub async fn list_peers(State(state): State<AppState>) -> Result<Json<serde_json::Value>> {
let peers = state.db.list_peers().await?;
@@ -50,7 +169,9 @@ pub async fn list_peers(State(state): State<AppState>) -> Result<Json<serde_json
reachable: p.last_ping_ok,
})
.collect();
Ok(Json(serde_json::json!({ "peers": resp, "count": resp.len() })))
Ok(Json(
serde_json::json!({ "peers": resp, "count": resp.len() }),
))
}
/// POST /api/v1/peers/announce (auth required)
@@ -59,11 +180,52 @@ pub async fn list_peers(State(state): State<AppState>) -> Result<Json<serde_json
/// The Authorization header's DID must match the `did` in the body.
pub async fn announce(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Json(req): Json<AnnounceRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
// Validate the URL is HTTP/HTTPS
if !req.http_url.starts_with("http://") && !req.http_url.starts_with("https://") {
return Err(AppError::BadRequest("http_url must start with http:// or https://".into()));
let announced_did: Did = req
.did
.parse()
.map_err(|e: gitlawb_core::Error| AppError::BadRequest(e.to_string()))?;
if let Some(Extension(auth)) = auth {
if auth.0 != announced_did.to_string() {
return Err(AppError::BadRequest(
"Signature keyid must match announced DID".into(),
));
}
} else {
tracing::warn!(
did = %announced_did,
"accepted unsigned peer announce; set GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true after all peers upgrade"
);
}
// Validate the URL is a public http(s) endpoint. The announce route is
// reachable unauthenticated (until all peers sign), so without this an
// attacker can register loopback/private "peers" (localhost:5432, etc.)
// and turn our outbound sync-notify fan-out into an SSRF probe — and bury
// the real peers under junk so node-origin repos stop replicating.
if !is_public_http_url(&req.http_url) {
return Err(AppError::BadRequest(
"http_url must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(),
));
}
// Reject self-announcements: a peer row whose http_url is our own public
// URL makes the HTTP-notify path fan out to ourselves. Seen in prod when
// misconfigured dev nodes announce with their upstream's URL.
// prune_self_peers clears stale rows at boot; this stops new ones.
if let Some(self_url) = state.config.public_url.as_deref() {
if req.http_url.trim_end_matches('/') == self_url.trim_end_matches('/') {
return Err(AppError::BadRequest(
"http_url is this node's own public URL; refusing to register self as peer".into(),
));
}
}
if announced_did.to_string() == state.node_did.to_string() {
return Err(AppError::BadRequest(
"did is this node's own DID; refusing to register self as peer".into(),
));
}
state.db.upsert_peer(&req.did, &req.http_url).await?;
@@ -71,13 +233,16 @@ pub async fn announce(
tracing::info!(did = %req.did, url = %req.http_url, "peer announced");
// Return our own info so the peer can add us back
Ok((StatusCode::OK, Json(serde_json::json!({
"status": "accepted",
"node_did": state.node_did.to_string(),
"node_url": state.config.public_url.as_deref().unwrap_or(""),
"peer_count": state.db.list_peers().await.map(|p| p.len()).unwrap_or(0),
"message": "added to peer list",
}))))
Ok((
StatusCode::OK,
Json(serde_json::json!({
"status": "accepted",
"node_did": state.node_did.to_string(),
"node_url": state.config.public_url.as_deref().unwrap_or(""),
"peer_count": state.db.list_peers().await.map(|p| p.len()).unwrap_or(0),
"message": "added to peer list",
})),
))
}
/// POST /api/v1/sync/trigger
@@ -86,9 +251,7 @@ pub async fn announce(
/// peer's repo list over HTTP and enqueues any repos we don't have or are
/// behind on into the sync_queue. This is the HTTP fallback when Gossipsub
/// p2p is not yet connected.
pub async fn trigger_sync(
State(state): State<AppState>,
) -> Result<Json<serde_json::Value>> {
pub async fn trigger_sync(State(state): State<AppState>) -> Result<Json<serde_json::Value>> {
let peers = state.db.list_peers().await?;
let client = &state.http_client;
let mut enqueued = 0u32;
@@ -99,39 +262,56 @@ pub async fn trigger_sync(
continue;
}
let url = format!("{}/api/v1/repos", peer.http_url.trim_end_matches('/'));
let result = tokio::time::timeout(
std::time::Duration::from_secs(5),
client.get(&url).send(),
).await;
let repos: Vec<serde_json::Value> = match result {
Ok(Ok(resp)) if resp.status().is_success() => {
peers_reached += 1;
resp.json().await.unwrap_or_default()
// 30s with the body read inside the timeout: 5s only covered the
// response headers, so canonical nodes serving large unpaginated repo
// lists (and transpacific round trips) aborted mid-body.
let result = tokio::time::timeout(std::time::Duration::from_secs(30), async {
let resp = client.get(&url).send().await?;
if !resp.status().is_success() {
return Err(anyhow::anyhow!("peer returned status {}", resp.status()));
}
_ => {
tracing::warn!(peer = %peer.did, "trigger_sync: could not reach peer");
let repos: Vec<serde_json::Value> = resp.json().await?;
Ok::<_, anyhow::Error>(repos)
})
.await;
let repos = match result {
Ok(Ok(repos)) => {
peers_reached += 1;
repos
}
Ok(Err(e)) => {
tracing::warn!(peer = %peer.did, err = %e, "trigger_sync: peer fetch failed");
continue;
}
Err(_) => {
tracing::warn!(peer = %peer.did, "trigger_sync: peer timed out");
continue;
}
};
for repo in repos {
let repo_slug = match (repo.get("owner_did").and_then(|v| v.as_str()),
repo.get("name").and_then(|v| v.as_str())) {
let repo_slug = match (
repo.get("owner_did").and_then(|v| v.as_str()),
repo.get("name").and_then(|v| v.as_str()),
) {
(Some(owner), Some(name)) => {
// Use short owner (last colon segment) matching DB convention
let short = owner.split(':').last().unwrap_or(owner);
let short = owner.split(':').next_back().unwrap_or(owner);
format!("{short}/{name}")
}
_ => continue,
};
let _ = state.db.enqueue_sync(
&repo_slug,
&peer.did,
"refs/heads/main",
"0000000000000000000000000000000000000000",
None,
).await;
let _ = state
.db
.enqueue_sync(
&repo_slug,
&peer.did,
"refs/heads/main",
"0000000000000000000000000000000000000000",
None,
)
.await;
enqueued += 1;
}
}
@@ -155,12 +335,38 @@ pub struct NotifyRequest {
pub ref_name: String,
pub new_sha: String,
pub node_did: String,
// Optional fields — older senders only included the four above. New
// senders include these so received_ref_updates has full provenance
// even when the libp2p mesh isn't delivering and the HTTP fallback
// is the only path that fired.
#[serde(default)]
pub pusher_did: Option<String>,
#[serde(default)]
pub old_sha: Option<String>,
#[serde(default)]
pub timestamp: Option<String>,
#[serde(default)]
pub cert_id: Option<String>,
}
pub async fn notify_sync(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Json(req): Json<NotifyRequest>,
) -> Result<Json<serde_json::Value>> {
if let Some(Extension(auth)) = auth {
if auth.0 != req.node_did {
return Err(AppError::BadRequest(
"Signature keyid must match notification node_did".into(),
));
}
} else {
tracing::warn!(
did = %req.node_did,
"accepted unsigned sync notify; set GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true after all peers upgrade"
);
}
// Only accept notifications from known peers
let peers = state.db.list_peers().await?;
let known = peers.iter().any(|p| p.did == req.node_did);
@@ -171,13 +377,31 @@ pub async fn notify_sync(
)));
}
state.db.enqueue_sync(
&req.repo,
&req.node_did,
&req.ref_name,
&req.new_sha,
None,
).await?;
state
.db
.enqueue_sync(&req.repo, &req.node_did, &req.ref_name, &req.new_sha, None)
.await?;
// Mirror the gossipsub-receive handler: insert the same record we'd
// get from the libp2p path, so /api/v1/events/ref-updates reflects
// pushes that arrive over either transport.
let now = chrono::Utc::now().to_rfc3339();
let update = crate::db::ReceivedRefUpdate {
id: uuid::Uuid::new_v4().to_string(),
node_did: req.node_did.clone(),
pusher_did: req.pusher_did.clone().unwrap_or_default(),
repo: req.repo.clone(),
ref_name: req.ref_name.clone(),
old_sha: req.old_sha.clone().unwrap_or_default(),
new_sha: req.new_sha.clone(),
timestamp: req.timestamp.clone().unwrap_or_else(|| now.clone()),
cert_id: req.cert_id.clone(),
received_at: now,
from_peer: format!("http:{}", req.node_did),
};
if let Err(e) = state.db.insert_ref_update(&update).await {
tracing::warn!(err = %e, repo = %req.repo, "failed to insert ref-update from sync notify");
}
tracing::info!(
repo = %req.repo,
@@ -201,12 +425,23 @@ pub async fn ping_peer(
Path(did): Path<String>,
) -> Result<Json<serde_json::Value>> {
let peers = state.db.list_peers().await?;
let peer = peers.into_iter().find(|p| p.did == did)
let peer = peers
.into_iter()
.find(|p| p.did == did)
.ok_or_else(|| AppError::RepoNotFound(format!("peer {did} not found")))?;
// Async ping
let url = format!("{}/health", peer.http_url.trim_end_matches('/'));
let ok = reqwest::get(&url).await.map(|r| r.status().is_success()).unwrap_or(false);
// Use the shared no-redirect client: bare `reqwest::get` follows redirects,
// so a peer could answer with `302 -> http://127.0.0.1/` and turn the ping
// into an SSRF probe.
let ok = state
.http_client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false);
let _ = state.db.mark_peer_ping(&did, ok).await;
@@ -216,3 +451,90 @@ pub async fn ping_peer(
"reachable": ok,
})))
}
#[cfg(test)]
mod tests {
use super::is_public_http_url;
#[test]
fn accepts_public_https_and_http() {
assert!(is_public_http_url("https://node.gitlawb.com"));
assert!(is_public_http_url("https://manila.gitlawb.com/"));
assert!(is_public_http_url("http://203.0.113.10:7545"));
}
#[test]
fn rejects_loopback_private_and_internal() {
for bad in [
"http://localhost:7545",
"http://127.0.0.1:5432/",
"http://localhost:22/",
"http://0.0.0.0:7545",
// RFC1122 "this host" block 0.0.0.0/8 beyond 0.0.0.0 itself,
// including the upper boundary (1.0.0.0 is public, tested below).
"http://0.0.0.1:7545",
"http://0.1.2.3/",
"http://0.255.255.255/",
"http://10.0.0.5:7545",
"http://192.168.1.10/",
"http://172.16.0.1:7545",
"http://169.254.1.1/",
"http://[::1]:7545",
"http://gitlawb-node.internal:7545",
"http://my-node.local/",
"ftp://node.gitlawb.com",
"not-a-url",
"",
// Trailing-dot FQDN of an internal host.
"http://localhost./",
// CGNAT (100.64.0.0/10).
"http://100.64.0.1:7545",
"http://100.127.255.255/",
// IPv4-mapped / IPv4-compatible IPv6 smuggling private/loopback v4.
"http://[::ffff:127.0.0.1]:7545",
"http://[::ffff:10.0.0.1]/",
"http://[::ffff:192.168.1.1]:7545",
"http://[::127.0.0.1]/",
// 6to4 (2002::/16) embedding loopback/private v4.
"http://[2002:7f00:1::]:7545",
"http://[2002:a00:1::]/",
"http://[2002:c0a8:101::]/",
// NAT64 well-known prefix (64:ff9b::/96) embedding loopback/private v4.
"http://[64:ff9b::7f00:1]/",
"http://[64:ff9b::a00:1]:7545",
// NAT64 local-use prefix (64:ff9b:1::/48, RFC 8215) — rejected
// outright rather than decoded, so the whole NAT64 space is closed.
"http://[64:ff9b:1::7f00:1]/",
"http://[64:ff9b:1::a00:1]:7545",
"http://[64:ff9b:1:a00::]/",
// Local-use NAT64 wrapping a *public* v4 (203.0.113.10) is still
// rejected: the block is prefix-based, not payload-based.
"http://[64:ff9b:1::cb00:710a]/",
] {
assert!(!is_public_http_url(bad), "{bad:?} must be rejected");
}
}
#[test]
fn accepts_6to4_and_nat64_wrapping_public_v4() {
// Fold-and-recheck stays consistent with the mapped/compatible handling:
// a transition address that wraps a public v4 is allowed, only the
// private/loopback-wrapping forms above are rejected. 203.0.113.10.
assert!(is_public_http_url("http://[2002:cb00:710a::]/"));
assert!(is_public_http_url("http://[64:ff9b::cb00:710a]/"));
}
#[test]
fn accepts_public_outside_cgnat_range() {
// 100.0.0.0/10 boundary: .63 and .128 are public, only 100.64-127 is CGNAT.
assert!(is_public_http_url("http://100.63.255.255:7545"));
assert!(is_public_http_url("http://100.128.0.1/"));
}
#[test]
fn accepts_first_address_above_this_host_block() {
// 1.0.0.0 is the first address above the rejected 0.0.0.0/8 block;
// pins the `o[0] == 0` check against an off-by-one into 1.x.
assert!(is_public_http_url("http://1.0.0.0/"));
}
}
+171
View File
@@ -0,0 +1,171 @@
//! Agent profile API handlers.
//!
//! - `PUT /api/v1/profile` — upsert the caller's profile (requires HTTP Signature)
//! - `GET /api/v1/agents/{did}/profile` — read any agent's profile (public)
use axum::{
extract::{Path, State},
http::StatusCode,
Json,
};
use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use crate::auth::AuthenticatedDid;
use crate::pinata;
use crate::state::AppState;
#[derive(Debug, Deserialize)]
pub struct SetProfileRequest {
pub display_name: Option<String>,
pub bio: Option<String>,
pub avatar_url: Option<String>,
pub website: Option<String>,
pub socials: Option<SocialsInput>,
pub pin_to_ipfs: Option<bool>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct SocialsInput {
pub twitter: Option<String>,
pub github: Option<String>,
pub farcaster: Option<String>,
pub telegram: Option<String>,
}
pub async fn set_profile(
State(state): State<AppState>,
axum::Extension(auth): axum::Extension<AuthenticatedDid>,
Json(req): Json<SetProfileRequest>,
) -> Result<(StatusCode, Json<Value>), (StatusCode, Json<Value>)> {
let did = auth.0;
if let Some(ref bio) = req.bio {
if bio.len() > 280 {
return Err((
StatusCode::BAD_REQUEST,
Json(json!({ "message": "bio must be 280 characters or fewer" })),
));
}
}
if let Some(ref name) = req.display_name {
if name.len() > 50 {
return Err((
StatusCode::BAD_REQUEST,
Json(json!({ "message": "display_name must be 50 characters or fewer" })),
));
}
}
let socials_json = req
.socials
.as_ref()
.map(|s| serde_json::to_string(s).unwrap_or_else(|_| "{}".to_string()));
let result = state
.db
.upsert_profile(
&did,
req.display_name.as_deref(),
req.bio.as_deref(),
req.avatar_url.as_deref(),
req.website.as_deref(),
socials_json.as_deref(),
)
.await;
match result {
Ok(profile) => {
let mut resp = json!({
"did": profile.did,
"display_name": profile.display_name,
"bio": profile.bio,
"avatar_url": profile.avatar_url,
"website": profile.website,
"updated_at": profile.updated_at,
});
if let Some(ref socials_str) = profile.socials {
if let Ok(socials) = serde_json::from_str::<Value>(socials_str) {
resp["socials"] = socials;
}
}
if req.pin_to_ipfs.unwrap_or(false) && !state.config.pinata_jwt.is_empty() {
let profile_json = serde_json::to_vec(&resp).unwrap_or_default();
match pinata::pin_object(
&state.http_client,
&state.config.pinata_upload_url,
&state.config.pinata_jwt,
&format!("profile-{}", &did),
&profile_json,
)
.await
{
Ok(cid) if !cid.is_empty() => {
if let Err(e) = state.db.set_profile_cid(&did, &cid).await {
tracing::warn!(did = %did, err = %e, "failed to store profile CID");
}
resp["profile_cid"] = json!(cid);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(did = %did, err = %e, "IPFS pin failed — profile saved without CID");
}
}
} else if let Some(cid) = profile.profile_cid {
resp["profile_cid"] = json!(cid);
}
Ok((StatusCode::OK, Json(resp)))
}
Err(e) => {
tracing::error!(did = %did, error = %e, "failed to upsert profile");
Err((
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "message": "failed to save profile" })),
))
}
}
}
pub async fn get_profile(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<Json<Value>, (StatusCode, Json<Value>)> {
let profile = state.db.get_profile(&did).await.map_err(|e| {
tracing::error!(did = %did, error = %e, "failed to fetch profile");
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "message": "failed to fetch profile" })),
)
})?;
match profile {
Some(p) => {
let mut resp = json!({
"did": p.did,
"display_name": p.display_name,
"bio": p.bio,
"avatar_url": p.avatar_url,
"website": p.website,
"profile_cid": p.profile_cid,
"created_at": p.created_at,
"updated_at": p.updated_at,
});
if let Some(ref socials_str) = p.socials {
if let Ok(socials) = serde_json::from_str::<Value>(socials_str) {
resp["socials"] = socials;
}
}
Ok(Json(resp))
}
None => Err((
StatusCode::NOT_FOUND,
Json(json!({ "message": "profile not found" })),
)),
}
}
+25 -15
View File
@@ -17,14 +17,16 @@ pub async fn protect_branch(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, branch)): Path<(String, String, String)>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Only the repo owner can protect branches
// Only the repo owner can protect branches (DID-safe match, shared idiom).
let caller = &auth.0;
let owner_short = record.owner_did.split(':').next_back().unwrap_or(&record.owner_did);
if caller != &record.owner_did && caller != owner_short {
return Err(AppError::BadRequest(
if !crate::api::did_matches(caller, &record.owner_did) {
return Err(AppError::Forbidden(
"only the repo owner can protect branches".into(),
));
}
@@ -33,11 +35,14 @@ pub async fn protect_branch(
tracing::info!(repo = %repo, branch = %branch, caller = %caller, "branch protected");
Ok((StatusCode::CREATED, Json(serde_json::json!({
"status": "protected",
"repo": format!("{owner}/{repo}"),
"branch": branch,
}))))
Ok((
StatusCode::CREATED,
Json(serde_json::json!({
"status": "protected",
"repo": format!("{owner}/{repo}"),
"branch": branch,
})),
))
}
/// DELETE /api/v1/repos/:owner/:repo/branches/:branch/protect
@@ -46,13 +51,15 @@ pub async fn unprotect_branch(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, branch)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = &auth.0;
let owner_short = record.owner_did.split(':').next_back().unwrap_or(&record.owner_did);
if caller != &record.owner_did && caller != owner_short {
return Err(AppError::BadRequest(
if !crate::api::did_matches(caller, &record.owner_did) {
return Err(AppError::Forbidden(
"only the repo owner can unprotect branches".into(),
));
}
@@ -73,7 +80,10 @@ pub async fn list_protected_branches(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let branches = state.db.list_protected_branches(&record.id).await?;
+140 -37
View File
@@ -8,7 +8,7 @@ use serde::Deserialize;
use uuid::Uuid;
use crate::auth::AuthenticatedDid;
use crate::db::{PullRequest, PrComment, PrReview};
use crate::db::{PrComment, PrReview, PullRequest};
use crate::error::{AppError, Result};
use crate::git::store;
use crate::state::AppState;
@@ -40,11 +40,16 @@ pub async fn create_pr(
Path((owner, name)): Path<(String, String)>,
Json(req): Json<CreatePrRequest>,
) -> Result<(StatusCode, Json<PullRequest>)> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Authorize the caller as a reader before accepting a PR: a non-reader must
// not be able to open a PR (and fire its webhooks) against a private repo
// they cannot read. Mirrors create_review / create_comment / create_bounty.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let author_did = auth.0;
let target_branch = req.target_branch.unwrap_or_else(|| record.default_branch.clone());
let target_branch = req
.target_branch
.unwrap_or_else(|| record.default_branch.clone());
let number = state.db.next_pr_number(&record.id).await?;
let now = Utc::now().to_rfc3339();
@@ -68,7 +73,11 @@ pub async fn create_pr(
// Bump trust score for the PR author — increment current score by 0.05
// (avoids the push_count=0 stuck-at-0.05 bug for agents who only open PRs)
let current = state.db.get_trust_score(&pr.author_did).await.unwrap_or(0.05);
let current = state
.db
.get_trust_score(&pr.author_did)
.await
.unwrap_or(0.05);
let new_score = (current + 0.05).min(1.0);
let _ = state.db.update_trust_score(&pr.author_did, new_score).await;
@@ -92,23 +101,32 @@ pub async fn create_pr(
pub async fn list_prs(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let prs = state.db.list_prs(&record.id).await?;
Ok(Json(serde_json::json!({ "pulls": prs, "count": prs.len() })))
Ok(Json(
serde_json::json!({ "pulls": prs, "count": prs.len() }),
))
}
/// GET /api/v1/repos/:owner/:repo/pulls/:number
pub async fn get_pr(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<PullRequest>> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
Ok(Json(pr))
@@ -118,15 +136,42 @@ pub async fn get_pr(
pub async fn get_pr_diff(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let disk_path = state.repo_store.acquire(&record.owner_did, &record.name).await
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
// Withhold the entire diff if it touches a path the caller cannot read, so a
// PR diff cannot leak private-subtree content of an otherwise-public repo.
let touched = store::branch_diff_names(&disk_path, &pr.target_branch, &pr.source_branch)
.map_err(|e| AppError::Git(e.to_string()))?;
for p in &touched {
let gate = format!("/{}", p.trim_start_matches('/'));
if crate::visibility::visibility_check(
&rules,
record.is_public,
&record.owner_did,
caller,
&gate,
) == crate::visibility::Decision::Deny
{
return Err(AppError::NotFound(format!("PR #{number} not found")));
}
}
let diff = store::branch_diff(&disk_path, &pr.target_branch, &pr.source_branch)
.map_err(|e| AppError::Git(e.to_string()))?;
@@ -143,24 +188,44 @@ pub async fn merge_pr(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, number)): Path<(String, String, i64)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let pr = state.db.get_pr(&record.id, number).await?
// Owner-only merge (N7). Merging writes the served tree, so this is the same
// trust boundary as owner-only push; it subsumes branch protection (a
// non-owner cannot merge to any branch, protected or not).
crate::api::require_repo_owner(&record, &auth.0)?;
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
if pr.status != "open" {
return Err(AppError::BadRequest(format!("PR is already {}", pr.status)));
}
let guard = state.repo_store.acquire_write(&record.owner_did, &record.name).await
let guard = state
.repo_store
.acquire_write(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let disk_path = guard.path().to_path_buf();
let merger_did = auth.0;
let merge_result = store::merge_branch(&disk_path, &pr.target_branch, &pr.source_branch, &merger_did, &pr.title);
let merge_result = store::merge_branch(
&disk_path,
&pr.target_branch,
&pr.source_branch,
&merger_did,
&pr.title,
);
// Always release the advisory lock — even on error
guard.release().await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(merge_result.is_ok()).await;
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
@@ -192,14 +257,30 @@ pub async fn merge_pr(
/// POST /api/v1/repos/:owner/:repo/pulls/:number/close
pub async fn close_pr(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, number)): Path<(String, String, i64)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
// Owner OR author may close (forge norm).
let is_owner = crate::api::require_repo_owner(&record, &auth.0).is_ok();
let is_author = crate::api::did_matches(&auth.0, &pr.author_did);
if !is_owner && !is_author {
return Err(AppError::Forbidden(
"only the repo owner or the PR author can close this PR".into(),
));
}
state.db.close_pr(&pr.id).await?;
webhooks::fire_event(
@@ -224,15 +305,21 @@ pub async fn create_review(
Path((owner, name, number)): Path<(String, String, i64)>,
Json(req): Json<CreateReviewRequest>,
) -> Result<(StatusCode, Json<PrReview>)> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Read-gate: a reviewer must be able to read the repo, but need not own it.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let valid_statuses = ["approved", "changes_requested", "comment"];
if !valid_statuses.contains(&req.status.as_str()) {
return Err(AppError::BadRequest("status must be approved, changes_requested, or comment".into()));
return Err(AppError::BadRequest(
"status must be approved, changes_requested, or comment".into(),
));
}
let review = PrReview {
@@ -266,11 +353,16 @@ pub async fn create_review(
pub async fn list_reviews(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let reviews = state.db.list_pr_reviews(&pr.id).await?;
@@ -285,13 +377,19 @@ pub async fn create_comment(
Json(req): Json<CreateCommentRequest>,
) -> Result<(StatusCode, Json<PrComment>)> {
if req.body.trim().is_empty() {
return Err(AppError::BadRequest("comment body must not be empty".into()));
return Err(AppError::BadRequest(
"comment body must not be empty".into(),
));
}
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Read-gate: a commenter must be able to read the repo, but need not own it.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let comment = PrComment {
@@ -311,11 +409,16 @@ pub async fn create_comment(
pub async fn list_comments(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state.db.get_pr(&record.id, number).await?
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let comments = state.db.list_pr_comments(&pr.id).await?;
+44 -16
View File
@@ -1,8 +1,8 @@
use axum::extract::State;
use axum::http::StatusCode;
use axum::Json;
use serde::{Deserialize, Serialize};
use chrono::Utc;
use serde::{Deserialize, Serialize};
use gitlawb_core::did::Did;
use gitlawb_core::ucan::Ucan;
@@ -35,38 +35,66 @@ pub struct RegisterResponse {
/// Accepts only requests with a valid HTTP Signature (enforced by middleware).
pub async fn register(
State(state): State<AppState>,
axum::Extension(auth): axum::Extension<crate::auth::AuthenticatedDid>,
headers: axum::http::HeaderMap,
Json(req): Json<RegisterRequest>,
) -> Result<(StatusCode, Json<RegisterResponse>)> {
// iCaptcha gate (inert unless ICAPTCHA_MODE is set). Verify up front; spend
// the proof only once the request is admissible, just before the write, so a
// rejected request never burns a valid proof.
let proof = crate::icaptcha::verify_request(&headers, &auth.0)?;
// Parse and validate the DID
let agent_did: Did = req.did.parse()
let agent_did: Did = req
.did
.parse()
.map_err(|e: gitlawb_core::Error| AppError::BadRequest(e.to_string()))?;
// Bind registration to the authenticated signer: an agent may only register
// itself, not create or refresh a trust row for a DID it does not control.
if !crate::api::did_matches(&auth.0, agent_did.as_str()) {
return Err(AppError::Forbidden(
"did must be the authenticated signer".into(),
));
}
// Request is admissible — spend the proof now, immediately before the write.
proof.consume(&state.db).await?;
// Store the agent in the local index
state.db.register_agent(agent_did.as_str(), &req.capabilities).await?;
state
.db
.register_agent(agent_did.as_str(), &req.capabilities)
.await?;
// Grant a small baseline trust score on first registration (verified via HTTP Signature).
// Score grows further with pushes, PRs, and issue activity.
let initial_trust = 0.05;
let _ = state.db.update_trust_score(agent_did.as_str(), initial_trust).await;
let _ = state
.db
.update_trust_score(agent_did.as_str(), initial_trust)
.await;
// Issue a bootstrap UCAN from the node's identity
let ucan = Ucan::bootstrap(&state.node_keypair, agent_did.clone())
.map_err(|e| AppError::Internal(e.into()))?;
let exp = Utc::now() + chrono::Duration::days(30);
let ucan_encoded = ucan.encode()
.map_err(|e| AppError::Internal(e.into()))?;
let ucan_encoded = ucan.encode().map_err(|e| AppError::Internal(e.into()))?;
tracing::info!(did = %agent_did, "registered new agent");
Ok((StatusCode::CREATED, Json(RegisterResponse {
status: "accepted".to_string(),
did: agent_did.to_string(),
ucan: ucan_encoded,
node: format!("{}:{}", state.config.host, state.config.port),
expires: exp.to_rfc3339(),
trust_score: initial_trust,
capabilities: req.capabilities,
message: "welcome to the network, agent.".to_string(),
})))
Ok((
StatusCode::CREATED,
Json(RegisterResponse {
status: "accepted".to_string(),
did: agent_did.to_string(),
ucan: ucan_encoded,
node: format!("{}:{}", state.config.host, state.config.port),
expires: exp.to_rfc3339(),
trust_score: initial_trust,
capabilities: req.capabilities,
message: "welcome to the network, agent.".to_string(),
}),
))
}
+203
View File
@@ -0,0 +1,203 @@
//! Repo replica registration API.
//!
//! Lets a node tell the origin "I'm hosting a replica of your repo." The
//! origin records the (replica DID, replica URL) pair and exposes the list
//! publicly so anyone can see how many nodes are mirroring a given repo.
//!
//! Endpoints:
//! - `PUT /api/v1/repos/:owner/:repo/replicas` (auth) register
//! - `DELETE /api/v1/repos/:owner/:repo/replicas` (auth) unregister
//! - `GET /api/v1/repos/:owner/:repo/replicas` (public) list
//!
//! Auth model: the caller's DID (verified via HTTP Signatures) is the
//! replica's identity. There's no separate "claim this URL" check — replicas
//! self-report their public URL. Operators viewing the list should treat
//! replica URLs as advisory until they actually reach out and verify.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use serde::Deserialize;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
#[derive(Debug, Deserialize)]
pub struct RegisterReplicaRequest {
/// Publicly reachable URL of the replica node (e.g. `https://my-node.example.com`).
pub url: String,
}
/// PUT /api/v1/repos/:owner/:repo/replicas
/// Idempotent — first registration returns 201, subsequent ones update the URL and return 200.
pub async fn register_replica(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<RegisterReplicaRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
validate_replica_url(&req.url)?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replica_did = &auth.0;
// Don't let an owner register themselves as a replica of their own repo
// (did_matches handles the full vs bare did:key owner form).
if crate::api::did_matches(replica_did, &record.owner_did) {
return Err(AppError::BadRequest(
"the repo owner is not a replica of their own repo".into(),
));
}
let inserted = state
.db
.register_replica(&record.id, replica_did, &req.url)
.await?;
let count = state.db.count_replicas(&record.id).await?;
let status = if inserted {
StatusCode::CREATED
} else {
StatusCode::OK
};
tracing::info!(
repo = %repo,
replica = %replica_did,
url = %req.url,
"replica registered"
);
Ok((
status,
Json(serde_json::json!({
"status": "registered",
"repo": format!("{owner}/{repo}"),
"replica_did": replica_did,
"replica_url": req.url,
"replica_count": count,
})),
))
}
/// DELETE /api/v1/repos/:owner/:repo/replicas
/// Idempotent — no error if the caller wasn't registered.
pub async fn unregister_replica(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replica_did = &auth.0;
state.db.unregister_replica(&record.id, replica_did).await?;
let count = state.db.count_replicas(&record.id).await?;
tracing::info!(repo = %repo, replica = %replica_did, "replica unregistered");
Ok(Json(serde_json::json!({
"status": "unregistered",
"repo": format!("{owner}/{repo}"),
"replica_count": count,
})))
}
/// GET /api/v1/repos/:owner/:repo/replicas
/// Public — returns the list of replicas (DID + URL + registration timestamp).
pub async fn list_replicas(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replicas = state.db.list_replicas(&record.id).await?;
Ok(Json(serde_json::json!({
"repo": format!("{owner}/{repo}"),
"replica_count": replicas.len(),
"replicas": replicas,
})))
}
/// Basic URL hygiene — must be http(s), parseable, length-bounded.
fn validate_replica_url(url: &str) -> Result<()> {
if url.is_empty() {
return Err(AppError::BadRequest("replica url is empty".into()));
}
if url.len() > 512 {
return Err(AppError::BadRequest("replica url exceeds 512 chars".into()));
}
if !(url.starts_with("http://") || url.starts_with("https://")) {
return Err(AppError::BadRequest(
"replica url must start with http:// or https://".into(),
));
}
// No spaces / control chars / newlines.
if url.chars().any(|c| c.is_whitespace() || c.is_control()) {
return Err(AppError::BadRequest(
"replica url contains whitespace or control characters".into(),
));
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn url_accepts_normal_https() {
validate_replica_url("https://node.example.com").unwrap();
validate_replica_url("https://my-node.example.com:7545").unwrap();
validate_replica_url("http://localhost:7545").unwrap();
}
#[test]
fn url_rejects_empty() {
assert!(validate_replica_url("").is_err());
}
#[test]
fn url_rejects_non_http_scheme() {
for bad in [
"ftp://host",
"file:///etc/passwd",
"javascript:alert(1)",
"/path",
] {
assert!(validate_replica_url(bad).is_err(), "{bad:?} must reject");
}
}
#[test]
fn url_rejects_whitespace_and_control() {
for bad in [
"https://host .com",
"https://host\n.com",
"https://host\t.com",
"https://host\0evil.com",
] {
assert!(validate_replica_url(bad).is_err(), "{bad:?} must reject");
}
}
#[test]
fn url_rejects_overlong() {
let long = format!("https://{}.com", "a".repeat(600));
assert!(validate_replica_url(&long).is_err());
}
}
File diff suppressed because it is too large. Load diff
+25 -9
View File
@@ -19,22 +19,32 @@ pub async fn star_repo(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = &auth.0;
let inserted = state.db.star_repo(&record.id, caller).await?;
let count = state.db.count_stars(&record.id).await?;
let status = if inserted { StatusCode::CREATED } else { StatusCode::OK };
let status = if inserted {
StatusCode::CREATED
} else {
StatusCode::OK
};
tracing::info!(repo = %repo, caller = %caller, "repo starred");
Ok((status, Json(serde_json::json!({
"status": "starred",
"repo": format!("{owner}/{repo}"),
"star_count": count,
}))))
Ok((
status,
Json(serde_json::json!({
"status": "starred",
"repo": format!("{owner}/{repo}"),
"star_count": count,
})),
))
}
/// DELETE /api/v1/repos/:owner/:repo/star
@@ -44,7 +54,10 @@ pub async fn unstar_repo(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = &auth.0;
@@ -66,7 +79,10 @@ pub async fn get_star_status(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &repo).await?
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let count = state.db.count_stars(&record.id).await?;
+85 -19
View File
@@ -9,7 +9,7 @@
//! POST /api/v1/tasks/{id}/fail — fail task
use axum::{
extract::{Path, Query, State},
extract::{Extension, Path, Query, State},
http::StatusCode,
Json,
};
@@ -18,9 +18,18 @@ use serde::Deserialize;
use serde_json::{json, Value};
use uuid::Uuid;
use crate::auth::AuthenticatedDid;
use crate::db::AgentTask;
use crate::state::{AppState, TaskEventBroadcast};
/// 403 in this module's error shape (`(StatusCode, Json<Value>)`, not `AppError`).
fn forbidden(msg: &str) -> (StatusCode, Json<Value>) {
(
StatusCode::FORBIDDEN,
Json(json!({ "error": "forbidden", "message": msg })),
)
}
// ── Request / response types ──────────────────────────────────────────────────
#[derive(Deserialize)]
@@ -55,13 +64,11 @@ pub struct ClaimTaskBody {
#[derive(Deserialize)]
pub struct CompleteTaskBody {
pub result: Option<String>,
pub by_did: Option<String>,
}
#[derive(Deserialize)]
pub struct FailTaskBody {
pub reason: Option<String>,
pub by_did: Option<String>,
}
fn task_to_json(t: &AgentTask) -> Value {
@@ -87,15 +94,20 @@ fn task_to_json(t: &AgentTask) -> Value {
/// POST /api/v1/tasks
pub async fn create_task(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Json(body): Json<CreateTaskBody>,
) -> Result<(StatusCode, Json<Value>), (StatusCode, Json<Value>)> {
// Bind the delegator to the authenticated signer (N13).
if !crate::api::did_matches(&auth.0, &body.delegator_did) {
return Err(forbidden("delegator_did must be the authenticated signer"));
}
let now = Utc::now().to_rfc3339();
let task = AgentTask {
id: Uuid::new_v4().to_string(),
repo_id: body.repo_id,
kind: body.kind,
status: "pending".to_string(),
delegator_did: body.delegator_did,
delegator_did: auth.0,
assignee_did: body.assignee_did,
capability: body.capability,
ucan_token: body.ucan_token,
@@ -154,24 +166,25 @@ pub async fn get_task(
/// POST /api/v1/tasks/{id}/claim
pub async fn claim_task(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
Json(body): Json<ClaimTaskBody>,
) -> Result<Json<Value>, (StatusCode, Json<Value>)> {
let task = state
.db
.claim_task(&id, &body.assignee_did)
.await
.map_err(|e| {
(
StatusCode::CONFLICT,
Json(json!({ "error": e.to_string() })),
)
})?;
// Bind the assignee to the authenticated signer (N13).
if !crate::api::did_matches(&auth.0, &body.assignee_did) {
return Err(forbidden("assignee_did must be the authenticated signer"));
}
let task = state.db.claim_task(&id, &auth.0).await.map_err(|e| {
(
StatusCode::CONFLICT,
Json(json!({ "error": e.to_string() })),
)
})?;
let _ = state.task_event_tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did: body.assignee_did,
by_did: auth.0,
at: Utc::now().to_rfc3339(),
});
Ok(Json(task_to_json(&task)))
@@ -180,17 +193,44 @@ pub async fn claim_task(
/// POST /api/v1/tasks/{id}/complete
pub async fn complete_task(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
Json(body): Json<CompleteTaskBody>,
) -> Result<Json<Value>, (StatusCode, Json<Value>)> {
let by_did = body.by_did.unwrap_or_default();
// Authorize the actor, not just bind their identity: the N13 signer-binding
// proved the caller was whoever they claimed, but never that they were the
// task's assignee. Load the task and require the caller to be its assignee;
// finish_task then transitions only a claimed task.
let existing = state
.db
.get_task(&id)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "error": e.to_string() })),
)
})?
.ok_or_else(|| {
(
StatusCode::NOT_FOUND,
Json(json!({ "error": "task not found" })),
)
})?;
if !crate::api::did_matches(
&auth.0,
existing.assignee_did.as_deref().unwrap_or_default(),
) {
return Err(forbidden("only the task assignee can complete it"));
}
let by_did = auth.0;
let task = state
.db
.finish_task(&id, "completed", body.result.as_deref())
.await
.map_err(|e| {
(
StatusCode::UNPROCESSABLE_ENTITY,
StatusCode::CONFLICT,
Json(json!({ "error": e.to_string() })),
)
})?;
@@ -207,10 +247,36 @@ pub async fn complete_task(
/// POST /api/v1/tasks/{id}/fail
pub async fn fail_task(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
Json(body): Json<FailTaskBody>,
) -> Result<Json<Value>, (StatusCode, Json<Value>)> {
let by_did = body.by_did.unwrap_or_default();
// Authorize the actor, not just bind their identity (see complete_task): only
// the task's assignee may fail it, and finish_task transitions only a claimed
// task.
let existing = state
.db
.get_task(&id)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "error": e.to_string() })),
)
})?
.ok_or_else(|| {
(
StatusCode::NOT_FOUND,
Json(json!({ "error": "task not found" })),
)
})?;
if !crate::api::did_matches(
&auth.0,
existing.assignee_did.as_deref().unwrap_or_default(),
) {
return Err(forbidden("only the task assignee can fail it"));
}
let by_did = auth.0;
let reason = body.reason.unwrap_or_default();
let task = state
.db
@@ -218,7 +284,7 @@ pub async fn fail_task(
.await
.map_err(|e| {
(
StatusCode::UNPROCESSABLE_ENTITY,
StatusCode::CONFLICT,
Json(json!({ "error": e.to_string() })),
)
})?;
+275
View File
@@ -0,0 +1,275 @@
//! Path-scoped visibility management API. Owner-only, mirrors `api/protect.rs`.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use serde::Deserialize;
use crate::auth::AuthenticatedDid;
use crate::db::VisibilityMode;
use crate::error::{AppError, Result};
use crate::state::AppState;
#[derive(Deserialize)]
pub struct SetVisibilityRequest {
pub path_glob: String,
/// "a" or "b"; defaults to "b" if absent or unrecognized.
#[serde(default)]
pub mode: Option<String>,
#[serde(default)]
pub reader_dids: Vec<String>,
}
#[derive(Deserialize)]
pub struct RemoveVisibilityRequest {
pub path_glob: String,
}
fn require_owner(record: &crate::db::RepoRecord, caller: &str) -> Result<()> {
// DID-safe owner match (collapses did:key full vs bare on both sides, never
// across methods), shared with require_repo_owner — not a trailing-segment
// compare that only normalized the owner side.
if crate::api::did_matches(caller, &record.owner_did) {
Ok(())
} else {
Err(AppError::Forbidden(
"only the repo owner can manage visibility".into(),
))
}
}
/// Reject malformed globs before they reach the store, where they would
/// silently misconfigure access (an empty glob behaves like "/", and a glob
/// without a leading "/" never matches a real repo path). The accepted forms
/// match what `visibility_check` understands: "/", "/prefix", or "/prefix/**".
fn validate_path_glob(path_glob: &str) -> Result<()> {
if !path_glob.starts_with('/') {
return Err(AppError::BadRequest("path_glob must start with '/'".into()));
}
if path_glob == "/**" {
return Err(AppError::BadRequest(
"use '/' for whole-repo scope, not '/**'".into(),
));
}
if path_glob != "/" && path_glob.ends_with('/') {
return Err(AppError::BadRequest(
"path_glob must not end with '/'".into(),
));
}
// Reject empty path segments. `glob_prefix` collapses "//**" to "/" (whole
// repo), so without this an operator could bypass the "/**" guard above and
// grant repo-wide read with "//**", or set a dead rule like "//secret/**"
// whose prefix never matches a real single-slash path and silently leaves
// the subtree readable.
if path_glob.contains("//") {
return Err(AppError::BadRequest(
"path_glob must not contain empty path segments".into(),
));
}
// Strip the one permitted trailing "/**" before checking for stray
// wildcards, otherwise a "*" elsewhere in the prefix (e.g. "/a*b/**" or
// "/a/**/**") slips through because the string still ends with "/**".
let prefix = path_glob.strip_suffix("/**").unwrap_or(path_glob);
if prefix.contains('*') {
return Err(AppError::BadRequest(
"the only supported wildcard is a single trailing '/**'".into(),
));
}
Ok(())
}
/// PUT /api/v1/repos/{owner}/{repo}/visibility
pub async fn set_visibility(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<SetVisibilityRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
validate_path_glob(&req.path_glob)?;
let mode = match req.mode.as_deref() {
Some("a") => VisibilityMode::A,
_ => VisibilityMode::B,
};
// Mode A hides existence and is only coherent for the whole repo; a subtree
// cannot hide its existence without rewriting git history (see spec).
if mode == VisibilityMode::A && req.path_glob != "/" {
return Err(AppError::BadRequest(
"mode 'a' (hide) is only allowed for the whole repo (path_glob '/'); use mode 'b' for subtrees".into(),
));
}
// An empty reader_dids list is valid and intentional: the owner is always
// allowed by visibility_check, so a "/" rule with no readers is exactly the
// whole-repo "private to owner only" case.
state
.db
.set_visibility_rule(&record.id, &req.path_glob, mode, &req.reader_dids, &auth.0)
.await?;
let subtree_warning = req.path_glob != "/";
tracing::info!(
repo = %repo, caller = %auth.0, path_glob = %req.path_glob, mode = %mode.as_str(),
subtree_pending = subtree_warning,
"visibility rule set"
);
Ok((
StatusCode::CREATED,
Json(serde_json::json!({
"status": "set",
"repo": format!("{owner}/{repo}"),
"path_glob": req.path_glob,
"mode": mode.as_str(),
"reader_dids": req.reader_dids,
"subtree_clone_enforcement_pending": subtree_warning,
})),
))
}
/// DELETE /api/v1/repos/{owner}/{repo}/visibility
pub async fn remove_visibility(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<RemoveVisibilityRequest>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
state
.db
.remove_visibility_rule(&record.id, &req.path_glob)
.await?;
tracing::info!(
repo = %repo, caller = %auth.0, path_glob = %req.path_glob,
"visibility rule removed"
);
Ok(Json(serde_json::json!({
"status": "removed",
"repo": format!("{owner}/{repo}"),
"path_glob": req.path_glob,
})))
}
/// GET /api/v1/repos/{owner}/{repo}/visibility
pub async fn list_visibility(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let rules_json: Vec<_> = rules
.into_iter()
.map(|r| {
serde_json::json!({
"path_glob": r.path_glob,
"mode": r.mode.as_str(),
"reader_dids": r.reader_dids,
"created_by": r.created_by,
})
})
.collect();
Ok(Json(serde_json::json!({
"repo": format!("{owner}/{repo}"),
"count": rules_json.len(),
"rules": rules_json,
})))
}
/// GET /api/v1/repos/{owner}/{repo}/withheld-paths
///
/// Returns the path globs the (optionally authenticated) caller is denied
/// (`withheld`) plus any more-specific globs that are allowed underneath a
/// denied one (`reinclude`), so a clean-clone client can sparse-exclude the
/// denied subtrees while re-including the allowed nested paths. Unlike
/// `list_visibility` this is not owner-gated and never exposes reader_dids.
pub async fn withheld_paths(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
// Whole-repo read gate: a caller who cannot read "/" gets repo-not-found,
// matching the git read endpoints, so this never discloses a private repo's
// existence or its path layout to an unauthorized caller.
if crate::visibility::visibility_check(&rules, record.is_public, &record.owner_did, caller, "/")
== crate::visibility::Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let withheld =
crate::visibility::withheld_globs(&rules, record.is_public, &record.owner_did, caller);
let reinclude =
crate::visibility::reincluded_globs(&rules, record.is_public, &record.owner_did, caller);
Ok(Json(serde_json::json!({
"repo": format!("{owner}/{repo}"),
"withheld": withheld,
"reinclude": reinclude,
})))
}
#[cfg(test)]
mod tests {
use super::validate_path_glob;
#[test]
fn accepts_supported_globs() {
for g in ["/", "/secret", "/secret/**", "/a/b/c", "/a/b/**"] {
assert!(validate_path_glob(g).is_ok(), "{g} should be valid");
}
}
#[test]
fn rejects_malformed_globs() {
// empty, no leading slash, whole-repo via "/**", trailing slash,
// non-trailing wildcards, double wildcards, and empty path segments
// (which would collapse to whole-repo or a dead rule) are all rejected.
for g in [
"",
"secret/**",
"/**",
"/**/**",
"/secret/",
"/a*b",
"/*/x",
"/a*b/**",
"/a/**/**",
"//**",
"//secret/**",
] {
assert!(validate_path_glob(g).is_err(), "{g} should be rejected");
}
}
}
+72 -12
View File
@@ -33,12 +33,22 @@ pub async fn create_webhook(
Path((owner, name)): Path<(String, String)>,
Json(req): Json<CreateWebhookRequest>,
) -> Result<(StatusCode, Json<Webhook>)> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
// Validate URL is http/https
if !req.url.starts_with("http://") && !req.url.starts_with("https://") {
return Err(AppError::BadRequest("webhook URL must be http:// or https://".into()));
// Gate the target through the same hardened public-host validator the peer
// announce path uses, so an authenticated owner cannot register a webhook
// that makes the node POST to loopback/private/link-local/metadata
// endpoints (SSRF). Delivery runs on the shared no-redirect client
// (main.rs), which closes the 3xx-to-internal bounce.
if !crate::api::peers::is_public_http_url(&req.url) {
return Err(AppError::BadRequest(
"webhook URL must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(),
));
}
let events = req.events.unwrap_or_else(|| vec!["*".into()]);
@@ -64,29 +74,47 @@ pub async fn list_webhooks(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let hooks = state.db.list_webhooks(&record.id).await?;
// Redact secrets in list response
let redacted: Vec<_> = hooks.into_iter().map(|mut h| {
if h.secret.is_some() { h.secret = Some("***".into()); }
h
}).collect();
let redacted: Vec<_> = hooks
.into_iter()
.map(|mut h| {
if h.secret.is_some() {
h.secret = Some("***".into());
}
h
})
.collect();
Ok(Json(serde_json::json!({ "webhooks": redacted, "count": redacted.len() })))
Ok(Json(
serde_json::json!({ "webhooks": redacted, "count": redacted.len() }),
))
}
/// DELETE /api/v1/repos/:owner/:repo/hooks/:id
pub async fn delete_webhook(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state.db.get_repo(&owner, &name).await?
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
// Verify the webhook belongs to this repo
let hook = state.db.get_webhook(&id).await?
let hook = state
.db
.get_webhook(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("webhook {id} not found")))?;
if hook.repo_id != record.id {
@@ -96,3 +124,35 @@ pub async fn delete_webhook(
state.db.delete_webhook(&id).await?;
Ok(Json(serde_json::json!({ "deleted": true, "id": id })))
}
#[cfg(test)]
mod tests {
use crate::api::peers::is_public_http_url;
// create_webhook gates req.url through is_public_http_url. Pin the exact
// SSRF targets from issue #81 so the webhook path can never regress to the
// old scheme-only check, independent of the validator's own peer tests.
#[test]
fn webhook_url_gate_rejects_ssrf_targets() {
for bad in [
"http://127.0.0.1:5432/",
"http://169.254.169.254/latest/meta-data/",
"http://localhost/",
"http://10.0.0.5/",
"http://[::1]/",
// IPv6 transition encodings smuggling loopback v4 (6to4 / NAT64).
"http://[2002:7f00:1::]/",
"http://[64:ff9b::7f00:1]/",
"ftp://example.com/",
"not-a-url",
] {
assert!(!is_public_http_url(bad), "{bad:?} must be rejected");
}
}
#[test]
fn webhook_url_gate_allows_public_targets() {
assert!(is_public_http_url("https://hooks.example.com/gitlawb"));
assert!(is_public_http_url("http://203.0.113.10:7545/"));
}
}
+226 -8
View File
@@ -20,7 +20,6 @@
use anyhow::Result;
use serde_json::json;
/// Data describing a ref-update event to be anchored.
#[derive(Debug, Clone)]
pub struct RefAnchor {
@@ -104,6 +103,108 @@ pub async fn anchor_ref_update(
Ok(tx_id)
}
/// A per-push manifest of the blobs encrypted this push (Option B3). The
/// `blobs` slice is `(oid, cid)` tuples. Anchored directly to Arweave as its JSON
/// body so the discovery index survives total node loss. Recipient identities are
/// never part of the manifest.
pub struct EncryptedManifest<'a> {
pub repo: &'a str,
pub owner_did: &'a str,
pub node_did: &'a str,
pub timestamp: &'a str,
pub blobs: &'a [(String, String)],
}
/// Anchor a per-push encrypted-blob manifest to Arweave via Irys. The manifest
/// JSON body is the payload (not a CID pointer to IPFS), so the index is
/// permanent and self-contained. Recipient identities are deliberately omitted:
/// the anchor is permanent and public, and the v2 envelopes no longer expose
/// recipients, so the reader set must not be written to Arweave either.
///
/// Returns the Irys/Arweave transaction ID, or `Ok("")` when `irys_url` is empty
/// (anchoring disabled) or there are no blobs to anchor.
pub async fn anchor_encrypted_manifest(
client: &reqwest::Client,
irys_url: &str,
manifest: &EncryptedManifest<'_>,
) -> Result<String> {
if irys_url.is_empty() || manifest.blobs.is_empty() {
return Ok(String::new());
}
let blobs_json: Vec<serde_json::Value> = manifest
.blobs
.iter()
.map(|(oid, cid)| manifest_blob_json(oid, cid))
.collect();
let payload = json!({
"schema": "gitlawb/encrypted-manifest/v1",
"repo": manifest.repo,
"owner_did": manifest.owner_did,
"node_did": manifest.node_did,
"timestamp": manifest.timestamp,
"blobs": blobs_json,
});
let body = serde_json::to_vec(&payload)?;
let url = format!("{}/upload", irys_url.trim_end_matches('/'));
let resp = client
.post(&url)
.header("Content-Type", "application/json")
.header("x-irys-tags", build_manifest_tags_header(manifest))
.body(body)
.send()
.await
.map_err(|e| anyhow::anyhow!("Irys upload failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(anyhow::anyhow!("Irys returned {status}: {body}"));
}
let json: serde_json::Value = resp
.json()
.await
.map_err(|e| anyhow::anyhow!("failed to parse Irys response: {e}"))?;
let tx_id = json["id"]
.as_str()
.ok_or_else(|| anyhow::anyhow!("no 'id' in Irys response: {json}"))?
.to_string();
tracing::info!(
repo = %manifest.repo,
tx_id = %tx_id,
blobs = manifest.blobs.len(),
"anchored encrypted manifest to Arweave"
);
Ok(tx_id)
}
/// Serialize one blob for the Arweave manifest. Recipient identities are
/// intentionally absent so the permanent public anchor never records who can
/// read a blob.
fn manifest_blob_json(oid: &str, cid: &str) -> serde_json::Value {
json!({ "oid": oid, "cid": cid })
}
/// Build the Irys tag header for an encrypted-blob manifest. `Repo` and `Schema`
/// are the tags the `gl` recovery query filters on.
fn build_manifest_tags_header(manifest: &EncryptedManifest<'_>) -> String {
[
"App-Name:gitlawb".to_string(),
"Schema:gitlawb/encrypted-manifest/v1".to_string(),
format!("Repo:{}", sanitize_tag(manifest.repo)),
format!("Owner-DID:{}", sanitize_tag(manifest.owner_did)),
format!("Node-DID:{}", sanitize_tag(manifest.node_did)),
]
.join(",")
}
/// Arweave permanent URL for a given Irys transaction ID.
pub fn arweave_url(tx_id: &str) -> String {
format!("https://arweave.net/{tx_id}")
@@ -112,9 +213,9 @@ pub fn arweave_url(tx_id: &str) -> String {
/// Build the Irys tag header value for Arweave indexing.
/// Format: comma-separated "name:value" pairs.
fn build_tags_header(anchor: &RefAnchor) -> String {
vec![
format!("App-Name:gitlawb"),
format!("Schema:gitlawb/ref-update/v1"),
[
"App-Name:gitlawb".to_string(),
"Schema:gitlawb/ref-update/v1".to_string(),
format!("Repo:{}", sanitize_tag(&anchor.repo)),
format!("Ref:{}", sanitize_tag(&anchor.ref_name)),
format!("SHA:{}", &anchor.new_sha[..anchor.new_sha.len().min(16)]),
@@ -142,7 +243,7 @@ mod tests {
repo: "alice/myrepo".into(),
owner_did: "did:key:z6Mk...".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0000000000000000000000000000000000000000000000000000000000000000".into(),
old_sha: "0000000000000000000000000000000000000000".into(),
new_sha: "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2".into(),
cid: Some("bafyreib5...".into()),
timestamp: "2026-03-14T00:00:00Z".into(),
@@ -169,7 +270,7 @@ mod tests {
repo: "alice/myrepo".into(),
owner_did: "did:key:z6Mk...".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(64),
old_sha: "0".repeat(40),
new_sha: "a1b2c3d4".repeat(8),
cid: None,
timestamp: "2026-03-14T00:00:00Z".into(),
@@ -178,14 +279,131 @@ mod tests {
let result = anchor_ref_update(&client, &server.url(), &anchor).await;
assert!(result.is_ok(), "anchor should succeed: {result:?}");
assert_eq!(result.unwrap(), "7xGpIoHUQ8j9GhD3Y2mKzP1NsVtXwRcFe4bEaLnMuOk");
assert_eq!(
result.unwrap(),
"7xGpIoHUQ8j9GhD3Y2mKzP1NsVtXwRcFe4bEaLnMuOk"
);
_mock.assert_async().await;
}
#[tokio::test]
async fn test_anchor_body_carries_real_old_sha() {
// The anchored body must serialize the real old→new transition the
// node was handed, never a zero placeholder. Regression guard for the
// push handler that used to hardcode `old_sha` to 64 zeros (#26).
let mut server = mockito::Server::new_async().await;
let real_old = "1111111111111111111111111111111111111111";
let real_new = "2222222222222222222222222222222222222222";
let _mock = server
.mock("POST", "/upload")
.match_body(mockito::Matcher::AllOf(vec![
mockito::Matcher::PartialJsonString(format!(r#"{{"old_sha":"{real_old}"}}"#)),
mockito::Matcher::PartialJsonString(format!(r#"{{"new_sha":"{real_new}"}}"#)),
]))
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"id":"TX_REAL_OLD_SHA","timestamp":1710000000000,"version":"1.0.0"}"#)
.create_async()
.await;
let client = reqwest::Client::new();
let anchor = RefAnchor {
repo: "alice/myrepo".into(),
owner_did: "did:key:z6Mk...".into(),
ref_name: "refs/heads/main".into(),
old_sha: real_old.into(),
new_sha: real_new.into(),
cid: None,
timestamp: "2026-03-14T00:00:00Z".into(),
node_did: "did:key:z6Mknnd...".into(),
};
let result = anchor_ref_update(&client, &server.url(), &anchor).await;
assert_eq!(result.unwrap(), "TX_REAL_OLD_SHA");
// The mock only matches when the posted JSON carries both real SHAs.
_mock.assert_async().await;
}
#[test]
fn test_arweave_url() {
let url = arweave_url("7xGpIoHUQ8j9GhD3Y2mKzP1NsVtXwRcFe4bEaLnMuOk");
assert_eq!(url, "https://arweave.net/7xGpIoHUQ8j9GhD3Y2mKzP1NsVtXwRcFe4bEaLnMuOk");
assert_eq!(
url,
"https://arweave.net/7xGpIoHUQ8j9GhD3Y2mKzP1NsVtXwRcFe4bEaLnMuOk"
);
}
#[tokio::test]
async fn test_manifest_anchor_noop_when_url_empty() {
let client = reqwest::Client::new();
let blobs = vec![("oid1".to_string(), "cid1".to_string())];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
assert_eq!(
anchor_encrypted_manifest(&client, "", &m).await.unwrap(),
""
);
}
#[tokio::test]
async fn test_manifest_anchor_noop_when_no_blobs() {
let client = reqwest::Client::new();
let blobs: Vec<(String, String)> = vec![];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
// Non-empty URL, but no blobs: still a no-op.
assert_eq!(
anchor_encrypted_manifest(&client, "https://example.invalid", &m)
.await
.unwrap(),
""
);
}
#[tokio::test]
async fn test_manifest_anchor_success() {
let mut server = mockito::Server::new_async().await;
let _mock = server
.mock("POST", "/upload")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"id":"MANIFESTTX123","timestamp":1710000000000,"version":"1.0.0"}"#)
.create_async()
.await;
let client = reqwest::Client::new();
let blobs = vec![("oid1".to_string(), "cid1".to_string())];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
let r = anchor_encrypted_manifest(&client, &server.url(), &m).await;
assert_eq!(r.unwrap(), "MANIFESTTX123");
_mock.assert_async().await;
}
#[test]
fn manifest_blob_json_omits_recipients() {
let v = manifest_blob_json("oid1", "cidA");
assert_eq!(v["oid"], "oid1");
assert_eq!(v["cid"], "cidA");
assert!(
v.get("recipients").is_none(),
"Arweave manifest must not anchor recipient identities"
);
}
#[test]
+393 -26
View File
@@ -1,5 +1,5 @@
use axum::body::Body;
use axum::extract::Request;
use axum::extract::{Request, State};
use axum::http::StatusCode;
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
@@ -8,10 +8,27 @@ use http_body_util::BodyExt;
use serde_json::json;
use std::collections::HashMap;
use gitlawb_core::did::Did;
use gitlawb_core::ucan::Ucan;
use crate::state::AppState;
/// The authenticated agent's DID, injected into request extensions by `require_signature`.
#[derive(Clone, Debug)]
pub struct AuthenticatedDid(pub String);
/// Whether `caller` is authorized to push to `record`.
///
/// Phase 1 (`GITLAWB_ENFORCE_OWNER_PUSH`): owner-only, via the canonical
/// [`crate::api::did_matches`] owner comparison (DID-safe on both sides). This is
/// intentionally a distinct, intent-named gate rather than a bare owner check so
/// that Phase 2 can extend it to honor a verified UCAN `git/push` capability as a
/// pure addition (`did_matches(..) || ucan_grants_push(..)`) without rewriting
/// call sites.
pub fn caller_authorized_to_push(record: &crate::db::RepoRecord, caller: &str) -> bool {
crate::api::did_matches(caller, &record.owner_did)
}
use gitlawb_core::http_sig::{
build_signing_string, compute_content_digest, HttpSignature, COVERED_COMPONENTS,
};
@@ -34,16 +51,17 @@ use gitlawb_core::identity::verify;
pub async fn require_signature(request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream
let (parts, body) = request.into_parts();
let body_bytes = match body.collect().await {
Ok(collected) => collected.to_bytes(),
Err(_) => {
return (
let body_bytes =
match body.collect().await {
Ok(collected) => collected.to_bytes(),
Err(_) => return (
StatusCode::BAD_REQUEST,
Json(json!({ "error": "unreadable_body", "message": "could not read request body" })),
Json(
json!({ "error": "unreadable_body", "message": "could not read request body" }),
),
)
.into_response()
}
};
.into_response(),
};
let sig_input = parts
.headers
@@ -61,8 +79,9 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
(Some(i), Some(s)) => (i, s),
_ => {
return human_detected(
"missing Signature-Input or Signature headers — use RFC 9421 HTTP Signatures"
).into_response();
"missing Signature-Input or Signature headers — use RFC 9421 HTTP Signatures",
)
.into_response();
}
};
@@ -156,22 +175,21 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
request_values.insert("content-digest".to_string(), content_digest);
// The @signature-params value is the part of Signature-Input after "sig1="
let sig_params_value = sig_input
.strip_prefix("sig1=")
.unwrap_or(&sig_input);
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(&sig_input);
let components_ref: Vec<&str> = sig.components.iter().map(String::as_str).collect();
let signing_string = match build_signing_string(&components_ref, sig_params_value, &request_values) {
Ok(s) => s,
Err(e) => {
return (
StatusCode::BAD_REQUEST,
Json(json!({ "error": "signing_string_error", "message": e.to_string() })),
)
.into_response()
}
};
let signing_string =
match build_signing_string(&components_ref, sig_params_value, &request_values) {
Ok(s) => s,
Err(e) => {
return (
StatusCode::BAD_REQUEST,
Json(json!({ "error": "signing_string_error", "message": e.to_string() })),
)
.into_response()
}
};
// Verify Ed25519 signature
let sig_array: [u8; 64] = match sig.signature_bytes.as_slice().try_into() {
@@ -200,7 +218,11 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
}
// Verify Content-Digest matches the actual request body
if let Some(claimed) = parts.headers.get("content-digest").and_then(|v| v.to_str().ok()) {
if let Some(claimed) = parts
.headers
.get("content-digest")
.and_then(|v| v.to_str().ok())
{
let actual = compute_content_digest(&body_bytes);
if claimed != actual {
return (
@@ -217,7 +239,130 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
tracing::info!(did = %sig.key_id, "✓ authenticated request");
let mut request = Request::from_parts(parts, Body::from(body_bytes));
request.extensions_mut().insert(AuthenticatedDid(sig.key_id.to_string()));
request
.extensions_mut()
.insert(AuthenticatedDid(sig.key_id.to_string()));
next.run(request).await
}
/// Optional variant for rolling upgrades: verify and inject `AuthenticatedDid` when
/// RFC 9421 signature headers are present, but allow legacy unsigned requests to
/// continue when no signature attempt was made.
pub async fn optional_signature(request: Request, next: Next) -> Response {
let has_signature_headers = request.headers().contains_key("signature-input")
|| request.headers().contains_key("signature");
if has_signature_headers {
return require_signature(request, next).await;
}
next.run(request).await
}
/// Validate a raw UCAN token string supplied in `X-Ucan`.
///
/// Checks performed:
/// 1. The token decodes to a valid [`Ucan`] structure.
/// 2. The UCAN issuer (`iss`) matches `signer_did` — the DID that signed the
/// HTTP request — preventing replay of another agent's UCAN.
/// 3. The UCAN audience (`aud`) matches `expected_aud` — the node's own DID.
/// 4. The full proof chain is cryptographically valid (signatures, expiry,
/// not-before, chain linkage, and capability attenuation).
fn validate_ucan_chain(
token: &str,
expected_aud: &Did,
signer_did: &Did,
) -> Result<(), (StatusCode, Json<serde_json::Value>)> {
let ucan = Ucan::decode(token).map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
if &ucan.payload.iss != signer_did {
return Err((
StatusCode::UNAUTHORIZED,
Json(json!({
"error": "invalid_ucan",
"message": format!(
"UCAN issuer {} does not match request signer {}",
ucan.payload.iss, signer_did
),
})),
));
}
ucan.verify_audience(expected_aud).map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
ucan.verify_chain().map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
Ok(())
}
/// Axum middleware that validates a UCAN chain when `X-Ucan` is present.
///
/// Must be layered so that it runs after [`require_signature`], which sets the
/// [`AuthenticatedDid`] extension consumed here.
///
/// When `X-Ucan` is absent the request passes through unchanged, preserving
/// backward compatibility for agents that pre-date UCAN delegation. When the
/// header is present the full chain is validated: the UCAN issuer must match
/// the HTTP Signature identity, the audience must be this node's DID, and
/// every proof in the chain must be cryptographically sound with no capability
/// escalation.
pub async fn require_ucan_chain(
State(state): State<AppState>,
request: Request,
next: Next,
) -> Response {
let token = match request
.headers()
.get("x-ucan")
.and_then(|v| v.to_str().ok())
.map(str::to_owned)
{
Some(t) => t,
None => return next.run(request).await,
};
let signer_did: Did = match request.extensions().get::<AuthenticatedDid>() {
Some(a) => match a.0.parse() {
Ok(did) => did,
Err(e) => {
tracing::warn!(raw_did = %a.0, err = %e, "failed to parse DID from authenticated identity");
return (
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_identity", "message": "invalid DID in token" })),
)
.into_response();
}
},
None => {
return (
StatusCode::UNAUTHORIZED,
Json(json!({
"error": "invalid_ucan",
"message": "UCAN validation requires a valid HTTP Signature",
})),
)
.into_response()
}
};
if let Err((status, body)) = validate_ucan_chain(&token, &state.node_did, &signer_did) {
return (status, body).into_response();
}
tracing::debug!(did = %signer_did, "UCAN chain validated");
next.run(request).await
}
@@ -239,3 +384,225 @@ fn human_detected(message: &str) -> impl IntoResponse {
})),
)
}
#[cfg(test)]
mod tests {
use super::*;
use axum::{middleware, Router};
use gitlawb_core::identity::Keypair;
use gitlawb_core::ucan::{caps, Capability, Ucan};
use std::{path::PathBuf, sync::Arc, time::Duration};
use tower::ServiceExt;
fn bootstrap_ucan(node: &Keypair, agent_did: Did) -> Ucan {
Ucan::bootstrap(node, agent_did).unwrap()
}
fn delegation_ucan(agent: &Keypair, node_did: Did, proof: &Ucan) -> Ucan {
Ucan::delegate(
agent,
node_did,
vec![Capability::new("gitlawb://alpha", caps::NETWORK_JOIN)],
None,
proof,
)
.unwrap()
}
#[test]
fn validate_ucan_chain_valid() {
let node = Keypair::generate();
let agent = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let proof = bootstrap_ucan(&node, agent_did.clone());
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
assert!(validate_ucan_chain(&token, &node_did, &agent_did).is_ok());
}
#[test]
fn validate_ucan_chain_wrong_issuer() {
let node = Keypair::generate();
let agent = Keypair::generate();
let other = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let proof = bootstrap_ucan(&node, agent_did.clone());
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
// signer_did is `other` but UCAN iss is `agent` — must be rejected
let err = validate_ucan_chain(&token, &node_did, &other.did()).unwrap_err();
assert_eq!(err.0, StatusCode::UNAUTHORIZED);
let body = err.1 .0.to_string();
assert!(body.contains("does not match request signer"));
}
#[test]
fn validate_ucan_chain_wrong_audience() {
let node = Keypair::generate();
let agent = Keypair::generate();
let other_node = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let proof = bootstrap_ucan(&node, agent_did.clone());
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
// expected_aud is a different node — must be rejected
let err = validate_ucan_chain(&token, &other_node.did(), &agent_did).unwrap_err();
assert_eq!(err.0, StatusCode::UNAUTHORIZED);
let body = err.1 .0.to_string();
assert!(body.contains("audience mismatch"));
}
#[test]
fn validate_ucan_chain_expired_proof() {
let node = Keypair::generate();
let agent = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let exp = chrono::Utc::now() - chrono::Duration::hours(1);
let proof = Ucan::issue(
&node,
agent_did.clone(),
vec![Capability::new("gitlawb://alpha", caps::NETWORK_JOIN)],
Some(exp),
)
.unwrap();
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
let err = validate_ucan_chain(&token, &node_did, &agent_did).unwrap_err();
assert_eq!(err.0, StatusCode::UNAUTHORIZED);
let body = err.1 .0.to_string();
assert!(body.contains("expired"));
}
fn make_test_state(node_did: gitlawb_core::did::Did) -> crate::state::AppState {
use crate::{config::Config, graphql, rate_limit::RateLimiter};
use clap::Parser;
let keypair = Keypair::generate();
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/gitlawb_test_placeholder")
.expect("lazy pool creation should not fail");
let db = Arc::new(crate::db::Db::for_testing(pool.clone()));
let schema = Arc::new(graphql::build_schema(
db.clone(),
ref_tx.clone(),
task_tx.clone(),
));
crate::state::AppState {
config: Arc::new(Config::parse_from(["gitlawb-node"])),
db,
node_did,
node_keypair: Arc::new(keypair),
p2p: None,
http_client: Arc::new(reqwest::Client::new()),
ref_update_tx: ref_tx,
task_event_tx: task_tx,
graphql_schema: schema,
machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
shutdown_tx: tokio::sync::watch::channel(false).0,
}
}
#[tokio::test]
async fn require_ucan_chain_no_header_passes_through() {
let state = make_test_state(Keypair::generate().did());
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
let req = Request::builder()
.uri("/")
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn require_ucan_chain_missing_did_returns_401() {
let state = make_test_state(Keypair::generate().did());
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
// x-ucan present but no AuthenticatedDid extension → 401
let req = Request::builder()
.uri("/")
.header("x-ucan", "any-token")
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn require_ucan_chain_wrong_issuer_returns_401() {
let node = Keypair::generate();
let agent = Keypair::generate();
let other = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
// Build a valid token where iss = agent, but supply `other` as the signer.
let proof = bootstrap_ucan(&node, agent_did.clone());
let token = delegation_ucan(&agent, node_did.clone(), &proof)
.encode()
.unwrap();
let state = make_test_state(node_did);
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
// AuthenticatedDid is `other`, UCAN iss is `agent` → issuer mismatch → 401
let req = Request::builder()
.uri("/")
.header("x-ucan", token)
.extension(AuthenticatedDid(other.did().to_string()))
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn require_ucan_chain_malformed_token_returns_401() {
let state = make_test_state(Keypair::generate().did());
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
// Malformed x-ucan (invalid JSON)
let req = Request::builder()
.uri("/")
.header("x-ucan", "invalid-token-structure")
.extension(AuthenticatedDid(Keypair::generate().did().to_string()))
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
let body_bytes = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
let body_json: serde_json::Value = serde_json::from_slice(&body_bytes).unwrap();
assert_eq!(body_json["error"], "invalid_ucan");
}
}
+316
View File
@@ -0,0 +1,316 @@
//! Embedded seed list of public Gitlawb network nodes.
//!
//! This module parses `bootstrap-peers.json` (embedded at compile time) and
//! merges its contents into the runtime config so a fresh `docker compose up`
//! joins the network without any manual peer configuration.
//!
//! Operators can opt out by setting `GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true` in
//! their environment — useful for isolated dev networks or testing.
//!
//! Add a node to the canonical list via PR to `bootstrap-peers.json`.
use std::str::FromStr;
use libp2p_core::Multiaddr;
use serde::Deserialize;
use tracing::{info, warn};
use crate::config::Config;
const EMBEDDED_PEERS_JSON: &str = include_str!("../../../bootstrap-peers.json");
const SUPPORTED_VERSION: u32 = 1;
#[derive(Debug, Deserialize)]
struct BootstrapList {
version: u32,
#[serde(default)]
peers: Vec<BootstrapPeer>,
}
#[derive(Debug, Deserialize)]
struct BootstrapPeer {
name: String,
#[allow(dead_code)]
operator: Option<String>,
#[allow(dead_code)]
did: Option<String>,
http_url: Option<String>,
p2p_multiaddr: Option<String>,
#[allow(dead_code)]
added: Option<String>,
}
/// Counts of newly-added entries returned by `merge_into_vecs`.
#[derive(Debug, Default, PartialEq, Eq)]
struct MergeCounts {
http: usize,
p2p: usize,
}
/// Returns true when `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` is set to a truthy value.
fn seeds_disabled() -> bool {
std::env::var("GITLAWB_BOOTSTRAP_DISABLE_SEEDS")
.ok()
.filter(|v| !v.is_empty() && v != "0" && !v.eq_ignore_ascii_case("false"))
.is_some()
}
/// Parse the seed list from a JSON string, rejecting unsupported versions.
fn parse_seed_list(json: &str) -> Result<BootstrapList, String> {
let list: BootstrapList = serde_json::from_str(json).map_err(|e| e.to_string())?;
if list.version != SUPPORTED_VERSION {
return Err(format!(
"unsupported bootstrap-peers.json version: {} (expected {})",
list.version, SUPPORTED_VERSION
));
}
Ok(list)
}
/// Pure merge: appends entries from `list` to the two vectors, deduping.
/// Returns counts of entries actually added (i.e. not already present).
fn merge_into_vecs(
list: BootstrapList,
http_peers: &mut Vec<String>,
p2p_bootstrap: &mut Vec<String>,
) -> MergeCounts {
let mut counts = MergeCounts::default();
for peer in list.peers {
if let Some(url) = peer
.http_url
.as_ref()
.filter(|u| !u.is_empty() && !http_peers.contains(u))
{
http_peers.push(url.clone());
counts.http += 1;
}
if let Some(addr_str) = peer.p2p_multiaddr.as_ref().filter(|s| !s.is_empty()) {
match Multiaddr::from_str(addr_str) {
Ok(_) => {
if !p2p_bootstrap.contains(addr_str) {
p2p_bootstrap.push(addr_str.clone());
counts.p2p += 1;
}
}
Err(e) => warn!(
name = %peer.name,
addr = %addr_str,
err = %e,
"invalid p2p_multiaddr in bootstrap-peers.json — skipping"
),
}
}
}
counts
}
/// Merge the embedded seed list into the runtime config.
///
/// - Appends any `http_url` to `config.bootstrap_peers` (used by gossip_task)
/// - Appends any valid `p2p_multiaddr` to `config.p2p_bootstrap` (used by libp2p)
/// - Dedupes against entries already present (env / CLI takes precedence)
/// - No-op when `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` is set to a truthy value
pub fn merge_seeds(config: &mut Config) {
if seeds_disabled() {
info!("bootstrap seed list disabled via GITLAWB_BOOTSTRAP_DISABLE_SEEDS");
return;
}
let list = match parse_seed_list(EMBEDDED_PEERS_JSON) {
Ok(l) => l,
Err(e) => {
warn!(err = %e, "failed to load embedded bootstrap-peers.json — skipping");
return;
}
};
let counts = merge_into_vecs(list, &mut config.bootstrap_peers, &mut config.p2p_bootstrap);
if counts.http > 0 || counts.p2p > 0 {
info!(
http_peers = counts.http,
p2p_peers = counts.p2p,
"merged bootstrap seed list into config"
);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_valid_v1_list() {
let json = r#"{
"version": 1,
"updated": "2026-04-29",
"peers": [
{
"name": "alpha",
"operator": "Alice",
"did": "did:key:z6MkAlice",
"http_url": "https://alpha.example.com",
"p2p_multiaddr": "/ip4/1.2.3.4/tcp/7546",
"added": "2026-04-29"
}
]
}"#;
let list = parse_seed_list(json).expect("should parse");
assert_eq!(list.version, 1);
assert_eq!(list.peers.len(), 1);
assert_eq!(list.peers[0].name, "alpha");
}
#[test]
fn parse_rejects_unknown_version() {
let json = r#"{ "version": 99, "peers": [] }"#;
let err = parse_seed_list(json).expect_err("should reject");
assert!(err.contains("unsupported"));
}
#[test]
fn parse_rejects_malformed_json() {
let err = parse_seed_list("{ not json").expect_err("should reject");
assert!(!err.is_empty());
}
#[test]
fn parse_accepts_empty_peers_array() {
let json = r#"{ "version": 1, "peers": [] }"#;
let list = parse_seed_list(json).expect("should parse");
assert!(list.peers.is_empty());
}
#[test]
fn parse_treats_missing_peers_as_empty() {
let json = r#"{ "version": 1 }"#;
let list = parse_seed_list(json).expect("should parse");
assert!(list.peers.is_empty());
}
#[test]
fn merge_appends_new_http_and_p2p() {
let list = parse_seed_list(
r#"{
"version": 1,
"peers": [
{
"name": "alpha",
"http_url": "https://alpha.example.com",
"p2p_multiaddr": "/ip4/1.2.3.4/tcp/7546"
}
]
}"#,
)
.unwrap();
let mut http = Vec::new();
let mut p2p = Vec::new();
let counts = merge_into_vecs(list, &mut http, &mut p2p);
assert_eq!(counts, MergeCounts { http: 1, p2p: 1 });
assert_eq!(http, vec!["https://alpha.example.com"]);
assert_eq!(p2p, vec!["/ip4/1.2.3.4/tcp/7546"]);
}
#[test]
fn merge_dedupes_existing_entries() {
let list = parse_seed_list(
r#"{
"version": 1,
"peers": [
{ "name": "alpha", "http_url": "https://alpha.example.com" }
]
}"#,
)
.unwrap();
let mut http = vec!["https://alpha.example.com".to_string()];
let mut p2p = Vec::new();
let counts = merge_into_vecs(list, &mut http, &mut p2p);
assert_eq!(counts.http, 0, "should not double-add");
assert_eq!(http.len(), 1);
}
#[test]
fn merge_skips_invalid_p2p_multiaddr() {
let list = parse_seed_list(
r#"{
"version": 1,
"peers": [
{
"name": "bad",
"http_url": "https://bad.example.com",
"p2p_multiaddr": "this is not a multiaddr"
}
]
}"#,
)
.unwrap();
let mut http = Vec::new();
let mut p2p = Vec::new();
let counts = merge_into_vecs(list, &mut http, &mut p2p);
assert_eq!(counts.http, 1, "http still added");
assert_eq!(counts.p2p, 0, "invalid p2p skipped");
assert!(p2p.is_empty());
}
#[test]
fn merge_skips_empty_strings() {
let list = parse_seed_list(
r#"{
"version": 1,
"peers": [
{ "name": "blank", "http_url": "", "p2p_multiaddr": "" }
]
}"#,
)
.unwrap();
let mut http = Vec::new();
let mut p2p = Vec::new();
let counts = merge_into_vecs(list, &mut http, &mut p2p);
assert_eq!(counts, MergeCounts::default());
}
#[test]
fn merge_handles_null_optional_fields() {
let list = parse_seed_list(
r#"{
"version": 1,
"peers": [
{
"name": "alpha",
"operator": null,
"did": null,
"http_url": "https://alpha.example.com",
"p2p_multiaddr": null,
"added": null
}
]
}"#,
)
.unwrap();
let mut http = Vec::new();
let mut p2p = Vec::new();
let counts = merge_into_vecs(list, &mut http, &mut p2p);
assert_eq!(counts, MergeCounts { http: 1, p2p: 0 });
}
#[test]
fn embedded_seed_list_parses_successfully() {
// Regression: the canonical bootstrap-peers.json shipped in the repo
// must always be valid, since it's compiled into the binary.
parse_seed_list(EMBEDDED_PEERS_JSON)
.expect("embedded bootstrap-peers.json must always parse");
}
}
+5 -5
View File
@@ -41,11 +41,11 @@ pub async fn issue_ref_certificate(
let signature = state.node_keypair.sign_b64(&payload_bytes);
let cert = RefCertificate {
id: Uuid::new_v4().to_string(),
repo_id: repo_id.to_string(),
ref_name: ref_name.to_string(),
old_sha: old_sha.to_string(),
new_sha: new_sha.to_string(),
id: Uuid::new_v4().to_string(),
repo_id: repo_id.to_string(),
ref_name: ref_name.to_string(),
old_sha: old_sha.to_string(),
new_sha: new_sha.to_string(),
pusher_did: pusher_did.to_string(),
node_did,
signature,
+88 -7
View File
@@ -1,15 +1,58 @@
use clap::{Args, Parser, Subcommand};
use std::path::PathBuf;
use clap::Parser;
/// Admin subcommands. When absent, the binary runs the node daemon as usual.
#[derive(Subcommand, Debug, Clone)]
pub enum Command {
/// Bulk-delete repos whose name matches a Postgres regex, removing their DB
/// rows, on-disk bare repo, and Tigris archive. Intended for spam cleanup.
/// Defaults to a dry run; pass --execute to actually delete.
PurgeSpam(PurgeSpamArgs),
}
#[derive(Args, Debug, Clone)]
pub struct PurgeSpamArgs {
/// Postgres regex matched against `repos.name` (e.g. '^keep[0-9]').
#[arg(long)]
pub regex: String,
/// Actually delete. Without this flag the command only reports what would
/// be removed (dry run).
#[arg(long, default_value_t = false)]
pub execute: bool,
/// Stop after processing at most this many repos (0 = no limit).
#[arg(long, default_value_t = 0)]
pub limit: i64,
/// Skip deleting the Tigris archive (DB + on-disk only).
#[arg(long, default_value_t = false)]
pub skip_tigris: bool,
/// Bulk mode: delete all matching rows with set-based SQL (a handful of
/// statements) + remove their on-disk repos, instead of one transaction per
/// repo. Far faster on large match sets / small DB instances. Skips Tigris.
#[arg(long, default_value_t = false)]
pub bulk: bool,
}
#[derive(Parser, Debug, Clone)]
#[command(name = "gitlawb-node", about = "gitlawb node daemon", version)]
pub struct Config {
/// Optional admin subcommand. When omitted, runs the node daemon.
#[command(subcommand)]
pub command: Option<Command>,
/// Directory where bare git repositories are stored
#[arg(long, env = "GITLAWB_REPOS_DIR", default_value = "./data/repos")]
pub repos_dir: PathBuf,
/// PostgreSQL connection URL (Supabase or any Postgres instance)
#[arg(long, env = "DATABASE_URL", default_value = "postgresql://localhost/gitlawb")]
#[arg(
long,
env = "DATABASE_URL",
default_value = "postgresql://localhost/gitlawb"
)]
pub database_url: String,
/// Host to bind to
@@ -24,7 +67,7 @@ pub struct Config {
#[arg(long, env = "GITLAWB_KEY", default_value = "~/.gitlawb/identity.pem")]
pub key_path: String,
/// Allow unauthenticated reads (default: true for public repos)
/// Reserved for private-read mode; per-repo read enforcement is not wired in alpha
#[arg(long, env = "GITLAWB_PUBLIC_READ", default_value_t = true)]
pub public_read: bool,
@@ -36,6 +79,23 @@ pub struct Config {
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>,
/// Require RFC 9421 signatures on peer announce/sync write routes.
/// Keep false during rolling upgrades so existing live nodes can still gossip.
#[arg(
long,
env = "GITLAWB_REQUIRE_SIGNED_PEER_WRITES",
default_value_t = false
)]
pub require_signed_peer_writes: bool,
/// Require the authenticated pusher to be the repo owner on `git-receive-pack`.
/// Authentication (a valid did:key signature) is not authorization on its own:
/// any party can sign as their own DID. When true, pushes whose authenticated
/// DID is not the repo owner are rejected. Keep false during rolling upgrades;
/// flip it on once owners are ready for owner-only writes.
#[arg(long, env = "GITLAWB_ENFORCE_OWNER_PUSH", default_value_t = false)]
pub enforce_owner_push: bool,
/// URL of local IPFS/Kubo node HTTP API (e.g. http://127.0.0.1:5001)
#[arg(long, env = "GITLAWB_IPFS_API", default_value = "")]
pub ipfs_api: String,
@@ -45,15 +105,19 @@ pub struct Config {
pub pinata_jwt: String,
/// Pinata v3 upload URL
#[arg(long, env = "GITLAWB_PINATA_UPLOAD_URL", default_value = "https://uploads.pinata.cloud/v3/files")]
#[arg(
long,
env = "GITLAWB_PINATA_UPLOAD_URL",
default_value = "https://uploads.pinata.cloud/v3/files"
)]
pub pinata_upload_url: String,
/// libp2p TCP port (0 = disabled)
/// libp2p QUIC/UDP port (0 = disabled)
#[arg(long, env = "GITLAWB_P2P_PORT", default_value_t = 7546)]
pub p2p_port: u16,
/// libp2p bootstrap multiaddrs (comma-separated)
/// Example: /ip4/1.2.3.4/tcp/7546/p2p/12D3KooW...
/// Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
#[arg(long, env = "GITLAWB_P2P_BOOTSTRAP", value_delimiter = ',')]
pub p2p_bootstrap: Vec<String>,
@@ -75,7 +139,11 @@ pub struct Config {
pub contract_name_registry: String,
/// Base L2 RPC URL
#[arg(long, env = "GITLAWB_CHAIN_RPC_URL", default_value = "https://sepolia.base.org")]
#[arg(
long,
env = "GITLAWB_CHAIN_RPC_URL",
default_value = "https://sepolia.base.org"
)]
pub chain_rpc_url: String,
/// Base L2 node staking contract address (GitlawbNodeStaking). When set
@@ -110,6 +178,19 @@ pub struct Config {
/// Default: 2 GB. Set lower on resource-constrained nodes.
#[arg(long, env = "GITLAWB_MAX_PACK_BYTES", default_value_t = 2_147_483_648)]
pub max_pack_bytes: usize,
/// Optional address to bind a Prometheus `/metrics` exposition endpoint on.
/// Example: `127.0.0.1:9091`. Leave empty (default) to disable.
/// Bind to localhost or a private interface — the metrics endpoint is
/// unauthenticated.
#[arg(long, env = "GITLAWB_METRICS_ADDR", default_value = "")]
pub metrics_addr: String,
/// Maximum time to wait for in-flight requests to drain on shutdown, in
/// seconds. After this elapses, the server returns 503 to anything still
/// in flight and exits. Default: 30s.
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
pub shutdown_grace_secs: u64,
}
impl Config {
File diff suppressed because it is too large. Load diff
+362
View File
@@ -0,0 +1,362 @@
//! Encrypt-then-pin for withheld blobs (Option B1). Each withheld blob is sealed
//! to its recipient DIDs and the envelope pinned to IPFS, recorded in
//! `encrypted_blobs`. Best-effort per blob: a failure is logged and skipped,
//! never pinned in plaintext.
use std::collections::{BTreeSet, HashMap};
use std::path::Path;
use std::str::FromStr;
use ed25519_dalek::VerifyingKey;
use gitlawb_core::did::Did;
use gitlawb_core::encrypt::seal_blob;
use crate::db::Db;
use hmac::{Hmac, Mac};
use sha2::Sha256;
type HmacSha256 = Hmac<Sha256>;
/// Opaque, node-keyed fingerprint of a blob's recipient set. Stored in place of
/// the cleartext DID list so a DB compromise cannot reveal the reader set; used
/// only to detect a recipient-set change so an unchanged blob is not re-sealed.
/// Order-insensitive (the input `BTreeSet` is already sorted).
pub fn recipients_tag(node_seed: &[u8; 32], dids: &BTreeSet<String>) -> String {
let mut mac = HmacSha256::new_from_slice(node_seed).expect("HMAC accepts any key length");
mac.update(b"gitlawb/recipients-tag/v1");
for did in dids {
mac.update(b"\n");
mac.update(did.as_bytes());
}
hex::encode(mac.finalize().into_bytes())
}
/// Resolve a DID string to its Ed25519 verifying key, or None if it carries no
/// inline key (e.g. did:web / did:gitlawb).
fn did_to_key(did: &str) -> Option<VerifyingKey> {
Did::from_str(did).ok()?.to_verifying_key().ok()
}
/// Resolve every recipient DID to its verifying key, all-or-nothing.
///
/// Returns `Ok(keys)` only when every DID resolves. If any DID fails, returns
/// `Err(unresolved)` listing the unresolvable DID strings so the caller can fail
/// closed rather than seal to a partial recipient set (#47): sealing to a subset
/// while recording the full set as covered permanently locks out the dropped
/// readers. Resolution is local-only, so `did:web`/`did:gitlawb` recipients (and
/// any malformed `did:key`) land in the unresolved set until off-`did:key`
/// resolution exists.
fn resolve_all_recipients(dids: &BTreeSet<String>) -> Result<Vec<VerifyingKey>, Vec<String>> {
let mut keys = Vec::with_capacity(dids.len());
let mut unresolved = Vec::new();
for did in dids {
match did_to_key(did) {
Some(k) => keys.push(k),
None => unresolved.push(did.clone()),
}
}
if unresolved.is_empty() {
Ok(keys)
} else {
Err(unresolved)
}
}
/// What to do with a single withheld blob, decided without any DB or IO so the
/// fail-closed invariant (#47) is unit-testable in isolation.
#[derive(Debug)]
enum SealPlan {
/// An existing envelope already covers exactly this recipient set; nothing to do.
SkipUnchanged,
/// No recipient DID resolved to a key, so there is nothing to seal to.
SkipNoRecipients,
/// At least one recipient DID is unresolvable. Fail closed: never seal to a
/// partial set. Carries the unresolvable DIDs for logging.
SkipUnresolvable(Vec<String>),
/// Seal to `keys` and record coverage under `tag`.
Seal {
keys: Vec<VerifyingKey>,
tag: String,
},
}
/// Decide what to do with one blob given its desired recipient set and the tag
/// already stored for it (if any). Pure: no DB, no IO.
///
/// This is the #47 fail-closed gate in isolation: it returns `Seal` only when
/// EVERY recipient DID resolves, so no caller can seal to a partial set. A
/// changed recipient set (different tag) re-seals so a newly added reader can
/// recover the blob; reader removal is not retroactive (the old envelope is
/// already public). The comparison is on the opaque node-keyed tag, never the
/// DID list.
fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&str>) -> SealPlan {
let tag = recipients_tag(node_seed, dids);
if stored_tag == Some(tag.as_str()) {
return SealPlan::SkipUnchanged;
}
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
}
}
/// Encrypt and pin every withheld blob. `recipients` maps blob oid -> DID set;
/// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob
/// actually sealed and recorded this call (the per-push delta), used by Option B3
/// to anchor a manifest. Recipient identities are never stored or returned.
pub async fn encrypt_and_pin(
ipfs_api: &str,
repo_path: &Path,
db: &Db,
repo_id: &str,
node_seed: &[u8; 32],
recipients: &HashMap<String, BTreeSet<String>>,
) -> Vec<(String, String)> {
let mut sealed = Vec::new();
let mut skipped_unresolvable = 0usize;
for (oid, dids) in recipients {
// A DB read failure is not a cache miss: re-sealing here would do an
// avoidable IPFS write during a partial outage. Skip and retry next push.
let stored_tag = match db.encrypted_blob_recipients_tag(repo_id, oid).await {
Ok(t) => t,
Err(e) => {
tracing::warn!(oid = %oid, err = %e, "recipients_tag lookup failed; skipping reseal");
continue;
}
};
// Fail closed: plan_seal returns Seal only when every recipient DID
// resolves, so we never seal to a partial set and record the full set as
// covered (which would permanently lock out the dropped readers, #47).
let (keys, tag) = match plan_seal(node_seed, dids, stored_tag.as_deref()) {
SealPlan::SkipUnchanged => continue,
SealPlan::SkipNoRecipients => {
tracing::warn!(oid = %oid, "no resolvable recipient keys; skipping encrypted pin");
continue;
}
SealPlan::SkipUnresolvable(unresolved) => {
skipped_unresolvable += 1;
// DIDs are user-controlled (rule reader_dids); log a bounded
// sample, not an unbounded dump. Wording stays neutral about the
// cause: a malformed did:key is not DHT-pending and never will
// resolve, unlike a did:gitlawb awaiting anchoring.
let sample: Vec<&String> = unresolved.iter().take(3).collect();
tracing::warn!(
oid = %oid,
unresolved_count = unresolved.len(),
unresolved_sample = ?sample,
"unresolvable recipient DID(s); skipping encrypted pin to avoid sealing to a partial set"
);
continue;
}
SealPlan::Seal { keys, tag } => (keys, tag),
};
let data = match crate::git::store::read_object(repo_path, oid) {
Ok(Some((_t, bytes))) => bytes,
Ok(None) => {
tracing::warn!(oid = %oid, "git object not found; skipping encrypted pin");
continue;
}
Err(e) => {
tracing::warn!(oid = %oid, err = %e, "read_object failed; skipping encrypted pin");
continue;
}
};
let envelope = match seal_blob(&data, &keys) {
Ok(e) => e,
Err(e) => {
tracing::warn!(oid = %oid, err = %e, "seal_blob failed; skipping");
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
tracing::warn!(oid = %oid, "pin_git_object returned no cid; skipping");
continue;
}
Err(e) => {
tracing::warn!(oid = %oid, err = %e, "pin_git_object failed; skipping");
continue;
}
};
if let Err(e) = db.record_encrypted_blob(repo_id, oid, &cid, &tag).await {
tracing::warn!(oid = %oid, err = %e, "record_encrypted_blob failed");
continue;
}
sealed.push((oid.clone(), cid.clone()));
}
// One aggregate signal so a coverage collapse is a single greppable line, not
// a per-oid scrape. In a fully-migrated did:gitlawb org every blob skips and
// recovery coverage silently drops to zero; this is the operator's cue that
// the gap is the deliberate fail-closed posture, not a malfunction.
if skipped_unresolvable > 0 {
tracing::warn!(
sealed = sealed.len(),
skipped = skipped_unresolvable,
"encrypted-pin coverage reduced: blobs skipped for unresolvable recipients"
);
}
sealed
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
fn did_key(seed: u8) -> String {
let vk = SigningKey::from_bytes(&[seed; 32]).verifying_key();
Did::from_verifying_key(&vk).to_string()
}
// Accepts both `&[String]` (resolve tests, built from `did_key`) and
// `&[&str]` (tag tests, built from literals).
fn set<S: AsRef<str>>(dids: &[S]) -> BTreeSet<String> {
dids.iter().map(|s| s.as_ref().to_string()).collect()
}
#[test]
fn all_did_key_recipients_resolve() {
let dids = set(&[did_key(1), did_key(2)]);
let keys = resolve_all_recipients(&dids).expect("all did:key resolve");
assert_eq!(keys.len(), 2);
}
#[test]
fn mixed_set_with_did_gitlawb_fails_closed() {
// Core #47 regression: a resolvable subset must never yield a sealable
// key set. Two did:key plus one did:gitlawb -> Err naming only the
// unresolvable DID.
let gitlawb = Did::gitlawb("zSomeDhtKey").to_string();
let dids = set(&[did_key(1), did_key(2), gitlawb.clone()]);
let unresolved = resolve_all_recipients(&dids).expect_err("must fail closed");
assert_eq!(unresolved, vec![gitlawb]);
}
#[test]
fn did_web_recipient_fails_closed() {
let web = Did::web("example.com").to_string();
let dids = set(&[did_key(1), web.clone()]);
let unresolved = resolve_all_recipients(&dids).expect_err("did:web cannot resolve locally");
assert_eq!(unresolved, vec![web]);
}
#[test]
fn malformed_did_key_fails_closed() {
// The third state: not a method-resolution gap but a permanently broken
// did:key (invalid multibase). Must also fail closed.
let broken = "did:key:z!!!invalid".to_string();
let dids = set(&[did_key(1), broken.clone()]);
let unresolved = resolve_all_recipients(&dids).expect_err("malformed did:key fails");
assert_eq!(unresolved, vec![broken]);
}
#[test]
fn empty_set_resolves_to_empty_keys() {
let dids = BTreeSet::new();
let keys = resolve_all_recipients(&dids).expect("empty set is not an error");
assert!(keys.is_empty());
}
#[test]
fn single_unresolvable_did_returns_that_did() {
let gitlawb = Did::gitlawb("zOnlyOne").to_string();
let dids: BTreeSet<String> = [gitlawb.clone()].into_iter().collect();
let unresolved = resolve_all_recipients(&dids).expect_err("must fail closed");
assert_eq!(unresolved, vec![gitlawb]);
}
#[test]
fn tag_is_order_insensitive() {
let seed = [7u8; 32];
let a = recipients_tag(&seed, &set(&["did:key:zA", "did:key:zB"]));
let b = recipients_tag(&seed, &set(&["did:key:zB", "did:key:zA"]));
assert_eq!(a, b);
}
#[test]
fn tag_differs_for_different_sets() {
let seed = [7u8; 32];
let a = recipients_tag(&seed, &set(&["did:key:zA"]));
let b = recipients_tag(&seed, &set(&["did:key:zA", "did:key:zB"]));
assert_ne!(a, b);
}
#[test]
fn tag_is_keyed_by_node_seed() {
let dids = set(&["did:key:zA", "did:key:zB"]);
let a = recipients_tag(&[1u8; 32], &dids);
let b = recipients_tag(&[2u8; 32], &dids);
assert_ne!(
a, b,
"tag must depend on the node seed, not be a plain hash"
);
}
// plan_seal is the seal/skip decision `encrypt_and_pin` acts on. Testing it
// directly pins the #47 fail-closed invariant at the function that owns it,
// which a unit test of `resolve_all_recipients` alone cannot do (it can't
// catch the caller falling through to a partial seal).
const SEED: [u8; 32] = [9u8; 32];
#[test]
fn plan_seal_seals_when_all_recipients_resolve() {
let dids = set(&[did_key(1), did_key(2)]);
match plan_seal(&SEED, &dids, None) {
SealPlan::Seal { keys, tag } => {
assert_eq!(keys.len(), 2, "must seal to the full recipient set");
assert_eq!(
tag,
recipients_tag(&SEED, &dids),
"records the full-set tag"
);
}
other => panic!("expected Seal, got {other:?}"),
}
}
#[test]
fn plan_seal_fails_closed_on_any_unresolvable_recipient() {
// The #47 invariant at the decision boundary: one unresolvable DID among
// resolvable ones must NOT yield a Seal (which would seal a partial set).
let gitlawb = Did::gitlawb("zPending").to_string();
let dids = set(&[did_key(1), did_key(2), gitlawb.clone()]);
match plan_seal(&SEED, &dids, None) {
SealPlan::SkipUnresolvable(unresolved) => assert_eq!(unresolved, vec![gitlawb]),
other => panic!("must fail closed, never seal a partial set; got {other:?}"),
}
}
#[test]
fn plan_seal_skips_empty_recipient_set() {
let dids = BTreeSet::new();
assert!(matches!(
plan_seal(&SEED, &dids, None),
SealPlan::SkipNoRecipients
));
}
#[test]
fn plan_seal_skips_when_tag_unchanged() {
let dids = set(&[did_key(1)]);
let stored = recipients_tag(&SEED, &dids);
assert!(matches!(
plan_seal(&SEED, &dids, Some(&stored)),
SealPlan::SkipUnchanged
));
}
#[test]
fn plan_seal_reseals_when_recipient_set_changed() {
// A stored tag for a DIFFERENT set is a miss: a newly added reader must
// trigger a re-seal, not be skipped as unchanged.
let dids = set(&[did_key(1), did_key(2)]);
let stale = recipients_tag(&SEED, &set(&[did_key(1)]));
match plan_seal(&SEED, &dids, Some(&stale)) {
SealPlan::Seal { keys, .. } => assert_eq!(keys.len(), 2),
other => panic!("changed recipient set must re-seal; got {other:?}"),
}
}
}
+15 -27
View File
@@ -23,6 +23,9 @@ pub enum AppError {
#[allow(dead_code)]
Forbidden(String),
#[error("icaptcha proof required: {0}")]
IcaptchaProofRequired(String),
#[error("invalid request: {0}")]
BadRequest(String),
@@ -49,36 +52,21 @@ impl IntoResponse for AppError {
"repo_exists",
format!("repository '{r}' already exists"),
),
AppError::NotFound(msg) => (
StatusCode::NOT_FOUND,
"not_found",
msg.clone(),
),
AppError::Unauthorized(msg) => (
StatusCode::UNAUTHORIZED,
"not_an_agent",
msg.clone(),
),
AppError::Forbidden(msg) => (
AppError::NotFound(msg) => (StatusCode::NOT_FOUND, "not_found", msg.clone()),
AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, "not_an_agent", msg.clone()),
AppError::Forbidden(msg) => (StatusCode::FORBIDDEN, "forbidden", msg.clone()),
// 403, not 401: the caller IS an authenticated agent (credentials are
// valid) but is forbidden from this action without a valid, fresh
// iCaptcha proof. The distinct `icaptcha_proof_required` code — which
// clients branch on — keeps it separable from a plain `forbidden`.
AppError::IcaptchaProofRequired(msg) => (
StatusCode::FORBIDDEN,
"forbidden",
"icaptcha_proof_required",
msg.clone(),
),
AppError::BadRequest(msg) => (
StatusCode::BAD_REQUEST,
"bad_request",
msg.clone(),
),
AppError::Git(msg) => (
StatusCode::INTERNAL_SERVER_ERROR,
"git_error",
msg.clone(),
),
AppError::Db(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
"db_error",
e.to_string(),
),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
+51 -27
View File
@@ -23,19 +23,26 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
use std::io::Write;
let mut child = hash_output;
if let Some(stdin) = child.stdin.take() {
let mut stdin = stdin;
stdin.write_all(json.as_bytes()).context("failed to write to git hash-object stdin")?;
}
// Write the blob to stdin, but always reap the child afterward even if the
// write fails, so a stdin-write error can't drop the Child unwaited and leak
// a zombie (#53).
let write_result = match child.stdin.take() {
Some(mut stdin) => stdin.write_all(json.as_bytes()),
None => Err(std::io::Error::new(
std::io::ErrorKind::BrokenPipe,
"git hash-object stdin unavailable",
)),
};
let output = child.wait_with_output().context("git hash-object failed")?;
write_result.context("failed to write to git hash-object stdin")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("git hash-object failed: {stderr}");
}
let hash = String::from_utf8(output.stdout)
.context("git hash-object output is not valid UTF-8")?;
let hash =
String::from_utf8(output.stdout).context("git hash-object output is not valid UTF-8")?;
let hash = hash.trim();
// Update the ref
@@ -58,7 +65,11 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
let list_output = Command::new("git")
.args(["for-each-ref", "--format=%(refname)", "refs/gitlawb/issues/"])
.args([
"for-each-ref",
"--format=%(refname)",
"refs/gitlawb/issues/",
])
.current_dir(repo_path)
.output()
.context("failed to run git for-each-ref")?;
@@ -121,10 +132,7 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
}
let output = String::from_utf8_lossy(&list.stdout);
let matches: Vec<&str> = output
.lines()
.filter(|l| !l.trim().is_empty())
.collect();
let matches: Vec<&str> = output.lines().filter(|l| !l.trim().is_empty()).collect();
match matches.len() {
0 => Ok(None),
@@ -156,8 +164,8 @@ pub fn close_issue(repo_path: &Path, issue_id: &str) -> Result<Option<String>> {
let raw = get_issue(repo_path, &full_id)?
.expect("ref existed in resolve but not in get — should be impossible");
let mut issue: serde_json::Value = serde_json::from_str(&raw)
.context("invalid issue JSON in git ref")?;
let mut issue: serde_json::Value =
serde_json::from_str(&raw).context("invalid issue JSON in git ref")?;
issue["status"] = serde_json::Value::String("closed".to_string());
let updated = serde_json::to_string(&issue).context("failed to serialize updated issue")?;
@@ -193,15 +201,6 @@ mod tests {
use std::process::Command;
use tempfile::TempDir;
fn init_bare_repo(dir: &TempDir) {
Command::new("git")
.args(["init", "--bare"])
.current_dir(dir.path())
.output()
.unwrap();
// git hash-object -w needs a non-bare repo; use a working repo instead
}
fn init_repo(dir: &TempDir) {
Command::new("git")
.args(["init"])
@@ -225,7 +224,12 @@ mod tests {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "abc12345-0000-0000-0000-000000000000";
create_issue(dir.path(), full_id, r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#).unwrap();
create_issue(
dir.path(),
full_id,
r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
let resolved = resolve_issue_id(dir.path(), full_id).unwrap();
assert_eq!(resolved, Some(full_id.to_string()));
}
@@ -235,7 +239,12 @@ mod tests {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "abc12345-0000-0000-0000-000000000000";
create_issue(dir.path(), full_id, r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#).unwrap();
create_issue(
dir.path(),
full_id,
r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
let resolved = resolve_issue_id(dir.path(), "abc12345").unwrap();
assert_eq!(resolved, Some(full_id.to_string()));
}
@@ -252,8 +261,18 @@ mod tests {
fn test_resolve_ambiguous_prefix_errors() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345-aaaa-0000-0000-000000000000", r#"{"status":"open"}"#).unwrap();
create_issue(dir.path(), "abc12345-bbbb-0000-0000-000000000000", r#"{"status":"open"}"#).unwrap();
create_issue(
dir.path(),
"abc12345-aaaa-0000-0000-000000000000",
r#"{"status":"open"}"#,
)
.unwrap();
create_issue(
dir.path(),
"abc12345-bbbb-0000-0000-000000000000",
r#"{"status":"open"}"#,
)
.unwrap();
let result = resolve_issue_id(dir.path(), "abc12345");
assert!(result.is_err());
assert!(result.unwrap_err().to_string().contains("ambiguous"));
@@ -264,7 +283,12 @@ mod tests {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "def99999-0000-0000-0000-000000000000";
create_issue(dir.path(), full_id, r#"{"id":"def99999-0000-0000-0000-000000000000","status":"open"}"#).unwrap();
create_issue(
dir.path(),
full_id,
r#"{"id":"def99999-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
let updated = close_issue(dir.path(), "def99999").unwrap().unwrap();
let v: serde_json::Value = serde_json::from_str(&updated).unwrap();
+5 -3
View File
@@ -1,5 +1,7 @@
pub mod store;
pub mod smart_http;
pub mod issues;
pub mod tigris;
pub mod push_delta;
pub mod repo_store;
pub mod smart_http;
pub mod store;
pub mod tigris;
pub mod visibility_pack;
+663
View File
@@ -0,0 +1,663 @@
//! Push-delta object enumeration for the IPFS/Pinata pin path.
//!
//! The pin path needs the set of git objects to consider for pinning after a
//! push. Historically both pin functions enumerated the *whole* repo
//! (`git cat-file --batch-all-objects`) on every push, so cost scaled with
//! repo size rather than push size (finding N16). This module computes the
//! per-push *delta* instead — the objects a push introduced — and keeps the
//! whole-repo lister for the reconciliation sweep and the fail-closed fallback.
//!
//! ## Correctness framing (do not confuse with the #84 withholding direction)
//!
//! The pin enumeration is part of the *exposure* set, not the withheld filter.
//! On the *delta* path, narrowing per push only *shrinks* what is pinned, so it
//! cannot create an under-withholding leak — the reachable-only withheld filter
//! (`visibility_pack`) still subtracts from the smaller set we feed it. The only
//! risk there is *under-pinning* (a durability gap) the sweep backstops.
//!
//! The *full-scan* path is different: `list_all_objects` includes dangling
//! objects the reachable withheld set never classified, so subtracting that set
//! is not enough — a dangling private blob would slip through (#99). The caller
//! signals a full scan via [`PinCandidateSet::full_scan`] and must then apply
//! the fail-closed blob filter (`visibility_pack::replicable_objects_fail_closed`)
//! instead of the plain reachable-only subtraction.
//!
//! Because the pin candidate set needs only the OID *set* (never the per-path
//! information the withheld classifier needs), `git rev-list --objects
//! --no-object-names` is safe here. The "rev-list reports one path per object"
//! trap from #84 applies only to `visibility_pack`'s per-path `ls-tree` walk.
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::process::Command;
use anyhow::{bail, Context, Result};
/// Env var that forces the push path to always full-scan, bypassing the delta
/// optimization (KTD7 kill-switch). Reuses the already-tested fallback branch
/// rather than introducing a second enumeration path. Does not gate the sweep.
const FORCE_FULL_SCAN_ENV: &str = "GITLAWB_PIN_FORCE_FULLSCAN";
/// The objects a push introduced, or a signal that the delta could not be
/// computed safely and the caller should fall back to a full scan.
///
/// Three-valued by design: `Delta(vec![])` (a no-op push — pin nothing) is a
/// *different* fact from `FullScanRequired` (could not compute — fall back),
/// and they drive different actions.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PinCandidates {
/// Objects reachable from the new tips but not the old tips.
Delta(Vec<String>),
/// The delta is not safe to use; the caller must enumerate the whole repo
/// with [`list_all_objects`] instead. Returned on a non-commit tip, any git
/// error, or the force-full-scan kill-switch.
FullScanRequired,
}
/// Resolve the set of objects introduced by a push from its ref updates.
///
/// `new_tips` / `old_tips` are the non-zero new/old SHAs of the push's ref
/// updates (the caller strips the all-zeros create/delete sentinel). Keeping
/// the helper on plain SHA slices (not the `api` `RefUpdate` type) keeps it
/// unit-testable without the HTTP layer.
///
/// Fail-closed: any condition where the introduced set cannot be safely
/// determined returns [`PinCandidates::FullScanRequired`] rather than a partial
/// set, so the caller full-scans instead of silently under-pinning.
pub fn resolve_push_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> PinCandidates {
// KTD7 kill-switch: force the (already-tested) full-scan fallback. The env
// read is split out from the pure logic so the resolver stays unit-testable
// without touching process-global state.
resolve_push_delta_inner(repo_path, new_tips, old_tips, force_full_scan())
}
/// Whether the force-full-scan kill-switch env var is set.
fn force_full_scan() -> bool {
std::env::var_os(FORCE_FULL_SCAN_ENV).is_some()
}
// Intentionally private but reachable from the test module (via `use super::*`)
// so the kill-switch can be exercised without mutating process-global env.
fn resolve_push_delta_inner(
repo_path: &Path,
new_tips: &[&str],
old_tips: &[&str],
force_full_scan: bool,
) -> PinCandidates {
if force_full_scan {
tracing::debug!("{FORCE_FULL_SCAN_ENV} set — forcing full scan");
return PinCandidates::FullScanRequired;
}
// A push that only deleted refs introduces no objects.
if new_tips.is_empty() {
return PinCandidates::Delta(Vec::new());
}
// Ref-type guard (the fail-closed mechanism). `git rev-list` does NOT error
// on a non-commit tip — a blob/tree/tag-of-non-commit all exit 0 and walk
// the object — so the rev-list exit code cannot catch this. Check each new
// tip's *fully-peeled* type with `cat-file -t '<sha>^{}'` (recursive peel:
// tag-of-commit -> commit, tag-of-tree -> tree, tag-of-tag-of-commit ->
// commit). Bare `cat-file -t` returns `tag` for an annotated tag, and
// `for-each-ref %(*objecttype)` peels only one level — neither is correct.
for tip in new_tips {
match peeled_object_type(repo_path, tip) {
Some(t) if t == "commit" => {}
other => {
tracing::debug!(
tip = %tip,
peeled_type = ?other,
"push tip is not a commit (or type lookup failed) — forcing full scan"
);
return PinCandidates::FullScanRequired;
}
}
}
match rev_list_delta(repo_path, new_tips, old_tips) {
Ok(oids) => PinCandidates::Delta(oids),
Err(e) => {
tracing::debug!(err = %e, "push-delta rev-list failed — forcing full scan");
PinCandidates::FullScanRequired
}
}
}
/// Return the fully-peeled object type of `sha` (e.g. `commit`, `tree`,
/// `blob`), or `None` if the object is missing/unpeelable or git errored.
fn peeled_object_type(repo_path: &Path, sha: &str) -> Option<String> {
let output = Command::new("git")
.args(["cat-file", "-t", &format!("{sha}^{{}}")])
.current_dir(repo_path)
.output()
.ok()?;
if !output.status.success() {
return None;
}
Some(String::from_utf8_lossy(&output.stdout).trim().to_string())
}
/// Run `git rev-list --objects --no-object-names <new> --not <old>` and return
/// the bare OID set. Decides on `status.success()` *before* parsing stdout, so
/// a walk that prints a valid prefix then errors mid-walk is discarded.
fn rev_list_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> Result<Vec<String>> {
let mut args: Vec<&str> = vec!["rev-list", "--objects", "--no-object-names"];
args.extend_from_slice(new_tips);
if !old_tips.is_empty() {
args.push("--not");
args.extend_from_slice(old_tips);
}
let output = Command::new("git")
.args(&args)
.current_dir(repo_path)
.output()
.context("failed to run git rev-list for push delta")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git rev-list failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
Ok(stdout
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect())
}
/// List every object in the repository via
/// `git cat-file --batch-all-objects --batch-check='%(objectname)'`.
///
/// This is the whole-repo enumeration the push path falls back to and the
/// reconciliation sweep relies on. It returns *all* objects (including
/// unreachable/dangling ones), which is what the sweep needs to catch
/// stragglers — do not swap it for a reachability walk.
pub fn list_all_objects(repo_path: &Path) -> Result<Vec<String>> {
let output = Command::new("git")
.args([
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname)",
])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git cat-file failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
Ok(stdout
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect())
}
/// Like [`list_all_objects`] but pairs each OID with its object type, via
/// `--batch-check='%(objectname) %(objecttype)'`. The pin path's fail-closed
/// filter needs to tell blobs (content, withholdable) from commits/trees
/// (structural, never withheld) without typing the candidate list itself.
pub fn list_all_objects_with_type(repo_path: &Path) -> Result<Vec<(String, String)>> {
let output = Command::new("git")
.args([
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname) %(objecttype)",
])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git cat-file failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
Ok(stdout
.lines()
.filter_map(|l| {
let mut it = l.split_whitespace();
let oid = it.next()?;
let ty = it.next()?;
Some((oid.to_string(), ty.to_string()))
})
.collect())
}
/// Every blob OID in the repo, including unreachable/dangling ones. The
/// fail-closed pin filter drops any candidate blob absent from the reachable,
/// visibility-allowed set; a dangling private blob is in this set but not the
/// allowed set, so it never replicates (#99).
pub fn all_blob_oids(repo_path: &Path) -> Result<HashSet<String>> {
Ok(list_all_objects_with_type(repo_path)?
.into_iter()
.filter(|(_, ty)| ty == "blob")
.map(|(oid, _)| oid)
.collect())
}
/// The pin candidate OIDs for a push plus whether they came from a full-repo
/// scan. `full_scan` is true when the delta could not be used and the whole
/// object DB (including dangling objects) was enumerated — the caller must then
/// apply the fail-closed blob filter, because the reachable-only withheld set
/// cannot classify dangling blobs (#99).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PinCandidateSet {
pub candidates: Vec<String>,
pub full_scan: bool,
}
/// Resolve the pin candidate OID set for a push, off the async worker.
///
/// Runs [`resolve_push_delta`] in `spawn_blocking` (git subprocess) and applies
/// the full-scan fallback for [`PinCandidates::FullScanRequired`]. Owns the
/// dispatch so the push call site stays thin and this wiring is unit-testable
/// without the HTTP layer.
///
/// Every degraded path is **logged**, not silent: a full-scan fallback, a
/// failed full scan, and a panicked blocking task each emit a warning. On a
/// failed full scan or a task panic the candidate set is empty (pin nothing
/// this push); that is a durability gap the reconciliation sweep backstops, and
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
pub async fn resolve_candidates_for_push(
repo_path: PathBuf,
new_tips: Vec<String>,
old_tips: Vec<String>,
) -> PinCandidateSet {
tokio::task::spawn_blocking(move || {
let new_refs: Vec<&str> = new_tips.iter().map(String::as_str).collect();
let old_refs: Vec<&str> = old_tips.iter().map(String::as_str).collect();
match resolve_push_delta(&repo_path, &new_refs, &old_refs) {
PinCandidates::Delta(objs) => {
tracing::info!(delta = objs.len(), repo = %repo_path.display(), "pin candidate set from push delta");
PinCandidateSet { candidates: objs, full_scan: false }
}
PinCandidates::FullScanRequired => {
tracing::warn!(repo = %repo_path.display(), "pin delta unavailable (non-commit tip, git error, or force-full-scan) — full-scan fallback");
match list_all_objects(&repo_path) {
Ok(objs) => PinCandidateSet { candidates: objs, full_scan: true },
Err(e) => {
tracing::warn!(repo = %repo_path.display(), err = %e, "full-scan fallback failed; pinning nothing this push (reconciliation sweep backstops)");
PinCandidateSet { candidates: Vec::new(), full_scan: false }
}
}
}
}
})
.await
.unwrap_or_else(|e| {
tracing::warn!(err = %e, "pin candidate computation task panicked; pinning nothing this push (reconciliation sweep backstops)");
PinCandidateSet { candidates: Vec::new(), full_scan: false }
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashSet;
use tempfile::TempDir;
/// Minimal git helper for building test repos.
struct Repo {
_td: TempDir,
path: std::path::PathBuf,
}
impl Repo {
fn new() -> Self {
let td = TempDir::new().unwrap();
let path = td.path().to_path_buf();
let r = Repo { _td: td, path };
r.git(&["init", "-q", "-b", "main"]);
r.git(&["config", "user.email", "t@t"]);
r.git(&["config", "user.name", "t"]);
r
}
fn git(&self, args: &[&str]) -> String {
let out = Command::new("git")
.args(args)
.current_dir(&self.path)
.output()
.unwrap();
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
fn commit_file(&self, name: &str, body: &str) -> String {
std::fs::write(self.path.join(name), body).unwrap();
self.git(&["add", name]);
self.git(&["commit", "-qm", &format!("add {name}")]);
self.git(&["rev-parse", "HEAD"])
}
fn rev(&self, r: &str) -> String {
self.git(&["rev-parse", r])
}
}
const ZERO: &str = "0000000000000000000000000000000000000000";
fn delta(c: PinCandidates) -> Vec<String> {
match c {
PinCandidates::Delta(v) => v,
PinCandidates::FullScanRequired => panic!("expected Delta, got FullScanRequired"),
}
}
#[test]
fn single_new_commit_delta_excludes_preexisting_objects() {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c2], &[&c1]))
.into_iter()
.collect();
// The new blob b.txt and commit c2 are in the delta; the old blob a.txt
// and commit c1 are not.
let new_blob = repo.rev("HEAD:b.txt");
let old_blob = repo.rev(&format!("{c1}:a.txt"));
assert!(got.contains(&c2), "new commit in delta");
assert!(got.contains(&new_blob), "new blob in delta");
assert!(!got.contains(&c1), "old commit excluded");
assert!(!got.contains(&old_blob), "old blob excluded");
}
#[test]
fn created_ref_is_superset_of_new_objects() {
// A created ref (old tip all-zeros, filtered out by caller) with no old
// tips walks everything reachable from the new tip — a superset of the
// genuinely new objects, never fewer.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c1], &[]))
.into_iter()
.collect();
assert!(got.contains(&c1));
assert!(got.contains(&repo.rev("HEAD:a.txt")));
assert!(got.contains(&repo.rev("HEAD^{tree}")));
}
#[test]
fn force_push_delta_is_objects_unique_to_new_tip() {
let repo = Repo::new();
let base = repo.commit_file("a.txt", "one\n");
let old_tip = repo.commit_file("b.txt", "two\n");
// Rewrite history: reset to base, commit a different file.
repo.git(&["reset", "-q", "--hard", &base]);
let new_tip = repo.commit_file("c.txt", "three\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&new_tip], &[&old_tip]))
.into_iter()
.collect();
assert!(got.contains(&new_tip), "new tip in delta");
assert!(got.contains(&repo.rev(&format!("{new_tip}:c.txt"))));
// No error; force-push computes new-minus-old cleanly.
}
#[test]
fn deleted_ref_only_yields_empty_delta_without_git() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
// All updates were deletions => new_tips empty after the caller strips zeros.
assert_eq!(
resolve_push_delta(&repo.path, &[], &[ZERO]),
PinCandidates::Delta(Vec::new())
);
}
#[test]
fn blob_tip_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
assert_eq!(
resolve_push_delta(&repo.path, &[&blob], &[]),
PinCandidates::FullScanRequired,
"a blob tip must force full scan (rev-list would exit 0 and walk it)"
);
}
#[test]
fn tree_tip_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let tree = repo.rev("HEAD^{tree}");
assert_eq!(
resolve_push_delta(&repo.path, &[&tree], &[]),
PinCandidates::FullScanRequired
);
}
#[test]
fn annotated_tag_of_tree_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let tree = repo.rev("HEAD^{tree}");
repo.git(&["tag", "-a", "treetag", "-m", "x", &tree]);
let tag = repo.rev("treetag");
assert_eq!(
resolve_push_delta(&repo.path, &[&tag], &[]),
PinCandidates::FullScanRequired,
"annotated tag peeling to a tree must force full scan"
);
}
#[test]
fn tag_of_tag_of_noncommit_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
repo.git(&["tag", "-a", "t1", "-m", "x", &blob]);
let t1 = repo.rev("t1");
repo.git(&["tag", "-a", "t2", "-m", "x", &t1]);
let t2 = repo.rev("t2");
assert_eq!(
resolve_push_delta(&repo.path, &[&t2], &[]),
PinCandidates::FullScanRequired
);
}
#[test]
fn annotated_tag_of_commit_proceeds_as_delta() {
// DISCRIMINATING positive-path regression: an annotated tag whose peeled
// target is a commit must return Delta, not FullScanRequired. A
// FullScanRequired here means the guard used bare `cat-file -t` (returns
// `tag`) instead of the `^{}` peel.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
repo.git(&["tag", "-a", "rel", "-m", "release", &c1]);
let tag = repo.rev("rel");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&tag], &[]))
.into_iter()
.collect();
assert!(
got.contains(&c1),
"peeled commit's objects are in the delta"
);
}
#[test]
fn tag_of_tag_of_commit_proceeds_as_delta() {
// Deep-peel positive path: tag -> tag -> commit must recurse to `commit`
// via `^{}` and return Delta. Complements tag_of_tag_of_noncommit (which
// forces full scan) — guards against a regression in peel depth that
// would silently downgrade a legitimate nested-tag push to full scan.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
repo.git(&["tag", "-a", "t1", "-m", "x", &c1]);
let t1 = repo.rev("t1");
repo.git(&["tag", "-a", "t2", "-m", "x", &t1]);
let t2 = repo.rev("t2");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&t2], &[]))
.into_iter()
.collect();
assert!(
got.contains(&c1),
"peeled commit's objects are in the delta"
);
}
#[tokio::test]
async fn resolve_candidates_for_push_returns_delta() {
// The extracted wiring returns the per-push delta on the happy path.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let set =
resolve_candidates_for_push(repo.path.clone(), vec![c2.clone()], vec![c1.clone()])
.await;
assert!(!set.full_scan, "happy-path delta is not a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
let new_blob = repo.rev("HEAD:b.txt");
assert!(
got.contains(&c2) && got.contains(&new_blob),
"new objects pinned"
);
assert!(!got.contains(&c1), "old commit excluded from delta");
}
#[tokio::test]
async fn resolve_candidates_for_push_falls_back_to_full_scan_on_noncommit_tip() {
// A non-commit tip forces FullScanRequired, and the wiring resolves that
// to the whole-repo list (a superset), never an empty set.
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
let all: HashSet<String> = list_all_objects(&repo.path).unwrap().into_iter().collect();
let set = resolve_candidates_for_push(repo.path.clone(), vec![blob], vec![]).await;
assert!(set.full_scan, "non-commit tip is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(got, all, "non-commit tip falls back to full repo scan");
}
#[test]
fn missing_oid_tip_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let bogus = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
assert_eq!(
resolve_push_delta(&repo.path, &[bogus], &[]),
PinCandidates::FullScanRequired,
"a missing/corrupt tip OID must force full scan"
);
}
#[test]
fn present_noncommit_old_tip_is_safe() {
// A present-but-non-commit old_sha (a --not arg) must not under-pin:
// either rev-list errors (FullScanRequired) or it over-lists (Delta).
// Either way the new tip's objects are covered; never fewer.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let old_tree = repo.rev(&format!("{c1}^{{tree}}"));
let result = resolve_push_delta(&repo.path, &[&c2], &[&old_tree]);
match result {
PinCandidates::FullScanRequired => {} // safe: caller full-scans
PinCandidates::Delta(objs) => {
// Safe direction: must still contain the new commit (over-list ok).
assert!(objs.contains(&c2), "new tip objects must be covered");
}
}
}
#[test]
fn multi_ref_push_unions_ranges() {
let repo = Repo::new();
let base = repo.commit_file("a.txt", "one\n");
// branch1 advances
let b1 = repo.commit_file("b.txt", "two\n");
// branch2 from base advances independently
repo.git(&["checkout", "-q", "-b", "branch2", &base]);
let b2 = repo.commit_file("c.txt", "three\n");
let got: HashSet<String> =
delta(resolve_push_delta(&repo.path, &[&b1, &b2], &[&base, &base]))
.into_iter()
.collect();
assert!(got.contains(&b1), "branch1 new commit");
assert!(got.contains(&b2), "branch2 new commit");
}
#[test]
fn empty_repo_no_tips_yields_empty_delta() {
let repo = Repo::new();
assert_eq!(
resolve_push_delta(&repo.path, &[], &[]),
PinCandidates::Delta(Vec::new())
);
}
#[test]
fn force_full_scan_flag_overrides() {
// Test the kill-switch via the pure inner fn so we never touch
// process-global env (which would race with parallel tests, and is
// unsafe in Rust 2024). The public resolve_push_delta wires the env
// read into this same flag.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
assert_eq!(
resolve_push_delta_inner(&repo.path, &[&c1], &[], true),
PinCandidates::FullScanRequired
);
// And with the flag off, the same push yields a Delta.
assert!(matches!(
resolve_push_delta_inner(&repo.path, &[&c1], &[], false),
PinCandidates::Delta(_)
));
}
#[test]
fn list_all_objects_returns_full_repo() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
repo.commit_file("b.txt", "two\n");
let all = list_all_objects(&repo.path).unwrap();
// 2 commits + 2 trees + 2 blobs = 6 objects.
assert_eq!(all.len(), 6, "got: {all:?}");
}
#[test]
fn all_blob_oids_includes_dangling_and_excludes_nonblobs() {
// The all-blob universe must include a dangling blob (the #99 hazard) and
// exclude commits/trees, so the fail-closed filter can tell a blob from a
// structural object without typing the candidate list.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let reachable_blob = repo.rev("HEAD:a.txt");
let tree = repo.rev("HEAD^{tree}");
// Write a blob referenced by no tree/commit — dangling.
std::fs::write(repo.path.join("orphan.bin"), b"dangling\n").unwrap();
let dangling = repo.git(&["hash-object", "-w", "orphan.bin"]);
let blobs = all_blob_oids(&repo.path).unwrap();
assert!(blobs.contains(&reachable_blob), "reachable blob present");
assert!(
blobs.contains(&dangling),
"dangling blob present in the all-blob universe"
);
assert!(!blobs.contains(&c1), "commit is not a blob");
assert!(!blobs.contains(&tree), "tree is not a blob");
// The typed lister tags each object; spot-check the dangling blob's type.
let typed = list_all_objects_with_type(&repo.path).unwrap();
assert!(
typed
.iter()
.any(|(oid, ty)| oid == &dangling && ty == "blob"),
"dangling object is typed as a blob"
);
}
}
+336 -24
View File
@@ -33,8 +33,23 @@ pub struct RepoStore {
}
impl RepoStore {
#[cfg(test)]
pub fn for_testing(repos_dir: PathBuf, pool: PgPool) -> Self {
Self {
repos_dir,
tigris: None,
pool,
migrated: Arc::new(tokio::sync::Mutex::new(std::collections::HashSet::new())),
}
}
pub fn new(repos_dir: PathBuf, tigris: Option<TigrisClient>, pool: PgPool) -> Self {
Self { repos_dir, tigris, pool, migrated: Arc::new(Mutex::new(HashSet::new())) }
Self {
repos_dir,
tigris,
pool,
migrated: Arc::new(Mutex::new(HashSet::new())),
}
}
/// Ensure a repo is available on local disk, downloading from Tigris if needed.
@@ -42,7 +57,7 @@ impl RepoStore {
/// spawned to lazily migrate it (on-demand migration for pre-Tigris repos).
/// Returns the local path to the bare repo.
pub async fn acquire(&self, owner_did: &str, repo_name: &str) -> Result<PathBuf> {
let (owner_slug, local_path) = self.local_path(owner_did, repo_name);
let (owner_slug, local_path) = self.local_path(owner_did, repo_name)?;
// Fast path: repo exists locally
if local_path.exists() {
@@ -87,10 +102,15 @@ impl RepoStore {
if let Some(ref tigris) = self.tigris {
if tigris.exists(&owner_slug, repo_name).await.unwrap_or(false) {
debug!(repo = %repo_name, "cache miss — downloading from tigris");
tigris.download(&owner_slug, repo_name, &local_path).await
tigris
.download(&owner_slug, repo_name, &local_path)
.await
.context("downloading repo from tigris")?;
// Mark as migrated since we just downloaded it
self.migrated.lock().await.insert(format!("{owner_slug}/{repo_name}"));
self.migrated
.lock()
.await
.insert(format!("{owner_slug}/{repo_name}"));
return Ok(local_path);
}
}
@@ -105,13 +125,24 @@ impl RepoStore {
/// `git-receive-pack`) so the client sees the same refs that `acquire_write()`
/// will operate on.
pub async fn acquire_fresh(&self, owner_did: &str, repo_name: &str) -> Result<PathBuf> {
let (owner_slug, local_path) = self.local_path(owner_did, repo_name);
let (owner_slug, local_path) = self.local_path(owner_did, repo_name)?;
if let Some(ref tigris) = self.tigris {
if tigris.exists(&owner_slug, repo_name).await.unwrap_or(false) {
debug!(repo = %repo_name, "acquire_fresh: downloading latest from tigris");
tigris.download(&owner_slug, repo_name, &local_path).await
.context("downloading repo from tigris (fresh)")?;
if let Err(e) = tigris.download(&owner_slug, repo_name, &local_path).await {
// The Tigris archive is present (HEAD ok) but unreadable — a
// corrupt/partial upload, or a transient GET failure. If we have a
// valid local copy, proceed with it rather than blocking the write;
// the post-write upload re-syncs (self-heals) Tigris. Only hard-fail
// when there is no local copy to fall back to.
if local_path.exists() {
warn!(repo = %repo_name, err = %e,
"acquire_fresh: tigris download failed — falling back to local copy");
return Ok(local_path);
}
return Err(e).context("downloading repo from tigris (fresh)");
}
return Ok(local_path);
}
}
@@ -123,7 +154,7 @@ impl RepoStore {
/// Take a write lock (Postgres advisory lock), ensure repo is local, return guard.
/// The lock prevents concurrent writes to the same repo across machines.
pub async fn acquire_write(&self, owner_did: &str, repo_name: &str) -> Result<RepoWriteGuard> {
let (owner_slug, local_path) = self.local_path(owner_did, repo_name);
let (owner_slug, local_path) = self.local_path(owner_did, repo_name)?;
let lock_key = advisory_lock_key(&owner_slug, repo_name);
// Acquire Postgres advisory lock with retry using pg_try_advisory_lock
@@ -152,8 +183,17 @@ impl RepoStore {
if let Some(ref tigris) = self.tigris {
if tigris.exists(&owner_slug, repo_name).await.unwrap_or(false) {
debug!(repo = %repo_name, "write acquire: downloading latest from tigris");
tigris.download(&owner_slug, repo_name, &local_path).await
.context("downloading repo from tigris for write")?;
if let Err(e) = tigris.download(&owner_slug, repo_name, &local_path).await {
// Same self-healing fallback as acquire_fresh: a corrupt/unreadable
// Tigris archive must not block a write when a valid local copy
// exists — release(success) will re-upload a good archive.
if local_path.exists() {
warn!(repo = %repo_name, err = %e,
"write acquire: tigris download failed — falling back to local copy");
} else {
return Err(e).context("downloading repo from tigris for write");
}
}
}
}
@@ -169,10 +209,9 @@ impl RepoStore {
/// Initialize a new bare repo on local disk and upload to Tigris.
pub async fn init(&self, owner_did: &str, repo_name: &str) -> Result<PathBuf> {
let (owner_slug, local_path) = self.local_path(owner_did, repo_name);
let (owner_slug, local_path) = self.local_path(owner_did, repo_name)?;
store::init_bare(&local_path)
.context("initializing bare repo")?;
store::init_bare(&local_path).context("initializing bare repo")?;
// Upload to Tigris in background
if let Some(ref tigris) = self.tigris {
@@ -194,7 +233,13 @@ impl RepoStore {
/// Call this after any operation that modifies the git repo on disk.
pub async fn release_after_write(&self, owner_did: &str, repo_name: &str) {
if let Some(ref tigris) = self.tigris {
let (owner_slug, local_path) = self.local_path(owner_did, repo_name);
let (owner_slug, local_path) = match self.local_path(owner_did, repo_name) {
Ok(p) => p,
Err(e) => {
warn!(repo = %repo_name, err = %e, "rejected unsafe path in release_after_write");
return;
}
};
if let Err(e) = tigris.upload(&owner_slug, repo_name, &local_path).await {
warn!(repo = %repo_name, err = %e, "failed to upload repo to tigris after write");
}
@@ -202,13 +247,106 @@ impl RepoStore {
}
/// Compute the local disk path and owner slug for a repo.
fn local_path(&self, owner_did: &str, repo_name: &str) -> (String, PathBuf) {
let owner_slug = owner_did.replace(':', "_").replace('/', "_");
let local_path = self.repos_dir.join(&owner_slug).join(format!("{repo_name}.git"));
(owner_slug, local_path)
///
/// Three-layer defence against path traversal:
/// 1. Strict allowlist on `owner_did` and `repo_name` (no `..`, slashes,
/// null bytes, leading dots; length-bounded).
/// 2. The joined path must remain rooted at `repos_dir`.
/// 3. Every component of the joined path must be `Component::Normal`
/// (or the prefix/root from `repos_dir`); any `ParentDir`/`CurDir`
/// segment is rejected. This is the CodeQL-recognised barrier
/// pattern for `rust/path-injection`.
fn local_path(&self, owner_did: &str, repo_name: &str) -> Result<(String, PathBuf)> {
validate_path_components(owner_did, repo_name)?;
let owner_slug = owner_did.replace([':', '/'], "_");
let local_path = self
.repos_dir
.join(&owner_slug)
.join(format!("{repo_name}.git"));
if !local_path.starts_with(&self.repos_dir) {
anyhow::bail!(
"computed repo path escaped repos_dir: {}",
local_path.display()
);
}
// Explicit component walk — sanitisation barrier that static analysers
// (CodeQL `rust/path-injection`) recognise. The path must be composed
// entirely of Normal segments after the root prefix; any ParentDir or
// CurDir component is a traversal attempt.
for component in local_path.components() {
use std::path::Component;
match component {
Component::Prefix(_) | Component::RootDir | Component::Normal(_) => {}
Component::ParentDir => {
anyhow::bail!("path contains parent-directory component");
}
Component::CurDir => {
anyhow::bail!("path contains current-directory component");
}
}
}
Ok((owner_slug, local_path))
}
}
/// Strict allowlist validator for `owner_did` and `repo_name`.
///
/// Rejects any character that isn't explicitly safe, plus length and
/// special-sequence checks (`..`, leading `.`, leading `-`).
fn validate_path_components(owner_did: &str, repo_name: &str) -> Result<()> {
validate_owner_did(owner_did)?;
validate_repo_name(repo_name)?;
Ok(())
}
fn validate_owner_did(owner_did: &str) -> Result<()> {
if owner_did.is_empty() {
anyhow::bail!("owner_did is empty");
}
if owner_did.len() > 256 {
anyhow::bail!("owner_did exceeds 256 chars");
}
// DIDs are `did:method:identifier` — `did:key:z6Mk...`, `did:web:host:user`, etc.
// Allow alnum + `:`, `.`, `_`, `-`. Reject `..` substring and any `/` or `\`.
if owner_did.contains("..") {
anyhow::bail!("owner_did contains '..' sequence");
}
for ch in owner_did.chars() {
let ok = ch.is_ascii_alphanumeric() || matches!(ch, ':' | '.' | '_' | '-');
if !ok {
anyhow::bail!("owner_did contains disallowed character: {ch:?}");
}
}
Ok(())
}
fn validate_repo_name(repo_name: &str) -> Result<()> {
if repo_name.is_empty() {
anyhow::bail!("repo_name is empty");
}
if repo_name.len() > 100 {
anyhow::bail!("repo_name exceeds 100 chars");
}
// Repo names are `[A-Za-z0-9._-]+` minus path-traversal traps.
if repo_name.contains("..") {
anyhow::bail!("repo_name contains '..' sequence");
}
if repo_name.starts_with('.') || repo_name.starts_with('-') {
anyhow::bail!("repo_name must not start with '.' or '-'");
}
for ch in repo_name.chars() {
let ok = ch.is_ascii_alphanumeric() || matches!(ch, '.' | '_' | '-');
if !ok {
anyhow::bail!("repo_name contains disallowed character: {ch:?}");
}
}
Ok(())
}
/// Guard returned by `acquire_write()`. Holds the Postgres advisory lock and
/// uploads to Tigris + releases the lock on `release()`.
pub struct RepoWriteGuard {
@@ -226,13 +364,24 @@ impl RepoWriteGuard {
&self.local_path
}
/// Upload to Tigris and release the advisory lock. Call this when the write is done.
pub async fn release(self) {
// Upload to Tigris
if let Some(ref tigris) = self.tigris {
if let Err(e) = tigris.upload(&self.owner_slug, &self.repo_name, &self.local_path).await {
warn!(repo = %self.repo_name, err = %e, "failed to upload repo to tigris after write");
/// Upload to Tigris (only when the write succeeded) and release the advisory
/// lock. Pass `success = false` when the write operation failed — uploading a
/// half-applied or otherwise inconsistent repo would propagate corruption to
/// Tigris (and to every node that later downloads it). The lock is always
/// released regardless, to avoid stale locks blocking future writes.
pub async fn release(self, success: bool) {
// Upload to Tigris only on success.
if success {
if let Some(ref tigris) = self.tigris {
if let Err(e) = tigris
.upload(&self.owner_slug, &self.repo_name, &self.local_path)
.await
{
warn!(repo = %self.repo_name, err = %e, "failed to upload repo to tigris after write");
}
}
} else {
warn!(repo = %self.repo_name, "write failed — skipping tigris upload to avoid propagating an inconsistent repo");
}
// Release advisory lock
@@ -251,3 +400,166 @@ fn advisory_lock_key(owner_slug: &str, repo_name: &str) -> i64 {
repo_name.hash(&mut hasher);
hasher.finish() as i64
}
#[cfg(test)]
mod tests {
use super::*;
// ── repo_name validation ───────────────────────────────────────────────
#[test]
fn repo_name_accepts_normal_names() {
for name in [
"hello",
"hello-world",
"hello_world",
"hello.world",
"Repo123",
"a",
] {
validate_repo_name(name).unwrap_or_else(|e| panic!("{name} should be valid: {e}"));
}
}
#[test]
fn repo_name_rejects_empty() {
assert!(validate_repo_name("").is_err());
}
#[test]
fn repo_name_rejects_path_traversal_dotdot() {
for name in ["..", "../etc", "../../passwd", "foo/../bar", "a..b"] {
assert!(
validate_repo_name(name).is_err(),
"{name:?} must be rejected"
);
}
}
#[test]
fn repo_name_rejects_slashes() {
for name in ["foo/bar", "foo\\bar", "/abs", "a/b/c"] {
assert!(
validate_repo_name(name).is_err(),
"{name:?} must be rejected"
);
}
}
#[test]
fn repo_name_rejects_leading_dot_or_dash() {
for name in [".hidden", ".", "-foo"] {
assert!(
validate_repo_name(name).is_err(),
"{name:?} must be rejected"
);
}
}
#[test]
fn repo_name_rejects_null_byte() {
assert!(validate_repo_name("foo\0bar").is_err());
}
#[test]
fn repo_name_rejects_overlong() {
let long = "a".repeat(101);
assert!(validate_repo_name(&long).is_err());
}
// ── owner_did validation ───────────────────────────────────────────────
#[test]
fn owner_did_accepts_did_key() {
validate_owner_did("did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr").unwrap();
}
#[test]
fn owner_did_accepts_did_web_with_dots() {
validate_owner_did("did:web:example.com:user").unwrap();
}
#[test]
fn owner_did_rejects_empty() {
assert!(validate_owner_did("").is_err());
}
#[test]
fn owner_did_rejects_path_traversal() {
for did in [
"did:key:..",
"did:key:../../etc",
"..",
"did:key:foo/../bar",
] {
assert!(validate_owner_did(did).is_err(), "{did:?} must be rejected");
}
}
#[test]
fn owner_did_rejects_slashes_and_backslashes() {
for did in ["did:key:foo/bar", "did:key:foo\\bar", "did/key/foo"] {
assert!(validate_owner_did(did).is_err(), "{did:?} must be rejected");
}
}
#[test]
fn owner_did_rejects_null_byte() {
assert!(validate_owner_did("did:key:z6Mk\0evil").is_err());
}
#[test]
fn owner_did_rejects_overlong() {
let long = format!("did:key:{}", "z".repeat(260));
assert!(validate_owner_did(&long).is_err());
}
// ── end-to-end local_path ──────────────────────────────────────────────
fn make_store() -> RepoStore {
// We only exercise the path-construction code, which doesn't touch
// the pool or the network. Fabricate a pool reference via PgPool::connect_lazy
// so we don't need a live DB.
let pool = sqlx::PgPool::connect_lazy("postgres://invalid").unwrap();
RepoStore::new(PathBuf::from("/var/lib/gitlawb/repos"), None, pool)
}
#[tokio::test]
async fn local_path_resolves_safe_inputs() {
let store = make_store();
let (slug, path) = store
.local_path(
"did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr",
"hello",
)
.unwrap();
assert_eq!(
slug,
"did_key_z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr"
);
assert!(path.starts_with("/var/lib/gitlawb/repos"));
assert!(path.ends_with("hello.git"));
}
#[tokio::test]
async fn local_path_rejects_traversal_in_repo_name() {
let store = make_store();
for bad in ["../etc/passwd", "..", "../../shadow"] {
assert!(
store.local_path("did:key:z6MkAlice", bad).is_err(),
"repo_name={bad:?} must be rejected"
);
}
}
#[tokio::test]
async fn local_path_rejects_traversal_in_owner_did() {
let store = make_store();
for bad in ["did:key:..", "..", "did/key/foo"] {
assert!(
store.local_path(bad, "hello").is_err(),
"owner_did={bad:?} must be rejected"
);
}
}
}
+767 -10
View File
@@ -1,12 +1,13 @@
use std::path::Path;
use std::process::Stdio;
use anyhow::{bail, Context, Result};
use axum::body::Body;
use axum::http::StatusCode;
use axum::response::Response;
use bytes::Bytes;
use std::collections::HashSet;
use std::path::Path;
use std::process::Stdio;
use tokio::io::AsyncWriteExt;
use tokio::process::Command;
use anyhow::{bail, Result};
/// Handle `GET /:owner/:repo/info/refs?service=git-upload-pack`
/// or `?service=git-receive-pack`
@@ -74,23 +75,83 @@ pub async fn receive_pack(repo_path: &Path, request_body: Bytes) -> Result<Respo
.body(Body::from(output))?)
}
/// Sends SIGTERM to a child's whole process group on drop, unless disarmed first.
///
/// A served `git upload-pack`/`receive-pack` forks helpers such as `pack-objects`.
/// If the request future is dropped (client disconnect) or returns early, dropping
/// the tokio `Child` does not signal `git`; it lingers until EPIPE, and its
/// `pack-objects` child can reparent to PID 1 and never be reaped — a zombie that
/// accumulates until `fork()` fails with EAGAIN (#53). Spawning the child in its
/// own process group and signalling that group here tears the whole tree down at
/// the source. SIGTERM (not SIGKILL) lets `git` run its cleanup — notably removing
/// `.git/*.lock` files mid-`receive-pack` — before it exits, so an aborted push
/// can't leave a stale lock that blocks the next one. The guard is disarmed once
/// `wait_with_output()` returns, so a request that completed cleanly never signals.
#[cfg(unix)]
struct KillGroupOnDrop {
pgid: Option<i32>,
}
#[cfg(unix)]
impl KillGroupOnDrop {
fn disarm(&mut self) {
self.pgid = None;
}
}
#[cfg(unix)]
impl Drop for KillGroupOnDrop {
fn drop(&mut self) {
if let Some(pgid) = self.pgid {
// SAFETY: kill(2) takes only integer arguments and borrows no Rust
// memory. Signalling a stale group just returns ESRCH, which we ignore.
unsafe {
libc::kill(-pgid, libc::SIGTERM);
}
}
}
}
async fn run_git_service(service: &str, repo_path: &Path, input: Bytes) -> Result<Vec<u8>> {
let mut child = Command::new("git")
let mut command = Command::new("git");
command
.arg(service_to_command(service))
.arg("--stateless-rpc")
.arg(repo_path)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?;
.stderr(Stdio::piped());
// Write request body to git's stdin
if let Some(mut stdin) = child.stdin.take() {
stdin.write_all(&input).await?;
}
// Run git in its own process group so the whole tree (git + pack-objects)
// can be signalled together on disconnect rather than orphaning a grandchild.
#[cfg(unix)]
command.process_group(0);
let mut child = command.spawn()?;
// Arm the group-kill guard for the lifetime of the request. With
// process_group(0) the child is its own group leader, so pgid == its pid.
#[cfg(unix)]
let mut group_guard = KillGroupOnDrop {
pgid: child.id().map(|id| id as i32),
};
// Write the request body to git's stdin, but don't early-return on a write
// error: always reap the child first (below), so the guard only ever fires on
// an actual future-drop (client disconnect), never on a pid we just reaped.
let write_result: std::io::Result<()> = match child.stdin.take() {
Some(mut stdin) => stdin.write_all(&input).await,
None => Ok(()),
};
let output = child.wait_with_output().await?;
// Child reaped, so its group is gone: disarm before surfacing any error.
#[cfg(unix)]
group_guard.disarm();
write_result.context("failed to write to git stdin")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("{service} failed: {stderr}");
@@ -120,3 +181,699 @@ fn pkt_line(data: &str) -> Vec<u8> {
let len = data.len() + 4;
format!("{len:04x}{data}").into_bytes()
}
/// Build a packfile containing every object reachable from all refs EXCEPT the
/// given blob OIDs. Commits and trees are always included, so SHAs stay intact;
/// only the named blobs are dropped.
pub fn build_filtered_pack(repo_path: &Path, withheld: &HashSet<String>) -> Result<Vec<u8>> {
// All reachable objects as "oid [path]" lines.
let rev = std::process::Command::new("git")
.args(["rev-list", "--objects", "--all"])
.current_dir(repo_path)
.output()?;
if !rev.status.success() {
bail!(
"git rev-list failed: {}",
String::from_utf8_lossy(&rev.stderr)
);
}
let mut keep = Vec::new();
for line in String::from_utf8_lossy(&rev.stdout).lines() {
let oid = line.split_whitespace().next().unwrap_or("");
if oid.is_empty() || withheld.contains(oid) {
continue;
}
keep.push(oid.to_string());
}
let mut child = std::process::Command::new("git")
.args(["pack-objects", "--stdout"])
.current_dir(repo_path)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?;
// Feed the object ids on stdin, but always reap the child afterward even if
// the write fails or stdin is missing, so an error can't drop the Child
// unwaited and leak a zombie (#53).
let write_result: std::io::Result<()> = {
use std::io::Write as _;
match child.stdin.take() {
Some(mut stdin) => {
let mut data = keep.join("\n").into_bytes();
data.push(b'\n');
stdin.write_all(&data)
}
None => Err(std::io::Error::new(
std::io::ErrorKind::BrokenPipe,
"git pack-objects stdin unavailable",
)),
}
};
let out = child.wait_with_output()?;
write_result.context("failed to write object ids to git pack-objects stdin")?;
if !out.status.success() {
bail!(
"git pack-objects failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
Ok(out.stdout)
}
/// Serve a clone/fetch with the withheld blobs removed from the response pack.
///
/// The framing is git protocol v0 (`NAK` then the pack), matching the v0 ref
/// advertisement that `info_refs` emits (it runs `git upload-pack
/// --advertise-refs` without `GIT_PROTOCOL=version=2`, so clients negotiate v0).
/// If `info_refs` ever advertises v2, this serve path must learn v2 framing too.
///
/// Because the pack deliberately omits blobs that the sent trees still
/// reference, the pack is not closed under reachability. A stock full clone
/// rejects it at fetch time ("remote did not send all necessary objects"); only
/// a partial clone (the client passes `--filter`, marking a promisor remote)
/// accepts the pack with the private blobs absent. Tree and commit SHAs stay
/// intact either way. The clean partial-clone client UX is a separate follow-up
/// (git-remote-gitlawb); the security guarantee (private bytes never leave the
/// node) holds regardless of client.
///
/// Negotiation is intentionally ignored: rather than honoring the client's
/// `want`/`have` lines, this always sends a self-contained pack of every object
/// across all refs minus the withheld blobs, and replies `NAK`. A fresh clone
/// and an incremental fetch are both correct (the client de-duplicates objects
/// it already has); the cost is that a fetch re-sends the full object set
/// instead of a thin delta. Honoring negotiation for smaller fetch packs is an
/// optimization follow-up, not a correctness requirement.
pub async fn upload_pack_excluding(
repo_path: &Path,
request_body: Bytes,
withheld: &HashSet<String>,
) -> Result<Response> {
// build_filtered_pack shells out to git (rev-list, pack-objects) with
// blocking std::process I/O; run it off the async worker so a large repo's
// pack build does not stall the tokio runtime.
let pack = {
let repo_path = repo_path.to_path_buf();
let withheld = withheld.clone();
tokio::task::spawn_blocking(move || build_filtered_pack(&repo_path, &withheld))
.await
.context("filtered-pack build task panicked")??
};
// The client lists its capabilities on the first `want` line. Honor
// side-band-64k when offered (every modern smart-HTTP client offers it);
// otherwise stream the raw pack after NAK.
let sideband = memmem(&request_body, b"side-band-64k");
let mut body = Vec::new();
body.extend_from_slice(&pkt_line("NAK\n"));
if sideband {
// Band 1 carries pack data, chunked under the pkt-line size limit.
for chunk in pack.chunks(65515) {
let mut framed = Vec::with_capacity(chunk.len() + 1);
framed.push(0x01);
framed.extend_from_slice(chunk);
let len = framed.len() + 4;
body.extend_from_slice(format!("{len:04x}").as_bytes());
body.extend_from_slice(&framed);
}
body.extend_from_slice(b"0000");
} else {
body.extend_from_slice(&pack);
}
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/x-git-upload-pack-result")
.header("Cache-Control", "no-cache")
.body(Body::from(body))?)
}
/// True if `needle` occurs anywhere in `haystack`. Small substring scan used to
/// detect a client capability token in the upload-pack request body.
fn memmem(haystack: &[u8], needle: &[u8]) -> bool {
if needle.is_empty() || haystack.len() < needle.len() {
return needle.is_empty();
}
haystack
.windows(needle.len())
.any(|window| window == needle)
}
#[cfg(test)]
mod tests {
use super::*;
use std::process::Command;
use tempfile::TempDir;
/// List OIDs in a pack by writing it to a temp dir and running verify-pack.
pub(super) fn pack_object_ids(pack: &[u8]) -> std::collections::HashSet<String> {
let dir = TempDir::new().unwrap();
let path = dir.path().join("test.pack");
std::fs::write(&path, pack).unwrap();
// index-pack creates the matching .idx next to the pack.
let ok = Command::new("git")
.args(["index-pack", path.to_str().unwrap()])
.status()
.unwrap()
.success();
assert!(ok, "index-pack failed");
let out = Command::new("git")
.args(["verify-pack", "-v", path.to_str().unwrap()])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout)
.lines()
.filter_map(|l| l.split_whitespace().next())
.filter(|t| t.len() == 40 && t.chars().all(|c| c.is_ascii_hexdigit()))
.map(|s| s.to_string())
.collect()
}
#[tokio::test]
async fn filtered_serve_excludes_withheld_blob() {
// Build a bare repo, capture the secret + public blob OIDs.
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("bare.git");
let g = |args: &[&str], dir: &std::path::Path| {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
};
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"pub\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"SECRET\n").unwrap();
g(&["init", "-q"], &work);
g(&["config", "user.email", "t@t"], &work);
g(&["config", "user.name", "t"], &work);
g(&["add", "."], &work);
g(&["commit", "-qm", "init"], &work);
let oid = |p: &str| {
let o = Command::new("git")
.args(["rev-parse", &format!("HEAD:{p}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let secret = oid("secret/b.txt");
let public = oid("public/a.txt");
g(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
let mut withheld = std::collections::HashSet::new();
withheld.insert(secret.clone());
let pack = build_filtered_pack(&bare, &withheld).unwrap();
let ids = pack_object_ids(&pack);
assert!(ids.contains(&public), "public blob must be in the pack");
assert!(
!ids.contains(&secret),
"secret blob must NOT be in the pack"
);
}
#[tokio::test]
async fn client_clone_lacks_withheld_blob_bytes() {
use axum::body::to_bytes;
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("bare.git");
let g = |args: &[&str], dir: &std::path::Path| {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
};
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"pub\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"SECRET\n").unwrap();
g(&["init", "-q"], &work);
g(&["config", "user.email", "t@t"], &work);
g(&["config", "user.name", "t"], &work);
g(&["add", "."], &work);
g(&["commit", "-qm", "init"], &work);
let oid = |p: &str| {
let o = Command::new("git")
.args(["rev-parse", &format!("HEAD:{p}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let secret_oid = oid("secret/b.txt");
let public_oid = oid("public/a.txt");
g(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
let mut withheld = std::collections::HashSet::new();
withheld.insert(secret_oid.clone());
// A realistic v0 request advertises side-band-64k, so the serve frames
// the pack in band 1 (the path real clients exercise).
let req = Bytes::from_static(
b"0098want 0000000000000000000000000000000000000000 \
side-band-64k ofs-delta agent=git/2\n00000009done\n",
);
let resp = upload_pack_excluding(&bare, req, &withheld).await.unwrap();
let body = to_bytes(resp.into_body(), usize::MAX).await.unwrap();
let ids = pack_object_ids(&extract_pack(&body));
assert!(
ids.contains(&public_oid),
"public blob must be present in served pack"
);
assert!(
!ids.contains(&secret_oid),
"withheld blob must be absent from served pack"
);
}
/// Strip the v0 upload-pack framing (NAK line + sideband-64k bands),
/// returning the raw pack. Mirrors how a client de-frames the band-1 stream.
fn extract_pack(body: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
let mut i = 0;
while i + 4 <= body.len() {
let len =
usize::from_str_radix(std::str::from_utf8(&body[i..i + 4]).unwrap_or("0000"), 16)
.unwrap_or(0);
if len == 0 {
i += 4;
continue;
}
let chunk = &body[i + 4..i + len];
// band 1 = pack data; skip the NAK line and any other bands.
if chunk.first() == Some(&0x01) {
out.extend_from_slice(&chunk[1..]);
}
i += len;
}
out
}
// Shared harness for the real-git server tests: a minimal smart-HTTP server
// backed by the real info_refs + upload_pack_excluding.
#[derive(Clone)]
struct FilterState {
repo: std::path::PathBuf,
withheld: HashSet<String>,
}
async fn refs_handler(
axum::extract::State(st): axum::extract::State<std::sync::Arc<FilterState>>,
axum::extract::Query(q): axum::extract::Query<std::collections::HashMap<String, String>>,
) -> Response {
let service = q.get("service").cloned().unwrap_or_default();
info_refs(&st.repo, &service).await.unwrap()
}
async fn pack_handler(
axum::extract::State(st): axum::extract::State<std::sync::Arc<FilterState>>,
body: Bytes,
) -> Response {
upload_pack_excluding(&st.repo, body, &st.withheld)
.await
.unwrap()
}
/// Spawn the server for `bare`, withholding `withheld`. Returns the clone URL
/// and the server task (abort it when done).
async fn spawn_filter_server(
bare: std::path::PathBuf,
withheld: HashSet<String>,
) -> (String, tokio::task::JoinHandle<()>) {
use axum::routing::{get, post};
let state = std::sync::Arc::new(FilterState {
repo: bare,
withheld,
});
let app = axum::Router::new()
.route("/repo.git/info/refs", get(refs_handler))
.route("/repo.git/git-upload-pack", post(pack_handler))
.with_state(state);
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let handle = tokio::spawn(async move {
axum::serve(listener, app).await.unwrap();
});
(format!("http://127.0.0.1:{port}/repo.git"), handle)
}
fn run_git(args: &[&str], dir: &std::path::Path) {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
}
/// Build a work repo (public/a.txt, secret/b.txt) and a bare clone of it.
/// Returns (work, bare, secret_blob_oid, public_blob_oid).
fn fixture_with_secret(
td: &TempDir,
) -> (std::path::PathBuf, std::path::PathBuf, String, String) {
let work = td.path().join("work");
let bare = td.path().join("bare.git");
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"pub\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"SECRET\n").unwrap();
run_git(&["init", "-q"], &work);
run_git(&["config", "user.email", "t@t"], &work);
run_git(&["config", "user.name", "t"], &work);
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "init"], &work);
let oid = |p: &str| {
let o = Command::new("git")
.args(["rev-parse", &format!("HEAD:{p}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let secret_oid = oid("secret/b.txt");
let public_oid = oid("public/a.txt");
run_git(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
(work, bare, secret_oid, public_oid)
}
/// Enumerate exactly the objects a repo physically has (no promisor lazy
/// fetch), so tests assert on what bytes actually crossed the wire.
fn local_object_ids(repo: &std::path::Path) -> String {
let out = Command::new("git")
.args(["cat-file", "--batch-all-objects", "--batch-check"])
.current_dir(repo)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).into_owned()
}
/// End-to-end: a real `git` client clones through `info_refs` +
/// `upload_pack_excluding` and ends up without the withheld blob's bytes
/// while still seeing its tree entry (SHA). Uses a partial clone
/// (`--filter`) because a pack that omits a referenced blob is only
/// accepted by a promisor-aware client; a stock full clone is refused at
/// fetch time by the connectivity check.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn real_git_partial_clone_omits_withheld_blob() {
let td = TempDir::new().unwrap();
let (_work, bare, secret_oid, public_oid) = fixture_with_secret(&td);
let (url, server) = spawn_filter_server(bare, HashSet::from([secret_oid.clone()])).await;
let dest = td.path().join("clone");
let dest_s = dest.to_str().unwrap().to_string();
let out = tokio::task::spawn_blocking(move || {
Command::new("git")
.args([
"-c",
"protocol.version=2",
"clone",
"--filter=blob:none",
"--no-checkout",
"-q",
&url,
&dest_s,
])
.output()
.unwrap()
})
.await
.unwrap();
assert!(
out.status.success(),
"clone failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// The public blob is present in the clone, the withheld blob is not.
let local = local_object_ids(&dest);
assert!(
local.contains(&public_oid),
"public blob should be present in the clone"
);
assert!(
!local.contains(&secret_oid),
"withheld blob bytes must be absent from the clone"
);
// The tree entry (and SHA) for the private file is still visible.
let tree = Command::new("git")
.args(["ls-tree", "-r", "HEAD"])
.current_dir(&dest)
.output()
.unwrap();
let tree = String::from_utf8_lossy(&tree.stdout);
assert!(
tree.contains(&secret_oid) && tree.contains("secret/b.txt"),
"the private path and its blob SHA must remain visible: {tree}"
);
server.abort();
}
/// End-to-end: an incremental `git fetch` after a partial clone still works
/// and still withholds the private blob. The serve path ignores the client's
/// have/want negotiation and always sends a self-contained pack of all refs
/// minus the withheld blobs (it replies NAK, so the client treats it as "no
/// common commits" and accepts the full set). This is correct, just not
/// bandwidth-optimal; thin-pack/negotiation is an optimization follow-up.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn real_git_fetch_after_partial_clone_still_withholds() {
let td = TempDir::new().unwrap();
let (work, bare, secret_oid, _public_oid) = fixture_with_secret(&td);
let branch = {
let o = Command::new("git")
.args(["symbolic-ref", "--short", "HEAD"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let (url, server) =
spawn_filter_server(bare.clone(), HashSet::from([secret_oid.clone()])).await;
// Partial-clone the initial state.
let dest = td.path().join("clone");
let dest_s = dest.to_str().unwrap().to_string();
let url_c = url.clone();
let out = tokio::task::spawn_blocking(move || {
Command::new("git")
.args([
"-c",
"protocol.version=2",
"clone",
"--filter=blob:none",
"--no-checkout",
"-q",
&url_c,
&dest_s,
])
.output()
.unwrap()
})
.await
.unwrap();
assert!(
out.status.success(),
"clone failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// Add a new public commit on the server side.
std::fs::write(work.join("public/c.txt"), b"v2\n").unwrap();
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "c2"], &work);
let new_oid = {
let o = Command::new("git")
.args(["rev-parse", "HEAD:public/c.txt"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
run_git(&["push", "-q", bare.to_str().unwrap(), &branch], &work);
// Incremental fetch: the client has c1 and asks for the update.
let dest_f = dest.clone();
let out = tokio::task::spawn_blocking(move || {
Command::new("git")
.args(["-c", "protocol.version=2", "fetch", "-q", "origin"])
.current_dir(&dest_f)
.output()
.unwrap()
})
.await
.unwrap();
assert!(
out.status.success(),
"fetch failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// The new commit's blob arrived; the withheld blob is still absent.
let local = local_object_ids(&dest);
assert!(
local.contains(&new_oid),
"the new commit's blob must be fetched"
);
assert!(
!local.contains(&secret_oid),
"withheld blob must remain absent after fetch"
);
server.abort();
}
// ── #53 regression: served-git process-group teardown ──────────────────
//
// run_git_service runs git in its own process group and SIGTERMs that group
// when the request future is dropped (client disconnect) or returns early, so
// git AND its pack-objects child die together instead of orphaning a zombie.
// These exercise the KillGroupOnDrop guard that wires that up.
#[cfg(unix)]
fn alive(pid: i32) -> bool {
// kill(pid, 0) probes existence: returns 0 while the pid exists, -1 once
// it's gone (ESRCH). Same-uid here, so EPERM never applies.
unsafe { libc::kill(pid, 0) == 0 }
}
#[cfg(unix)]
#[tokio::test]
async fn kill_group_guard_terminates_child_on_drop() {
let mut child = tokio::process::Command::new("sleep")
.arg("300")
.process_group(0)
.spawn()
.unwrap();
let pgid = child.id().map(|id| id as i32);
{
let _guard = KillGroupOnDrop { pgid };
} // guard drops here -> SIGTERM to the group
use std::os::unix::process::ExitStatusExt;
let status = child.wait().await.unwrap();
assert_eq!(
status.signal(),
Some(libc::SIGTERM),
"child must be terminated by SIGTERM via its process group"
);
}
#[cfg(unix)]
#[tokio::test]
async fn kill_group_guard_reaps_grandchild_on_drop() {
// The #53 scenario: git forks pack-objects. A group kill must reach the
// grandchild, not just the direct child. sh (the group leader) backgrounds
// a sleep (standing in for pack-objects) and prints its pid.
use tokio::io::AsyncReadExt;
let mut child = tokio::process::Command::new("sh")
.arg("-c")
.arg("sleep 300 & echo \"$!\"; wait")
.stdout(Stdio::piped())
.process_group(0)
.spawn()
.unwrap();
let pgid = child.id().map(|id| id as i32);
// Read the backgrounded grandchild's pid from the first stdout line.
let mut stdout = child.stdout.take().unwrap();
let mut buf = Vec::new();
loop {
let mut byte = [0u8; 1];
let n = stdout.read(&mut byte).await.unwrap();
if n == 0 || byte[0] == b'\n' {
break;
}
buf.push(byte[0]);
}
let grandchild: i32 = String::from_utf8(buf).unwrap().trim().parse().unwrap();
assert!(alive(grandchild), "grandchild should be running");
{
let _guard = KillGroupOnDrop { pgid };
} // group SIGTERM reaches sh AND the sleep grandchild
let _ = child.wait().await; // reap sh
// The grandchild reparents to init and is reaped; poll until it's gone.
let mut gone = false;
for _ in 0..200 {
if !alive(grandchild) {
gone = true;
break;
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
assert!(
gone,
"grandchild must be terminated by the group signal (#53)"
);
}
#[cfg(unix)]
#[tokio::test]
async fn kill_group_guard_disarmed_does_not_kill() {
// A request that completed cleanly disarms the guard; dropping it must not
// signal anything.
let mut child = tokio::process::Command::new("sleep")
.arg("300")
.process_group(0)
.spawn()
.unwrap();
{
let mut guard = KillGroupOnDrop {
pgid: child.id().map(|id| id as i32),
};
guard.disarm();
} // disarmed -> no kill
assert!(
child.try_wait().unwrap().is_none(),
"disarmed guard must not kill the child"
);
// Clean up the still-running child.
let _ = child.kill().await;
let _ = child.wait().await;
}
}
+125 -13
View File
@@ -1,6 +1,6 @@
use anyhow::{bail, Context, Result};
use std::path::{Path, PathBuf};
use std::process::Command;
use anyhow::{Result, Context, bail};
/// Initialize a new bare git repository with SHA-1 object format (default).
///
@@ -107,7 +107,8 @@ pub fn resolve_head(repo_path: &Path, preferred_branch: &str) -> String {
// 3. Walk all refs — prefer main/master, then take the first one
if let Ok(refs) = list_refs(repo_path) {
let branches: Vec<_> = refs.iter()
let branches: Vec<_> = refs
.iter()
.filter(|(r, _)| r.starts_with("refs/heads/"))
.collect();
// Preferred names in order
@@ -199,7 +200,13 @@ pub fn ls_tree(repo_path: &Path, refname: &str, tree_path: &str) -> Result<Vec<T
let kind = parts.next()?.to_string();
let hash = parts.next()?.to_string();
let size: Option<u64> = parts.next().and_then(|s| s.parse().ok());
Some(TreeEntry { mode, kind, hash, path: name.to_string(), size })
Some(TreeEntry {
mode,
kind,
hash,
path: name.to_string(),
size,
})
})
.collect();
@@ -239,7 +246,9 @@ pub struct CommitInfo {
fn serialize_timestamp<S: serde::Serializer>(ts: &i64, s: S) -> Result<S::Ok, S::Error> {
use chrono::TimeZone;
let dt = chrono::Utc.timestamp_opt(*ts, 0).single()
let dt = chrono::Utc
.timestamp_opt(*ts, 0)
.single()
.unwrap_or_else(chrono::Utc::now);
s.serialize_str(&dt.to_rfc3339())
}
@@ -275,7 +284,9 @@ pub fn read_object(repo_path: &Path, sha256_hex: &str) -> Result<Option<(String,
return Ok(None);
}
let obj_type = String::from_utf8_lossy(&type_output.stdout).trim().to_string();
let obj_type = String::from_utf8_lossy(&type_output.stdout)
.trim()
.to_string();
// Read the raw content bytes
let content_output = Command::new("git")
@@ -303,9 +314,48 @@ pub fn branch_diff(repo_path: &Path, target_branch: &str, source_branch: &str) -
Ok(String::from_utf8_lossy(&output.stdout).to_string())
}
/// The repo-relative paths changed by `git diff target...source` (the same range
/// as `branch_diff`). Used to enforce per-path visibility on a PR diff: if the
/// caller cannot read one of these paths, the diff is withheld.
pub fn branch_diff_names(
repo_path: &Path,
target_branch: &str,
source_branch: &str,
) -> Result<Vec<String>> {
let output = Command::new("git")
.args([
"diff",
"--name-only",
"-z",
&format!("{target_branch}...{source_branch}"),
])
.current_dir(repo_path)
.output()
.context("failed to run git diff --name-only")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git diff --name-only failed: {stderr}");
}
// Split on NUL (`-z`) so paths containing newlines keep their exact bytes;
// `--name-only` without `-z` would quote/escape such paths and they would no
// longer match the visibility globs in get_pr_diff, leaking the diff.
Ok(output
.stdout
.split(|b| *b == b'\0')
.filter(|s| !s.is_empty())
.map(|s| String::from_utf8_lossy(s).into_owned())
.collect())
}
/// Merge source_branch into target_branch in a bare repo using a temporary worktree.
/// Returns the new merge commit hash.
pub fn merge_branch(repo_path: &Path, target_branch: &str, source_branch: &str, author_did: &str, pr_title: &str) -> Result<String> {
pub fn merge_branch(
repo_path: &Path,
target_branch: &str,
source_branch: &str,
author_did: &str,
pr_title: &str,
) -> Result<String> {
let worktree_path = repo_path.join("_merge_worktree");
// Clean up any leftover worktree
@@ -324,13 +374,24 @@ pub fn merge_branch(repo_path: &Path, target_branch: &str, source_branch: &str,
.output()
.context("failed to create worktree")?;
if !wt.status.success() {
bail!("git worktree add failed: {}", String::from_utf8_lossy(&wt.stderr));
bail!(
"git worktree add failed: {}",
String::from_utf8_lossy(&wt.stderr)
);
}
// Run merge in worktree
let merge = Command::new("git")
.args(["merge", "--no-ff", source_branch, "-m",
&format!("Merge branch '{}' into {} ({})", source_branch, target_branch, pr_title)])
.args([
"merge",
"--no-ff",
source_branch,
"-m",
&format!(
"Merge branch '{}' into {} ({})",
source_branch, target_branch, pr_title
),
])
.current_dir(&worktree_path)
.env("GIT_AUTHOR_NAME", author_did)
.env("GIT_AUTHOR_EMAIL", format!("{}@gitlawb", author_did))
@@ -349,7 +410,10 @@ pub fn merge_branch(repo_path: &Path, target_branch: &str, source_branch: &str,
let _ = std::fs::remove_dir_all(&worktree_path);
if !success {
bail!("git merge failed: {}", String::from_utf8_lossy(&merge.stderr));
bail!(
"git merge failed: {}",
String::from_utf8_lossy(&merge.stderr)
);
}
// Get new HEAD of target branch
@@ -365,8 +429,56 @@ pub fn merge_branch(repo_path: &Path, target_branch: &str, source_branch: &str,
/// Resolve a repo disk path: {repos_dir}/{owner_slug}/{repo_name}.git
pub fn repo_disk_path(repos_dir: &Path, owner_did: &str, repo_name: &str) -> PathBuf {
// Sanitize the DID for use as a directory name
let owner_slug = owner_did
.replace(':', "_")
.replace('/', "_");
let owner_slug = owner_did.replace([':', '/'], "_");
repos_dir.join(owner_slug).join(format!("{repo_name}.git"))
}
#[cfg(test)]
mod tests {
use super::branch_diff_names;
use std::path::Path;
use std::process::Command;
#[test]
fn branch_diff_names_lists_changed_paths() {
let td = tempfile::TempDir::new().unwrap();
let work: &Path = td.path();
let g = |args: &[&str]| {
assert!(Command::new("git")
.args(args)
.current_dir(work)
.status()
.unwrap()
.success());
};
g(&["init", "-q"]);
g(&["config", "user.email", "t@t"]);
g(&["config", "user.name", "t"]);
std::fs::write(work.join("base.txt"), b"base\n").unwrap();
g(&["add", "."]);
g(&["commit", "-qm", "base"]);
let main = {
let o = Command::new("git")
.args(["symbolic-ref", "--short", "HEAD"])
.current_dir(work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
g(&["checkout", "-q", "-b", "feature"]);
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::write(work.join("secret/x.txt"), b"secret\n").unwrap();
g(&["add", "."]);
g(&["commit", "-qm", "feat"]);
let names = branch_diff_names(work, &main, "feature").unwrap();
assert!(
names.iter().any(|p| p == "secret/x.txt"),
"expected secret/x.txt in changed paths, got {names:?}"
);
assert!(
!names.iter().any(|p| p == "base.txt"),
"unchanged file must not appear: {names:?}"
);
}
}
+92 -21
View File
@@ -3,7 +3,9 @@
//! Repos are stored as `repos/v1/{owner_slug}/{repo_name}.tar.zst` — a
//! zstd-compressed tar archive of the bare repo directory.
use std::path::Path;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, OnceLock};
use anyhow::{Context, Result};
use aws_sdk_s3::Client as S3Client;
@@ -23,7 +25,10 @@ impl TigrisClient {
let config = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await;
let s3 = S3Client::new(&config);
info!(bucket = %bucket, "tigris storage client initialized");
Ok(Self { s3, bucket: bucket.to_string() })
Ok(Self {
s3,
bucket: bucket.to_string(),
})
}
/// S3 key for a given repo: `repos/v1/{owner_slug}/{repo_name}.tar.zst`
@@ -34,7 +39,9 @@ impl TigrisClient {
/// Check if a repo archive exists in Tigris.
pub async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool> {
let key = Self::repo_key(owner_slug, repo_name);
match self.s3.head_object()
match self
.s3
.head_object()
.bucket(&self.bucket)
.key(&key)
.send()
@@ -42,7 +49,7 @@ impl TigrisClient {
{
Ok(_) => Ok(true),
Err(e) => {
if e.as_service_error().map_or(false, |e| e.is_not_found()) {
if e.as_service_error().is_some_and(|e| e.is_not_found()) {
Ok(false)
} else {
Err(anyhow::anyhow!("tigris HEAD {key}: {e}"))
@@ -67,7 +74,8 @@ impl TigrisClient {
let body = aws_sdk_s3::primitives::ByteStream::from(archive_bytes);
self.s3.put_object()
self.s3
.put_object()
.bucket(&self.bucket)
.key(&key)
.body(body)
@@ -81,21 +89,30 @@ impl TigrisClient {
}
/// Download a repo archive from Tigris and extract to local disk.
pub async fn download(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
pub async fn download(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), "downloading repo from tigris");
let resp = self.s3.get_object()
let resp = self
.s3
.get_object()
.bucket(&self.bucket)
.key(&key)
.send()
.await
.context(format!("tigris GET {key}"))?;
let data = resp.body.collect().await
let data = resp
.body
.collect()
.await
.context("reading tigris response body")?
.into_bytes()
.to_vec();
.into_bytes();
// Extract tar.zst to local path
tokio::task::spawn_blocking({
@@ -114,7 +131,8 @@ impl TigrisClient {
#[allow(dead_code)]
pub async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
self.s3.delete_object()
self.s3
.delete_object()
.bucket(&self.bucket)
.key(&key)
.send()
@@ -139,17 +157,70 @@ fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
Ok(compressed)
}
/// Decompress a tar.zst byte vector into a local directory.
fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
// Ensure parent directory exists
if let Some(parent) = local_path.parent() {
std::fs::create_dir_all(parent).context("creating parent dir")?;
}
std::fs::create_dir_all(local_path).context("creating repo dir")?;
/// Per-repo-path lock serializing the publish (swap-into-place) step of
/// `decompress_repo`. Concurrent extractions unpack into isolated temp dirs in
/// parallel, but the final `remove_dir_all` + `rename` must not interleave for
/// the same `local_path`, or they race to a nondeterministic overwrite/failure.
fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
// KNOWN LIMITATION: this map is never evicted — one (PathBuf, Arc<Mutex>)
// entry accrues per distinct repo path for the process lifetime. Bounded by
// the number of repos a node hosts, so it's negligible for normal use, but
// high-volume/churning deployments may want LRU or weak-ref eviction here.
static LOCKS: OnceLock<Mutex<HashMap<PathBuf, Arc<Mutex<()>>>>> = OnceLock::new();
let locks = LOCKS.get_or_init(|| Mutex::new(HashMap::new()));
let mut map = locks.lock().expect("publish lock map poisoned");
map.entry(local_path.to_path_buf())
.or_insert_with(|| Arc::new(Mutex::new(())))
.clone()
}
let decoder = zstd::stream::Decoder::new(data)?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(local_path).context("unpacking tar.zst")?;
/// Decompress a tar.zst byte vector into a local directory.
///
/// Extraction is atomic with respect to `local_path`: the archive is unpacked
/// into a sibling temp directory first, and only swapped into place once it
/// fully succeeds. A corrupt or truncated archive therefore can never clobber a
/// good existing copy at `local_path` — on failure we discard the temp dir and
/// leave `local_path` exactly as it was.
fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
let parent = local_path.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = local_path
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
// Unique per-extraction temp dir: a fixed name would let two concurrent
// extractions of the same repo share one dir and clobber each other's
// in-progress unpack. A fresh UUID also means it can't collide with a
// leftover dir from a previously-interrupted run.
let tmp_dir = parent.join(format!(".{file_name}.tmp-extract.{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&tmp_dir).context("creating temp extract dir")?;
// Unpack into the temp dir; on any failure, clean up and bail without
// touching local_path.
let unpack = (|| -> Result<()> {
let decoder = zstd::stream::Decoder::new(data)?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(&tmp_dir).context("unpacking tar.zst")?;
Ok(())
})();
if let Err(e) = unpack {
let _ = std::fs::remove_dir_all(&tmp_dir);
return Err(e);
}
// Swap the freshly-extracted repo into place. rename within the same parent
// is effectively atomic, but most platforms refuse to rename onto a
// non-empty dir, so remove the old copy first. Serialize this per repo path:
// concurrent extractions unpack into isolated temp dirs, but their swaps
// must not interleave or they race to a nondeterministic overwrite/failure.
let lock = publish_lock(local_path);
let _publish = lock.lock().expect("publish lock poisoned");
if local_path.exists() {
std::fs::remove_dir_all(local_path).context("removing stale repo dir")?;
}
std::fs::rename(&tmp_dir, local_path).context("swapping extracted repo into place")?;
Ok(())
}
File diff suppressed because it is too large. Load diff
+184 -6
View File
@@ -3,11 +3,21 @@ use chrono::Utc;
use std::sync::Arc;
use uuid::Uuid;
use crate::auth::AuthenticatedDid;
use crate::db::{AgentTask, Db};
use crate::state::TaskEventBroadcast;
use super::types::{AgentTaskType, CreateTaskInput, FinishTaskInput};
/// The verified signer DID for this request, or an auth error (N2). The DID is
/// attached request-scoped by the `/graphql` `optional_signature` layer; its
/// absence means the request was unsigned, so every mutation rejects.
fn require_signer<'a>(ctx: &'a Context<'_>) -> Result<&'a str> {
ctx.data::<AuthenticatedDid>()
.map(|d| d.0.as_str())
.map_err(|_| async_graphql::Error::new("authentication required"))
}
pub struct MutationRoot;
#[Object]
@@ -18,6 +28,13 @@ impl MutationRoot {
delegator_did: String,
input: CreateTaskInput,
) -> Result<AgentTaskType> {
let caller = require_signer(ctx)?;
if !crate::api::did_matches(caller, &delegator_did) {
return Err(async_graphql::Error::new(
"delegator_did must be the authenticated signer",
));
}
let delegator_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let now = Utc::now().to_rfc3339();
let task = AgentTask {
@@ -47,9 +64,15 @@ impl MutationRoot {
id: String,
assignee_did: String,
) -> Result<AgentTaskType> {
let caller = require_signer(ctx)?;
if !crate::api::did_matches(caller, &assignee_did) {
return Err(async_graphql::Error::new(
"assignee_did must be the authenticated signer",
));
}
let assignee_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx
.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
let task = db
.claim_task(&id, &assignee_did)
.await
@@ -71,9 +94,27 @@ impl MutationRoot {
by_did: String,
input: FinishTaskInput,
) -> Result<AgentTaskType> {
let caller = require_signer(ctx)?;
if !crate::api::did_matches(caller, &by_did) {
return Err(async_graphql::Error::new(
"by_did must be the authenticated signer",
));
}
let by_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx
.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
// Authorize the actor: binding by_did to the signer is necessary but not
// sufficient — only the task's assignee may finish it.
let existing = db
.get_task(&id)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
"only the task assignee can complete it",
));
}
let task = db
.finish_task(&id, "completed", input.result.as_deref())
.await
@@ -95,9 +136,26 @@ impl MutationRoot {
by_did: String,
input: FinishTaskInput,
) -> Result<AgentTaskType> {
let caller = require_signer(ctx)?;
if !crate::api::did_matches(caller, &by_did) {
return Err(async_graphql::Error::new(
"by_did must be the authenticated signer",
));
}
let by_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx
.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
// Authorize the actor: only the task's assignee may fail it.
let existing = db
.get_task(&id)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
"only the task assignee can fail it",
));
}
let reason = input.reason.unwrap_or_default();
let task = db
.finish_task(&id, "failed", Some(&reason))
@@ -113,3 +171,123 @@ impl MutationRoot {
Ok(AgentTaskType::from(task))
}
}
#[cfg(test)]
mod tests {
use crate::auth::AuthenticatedDid;
use async_graphql::{Request, Response};
use sqlx::PgPool;
fn errors(resp: &Response) -> String {
resp.errors
.iter()
.map(|e| e.message.clone())
.collect::<Vec<_>>()
.join("; ")
}
/// N2: GraphQL mutations require a verified signer and bind the acting DID to
/// it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate.
#[sqlx::test]
async fn mutation_requires_and_binds_signer(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let schema = state.graphql_schema.as_ref();
let assignee = "did:key:zASSIGNEEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let q = format!(
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{assignee}") {{ id }} }}"#
);
// 1. Unsigned → rejected before any DB work.
let resp = schema.execute(Request::new(&q)).await;
assert!(
errors(&resp).contains("authentication required"),
"unsigned mutation must be rejected: {}",
errors(&resp)
);
// 2. Signed as someone other than the claimed assignee → rejected.
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(
"did:key:zOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB".into(),
)))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"DID mismatch must be rejected: {}",
errors(&resp)
);
// 3. Signed as the claimed assignee → passes the auth gate (any remaining
// error is the missing task, not an auth error).
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"matching signer must pass the auth gate: {errs}"
);
}
/// Adversarial-review GATE-1 (GraphQL): completing a task requires being its
/// assignee, not merely signing as the by_did you pass. A signer who is not
/// the assignee is rejected even though the by_did binding passes; the
/// assignee completes.
#[sqlx::test]
async fn complete_task_requires_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let assignee = "did:key:zGQLASSIGNEEAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zGQLSTRANGERBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-g".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed task");
state
.db
.claim_task("task-g", assignee)
.await
.expect("claim");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ completeTask(id: "task-g", byDid: "{actor}", input: {{}}) {{ id status }} }}"#
)
};
// Stranger signs as themselves and passes byDid=self (so the signer
// binding passes), but is not the assignee → rejected by authorization.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).contains("assignee"),
"a non-assignee signer must be rejected: {}",
errors(&resp)
);
// The assignee completes with no error.
let resp = schema
.execute(Request::new(q(assignee)).data(AuthenticatedDid(assignee.into())))
.await;
assert!(
errors(&resp).is_empty(),
"the assignee should complete the task: {}",
errors(&resp)
);
}
}
+19 -1
View File
@@ -12,11 +12,29 @@ impl QueryRoot {
async fn repos(&self, ctx: &Context<'_>) -> Result<Vec<RepoType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
let repos = db
.list_all_repos()
.list_all_repos_deduped()
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
// threaded onto the context by optional_signature; absent = anonymous.
let caller = ctx
.data::<crate::auth::AuthenticatedDid>()
.ok()
.map(|d| d.0.as_str());
let ids: Vec<String> = repos.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(repos
.into_iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, caller)
})
.map(|r| RepoType {
name: r.name,
owner_did: r.owner_did,
@@ -1,5 +1,5 @@
use async_graphql::{Context, Subscription};
use async_graphql::futures_util::Stream;
use async_graphql::{Context, Subscription};
use tokio_stream::wrappers::BroadcastStream;
use tokio_stream::StreamExt as _;
@@ -22,7 +22,7 @@ impl SubscriptionRoot {
BroadcastStream::new(rx).filter_map(move |r| {
let rf = repo.clone();
match r {
Ok(ev) if rf.as_deref().map_or(true, |r| ev.repo == r) => Some(RefUpdateType {
Ok(ev) if rf.as_deref().is_none_or(|r| ev.repo == r) => Some(RefUpdateType {
repo: ev.repo,
ref_name: ev.ref_name,
old_sha: ev.old_sha,
@@ -47,15 +47,13 @@ impl SubscriptionRoot {
BroadcastStream::new(rx).filter_map(move |r| {
let tf = task_id.clone();
match r {
Ok(ev) if tf.as_deref().map_or(true, |id| ev.task_id == id) => {
Some(TaskEventType {
task_id: ev.task_id,
old_status: ev.old_status,
new_status: ev.new_status,
by_did: ev.by_did,
at: ev.at,
})
}
Ok(ev) if tf.as_deref().is_none_or(|id| ev.task_id == id) => Some(TaskEventType {
task_id: ev.task_id,
old_status: ev.old_status,
new_status: ev.new_status,
by_did: ev.by_did,
at: ev.at,
}),
_ => None,
}
})
+633
View File
@@ -0,0 +1,633 @@
//! iCaptcha proof-of-intelligence gate.
//!
//! Spam-prone endpoints (repo creation, agent registration) can require the
//! caller to present an iCaptcha proof: a small Ed25519-signed token minted by
//! <https://icaptcha.gitlawb.com> after the caller solves an escalating
//! challenge. We verify the proof OFFLINE (no per-request call to iCaptcha)
//! using its published public key, and bind each proof to the authenticated
//! agent DID so a proof cannot be shared between identities.
//!
//! Behaviour is controlled by `ICAPTCHA_MODE`:
//! * `off` (default) — gate is inert, nothing is checked.
//! * `shadow` — verify and log would-be rejections, but always allow.
//! * `enforce` — reject requests without a valid, sufficiently-strong proof.
//!
//! Config (env):
//! ICAPTCHA_MODE off | shadow | enforce (default off)
//! ICAPTCHA_URL base URL (default https://icaptcha.gitlawb.com)
//! ICAPTCHA_PUBKEY base64url Ed25519 public key (optional; else fetched from /v1/pubkey)
//! ICAPTCHA_REQUIRED_LEVEL minimum proof level (default 3)
use std::sync::OnceLock;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use axum::http::HeaderMap;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use base64::Engine;
use ed25519_dalek::{Signature, VerifyingKey};
use serde::Deserialize;
use crate::error::AppError;
const PROOF_HEADER: &str = "x-icaptcha-proof";
/// How long a mirror-admission `jti` stays in the replay ledger. The direct
/// request path spends a proof against its own `exp` (it can't be presented once
/// expired). The propagation path deliberately accepts already-expired proofs,
/// so spending against `exp` would store an already-past expiry that the next
/// sweep deletes — letting the same token admit another mirror minutes later.
/// We instead retain the mirror replay record for a long, fixed window so the
/// per-node single-use property is durable.
const MIRROR_REPLAY_RETENTION_SECS: i64 = 365 * 24 * 60 * 60;
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum Mode {
Off,
Shadow,
Enforce,
}
impl Mode {
fn as_str(self) -> &'static str {
match self {
Mode::Off => "off",
Mode::Shadow => "shadow",
Mode::Enforce => "enforce",
}
}
}
/// Parse `ICAPTCHA_MODE`. Returns `None` for unrecognized values so the caller
/// can surface the typo instead of silently disabling the gate.
fn parse_mode(s: &str) -> Option<Mode> {
match s.trim().to_ascii_lowercase().as_str() {
"" | "off" => Some(Mode::Off),
"shadow" => Some(Mode::Shadow),
"enforce" => Some(Mode::Enforce),
_ => None,
}
}
/// Parse `ICAPTCHA_REQUIRED_LEVEL`. Defaults to 3; warns (rather than silently
/// lowering the threshold) when a non-empty value fails to parse.
fn parse_required_level() -> u32 {
const DEFAULT: u32 = 3;
match std::env::var("ICAPTCHA_REQUIRED_LEVEL") {
Ok(v) if !v.trim().is_empty() => v.trim().parse().unwrap_or_else(|_| {
tracing::warn!(
value = %v,
default = DEFAULT,
"invalid ICAPTCHA_REQUIRED_LEVEL; using default"
);
DEFAULT
}),
_ => DEFAULT,
}
}
struct Verifier {
mode: Mode,
url: String,
required_level: u32,
key: Option<VerifyingKey>,
}
static VERIFIER: OnceLock<Verifier> = OnceLock::new();
#[derive(Deserialize, Debug)]
struct ProofClaims {
sub: String,
level: u32,
exp: i64,
/// Unique proof id, consumed once so a proof cannot be replayed.
jti: String,
}
#[derive(Deserialize)]
struct Jwk {
x: String,
}
#[derive(Deserialize)]
struct Jwks {
keys: Vec<Jwk>,
}
fn now_secs() -> i64 {
SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
.unwrap_or(0)
}
fn decode_key(b64url: &str) -> Option<VerifyingKey> {
let bytes = URL_SAFE_NO_PAD.decode(b64url.trim()).ok()?;
let arr: [u8; 32] = bytes.try_into().ok()?;
VerifyingKey::from_bytes(&arr).ok()
}
async fn fetch_key(url: &str) -> Option<VerifyingKey> {
let endpoint = format!("{}/v1/pubkey", url.trim_end_matches('/'));
// Bounded request: a hung /v1/pubkey must never block node startup. On
// timeout/error we return None and the gate stays inert (fail safe).
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(5))
.build()
.ok()?;
let jwks: Jwks = client.get(&endpoint).send().await.ok()?.json().await.ok()?;
decode_key(&jwks.keys.first()?.x)
}
/// Initialize the gate from the environment. Call once at startup. Never panics;
/// if the gate is active but no key can be loaded, it stays inert and warns.
pub async fn init() {
let raw_mode = std::env::var("ICAPTCHA_MODE").unwrap_or_default();
let mode = parse_mode(&raw_mode).unwrap_or_else(|| {
tracing::warn!(value = %raw_mode, "invalid ICAPTCHA_MODE; disabling iCaptcha gate");
Mode::Off
});
let url = std::env::var("ICAPTCHA_URL")
.unwrap_or_else(|_| "https://icaptcha.gitlawb.com".to_string());
let required_level = parse_required_level();
let key = if mode == Mode::Off {
None
} else {
match std::env::var("ICAPTCHA_PUBKEY") {
Ok(b64) if !b64.is_empty() => decode_key(&b64),
_ => fetch_key(&url).await,
}
};
if mode != Mode::Off {
if key.is_some() {
tracing::info!(mode = mode.as_str(), required_level, "iCaptcha gate active");
} else {
tracing::warn!(
mode = mode.as_str(),
"iCaptcha gate enabled but no public key could be loaded; staying inert"
);
}
}
let _ = VERIFIER.set(Verifier {
mode,
url,
required_level,
key,
});
}
/// Outcome of the synchronous, IO-free decision step.
#[derive(Debug)]
enum Decision {
/// Allow the request (off, shadow, inert/no-key, or verified non-enforcing).
Allow,
/// Enforce mode and verification failed; reject with this reason.
Reject(String),
/// Enforce mode and the proof verified; the caller must consume its `jti`
/// (and reject replays) before allowing, then may persist it.
Consume(VerifiedProof),
}
/// A verified proof: the raw token plus the claims we act on. Carried by a
/// [`ProofGuard`] so that, once consumed, the gated handler can persist it with
/// the created repo — letting the proof travel with the repo when it propagates
/// to peers (see [`admit_mirror`]).
#[derive(Debug)]
pub struct VerifiedProof {
token: String,
sub: String,
level: u32,
jti: String,
exp: i64,
}
impl VerifiedProof {
/// Persist this proof against a freshly-created repo so a mirroring peer can
/// re-verify it. Best-effort callers may ignore failures; the gate's
/// security does not depend on it (propagation just falls back to quarantine).
pub async fn record_for_repo(&self, db: &crate::db::Db, repo_id: &str) -> Result<(), AppError> {
db.record_repo_proof(
repo_id,
&self.token,
&self.sub,
self.level as i32,
&self.jti,
self.exp,
)
.await?;
Ok(())
}
}
/// A verified proof awaiting consumption. Verification (which rejects invalid or
/// missing proofs) is separated from consumption (which spends the single-use
/// `jti`) so a request rejected by later validation never burns a valid proof.
/// The caller must `consume()` this guard immediately before the gated write.
/// For off/shadow/inert there is nothing to consume.
#[must_use = "a verified iCaptcha proof must be consumed before the gated action"]
pub struct ProofGuard(Option<VerifiedProof>);
impl ProofGuard {
/// Spend the proof's `jti` (single-use) and return the verified proof so the
/// caller can persist it. A replay is rejected. Returns `None` when there is
/// nothing to consume (off/shadow/inert).
pub async fn consume(self, db: &crate::db::Db) -> Result<Option<VerifiedProof>, AppError> {
match self.0 {
Some(p) => {
if !db.consume_proof_jti(&p.jti, p.exp).await? {
return Err(AppError::IcaptchaProofRequired(
"iCaptcha proof already used (replay); solve a fresh challenge".to_string(),
));
}
Ok(Some(p))
}
None => Ok(None),
}
}
}
/// Verify the proof in `headers` belongs to `did`. Rejects missing/invalid
/// proofs early (enforce mode); off/shadow never reject. Returns a [`ProofGuard`]
/// the caller must `consume()` right before the gated write. Off/shadow/inert
/// yield a no-op guard that consumes nothing.
pub fn verify_request(headers: &HeaderMap, did: &str) -> Result<ProofGuard, AppError> {
let v = match VERIFIER.get() {
Some(v) => v,
None => return Ok(ProofGuard(None)), // not initialized -> inert
};
match decide(v, headers, did, now_secs()) {
Decision::Allow => Ok(ProofGuard(None)),
Decision::Reject(reason) => Err(reject_error(v, &reason)),
Decision::Consume(proof) => Ok(ProofGuard(Some(proof))),
}
}
fn reject_error(v: &Verifier, reason: &str) -> AppError {
AppError::IcaptchaProofRequired(format!(
"iCaptcha proof required ({reason}). Solve a challenge at {} for level >= {} and resend with the {} header.",
v.url, v.required_level, PROOF_HEADER
))
}
/// Mode-aware decision. Pure and IO-free (no DB; clock injected via `now`) so it
/// is fully unit-testable. The caller performs jti consumption for `Consume`.
fn decide(v: &Verifier, headers: &HeaderMap, did: &str, now: i64) -> Decision {
if v.mode == Mode::Off {
return Decision::Allow;
}
// Fail safe: if no public key could be loaded (e.g. iCaptcha was unreachable
// at startup), stay inert rather than rejecting every request. The operator
// already saw a startup warning. An iCaptcha hiccup must never break repo
// creation or registration.
if v.key.is_none() {
return Decision::Allow;
}
let token = headers.get(PROOF_HEADER).and_then(|h| h.to_str().ok());
let result = match token {
Some(t) => verify_token(v, t, did, now, true).map(|claims| (t, claims)),
None => Err("missing proof header".to_string()),
};
match result {
Ok((token, claims)) => match v.mode {
Mode::Enforce => Decision::Consume(VerifiedProof {
token: token.to_string(),
sub: claims.sub,
level: claims.level,
jti: claims.jti,
exp: claims.exp,
}),
// Shadow/Off: never reject, and do not consume (observational only).
_ => Decision::Allow,
},
Err(reason) => match v.mode {
Mode::Shadow => {
tracing::warn!(did = %did, reason, "iCaptcha (shadow) would reject");
Decision::Allow
}
Mode::Enforce => Decision::Reject(reason),
Mode::Off => Decision::Allow,
},
}
}
/// Header-extracting wrapper over [`verify_token`] (enforces expiry). The
/// production path inlines this in [`decide`]; retained as the unit-test entry
/// point that also exercises header extraction.
#[cfg(test)]
fn verify(v: &Verifier, headers: &HeaderMap, did: &str, now: i64) -> Result<ProofClaims, String> {
let proof = headers
.get(PROOF_HEADER)
.and_then(|h| h.to_str().ok())
.ok_or("missing proof header")?;
verify_token(v, proof, did, now, true)
}
/// Core verification of a raw proof token, separated for testability and reuse.
/// Checks the Ed25519 signature, the required level, and the DID binding. `exp`
/// is only enforced when `check_exp` is true: the direct request path enforces
/// freshness, but the propagation path ([`admit_mirror`]) does not — a proof has
/// usually expired by the time a repo mirrors, and the origin enforced freshness
/// at creation. `now` is unix seconds.
fn verify_token(
v: &Verifier,
token: &str,
did: &str,
now: i64,
check_exp: bool,
) -> Result<ProofClaims, String> {
let key = v.key.as_ref().ok_or("verifier has no public key")?;
let (payload, sig_b64) = token.split_once('.').ok_or("malformed proof")?;
let sig_bytes = URL_SAFE_NO_PAD
.decode(sig_b64)
.map_err(|_| "bad signature encoding")?;
let sig = Signature::from_slice(&sig_bytes).map_err(|_| "bad signature length")?;
key.verify_strict(payload.as_bytes(), &sig)
.map_err(|_| "signature verification failed")?;
let claims_bytes = URL_SAFE_NO_PAD
.decode(payload)
.map_err(|_| "bad payload encoding")?;
let claims: ProofClaims = serde_json::from_slice(&claims_bytes).map_err(|_| "bad claims")?;
if check_exp && claims.exp < now {
return Err("proof expired".to_string());
}
if claims.level < v.required_level {
return Err(format!(
"level {} below required {}",
claims.level, v.required_level
));
}
if !crate::api::did_matches(did, &claims.sub) {
return Err("proof subject does not match authenticated DID".to_string());
}
Ok(claims)
}
/// Outcome of admitting a repo mirrored from a peer.
#[derive(Debug, PartialEq, Eq)]
pub enum MirrorAdmission {
/// Admit the mirror normally (off/shadow/inert, or a valid proof in enforce).
Admit,
/// Enforce mode and the mirror's proof is missing/invalid/replayed — mirror
/// it but quarantine until an operator releases it. Carries the reason.
Quarantine(String),
}
/// Decide whether to admit a repo being mirrored from a peer, given the proof
/// token the origin served (`None` if it had none or the fetch failed) and the
/// mirrored repo's owner DID.
///
/// Off/shadow/inert always admit (shadow logs the would-be quarantine and writes
/// nothing). In enforce mode the proof is re-verified offline against the local
/// public key — signature, level, and owner-DID binding, but NOT expiry — and
/// its `jti` is consumed in the local ledger, so one solved challenge admits at
/// most one mirror per node (a malicious origin cannot reuse one proof to bless
/// many spam repos here).
///
/// LIMITATION (proof is not repo-bound): an iCaptcha proof commits only to the
/// solver's DID (`sub`), never to a repo — the proof is minted before the repo
/// exists. So the guarantee enforced here is "a level≥N challenge was solved by
/// this repo's owner DID", not "the owner intended THIS repo". A malicious origin
/// that harvests a victim's public proof (served by `get_icaptcha_proof`) can use
/// it to get ONE spam repo nominally owned by that victim admitted per node — the
/// per-node `jti` single-use caps the amplification at one repo per distinct
/// harvested proof, but does not eliminate it. Fully binding a proof to a repo
/// would require the iCaptcha service to embed a repo/target identifier in the
/// signed challenge (a protocol change, tracked as future work).
pub async fn admit_mirror(
db: &crate::db::Db,
token: Option<&str>,
owner_did: &str,
) -> MirrorAdmission {
let v = match VERIFIER.get() {
Some(v) => v,
None => return MirrorAdmission::Admit, // not initialized -> inert
};
// Off, or inert because no key could be loaded: admit unconditionally.
if v.mode == Mode::Off || v.key.is_none() {
return MirrorAdmission::Admit;
}
let shadow = v.mode == Mode::Shadow;
// Quarantine in enforce; in shadow just log and admit (observational, no IO).
let quarantine = |reason: String| -> MirrorAdmission {
if shadow {
tracing::warn!(owner = %owner_did, reason, "iCaptcha (shadow) would quarantine mirror");
MirrorAdmission::Admit
} else {
MirrorAdmission::Quarantine(reason)
}
};
let token = match token {
Some(t) => t,
None => return quarantine("origin served no iCaptcha proof".to_string()),
};
match verify_token(v, token, owner_did, now_secs(), false) {
Ok(claims) => {
if shadow {
// Observational: a valid proof would admit. Do not consume, so a
// later switch to enforce still sees the jti as fresh.
return MirrorAdmission::Admit;
}
// Retain the replay record on a fixed forward window, NOT the proof's
// own (already-past) exp — otherwise the next sweep frees the jti and
// the same token could admit another mirror here.
let retain_until = now_secs().saturating_add(MIRROR_REPLAY_RETENTION_SECS);
match db.consume_proof_jti(&claims.jti, retain_until).await {
Ok(true) => MirrorAdmission::Admit,
Ok(false) => MirrorAdmission::Quarantine(
"iCaptcha proof already used to admit another mirror".to_string(),
),
Err(e) => {
tracing::warn!(owner = %owner_did, err = %e, "iCaptcha mirror ledger error; quarantining");
MirrorAdmission::Quarantine("iCaptcha proof ledger unavailable".to_string())
}
}
}
Err(reason) => quarantine(reason),
}
}
#[cfg(test)]
mod tests {
use super::*;
// Real values captured from https://icaptcha.gitlawb.com (a live proof).
const PUBKEY_X: &str = "xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8";
const PROOF: &str = "eyJzdWIiOiJkaWQ6a2V5Onp0ZXN0IiwibGV2ZWwiOjMsImlzcyI6ImljYXB0Y2hhIiwiaWF0IjoxNzgyNTcyODUxLCJleHAiOjE3ODI1NzMxNTEsImp0aSI6IjRiNTIyOGE1YmVkNzEyMmRlZTlmNDdmZiJ9.5UXVPZ8Eo91VnlcvgDXtW-Fx7J2jr7h535SAstQEpigxBr7FF7V6R0XB4PBDgdoBPnhdH_kVEfRPfdHPSdB0CA";
const SUB: &str = "did:key:ztest";
const IAT: i64 = 1782572851; // within the proof's validity window
fn verifier(level: u32) -> Verifier {
Verifier {
mode: Mode::Enforce,
url: "https://icaptcha.gitlawb.com".to_string(),
required_level: level,
key: decode_key(PUBKEY_X),
}
}
fn headers_with(proof: &str) -> HeaderMap {
let mut h = HeaderMap::new();
h.insert(PROOF_HEADER, proof.parse().unwrap());
h
}
#[test]
fn accepts_a_real_proof() {
let v = verifier(3);
assert!(verify(&v, &headers_with(PROOF), SUB, IAT).is_ok());
}
#[test]
fn rejects_expired_proof() {
let v = verifier(3);
let err = verify(&v, &headers_with(PROOF), SUB, 9_999_999_999).unwrap_err();
assert!(err.contains("expired"), "{err}");
}
#[test]
fn rejects_wrong_did() {
let v = verifier(3);
let err = verify(&v, &headers_with(PROOF), "did:key:zsomeoneelse", IAT).unwrap_err();
assert!(err.contains("subject"), "{err}");
}
#[test]
fn rejects_insufficient_level() {
let v = verifier(5); // proof is level 3
let err = verify(&v, &headers_with(PROOF), SUB, IAT).unwrap_err();
assert!(err.contains("below required"), "{err}");
}
#[test]
fn rejects_tampered_signature() {
let v = verifier(3);
// Flip one base64url char in the signature so it is guaranteed different.
let (payload, sig) = PROOF.split_once('.').unwrap();
let mut chars: Vec<char> = sig.chars().collect();
chars[0] = if chars[0] == 'A' { 'B' } else { 'A' };
let tampered = format!("{}.{}", payload, chars.into_iter().collect::<String>());
assert!(verify(&v, &headers_with(&tampered), SUB, IAT).is_err());
}
#[test]
fn rejects_missing_header() {
let v = verifier(3);
let err = verify(&v, &HeaderMap::new(), SUB, IAT).unwrap_err();
assert!(err.contains("missing"), "{err}");
}
#[test]
fn parse_mode_accepts_documented_values_and_rejects_junk() {
assert_eq!(parse_mode(""), Some(Mode::Off));
assert_eq!(parse_mode("off"), Some(Mode::Off));
assert_eq!(parse_mode(" Shadow "), Some(Mode::Shadow));
assert_eq!(parse_mode("ENFORCE"), Some(Mode::Enforce));
// Typos must NOT silently disable the gate.
assert_eq!(parse_mode("enforced"), None);
assert_eq!(parse_mode("on"), None);
}
#[test]
fn off_mode_allows_everything() {
let mut v = verifier(3);
v.mode = Mode::Off;
assert!(matches!(
decide(&v, &HeaderMap::new(), SUB, IAT),
Decision::Allow
));
}
#[test]
fn enforce_without_key_stays_inert() {
// iCaptcha unreachable at startup -> no key -> must not reject.
let v = Verifier {
mode: Mode::Enforce,
url: "https://icaptcha.gitlawb.com".to_string(),
required_level: 3,
key: None,
};
assert!(matches!(
decide(&v, &HeaderMap::new(), SUB, IAT),
Decision::Allow
));
}
#[test]
fn enforce_with_key_rejects_missing_proof() {
let v = verifier(3);
assert!(matches!(
decide(&v, &HeaderMap::new(), SUB, IAT),
Decision::Reject(_)
));
}
#[test]
fn shadow_allows_despite_bad_proof() {
let mut v = verifier(3);
v.mode = Mode::Shadow;
assert!(matches!(
decide(&v, &HeaderMap::new(), SUB, IAT),
Decision::Allow
));
}
#[test]
fn enforce_valid_proof_requires_consuming_its_jti() {
// A verified proof under enforce must yield Consume carrying the jti, so
// the caller can spend it once and reject replays.
let v = verifier(3);
match decide(&v, &headers_with(PROOF), SUB, IAT) {
Decision::Consume(p) => {
assert_eq!(p.jti, "4b5228a5bed7122dee9f47ff");
assert_eq!(p.exp, 1782573151);
assert_eq!(p.token, PROOF, "the raw token is retained for persistence");
assert_eq!(p.sub, SUB);
}
other => panic!("expected Consume, got {other:?}"),
}
}
#[test]
fn propagation_accepts_an_expired_proof() {
// The mirror-admission path verifies sig/level/DID but NOT expiry, since a
// proof has usually expired by the time a repo propagates. The same proof
// that the direct path rejects as expired must verify here.
let v = verifier(3);
let now = 9_999_999_999; // far past the proof's exp
assert!(
verify_token(&v, PROOF, SUB, now, true).is_err(),
"direct path (check_exp=true) rejects the expired proof"
);
assert!(
verify_token(&v, PROOF, SUB, now, false).is_ok(),
"propagation path (check_exp=false) accepts it"
);
}
#[test]
fn propagation_still_enforces_signature_level_and_did() {
let v = verifier(3);
// wrong owner DID
assert!(verify_token(&v, PROOF, "did:key:zother", IAT, false).is_err());
// insufficient level
let v5 = verifier(5);
assert!(verify_token(&v5, PROOF, SUB, IAT, false).is_err());
// tampered signature
let (payload, sig) = PROOF.split_once('.').unwrap();
let mut chars: Vec<char> = sig.chars().collect();
chars[0] = if chars[0] == 'A' { 'B' } else { 'A' };
let tampered = format!("{}.{}", payload, chars.into_iter().collect::<String>());
assert!(verify_token(&v, &tampered, SUB, IAT, false).is_err());
}
}
+24 -40
View File
@@ -63,35 +63,47 @@ pub async fn pin_git_object(ipfs_api: &str, sha256_hex: &str, data: &[u8]) -> Re
let v: serde_json::Value = serde_json::from_str(line).ok()?;
v["Hash"].as_str().map(|s| s.to_string())
})
.last()
.next_back()
.unwrap_or(expected_cid.clone());
tracing::debug!(sha256 = %sha256_hex, %cid, "pinned git object to IPFS");
Ok(cid)
}
/// List all git objects in the given bare repo and pin any that are not yet
/// recorded in `pinned_cids`.
/// Fetch raw bytes for a CID from the local Kubo node (`/api/v0/cat`).
pub async fn cat(ipfs_api: &str, cid: &str) -> Result<Vec<u8>> {
if ipfs_api.is_empty() {
return Err(anyhow::anyhow!("IPFS not configured"));
}
let url = format!("{}/api/v0/cat?arg={}", ipfs_api.trim_end_matches('/'), cid);
let resp = reqwest::Client::new().post(&url).send().await?;
if !resp.status().is_success() {
return Err(anyhow::anyhow!("ipfs cat {cid}: {}", resp.status()));
}
Ok(resp.bytes().await?.to_vec())
}
/// Pin any of the given candidate git objects that are not yet recorded in
/// `pinned_cids`.
///
/// `object_list` is the already-withheld-filtered OID set to pin: the caller
/// applies `visibility_pack::replicable_objects` on the delta path or the
/// `..._fail_closed` filter on the full-scan path before calling, so this
/// function never sees a withheld blob. `repo_path` is still needed to read each
/// object's bytes. The twin in `pinata.rs` mirrors this shape — change both in
/// lockstep.
///
/// Returns a list of `(sha256_hex, cid)` pairs for objects pinned this call.
pub async fn pin_new_objects(
ipfs_api: &str,
repo_path: &std::path::Path,
object_list: Vec<String>,
db: &crate::db::Db,
) -> Vec<(String, String)> {
if ipfs_api.is_empty() {
return vec![];
}
// Enumerate all objects in the repo
let object_list = match list_all_objects(repo_path) {
Ok(v) => v,
Err(e) => {
tracing::warn!(repo = %repo_path.display(), err = %e, "failed to list git objects for IPFS pinning");
return vec![];
}
};
let mut pinned = Vec::new();
for sha in object_list {
@@ -132,31 +144,3 @@ pub async fn pin_new_objects(
pinned
}
/// Run `git cat-file --batch-all-objects --batch-check='%(objectname)'`
/// to get all object SHA-256 hashes in the repository.
fn list_all_objects(repo_path: &std::path::Path) -> Result<Vec<String>> {
let output = std::process::Command::new("git")
.args([
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname)",
])
.current_dir(repo_path)
.output()
.map_err(|e| anyhow::anyhow!("failed to run git cat-file: {e}"))?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(anyhow::anyhow!("git cat-file failed: {stderr}"));
}
let stdout = String::from_utf8_lossy(&output.stdout);
let hashes = stdout
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect();
Ok(hashes)
}
+402 -64
View File
@@ -1,27 +1,38 @@
mod admin;
mod api;
mod arweave;
mod auth;
mod bootstrap;
mod cert;
mod config;
mod db;
mod encrypted_pin;
mod error;
mod git;
mod graphql;
mod icaptcha;
mod ipfs_pin;
mod metrics;
mod operator;
mod p2p;
mod pinata;
mod rate_limit;
mod server;
mod state;
mod sync;
#[cfg(test)]
mod test_support;
mod visibility;
mod webhooks;
use std::sync::Arc;
use anyhow::{Context, Result};
use clap::Parser;
use std::sync::Arc;
use tokio::net::TcpListener;
use tracing::info;
use tokio::sync::watch;
use tracing::{info, warn};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use config::Config;
@@ -38,33 +49,98 @@ async fn main() -> Result<()> {
)
.init();
let config = Config::parse();
let mut config = Config::parse();
// Admin subcommands run instead of the daemon and exit.
if let Some(cmd) = config.command.clone() {
match cmd {
config::Command::PurgeSpam(args) => {
return admin::purge_spam(&config, &args).await;
}
}
}
// Merge the embedded seed list of public network nodes into the runtime
// bootstrap peers. Operators can opt out via GITLAWB_BOOTSTRAP_DISABLE_SEEDS.
bootstrap::merge_seeds(&mut config);
if !config.public_read {
warn!(
"GITLAWB_PUBLIC_READ=false is reserved; per-repository private-read enforcement is not wired in alpha"
);
}
// Load or generate the node's identity keypair
let keypair = load_or_create_keypair(&config)?;
let node_did = keypair.did();
// One-time metrics init. Must run before any handler that calls into
// `metrics::record_*` so the registry exists when the first event fires.
// Safe to call even when GITLAWB_METRICS_ADDR is unset — those helpers
// are simply no-ops until something reads from the registry.
metrics::init(env!("CARGO_PKG_VERSION"), &node_did.to_string());
info!("╔══════════════════════════════════════════╗");
info!("║ gitlawb node v{} ║", env!("CARGO_PKG_VERSION"));
info!(
"║ gitlawb node v{} ║",
env!("CARGO_PKG_VERSION")
);
info!("╚══════════════════════════════════════════╝");
info!(did = %node_did, "node identity");
info!(addr = %config.bind_addr(), "listening");
// Process-wide shutdown signal. One sender lives in AppState (cloned
// into every handler); main() keeps a clone and flips it on SIGINT
// or SIGTERM. Tasks that hold a watch::Receiver get notified at
// their next await point.
let (shutdown_tx, _shutdown_rx_for_main) = watch::channel(false);
// Connect to PostgreSQL database
let db = Arc::new(Db::connect(&config.database_url)
.await
.context("failed to connect to database")?);
let db = Arc::new(
Db::connect(&config.database_url)
.await
.context("failed to connect to database")?,
);
// Prune peer rows that point back at this node (stale self-loop entries)
if let Some(public_url) = config.public_url.as_deref() {
match db.prune_self_peers(public_url).await {
Ok(0) => {}
Ok(n) => info!(removed = n, public_url, "pruned self-loop peer rows"),
Err(e) => warn!(err = %e, "prune_self_peers failed (non-fatal)"),
}
}
// Prune peer rows with non-public hosts (loopback/private/internal) that
// were injected via the unauthenticated announce route — they poison the
// sync-notify fan-out (SSRF + crowding out real peers).
match db.prune_non_public_peers().await {
Ok(0) => {}
Ok(n) => info!(removed = n, "pruned non-public (poisoned) peer rows"),
Err(e) => warn!(err = %e, "prune_non_public_peers failed (non-fatal)"),
}
// Ensure repos directory exists
std::fs::create_dir_all(&config.repos_dir)
.context("failed to create repos directory")?;
std::fs::create_dir_all(&config.repos_dir).context("failed to create repos directory")?;
// Start libp2p swarm (if p2p_port > 0)
let p2p_handle = if config.p2p_port > 0 {
let bootstrap_addrs = config.p2p_bootstrap.iter()
let bootstrap_addrs = config
.p2p_bootstrap
.iter()
.filter_map(|s| s.parse().ok())
.collect();
match p2p::start(&node_did.to_string(), config.p2p_port, bootstrap_addrs, Arc::clone(&db), config.auto_sync).await {
let shutdown_rx = shutdown_tx.subscribe();
match p2p::start(
&node_did.to_string(),
config.p2p_port,
bootstrap_addrs,
Arc::clone(&db),
config.auto_sync,
shutdown_rx,
)
.await
{
Ok(handle) => {
info!(port = config.p2p_port, peer_id = %handle.local_peer_id, "libp2p swarm started");
Some(Arc::new(handle))
@@ -79,9 +155,16 @@ async fn main() -> Result<()> {
None
};
let http_client = Arc::new(reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()?);
// Do not follow redirects: this client fans out to peer-supplied URLs
// (sync trigger, profile/repo fetches). A peer answering `302 Location:
// http://127.0.0.1/` would otherwise bypass the announce-time public-URL
// check and turn the redirect into an SSRF.
let http_client = Arc::new(
reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()?,
);
let (ref_update_tx, _) = tokio::sync::broadcast::channel::<state::RefUpdateBroadcast>(256);
let (task_event_tx, _) = tokio::sync::broadcast::channel::<state::TaskEventBroadcast>(256);
@@ -114,11 +197,13 @@ async fn main() -> Result<()> {
None
};
let repo_store = git::repo_store::RepoStore::new(
config.repos_dir.clone(),
tigris,
db.pool().clone(),
);
let repo_store =
git::repo_store::RepoStore::new(config.repos_dir.clone(), tigris, db.pool().clone());
let rate_limiter = rate_limit::RateLimiter::new(10, std::time::Duration::from_secs(3600));
// Initialize the iCaptcha proof gate (inert unless ICAPTCHA_MODE is set).
icaptcha::init().await;
let state = AppState {
config: Arc::new(config.clone()),
@@ -132,15 +217,120 @@ async fn main() -> Result<()> {
graphql_schema,
machine_id,
repo_store,
rate_limiter,
shutdown_tx: shutdown_tx.clone(),
};
// Periodic peer-count poll for the metrics gauge. If p2p is disabled
// we still set the gauge to 0 so dashboards don't show "no data".
{
let p2p_for_metrics = state.p2p.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(15));
loop {
tokio::select! {
_ = interval.tick() => {
let count = match &p2p_for_metrics {
Some(h) => h.status().await.map(|s| s.connected_peers).unwrap_or(0),
None => 0,
};
metrics::set_peers_connected(count as i64);
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
return;
}
}
}
}
});
}
// Spawn a task that flips the shutdown signal on SIGINT or SIGTERM.
// On Unix, both signals are handled. On Windows, only Ctrl-C is
// supported by tokio::signal::ctrl_c.
{
let tx = shutdown_tx.clone();
tokio::spawn(async move {
#[cfg(unix)]
{
use tokio::signal::unix::{signal as unix_signal, SignalKind};
let mut sigterm =
unix_signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut sigint =
unix_signal(SignalKind::interrupt()).expect("install SIGINT handler");
tokio::select! {
_ = sigterm.recv() => info!("SIGTERM received, shutting down"),
_ = sigint.recv() => info!("SIGINT received, shutting down"),
}
}
#[cfg(not(unix))]
{
use tokio::signal;
let _ = signal::ctrl_c().await;
info!("Ctrl-C received, shutting down");
}
tx.send(true).ok();
});
}
// Periodic cleanup of expired rate limit entries + consumed-proof ledger
{
let rl = state.rate_limiter.clone();
let db = state.db.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
loop {
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(300)) => {
rl.cleanup().await;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
.unwrap_or(0);
if let Err(e) = db.sweep_expired_proofs(now).await {
tracing::warn!(err = %e, "failed to sweep expired iCaptcha proofs");
}
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
break;
}
}
}
}
});
}
let router = server::build_router(state.clone());
let listener = TcpListener::bind(config.bind_addr()).await
let listener = TcpListener::bind(config.bind_addr())
.await
.with_context(|| format!("failed to bind to {}", config.bind_addr()))?;
info!("✓ node started — did:{}", node_did);
info!(" repos dir: {}", config.repos_dir.display());
info!(" database: PostgreSQL ({})", &config.database_url.split('@').last().unwrap_or("?"));
info!(
" database: PostgreSQL ({})",
&config.database_url.split('@').next_back().unwrap_or("?")
);
// Optional Prometheus metrics listener on a separate port.
let metrics_handle = if !config.metrics_addr.is_empty() {
match spawn_metrics_server(&config.metrics_addr, state.clone()).await {
Ok(handle) => {
info!(addr = %config.metrics_addr, "metrics endpoint listening");
Some(handle)
}
Err(e) => {
warn!(err = %e, addr = %config.metrics_addr, "failed to start metrics endpoint — continuing without");
None
}
}
} else {
info!("metrics endpoint disabled (GITLAWB_METRICS_ADDR not set)");
None
};
// Publish our DID record to the Kademlia DHT shortly after startup
if let Some(p2p) = &state.p2p {
@@ -152,26 +342,36 @@ async fn main() -> Result<()> {
timestamp: chrono::Utc::now().to_rfc3339(),
};
let p2p_clone = Arc::clone(p2p);
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
// Small delay so Kademlia can find peers first
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(5)) => {}
_ = shutdown_rx.changed() => return,
}
p2p_clone.put_did(did_record).await;
info!("DID record published to Kademlia DHT");
});
}
// Spawn background gossip: announce to bootstrap peers, then ping known peers periodically
if !config.bootstrap_peers.is_empty() || true {
{
let gossip_state = state.clone();
let bootstrap_peers = config.bootstrap_peers.clone();
let shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
gossip_task(gossip_state, bootstrap_peers).await;
gossip_task(gossip_state, bootstrap_peers, shutdown_rx).await;
});
}
// Start multi-node sync worker if auto_sync is enabled
if config.auto_sync {
sync::start(Arc::clone(&state.db), Arc::clone(&state.config));
sync::start(
Arc::clone(&state.db),
Arc::clone(&state.config),
Arc::clone(&state.node_keypair),
state.subscribe_shutdown(),
);
info!("auto-sync worker started");
}
@@ -180,20 +380,18 @@ async fn main() -> Result<()> {
&& !state.config.operator_private_key.is_empty()
{
match build_operator_client(&state.config, &state.node_did.to_string()) {
Ok(client) => {
match operator::startup_check(&client).await {
Ok(_) => {
let arc_client = Arc::new(client);
arc_client.spawn_heartbeat_loop();
}
Err(e) => {
if state.config.operator_strict_mode {
return Err(e.context("strict-mode operator check failed"));
}
tracing::warn!(err = %e, "operator startup check failed — continuing without heartbeat loop");
}
Ok(client) => match operator::startup_check(&client).await {
Ok(_) => {
let arc_client = Arc::new(client);
arc_client.spawn_heartbeat_loop(state.subscribe_shutdown());
}
}
Err(e) => {
if state.config.operator_strict_mode {
return Err(e.context("strict-mode operator check failed"));
}
tracing::warn!(err = %e, "operator startup check failed — continuing without heartbeat loop");
}
},
Err(e) => {
if state.config.operator_strict_mode {
return Err(e.context("strict-mode: failed to build operator client"));
@@ -205,11 +403,94 @@ async fn main() -> Result<()> {
info!("on-chain PoS disabled (GITLAWB_CONTRACT_NODE_STAKING or GITLAWB_OPERATOR_PRIVATE_KEY unset)");
}
axum::serve(listener, router).await?;
// axum's `with_graceful_shutdown` waits for in-flight requests to
// complete (up to the configured grace) once the future resolves.
let shutdown_signal_for_axum = state.subscribe_shutdown();
let grace = std::time::Duration::from_secs(config.shutdown_grace_secs);
info!(grace_secs = config.shutdown_grace_secs, "axum server ready");
let serve_result = axum::serve(listener, router)
.with_graceful_shutdown(async move {
let mut rx = shutdown_signal_for_axum;
// Wait until the watcher flips to true, then return so axum
// can begin draining.
while !*rx.borrow_and_update() {
if rx.changed().await.is_err() {
// Sender dropped — treat as shutdown.
break;
}
}
})
.await;
// Server has stopped accepting new connections and drained in-flight
// requests. Tear the rest of the system down.
info!("HTTP server stopped, beginning process shutdown");
if let Some(h) = metrics_handle {
h.abort();
}
let _ = grace; // recorded for operators in the log above; not enforced
serve_result?;
info!("clean exit");
Ok(())
}
fn build_operator_client(config: &config::Config, node_did: &str) -> Result<operator::OperatorClient> {
/// Spawn a small axum router that exposes only `GET /metrics` on its own
/// listener. Returns the JoinHandle so `main()` can abort it on shutdown.
/// This is deliberately separate from the main router so the metrics port
/// can be firewalled differently from the API port — bind to localhost
/// or a private interface only.
async fn spawn_metrics_server(addr: &str, state: AppState) -> Result<tokio::task::JoinHandle<()>> {
use axum::{response::IntoResponse, routing::get, Router};
async fn metrics_handler() -> impl IntoResponse {
match metrics::encode() {
Ok(body) => (
axum::http::StatusCode::OK,
[(
axum::http::header::CONTENT_TYPE,
"text/plain; version=0.0.4; charset=utf-8",
)],
body,
),
Err(e) => (
axum::http::StatusCode::INTERNAL_SERVER_ERROR,
[(
axum::http::header::CONTENT_TYPE,
"text/plain; charset=utf-8",
)],
format!("metrics encode error: {e}"),
),
}
}
let mut shutdown_rx = state.subscribe_shutdown();
let listener = TcpListener::bind(addr)
.await
.with_context(|| format!("failed to bind metrics listener to {addr}"))?;
let app = Router::new().route("/metrics", get(metrics_handler));
let handle = tokio::spawn(async move {
if let Err(e) = axum::serve(listener, app)
.with_graceful_shutdown(async move {
while !*shutdown_rx.borrow_and_update() {
if shutdown_rx.changed().await.is_err() {
break;
}
}
})
.await
{
warn!(err = %e, "metrics server exited with error");
}
});
Ok(handle)
}
fn build_operator_client(
config: &config::Config,
node_did: &str,
) -> Result<operator::OperatorClient> {
use alloy::primitives::Address;
use std::str::FromStr;
@@ -228,9 +509,21 @@ fn build_operator_client(config: &config::Config, node_did: &str) -> Result<oper
}
/// Announce to bootstrap peers on startup, then periodically ping all known peers.
async fn gossip_task(state: AppState, bootstrap_peers: Vec<String>) {
// Small delay to let the HTTP server come up before we try to announce
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
async fn gossip_task(
state: AppState,
bootstrap_peers: Vec<String>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) {
// If shutdown arrives during the initial delay, exit before announcing.
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(2)) => {}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("gossip: shutdown during startup delay, exiting");
return;
}
}
}
let client = reqwest::Client::new();
let my_did = state.node_did.to_string();
@@ -238,13 +531,43 @@ async fn gossip_task(state: AppState, bootstrap_peers: Vec<String>) {
// Announce ourselves to each bootstrap peer
for peer_url in &bootstrap_peers {
let announce_url = format!("{}/api/v1/peers/announce", peer_url.trim_end_matches('/'));
// Cooperative shutdown between peers — a slow peer shouldn't
// block the node exiting.
if *shutdown_rx.borrow() {
info!("gossip: shutdown signalled during peer announce, exiting");
return;
}
let path = "/api/v1/peers/announce";
let announce_url = format!("{}{}", peer_url.trim_end_matches('/'), path);
let body = serde_json::json!({
"did": my_did,
"http_url": my_url,
"did": my_did.clone(),
"http_url": my_url.clone(),
});
match client.post(&announce_url).json(&body).send().await {
Ok(resp) => {
let body_bytes = match serde_json::to_vec(&body) {
Ok(bytes) => bytes,
Err(e) => {
tracing::warn!(err = %e, "failed to serialize peer announce body");
continue;
}
};
let signed = sign_request(state.node_keypair.as_ref(), "POST", path, &body_bytes);
// Per-request timeout inside the loop; do not let one hung peer
// block others. The request itself is a normal tokio future so
// it's cancel-safe on shutdown.
match tokio::time::timeout(
std::time::Duration::from_secs(5),
client
.post(&announce_url)
.header("Content-Type", "application/json")
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature)
.body(body_bytes)
.send(),
)
.await
{
Ok(Ok(resp)) => {
if resp.status().is_success() {
if let Ok(json) = resp.json::<serde_json::Value>().await {
// Add them back to our peer list
@@ -260,24 +583,39 @@ async fn gossip_task(state: AppState, bootstrap_peers: Vec<String>) {
}
}
}
Err(e) => tracing::warn!(url = %announce_url, err = %e, "failed to announce to bootstrap peer"),
Ok(Err(e)) => {
tracing::warn!(url = %announce_url, err = %e, "failed to announce to bootstrap peer")
}
Err(_) => tracing::warn!(url = %announce_url, "bootstrap peer announce timed out (5s)"),
}
}
// Periodic ping every 5 minutes
// Periodic ping every 5 minutes — exit on shutdown.
let mut interval = tokio::time::interval(std::time::Duration::from_secs(300));
loop {
interval.tick().await;
let peers = match state.db.list_peers().await {
Ok(p) => p,
Err(_) => continue,
};
for peer in peers {
let url = format!("{}/health", peer.http_url.trim_end_matches('/'));
let ok = client.get(&url).send().await
.map(|r| r.status().is_success())
.unwrap_or(false);
let _ = state.db.mark_peer_ping(&peer.did, ok).await;
tokio::select! {
_ = interval.tick() => {
let peers = match state.db.list_peers().await {
Ok(p) => p,
Err(_) => continue,
};
for peer in peers {
let url = format!("{}/health", peer.http_url.trim_end_matches('/'));
let ok = client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false);
let _ = state.db.mark_peer_ping(&peer.did, ok).await;
}
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("gossip task: shutdown signal received, exiting");
return;
}
}
}
}
}
@@ -288,13 +626,13 @@ fn load_or_create_keypair(config: &Config) -> Result<Keypair> {
if key_path.exists() {
let pem = std::fs::read_to_string(&key_path)
.with_context(|| format!("failed to read key from {}", key_path.display()))?;
let kp = Keypair::from_pem(&pem)
.map_err(|e| anyhow::anyhow!("invalid PEM key: {e}"))?;
let kp = Keypair::from_pem(&pem).map_err(|e| anyhow::anyhow!("invalid PEM key: {e}"))?;
info!(path = %key_path.display(), "loaded existing identity");
Ok(kp)
} else {
let kp = Keypair::generate();
let pem = kp.to_pem()
let pem = kp
.to_pem()
.map_err(|e| anyhow::anyhow!("failed to serialize key: {e}"))?;
if let Some(parent) = key_path.parent() {
+352
View File
@@ -0,0 +1,352 @@
//! Prometheus metrics for the gitlawb node.
//!
//! Exposes a small, deliberately conservative set of counters and histograms
//! that cover the questions an operator actually asks of a live node:
//!
//! * is push traffic flowing? — `gitlawb_pushes_total`
//! * is fetch traffic flowing? — `gitlawb_fetches_total`
//! * are signature checks passing or failing? —
//! `gitlawb_auth_successes_total` / `gitlawb_auth_failures_total`
//! * is the sync worker making progress? —
//! `gitlawb_sync_queue_processed_total{status}`
//! * are webhooks reaching their endpoints? —
//! `gitlawb_webhook_deliveries_total{result}`
//! * how big are the packs we're sending and receiving? —
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
//! * currently-connected peer count — `gitlawb_peers_connected`
//!
//! All metrics live in a single process-wide registry initialized by
//! [`init`]. Increment helpers (`record_push`, `record_auth_failure`, ...)
//! are no-ops until [`init`] has been called, so unit tests and benchmarks
//! that don't go through `main()` don't need to do anything special.
//!
//! The endpoint is opt-in via `GITLAWB_METRICS_ADDR`; if unset, no listener
//! is bound and no exposition happens. Increments are atomic and effectively
//! free.
//!
//! Follow-ups (not in this module):
//! * per-route latency histograms (TraceLayer already gives us spans)
//! * per-peer p2p counters
//! * ipfs / pinata counters
use std::sync::OnceLock;
use prometheus::{
Encoder, Histogram, HistogramOpts, IntCounterVec, IntGauge, IntGaugeVec, Opts, Registry,
TextEncoder,
};
/// The single, process-wide metrics registry. Initialized by [`init`].
/// Wrapped in `OnceLock` so unit tests can run without going through
/// `main()` — the public `record_*` helpers treat an uninitialized
/// registry as a silent no-op.
static REGISTRY: OnceLock<Registry> = OnceLock::new();
static INFO: OnceLock<IntGaugeVec> = OnceLock::new();
static PUSHES: OnceLock<IntCounterVec> = OnceLock::new();
static FETCHES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_SUCCESSES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_FAILURES: OnceLock<IntCounterVec> = OnceLock::new();
static SYNC_PROCESSED: OnceLock<IntCounterVec> = OnceLock::new();
static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
/// One-time initializer. Builds the registry, registers every metric,
/// and sets the constant `gitlawb_info` gauge. Idempotent — calling
/// more than once is a silent no-op. MUST be called from `main()` after
/// the node DID is known.
pub fn init(version: &str, node_did: &str) {
if REGISTRY.get().is_some() {
return;
}
let registry = Registry::new();
let info = IntGaugeVec::new(
Opts::new(
"gitlawb_info",
"Constant 1 labelled with version and node DID",
),
&["version", "did"],
)
.expect("gitlawb_info metric definition");
registry
.register(Box::new(info.clone()))
.expect("register gitlawb_info");
info.with_label_values(&[version, node_did]).set(1);
INFO.set(info).expect("set INFO once");
let pushes = IntCounterVec::new(
Opts::new(
"gitlawb_pushes_total",
"Total successful git push (receive-pack) completions",
),
&["repo"],
)
.expect("gitlawb_pushes_total definition");
registry
.register(Box::new(pushes.clone()))
.expect("register gitlawb_pushes_total");
PUSHES.set(pushes).expect("set PUSHES once");
let fetches = IntCounterVec::new(
Opts::new(
"gitlawb_fetches_total",
"Total successful git fetch (upload-pack) completions",
),
&["repo"],
)
.expect("gitlawb_fetches_total definition");
registry
.register(Box::new(fetches.clone()))
.expect("register gitlawb_fetches_total");
FETCHES.set(fetches).expect("set FETCHES once");
let auth_successes = IntCounterVec::new(
Opts::new(
"gitlawb_auth_successes_total",
"Total HTTP signature checks that passed",
),
&["route"],
)
.expect("gitlawb_auth_successes_total definition");
registry
.register(Box::new(auth_successes.clone()))
.expect("register gitlawb_auth_successes_total");
AUTH_SUCCESSES
.set(auth_successes)
.expect("set AUTH_SUCCESSES once");
let auth_failures = IntCounterVec::new(
Opts::new(
"gitlawb_auth_failures_total",
"Total HTTP signature checks that failed",
),
&["route", "reason"],
)
.expect("gitlawb_auth_failures_total definition");
registry
.register(Box::new(auth_failures.clone()))
.expect("register gitlawb_auth_failures_total");
AUTH_FAILURES
.set(auth_failures)
.expect("set AUTH_FAILURES once");
let sync_processed = IntCounterVec::new(
Opts::new(
"gitlawb_sync_queue_processed_total",
"Total sync_queue items the worker has processed",
),
&["status"],
)
.expect("gitlawb_sync_queue_processed_total definition");
registry
.register(Box::new(sync_processed.clone()))
.expect("register gitlawb_sync_queue_processed_total");
SYNC_PROCESSED
.set(sync_processed)
.expect("set SYNC_PROCESSED once");
let webhook_deliveries = IntCounterVec::new(
Opts::new(
"gitlawb_webhook_deliveries_total",
"Total webhook delivery attempts and their outcomes",
),
&["result"],
)
.expect("gitlawb_webhook_deliveries_total definition");
registry
.register(Box::new(webhook_deliveries.clone()))
.expect("register gitlawb_webhook_deliveries_total");
WEBHOOK_DELIVERIES
.set(webhook_deliveries)
.expect("set WEBHOOK_DELIVERIES once");
let pack_size = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_pack_size_bytes",
"Distribution of git pack body sizes processed (bytes)",
)
.buckets(vec![
1_024.0, // 1 KB
64_000.0, // ~64 KB
1_000_000.0, // 1 MB
16_000_000.0, // 16 MB
64_000_000.0, // 64 MB
256_000_000.0, // 256 MB
1_073_741_824.0, // 1 GiB
2_147_483_648.0, // 2 GiB (current cap)
]),
)
.expect("gitlawb_pack_size_bytes definition");
registry
.register(Box::new(pack_size.clone()))
.expect("register gitlawb_pack_size_bytes");
PACK_SIZE.set(pack_size).expect("set PACK_SIZE once");
let peers_connected = IntGauge::with_opts(Opts::new(
"gitlawb_peers_connected",
"Currently connected libp2p peers",
))
.expect("gitlawb_peers_connected definition");
registry
.register(Box::new(peers_connected.clone()))
.expect("register gitlawb_peers_connected");
PEERS_CONNECTED
.set(peers_connected)
.expect("set PEERS_CONNECTED once");
REGISTRY
.set(registry)
.expect("set REGISTRY once (init must be called exactly once)");
}
/// Record one successful push (receive-pack completion). Labelled with
/// the repo in the form `owner_short/repo_name`. No-op if `init` was
/// never called (e.g. inside a unit test that doesn't go through main).
pub fn record_push(repo: &str) {
if let Some(c) = PUSHES.get() {
c.with_label_values(&[repo]).inc();
}
}
/// Record one successful fetch (upload-pack completion).
pub fn record_fetch(repo: &str) {
if let Some(c) = FETCHES.get() {
c.with_label_values(&[repo]).inc();
}
}
/// Record one HTTP signature check that passed.
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_success(route: &str) {
if let Some(c) = AUTH_SUCCESSES.get() {
c.with_label_values(&[route]).inc();
}
}
/// Record one HTTP signature check that failed. `reason` is a short,
/// machine-friendly token (e.g. `expired`, `bad_signature`, `missing_header`).
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_failure(route: &str, reason: &str) {
if let Some(c) = AUTH_FAILURES.get() {
c.with_label_values(&[route, reason]).inc();
}
}
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped}.
pub fn record_sync_processed(status: &str) {
if let Some(c) = SYNC_PROCESSED.get() {
c.with_label_values(&[status]).inc();
}
}
/// Record one webhook delivery attempt outcome. `result` ∈
/// {ok, http_error, network_error, skipped}.
pub fn record_webhook_delivery(result: &str) {
if let Some(c) = WEBHOOK_DELIVERIES.get() {
c.with_label_values(&[result]).inc();
}
}
/// Record a pack body size observation (bytes).
pub fn observe_pack_size(bytes: f64) {
if let Some(h) = PACK_SIZE.get() {
h.observe(bytes);
}
}
/// Update the currently-connected peer count gauge.
pub fn set_peers_connected(count: i64) {
if let Some(g) = PEERS_CONNECTED.get() {
g.set(count);
}
}
/// Encode the registry as the standard Prometheus text exposition format.
/// Returns an error if `init` was never called.
pub fn encode() -> Result<String, prometheus::Error> {
let registry = REGISTRY
.get()
.ok_or_else(|| prometheus::Error::Msg("metrics::init() was never called".into()))?;
let metric_families = registry.gather();
let mut buf = Vec::new();
let encoder = TextEncoder::new();
encoder.encode(&metric_families, &mut buf)?;
String::from_utf8(buf)
.map_err(|e| prometheus::Error::Msg(format!("non-UTF8 in metric buffer: {e}")))
}
#[cfg(test)]
mod tests {
use super::*;
// Note: these tests are not run in parallel by default (cargo runs
// tests in a binary in parallel via threads but the OnceLock guard
// means only the first init() call succeeds; subsequent ones are
// no-ops). The encode test below is structured so it's safe to run
// alongside other tests in the same binary.
#[test]
fn encode_after_init_returns_prometheus_text() {
// Reset is not supported by OnceLock, so this test relies on
// either being the first to run, or `init` being a no-op on
// second call. Both paths are exercised: if init() was already
// called by another test, the no-op branch still leaves the
// registry usable.
init("0.0.0-test", "did:key:test");
PUSHES
.get()
.expect("PUSHES set after init")
.with_label_values(&["alice/repo"])
.inc();
let body = encode().expect("encode should succeed after init");
assert!(
body.contains("# HELP gitlawb_info"),
"expected gitlawb_info HELP line in: {body}"
);
assert!(
body.contains("# TYPE gitlawb_pushes_total counter"),
"expected gitlawb_pushes_total TYPE line in: {body}"
);
assert!(
body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"),
"expected the incremented counter to be visible in: {body}"
);
}
#[test]
fn record_helpers_are_noops_before_init() {
// These don't panic. They also don't show up in encode() output
// because the registry doesn't exist yet.
record_push("test/repo");
record_fetch("test/repo");
record_auth_success("test/route");
record_auth_failure("test/route", "test_reason");
record_sync_processed("done");
record_webhook_delivery("ok");
observe_pack_size(1024.0);
set_peers_connected(0);
}
#[test]
fn encode_before_init_returns_error() {
// encode() must error rather than panic if init() was never called.
// This test is meaningful only if NO other test in this binary
// ran first. cargo runs tests on multiple threads within a
// single binary, so this is racy — but worst case the assertion
// below is "encode succeeded with a non-empty body", which is
// still a valid state.
let result = encode();
match result {
Err(_) => { /* expected when init wasn't called */ }
Ok(body) => {
// Some other test ran init() first; that's fine, the
// body should still be a valid exposition payload.
assert!(!body.is_empty(), "non-empty exposition body");
}
}
}
}
+20 -9
View File
@@ -129,9 +129,7 @@ impl OperatorClient {
.rpc_url
.parse()
.with_context(|| format!("invalid RPC URL: {}", self.cfg.rpc_url))?;
let provider = ProviderBuilder::new()
.wallet(wallet)
.connect_http(rpc_url);
let provider = ProviderBuilder::new().wallet(wallet).connect_http(rpc_url);
let contract = IGitlawbNodeStaking::new(self.cfg.contract_address, provider);
let pending = contract
@@ -147,8 +145,12 @@ impl OperatorClient {
}
/// Spawn a background task that posts a heartbeat on `heartbeat_interval` cadence.
/// Errors are logged and retried on the next tick.
pub fn spawn_heartbeat_loop(self: Arc<Self>) {
/// Errors are logged and retried on the next tick. The task exits cleanly
/// when `shutdown_rx` flips to `true`.
pub fn spawn_heartbeat_loop(
self: Arc<Self>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) {
let interval_secs = self.cfg.heartbeat_interval.as_secs();
info!(
interval_hours = interval_secs / 3600,
@@ -160,11 +162,20 @@ impl OperatorClient {
// First tick fires immediately — use it to send an initial heartbeat
interval.tick().await;
loop {
match self.send_heartbeat().await {
Ok(tx) => info!(tx = %tx, "operator heartbeat sent"),
Err(e) => warn!(err = %e, "operator heartbeat failed — will retry"),
tokio::select! {
_ = interval.tick() => {
match self.send_heartbeat().await {
Ok(tx) => info!(tx = %tx, "operator heartbeat sent"),
Err(e) => warn!(err = %e, "operator heartbeat failed — will retry"),
}
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("operator heartbeat loop: shutdown signal received, exiting");
return;
}
}
}
interval.tick().await;
}
});
}
+119 -86
View File
@@ -3,26 +3,24 @@
//! Provides:
//! - Peer discovery via Kademlia DHT (DID → multiaddr mapping)
//! - Real-time ref-update events via Gossipsub
//! - Local network discovery via mDNS
//!
//! The node's PeerId is derived from its Ed25519 identity keypair,
//! so the gitlawb DID and libp2p PeerId share the same key.
use std::collections::{HashMap, hash_map::DefaultHasher};
use std::collections::{hash_map::DefaultHasher, HashMap};
use std::hash::{Hash, Hasher};
use std::sync::Arc;
use std::time::Duration;
use anyhow::Result;
use chrono::Utc;
use libp2p::futures::StreamExt;
use libp2p::{
gossipsub, identify, kad,
mdns,
noise, tcp, yamux,
swarm::{NetworkBehaviour, SwarmEvent},
Multiaddr, PeerId,
};
use futures::StreamExt;
use libp2p_core::{muxing::StreamMuxerBox, Multiaddr, PeerId, Transport};
use libp2p_gossipsub as gossipsub;
use libp2p_identify as identify;
use libp2p_identity as identity;
use libp2p_kad as kad;
use libp2p_swarm::{NetworkBehaviour, Swarm, SwarmEvent};
use tokio::sync::{mpsc, oneshot};
use tracing::{debug, info, warn};
use uuid::Uuid;
@@ -65,6 +63,15 @@ pub struct DidRecord {
pub timestamp: String,
}
/// Snapshot of the libp2p swarm state for observability.
#[derive(Debug, Clone, serde::Serialize)]
pub struct SwarmStatus {
pub connected_peers: usize,
pub gossipsub_mesh_peers: usize,
pub gossipsub_all_peers: usize,
pub listen_addrs: Vec<String>,
}
/// Commands sent to the swarm task from the rest of the node.
#[derive(Debug)]
pub enum P2pCommand {
@@ -79,7 +86,12 @@ pub enum P2pCommand {
/// Store a DID record in the Kademlia DHT (fire-and-forget)
PutDid(DidRecord),
/// Look up a DID in the Kademlia DHT; reply on the oneshot sender
GetDid { did: String, reply: oneshot::Sender<Option<DidRecord>> },
GetDid {
did: String,
reply: oneshot::Sender<Option<DidRecord>>,
},
/// Get a snapshot of the swarm status
GetStatus { reply: oneshot::Sender<SwarmStatus> },
}
/// Handle returned to the rest of the node for sending commands to the swarm.
@@ -96,7 +108,10 @@ impl P2pHandle {
#[allow(dead_code)]
pub async fn add_peer(&self, peer_id: PeerId, addr: Multiaddr) {
let _ = self.tx.send(P2pCommand::AddKnownPeer { peer_id, addr }).await;
let _ = self
.tx
.send(P2pCommand::AddKnownPeer { peer_id, addr })
.await;
}
#[allow(dead_code)]
@@ -109,6 +124,15 @@ impl P2pHandle {
let _ = self.tx.send(P2pCommand::PutDid(record)).await;
}
pub async fn status(&self) -> Option<SwarmStatus> {
let (tx, rx) = oneshot::channel();
let _ = self.tx.send(P2pCommand::GetStatus { reply: tx }).await;
tokio::time::timeout(std::time::Duration::from_secs(2), rx)
.await
.ok()
.and_then(|r| r.ok())
}
/// Look up a DID in the DHT. Returns None if not found or timeout (10s).
pub async fn get_did(&self, did: String) -> Option<DidRecord> {
let (tx, rx) = oneshot::channel();
@@ -128,21 +152,23 @@ fn did_to_kad_key(did: &str) -> kad::RecordKey {
/// Combined libp2p behaviour.
#[derive(NetworkBehaviour)]
#[behaviour(prelude = "libp2p_swarm::derive_prelude")]
struct GitlawbBehaviour {
kademlia: kad::Behaviour<kad::store::MemoryStore>,
gossipsub: gossipsub::Behaviour,
identify: identify::Behaviour,
mdns: mdns::tokio::Behaviour,
}
/// Start the libp2p swarm. Returns a handle for sending commands and the
/// listening multiaddrs. Runs the event loop as a background tokio task.
/// listening multiaddrs. Runs the event loop as a background tokio task
/// that exits cleanly when `shutdown_rx` flips to `true`.
pub async fn start(
node_did: &str,
listen_port: u16,
bootstrap_addrs: Vec<Multiaddr>,
db: Arc<Db>,
auto_sync: bool,
shutdown_rx: tokio::sync::watch::Receiver<bool>,
) -> Result<P2pHandle> {
// Derive a stable libp2p Ed25519 key from a seed based on the node DID.
// In production you'd load/persist this key alongside the identity PEM.
@@ -159,7 +185,7 @@ pub async fn start(
seed_bytes[i * 8..(i + 1) * 8].copy_from_slice(&seed.wrapping_add(i as u64).to_le_bytes());
}
let local_key = libp2p::identity::Keypair::ed25519_from_bytes(seed_bytes)
let local_key = identity::Keypair::ed25519_from_bytes(seed_bytes)
.map_err(|e| anyhow::anyhow!("failed to create p2p keypair: {e}"))?;
let local_peer_id = PeerId::from(local_key.public());
@@ -172,68 +198,66 @@ pub async fn start(
local_peer_id,
};
// Build the swarm
let mut swarm = libp2p::SwarmBuilder::with_existing_identity(local_key)
.with_tokio()
.with_tcp(
tcp::Config::default(),
noise::Config::new,
yamux::Config::default,
)?
.with_behaviour(|key| {
let peer_id = PeerId::from(key.public());
let kad_store = kad::store::MemoryStore::new(local_peer_id);
let mut kademlia = kad::Behaviour::new(local_peer_id, kad_store);
kademlia.set_mode(Some(kad::Mode::Server));
// Kademlia
let kad_store = kad::store::MemoryStore::new(peer_id);
let mut kademlia = kad::Behaviour::new(peer_id, kad_store);
kademlia.set_mode(Some(kad::Mode::Server));
let gossipsub_config = gossipsub::ConfigBuilder::default()
.heartbeat_interval(Duration::from_secs(10))
.validation_mode(gossipsub::ValidationMode::Permissive)
.message_id_fn(|msg: &gossipsub::Message| {
let mut h = DefaultHasher::new();
msg.data.hash(&mut h);
gossipsub::MessageId::from(h.finish().to_string())
})
.build()
.expect("gossipsub config");
let gossipsub = gossipsub::Behaviour::new(
gossipsub::MessageAuthenticity::Signed(local_key.clone()),
gossipsub_config,
)
.expect("gossipsub behaviour");
// Gossipsub — message_id based on content hash to deduplicate
let gossipsub_config = gossipsub::ConfigBuilder::default()
.heartbeat_interval(Duration::from_secs(10))
.validation_mode(gossipsub::ValidationMode::Permissive)
.message_id_fn(|msg: &gossipsub::Message| {
let mut h = DefaultHasher::new();
msg.data.hash(&mut h);
gossipsub::MessageId::from(h.finish().to_string())
})
.build()
.expect("gossipsub config");
let gossipsub = gossipsub::Behaviour::new(
gossipsub::MessageAuthenticity::Signed(key.clone()),
gossipsub_config,
)
.expect("gossipsub behaviour");
let identify = identify::Behaviour::new(identify::Config::new(
"/gitlawb/1.0.0".to_string(),
local_key.public(),
));
// Identify — so peers can exchange their observed addresses
let identify = identify::Behaviour::new(identify::Config::new(
"/gitlawb/1.0.0".to_string(),
key.public(),
));
// mDNS — local network peer discovery
let mdns = mdns::tokio::Behaviour::new(
mdns::Config::default(),
peer_id,
)
.expect("mdns behaviour");
GitlawbBehaviour {
kademlia,
gossipsub,
identify,
mdns,
}
})?
.build();
let behaviour = GitlawbBehaviour {
kademlia,
gossipsub,
identify,
};
// DNS wraps QUIC so multiaddrs like /dns6/<app>.internal/udp/…/quic-v1
// resolve at dial time. On Fly, peer nodes must dial each other over the
// private 6PN network via <app>.internal hostnames — dialing through the
// public anycast edge breaks the handshake (the proxy closes the
// connection mid-stream).
let quic = libp2p_quic::tokio::Transport::new(libp2p_quic::Config::new(&local_key))
.map(|(peer_id, muxer), _| (peer_id, StreamMuxerBox::new(muxer)));
let transport = libp2p_dns::tokio::Transport::system(quic)?.boxed();
let mut swarm = Swarm::new(
transport,
behaviour,
local_peer_id,
libp2p_swarm::Config::with_tokio_executor(),
);
// Subscribe to the ref-updates topic
let topic = gossipsub::IdentTopic::new(REF_UPDATES_TOPIC);
swarm.behaviour_mut().gossipsub.subscribe(&topic)?;
// Listen
let listen_addr: Multiaddr = format!("/ip4/0.0.0.0/tcp/{listen_port}").parse()?;
swarm.listen_on(listen_addr)?;
// Listen on both IPv4 (local/mDNS + any IPv4 dials) and IPv6 (required
// for Fly's 6PN inter-app network — <app>.internal DNS only returns AAAA
// records, so peers dial us via IPv6 and need a matching IPv6 socket).
let v4: Multiaddr = format!("/ip4/0.0.0.0/udp/{listen_port}/quic-v1").parse()?;
if let Err(e) = swarm.listen_on(v4) {
warn!(err = %e, "failed to listen on IPv4");
}
let v6: Multiaddr = format!("/ip6/::/udp/{listen_port}/quic-v1").parse()?;
if let Err(e) = swarm.listen_on(v6) {
warn!(err = %e, "failed to listen on IPv6");
}
// Bootstrap Kademlia with known peers
for addr in bootstrap_addrs {
@@ -249,8 +273,18 @@ pub async fn start(
// Start the event loop as a background task
tokio::spawn(async move {
let mut shutdown_rx = shutdown_rx;
loop {
tokio::select! {
// Graceful shutdown: exit the swarm loop when the
// process-wide signal flips. This drops the Swarm
// which closes all libp2p connections cleanly.
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("p2p swarm: shutdown signal received, exiting event loop");
break;
}
}
// Handle swarm events
event = swarm.select_next_some() => {
match event {
@@ -298,14 +332,12 @@ pub async fn start(
kad::Event::OutboundQueryProgressed { id, result, .. }
)) => {
match result {
kad::QueryResult::GetRecord(Ok(ok)) => {
if let kad::GetRecordOk::FoundRecord(pr) = ok {
if let Some(reply) = pending_get_did.remove(&id) {
let record = serde_json::from_slice::<DidRecord>(
&pr.record.value
).ok();
let _ = reply.send(record);
}
kad::QueryResult::GetRecord(Ok(kad::GetRecordOk::FoundRecord(pr))) => {
if let Some(reply) = pending_get_did.remove(&id) {
let record = serde_json::from_slice::<DidRecord>(
&pr.record.value
).ok();
let _ = reply.send(record);
}
}
kad::QueryResult::GetRecord(Err(e)) => {
@@ -324,15 +356,6 @@ pub async fn start(
}
}
SwarmEvent::Behaviour(GitlawbBehaviourEvent::Mdns(
mdns::Event::Discovered(peers)
)) => {
for (peer_id, addr) in peers {
debug!(peer = %peer_id, addr = %addr, "mDNS discovered peer");
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr.clone());
let _ = swarm.dial(addr);
}
}
SwarmEvent::Behaviour(GitlawbBehaviourEvent::Identify(
identify::Event::Received { peer_id, info, .. }
)) => {
@@ -391,6 +414,16 @@ pub async fn start(
let query_id = swarm.behaviour_mut().kademlia.get_record(key);
pending_get_did.insert(query_id, reply);
}
P2pCommand::GetStatus { reply } => {
let topic_hash = gossipsub::IdentTopic::new(REF_UPDATES_TOPIC).hash();
let status = SwarmStatus {
connected_peers: swarm.connected_peers().count(),
gossipsub_mesh_peers: swarm.behaviour().gossipsub.mesh_peers(&topic_hash).count(),
gossipsub_all_peers: swarm.behaviour().gossipsub.all_peers().count(),
listen_addrs: swarm.listeners().map(|a| a.to_string()).collect(),
};
let _ = reply.send(status);
}
}
}
}
+38 -38
View File
@@ -66,33 +66,28 @@ pub async fn pin_object(
Ok(cid)
}
/// Pin all git objects in a repo that haven't yet been sent to Pinata.
/// Pin any of the given candidate git objects that haven't yet been sent to
/// Pinata.
///
/// Objects already recorded with a `pinata_cid` in the DB are skipped.
/// Returns `(sha_hex, cid)` pairs for each newly pinned object.
/// `object_list` is the already-withheld-filtered OID set to pin: the caller
/// applies `visibility_pack::replicable_objects` on the delta path or the
/// `..._fail_closed` filter on the full-scan path before calling. `repo_path` is
/// still needed to read each object's bytes. The twin in `ipfs_pin.rs` mirrors
/// this shape — change both in lockstep. Objects already recorded with a
/// `pinata_cid` are skipped. Returns `(sha_hex, cid)` pairs for each newly
/// pinned object.
pub async fn pin_new_objects(
client: &reqwest::Client,
upload_url: &str,
jwt: &str,
repo_path: &std::path::Path,
object_list: Vec<String>,
db: &crate::db::Db,
) -> Vec<(String, String)> {
if jwt.is_empty() {
return vec![];
}
let object_list = match list_all_objects(repo_path) {
Ok(v) => v,
Err(e) => {
tracing::warn!(
repo = %repo_path.display(),
err = %e,
"failed to list git objects for Pinata upload"
);
return vec![];
}
};
let mut pinned = Vec::new();
for sha in object_list {
@@ -131,25 +126,6 @@ pub async fn pin_new_objects(
pinned
}
fn list_all_objects(repo_path: &std::path::Path) -> Result<Vec<String>> {
let out = std::process::Command::new("git")
.args(["cat-file", "--batch-all-objects", "--batch-check=%(objectname)"])
.current_dir(repo_path)
.output()
.map_err(|e| anyhow::anyhow!("failed to run git cat-file: {e}"))?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!("git cat-file failed: {stderr}"));
}
Ok(String::from_utf8_lossy(&out.stdout)
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect())
}
// ── Tests ─────────────────────────────────────────────────────────────────────
#[cfg(test)]
@@ -159,7 +135,14 @@ mod tests {
#[tokio::test]
async fn test_pin_skipped_when_jwt_empty() {
let client = reqwest::Client::new();
let result = pin_object(&client, "https://uploads.pinata.cloud/v3/files", "", "deadbeef", b"data").await;
let result = pin_object(
&client,
"https://uploads.pinata.cloud/v3/files",
"",
"deadbeef",
b"data",
)
.await;
assert!(result.is_ok());
assert_eq!(result.unwrap(), "", "empty JWT must return empty CID");
}
@@ -186,7 +169,10 @@ mod tests {
.await;
assert!(result.is_ok(), "pin should succeed: {result:?}");
assert_eq!(result.unwrap(), "QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG");
assert_eq!(
result.unwrap(),
"QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG"
);
_mock.assert_async().await;
}
@@ -201,7 +187,14 @@ mod tests {
.await;
let client = reqwest::Client::new();
let result = pin_object(&client, &server.url(), "bad-jwt", "deadbeef00000000", b"data").await;
let result = pin_object(
&client,
&server.url(),
"bad-jwt",
"deadbeef00000000",
b"data",
)
.await;
assert!(result.is_err());
assert!(result.unwrap_err().to_string().contains("401"));
@@ -255,7 +248,14 @@ mod tests {
.await;
let client = reqwest::Client::new();
let result = pin_object(&client, &server.url(), "my-pinata-jwt", "deadbeef00000000", b"data").await;
let result = pin_object(
&client,
&server.url(),
"my-pinata-jwt",
"deadbeef00000000",
b"data",
)
.await;
assert!(result.is_ok());
_mock.assert_async().await;
+137
View File
@@ -0,0 +1,137 @@
use std::collections::HashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};
use axum::extract::Request;
use axum::http::StatusCode;
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
use tokio::sync::Mutex;
use crate::auth::AuthenticatedDid;
#[derive(Clone)]
struct Window {
timestamps: Vec<Instant>,
}
#[derive(Clone)]
pub struct RateLimiter {
state: Arc<Mutex<HashMap<String, Window>>>,
max_requests: usize,
window: Duration,
}
impl RateLimiter {
pub fn new(max_requests: usize, window: Duration) -> Self {
Self {
state: Arc::new(Mutex::new(HashMap::new())),
max_requests,
window,
}
}
async fn check(&self, key: &str) -> bool {
let now = Instant::now();
let mut state = self.state.lock().await;
// Look up before inserting so the common case (key already tracked)
// doesn't allocate a String per request.
if !state.contains_key(key) {
state.insert(
key.to_string(),
Window {
timestamps: Vec::new(),
},
);
}
let window = state.get_mut(key).expect("window just ensured");
window
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
if window.timestamps.len() >= self.max_requests {
return false;
}
window.timestamps.push(now);
true
}
pub async fn cleanup(&self) {
let now = Instant::now();
let mut state = self.state.lock().await;
state.retain(|_, w| {
w.timestamps
.retain(|t| now.duration_since(*t) < self.window);
!w.timestamps.is_empty()
});
}
}
pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
let limiter = request.extensions().get::<RateLimiter>().cloned();
let did = request
.extensions()
.get::<AuthenticatedDid>()
.map(|a| a.0.clone());
if let (Some(limiter), Some(did)) = (limiter, did) {
if !limiter.check(&did).await {
return (
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response();
}
}
next.run(request).await
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn allows_within_limit() {
let limiter = RateLimiter::new(3, Duration::from_secs(60));
assert!(limiter.check("did:key:test1").await);
assert!(limiter.check("did:key:test1").await);
assert!(limiter.check("did:key:test1").await);
}
#[tokio::test]
async fn blocks_over_limit() {
let limiter = RateLimiter::new(2, Duration::from_secs(60));
assert!(limiter.check("did:key:test2").await);
assert!(limiter.check("did:key:test2").await);
assert!(!limiter.check("did:key:test2").await);
}
#[tokio::test]
async fn separate_keys_independent() {
let limiter = RateLimiter::new(1, Duration::from_secs(60));
assert!(limiter.check("did:key:alice").await);
assert!(limiter.check("did:key:bob").await);
assert!(!limiter.check("did:key:alice").await);
}
#[tokio::test]
async fn window_expires() {
let limiter = RateLimiter::new(1, Duration::from_millis(50));
assert!(limiter.check("did:key:test3").await);
assert!(!limiter.check("did:key:test3").await);
tokio::time::sleep(Duration::from_millis(60)).await;
assert!(limiter.check("did:key:test3").await);
}
#[tokio::test]
async fn cleanup_removes_expired() {
let limiter = RateLimiter::new(1, Duration::from_millis(50));
limiter.check("did:key:stale").await;
tokio::time::sleep(Duration::from_millis(60)).await;
limiter.cleanup().await;
let state = limiter.state.lock().await;
assert!(state.is_empty());
}
}
+359 -86
View File
@@ -1,4 +1,5 @@
use async_graphql_axum::{GraphQLRequest, GraphQLResponse, GraphQLSubscription};
use axum::extract::DefaultBodyLimit;
use axum::{
extract::State,
middleware,
@@ -6,21 +7,32 @@ use axum::{
routing::{get, post},
Json, Router,
};
use axum::extract::DefaultBodyLimit;
use tower_http::limit::RequestBodyLimitLayer;
use serde_json::json;
use tower_http::trace::{DefaultOnResponse, DefaultOnFailure, TraceLayer};
use tower_http::limit::RequestBodyLimitLayer;
use tower_http::trace::{DefaultOnFailure, DefaultOnResponse, TraceLayer};
use tracing::Level;
use crate::api::{agents, arweave, bounties, certs, changelog, events, ipfs, issues, labels, peers, protect, pulls, register, repos, resolve, stars, tasks, webhooks};
use crate::api::{
agents, arweave, bounties, certs, changelog, events, ipfs, issues, labels, peers, profiles,
protect, pulls, register, replicas, repos, resolve, stars, tasks, visibility, webhooks,
};
use crate::auth;
use crate::rate_limit;
use crate::state::AppState;
async fn graphql_handler(
State(state): State<AppState>,
auth: Option<axum::Extension<crate::auth::AuthenticatedDid>>,
req: GraphQLRequest,
) -> GraphQLResponse {
state.graphql_schema.execute(req.into_inner()).await.into()
// `optional_signature` attaches the verified DID when a signature is present.
// Thread it into request-scoped GraphQL data; mutations enforce its presence
// in-resolver (N2) while queries stay open.
let mut inner = req.into_inner();
if let Some(axum::Extension(did)) = auth {
inner = inner.data(did);
}
state.graphql_schema.execute(inner).await.into()
}
async fn graphql_playground() -> impl IntoResponse {
@@ -30,140 +42,364 @@ async fn graphql_playground() -> impl IntoResponse {
))
}
/// Applies the standard auth middleware pair to a router: HTTP Signature verification
/// followed by UCAN chain validation. The two layers run in this order for every
/// matched request: `require_signature` first (sets `AuthenticatedDid`), then
/// `require_ucan_chain` (reads it).
fn add_auth_layers(router: Router<AppState>, state: AppState) -> Router<AppState> {
router
.layer(middleware::from_fn_with_state(
state,
auth::require_ucan_chain,
))
.layer(middleware::from_fn(auth::require_signature))
}
pub fn build_router(state: AppState) -> Router {
// ── GraphQL routes ─────────────────────────────────────────────────────
let schema = state.graphql_schema.as_ref().clone();
let graphql_routes = Router::new()
.route("/graphql", get(graphql_playground).post(graphql_handler))
// Attach the verified DID to /graphql when a signature is present. The
// layer covers only routes added before it, so /graphql/ws (added after,
// read-only subscriptions) stays open.
.layer(middleware::from_fn(auth::optional_signature))
.route_service("/graphql/ws", GraphQLSubscription::new(schema));
// ── Task routes (write — require HTTP Signature) ───────────────────────
let task_write_routes = Router::new()
.route("/api/v1/tasks", post(tasks::create_task))
.route("/api/v1/tasks/{id}/claim", post(tasks::claim_task))
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task))
.layer(middleware::from_fn(auth::require_signature));
let task_write_routes = add_auth_layers(
Router::new()
.route("/api/v1/tasks", post(tasks::create_task))
.route("/api/v1/tasks/{id}/claim", post(tasks::claim_task))
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task)),
state.clone(),
);
// ── Task routes (read — open) ──────────────────────────────────────────
let task_read_routes = Router::new()
.route("/api/v1/tasks", get(tasks::list_tasks))
.route("/api/v1/tasks/{id}", get(tasks::get_task));
// ── Write routes — require HTTP Signature ──────────────────────────────
let write_routes = Router::new()
.route("/api/v1/repos", post(repos::create_repo))
.route("/api/register", post(register::register))
.route("/api/v1/repos/{owner}/{repo}/pulls", post(pulls::create_pr))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/merge", post(pulls::merge_pr))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/close", post(pulls::close_pr))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/reviews", post(pulls::create_review))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/comments", post(pulls::create_comment))
.route("/api/v1/repos/{owner}/{repo}/hooks", post(webhooks::create_webhook))
.route("/api/v1/repos/{owner}/{repo}/hooks/{id}", axum::routing::delete(webhooks::delete_webhook))
.route("/api/v1/repos/{owner}/{repo}/branches/{branch}/protect", post(protect::protect_branch))
.route("/api/v1/repos/{owner}/{repo}/branches/{branch}/protect", axum::routing::delete(protect::unprotect_branch))
.route("/api/v1/repos/{owner}/{repo}/star", axum::routing::put(stars::star_repo))
.route("/api/v1/repos/{owner}/{repo}/star", axum::routing::delete(stars::unstar_repo))
.route("/api/v1/repos/{owner}/{repo}/fork", post(repos::fork_repo))
.route("/api/v1/repos/{owner}/{repo}/labels", post(labels::add_label))
.route("/api/v1/repos/{owner}/{repo}/labels/{label}", axum::routing::delete(labels::remove_label))
.layer(middleware::from_fn(auth::require_signature));
// ── Rate-limited creation routes — require HTTP Signature + per-DID throttle
let limiter = state.rate_limiter.clone();
let creation_routes = add_auth_layers(
Router::new()
.route("/api/v1/repos", post(repos::create_repo))
.route("/api/register", post(register::register))
.route("/api/v1/repos/{owner}/{repo}/fork", post(repos::fork_repo))
.route(
"/api/v1/repos/{owner}/{repo}/issues",
post(issues::create_issue),
)
.route("/api/v1/repos/{owner}/{repo}/pulls", post(pulls::create_pr))
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(limiter)),
state.clone(),
);
// ── Write routes — require HTTP Signature (no rate limit) ─────────────
let write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/merge",
post(pulls::merge_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/close",
post(pulls::close_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/reviews",
post(pulls::create_review),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/comments",
post(pulls::create_comment),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks",
post(webhooks::create_webhook),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks/{id}",
axum::routing::delete(webhooks::delete_webhook),
)
.route(
"/api/v1/repos/{owner}/{repo}/branches/{branch}/protect",
post(protect::protect_branch),
)
.route(
"/api/v1/repos/{owner}/{repo}/branches/{branch}/protect",
axum::routing::delete(protect::unprotect_branch),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
axum::routing::put(stars::star_repo),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
axum::routing::delete(stars::unstar_repo),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
axum::routing::put(replicas::register_replica),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
axum::routing::delete(replicas::unregister_replica),
)
.route(
"/api/v1/repos/{owner}/{repo}/labels",
post(labels::add_label),
)
.route(
"/api/v1/repos/{owner}/{repo}/labels/{label}",
axum::routing::delete(labels::remove_label),
)
.route(
"/api/v1/repos/{owner}/{repo}/visibility",
axum::routing::put(visibility::set_visibility)
.delete(visibility::remove_visibility)
.get(visibility::list_visibility),
)
.route(
"/api/v1/agents/{did}",
axum::routing::delete(agents::deregister_agent),
),
state.clone(),
);
// Body limit is raised to GITLAWB_MAX_PACK_BYTES (default 2 GB) for git
// routes only — all other API routes keep axum's default 2 MB cap.
// HTTP Signature is enforced on receive-pack (push) — the git-remote-gitlawb
// helper signs requests with RFC 9421 signatures using the agent's keypair.
let pack_limit = state.config.max_pack_bytes;
let git_write_routes = Router::new()
.route("/{owner}/{repo}/git-receive-pack", post(repos::git_receive_pack))
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit))
.layer(middleware::from_fn(auth::require_signature));
let git_write_routes = add_auth_layers(
Router::new()
.route(
"/{owner}/{repo}/git-receive-pack",
post(repos::git_receive_pack),
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit)),
state.clone(),
);
// ── IPFS content-addressed retrieval and pin listing ──────────────────
// `/ipfs/{cid}` carries `optional_signature` so `get_by_cid` sees the caller
// identity and can apply per-repo visibility (#110); anonymous callers stay
// anonymous and still read genuinely public content. `/api/v1/ipfs/pins`
// stays unsigned — gating the pin index is tracked separately (#121).
let ipfs_routes = Router::new()
.route("/ipfs/{cid}", get(ipfs::get_by_cid))
.route("/api/v1/ipfs/pins", get(ipfs::list_pins));
.layer(middleware::from_fn(auth::optional_signature))
.merge(Router::new().route("/api/v1/ipfs/pins", get(ipfs::list_pins)));
// ── Arweave permanent anchors ──────────────────────────────────────────
let arweave_routes = Router::new()
.route("/api/v1/arweave/anchors", get(arweave::list_anchors));
let arweave_routes = Router::new().route("/api/v1/arweave/anchors", get(arweave::list_anchors));
// ── Bounty routes (write — require HTTP Signature) ─────────────────
let bounty_write_routes = Router::new()
.route("/api/v1/repos/{owner}/{repo}/bounties", post(bounties::create_bounty))
.route("/api/v1/bounties/{id}/claim", post(bounties::claim_bounty))
.route("/api/v1/bounties/{id}/submit", post(bounties::submit_bounty))
.route("/api/v1/bounties/{id}/approve", post(bounties::approve_bounty))
.route("/api/v1/bounties/{id}/cancel", post(bounties::cancel_bounty))
.layer(middleware::from_fn(auth::require_signature));
let bounty_write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/bounties",
post(bounties::create_bounty),
)
.route("/api/v1/bounties/{id}/claim", post(bounties::claim_bounty))
.route(
"/api/v1/bounties/{id}/submit",
post(bounties::submit_bounty),
)
.route(
"/api/v1/bounties/{id}/approve",
post(bounties::approve_bounty),
)
.route(
"/api/v1/bounties/{id}/cancel",
post(bounties::cancel_bounty),
)
.route(
"/api/v1/bounties/{id}/dispute",
post(bounties::dispute_bounty),
),
state.clone(),
);
// ── Bounty routes (read — open) ──────────────────────────────────────
let bounty_read_routes = Router::new()
.route("/api/v1/repos/{owner}/{repo}/bounties", get(bounties::list_repo_bounties))
.route(
"/api/v1/repos/{owner}/{repo}/bounties",
get(bounties::list_repo_bounties),
)
.route("/api/v1/bounties", get(bounties::list_all_bounties))
.route("/api/v1/bounties/{id}", get(bounties::get_bounty))
.route("/api/v1/bounties/{id}/dispute", post(bounties::dispute_bounty))
.route("/api/v1/bounties/stats", get(bounties::bounty_stats))
.route("/api/v1/agents/{did}/bounties", get(bounties::agent_bounty_stats));
.route(
"/api/v1/agents/{did}/bounties",
get(bounties::agent_bounty_stats),
);
// ── Issue routes ──────────────────────────────────────────────────────
let issue_write_routes = Router::new()
.route("/api/v1/repos/{owner}/{repo}/issues", post(issues::create_issue))
.route("/api/v1/repos/{owner}/{repo}/issues/{id}/close", post(issues::close_issue))
.route("/api/v1/repos/{owner}/{repo}/issues/{id}/comments", post(issues::create_issue_comment))
.layer(middleware::from_fn(auth::require_signature));
// ── Profile routes (write — require HTTP Signature) ─────────────────
let profile_write_routes = add_auth_layers(
Router::new().route("/api/v1/profile", axum::routing::put(profiles::set_profile)),
state.clone(),
);
// ── Issue routes (write — require HTTP Signature, no rate limit) ─────
let issue_write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/close",
post(issues::close_issue),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/comments",
post(issues::create_issue_comment),
),
state.clone(),
);
// ── Peer discovery routes ─────────────────────────────────────────────
// announce is intentionally unauthenticated — nodes announce by DID which
// is self-proving; rate-limiting and blocklisting are handled elsewhere.
// Peer writes accept signatures when present and can require them after a
// coordinated live-network upgrade.
let peer_read_routes = Router::new()
.route("/api/v1/peers", get(peers::list_peers))
.route("/api/v1/peers/{did}/ping", get(peers::ping_peer));
let mut peer_write_routes = Router::new()
.route("/api/v1/peers/announce", post(peers::announce))
.route("/api/v1/peers/{did}/ping", get(peers::ping_peer))
.route("/api/v1/sync/trigger", post(peers::trigger_sync))
.route("/api/v1/sync/notify", post(peers::notify_sync));
peer_write_routes = if state.config.require_signed_peer_writes {
add_auth_layers(peer_write_routes, state.clone())
} else {
peer_write_routes.layer(middleware::from_fn(auth::optional_signature))
};
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
.route("/api/v1/repos", get(repos::list_repos))
.route("/api/v1/repos/federated", get(repos::list_federated_repos))
.route("/api/v1/repos/{owner}/{repo}", get(repos::get_repo))
.route("/api/v1/repos/{owner}/{repo}/commits", get(repos::list_commits))
.route("/api/v1/repos/{owner}/{repo}/tree", get(repos::get_tree_root))
.route("/api/v1/repos/{owner}/{repo}/tree/{*path}", get(repos::get_tree))
.route("/api/v1/repos/{owner}/{repo}/blob/{*path}", get(repos::get_blob))
.route("/api/v1/repos/{owner}/{repo}/issues", get(issues::list_issues))
.route("/api/v1/repos/{owner}/{repo}/issues/{id}", get(issues::get_issue))
.route("/api/v1/repos/{owner}/{repo}/issues/{id}/comments", get(issues::list_issue_comments))
.route("/api/v1/repos/{owner}/{repo}/labels", get(labels::list_labels))
.route(
"/api/v1/repos/{owner}/{repo}/commits",
get(repos::list_commits),
)
.route(
"/api/v1/repos/{owner}/{repo}/tree",
get(repos::get_tree_root),
)
.route(
"/api/v1/repos/{owner}/{repo}/tree/{*path}",
get(repos::get_tree),
)
.route(
"/api/v1/repos/{owner}/{repo}/blob/{*path}",
get(repos::get_blob),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues",
get(issues::list_issues),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}",
get(issues::get_issue),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/comments",
get(issues::list_issue_comments),
)
.route(
"/api/v1/repos/{owner}/{repo}/labels",
get(labels::list_labels),
)
.route("/api/v1/repos/{owner}/{repo}/certs", get(certs::list_certs))
.route("/api/v1/repos/{owner}/{repo}/certs/{id}", get(certs::get_cert))
.route("/api/v1/repos/{owner}/{repo}/events", get(events::list_repo_events))
.route(
"/api/v1/repos/{owner}/{repo}/certs/{id}",
get(certs::get_cert),
)
.route(
"/api/v1/repos/{owner}/{repo}/events",
get(events::list_repo_events),
)
.route("/api/v1/agents", get(agents::list_agents))
.route("/api/v1/agents/{did}", get(agents::show_agent))
.route("/api/v1/agents/{did}/trust", get(agents::get_trust))
.route("/api/v1/agents/{did}/profile", get(profiles::get_profile))
.route("/api/v1/events/ref-updates", get(events::list_ref_updates))
.route("/api/v1/resolve/{did}", get(resolve::resolve_did))
.route("/api/v1/repos/{owner}/{repo}/pulls", get(pulls::list_prs))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}", get(pulls::get_pr))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/diff", get(pulls::get_pr_diff))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/reviews", get(pulls::list_reviews))
.route("/api/v1/repos/{owner}/{repo}/pulls/{number}/comments", get(pulls::list_comments))
.route("/api/v1/repos/{owner}/{repo}/hooks", get(webhooks::list_webhooks))
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}",
get(pulls::get_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/diff",
get(pulls::get_pr_diff),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/reviews",
get(pulls::list_reviews),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/comments",
get(pulls::list_comments),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks",
get(webhooks::list_webhooks),
)
.route("/api/v1/repos/{owner}/{repo}/refs", get(repos::list_refs))
.route("/api/v1/repos/{owner}/{repo}/branches/protected", get(protect::list_protected_branches))
.route("/api/v1/repos/{owner}/{repo}/changelog", get(changelog::get_changelog))
.route("/api/v1/repos/{owner}/{repo}/star", get(stars::get_star_status))
.route("/{owner}/{repo}/info/refs", get(repos::git_info_refs));
.route(
"/api/v1/repos/{owner}/{repo}/branches/protected",
get(protect::list_protected_branches),
)
.route(
"/api/v1/repos/{owner}/{repo}/changelog",
get(changelog::get_changelog),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
get(stars::get_star_status),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
get(replicas::list_replicas),
)
.layer(middleware::from_fn(auth::optional_signature));
// git-upload-pack (clone/fetch) — same raised body limit as receive-pack so
// large pack responses from the server don't get truncated on the client side.
let git_read_routes = Router::new()
.route("/{owner}/{repo}/git-upload-pack", post(repos::git_upload_pack))
.route("/{owner}/{repo}/info/refs", get(repos::git_info_refs))
.route(
"/{owner}/{repo}/git-upload-pack",
post(repos::git_upload_pack),
)
.route(
"/api/v1/repos/{owner}/{repo}/withheld-paths",
axum::routing::get(visibility::withheld_paths),
)
.route(
"/api/v1/repos/{owner}/{repo}/icaptcha-proof",
axum::routing::get(repos::get_icaptcha_proof),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blobs",
axum::routing::get(crate::api::encrypted::list_encrypted_blobs),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blob/{oid}",
axum::routing::get(crate::api::encrypted::get_encrypted_blob),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate",
axum::routing::get(crate::api::encrypted::replicate_encrypted_blobs),
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit));
.layer(RequestBodyLimitLayer::new(pack_limit))
.layer(middleware::from_fn(auth::optional_signature));
// ── Meta ──────────────────────────────────────────────────────────────
let meta_routes = Router::new()
@@ -179,12 +415,15 @@ pub fn build_router(state: AppState) -> Router {
.merge(task_read_routes)
.merge(bounty_write_routes)
.merge(bounty_read_routes)
.merge(profile_write_routes)
.merge(creation_routes)
.merge(write_routes)
.merge(git_write_routes)
.merge(git_read_routes)
.merge(issue_write_routes)
.merge(read_routes)
.merge(peer_read_routes)
.merge(peer_write_routes)
.merge(ipfs_routes)
.merge(arweave_routes)
.merge(meta_routes)
@@ -221,8 +460,31 @@ async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
}))
}
async fn stats(State(state): State<AppState>) -> Json<serde_json::Value> {
let repos = state.db.list_all_repos().await.map(|r| r.len() as i64).unwrap_or(0);
pub(crate) async fn stats(State(state): State<AppState>) -> Json<serde_json::Value> {
// Count only the repos an anonymous caller could list, so the aggregate does
// not leak the existence of private/mode-A repos (#104 count oracle). Mirror
// the listing seam (api/repos.rs): over-fetch the deduped set, batch-load the
// visibility rules, and keep rows that pass listable_at_root. The caller is
// always None — meta_routes carries no auth layer (see the route group in this
// file). Fail closed: any DB error collapses the whole count to 0 (an
// under-count never leaks existence), preserving the prior `.unwrap_or(0)`.
let repos = async {
// stats only needs the count, so use the no-stars deduped list (same
// DEDUP_CTE) and skip the repo_stars aggregation the listing path needs.
let rows = state.db.list_all_repos_deduped().await?;
let ids: Vec<String> = rows.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = state.db.list_visibility_rules_for_repos(&ids).await?;
let count = rows
.iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, None)
})
.count() as i64;
Ok::<i64, anyhow::Error>(count)
}
.await
.unwrap_or(0);
let agents = state.db.count_agents().await.unwrap_or(0);
let pushes = state.db.count_pushes().await.unwrap_or(0);
Json(json!({
@@ -237,7 +499,11 @@ async fn contracts_info(State(state): State<AppState>) -> Json<serde_json::Value
let did_registry = &state.config.contract_did_registry;
let name_registry = &state.config.contract_name_registry;
let rpc_url = &state.config.chain_rpc_url;
let chain_id: u64 = if rpc_url.contains("sepolia") { 84532 } else { 8453 };
let chain_id: u64 = if rpc_url.contains("sepolia") {
84532
} else {
8453
};
Json(serde_json::json!({
"chain": if chain_id == 8453 { "base" } else { "base-sepolia" },
"chain_id": chain_id,
@@ -255,11 +521,18 @@ async fn contracts_info(State(state): State<AppState>) -> Json<serde_json::Value
async fn p2p_info(State(state): State<AppState>) -> Json<serde_json::Value> {
match &state.p2p {
Some(h) => Json(json!({
"enabled": true,
"peer_id": h.local_peer_id.to_string(),
"topics": [crate::p2p::REF_UPDATES_TOPIC],
})),
Some(h) => {
let status = h.status().await;
Json(json!({
"enabled": true,
"peer_id": h.local_peer_id.to_string(),
"topics": [crate::p2p::REF_UPDATES_TOPIC],
"connected_peers": status.as_ref().map(|s| s.connected_peers),
"gossipsub_mesh_peers": status.as_ref().map(|s| s.gossipsub_mesh_peers),
"gossipsub_all_peers": status.as_ref().map(|s| s.gossipsub_all_peers),
"listen_addrs": status.as_ref().map(|s| s.listen_addrs.clone()),
}))
}
None => Json(json!({ "enabled": false })),
}
}
+41 -1
View File
@@ -1,11 +1,12 @@
use std::sync::Arc;
use gitlawb_core::did::Did;
use gitlawb_core::identity::Keypair;
use std::sync::Arc;
use crate::config::Config;
use crate::db::Db;
use crate::git::repo_store::RepoStore;
use crate::p2p::P2pHandle;
use crate::rate_limit::RateLimiter;
#[derive(Clone, Debug)]
pub struct RefUpdateBroadcast {
@@ -48,4 +49,43 @@ pub struct AppState {
pub machine_id: Option<String>,
/// Centralized repo storage: local disk cache + optional Tigris backend
pub repo_store: RepoStore,
/// Per-DID rate limiter for creation endpoints (repos, issues, PRs)
pub rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
/// * the SIGINT/SIGTERM handler in `main()`
/// * axum's `with_graceful_shutdown` to drain in-flight HTTP requests
/// * the libp2p swarm task
/// * the gossip, sync, operator heartbeat, and rate-limit cleanup loops
pub shutdown_tx: tokio::sync::watch::Sender<bool>,
}
impl AppState {
/// Subscribe to the shutdown signal. Returns a fresh receiver whose
/// initial value matches the current state.
pub fn subscribe_shutdown(&self) -> tokio::sync::watch::Receiver<bool> {
self.shutdown_tx.subscribe()
}
/// Trigger graceful shutdown. Idempotent — calling more than once
/// has no effect. Returns `true` if this call was the one that
/// flipped the signal.
#[allow(dead_code)] // used by tests; main() drives the signal directly
pub fn shutdown(&self) -> bool {
self.shutdown_tx.send_if_modified(|v| {
if *v {
false
} else {
*v = true;
true
}
})
}
/// `true` once shutdown has been signalled.
#[allow(dead_code)] // used by tests and any future handler that wants to short-circuit
pub fn is_shutting_down(&self) -> bool {
*self.shutdown_tx.borrow()
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+439
View File
@@ -0,0 +1,439 @@
//! Pure read-authorization logic for path-scoped visibility.
//!
//! `visibility_check` decides whether a caller may read a given path in a repo,
//! based on the repo's visibility rules with a fallback to the legacy
//! `is_public` flag. It performs no I/O so it is exhaustively unit tested.
use crate::db::VisibilityRule;
use unicode_normalization::UnicodeNormalization;
#[derive(Debug, PartialEq, Eq)]
pub enum Decision {
Allow,
Deny,
}
/// NFC-normalize a glob prefix or path so the matcher compares canonically
/// equivalent strings byte-for-byte. Without this, a deny rule authored NFC
/// (`é` = U+00E9) byte-compares unequal to a path committed NFD (`e` + U+0301)
/// and the blob slips past the rule on the replication path (#101). NFC, not
/// NFKC: compatibility folding (ligatures, full-width forms) would merge paths
/// the filesystem treats as distinct and over-withhold. Both sides of every
/// comparison must pass through this, or the skew just moves.
fn nfc(s: &str) -> String {
s.nfc().collect()
}
/// True if `caller` is the repo owner (matches full did:key or its short form),
/// mirroring the owner-match idiom in `api/protect.rs`.
fn is_owner(owner_did: &str, caller: &str) -> bool {
let owner_short = owner_did.split(':').next_back().unwrap_or(owner_did);
caller == owner_did || caller == owner_short
}
/// The match prefix for a glob: "/" stays "/", "/secret/**" becomes "/secret".
fn glob_prefix(glob: &str) -> &str {
let p = glob.trim_end_matches("**").trim_end_matches('/');
if p.is_empty() {
"/"
} else {
p
}
}
/// Does `glob` match `path`? "/" matches everything; "/secret" matches
/// "/secret" and any "/secret/..." descendant.
fn glob_matches(glob: &str, path: &str) -> bool {
let prefix = glob_prefix(glob);
if prefix == "/" {
return true;
}
// Compare in NFC so an NFC rule matches a canonically-equivalent NFD path
// (and vice versa). Both sides normalized here — the single matcher seam.
let prefix = nfc(prefix);
let path = nfc(path);
path == prefix || path.starts_with(&format!("{prefix}/"))
}
/// Specificity = length of the (normalized) match prefix; longer is more
/// specific. Normalized so ranking stays consistent with `glob_matches`.
fn specificity(glob: &str) -> usize {
nfc(glob_prefix(glob)).len()
}
/// Decide whether `caller` (None = anonymous) may read `path` in a repo.
/// `path` is "/" for a whole-repo clone/fetch.
///
/// Reader DIDs in a rule are matched exactly, so they must be stored in full
/// `did:key:...` form. The owner is the only identity matched in both full and
/// short form.
pub fn visibility_check(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
path: &str,
) -> Decision {
// The owner can always read everything.
if let Some(c) = caller {
if is_owner(owner_did, c) {
return Decision::Allow;
}
}
// Most-specific matching rule wins. On equal specificity the last rule in
// DB order is chosen; `list_visibility_rules` orders by `path_glob`, so this
// is deterministic.
let best = rules
.iter()
.filter(|r| glob_matches(&r.path_glob, path))
.max_by_key(|r| specificity(&r.path_glob));
match best {
Some(rule) => {
// Phase 1 treats every matching rule as an allow-list keyed by
// `reader_dids`. `rule.mode` (A vs B) is stored from day one but not
// acted on here; it governs replication behavior in Phases 2-3.
let allowed = caller
.map(|c| rule.reader_dids.iter().any(|d| d == c))
.unwrap_or(false);
if allowed {
Decision::Allow
} else {
Decision::Deny
}
}
None => {
if is_public {
Decision::Allow
} else {
Decision::Deny
}
}
}
}
/// Whether `caller` (None = anonymous) may see a repo in a listing — the `"/"`
/// visibility decision, shared by every repo-listing surface (REST list,
/// federated list, GraphQL `repos`) so they enforce one rule, not three drifting
/// copies. Not a bare `is_public` test: a repo can be `is_public=false` with a
/// root rule granting readers, or `is_public=true` with a root deny (#97).
pub fn listable_at_root(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> bool {
visibility_check(rules, is_public, owner_did, caller, "/") == Decision::Allow
}
/// The subtree path globs that `caller` (None = anonymous) may NOT read, given
/// the repo's rules. Whole-repo ("/") rules are excluded: a denied whole-repo
/// read is handled by the 404 gate before a clone ever starts. Each remaining
/// rule is reported when `visibility_check` denies the caller at the glob's
/// representative path. Used by the clean-clone client to sparse-exclude the
/// private paths from checkout.
pub fn withheld_globs(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Vec<String> {
rules
.iter()
.filter(|r| r.path_glob != "/")
.filter(|r| {
let probe = glob_prefix(&r.path_glob);
visibility_check(rules, is_public, owner_did, caller, probe) == Decision::Deny
})
.map(|r| r.path_glob.clone())
.collect()
}
/// The allowed globs that sit strictly underneath a denied glob. A clean-clone
/// client sparse-excludes everything in `withheld_globs`, which would also hide
/// these nested allowed paths; re-including them restores the caller's access.
/// Example: with `/secret/**` denied and `/secret/public/**` allowed for the
/// same caller, `/secret/public/**` is returned here so the client re-includes
/// it after excluding `/secret/`.
pub fn reincluded_globs(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Vec<String> {
let denied: Vec<&str> = rules
.iter()
.filter(|r| r.path_glob != "/")
.filter(|r| {
visibility_check(
rules,
is_public,
owner_did,
caller,
glob_prefix(&r.path_glob),
) == Decision::Deny
})
.map(|r| glob_prefix(&r.path_glob))
.collect();
rules
.iter()
.filter(|r| r.path_glob != "/")
.filter(|r| {
visibility_check(
rules,
is_public,
owner_did,
caller,
glob_prefix(&r.path_glob),
) == Decision::Allow
})
.filter(|r| {
let p = glob_prefix(&r.path_glob);
denied
.iter()
.any(|d| *d != p && p.starts_with(&format!("{d}/")))
})
.map(|r| r.path_glob.clone())
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::VisibilityMode;
use chrono::Utc;
fn rule(path_glob: &str, mode: VisibilityMode, readers: &[&str]) -> VisibilityRule {
VisibilityRule {
id: "x".into(),
repo_id: "r1".into(),
path_glob: path_glob.into(),
mode,
reader_dids: readers.iter().map(|s| s.to_string()).collect(),
created_by: "did:key:z6MkOwner".into(),
created_at: Utc::now(),
}
}
const OWNER: &str = "did:key:z6MkOwner";
#[test]
fn withheld_globs_lists_only_denied_subtrees() {
let rules = [
rule("/secret/**", VisibilityMode::B, &["did:key:z6MkFriend"]),
rule("/docs/**", VisibilityMode::B, &["did:key:z6MkStranger"]),
];
// Stranger is denied /secret but allowed /docs.
let mut got = withheld_globs(&rules, true, OWNER, Some("did:key:z6MkStranger"));
got.sort();
assert_eq!(got, vec!["/secret/**".to_string()]);
// Owner is denied nothing.
assert!(withheld_globs(&rules, true, OWNER, Some(OWNER)).is_empty());
// Anonymous is denied both.
let mut anon = withheld_globs(&rules, true, OWNER, None);
anon.sort();
assert_eq!(anon, vec!["/docs/**".to_string(), "/secret/**".to_string()]);
}
#[test]
fn reincluded_globs_restores_allowed_nested_path() {
let rules = [
rule("/secret/**", VisibilityMode::B, &["did:key:z6MkFriend"]),
rule(
"/secret/public/**",
VisibilityMode::B,
&["did:key:z6MkFriend", "did:key:z6MkStranger"],
),
];
// Stranger is denied /secret/** but allowed the nested /secret/public/**.
let withheld = withheld_globs(&rules, true, OWNER, Some("did:key:z6MkStranger"));
assert_eq!(withheld, vec!["/secret/**".to_string()]);
let reinc = reincluded_globs(&rules, true, OWNER, Some("did:key:z6MkStranger"));
assert_eq!(reinc, vec!["/secret/public/**".to_string()]);
// Owner is denied nothing, so there is nothing to re-include.
assert!(reincluded_globs(&rules, true, OWNER, Some(OWNER)).is_empty());
}
#[test]
fn no_rules_public_allows_anonymous() {
assert_eq!(
visibility_check(&[], true, OWNER, None, "/"),
Decision::Allow
);
}
#[test]
fn no_rules_private_denies_anonymous() {
assert_eq!(
visibility_check(&[], false, OWNER, None, "/"),
Decision::Deny
);
}
#[test]
fn root_rule_denies_anonymous() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/"),
Decision::Deny
);
}
#[test]
fn root_rule_allows_owner() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some(OWNER), "/"),
Decision::Allow
);
}
#[test]
fn root_rule_allows_owner_short_form() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some("z6MkOwner"), "/"),
Decision::Allow
);
}
#[test]
fn root_rule_allows_listed_reader() {
let rules = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some("did:key:z6MkFriend"), "/"),
Decision::Allow
);
}
#[test]
fn root_rule_denies_unlisted_reader() {
let rules = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some("did:key:z6MkStranger"), "/"),
Decision::Deny
);
}
#[test]
fn subtree_rule_matches_descendant_path() {
let rules = [rule(
"/secret/**",
VisibilityMode::B,
&["did:key:z6MkFriend"],
)];
assert_eq!(
visibility_check(
&rules,
true,
OWNER,
Some("did:key:z6MkStranger"),
"/secret/a.rs"
),
Decision::Deny
);
assert_eq!(
visibility_check(
&rules,
true,
OWNER,
Some("did:key:z6MkFriend"),
"/secret/a.rs"
),
Decision::Allow
);
}
#[test]
fn subtree_rule_does_not_affect_root_clone() {
// A subtree rule must not gate a whole-repo (path "/") read: the public
// fallback applies because the subtree glob does not match "/".
let rules = [rule("/secret/**", VisibilityMode::B, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/"),
Decision::Allow
);
}
#[test]
fn most_specific_rule_wins() {
// Public repo, but /secret is locked. A stranger reading /secret is denied
// by the more specific rule even though "/" would allow.
let rules = [
rule("/", VisibilityMode::A, &["did:key:z6MkStranger"]),
rule("/secret/**", VisibilityMode::B, &["did:key:z6MkFriend"]),
];
// stranger is a root reader but not a /secret reader
assert_eq!(
visibility_check(
&rules,
true,
OWNER,
Some("did:key:z6MkStranger"),
"/secret/a.rs"
),
Decision::Deny
);
// stranger can still read root
assert_eq!(
visibility_check(&rules, true, OWNER, Some("did:key:z6MkStranger"), "/"),
Decision::Allow
);
}
// Mirrors the gossip-announce gate in git_receive_pack: announce iff an
// anonymous caller can read "/".
#[test]
fn announce_gate_matches_public_readability() {
let announce = |rules: &[VisibilityRule], is_public: bool| {
visibility_check(rules, is_public, OWNER, None, "/") == Decision::Allow
};
// Public repo, no rules → announce.
assert!(announce(&[], true));
// Legacy private repo (is_public false, no rules) → silent.
assert!(!announce(&[], false));
// Mode A whole-repo rule with no public readers → silent.
assert!(!announce(&[rule("/", VisibilityMode::A, &[])], true));
// Mode B public repo with a private subtree → still announce.
assert!(announce(
&[rule("/secret/**", VisibilityMode::B, &[])],
true
));
}
// #101: a deny rule must withhold a path that denotes the same characters in
// a different Unicode normalization form. Without NFC normalization at the
// matcher seam, an NFC-authored rule byte-compares unequal to an NFD-stored
// path and the blob leaks on the replication path.
#[test]
fn matcher_withholds_across_nfc_nfd_normalization_skew() {
// "é" composed (NFC, U+00E9) in the rule; decomposed (NFD, e + U+0301)
// in the committed path.
let nfc_rule = "/s\u{00e9}cret/**";
let nfd_path = "/se\u{0301}cret/key.pem";
let rules = [rule(nfc_rule, VisibilityMode::B, &["did:key:z6MkFriend"])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, nfd_path),
Decision::Deny,
"NFC-authored deny rule must withhold the canonically-equivalent NFD path"
);
// Mirror: rule authored NFD, path committed NFC.
let nfd_rule = "/se\u{0301}cret/**";
let nfc_path = "/s\u{00e9}cret/key.pem";
let rules2 = [rule(nfd_rule, VisibilityMode::B, &["did:key:z6MkFriend"])];
assert_eq!(
visibility_check(&rules2, true, OWNER, None, nfc_path),
Decision::Deny,
"NFD-authored deny rule must withhold the canonically-equivalent NFC path"
);
// A genuinely different path is still allowed (no over-withholding).
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/public/x.txt"),
Decision::Allow
);
}
}
+25 -14
View File
@@ -24,8 +24,8 @@ type HmacSha256 = Hmac<Sha256>;
/// Compute `sha256=<hex>` HMAC signature for a webhook payload.
fn sign_payload(secret: &str, payload: &[u8]) -> String {
let mut mac = HmacSha256::new_from_slice(secret.as_bytes())
.expect("HMAC accepts any key length");
let mut mac =
HmacSha256::new_from_slice(secret.as_bytes()).expect("HMAC accepts any key length");
mac.update(payload);
format!("sha256={}", hex::encode(mac.finalize().into_bytes()))
}
@@ -93,18 +93,29 @@ async fn fire_event_async(
}
match req.send().await {
Ok(resp) => tracing::info!(
url = %hook.url,
event = %event_name,
status = %resp.status(),
"webhook delivered"
),
Err(e) => tracing::warn!(
url = %hook.url,
event = %event_name,
err = %e,
"webhook delivery failed"
),
Ok(resp) => {
let result_label = if resp.status().is_success() {
"ok"
} else {
"http_error"
};
crate::metrics::record_webhook_delivery(result_label);
tracing::info!(
url = %hook.url,
event = %event_name,
status = %resp.status(),
"webhook delivered"
)
}
Err(e) => {
crate::metrics::record_webhook_delivery("network_error");
tracing::warn!(
url = %hook.url,
event = %event_name,
err = %e,
"webhook delivery failed"
)
}
}
});
}
+2 -1
View File
@@ -1,8 +1,9 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version.workspace = true
version = "0.4.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
+50 -13
View File
@@ -36,7 +36,11 @@ pub enum AgentCmd {
pub async fn run(args: AgentArgs) -> Result<()> {
match args.cmd {
AgentCmd::List { node, capability, dir: _ } => cmd_list(node, capability).await,
AgentCmd::List {
node,
capability,
dir: _,
} => cmd_list(node, capability).await,
AgentCmd::Show { did, node } => cmd_show(did, node).await,
}
}
@@ -49,7 +53,10 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
None => "/api/v1/agents".to_string(),
};
let resp = client.get(&path).await.context("failed to connect to node")?;
let resp = client
.get(&path)
.await
.context("failed to connect to node")?;
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
@@ -59,7 +66,10 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
);
}
let body: Value = resp.json().await.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
let body: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
@@ -76,10 +86,20 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
println!("Agents on {} ({} total)\n", node, agents.len());
for agent in &agents {
let did = agent["did"].as_str().unwrap_or("?");
let short = did.split(':').next_back().map(|s| &s[..s.len().min(16)]).unwrap_or("?");
let short = did
.split(':')
.next_back()
.map(|s| &s[..s.len().min(16)])
.unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
let caps = agent["capabilities"].as_array()
.map(|a| a.iter().filter_map(|v| v.as_str()).collect::<Vec<_>>().join(", "))
let caps = agent["capabilities"]
.as_array()
.map(|a| {
a.iter()
.filter_map(|v| v.as_str())
.collect::<Vec<_>>()
.join(", ")
})
.unwrap_or_default();
let model = agent["model"].as_str().unwrap_or("");
println!(" {short}… trust={trust:.2} {caps}");
@@ -106,7 +126,10 @@ async fn cmd_show(did: String, node: String) -> Result<()> {
);
}
let agent: Value = resp.json().await.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
let agent: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = agent["message"].as_str().unwrap_or("unknown error");
@@ -115,8 +138,14 @@ async fn cmd_show(did: String, node: String) -> Result<()> {
let full_did = agent["did"].as_str().unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
let caps = agent["capabilities"].as_array()
.map(|a| a.iter().filter_map(|v| v.as_str()).collect::<Vec<_>>().join(", "))
let caps = agent["capabilities"]
.as_array()
.map(|a| {
a.iter()
.filter_map(|v| v.as_str())
.collect::<Vec<_>>()
.join(", ")
})
.unwrap_or_default();
let registered = agent["registered_at"].as_str().unwrap_or("?");
let model = agent["model"].as_str().unwrap_or("(none)");
@@ -172,7 +201,9 @@ mod tests {
.create_async()
.await;
cmd_list(server.url(), Some("git:push".to_string())).await.unwrap();
cmd_list(server.url(), Some("git:push".to_string()))
.await
.unwrap();
_m.assert_async().await;
}
@@ -217,7 +248,9 @@ mod tests {
.create_async()
.await;
cmd_show("did:key:z6MkTest".to_string(), server.url()).await.unwrap();
cmd_show("did:key:z6MkTest".to_string(), server.url())
.await
.unwrap();
}
#[tokio::test]
@@ -231,7 +264,9 @@ mod tests {
.create_async()
.await;
let err = cmd_show("did:key:z6MkMissing".to_string(), server.url()).await.unwrap_err();
let err = cmd_show("did:key:z6MkMissing".to_string(), server.url())
.await
.unwrap_err();
assert!(err.to_string().contains("agents API") || err.to_string().contains("not found"));
}
@@ -246,6 +281,8 @@ mod tests {
.create_async()
.await;
cmd_show("did:key:z6MkHigh".to_string(), server.url()).await.unwrap();
cmd_show("did:key:z6MkHigh".to_string(), server.url())
.await
.unwrap();
}
}
Loaded 100 of 178 files, more files were not shown because too many files have changed in this diff. Show more