Compare commits

...
Author SHA1 Message Date
beardthelion 9a46e8cbb7 fix(core): zeroize the derived X25519 secret (#65) (#66)
x25519_secret_from_seed derived the X25519 private scalar from the Ed25519
seed and returned it as a bare [u8; 32]. Because [u8; 32] is Copy with no
Drop (and sha2's digest output likewise has no zeroize), the scalar, the
SHA-512 digest, and the returned temporary were released without being
scrubbed, leaving secret-derived material in freed memory. This is the same
Copy-no-Drop class fixed for the raw seed in #41, on the derived secret.

Return Zeroizing<[u8; 32]>, build the scalar directly into a zeroizing
buffer so no bare secret local persists, and explicitly wipe the SHA-512
digest before it drops. Callers deref the result into crypto_box::SecretKey,
which already scrubs its own copy. No behavior change: the derived scalar and
all decryption output are byte-identical, covered by the existing
ed25519_to_x25519_keypair_agrees and seal_open_round_trip_for_recipients tests.

Stacked on #41 (#64).
2026-06-23 22:05:56 +08:00
beardthelion e4f5e87e0b fix(node): migrate node_seed pin caller to to_seed()
main grew a third seed_bytes() caller (repos.rs, from #40) after this
branch was cut. Route it through to_seed() too: node_seed is moved into
a long-lived tokio::spawn pin task, so the Zeroizing wrapper scrubs the
copy when that task ends. encrypt_and_pin takes &[u8; 32]; the wrapper
deref-coerces, so the call site is unchanged.
2026-06-22 12:14:40 -05:00
beardthelion 38e654a289 Merge remote-tracking branch 'origin/main' into fix/zeroize-seed-bytes 2026-06-22 11:59:42 -05:00
beardthelion 4f814459cc test(core): pin to_seed's Zeroizing return type, drop duplicate round-trip
The to_seed_is_sole_seed_accessor test was named for a compile-time
property its body couldn't assert, and its from_seed -> DID check
duplicated from_seed_round_trip. Replace it with a type-annotated
binding that fails to compile if to_seed ever returns a bare [u8; 32]
again, plus the determinism check, which is the only behavior it
uniquely covered.
2026-06-22 11:52:38 -05:00
beardthelion 7533b74f79 ci: harden PR pipeline (audit, MSRV, beta matrix, release + docker smoke) (#57)
* ci: harden PR pipeline (audit, MSRV, release build, beta matrix, docker smoke)

Restructure the single stable-only fmt+clippy+test job into separate jobs:
fmt+clippy (stable), a test matrix on stable+beta (beta non-blocking),
build --release, cargo audit, an MSRV check pinned to Rust 1.91, and a
Docker build smoke test mirroring the release workflow. Add concurrency
cancellation for superseded PR runs and per-job timeouts.

Declare rust-version = "1.91" in [workspace.package] and inherit it in
every member crate so the stated MSRV is a real, CI-enforced contract.

* ci: disable credential persistence on checkout steps

Set persist-credentials: false on all six actions/checkout steps. These
jobs run untrusted PR code via cargo and docker build, and checkout's
default leaves the GITHUB_TOKEN in the local git config where that code
could read it. None of the jobs push or otherwise need git credentials.

* fix(deps): bump rustls-webpki to 0.103.13

Clears three advisories flagged by the new cargo audit job: a reachable
panic in CRL parsing and two name-constraint bypasses (wildcard and URI).
Semver-compatible patch, lockfile only, no manifest or code change.

cargo also corrects a stale lockfile reference while rewriting Cargo.lock:
data-encoding-macro-internal 0.1.17 (a build-time macro crate) requires
syn 1.x, and its dependency entry is re-pinned from syn 2.0.117 to the
1.0.109 it actually resolves to. Unrelated to the bump, unavoidable when
cargo touches the lock, and the more-correct state.

* ci(audit): suppress no-fix advisories with justification and a re-check

The cargo audit job stays hard-fail. The remaining three advisories have
no available upstream fix, so they are suppressed in .cargo/audit.toml
(the location cargo-audit actually reads; a repo-root audit.toml is not
read), each with an inline rationale and removal condition:

- RUSTSEC-2026-0118/0119 (hickory-proto 0.25.2, via libp2p-dns 0.44):
  reachable, since the node wraps QUIC in the system DNS resolver. No fix
  until libp2p-dns ships hickory 0.26. Accepted, reachable risk.
- RUSTSEC-2023-0071 (rsa Marvin): in the lockfile via sqlx's unconditional
  sqlx-mysql, but not linked (sqlx is postgres-only), so no RSA oracle
  exists; no upstream fix regardless.

Two controls keep the suppression honest. The PR audit job echoes the
active ignore list to the job summary, so a green check is not read as
advisory-clean. A weekly scheduled job re-runs cargo audit without the
ignores for visibility and hard-fails if hickory-proto moves off 0.25.x
while the ignores remain, so the suppression cannot silently outlive the
upstream fix.

* ci(audit): reference tracking issue #76 in the hickory ignores

* ci(audit): treat hickory-proto removal as drift too

The scheduled drift guard only failed when hickory-proto was present and
off 0.25.x, so removing it from Cargo.lock entirely left the RUSTSEC-2026-0118
/0119 ignores in place with the guard still passing. Treat an absent version
as drift as well, so dead ignore entries are caught whether the dependency is
bumped past 0.25.x or removed.
2026-06-22 19:02:38 +08:00
beardthelion abdc775770 feat(node): blind recipient identities at rest and gate B1 by repo readability (#40)
* feat(node): blind recipient identities at rest and gate B1 by repo readability

The origin no longer stores recipient DIDs. Migration v5 replaces the
encrypted_blobs.recipients column with an opaque, node-keyed recipients_tag
used only to detect a recipient-set change for re-seal. B1 discovery and fetch
are now gated by the same repo-readability check the git read path uses, not by
per-recipient matching; decryption is gated by the envelope crypto, so a
non-recipient who can read the repo sees a blob's {oid, cid} but cannot open it.
encrypt_and_pin keys the tag from the node seed and returns {oid, cid}; the
Arweave manifest tuple drops the now-unused recipient vec.

A DB compromise no longer reveals the reader set; recovering it would require
brute-forcing candidate DID sets against the keyed tag with the node key.

* chore: remove redundant .gitignore entry (covered by local exclude)

* fix(node): gate /replicate by repo readability; harden reseal on DB error

Address review on the at-rest blinding change:
- The encrypted-blobs/replicate listing returned {oid, cid} with no visibility
  check, so a non-readable repo's blob index was reachable by an unauthenticated
  caller who guessed {owner}/{repo}. Gate it by the same repo-readability check
  discovery and fetch use. For the intended case (a public repo with withheld
  subtrees) the public root keeps this open to peers; only fully non-readable
  repos are withheld, which is the desired behavior.
- encrypt_and_pin treated a recipients_tag DB read error as a cache miss and
  resealed, causing avoidable IPFS writes during a partial outage; skip and retry
  on the next push instead.
- Correct the get_encrypted_blob doc comment to describe repo-readability access.
2026-06-22 19:01:19 +08:00
beardthelion d45318281c fix(core): route seed access through the zeroizing wrapper (#41)
Remove Keypair::seed_bytes(), which returned a bare [u8; 32] copy of the
Ed25519 private seed. Because [u8; 32] is Copy with no Drop, each caller's
copy was released without being scrubbed, leaving secret key material in
freed memory. The zeroizing accessor to_seed() -> Zeroizing<[u8; 32]>
already existed beside it; this makes it the sole seed accessor and migrates
the two in-crate callers (envelope open path and a crypto test) to it via
deref coercion. No behavior change to envelope encryption: the derived
X25519 secret is byte-identical, covered by the existing round-trip tests.

Adds an identity test pinning to_seed() as the only seed accessor.
2026-06-19 22:01:08 -05:00
beardthelion e37ea7fec6 fix(node): enforce path-scoped visibility on the REST read API (#52)
* fix(node): enforce path-scoped visibility on the REST read API

The repo-scoped REST read endpoints (blob, tree, commits, refs, changelog,
pulls, pr diff, repo metadata) read straight from disk after a bare repo
lookup, with no visibility check, so private repositories and private
subtrees of public repositories were served to unauthenticated callers,
bypassing the withholding the git and replication paths enforce.

Add a shared authorize_repo_read gate (resolve repo, load rules,
visibility_check, 404 on deny) and call it from every repo-scoped read
handler. get_blob gates at the requested file path; get_pr_diff withholds
the whole diff when it touches a path the caller cannot read. Attach
optional_signature to the read routes so authorized readers authenticate.
Listings stay gated at the repo root, consistent with mode B exposing tree
and commit SHAs over git.

Closes #51

* fix(node): gate PR reviews/comments and fork on repo read visibility

The REST visibility gate added in this PR missed three read surfaces:
list_reviews and list_comments served a private repo's PR reviews and
comments to unauthenticated callers, and fork_repo cloned a full mirror of
the source without checking the caller could read it. Route all three through
authorize_repo_read at the repo root, matching the git-protocol gate.

Also make branch_diff_names fail closed: it ignored git's exit status and
returned an empty path list on error, which would let get_pr_diff serve a diff
it should have withheld. Fix changelog.rs import order to satisfy cargo fmt.

* fix(node): NUL-delimit branch_diff_names to keep path-gate enforcement exact

git diff --name-only quotes and escapes paths containing newlines, so they
no longer match the visibility globs in get_pr_diff. On a public repo where /
is allowed but a subtree is denied, a touched restricted path could be missed
by the deny check and the full PR diff disclosed. Use -z and split on NUL to
preserve exact path bytes.
2026-06-19 22:53:18 +08:00
beardthelion 197c4e1cf1 Add contributor templates, issue forms, and PR triage automation (#58)
* docs: add pull request template

* docs: add structured issue forms and template chooser config

* chore: add CODEOWNERS for CI, release, and security paths

* ci: add quality-signal PR triage workflow

* ci: add label-gated auto-stale workflow for ignored PRs

* docs: document PR quality bar and triage/stale policy in CONTRIBUTING

* ci: recognize inline #[cfg(test)] tests and fix triage comment grammar
2026-06-19 22:52:32 +08:00
a5d2264cc0 ci(release): smoke-test packaged binaries' --version before attaching (#59)
The release-binaries job packaged gl, git-remote-gitlawb, and gitlawb-node
into the release tarball but never ran them, so a broken artifact could ship.
This is the follow-up promised in #30 and the gap noted in OSS-READINESS-AUDIT
("release binaries are not smoke-tested after packaging"), now unblocked by
git-remote-gitlawb gaining --version.

Run each binary's --version after build and assert the released version string
is present. Skipped for aarch64-unknown-linux-musl, which is cross-compiled and
can't execute on the x86_64 runner.

Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-19 22:50:35 +08:00
beardthelionandCommandCodeBot 5ff7af84fa feat: encrypted replication for private subtrees (B1/B2/B3) for #18 (#36)
* feat(core): ed25519 to x25519 key conversion for envelope crypto

* feat(core): seal/open per-blob encryption envelopes

* feat(node): resolve per-blob encryption recipients (least-privilege)

* feat(node): encrypted_blobs table and recipient-scoped queries

* feat(node): encrypt-then-pin withheld blobs at push (IPFS)

At the push chokepoint, after pinning withheld objects, resolve each
withheld blob's recipient DIDs and seal it to their Ed25519 keys with
seal_blob, pinning the ciphertext to IPFS and recording it in
encrypted_blobs. Best-effort per blob: failures are logged and skipped,
never pinned in plaintext. Pinata replication is unchanged; B1
encrypts to IPFS only.

Adds ed25519-dalek as a direct dependency of gitlawb-node (it was only
declared in the workspace Cargo.toml).

* feat(node): authenticated discovery and fetch for encrypted blobs

* feat(gl): transparent recovery of authorized encrypted blobs on clone

* style: cargo fmt

* fix(node): re-seal encrypted blob when recipient set changes

Skip re-pinning only when an existing envelope already covers exactly the
current recipients. A reader added to a rule after the first pin now gets
a re-seal on the next push instead of being permanently locked out.
Reader removal stays non-retroactive (the old envelope is already public).

* feat(node): unscoped encrypted-blob listing for replication

* feat(node): encrypted-blobs/replicate endpoint for peer mirrors

* feat(node): route encrypted-blobs/replicate under git_read_routes

* feat(node): peer mirrors replicate encrypted withheld blobs (Option B2)

* feat(node): anchor_encrypted_manifest for Option B3 Arweave manifests

* feat(node): encrypt_and_pin returns the per-push sealed delta

* feat(node): anchor encrypted-blob manifest on push (Option B3)

* feat(gl): Arweave/IPFS gateway recovery for encrypted blobs (Option B3)

* feat(gl): transparent Arweave fallback recovery on clone (Option B3)

* style: cargo fmt

* fix(gl): bound Arweave recovery gateway requests with a 30s timeout

* test(gl): read-path recovery tests for Arweave fallback; quiet promisor present-check

Add two hermetic integration tests for recover_from_arweave that drive the
full read path over mocked Arweave GraphQL + IPFS gateways: discover the
manifest, fetch it, fetch the envelope, decrypt, and install the withheld
blob. One covers an authorized recipient (blob installed), the other a
non-recipient (nothing recovered). Both simulate origin death by removing
the promisor remote and enable uploadpack.allowFilter so the blob is truly
withheld over file://.

Also harden the local presence check in recover_from_arweave with
GIT_NO_LAZY_FETCH=1 and .output() so the expected 'missing object' case does
not trigger a wasted promisor fetch or leak git stderr to the user.

* fix: surface corrupt recipients JSON and silent recovery I/O failures

db: parse_recipients now propagates a descriptive error instead of
defaulting corrupt recipients JSON to an empty list, which would have
denied authorized readers and handed peers incomplete metadata.

gl: clone recovery now warns when the sparse-checkout file cannot be read
or written, or when the post-recovery checkout fails, instead of silently
discarding those errors and claiming files were recovered.

* feat(core): blind recipient identities in withheld-blob envelopes

Envelope v2 drops the cleartext recipient public key (kid) from each
wrapped-key header entry, so a party holding the public IPFS/Arweave copy
can no longer enumerate who is authorized to decrypt a withheld blob. The
version is bumped to 2 and v1 envelopes are rejected. open_blob now selects
the reader's entry by trial decryption (the AEAD tag authenticates exactly
one entry) instead of matching on the public key.

Recipient count is still visible; identity is not. Scope is envelope-only,
the node DB recipients column and peer replication metadata are unchanged.

Co-authored-by: CommandCodeBot <[email protected]>

* fix(core): reject malformed envelope nonces instead of panicking

open_blob fed attacker-controlled envelopes to GenericArray::from_slice and
XNonce::from_slice, which panic on a wrong-length input. Validate both the
per-recipient box nonce and the body nonce to 24 bytes before use: skip a
recipient entry whose nonce is malformed, and return an error for a malformed
body nonce, so the public recovery path surfaces an error rather than panicking.

* feat(node): blind recipient identities on the B2 replication surface

The encrypted-blobs/replicate endpoint shipped every blob's full recipient
DID list to mirrors, which persisted it, so any mirroring peer learned the
reader set. The v2 envelope blinding already removed recipient public keys
from the pinned envelopes, so the comment justifying this (DIDs are already
public) was no longer true.

/replicate now returns {oid, cid} only. Mirrors detect a re-seal by the CID
changing (the OID is stable across re-seals) instead of comparing recipient
sets, and store no recipient identities. Origin-side authz and the at-rest
recipients column are unchanged; this blinds only the peer-facing surface.

* feat(node): stop anchoring recipient identities to Arweave

The B3 encrypted-blob manifest anchored {oid, cid, recipients} to Arweave, a
permanent public record of every blob's reader set. The v2 envelope blinding
already removed recipient keys from the pinned envelopes, so the comment
justifying this (recipient DIDs are already public) was no longer true. The
manifest now anchors {oid, cid} only. The gl reader already ignores the
recipients field (it recovers by trial decryption), so no reader changes.

---------

Co-authored-by: CommandCodeBot <[email protected]>
2026-06-19 22:48:33 +08:00
beardthelionandKevin Codex 8680d0f9d6 feat(node): replication enforcement (Phase 2) for #18 (#34)
* docs(node): Phase 3 subtree-withholding plan with Task 0 spike findings

* feat(node): resolve withheld blob OIDs for path-scoped visibility

* feat(node): filtered upload-pack serve that omits withheld blobs

* feat(node): serve filtered pack when caller has withheld subtree blobs

* test(node): end-to-end assert served pack omits withheld blob

* docs(node): note why info/refs is not gated on subtree visibility

* fix(node): frame filtered serve as protocol v0 and prove it with a real clone

upload_pack_excluding emitted a v2 packfile section, but info_refs
advertises v0, so real clients negotiated v0 and rejected the response
with 'expected ACK/NAK, got packfile'. Frame the v0 stateless-rpc shape
instead (NAK, then the pack via side-band-64k when offered).

Add an end-to-end test that stands up info_refs + upload_pack_excluding
and runs a real git partial clone, asserting the withheld blob's bytes
never reach the client while its tree entry and SHA stay visible. A stock
full clone cannot consume the pack (it is not closed under reachability,
so fetch fails the connectivity check); a partial clone is required.

* docs(node): correct Phase 3 caveat (full clone refused at fetch, not checkout)

* test(node): prove incremental fetch still withholds; document negotiation choice

Add a real-git test that partial-clones, pushes a new commit server-side,
then fetches: the new object arrives and the withheld blob stays absent.
This pins down that ignoring have/want negotiation (always sending a
self-contained pack of all refs minus withheld, with NAK) is correct for
both clone and fetch; the only cost is a fetch re-sends the full object
set. Refactor the real-git tests onto a shared server harness and document
the negotiation decision in code and in the plan's follow-ups.

* refactor(node): address CodeRabbit review on PR #28

Move the two blocking git shell-outs in the filtered upload-pack path off
the async worker thread, matching the tokio::process / spawn_blocking usage
already in this file: build_filtered_pack (rev-list + pack-objects) and
withheld_blob_oids (per-ref ls-tree) now run inside spawn_blocking so a large
repo cannot stall the tokio runtime. Behavior is unchanged.

Also fix the Task 0 findings block in the Phase 3 plan: it still recorded v2
packfile framing, which is the exact path that failed against a real client
and was corrected to v0. The block now documents the shipped v0 contract.
Drop a stray trailing code fence flagged by markdownlint (MD040).

The speculative ls-tree timeout and the public/no-rules fast-path from the
review are intentionally left out: the timeout guards against adversarial
repos we do not yet host, and the fast-path is a micro-optimization not worth
the extra branch right now.

* chore(node): drop planning doc from PR per maintainer request

kevincodex1 asked to keep the superpowers planning docs out of the repo. The
Phase 3 plan was scaffolding for this change, not something the project needs
to carry. Removing it leaves only the code and tests in the PR.

* chore: gitignore local planning docs (docs/superpowers/)

* feat(node): replicable_objects filter for replication enforcement

* test(node): pin anonymous-caller contract of withheld_blob_oids

* feat(node): IPFS pinning skips withheld blob OIDs

* feat(node): Pinata pinning skips withheld blob OIDs

* feat(node): enforce visibility on push replication (IPFS/Pinata/gossip/Arweave)

* test(node): announce gate matches anonymous repo readability

* fix(node): run withheld_blob_oids off the async worker on push

The receive-pack replication chokepoint called withheld_blob_oids
directly on the tokio worker, where its blocking git ls-tree walk can
stall the runtime for repos with many refs. Wrap it in spawn_blocking
to match the upload-pack serve path.

---------

Co-authored-by: Kevin Codex <[email protected]>
2026-06-18 22:51:36 +08:00
Kevin CodexandOpenClaude c03c9af8fa feat(sync): auto-register as replica with origin after successful mirror (#56)
The sync worker mirrored repos from peers but never told the origin, so
replica_count (repo_replicas table, GET /api/v1/repos/:owner/:repo/replicas)
stayed at 0 network-wide — the dashboard showed 0 replication even for
repos that were actually mirrored.

After each successful clone/fetch the worker now sends a signed
PUT /api/v1/repos/{owner}/{repo}/replicas to the origin with our
public_url, using the same HTTP-Signature scheme as push notifies.
Best-effort by design: skipped when public_url is unset, failures are
logged without failing the sync, and the PUT is idempotent on the origin
(201 first, 200 after) so re-registering every sync is self-healing.

Nodes need this build before their mirrors are counted: a new node's
repos show replication once its peers upgrade.

Co-authored-by: OpenClaude <[email protected]>
2026-06-18 22:31:29 +08:00
Kevin CodexandOpenClaude 7a8b6c4974 Aws ecr image (#55)
* feat(infra): optional private ECR repo for the node image

create_ecr_repo = true provisions an ECR repository in-account and
switches the instance to pull from it: the instance role gets
AmazonEC2ContainerRegistryReadOnly and user-data configures
amazon-ecr-credential-helper, so first-boot pulls and SSM upgrades
authenticate via the role (no expiring login tokens). README documents
the local build-and-push flow.

Motivation: ghcr.io/gitlawb/node is not anonymously pullable (403),
which left first boot unable to start the stack.

Co-Authored-By: OpenClaude <[email protected]>

* feat(infra): HTTPS via Caddy + production tier (RDS, alarms, S3 state)

- domain_name: Caddy sidecar terminates TLS on 80/443 with auto-renewed
  Let's Encrypt certs (persisted on the data volume); public_url
  defaults to https://<domain>; SG opens 80/443 — matches the Fly nodes
- use_rds: managed Postgres (db.t4g.micro, encrypted, 7-day PITR,
  deletion protection) replaces the in-container postgres; DB security
  group only admits the node instance
- alert_email: SNS email alerts wired to CloudWatch alarms — external
  HTTPS /health via Route53 health check, EC2 status checks, CPU,
  RDS free storage
- user-data grows the filesystem when the EBS volume is enlarged
- terraform state moved to S3 (versioned, KMS-encrypted, public-blocked
  bucket) with lockfile locking (requires terraform >= 1.10)

Co-Authored-By: OpenClaude <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
2026-06-18 22:05:50 +08:00
Kevin CodexandOpenClaude 7a99d0f27c fix(node): make Tigris repo hydration resilient to corrupt archives & failed writes (#54)
Repo pushes 500'd whenever a repo's Tigris archive was unreadable (corrupt
or truncated) or the node was under disk pressure, because hydration was
treated as fatal and a failed extraction could clobber the good local copy.
Three hardening changes:

- tigris: extract archives atomically (unpack into a temp dir, swap into
  place only on success) so a corrupt/truncated archive can never clobber a
  good existing local copy.
- repo_store: acquire_fresh / acquire_write fall back to the valid local
  copy when a Tigris download fails, instead of hard-failing the operation;
  the post-write upload then re-syncs (self-heals) Tigris.
- repo_store: RepoWriteGuard::release(success) uploads to Tigris only when
  the write succeeded, so a half-applied / inconsistent repo is never
  propagated to Tigris or to other nodes.

Co-authored-by: OpenClaude <[email protected]>
2026-06-18 21:38:47 +08:00
Kevin CodexandClaude Fable 5 c7df6cad20 Fix/backport prod fixes (#39)
* fix: backport production fixes from monorepo

Three fixes that landed on the production node but never made it here:

- trigger_sync: 30s timeout with the body read inside it (was 5s
  covering headers only — large unpaginated repo lists from canonical
  nodes and transpacific peers aborted mid-body), plus distinct logging
  for non-2xx vs timeout
- announce: reject self-announcements by public URL or own DID;
  complements the existing boot-time prune_self_peers so self-loop
  rows never enter the peers table in the first place
- list_all_repos_paged: report MAX(updated_at) across each logical
  repo group instead of the canonical row's own value — gossip pushes
  touch only the mirror row, so canonical timestamps go stale

Monorepo refs: 8c2fe1b, c1dc33e, ef21c75.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(p2p): dual-stack QUIC listen + DNS multiaddr dialing for Fly 6PN

Backports the monorepo TCP-era p2p fixes (9f875b6, 267d371) adapted to
the QUIC transport:

- listen on /ip6/:: as well as /ip4/0.0.0.0 — Fly's 6PN inter-app
  network is IPv6-only (<app>.internal resolves to AAAA records), so
  without an IPv6 socket peers on the private network can't reach us
- wrap the QUIC transport in libp2p-dns so bootstrap multiaddrs can use
  /dns6/<app>.internal — dialing peers through the public anycast edge
  breaks the handshake when the proxy closes the connection mid-stream

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(sync): HTTP notify records received_ref_updates like the gossip path

A push propagated via the /api/v1/sync/notify HTTP fallback replicated
the repo but left no trace in received_ref_updates — only the libp2p
gossip path recorded it, so /api/v1/events/ref-updates went blind
whenever the mesh wasn't delivering. Insert the same record from the
notify handler, with optional provenance fields (pusher_did, old_sha,
timestamp, cert_id) that new senders already include.

Monorepo ref: 087eeac.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-06-18 21:36:11 +08:00
Kevin CodexandClaude Fable 5 b61a1bd46f fix(api): blob endpoint returns 400/404 instead of 500 on bad paths (#37)
Unnormalized "../.." paths from crawler-exploded relative links made
`git show` fail and surfaced as 500s. Reject traversal segments with
400 up front, and map paths absent from the tree to 404 so only real
git failures count as server errors.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-06-18 21:35:36 +08:00
beardthelion e365a57b51 feat(node): peer partial-mirrors for repos with private subtrees (#35)
* feat(node): withheld-paths endpoint reporting a caller's denied globs

* feat(gl): gl clone with promisor + sparse-exclude for private subtrees

* test(gl): gl clone repo-argument parsing

* fix(visibility): gate withheld-paths and handle nested/exact globs

Three fixes from the PR #33 review:

- withheld_paths now applies the whole-repo "/" read gate (returns
  repo-not-found when the caller cannot read the root), matching the git read
  endpoints. Without it the endpoint disclosed a private repo's existence and
  path layout to unauthorized callers. The withheld_globs doc already assumed
  this gate existed; now it does.

- A nested allow under a denied parent (e.g. "/secret/public/**" allowed,
  "/secret/**" denied) was over-withheld: the client sparse-excluded the whole
  parent and hid paths the caller may read. The endpoint now also returns a
  "reinclude" list (allowed globs strictly under a denied one) and gl clone
  re-includes them in the sparse spec after the excludes.

- Wildcard-free globs like "/docs/private" match both the exact path and a
  subtree (per glob_matches), but the client only emitted the subtree exclude.
  sparse_patterns now emits both "/docs/private" and "/docs/private/".

Verified the exclude-then-reinclude sparse ordering checks out cleanly with
real git, plus unit tests for reincluded_globs, the nested re-include, the
exact-path exclude, and sparse_patterns.

* fix(gl): reject multi-slash repo input and stop failing open on withheld-path errors

split_once('/') accepted owner/name/extra, smuggling a path segment into
the repo name that then flowed into the API path and clone URL; reject it.

fetch_withheld swallowed every network/auth/5xx/JSON error into an empty
result, dropping to a stock clone that the node refuses once blobs are
withheld. Now only 404/501 (endpoint unsupported) fall back to empty; the
rest propagate so the real cause surfaces.

* test(gl): guard three-level nested visibility in sparse clone

Add a regression test for deny /secret, allow /secret/public, deny
/secret/public/admin and clarify the sparse-checkout comment. git does
not re-traverse an explicitly excluded directory, so emitting all
excludes before re-includes keeps the deepest deny in force; the broader
parent re-include does not resurrect it.

* fix(gl): robust default-branch detection and strict withheld-paths schema

Read the default branch from the local origin/HEAD symref clone already
set, instead of parsing the localized, network-dependent output of
git remote show origin. Deserialize the withheld-paths body into a typed
struct so a missing or mistyped withheld/reinclude field is a hard error
rather than silently becoming an empty list, which would mask a server
regression behind a later clone failure.

* feat(node): classify mirror mode from origin withheld-paths

* feat(node): promisor-aware mirror clone for withheld repos

* feat(node): promisor-aware mirror fetch via origin remote

* feat(node): classify and mirror peers per withheld-paths

* style: cargo fmt

* fix(node): bound sync withheld lookup and rehydrate on promisor->plain

Give the sync worker's HTTP client a 30s timeout so a stalled peer
withheld-paths lookup cannot hang the worker loop. When a repo that was
mirrored as a promisor (mode B) becomes fully public, fetch_repo now
clears remote.origin.promisor and partialclonefilter and refetches, so
the once-withheld blobs are backfilled instead of the mirror staying
permanently partial.
2026-06-18 21:33:31 +08:00
beardthelion 4a59056a90 gl clone: clean partial clone of repos with private subtrees (#33)
* feat(node): withheld-paths endpoint reporting a caller's denied globs

* feat(gl): gl clone with promisor + sparse-exclude for private subtrees

* test(gl): gl clone repo-argument parsing

* fix(visibility): gate withheld-paths and handle nested/exact globs

Three fixes from the PR #33 review:

- withheld_paths now applies the whole-repo "/" read gate (returns
  repo-not-found when the caller cannot read the root), matching the git read
  endpoints. Without it the endpoint disclosed a private repo's existence and
  path layout to unauthorized callers. The withheld_globs doc already assumed
  this gate existed; now it does.

- A nested allow under a denied parent (e.g. "/secret/public/**" allowed,
  "/secret/**" denied) was over-withheld: the client sparse-excluded the whole
  parent and hid paths the caller may read. The endpoint now also returns a
  "reinclude" list (allowed globs strictly under a denied one) and gl clone
  re-includes them in the sparse spec after the excludes.

- Wildcard-free globs like "/docs/private" match both the exact path and a
  subtree (per glob_matches), but the client only emitted the subtree exclude.
  sparse_patterns now emits both "/docs/private" and "/docs/private/".

Verified the exclude-then-reinclude sparse ordering checks out cleanly with
real git, plus unit tests for reincluded_globs, the nested re-include, the
exact-path exclude, and sparse_patterns.

* fix(gl): reject multi-slash repo input and stop failing open on withheld-path errors

split_once('/') accepted owner/name/extra, smuggling a path segment into
the repo name that then flowed into the API path and clone URL; reject it.

fetch_withheld swallowed every network/auth/5xx/JSON error into an empty
result, dropping to a stock clone that the node refuses once blobs are
withheld. Now only 404/501 (endpoint unsupported) fall back to empty; the
rest propagate so the real cause surfaces.

* test(gl): guard three-level nested visibility in sparse clone

Add a regression test for deny /secret, allow /secret/public, deny
/secret/public/admin and clarify the sparse-checkout comment. git does
not re-traverse an explicitly excluded directory, so emitting all
excludes before re-includes keeps the deepest deny in force; the broader
parent re-include does not resurrect it.

* fix(gl): robust default-branch detection and strict withheld-paths schema

Read the default branch from the local origin/HEAD symref clone already
set, instead of parsing the localized, network-dependent output of
git remote show origin. Deserialize the withheld-paths body into a typed
struct so a missing or mistyped withheld/reinclude field is a hard error
rather than silently becoming an empty list, which would mask a server
regression behind a later clone failure.
2026-06-18 21:25:13 +08:00
3a401eb079 feat(git-remote-gitlawb): add --version and --help flags (#30)
git-remote-gitlawb was the only workspace binary without `--version`, so
release smoke tests could not validate the packaged helper directly. This
is called out in docs/MAINTAINER-ROADMAP.md ("Add --version support to
git-remote-gitlawb so release smoke tests can validate it directly") and
docs/OSS-READINESS-AUDIT.md.

- `--version`/`-V` prints `git-remote-gitlawb <version>`, matching the
  `<bin> <version>` format the clap-based `gl` and `gitlawb-node` emit so
  release tooling can treat all three binaries uniformly. The version
  comes from CARGO_PKG_VERSION.
- `--help`/`-h` prints concise usage (URL form, env vars, flags).
- Flags are only recognized as the first argument, so git's
  `<remote-name> <url>` remote-helper invocation path is untouched; a
  remote literally named "--version" is not mistaken for the flag.
- Informational flags are handled before tracing init so output stays
  clean for scripts.

Adds unit tests for argument classification (including the no-args and
misplaced-flag edge cases) and the version/help strings.

Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-18 21:13:01 +08:00
beardthelion 61b383019f feat(node): subtree content withholding (Phase 3) for #18 (#28)
* docs(node): Phase 3 subtree-withholding plan with Task 0 spike findings

* feat(node): resolve withheld blob OIDs for path-scoped visibility

* feat(node): filtered upload-pack serve that omits withheld blobs

* feat(node): serve filtered pack when caller has withheld subtree blobs

* test(node): end-to-end assert served pack omits withheld blob

* docs(node): note why info/refs is not gated on subtree visibility

* fix(node): frame filtered serve as protocol v0 and prove it with a real clone

upload_pack_excluding emitted a v2 packfile section, but info_refs
advertises v0, so real clients negotiated v0 and rejected the response
with 'expected ACK/NAK, got packfile'. Frame the v0 stateless-rpc shape
instead (NAK, then the pack via side-band-64k when offered).

Add an end-to-end test that stands up info_refs + upload_pack_excluding
and runs a real git partial clone, asserting the withheld blob's bytes
never reach the client while its tree entry and SHA stay visible. A stock
full clone cannot consume the pack (it is not closed under reachability,
so fetch fails the connectivity check); a partial clone is required.

* docs(node): correct Phase 3 caveat (full clone refused at fetch, not checkout)

* test(node): prove incremental fetch still withholds; document negotiation choice

Add a real-git test that partial-clones, pushes a new commit server-side,
then fetches: the new object arrives and the withheld blob stays absent.
This pins down that ignoring have/want negotiation (always sending a
self-contained pack of all refs minus withheld, with NAK) is correct for
both clone and fetch; the only cost is a fetch re-sends the full object
set. Refactor the real-git tests onto a shared server harness and document
the negotiation decision in code and in the plan's follow-ups.

* refactor(node): address CodeRabbit review on PR #28

Move the two blocking git shell-outs in the filtered upload-pack path off
the async worker thread, matching the tokio::process / spawn_blocking usage
already in this file: build_filtered_pack (rev-list + pack-objects) and
withheld_blob_oids (per-ref ls-tree) now run inside spawn_blocking so a large
repo cannot stall the tokio runtime. Behavior is unchanged.

Also fix the Task 0 findings block in the Phase 3 plan: it still recorded v2
packfile framing, which is the exact path that failed against a real client
and was corrected to v0. The block now documents the shipped v0 contract.
Drop a stray trailing code fence flagged by markdownlint (MD040).

The speculative ls-tree timeout and the public/no-rules fast-path from the
review are intentionally left out: the timeout guards against adversarial
repos we do not yet host, and the fast-path is a micro-optimization not worth
the extra branch right now.

* chore(node): drop planning doc from PR per maintainer request

kevincodex1 asked to keep the superpowers planning docs out of the repo. The
Phase 3 plan was scaffolding for this change, not something the project needs
to carry. Removing it leaves only the code and tests in the PR.

* chore: gitignore local planning docs (docs/superpowers/)
2026-06-18 21:11:46 +08:00
Kevin CodexandClaude Fable 5 a2217bff27 fix(infra): drop fly idle_timeout 600 -> 120 (#38)
600s idle windows let hung clients pin Fly proxy connection slots up to
hard_limit, which amplified the 2026-06-12 production outage. 120s was
deployed there as the fix; carry it into the reference config.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-06-12 21:55:57 +08:00
beardthelion 6abaf1d7ed feat: path-scoped repository visibility (Phase 1) for #18 (#25)
* feat: path-scoped repository visibility (Phase 1)

Implements per-path read visibility for repositories, addressing #18.
A repo's private status becomes a property of a path subtree rather
than a whole-repo boolean, enforced on the git read path via the node's
existing DID identity and RFC 9421 HTTP signatures.

Phase 1 scope (whole-repo enforcement, data model for the rest):
- visibility_rules data model (path_glob, mode, reader_dids) + DB access
- visibility_check pure decision function (owner-always-allow, most-
  specific rule wins, reader-DID allow-list, is_public fallback)
- enforcement on git_info_refs and git_upload_pack: unauthorized reads
  return 404 byte-identical to a missing repo (no existence leak)
- owner-only management API: PUT/DELETE/GET /api/v1/repos/{o}/{r}/visibility
- gl visibility set/remove/list CLI

mode A (hide) is whole-repo ("/") only by construction: hiding a
subtree's existence would rewrite ancestor tree hashes and diverge
history. Subtrees use mode B (content withheld, SHAs intact); B-subtree
clone filtering is deferred to a later phase and the CLI/API say so.

260 tests (69 node + 191 gl); fmt + clippy clean.

* fix: address review on path-scoped visibility

- migrations: move visibility_rules DDL out of the v1 migration into a
  new v3 (CodeRabbit, critical). Appending to v1 meant nodes that had
  already recorded v1 would skip the statements forever, so the table
  would be missing after upgrade. v3 matches the convention #23 set with
  v2 (agent_profiles) and the documented 'never append to v1' rule.
- api: validate path_glob on set (must start with '/', no trailing '/',
  only a trailing '/**' wildcard, '/' for whole-repo not '/**'). Empty
  or slash-less globs would otherwise silently misconfigure access.
  Added unit tests for the accepted/rejected forms.

281 -> 283 tests pass (88 node + 195 gl); fmt + clippy clean.
2026-06-05 22:42:26 +08:00
Kevin CodexandOpenClaude 9e6e832ec5 Infra reorg (#27)
* refactor: move fly.toml into infra/fly/

Deployment configs now live under infra/, organized per target.
Dockerfiles and docker-compose.yml intentionally stay at the repo
root (shared by CI, build scripts, and the macOS app).

Deploy with: fly deploy -c infra/fly/fly.toml

Co-Authored-By: OpenClaude <[email protected]>

* feat(infra): Terraform for single-EC2 AWS deployment

infra/aws/ provisions a production-leaning single-node deployment:
- t4g.small (AL2023 arm64) running the published ghcr.io/gitlawb/node
  image + postgres:16 via docker compose
- separate encrypted EBS data volume (prevent_destroy) with daily DLM
  snapshots; survives instance replacement
- postgres password via random_password -> SSM SecureString, fetched at
  boot by instance profile (never in user-data); optional secrets
  (operator key, Pinata JWT, S3 secret) follow the same path
- SSM Session Manager access (no SSH by default), IMDSv2 required,
  metrics port closed unless explicitly opened
- SSM command document for image upgrades (user-data runs once)

Verified: terraform fmt/init/validate clean; rendered compose passes
docker compose config; rendered user-data passes bash -n.

Co-Authored-By: OpenClaude <[email protected]>

* fix(infra): address CodeRabbit review feedback

- api_url output now honors the public_url override (local.public_url)
- disable xtrace around SSM secret fetch + .env write so secret values
  never reach the bootstrap log
- add retry/timeout flags to the docker compose download
- validate snapshot_retain_count (1-1000) at plan time
- add language tags to fenced tree blocks (MD040)

Co-Authored-By: OpenClaude <[email protected]>

* fix(infra): address PR #27 review feedback

- security group now derives its VPC from the selected subnet, so
  subnet_id overrides into non-default VPCs work
- s3_access_key_id moves to SSM SecureString (same path as the secret
  key) instead of being embedded in user-data and state
- ignore user_data drift on the instance: it only runs at first boot,
  so re-rendering caused pointless stop/starts; README documents the
  -replace workflow for config changes
- DLM snapshot targeting is stack-specific (Snapshot + Name tags), not
  any Snapshot=true volume in the account
- comment in fly.toml that it targets the shared test instance
- user-data compose up gains --remove-orphans (consistent with the
  upgrade SSM document)
- node service gets an HTTP /health healthcheck in compose

Co-Authored-By: OpenClaude <[email protected]>

* feat(infra): optional customer-managed KMS key for SSM secrets

Add ssm_kms_key_id (default null = AWS-managed aws/ssm key). When set,
all five SecureString parameters are encrypted with the CMK and the
instance role gains kms:Decrypt scoped to that key — without it the
boot-time secret fetch would fail.

Co-Authored-By: OpenClaude <[email protected]>

* fix(infra): scope CMK kms:Decrypt to this stack's SSM parameters

The grant on the customer-managed key was unconditioned, allowing the
instance role to decrypt any ciphertext under the same CMK. Restrict it
with kms:ViaService = ssm.<region>.amazonaws.com and
kms:EncryptionContext:PARAMETER_ARN limited to the stack's parameter
ARNs.

Co-Authored-By: OpenClaude <[email protected]>

---------

Co-authored-by: OpenClaude <[email protected]>
2026-06-05 22:38:53 +08:00
09a339745e feat: agent profiles (display name, bio, avatar, social links) (#23)
* feat: agent profiles — display name, bio, avatar, social links

Adds a profile system so agents and users can set metadata that the
frontend can render on profile pages.

CLI:
  gl profile set --name "Axiom" --bio "AI builder" --twitter AxiomBot
  gl profile show
  gl profile get <did>
  gl profile import profile.json
  gl profile export

API:
  PUT  /api/v1/profile              (authenticated — upsert own profile)
  GET  /api/v1/agents/{did}/profile (public — read any profile)

Database:
  Migration v2 adds `agent_profiles` table with display_name, bio,
  avatar_url, website, socials (JSON), profile_cid, timestamps.

Profile updates merge fields — setting --bio doesn't clear --name.
Bio capped at 280 chars, display_name at 50. Social links stored as
JSON (twitter, github, farcaster, telegram). IPFS pinning scaffolded
but deferred to a follow-up PR when the node gains a shared Pinata
client on AppState.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: apply cargo fmt to pass CI formatting check

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: suppress dead_code warnings for pin_to_ipfs field and set_profile_cid method

Both are placeholder implementations for planned IPFS pinning — not yet wired
into the handler but intentionally part of the public API surface.

Co-Authored-By: Axiom Bot <[email protected]>

* feat: wire up IPFS pinning for agent profiles via Pinata

When pin_to_ipfs: true is sent in the profile update request and the
node has GITLAWB_PINATA_JWT configured, the profile JSON is pinned to
IPFS via Pinata and the CID is stored in the database.

- Removed #[allow(dead_code)] from pin_to_ipfs field and set_profile_cid
- Profiles serialize to JSON and pin via existing pinata::pin_object
- Graceful fallback: if Pinata JWT is empty or pin fails, profile is
  saved normally without CID
- CID returned in response when available

---------

Co-authored-by: Axiom Bot <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-06-04 16:17:22 +08:00
Vasanth TandClaude Opus 4.8 2ce4da9cc5 feat(node): graceful shutdown + Prometheus metrics endpoint (#22)
* feat(db): versioned schema migrations with idempotent backfill

Replace the inline &str SQL array that re-runs on every node startup with a
versioned migration system. Each migration is recorded in a new
schema_migrations table and applied at most once per node, inside a single
transaction.

For nodes upgrading from a pre-migration-versioning build, the canonical
`repos` table is used as a signal to mark v1 as already applied without
re-running its ~140 statements — so existing operators see zero behavior
change on first restart after this commit.

Closes the gap called out in docs/OSS-READINESS-AUDIT.md:78.

- Adds Db::migration_status() returning applied (version, name, applied_at)
- Adds 6 unit tests validating the static migration catalogue
  (versions strictly increasing, names distinct, bodies non-empty, v1
  name locked to initial_schema)
- cargo fmt + clippy -D warnings + cargo test --workspace all clean

* fix(db): make legacy migration backfill safe and lock-guarded

Address two robustness gaps in the v1 migration runner on the live network:

- Remove the "repos table present => v1 complete" backfill short-circuit.
  It assumed every existing node already had the full current schema; a node
  that was behind would be marked v1-applied while still missing newer tables
  or columns. Every v1 statement is idempotent (CREATE TABLE/INDEX IF NOT
  EXISTS, ADD COLUMN IF NOT EXISTS), so legacy installs now simply run v1 once:
  existing objects are no-ops, missing ones get created.

- Guard the whole runner with a Postgres advisory lock so two instances
  against the same database (blue/green or rolling deploy) can't race to apply
  the same migration and trip the schema_migrations primary key. The lock is
  released explicitly, and automatically if the connection drops.

Also document that per-migration transactions preclude CREATE INDEX
CONCURRENTLY in future migrations.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* feat(node): graceful shutdown + Prometheus metrics endpoint

Two operational-safety changes that the live network needs:

1) Graceful shutdown. A process-wide tokio::sync::watch::Sender<bool>
   in AppState is flipped on SIGINT (all platforms) or SIGTERM (Unix).
   Every long-lived loop now selects on the shutdown arm:
     * axum server (with_graceful_shutdown — drains in-flight)
     * libp2p swarm (drops Swarm, closes connections)
     * gossip task (exits between peer announces)
     * sync worker (exits between batches)
     * operator heartbeat loop
     * rate-limit cleanup loop
     * peer-count metrics poller
     * the deferred DID-record publish task

   New GITLAWB_SHUTDOWN_GRACE_SECS (default 30s) controls how long
   axum waits for in-flight requests before returning.

2) Prometheus /metrics endpoint. New metrics module exposes:
     * gitlawb_info{version, did}                 (constant gauge = 1)
     * gitlawb_pushes_total{repo}                 (counter)
     * gitlawb_fetches_total{repo}                (counter)
     * gitlawb_auth_successes_total{route}        (counter, helper)
     * gitlawb_auth_failures_total{route,reason}  (counter, helper)
     * gitlawb_sync_queue_processed_total{status} (counter)
     * gitlawb_webhook_deliveries_total{result}   (counter)
     * gitlawb_pack_size_bytes                    (histogram)
     * gitlawb_peers_connected                    (gauge)

   Opt-in via GITLAWB_METRICS_ADDR (e.g. 127.0.0.1:9091). The endpoint
   is on a separate listener so the public port stays unexposed;
   bind to localhost or a private interface.

3) Metric instrumentation hooks in api/repos.rs (push/fetch),
   webhooks.rs (delivery outcomes), and sync.rs (per-item status).
   The auth helpers are defined but not yet wired — that's a
   focused follow-up.

Tests:
  * 3 new metrics::tests::* — encode returns valid prometheus text,
    record_* helpers are no-ops before init(), encode() errors if
    init() was never called.
  * cargo fmt + clippy -D warnings + cargo test --workspace all clean.

Closes the metrics and graceful-shutdown gaps in
docs/OSS-READINESS-AUDIT.md:131 and docs/MAINTAINER-ROADMAP.md:35.

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-02 08:58:05 +08:00
Vasanth TandClaude Opus 4.8 927e4d0cfb feat(db): versioned schema migrations with idempotent backfill (#21)
* feat(db): versioned schema migrations with idempotent backfill

Replace the inline &str SQL array that re-runs on every node startup with a
versioned migration system. Each migration is recorded in a new
schema_migrations table and applied at most once per node, inside a single
transaction.

For nodes upgrading from a pre-migration-versioning build, the canonical
`repos` table is used as a signal to mark v1 as already applied without
re-running its ~140 statements — so existing operators see zero behavior
change on first restart after this commit.

Closes the gap called out in docs/OSS-READINESS-AUDIT.md:78.

- Adds Db::migration_status() returning applied (version, name, applied_at)
- Adds 6 unit tests validating the static migration catalogue
  (versions strictly increasing, names distinct, bodies non-empty, v1
  name locked to initial_schema)
- cargo fmt + clippy -D warnings + cargo test --workspace all clean

* fix(db): make legacy migration backfill safe and lock-guarded

Address two robustness gaps in the v1 migration runner on the live network:

- Remove the "repos table present => v1 complete" backfill short-circuit.
  It assumed every existing node already had the full current schema; a node
  that was behind would be marked v1-applied while still missing newer tables
  or columns. Every v1 statement is idempotent (CREATE TABLE/INDEX IF NOT
  EXISTS, ADD COLUMN IF NOT EXISTS), so legacy installs now simply run v1 once:
  existing objects are no-ops, missing ones get created.

- Guard the whole runner with a Postgres advisory lock so two instances
  against the same database (blue/green or rolling deploy) can't race to apply
  the same migration and trip the schema_migrations primary key. The lock is
  released explicitly, and automatically if the connection drops.

Also document that per-migration transactions preclude CREATE INDEX
CONCURRENTLY in future migrations.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-01 14:42:50 +08:00
Achille Wasque 924bccd8e5 feat(gitlawb-attest): External Attestation v1 for ref-update certs (#20)
Adds an optional `attestations` field on ref-update certs plus a small
crate to sign and verify them. An envelope with no attestations
serializes to identical bytes as a bare cert, so nothing on the wire
changes for nodes that don't opt in.

Each attestation carries a `type` discriminator (`covenant/exec/v1`,
`slsa/v1.0`, `sigstore/dsse/v1`, ...), an opaque payload, a `cert_hash`
that binds it to one specific cert, a `did:key` signer, and a base64url
ed25519 signature over a domain-separated JCS-encoded input.

## Wire shape

  cert_hash = SHA-256(JCS(cert_body \ {signatures, attestations}))
              encoded as lowercase hex.

  signing_input = b"gitlawb-attest-sig/v1\n"
                  || JCS({type, payload, cert_hash})

  sig = base64url-no-pad(Ed25519(signing_input))

The cert hash is reproducible across implementations because JCS pins
key order, number formatting, and string escaping (RFC 8785). Lowercase
hex is required on the wire: the JCS signing input includes `cert_hash`
verbatim, so accepting mixed case would force every signer and verifier
to normalize before computing the signing bytes — an easy spec rule to
miss across language ports.

The domain tag lives in the byte stream, not in the canonical JSON, so
an attestation cannot be confused with a same-shape JCS document signed
for a different protocol that uses the same Ed25519 key.

Signers are `did:key` over Ed25519 with the multibase base58btc (`z`)
prefix. Non-base58btc encodings of the same public key are rejected on
verify so allowlists and signer-pinning policies stay consistent across
peers.

## Verification

`Registry` maps type to verifier. `Policy::AcceptKnown` (default) lets
unknown types pass without trust so adoption stays incremental.
`Policy::RequireAll` enforces a per-repo allowlist but stays lenient on
unknown types in the batch — anyone can attach an attestation, so
short-circuiting on an unrelated `attacker/spam/v1` would be a DoS
vector. `Policy::RejectUnknown` is strict.

`AttestedRefUpdateCert::verify_attestations(&Registry)` is the one-call
convenience over `registry.verify_all(&env.attestations, env.cert_hash()?)`.

## Canonical test vectors

Ed25519, JCS, and SHA-256 are all deterministic, so a fixed seed plus a
fixed cert body produce exactly the same `cert_hash`, `signer` DID, and
`sig` across implementations. `tests/canonical_vectors.rs` pins these
values so a port to Go, JS, Python, or any other language has a concrete
oracle to check against without reading Rust. If a vector drifts, that
is a wire-protocol break.

## Tests

42 pass: 33 unit + 4 cross-crate integration against the real
`gitlawb_core::cert::RefUpdateCert` + 4 canonical vector pins + 1
rustdoc quick-start doctest.

`cargo fmt --all -- --check` clean.
`cargo clippy --workspace --all-targets -- -D warnings` clean.
`cargo test --workspace` green; the new crate breaks no existing suite.

## Not in this PR

  - Node integration: storage, GraphQL, register flow.
  - UI surfacing.
  - Trust-score weighting policy for typed attestations.
  - Rollout ordering: old nodes silently drop the new `attestations`
    field (`RefUpdateBody` does not set `deny_unknown_fields`); signers
    and verifiers roll out independently.
  - wasm / no_std: not targeted; ed25519-dalek and serde_json are
    std-only.

## Reference implementation

End-to-end demo at open-covenant/covenant@64ec780,
`agent-os/examples/gitlawb-attest-demo/` (`covenant/exec/v1`).

Closes #7.
2026-05-31 10:05:28 +08:00
R Sundareswaran a9f208a378 Feat/ucan chain validation (#19)
* feat(ucan): validate proof chains and attenuation

* feat(auth): validate UCAN chain in middleware

* feat(server): enforce UCAN chain on write routes

* docs(ucan): clarify wildcard attenuation semantics in doc comment

* test(node): add test-only constructors for Db and RepoStore

* fix(auth): harden require_ucan_chain middleware, lower log level, and add tests

* refactor(server): centralize and deduplicate auth middleware layering
2026-05-30 14:44:40 +08:00
R Sundareswaran 491978a809 Test/core crypto primitives (#16)
* test(core/cert): add RefUpdateCert validation coverage

* test(core/cid): add CID parsing and hash tests

* test(core/did): add DID validation and predicate tests

* test(core/identity): add keypair and signed payload tests

* style(did): wrap long assert! messages to satisfy rustfmt

* style(identity): wrap long assert! messages to satisfy rustfmt
2026-05-29 16:42:32 +08:00
Vasanth T 0b056f3ad0 [codex] fix Dependabot Rust alerts (#15)
* fix(deps): resolve dependabot rust alerts

* docs: clarify dependabot validation
2026-05-28 16:44:05 +08:00
Vasanth T d366fc6b97 [codex] chore: add live-safe OSS readiness pass (#14)
* chore: add live-safe OSS readiness pass

* fix(node): verify optional peer signatures during rollout

* docs: rewrite README for node roadmap
2026-05-28 15:34:47 +08:00
Kevin CodexandClaude Opus 4.7 b12c6bc328 feat: per-DID rate limiting on creation endpoints (10/hour) (#13)
Add sliding-window rate limiter applied to resource creation endpoints:
- POST /api/v1/repos (repo creation)
- POST /api/register (agent registration)
- POST /api/v1/repos/:owner/:repo/fork
- POST /api/v1/repos/:owner/:repo/issues
- POST /api/v1/repos/:owner/:repo/pulls

Returns 429 Too Many Requests with Retry-After header when exceeded.
Other write operations (merge, close, comment, star) are not limited
since they're bounded by existing resources.

Includes 5 unit tests covering: within-limit, over-limit, per-key
isolation, window expiry, and cleanup of stale entries.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-27 12:29:02 +08:00
99 changed files with 13484 additions and 1199 deletions

No files matched your search

+38
View File
@@ -0,0 +1,38 @@
# cargo-audit configuration.
#
# cargo-audit reads its config from .cargo/audit.toml (relative to the working
# directory) or ~/.cargo/audit.toml. It does NOT read a repo-root audit.toml,
# and there is no CLI flag to point at a config file (`-f` is the lockfile path).
# This file lives at .cargo/audit.toml for that reason.
#
# The audit job stays hard-fail: only the advisories explicitly listed below are
# suppressed, so any NEW advisory still breaks CI. Each entry below has no
# available upstream fix; everything fixable is fixed in Cargo.lock instead.
#
# Tracking: the hickory ignores are reachable accepted risk. A scheduled re-check
# (.github/workflows/audit-schedule.yml) runs without these ignores and fails if
# the lockfile moves off the pinned vulnerable versions while the ignores remain.
[advisories]
ignore = [
# hickory-proto 0.25.2 (pulled by libp2p-dns 0.44 -> hickory-resolver 0.25).
# REACHABLE: the node wraps its QUIC transport in the system DNS resolver
# (crates/gitlawb-node/src/p2p/mod.rs:238), so a malicious DNS response can
# trigger these DoS paths. No fix is available: the latest libp2p-dns on
# crates.io is 0.44 and pins hickory 0.25; the fix requires hickory >=0.26.1.
# REMOVE both once libp2p-dns ships a release using hickory 0.26 (verified
# 2026-06-21: cargo update -p hickory-proto locks 0 packages; still 0.25.2).
# Tracking: https://github.com/Gitlawb/node/issues/76
"RUSTSEC-2026-0118", # hickory NSEC3 closest-encloser unbounded loop
"RUSTSEC-2026-0119", # hickory O(n^2) name-compression CPU exhaustion
# rsa 0.9.10 (Marvin timing sidechannel). Present in Cargo.lock because sqlx
# resolves sqlx-mysql unconditionally, and sqlx-mysql depends on rsa. cargo
# audit scans the lockfile, so it flags rsa even though it is NOT linked into
# our binary: sqlx is built postgres-only (crates/gitlawb-node/Cargo.toml),
# so the MySQL RSA auth path is never compiled and there is no RSA oracle to
# attack. No upstream fix exists regardless. REMOVE this ignore if sqlx is
# ever built with the `mysql` feature, or any other consumer of rsa enters
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
]
+16 -2
View File
@@ -63,13 +63,27 @@ POSTGRES_PASSWORD=changeme
# ── P2P networking ────────────────────────────────────────────────────────
GITLAWB_P2P_PORT=7546
# Comma-separated multiaddrs of bootstrap peers
# Comma-separated HTTP peer URLs to announce to on startup.
# The binary also ships with bootstrap-peers.json; set
# GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true for isolated dev/test networks.
GITLAWB_BOOTSTRAP_PEERS=
GITLAWB_BOOTSTRAP_DISABLE_SEEDS=false
# Require signatures on peer announce/sync write routes. Keep false during
# rolling upgrades so existing live nodes can still communicate.
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Comma-separated libp2p multiaddrs.
# Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
GITLAWB_P2P_BOOTSTRAP=
# ── Access control ────────────────────────────────────────────────────────
# Set to false to require auth on all read endpoints
# Reserved for private-read mode. Public/private repo read enforcement is not
# wired in the current live release; do not rely on this for private repositories.
GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
GITLAWB_MAX_PACK_BYTES=2147483648
# ── Sync ─────────────────────────────────────────────────────────────────
# Enable automatic background sync from known peers
GITLAWB_AUTO_SYNC=false
+11
View File
@@ -0,0 +1,11 @@
# Ownership for CI, release, and security-sensitive infrastructure.
# A PR that edits these paths requires review from a code owner once
# branch protection's "Require review from Code Owners" is enabled.
/.github/ @beardthelion
/Dockerfile @beardthelion
/Dockerfile.bins @beardthelion
/docker-compose.yml @beardthelion
/release-please-config.json @beardthelion
/.release-please-manifest.json @beardthelion
/SECURITY.md @beardthelion
+66
View File
@@ -0,0 +1,66 @@
name: Bug report
description: Report a defect in a Gitlawb crate or node behavior
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for the report. Please fill in the fields below so we can reproduce the issue.
For security vulnerabilities, do NOT open an issue — follow SECURITY.md instead.
- type: textarea
id: what-happened
attributes:
label: What happened?
description: A clear description of the bug.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: repro
attributes:
label: Steps to reproduce
placeholder: |
1. ...
2. ...
3. ...
validations:
required: true
- type: dropdown
id: crate
attributes:
label: Affected crate
options:
- gitlawb-node
- gl
- git-remote-gitlawb
- gitlawb-core
- not sure / other
validations:
required: true
- type: input
id: version
attributes:
label: Version or commit
description: Output of the binary's `--version`, or the git commit you built.
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
description: OS, Rust version (`rustc --version`), and anything else relevant.
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste logs or errors. This will be rendered as code, no need for backticks.
render: shell
validations:
required: false
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/Gitlawb/node/blob/main/SECURITY.md
about: Please report security issues privately per our security policy, not as a public issue.
- name: Questions & discussion
url: https://github.com/Gitlawb/node/discussions
about: For usage questions and open-ended discussion.
@@ -0,0 +1,34 @@
name: Feature request
description: Suggest an improvement or new capability
labels: ["enhancement"]
body:
- type: textarea
id: problem
attributes:
label: Problem / use case
description: What are you trying to do that's hard or impossible today?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
validations:
required: false
- type: dropdown
id: protocol
attributes:
label: Does this touch the protocol?
description: Identity, signatures, UCAN, ref certs, or P2P wire formats. Protocol changes need an issue discussion before any PR.
options:
- "No"
- "Yes"
- "Not sure"
validations:
required: true
+63
View File
@@ -0,0 +1,63 @@
<!--
Thanks for contributing to Gitlawb. Keep PRs focused on one change.
Protocol-level changes (identity, signatures, wire format) should start
as an issue before code — see CONTRIBUTING.md.
-->
## Summary
<!-- One or two sentences: what changes and why it matters. -->
## Motivation & context
<!-- What problem does this solve? Link prior discussion if any. -->
Closes #
## Kind of change
- [ ] Bug fix
- [ ] Feature
- [ ] Security fix
- [ ] Docs
- [ ] Tests / CI
- [ ] Refactor (no behavior change)
- [ ] Breaking or protocol change (issue required first)
## What changed
<!-- Bullet the concrete changes. Note the crate(s) touched:
gitlawb-core / gitlawb-node / gl / git-remote-gitlawb -->
-
## How a reviewer can verify
<!-- Commands to run, or repro steps for a bug plus proof it's fixed. -->
```sh
```
## Before you request review
- [ ] Scope is one logical change; no unrelated churn
- [ ] `cargo test --workspace` passes locally
- [ ] New behavior is covered by tests (required for fixes)
- [ ] `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` are clean
- [ ] Commit titles use Conventional Commits (`feat(...)`, `fix(...)`, `docs(...)`)
- [ ] Docs / `.env.example` updated if behavior or config changed (or N/A)
- [ ] Checked existing PRs so this isn't a duplicate
## Protocol & signing impact
<!-- Delete this block if your change doesn't touch the protocol. -->
- [ ] Touches DID / `did:key`, Ed25519 / RFC 9421 signatures, UCAN, ref certs, or P2P wire formats
- [ ] Discussed in an issue before implementation
- [ ] Backward-compatible with existing nodes and previously signed history
## Notes for reviewers
<!-- Anything out of scope, follow-ups, or known limitations. Optional. -->
+77
View File
@@ -0,0 +1,77 @@
name: Scheduled Audit
# Self-expiring suppression check (companion to the PR-blocking audit gate in
# pr-checks.yml). Runs weekly. It re-runs cargo audit WITHOUT the .cargo/audit.toml
# ignore list to surface the full advisory set for visibility, and it hard-fails
# if the lockfile has moved off the pinned vulnerable versions while the ignores
# are still present, i.e. the upstream fix landed but the ignore was not dropped.
on:
schedule:
- cron: "0 6 * * 1" # Mondays 06:00 UTC
workflow_dispatch:
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
jobs:
scheduled-audit:
name: scheduled audit (no ignores)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit-schedule
- name: Install cargo-audit
run: cargo install --locked cargo-audit
# Full advisory report with NO suppressions. Run from a scratch dir that
# has no .cargo/audit.toml so the ignore list does not apply; never fail
# the build on this step, it is visibility only.
- name: Full audit report (ignores not applied)
run: |
mkdir -p "$RUNNER_TEMP/full-audit"
cp Cargo.lock "$RUNNER_TEMP/full-audit/Cargo.lock"
cd "$RUNNER_TEMP/full-audit"
{
echo '### Full cargo audit (no ignores applied)'
echo '```'
cargo audit -f Cargo.lock || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
# Drift guard: the hickory ignores are only valid while hickory-proto is
# pinned at 0.25.x. If the lockfile has moved past that, the upstream fix
# is in and the ignores in .cargo/audit.toml must be removed. Fail loudly.
- name: Fail if hickory moved past 0.25.x while ignores remain
run: |
set -euo pipefail
version="$(grep -A1 'name = "hickory-proto"' Cargo.lock | grep '^version' | head -1 | cut -d'"' -f2)"
echo "hickory-proto in Cargo.lock: ${version:-not present}"
ignores_present=false
if grep -q 'RUSTSEC-2026-011' .cargo/audit.toml; then
ignores_present=true
fi
# Drift = the lockfile no longer matches what the ignores assume:
# hickory moved off 0.25.x (fix shipped) OR was removed entirely
# (dead ignore entries left behind). Both must fail.
if [ "$ignores_present" = true ] && { [ -z "${version}" ] || [[ "${version}" != 0.25.* ]]; }; then
echo "::error::hickory-proto is ${version:-absent from Cargo.lock} but the RUSTSEC-2026-0118/0119 ignores are still in .cargo/audit.toml. The upstream fix appears to be available or the dependency is gone; remove the ignores."
exit 1
fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version."
+137 -3
View File
@@ -9,18 +9,25 @@ on:
permissions:
contents: read
# Cancel superseded runs on the same PR/branch; never cancel runs on main.
concurrency:
group: pr-checks-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
jobs:
fmt-clippy-test:
name: fmt + clippy + test
fmt-clippy:
name: fmt + clippy
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
@@ -37,5 +44,132 @@ jobs:
- name: cargo clippy
run: cargo clippy --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
toolchain: [stable, beta]
# Beta is informational: an upstream beta regression should warn, not block merges.
continue-on-error: ${{ matrix.toolchain == 'beta' }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: ${{ matrix.toolchain }}
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: ${{ matrix.toolchain }}
- name: cargo test
run: cargo test --workspace
build-release:
name: build --release
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release
- name: cargo build --release
run: cargo build --release --workspace
audit:
name: cargo audit
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit
- name: Install cargo-audit
run: cargo install --locked cargo-audit
# Hard-fail gate. Suppressions live in .cargo/audit.toml (read automatically
# from the repo root), each with no available upstream fix. A green check
# here means "no NEW advisories", not "advisory-clean": surface the active
# ignore list so a reviewer sees exactly what is being accepted.
- name: Show active audit ignores
run: |
{
echo '### cargo audit: accepted advisories (.cargo/audit.toml)'
echo '```toml'
sed -n '/^\[advisories\]/,$p' .cargo/audit.toml
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- name: cargo audit
run: cargo audit
msrv:
name: MSRV (Rust 1.91)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust 1.91
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # 1.91
with:
toolchain: "1.91"
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: msrv
- name: cargo check on MSRV
run: cargo check --workspace --all-targets
docker-build:
name: Docker build smoke test
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Build image
run: docker build -t gitlawb-node:ci-test .
- name: Smoke test --version
run: docker run --rm gitlawb-node:ci-test --version
+120
View File
@@ -0,0 +1,120 @@
name: PR Triage
on:
pull_request:
types: [opened, edited, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: write
concurrency:
group: pr-triage-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
triage:
name: Quality-signal triage
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Label and guide
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const pr = context.payload.pull_request;
const { owner, repo } = context.repo;
const prNumber = pr.number;
// Trusted authors bypass triage entirely.
const trusted = ["OWNER", "MEMBER", "COLLABORATOR"];
if (trusted.includes(pr.author_association)) {
core.info(`Author association ${pr.author_association} is trusted; skipping triage.`);
return;
}
// Labels this workflow owns. Anything here may be added or removed each run.
const MANAGED = ["needs-issue", "needs-description", "needs-tests", "workflow-change"];
const want = new Set();
const body = pr.body || "";
// Linked issue: look for "closes/fixes/resolves #N" or any bare #N reference.
const hasLinkedIssue =
/\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\b[^\n]*#\d+/i.test(body) ||
/#\d+/.test(body);
if (!hasLinkedIssue) want.add("needs-issue");
// Description substance: strip HTML comments and markdown headings, then measure.
const stripped = body
.replace(/<!--[\s\S]*?-->/g, "")
.replace(/^#+.*$/gm, "")
.trim();
if (stripped.length < 50) want.add("needs-description");
// File-based signals.
const files = await github.paginate(github.rest.pulls.listFiles, {
owner, repo, pull_number: prNumber, per_page: 100,
});
const names = files.map(f => f.filename);
const isFork = pr.head.repo && pr.head.repo.full_name !== pr.base.repo.full_name;
const touchesWorkflows = names.some(n => n.startsWith(".github/workflows/"));
if (isFork && touchesWorkflows) want.add("workflow-change");
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or — the common Rust
// pattern — an inline #[test] / #[cfg(test)] block added inside the source file.
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
// Reconcile labels: add wanted managed labels, remove managed labels no longer wanted.
const current = (await github.rest.issues.listLabelsOnIssue({
owner, repo, issue_number: prNumber,
})).data.map(l => l.name);
const toAdd = [...want].filter(l => !current.includes(l));
const toRemove = MANAGED.filter(l => current.includes(l) && !want.has(l));
if (toAdd.length) {
await github.rest.issues.addLabels({ owner, repo, issue_number: prNumber, labels: toAdd });
}
for (const name of toRemove) {
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNumber, name })
.catch(() => {}); // ignore if already gone
}
// Guidance comment: only for advisory signals, and only once.
const advisory = ["needs-issue", "needs-description", "needs-tests"].filter(l => want.has(l));
if (!advisory.length) return;
const MARKER = "<!-- gitlawb-triage -->";
const comments = await github.paginate(github.rest.issues.listComments, {
owner, repo, issue_number: prNumber, per_page: 100,
});
if (comments.some(c => c.body && c.body.includes(MARKER))) {
core.info("Guidance comment already present; not reposting.");
return;
}
const reasons = {
"needs-issue": "Link the issue this addresses (`Closes #123`). For protocol changes, open an issue first.",
"needs-description": "Add a short summary of what changes and why — the template's Summary and Motivation sections.",
"needs-tests": "This changes Rust source but no tests changed. Tests are required for fixes and strongly encouraged for features.",
};
const lines = advisory.map(l => `- ${reasons[l]}`).join("\n");
const comment = [
MARKER,
"Thanks for the contribution. A couple of things will help us review this faster:",
"",
lines,
"",
"See [CONTRIBUTING.md](https://github.com/Gitlawb/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
].join("\n");
await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: comment });
+21
View File
@@ -146,6 +146,27 @@ jobs:
- name: Build
run: cargo build --release --target ${{ matrix.target }} -p gl -p git-remote-gitlawb -p gitlawb-node
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Skipped for cross-compiled targets that can't run
# on the host runner (aarch64 linux on an x86_64 runner).
- name: Smoke test binaries
if: ${{ matrix.target != 'aarch64-unknown-linux-musl' }}
shell: bash
run: |
set -euo pipefail
VERSION="${{ needs.release-please.outputs.version }}"
BIN_DIR="target/${{ matrix.target }}/release"
for bin in gl git-remote-gitlawb gitlawb-node; do
echo "== $bin --version =="
out="$("$BIN_DIR/$bin" --version)"
echo "$out"
# Each binary prints "<name> <version>"; assert the released version is present.
grep -qF "$VERSION" <<<"$out" || {
echo "::error::$bin --version did not report expected version $VERSION (got: $out)"
exit 1
}
done
- name: Package
id: pkg
shell: bash
+39
View File
@@ -0,0 +1,39 @@
name: Stale PRs
on:
schedule:
- cron: "30 1 * * *" # daily at 01:30 UTC
workflow_dispatch:
permissions:
contents: read
pull-requests: write
issues: write
jobs:
stale:
name: Flag and close ignored PRs
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9
with:
days-before-stale: 14
days-before-close: 7
# Act on PRs only; never on issues.
days-before-issue-stale: -1
days-before-issue-close: -1
# Only PRs the triage step flagged as needing an author response.
only-pr-labels: "needs-issue,needs-description"
stale-pr-label: "stale"
exempt-pr-labels: "workflow-change,security,pinned"
remove-stale-when-updated: true
stale-pr-message: >
This PR has been inactive for 14 days and is still missing a linked issue
or a description. It will be closed in 7 days if there's no update. Push a
change or leave a comment to keep it open — no hard feelings, you can reopen
anytime.
close-pr-message: >
Closing due to inactivity. Reopen whenever you're ready to update it — see
CONTRIBUTING.md for what helps us review quickly.
ascending: true
+19 -1
View File
@@ -55,10 +55,28 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
## What gets merged, what gets closed
We welcome contributions from humans and agents alike. To keep review sustainable, PRs are
expected to clear a basic quality bar:
- **Link an issue.** Bug fixes and features should reference an issue (`Closes #123`). For
protocol-level changes (identity, signatures, UCAN, ref certs, wire formats), open the
issue *before* writing code.
- **One change per PR.** Unrelated churn slows review and gets sent back.
- **Tests and a green pipeline.** New behavior needs tests; `cargo fmt`, `cargo clippy`,
and the full CI suite must pass.
- **A real description.** Say what changes and why. "Update code" is not a description.
A triage bot labels PRs that are missing these and leaves a short note. Nothing is closed
automatically while you're engaging. A flagged PR that goes 14 days with no linked issue or
description gets a stale warning, and is closed 7 days later if still untouched. Closed PRs can
be reopened at any time once updated.
## Development environment
**Requirements:**
- Rust stable (≥ 1.85) — install via [rustup](https://rustup.rs)
- Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs)
- PostgreSQL — required for the node. Use the bundled `docker-compose.yml` for local dev.
- Docker (optional, for full-stack local testing)
Generated
+186 -562
View File
File diff suppressed because it is too large. Load diff
+4 -1
View File
@@ -5,11 +5,13 @@ members = [
"crates/gitlawb-node",
"crates/gl",
"crates/git-remote-gitlawb",
"crates/gitlawb-attest",
]
[workspace.package]
version = "0.3.9"
edition = "2021"
rust-version = "1.91"
license = "MIT OR Apache-2.0"
authors = ["gitlawb contributors"]
repository = "https://github.com/gitlawb/node"
@@ -20,12 +22,13 @@ tokio = { version = "1", features = ["full"] }
# serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
serde_jcs = "0.2"
# errors
thiserror = "2"
anyhow = "1"
# crypto
ed25519-dalek = { version = "2", features = ["rand_core", "pkcs8", "pem"] }
rand = "0.8"
rand = "0.8.6"
sha2 = "0.10"
# encoding
hex = "0.4.3"
+1 -1
View File
@@ -65,7 +65,7 @@ ENV GITLAWB_REPOS_DIR=/data/repos \
GITLAWB_PORT=7545 \
GITLAWB_P2P_PORT=7546
EXPOSE 7545 7546
EXPOSE 7545 7546/udp
VOLUME ["/data"]
+372 -79
View File
@@ -1,49 +1,273 @@
# gitlawb node
# Gitlawb Node
**Decentralized git for AI agents and developers.** Run your own node, host repos under your own DID, and connect to the gitlawb network.
**Decentralized git infrastructure for developers, AI agents, and app delivery.**
This repository contains the open-source node software:
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
- `gitlawb-node` — Axum HTTP server, git smart-HTTP, libp2p gossip, optional S3-compatible storage
- `gl` — CLI for identity, repos, PRs, MCP server
- `git-remote-gitlawb` — `git remote helper` for `gitlawb://` URLs
- `gitlawb-core` — shared crypto primitives (DID, CID, HTTP signatures, UCAN)
Gitlawb is not trying to be only “another git host.” The long-term direction is:
```txt
Decentralized GitHub
+ signed agent-native workflows
+ resilient repo replication
+ CDN-style app/code delivery
```
The mission is simple: once code is pushed to the network, it should not disappear because one server went down.
---
## Quickstart
## What is in this repository?
The fastest way to run a node is via Docker. You'll get a node + Postgres locally with one command.
This is a Rust workspace with four crates:
| Crate | Purpose |
|---|---|
| `gitlawb-node` | The node daemon: Axum HTTP server, git smart-HTTP, Postgres metadata, libp2p gossip, optional S3/Tigris/IPFS/Arweave/Base PoS hooks. |
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `gitlawb-core` | Shared primitives: Ed25519 identities, `did:key`, CIDs, RFC 9421 HTTP signatures, certificates, and UCAN tokens. |
---
## Why Gitlawb Nodes?
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
---
## Current status
Gitlawb Node is live early infrastructure. It is useful today, but some security and reliability features are intentionally staged for compatibility with existing nodes.
Good today:
- Local or Docker node startup.
- Postgres-backed repo metadata.
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- DID identities.
- `gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
- Optional Tigris/S3 storage.
- Optional IPFS/Pinata and Arweave/Irys hooks.
- Optional Base node-operator staking/heartbeat hooks.
Known limitations:
- Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
- Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
See:
- [`SECURITY.md`](SECURITY.md)
- [`docs/OSS-READINESS-AUDIT.md`](docs/OSS-READINESS-AUDIT.md)
- [`docs/MAINTAINER-ROADMAP.md`](docs/MAINTAINER-ROADMAP.md)
---
## Quickstart: run a local node
The fastest path is Docker Compose. It starts a node and Postgres.
```bash
git clone https://github.com/gitlawb/node.git
git clone https://github.com/Gitlawb/node.git
cd node
cp .env.example .env
docker compose up -d
```
Your node is now serving:
Your local node will serve:
- HTTP + git smart-HTTP on `:7545`
- libp2p gossip on `:7546`
| Service | Default |
|---|---|
| HTTP API + git smart-HTTP | `http://localhost:7545` |
| libp2p QUIC/UDP | `7546` |
| Postgres | compose-managed |
Verify:
```bash
curl http://localhost:7545/health
curl http://localhost:7545/api/v1/stats
```
Expected health response:
```json
{ "status": "ok" }
```
Stop it:
```bash
docker compose down
```
---
## How it works
## Install the CLI
- **Identity** — every actor (human or agent) is an Ed25519 keypair → `did:key:z6Mk...`. No accounts, no passwords.
- **Auth** — every write is signed via HTTP Signatures (RFC 9421).
- **Delegation** — UCAN capability tokens.
- **Storage** — local disk + optional S3-compatible bucket (Tigris, MinIO, AWS S3).
- **Networking** — libp2p Gossipsub for ref-update events, Kademlia DHT for node discovery.
Use the installer once releases are available for your platform:
The node is **stateless beyond Postgres + the repos directory.** You can scale horizontally by pointing multiple nodes at a shared S3 bucket and database.
```bash
curl -fsSL https://gitlawb.com/install.sh | sh
```
Or build from source:
```bash
cargo build --release -p gl -p git-remote-gitlawb -p gitlawb-node
```
Put these binaries on your `PATH`:
```txt
target/release/gl
target/release/git-remote-gitlawb
target/release/gitlawb-node
```
Check your setup:
```bash
gl doctor
```
---
## First repo flow
Create an identity:
```bash
gl identity new
gl identity show
```
Register against your local node:
```bash
gl register --node http://localhost:7545
```
Create a repo:
```bash
gl repo create my-repo --description "My first Gitlawb repo" --node http://localhost:7545
```
Use the git remote helper:
```bash
export GITLAWB_NODE=http://localhost:7545
git clone gitlawb://did:key:z6Mk.../my-repo
```
For public-network use, make sure `GITLAWB_NODE` points to the node you want. The helper defaults to localhost for local development.
---
## Architecture
```txt
┌──────────────────────────┐
│ gl CLI / git / AI agents │
└────────────┬─────────────┘
│ signed HTTP writes / git smart-HTTP
↓
┌──────────────────────────┐
│ gitlawb-node │
│ Axum API + git routes │
└────────────┬─────────────┘
│
┌────────┴────────┐
↓ ↓
Postgres Bare git repos
metadata local disk / optional S3
│ │
└────────┬────────┘
↓
libp2p peers
gossip + discovery + sync
↓
optional IPFS / Arweave / Base PoS
```
### Core concepts
| Concept | Meaning |
|---|---|
| DID | A user, agent, or node identity derived from an Ed25519 public key. |
| HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. |
| Ref certificate | Signed record of a ref update. Useful for audit and replication. |
| UCAN | Delegation token for future capability-based workflows. |
| Peer announce | Node-to-node HTTP announcement of DID + public URL. |
| Gossipsub | libp2p topic for ref-update events. |
| Smart HTTP | Standard git protocol over HTTP for clone/fetch/push. |
---
## API surface
The node exposes both git smart-HTTP routes and JSON APIs.
Common public read routes:
```txt
GET /health
GET /
GET /api/v1/stats
GET /api/v1/contracts
GET /api/v1/repos
GET /api/v1/repos/{owner}/{repo}
GET /api/v1/repos/{owner}/{repo}/tree
GET /api/v1/repos/{owner}/{repo}/blob/{path}
GET /api/v1/repos/{owner}/{repo}/issues
GET /api/v1/repos/{owner}/{repo}/pulls
GET /api/v1/peers
GET /{owner}/{repo}/info/refs
POST /{owner}/{repo}/git-upload-pack
```
Signed write routes include:
```txt
POST /api/v1/repos
POST /api/register
POST /api/v1/repos/{owner}/{repo}/fork
POST /api/v1/repos/{owner}/{repo}/issues
POST /api/v1/repos/{owner}/{repo}/pulls
POST /api/v1/repos/{owner}/{repo}/pulls/{number}/merge
POST /api/v1/repos/{owner}/{repo}/hooks
POST /api/v1/bounties/{id}/...
POST /{owner}/{repo}/git-receive-pack
```
Peer write routes support staged rollout:
```txt
POST /api/v1/peers/announce
POST /api/v1/sync/notify
POST /api/v1/sync/trigger
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
```
---
@@ -51,116 +275,185 @@ The node is **stateless beyond Postgres + the repos directory.** You can scale h
All configuration is via environment variables. See [`.env.example`](.env.example) for the full reference.
The minimum required:
Minimum required for a persistent node:
- `DATABASE_URL` — Postgres connection string
```env
DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
```
Everything else has sensible defaults. Optional features:
Important node settings:
- `GITLAWB_TIGRIS_BUCKET` (+ AWS credentials) — S3-compatible shared storage
- `GITLAWB_PINATA_JWT` — IPFS warm-storage pinning
- `GITLAWB_IRYS_URL` — Arweave permanent anchoring
- `GITLAWB_BOOTSTRAP_PEERS` — comma-separated peer URLs to announce to on startup
- `GITLAWB_P2P_BOOTSTRAP` — comma-separated libp2p multiaddrs
| Variable | Purpose |
|---|---|
| `GITLAWB_HOST` / `GITLAWB_PORT` | HTTP bind address and port. |
| `GITLAWB_REPOS_DIR` | Local bare repo storage directory. |
| `GITLAWB_PUBLIC_URL` | Public HTTP URL announced to peers. |
| `GITLAWB_P2P_PORT` | libp2p QUIC/UDP port. Use `0` to disable. |
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
### On-chain Proof-of-Stake (advanced, optional)
The node supports an optional on-chain PoS layer where operators register their DID, stake $GITLAWB on Base L2, and receive a share of protocol fees. Smart contracts are in a separate repository at [github.com/gitlawb/contracts](https://github.com/gitlawb/contracts).
PoS is **disabled by default** and currently paused pending external audit. To enable, set `GITLAWB_CONTRACT_NODE_STAKING` and `GITLAWB_OPERATOR_PRIVATE_KEY`. See [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md) for details.
Production note: change the default Postgres password before exposing a node publicly.
---
## Build from source
## Optional node staking
Requires Rust 1.85+ and a Postgres instance.
Gitlawb Node includes optional Base L2 node-operator hooks. Operators can register a node DID, stake `$GITLAWB`, and post heartbeats.
PoS is disabled unless these are configured:
```env
GITLAWB_CONTRACT_NODE_STAKING=0x...
GITLAWB_OPERATOR_PRIVATE_KEY=0x...
GITLAWB_CHAIN_RPC_URL=https://mainnet.base.org
```
Recommended for operators:
```env
GITLAWB_OPERATOR_STRICT_MODE=true
GITLAWB_HEARTBEAT_INTERVAL_HOURS=20
```
Read:
- [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md)
- [`docs/ECONOMICS.md`](docs/ECONOMICS.md)
Use a dedicated low-balance operator wallet. Do not use a treasury wallet as the heartbeat key.
---
## Building from source
Requires Rust 1.91+.
```bash
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb
```
Binaries are placed in `target/release/`. The `gl` and `git-remote-gitlawb` binaries should be on your `$PATH` for client use.
---
## CLI usage
Run tests:
```bash
gl identity new
gl register --node http://localhost:7545
gl repo create my-repo --description "..."
git clone gitlawb://did:key:z6Mk.../my-repo
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
```
Full CLI reference: `gl --help`.
Run the node from source:
---
## Architecture
```
crates/
├── gitlawb-core/ crypto primitives (DID, CID, HTTP sigs, UCAN)
├── gitlawb-node/ node daemon (Axum + git smart-HTTP)
├── gl/ CLI
└── git-remote-gitlawb/ git remote helper
```bash
DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb \
cargo run -p gitlawb-node --release
```
---
## macOS Menu Bar App
## macOS menu bar app
A native Swift/AppKit menu bar app that manages the Docker Compose stack (node + Postgres) without touching the terminal.
A native Swift/AppKit menu bar app is included for managing a local Docker Compose stack without living in the terminal.
**Requirements:** macOS 26+, Xcode Command Line Tools (`xcode-select --install`), and a Docker runtime (Docker Desktop, OrbStack, or Colima).
Requirements:
### Build
- macOS 26+
- Xcode Command Line Tools
- Docker Desktop, OrbStack, or Colima
Build:
```bash
./scripts/build-macos-app.sh
```
The resulting `Gitlawb Node.app` and `.dmg` are placed in `dist/`.
Output:
To codesign for distribution:
```bash
./scripts/build-macos-app.sh --sign "Developer ID Application: ..."
```txt
dist/Gitlawb Node.app
dist/Gitlawb Node.dmg
```
### Features
Features:
- Start/Stop the node from the menu bar
- Status indicator (green = running, yellow = starting, red = stopped)
- Settings GUI (ports, Postgres password, operator config)
- Auto-start on login
- Detects Docker Desktop, OrbStack, and Colima automatically
- Start/stop local node stack.
- Status indicator.
- Settings GUI.
- Auto-start on login.
- Docker runtime detection.
### Running an unsigned build
If you built the app locally without a Developer ID, macOS Gatekeeper will block it. To allow it:
Unsigned local build:
```bash
xattr -cr "dist/Gitlawb Node.app"
```
Then open the app normally. Alternatively, go to **System Settings → Privacy & Security** and click **Open Anyway** after the first blocked launch attempt.
---
## Roadmap
The current maintainer focus is live-network stability first.
Short-term priorities:
1. Keep CI green: fmt, clippy, tests, release build.
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Implement repo write authorization: owner checks, protected branches, and UCAN capability checks.
6. Implement private-read enforcement or remove private repo affordances until it exists.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
Product direction:
1. Reliable repo replication.
2. Health-aware peer syncing.
3. CDN-style clone/fetch routing to healthy replicas.
4. App asset/build delivery from nodes.
5. Operator dashboard and desktop UX.
Read the maintainer roadmap:
```txt
docs/MAINTAINER-ROADMAP.md
```
---
## Contributing
See [`CONTRIBUTING.md`](CONTRIBUTING.md). Security issues: see [`SECURITY.md`](SECURITY.md).
Start here:
- [`CONTRIBUTING.md`](CONTRIBUTING.md)
- [`docs/MAINTAINER-ROADMAP.md`](docs/MAINTAINER-ROADMAP.md)
- [`docs/OSS-READINESS-AUDIT.md`](docs/OSS-READINESS-AUDIT.md)
- [`SECURITY.md`](SECURITY.md)
Good first contribution areas:
- docs and install polish
- Docker smoke tests
- CLI error messages
- `gl doctor` checks
- operator dashboard UX
- test coverage for peer sync and signed writes
Security issues should follow [`SECURITY.md`](SECURITY.md).
---
## License
Licensed under either of
Licensed under either of:
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project shall be dual licensed as above, without any additional terms or conditions.
+22 -11
View File
@@ -16,7 +16,7 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
## Current Security Architecture
### What is production-ready (v0.1)
### Current live security controls
**Ed25519 identity and HTTP Signatures**
- Every write operation is signed with RFC 9421 HTTP Signatures
@@ -42,35 +42,46 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
## Dependency Vulnerability Status
| Crate | Advisory | Severity | Status |
|-------|----------|----------|--------|
| `rsa 0.9.x` | RUSTSEC-2023-0071 Marvin Attack | Medium | Unexploitable — bundled via `sqlx-mysql` internals; gitlawb uses PostgreSQL only, MySQL auth code is never executed. No upstream fix available. |
| `lru 0.12.x` | RUSTSEC-2026-0002 unsound IterMut | Warning | In `libp2p-swarm`; the specific unsafe code path is internal to libp2p and not reachable through gitlawb's usage. No upstream fix yet. |
| Area | Status |
|------|--------|
| Dependabot alerts | Current open Rust alerts were addressed by updating vulnerable dependencies, removing libp2p mDNS, and moving P2P transport from TCP/Yamux to QUIC/UDP. |
---
## Known Limitations (Planned for v0.2)
These are **documented, accepted limitations** for the current testnet release. They will be addressed before mainnet.
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
### UCAN chain validation
- The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Nodes can set `GITLAWB_PUBLIC_READ=false` to restrict access. The v0.1 trust score system provides soft rate limiting.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### UCAN revocation
- Issued UCAN tokens cannot be revoked before expiry.
- **Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Nodes will reject the old DID on sight if manually blocklisted.
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth via the `git-remote-gitlawb` remote helper. Direct HTTP push without the helper is currently unauthenticated on v0.1 nodes.
- **Impact:** A node operator accepting direct HTTP git pushes (not via `gitlawb://`) cannot verify the pusher's identity.
- **Mitigation:** Use `gitlawb://` remote URLs. Do not expose `POST /*/git-receive-pack` to untrusted networks.
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
- **Impact:** Direct HTTP pushes without RFC 9421 headers are rejected; users need `gitlawb://` remotes or equivalent signed clients.
- **Mitigation:** Use `gitlawb://` remote URLs and keep `git-remote-gitlawb` on the user's `PATH`.
- **Fix target:** v0.2
### Private repository reads
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
- **Impact:** Do not store private repositories or secrets on public nodes.
- **Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
- **Fix target:** v0.2
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
- **Impact:** Unsigned peer writes are still accepted by default so existing live nodes can keep communicating during rolling upgrades.
- **Mitigation:** After all active peers run signed-node builds, operators can set `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true`.
- **Fix target:** staged rollout
---
## Supported Versions
+1
View File
@@ -3,6 +3,7 @@ name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
+156 -1
View File
@@ -22,15 +22,33 @@ use gitlawb_core::identity::Keypair;
use std::io::{self, BufRead, Read, Write};
fn main() -> Result<()> {
let args: Vec<String> = std::env::args().collect();
// Handle informational flags before anything else. Git always invokes a
// remote helper as `git-remote-gitlawb <remote-name> <url>`, so these flags
// only ever appear on direct user or release-smoke-test invocations. Print
// to stdout and exit before tracing init so the output stays clean.
match classify_args(&args) {
Invocation::Version => {
println!("{}", version_line());
return Ok(());
}
Invocation::Help => {
print!("{}", help_text());
return Ok(());
}
Invocation::Helper => {}
}
// All logging goes to stderr so it doesn't corrupt the git protocol on stdout
tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.with_env_filter(std::env::var("GITLAWB_LOG").unwrap_or_else(|_| "warn".to_string()))
.init();
let args: Vec<String> = std::env::args().collect();
if args.len() < 3 {
eprintln!("usage: git-remote-gitlawb <remote-name> <url>");
eprintln!("try 'git-remote-gitlawb --help' for more information");
std::process::exit(1);
}
@@ -51,6 +69,62 @@ fn main() -> Result<()> {
run_helper(&repo_base, keypair.as_ref())
}
// ── CLI argument handling ──────────────────────────────────────────────────────
/// How the binary was invoked, derived from its CLI arguments.
#[derive(Debug, PartialEq, Eq)]
enum Invocation {
/// `--version` / `-V`: print the version line and exit.
Version,
/// `--help` / `-h`: print usage and exit.
Help,
/// Normal git remote-helper invocation: `<remote-name> <url>`.
Helper,
}
/// Classify the process arguments.
///
/// Git always calls a remote helper as `git-remote-gitlawb <remote-name> <url>`,
/// so the informational flags are only recognized as the first argument; this
/// keeps the remote-helper protocol path untouched for git's own invocations.
fn classify_args(args: &[String]) -> Invocation {
match args.get(1).map(String::as_str) {
Some("--version") | Some("-V") => Invocation::Version,
Some("--help") | Some("-h") => Invocation::Help,
_ => Invocation::Helper,
}
}
/// The version line, matching the `<bin> <version>` format that the clap-based
/// `gl` and `gitlawb-node` binaries emit, so release smoke tests can treat all
/// three binaries uniformly.
fn version_line() -> String {
format!("{} {}", env!("CARGO_PKG_NAME"), env!("CARGO_PKG_VERSION"))
}
/// Usage text for `--help`.
fn help_text() -> String {
format!(
"{}\n\
\n\
Git remote helper for gitlawb:// URLs. Git invokes this automatically\n\
when it encounters a gitlawb:// remote; you normally do not run it directly.\n\
\n\
USAGE:\n\
\x20 git clone gitlawb://did:key:z6Mk.../<repo>\n\
\n\
ENVIRONMENT:\n\
\x20 GITLAWB_NODE Node base URL (default: http://127.0.0.1:7545)\n\
\x20 GITLAWB_KEY Identity PEM path for signed pushes (default: ~/.gitlawb/identity.pem)\n\
\x20 GITLAWB_LOG Log filter (default: warn)\n\
\n\
FLAGS:\n\
\x20 -V, --version Print version and exit\n\
\x20 -h, --help Print this help and exit\n",
version_line()
)
}
// ── Remote helper protocol loop ───────────────────────────────────────────────
fn run_helper(repo_base: &str, keypair: Option<&Keypair>) -> Result<()> {
@@ -445,4 +519,85 @@ mod tests {
"/z6Mk/myrepo/git-receive-pack"
);
}
fn args(items: &[&str]) -> Vec<String> {
items.iter().map(|s| s.to_string()).collect()
}
#[test]
fn classify_version_flags() {
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "--version"])),
Invocation::Version
);
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "-V"])),
Invocation::Version
);
}
#[test]
fn classify_help_flags() {
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "--help"])),
Invocation::Help
);
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "-h"])),
Invocation::Help
);
}
#[test]
fn classify_normal_helper_invocation() {
// How git actually calls us: `<remote-name> <url>`.
assert_eq!(
classify_args(&args(&[
"git-remote-gitlawb",
"origin",
"gitlawb://did:key:z6MkFoo123/my-repo",
])),
Invocation::Helper
);
}
#[test]
fn classify_no_args_is_helper() {
// No flag → falls through to the helper path, which reports its own
// usage error. `--version`/`--help` must not be inferred from emptiness.
assert_eq!(
classify_args(&args(&["git-remote-gitlawb"])),
Invocation::Helper
);
}
#[test]
fn classify_flag_only_in_first_position() {
// A remote literally named "--version" must not be treated as the flag.
assert_eq!(
classify_args(&args(&["git-remote-gitlawb", "origin", "--version",])),
Invocation::Helper
);
}
#[test]
fn version_line_matches_package_metadata() {
let line = version_line();
assert_eq!(
line,
format!("git-remote-gitlawb {}", env!("CARGO_PKG_VERSION"))
);
// Single line, no trailing newline (println! adds the newline).
assert!(!line.contains('\n'));
}
#[test]
fn help_text_includes_version_and_flags() {
let help = help_text();
assert!(help.starts_with(&version_line()));
assert!(help.contains("--version"));
assert!(help.contains("--help"));
assert!(help.contains("GITLAWB_NODE"));
assert!(help.ends_with('\n'));
}
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
serde = { workspace = true }
serde_json = { workspace = true }
serde_jcs = { workspace = true }
thiserror = { workspace = true }
ed25519-dalek = { workspace = true }
sha2 = { workspace = true }
hex = { workspace = true }
base64 = { workspace = true }
multibase = { workspace = true }
[dev-dependencies]
gitlawb-core = { path = "../gitlawb-core" }
rand = { workspace = true }
+557
View File
@@ -0,0 +1,557 @@
//! One typed, signed, cert-bound provenance blob.
//!
//! Fields: `type` (discriminator), `payload` (opaque, type-specific JSON),
//! `cert_hash` (binds to one ref-update cert), `signer` (`did:key`), and
//! `sig` (base64url-no-pad ed25519).
//!
//! The signature covers the domain-separated input
//! `b"gitlawb-attest-sig/v1\n"` followed by JCS-encoded `{type, payload,
//! cert_hash}` per RFC 8785. The domain tag is in the byte stream rather
//! than the canonical JSON so the wire shape stays additive: an `Attestation`
//! deserialized today and one constructed by a future `v2` signer differ only
//! by which tag the verifier prepends.
//!
//! Type discriminators are slash-separated namespace + version strings
//! (`covenant/exec/v1`, `slsa/v1.0`, `sigstore/dsse/v1`); verifiers register
//! by exact match.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64U, Engine};
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use crate::error::{Error, Result};
/// Domain-separation tag prepended to the JCS signing input. Bumped to
/// `v2` if the signing-input shape changes.
const SIGNING_DOMAIN: &[u8] = b"gitlawb-attest-sig/v1\n";
/// Multicodec varint prefix for Ed25519 public keys (0xed 0x01).
const ED25519_MULTICODEC: [u8; 2] = [0xed, 0x01];
/// A signed provenance blob attached to a ref-update cert.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Attestation {
/// Type discriminator, e.g. `covenant/exec/v1`.
#[serde(rename = "type")]
pub type_: String,
/// Type-specific payload. The verifier reparses into its concrete shape.
pub payload: serde_json::Value,
/// Lowercase SHA-256 hex of the cert body. Binds this attestation to one
/// cert. The lowercase form is canonical because the JCS signing input
/// includes this field verbatim — accepting mixed case on the wire would
/// require both signer and verifier to normalize before computing the
/// JCS bytes, an easy spec rule to miss across language ports.
pub cert_hash: String,
/// `did:key` of the signer; the verifying key is recoverable from it.
pub signer: String,
/// base64url-no-pad ed25519 signature over the JCS signing input.
pub sig: String,
}
/// Type-specific payload. Implement on a struct that is `Serialize +
/// DeserializeOwned` to participate.
pub trait AttestationPayload: Serialize + DeserializeOwned + Send + Sync {
/// The discriminator string written on the wire.
fn payload_type() -> &'static str;
}
#[derive(Serialize)]
struct SigningInput<'a> {
#[serde(rename = "type")]
type_: &'a str,
payload: &'a serde_json::Value,
cert_hash: &'a str,
}
impl Attestation {
/// Sign a fresh attestation. `cert_hash_bytes` comes from
/// [`crate::cert::cert_hash`].
pub fn sign<P: AttestationPayload>(
signing_key: &SigningKey,
payload: P,
cert_hash_bytes: [u8; 32],
) -> Result<Self> {
let type_ = P::payload_type().to_string();
validate_type(&type_)?;
let payload_value = serde_json::to_value(payload)?;
let cert_hash_hex = hex::encode(cert_hash_bytes);
let bytes = canonical_signing_bytes(&type_, &payload_value, &cert_hash_hex)?;
let sig: Signature = signing_key.sign(&bytes);
Ok(Self {
type_,
payload: payload_value,
cert_hash: cert_hash_hex,
signer: did_key_from_verifying_key(&signing_key.verifying_key()),
sig: B64U.encode(sig.to_bytes()),
})
}
/// Verify the signature and check that `cert_hash` matches
/// `expected_cert_hash`. Returns the recovered verifying key so the caller
/// can check it against an allowlist.
pub fn verify_signature(&self, expected_cert_hash: [u8; 32]) -> Result<VerifyingKey> {
let expected_hex = hex::encode(expected_cert_hash);
if self.cert_hash != expected_hex {
return Err(Error::CertHashMismatch);
}
validate_type(&self.type_)?;
let bytes = canonical_signing_bytes(&self.type_, &self.payload, &self.cert_hash)?;
let vk = verifying_key_from_did_key(&self.signer)?;
let sig_bytes: [u8; 64] = B64U
.decode(&self.sig)
.map_err(|e| Error::Signature(format!("base64url: {e}")))?
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify(&bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(vk)
}
/// Reparse `payload` as `P`. Errors if the type discriminator does not
/// match `P::payload_type()`.
pub fn payload_as<P: AttestationPayload>(&self) -> Result<P> {
if self.type_ != P::payload_type() {
return Err(Error::Type(format!(
"expected '{}', got '{}'",
P::payload_type(),
self.type_
)));
}
Ok(serde_json::from_value(self.payload.clone())?)
}
}
/// Discriminator grammar: nonempty, ASCII-graphic only, slash-separated
/// segments of `[a-zA-Z0-9._+-]`, no empty segment, capped at 128 bytes.
/// Matches `covenant/exec/v1`, `slsa/v1.0`, `sigstore/dsse/v1`.
fn validate_type(t: &str) -> Result<()> {
if t.is_empty() {
return Err(Error::Type("empty discriminator".to_string()));
}
if t.len() > 128 {
return Err(Error::Type(format!(
"discriminator longer than 128 bytes ({})",
t.len()
)));
}
if !t.is_ascii() {
return Err(Error::Type(format!("discriminator must be ASCII: '{t}'")));
}
for segment in t.split('/') {
if segment.is_empty() {
return Err(Error::Type(format!(
"discriminator has an empty segment: '{t}'"
)));
}
for c in segment.chars() {
if !(c.is_ascii_alphanumeric() || c == '.' || c == '_' || c == '+' || c == '-') {
return Err(Error::Type(format!(
"discriminator segment '{segment}' has disallowed character '{c}'"
)));
}
}
}
Ok(())
}
fn canonical_signing_bytes(
type_: &str,
payload: &serde_json::Value,
cert_hash_hex: &str,
) -> Result<Vec<u8>> {
let input = SigningInput {
type_,
payload,
cert_hash: cert_hash_hex,
};
let jcs = serde_jcs::to_vec(&input).map_err(|e| Error::Jcs(e.to_string()))?;
let mut out = Vec::with_capacity(SIGNING_DOMAIN.len() + jcs.len());
out.extend_from_slice(SIGNING_DOMAIN);
out.extend_from_slice(&jcs);
Ok(out)
}
fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&key.to_bytes());
let encoded = multibase::encode(multibase::Base::Base58Btc, &buf);
format!("did:key:{encoded}")
}
/// The DID:key spec mandates base58btc (multibase `z`-prefix). Other multibase
/// encodings of the same key bytes are rejected so allowlists, dedup, and
/// signer-pinning policies that compare `signer` strings remain consistent
/// across peers.
fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
let method_id = did
.strip_prefix("did:key:")
.ok_or_else(|| Error::Did(format!("not a did:key: {did}")))?;
if !method_id.starts_with('z') {
return Err(Error::Did(format!(
"did:key must use base58btc (z-prefix): {did}"
)));
}
let (base, bytes) =
multibase::decode(method_id).map_err(|e| Error::Did(format!("multibase: {e}")))?;
if base != multibase::Base::Base58Btc {
return Err(Error::Did(format!(
"did:key must use base58btc (z-prefix): {did}"
)));
}
if bytes.len() != ED25519_MULTICODEC.len() + 32
|| bytes[..ED25519_MULTICODEC.len()] != ED25519_MULTICODEC
{
return Err(Error::Did("not an ed25519 did:key".to_string()));
}
let key_bytes: [u8; 32] = bytes[ED25519_MULTICODEC.len()..]
.try_into()
.expect("length checked above");
VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
fn fresh() -> SigningKey {
SigningKey::generate(&mut OsRng)
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct DummyPayload {
agent: String,
commit: String,
}
impl AttestationPayload for DummyPayload {
fn payload_type() -> &'static str {
"test/dummy/v1"
}
}
fn sample_cert_hash() -> [u8; 32] {
let mut h = [0u8; 32];
for (i, b) in h.iter_mut().enumerate() {
*b = i as u8;
}
h
}
fn dummy_attestation(sk: &SigningKey, cert_hash: [u8; 32]) -> Attestation {
Attestation::sign(
sk,
DummyPayload {
agent: "a".to_string(),
commit: "b".to_string(),
},
cert_hash,
)
.unwrap()
}
#[test]
fn sign_then_verify_roundtrip() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let payload = DummyPayload {
agent: "did:key:z6MkTest".to_string(),
commit: "deadbeef".to_string(),
};
let att = Attestation::sign(&sk, payload.clone(), cert_hash).unwrap();
let vk = att.verify_signature(cert_hash).unwrap();
assert_eq!(vk.to_bytes(), sk.verifying_key().to_bytes());
assert_eq!(att.payload_as::<DummyPayload>().unwrap(), payload);
}
#[test]
fn cross_cert_replay_fails() {
let sk = fresh();
let cert_a = sample_cert_hash();
let mut cert_b = sample_cert_hash();
cert_b[0] ^= 0xff;
let att = dummy_attestation(&sk, cert_a);
let err = att.verify_signature(cert_b).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
#[test]
fn tampered_payload_fails_verify() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&sk, cert_hash);
att.payload = serde_json::json!({ "agent": "evil", "commit": "b" });
let err = att.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
#[test]
fn payload_as_wrong_type_errors() {
let sk = fresh();
let att = dummy_attestation(&sk, sample_cert_hash());
#[derive(Debug, Serialize, Deserialize)]
struct Wrong {
#[allow(dead_code)]
x: String,
}
impl AttestationPayload for Wrong {
fn payload_type() -> &'static str {
"test/other/v1"
}
}
let err = att.payload_as::<Wrong>().unwrap_err();
assert!(matches!(err, Error::Type(_)));
}
#[test]
fn json_roundtrip_preserves_signature() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let att = dummy_attestation(&sk, cert_hash);
let json = serde_json::to_string(&att).unwrap();
let back: Attestation = serde_json::from_str(&json).unwrap();
back.verify_signature(cert_hash).unwrap();
}
/// Wire shape is part of the public protocol — pin field names so an
/// accidental rename surfaces as a test failure, not silent interop drift.
#[test]
fn wire_shape_pins_field_names() {
let sk = fresh();
let att = dummy_attestation(&sk, sample_cert_hash());
let v: serde_json::Value = serde_json::to_value(&att).unwrap();
let obj = v.as_object().expect("attestation must serialize as object");
let keys: std::collections::BTreeSet<&str> = obj.keys().map(String::as_str).collect();
let expected: std::collections::BTreeSet<&str> =
["type", "payload", "cert_hash", "signer", "sig"]
.into_iter()
.collect();
assert_eq!(keys, expected);
assert!(obj["type"].is_string());
assert!(obj["cert_hash"].is_string());
assert!(obj["signer"].is_string());
assert!(obj["sig"].is_string());
}
/// The spec mandates lowercase hex on the wire. An uppercase cert_hash
/// is rejected as `CertHashMismatch` so a non-canonical signer can't
/// produce attestations that verify against some peers but not others.
#[test]
fn uppercase_cert_hash_is_rejected_as_non_canonical() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&sk, cert_hash);
att.cert_hash = att.cert_hash.to_uppercase();
let err = att.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
#[test]
fn empty_type_rejected_at_signing() {
#[derive(Serialize, Deserialize)]
struct Empty {}
impl AttestationPayload for Empty {
fn payload_type() -> &'static str {
""
}
}
let sk = fresh();
let err = Attestation::sign(&sk, Empty {}, sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Type(_)));
}
#[test]
fn validate_type_grammar() {
validate_type("covenant/exec/v1").unwrap();
validate_type("slsa/v1.0").unwrap();
validate_type("sigstore/dsse/v1").unwrap();
validate_type("a").unwrap();
validate_type("a.b-c_d+e/x").unwrap();
for bad in [
"",
" ",
"covenant/exec/v1 ",
"covenant//v1",
"/covenant/v1",
"covenant/v1/",
"covenant\x00exec/v1",
"covenant/exec\nv1",
"сovenant/exec/v1",
"covenant/ex ec/v1",
"covenant/exec/v1#frag",
] {
let err = validate_type(bad).unwrap_err();
assert!(matches!(err, Error::Type(_)), "must reject '{bad}'");
}
let too_long = "a/".repeat(70);
let err = validate_type(&too_long).unwrap_err();
assert!(matches!(err, Error::Type(_)));
}
#[test]
fn did_key_must_be_base58btc() {
// Sign with a real key, then re-encode the public-key bytes in a
// different multibase. Verification must reject.
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
let mut buf = Vec::with_capacity(34);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&sk.verifying_key().to_bytes());
for base in [
multibase::Base::Base64Url,
multibase::Base::Base32Lower,
multibase::Base::Base16Lower,
] {
att.signer = format!("did:key:{}", multibase::encode(base, &buf));
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)), "must reject base {base:?}");
}
}
#[test]
fn verify_rejects_did_without_did_key_prefix() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
att.signer = "ed25519:abc".to_string();
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_did_with_wrong_multicodec() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
// 0x12 0x00 = sha2-256 multicodec — wrong for an ed25519 public key.
let mut buf = vec![0x12, 0x00];
buf.extend_from_slice(&sk.verifying_key().to_bytes());
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_did_with_wrong_key_length() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
let mut buf = Vec::with_capacity(33);
buf.extend_from_slice(&ED25519_MULTICODEC);
// 31 bytes instead of 32.
buf.extend_from_slice(&sk.verifying_key().to_bytes()[..31]);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_bad_signature_base64() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
att.sig = "!!not-base64!!".to_string();
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
#[test]
fn verify_rejects_signature_wrong_length() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
att.sig = B64U.encode([0u8; 32]); // valid base64, wrong size.
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
/// A payload that happens to contain a `cert_hash` field of its own does
/// not interfere with the outer binding: the attestation envelope's
/// `cert_hash` is the only field consulted by `verify_signature`, and the
/// payload is treated as opaque JSON throughout. A reviewer worried about
/// shadowing or smuggling can verify this directly.
#[test]
fn payload_cert_hash_field_does_not_shadow_outer_binding() {
#[derive(Serialize, Deserialize)]
struct Sneaky {
cert_hash: String,
x: String,
}
impl AttestationPayload for Sneaky {
fn payload_type() -> &'static str {
"test/sneaky/v1"
}
}
let sk = fresh();
let outer_hash = sample_cert_hash();
let att = Attestation::sign(
&sk,
Sneaky {
cert_hash: "forged-by-the-attacker".to_string(),
x: "y".to_string(),
},
outer_hash,
)
.unwrap();
// Envelope's cert_hash is the outer (real) hash, lowercase hex.
assert_eq!(att.cert_hash, hex::encode(outer_hash));
// Payload retains its own field — opaque to the protocol.
assert_eq!(
att.payload.get("cert_hash").and_then(|v| v.as_str()),
Some("forged-by-the-attacker"),
);
// Verification binds to the outer envelope's cert_hash.
att.verify_signature(outer_hash).unwrap();
// And rejects when the outer cert hash differs.
let mut wrong = outer_hash;
wrong[0] ^= 0xff;
let err = att.verify_signature(wrong).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
/// A different protocol that also signs `{type, payload, cert_hash}` with
/// the same key would still fail attestation verification because the
/// signing input carries a domain tag the other protocol does not include.
#[test]
fn cross_protocol_replay_is_rejected_by_domain_separation() {
let sk = fresh();
let cert_hash = sample_cert_hash();
let att = dummy_attestation(&sk, cert_hash);
// Sign the same JCS bytes WITHOUT the domain tag and graft the
// signature back onto the attestation.
let cert_hash_hex = hex::encode(cert_hash);
let bare = serde_jcs::to_vec(&SigningInput {
type_: &att.type_,
payload: &att.payload,
cert_hash: &cert_hash_hex,
})
.unwrap();
let other_sig: Signature = sk.sign(&bare);
let mut tampered = att.clone();
tampered.sig = B64U.encode(other_sig.to_bytes());
let err = tampered.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
}
+327
View File
@@ -0,0 +1,327 @@
//! Envelope around a `gitlawb/ref-update/v1` cert plus optional attestations,
//! and the cert-hash helper attestations bind to.
//!
//! An empty envelope serializes to the same bytes as the underlying bare cert.
//! `cert_hash` strips `signatures` and `attestations`, JCS-encodes the rest,
//! and SHA-256s; the result is what every attestation binds to.
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use crate::attestation::Attestation;
use crate::error::{Error, Result};
use crate::verifier::{Registry, VerifiedAttestation};
/// The discriminator the wrapped cert must carry. Matches the constant of the
/// same name in `gitlawb-core::cert`.
pub const CERT_TYPE: &str = "gitlawb/ref-update/v1";
/// A ref-update cert with optional attestations.
///
/// The cert is flattened into the top-level JSON, so an empty envelope
/// produces the same bytes as a bare cert and existing decoders ignore the
/// extra `attestations` field. The cert is held as `serde_json::Value` so
/// this crate has no runtime dependency on `gitlawb-core::cert::RefUpdateCert`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AttestedRefUpdateCert {
/// The cert body and signatures, flattened to the top level.
#[serde(flatten)]
pub cert: serde_json::Value,
/// Attached attestations. Empty when absent on the wire.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub attestations: Vec<Attestation>,
}
impl AttestedRefUpdateCert {
/// Wrap a cert. No attestations are attached yet.
///
/// The serialized cert must be a JSON object whose `type` field equals
/// [`CERT_TYPE`] and which does **not** already carry an `attestations`
/// field. The second rule catches two cases that would otherwise leave
/// the envelope in an inconsistent in-memory state:
///
/// - Double-wrap: `from_cert(&previously_wrapped_envelope)` would put the
/// inner attestations on `self.cert` and leave `self.attestations` empty.
/// - Pre-attested cert from a peer: a malformed or malicious cert with an
/// attached `attestations` field would survive `cert_hash` (which strips
/// the field) but bypass the wrapper's append-and-verify path entirely.
///
/// Both are rejected with [`Error::Payload`]. To re-wrap, deserialize the
/// envelope as `AttestedRefUpdateCert` directly.
pub fn from_cert<T: Serialize>(cert: &T) -> Result<Self> {
let value = serde_json::to_value(cert)?;
let obj = value
.as_object()
.ok_or_else(|| Error::Payload("cert must serialize as a JSON object".to_string()))?;
match obj.get("type").and_then(|v| v.as_str()) {
Some(t) if t == CERT_TYPE => {}
Some(other) => {
return Err(Error::Payload(format!(
"cert type must be '{CERT_TYPE}', got '{other}'"
)));
}
None => {
return Err(Error::Payload(format!(
"cert missing 'type' field (expected '{CERT_TYPE}')"
)));
}
}
if obj.contains_key("attestations") {
return Err(Error::Payload(
"cert already carries an 'attestations' field; deserialize as \
AttestedRefUpdateCert directly instead of re-wrapping"
.to_string(),
));
}
Ok(Self {
cert: value,
attestations: Vec::new(),
})
}
/// The 32-byte cert hash every attached attestation must bind to.
pub fn cert_hash(&self) -> Result<[u8; 32]> {
cert_hash(&self.cert)
}
/// Append an already-signed attestation.
pub fn attach(&mut self, attestation: Attestation) {
self.attestations.push(attestation);
}
/// Verify every attached attestation against this envelope's cert hash,
/// applying the registry's policy. Equivalent to
/// `registry.verify_all(&self.attestations, self.cert_hash()?)` but
/// reads as one operation at call sites.
pub fn verify_attestations(&self, registry: &Registry) -> Result<Vec<VerifiedAttestation>> {
registry.verify_all(&self.attestations, self.cert_hash()?)
}
}
/// SHA-256 of the cert body. The body must be a JSON object; `signatures` and
/// `attestations` are stripped before JCS encoding (RFC 8785), so the hash is
/// reproducible across languages and JSON libraries and is stable across
/// countersignatures.
pub fn cert_hash(cert: &serde_json::Value) -> Result<[u8; 32]> {
let mut body = cert.clone();
let obj = body
.as_object_mut()
.ok_or_else(|| Error::Payload("cert body must be a JSON object".to_string()))?;
obj.remove("signatures");
obj.remove("attestations");
let bytes = serde_jcs::to_vec(&body).map_err(|e| Error::Jcs(e.to_string()))?;
let mut h = Sha256::new();
h.update(&bytes);
let out = h.finalize();
let mut arr = [0u8; 32];
arr.copy_from_slice(&out);
Ok(arr)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn bare_cert() -> serde_json::Value {
json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkRepo",
"ref_name": "refs/heads/main",
"from": "0".repeat(64),
"to": "a".repeat(64),
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "n1",
"signatures": [{"signer": "did:key:z6MkRepo", "sig": "abc"}]
})
}
#[test]
fn empty_envelope_serializes_as_bare_cert() {
let bare = bare_cert();
let env = AttestedRefUpdateCert::from_cert(&bare).unwrap();
let env_json: serde_json::Value =
serde_json::from_str(&serde_json::to_string(&env).unwrap()).unwrap();
let bare_keys: std::collections::BTreeSet<&String> =
bare.as_object().unwrap().keys().collect();
let env_keys: std::collections::BTreeSet<&String> =
env_json.as_object().unwrap().keys().collect();
assert_eq!(bare_keys, env_keys);
}
#[test]
fn bare_cert_parses_as_envelope_with_no_attestations() {
let bare = bare_cert();
let env: AttestedRefUpdateCert = serde_json::from_value(bare.clone()).unwrap();
assert!(env.attestations.is_empty());
assert_eq!(env.cert, bare);
}
#[test]
fn cert_hash_is_stable_across_countersignatures() {
let base_body = json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkRepo",
"ref_name": "refs/heads/main",
"from": "0".repeat(64),
"to": "a".repeat(64),
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "n1"
});
let mut one_sig = base_body.clone();
one_sig["signatures"] = json!([{"signer": "x", "sig": "y"}]);
let mut two_sigs = base_body.clone();
two_sigs["signatures"] = json!([
{"signer": "x", "sig": "y"},
{"signer": "a", "sig": "b"}
]);
assert_eq!(cert_hash(&one_sig).unwrap(), cert_hash(&two_sigs).unwrap());
}
#[test]
fn cert_hash_changes_when_body_changes() {
let body_a = json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkRepo",
"ref_name": "refs/heads/main",
"from": "0".repeat(64),
"to": "a".repeat(64),
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "n1",
"signatures": []
});
let mut body_b = body_a.clone();
body_b["to"] = json!("b".repeat(64));
assert_ne!(cert_hash(&body_a).unwrap(), cert_hash(&body_b).unwrap());
}
#[test]
fn cert_hash_rejects_non_object_root() {
let err = cert_hash(&json!(null)).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
let err = cert_hash(&json!([1, 2, 3])).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
let err = cert_hash(&json!("scalar")).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn from_cert_rejects_wrong_type() {
let wrong = json!({"type": "gitlawb/ref-update/v0", "repo": "did:key:z"});
let err = AttestedRefUpdateCert::from_cert(&wrong).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn from_cert_rejects_missing_type() {
let no_type = json!({"repo": "did:key:z"});
let err = AttestedRefUpdateCert::from_cert(&no_type).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn from_cert_rejects_non_object() {
let arr = json!([1, 2, 3]);
let err = AttestedRefUpdateCert::from_cert(&arr).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
#[test]
fn mutating_env_cert_changes_the_hash() {
let mut env = AttestedRefUpdateCert::from_cert(&bare_cert()).unwrap();
let hash_before = env.cert_hash().unwrap();
env.cert.as_object_mut().unwrap().insert(
"nonce".to_string(),
serde_json::Value::String("mutated".to_string()),
);
let hash_after = env.cert_hash().unwrap();
assert_ne!(hash_before, hash_after);
}
/// A cert that already carries an `attestations` field is rejected
/// regardless of the field's contents. Catches double-wrap (where the
/// inner attestations would otherwise be smuggled into `self.cert`) and
/// peer-supplied "pre-attested" certs that would bypass the wrapper.
#[test]
fn from_cert_rejects_preexisting_attestations_field() {
let mut sneaky = bare_cert();
sneaky["attestations"] = json!([{"smuggled": true}]);
let err = AttestedRefUpdateCert::from_cert(&sneaky).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
// Empty array still rejected — the field's mere presence is the signal.
let mut empty = bare_cert();
empty["attestations"] = json!([]);
let err = AttestedRefUpdateCert::from_cert(&empty).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// `from_cert` on an already-wrapped envelope is the same case as
/// `from_cert` on a cert with a pre-existing `attestations` field: serde
/// flattens the envelope back into a JSON object that carries the field.
#[test]
fn from_cert_rejects_a_previously_wrapped_envelope() {
let mut env = AttestedRefUpdateCert::from_cert(&bare_cert()).unwrap();
// Inject an attestation-shaped value so the wire round-trip would
// actually re-emit the field.
env.cert
.as_object_mut()
.unwrap()
.insert("attestations".to_string(), json!([]));
let err = AttestedRefUpdateCert::from_cert(&env).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// `attestations: null` is explicitly not the same as missing. The
/// `#[serde(default)]` attribute only fills in when the key is absent —
/// a present-but-null wire field is rejected, which surfaces malformed
/// payloads early instead of silently treating them as empty.
#[test]
fn attestations_null_on_wire_is_rejected() {
let mut with_null = bare_cert();
with_null["attestations"] = serde_json::Value::Null;
let err = serde_json::from_value::<AttestedRefUpdateCert>(with_null).unwrap_err();
let msg = format!("{err}");
assert!(
msg.contains("expected a sequence") || msg.contains("invalid type"),
"unexpected error: {msg}"
);
}
#[test]
fn attestations_string_on_wire_is_rejected() {
let mut with_str = bare_cert();
with_str["attestations"] = serde_json::Value::String("oops".to_string());
let err = serde_json::from_value::<AttestedRefUpdateCert>(with_str).unwrap_err();
let msg = format!("{err}");
assert!(msg.contains("expected a sequence") || msg.contains("invalid type"));
}
/// `attach` does not deduplicate. Duplicate attestations are a verifier-
/// policy concern (e.g., "at most one SLSA provenance"), not a wrapper
/// concern. Pinning the behavior so a future deduplication change is an
/// intentional break.
#[test]
fn attach_does_not_deduplicate() {
let mut env = AttestedRefUpdateCert::from_cert(&bare_cert()).unwrap();
let att = Attestation {
type_: "demo/v1".to_string(),
payload: serde_json::json!({}),
cert_hash: "deadbeef".to_string(),
signer: "did:key:z6MkDup".to_string(),
sig: "AAAA".to_string(),
};
env.attach(att.clone());
env.attach(att);
assert_eq!(env.attestations.len(), 2);
}
}
+46
View File
@@ -0,0 +1,46 @@
use thiserror::Error;
/// Errors from building or verifying an attestation.
#[derive(Debug, Error)]
pub enum Error {
/// Discriminator failed validation (empty, whitespace, control char,
/// disallowed grammar).
#[error("attestation type: {0}")]
Type(String),
/// Attestation was signed against a different cert.
#[error("cert hash mismatch")]
CertHashMismatch,
/// Ed25519 signature failed verification.
#[error("signature: {0}")]
Signature(String),
/// DID could not be parsed.
#[error("did: {0}")]
Did(String),
/// Payload structure check failed, or the cert body was not a JSON object.
#[error("payload: {0}")]
Payload(String),
/// No verifier registered for the type and policy required one.
#[error("no verifier registered for type '{0}'")]
UnknownType(String),
/// `RequireAll` policy did not find an entry for a required type, or the
/// entry was not fully verified.
#[error("required attestation type missing or unverified: '{0}'")]
RequiredMissing(String),
/// JSON failure.
#[error("json: {0}")]
Json(#[from] serde_json::Error),
/// JCS encoding failure.
#[error("jcs: {0}")]
Jcs(String),
}
/// Crate result alias.
pub type Result<T> = std::result::Result<T, Error>;
+132
View File
@@ -0,0 +1,132 @@
//! Pluggable provenance attachments for gitlawb ref-update certs.
//!
//! A `gitlawb/ref-update/v1` cert proves who pushed and where a ref moved to.
//! It does not say how the commit was produced. This crate adds an optional
//! `attestations` field that lets any provenance system (SLSA, Sigstore,
//! in-toto, agent runtimes) attach a typed, signed blob bound to the cert by
//! its hash.
//!
//! The cert format stays additive. A cert with no attestations serializes the
//! same bytes it does today, and existing decoders (which do not set
//! `deny_unknown_fields`) silently drop the new `attestations` field rather
//! than reject the cert.
//!
//! ## Wire shape
//!
//! ```json
//! {
//! "type": "gitlawb/ref-update/v1",
//! // ...standard cert fields...,
//! "signatures": [...],
//! "attestations": [
//! {
//! "type": "covenant/exec/v1",
//! "payload": { /* opaque, type-specific */ },
//! "cert_hash": "<sha256 hex of JCS-encoded cert body>",
//! "signer": "did:key:z6Mk...",
//! "sig": "<base64url ed25519 over the signing input>"
//! }
//! ]
//! }
//! ```
//!
//! ## Canonical bytes
//!
//! Hashing and signing inputs use JCS (RFC 8785) so they reproduce across
//! implementations regardless of struct field order or JSON library.
//!
//! The cert hash is `SHA-256(JCS(cert_body_without_signatures_or_attestations))`,
//! expressed as lowercase hex on the wire. Uppercase is non-canonical and
//! rejected on verify so the JCS signing input is uniquely determined by the
//! cert bytes alone — there is no implicit normalization rule that
//! cross-language signers might miss.
//!
//! The attestation signing input is the byte string
//! `b"gitlawb-attest-sig/v1\n" || JCS({type, payload, cert_hash})`. The
//! domain tag is in the byte stream, not the canonical JSON, so an attestation
//! cannot be confused with a same-shape JCS document signed for a different
//! protocol that uses the same Ed25519 key.
//!
//! ## DID:key
//!
//! Signers are `did:key` DIDs over Ed25519, encoded with the multibase
//! base58btc (`z`) prefix per the W3C DID:key specification. Non-base58btc
//! encodings of the same public key are rejected on verify so allowlists,
//! deduplication, and signer-pinning policies that compare `signer` strings
//! stay consistent across peers.
//!
//! ## Verification
//!
//! [`Registry`] looks up a verifier by type discriminator. [`Policy`] decides
//! what to do when no verifier matches: `AcceptKnown` (default) lets unknown
//! types pass without trust; `RequireAll` enforces a per-repo allowlist while
//! still letting unrelated attached attestations through unverified (so a
//! third-party attachment cannot DoS the cert); `RejectUnknown` rejects
//! anything unregistered.
//!
//! ## Quick start
//!
//! ```
//! use ed25519_dalek::SigningKey;
//! use gitlawb_attest::{
//! Attestation, AttestationPayload, AttestationVerifier, AttestedRefUpdateCert,
//! Error, Policy, Registry, Result, CERT_TYPE,
//! };
//! use serde::{Deserialize, Serialize};
//!
//! #[derive(Serialize, Deserialize)]
//! struct ExecPayload { agent: String }
//! impl AttestationPayload for ExecPayload {
//! fn payload_type() -> &'static str { "covenant/exec/v1" }
//! }
//!
//! struct ExecVerifier;
//! impl AttestationVerifier for ExecVerifier {
//! fn type_(&self) -> &'static str { "covenant/exec/v1" }
//! fn verify_payload(&self, payload: &serde_json::Value) -> Result<()> {
//! payload.get("agent").and_then(|v| v.as_str())
//! .ok_or_else(|| Error::Payload("missing agent".into()))?;
//! Ok(())
//! }
//! }
//!
//! # fn run() -> Result<()> {
//! let cert = serde_json::json!({
//! "type": CERT_TYPE, "repo": "did:key:z6MkRepo",
//! "ref_name": "refs/heads/main",
//! "from": "0".repeat(64), "to": "a".repeat(64),
//! "seq": 1, "timestamp": "2026-01-01T00:00:00Z", "nonce": "n1",
//! "signatures": []
//! });
//! let mut env = AttestedRefUpdateCert::from_cert(&cert)?;
//!
//! let sk = SigningKey::from_bytes(&[7u8; 32]);
//! let payload = ExecPayload { agent: "did:key:z6MkAgent".into() };
//! let att = Attestation::sign(&sk, payload, env.cert_hash()?)?;
//! env.attach(att);
//!
//! let mut registry = Registry::new().with_policy(Policy::RejectUnknown);
//! registry.register(ExecVerifier);
//! let verified = env.verify_attestations(&registry)?;
//! assert_eq!(verified[0].type_, "covenant/exec/v1");
//! assert!(verified[0].fully_verified);
//! # Ok(()) }
//! # run().unwrap();
//! ```
pub mod attestation;
pub mod cert;
pub mod error;
pub mod verifier;
pub use attestation::{Attestation, AttestationPayload};
pub use cert::{cert_hash, AttestedRefUpdateCert, CERT_TYPE};
pub use error::{Error, Result};
pub use verifier::{AttestationVerifier, Policy, Registry, VerifiedAttestation};
/// Version string for the attestation envelope wire shape.
///
/// Bumped only when the binding rules — JCS input shape, domain-separation
/// tag, hash algorithm — change. Independent of [`CERT_TYPE`], which versions
/// the underlying ref-update cert.
pub const ATTEST_ENVELOPE_VERSION: &str = "v1";
+377
View File
@@ -0,0 +1,377 @@
//! Verifier trait, registry, and policy.
//!
//! A [`Registry`] maps a type discriminator to a verifier. [`Policy`] decides
//! what happens for types nothing is registered for: accept without trust,
//! reject, or require an allowlist.
//!
//! Under [`Policy::RequireAll`] the registry never short-circuits on unknown
//! types — any attestation can be attached by any third party, so blocking
//! the cert because an attacker added an unrelated `attacker/spam/v1` would
//! be a denial-of-service vector. Unknown types are silently treated as
//! `fully_verified = false`; the allowlist is enforced only by the
//! presence-and-verified check on `required_types` after the batch runs.
use std::collections::HashMap;
use crate::attestation::Attestation;
use crate::error::{Error, Result};
/// Verifier for one attestation type. Implementors declare the discriminator
/// they handle and validate the payload's shape. The signature and cert-hash
/// binding are checked by [`Registry::verify`] before `verify_payload` runs.
pub trait AttestationVerifier: Send + Sync {
/// The discriminator this verifier handles. Must be a `'static` literal
/// so the registry can key on it without copying.
fn type_(&self) -> &'static str;
/// Validate payload structure. Signature is already verified.
fn verify_payload(&self, payload: &serde_json::Value) -> Result<()>;
}
/// What to do for attestation types with no registered verifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum Policy {
/// Unknown types pass but are flagged `fully_verified = false`. Default.
#[default]
AcceptKnown,
/// Every entry in `required_types` must be present and fully verified.
/// Unknown types beyond the required set are still accepted unverified,
/// so a malicious extra attestation cannot DoS the cert.
RequireAll,
/// Reject every attestation whose type is not registered.
RejectUnknown,
}
/// Result of verifying one attestation.
#[derive(Debug, Clone)]
pub struct VerifiedAttestation {
/// Type discriminator.
pub type_: String,
/// Signer's `did:key`.
pub signer: String,
/// Lowercase SHA-256 hex of the cert body the attestation bound to.
pub cert_hash: String,
/// `true` when a verifier ran a payload check and accepted it. `false`
/// when the policy let an unknown type through without one.
pub fully_verified: bool,
}
/// Verifiers keyed by type discriminator.
#[derive(Default)]
pub struct Registry {
by_type: HashMap<&'static str, Box<dyn AttestationVerifier>>,
policy: Policy,
required_types: Vec<String>,
}
impl Registry {
/// Empty registry with the default policy.
pub fn new() -> Self {
Self::default()
}
/// Register a verifier by value. Returns the prior verifier for the same
/// type, if any, so callers can detect double-registration.
pub fn register<V: AttestationVerifier + 'static>(
&mut self,
verifier: V,
) -> Option<Box<dyn AttestationVerifier>> {
self.register_boxed(Box::new(verifier))
}
/// Like [`Self::register`] but takes an already-boxed trait object — useful
/// when verifiers are constructed dynamically.
pub fn register_boxed(
&mut self,
verifier: Box<dyn AttestationVerifier>,
) -> Option<Box<dyn AttestationVerifier>> {
let key = verifier.type_();
self.by_type.insert(key, verifier)
}
/// Switch policy.
pub fn with_policy(mut self, policy: Policy) -> Self {
self.policy = policy;
self
}
/// Type discriminators that must be present and verified under
/// `Policy::RequireAll`. Ignored under other policies.
pub fn require_types<I, S>(mut self, types: I) -> Self
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
self.required_types = types.into_iter().map(Into::into).collect();
self
}
/// Active policy.
pub fn policy(&self) -> Policy {
self.policy
}
/// Verify signature, cert-hash binding, and (if a verifier is registered)
/// payload structure. Under `RequireAll` this is lenient on unknown types
/// — see the module docstring.
pub fn verify(
&self,
attestation: &Attestation,
expected_cert_hash: [u8; 32],
) -> Result<VerifiedAttestation> {
attestation.verify_signature(expected_cert_hash)?;
let fully = match self.by_type.get(attestation.type_.as_str()) {
Some(v) => {
v.verify_payload(&attestation.payload)?;
true
}
None => match self.policy {
Policy::AcceptKnown | Policy::RequireAll => false,
Policy::RejectUnknown => {
return Err(Error::UnknownType(attestation.type_.clone()));
}
},
};
Ok(VerifiedAttestation {
type_: attestation.type_.clone(),
signer: attestation.signer.clone(),
cert_hash: attestation.cert_hash.clone(),
fully_verified: fully,
})
}
/// Verify a batch, then enforce `RequireAll`.
pub fn verify_all(
&self,
attestations: &[Attestation],
expected_cert_hash: [u8; 32],
) -> Result<Vec<VerifiedAttestation>> {
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
if self.policy == Policy::RequireAll {
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
}
}
}
Ok(verified)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::attestation::{Attestation, AttestationPayload};
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
fn fresh() -> SigningKey {
SigningKey::generate(&mut OsRng)
}
#[derive(Serialize, Deserialize)]
struct Demo {
a: String,
}
impl AttestationPayload for Demo {
fn payload_type() -> &'static str {
"demo/v1"
}
}
struct DemoVerifier;
impl AttestationVerifier for DemoVerifier {
fn type_(&self) -> &'static str {
"demo/v1"
}
fn verify_payload(&self, payload: &serde_json::Value) -> Result<()> {
payload
.get("a")
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.map(|_| ())
.ok_or_else(|| Error::Payload("missing or empty 'a'".to_string()))
}
}
/// A second verifier that has the same discriminator as `DemoVerifier` —
/// used to test last-wins on `register`.
struct DemoVerifierAlt;
impl AttestationVerifier for DemoVerifierAlt {
fn type_(&self) -> &'static str {
"demo/v1"
}
fn verify_payload(&self, _payload: &serde_json::Value) -> Result<()> {
Err(Error::Payload("alt always rejects".to_string()))
}
}
#[derive(Serialize, Deserialize)]
struct Other {
x: u32,
}
impl AttestationPayload for Other {
fn payload_type() -> &'static str {
"other/v1"
}
}
fn sample_hash() -> [u8; 32] {
let mut h = [0u8; 32];
for (i, b) in h.iter_mut().enumerate() {
*b = (i + 1) as u8;
}
h
}
fn signed_demo(sk: &SigningKey, cert_hash: [u8; 32], a: &str) -> Attestation {
Attestation::sign(sk, Demo { a: a.to_string() }, cert_hash).unwrap()
}
fn signed_other(sk: &SigningKey, cert_hash: [u8; 32], x: u32) -> Attestation {
Attestation::sign(sk, Other { x }, cert_hash).unwrap()
}
#[test]
fn registry_accepts_registered_type() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new();
reg.register(DemoVerifier);
let att = signed_demo(&sk, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
assert!(v.fully_verified);
assert_eq!(v.type_, "demo/v1");
assert_eq!(v.cert_hash, hex::encode(cert_hash));
}
#[test]
fn accept_known_lets_unknown_pass_unverified() {
let sk = fresh();
let cert_hash = sample_hash();
let reg = Registry::new().with_policy(Policy::AcceptKnown);
let att = signed_demo(&sk, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
assert!(!v.fully_verified);
}
#[test]
fn reject_unknown_blocks_unregistered_types() {
let sk = fresh();
let cert_hash = sample_hash();
let reg = Registry::new().with_policy(Policy::RejectUnknown);
let att = signed_demo(&sk, cert_hash, "ok");
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::UnknownType(_)));
}
#[test]
fn require_all_enforces_presence() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
let err = reg.verify_all(&[], cert_hash).unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
let att = signed_demo(&sk, cert_hash, "ok");
let v = reg.verify_all(&[att], cert_hash).unwrap();
assert_eq!(v.len(), 1);
}
/// Even when a junk attestation of an unrelated type is attached first, a
/// `RequireAll` registry must still accept the cert as long as the
/// required type is present and fully verified — the attacker shouldn't
/// be able to DoS the cert by attaching `attacker/spam/v1` before
/// `covenant/exec/v1`.
#[test]
fn require_all_is_lenient_on_unknown_types_in_the_batch() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
let spam = signed_other(&sk, cert_hash, 7);
let real = signed_demo(&sk, cert_hash, "ok");
let verified = reg.verify_all(&[spam, real], cert_hash).unwrap();
assert_eq!(verified.len(), 2);
assert!(verified
.iter()
.any(|v| v.type_ == "demo/v1" && v.fully_verified));
assert!(verified
.iter()
.any(|v| v.type_ == "other/v1" && !v.fully_verified));
}
/// A required type that is present but unverified (no verifier registered
/// even though the type appears in `required_types`) must fail
/// `verify_all` with `RequiredMissing`. Catches a misconfiguration where
/// the operator listed a type but forgot to register a verifier.
#[test]
fn require_all_rejects_present_but_unverified_required_type() {
let sk = fresh();
let cert_hash = sample_hash();
let reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]); // type listed, but no verifier registered.
let att = signed_demo(&sk, cert_hash, "ok");
let err = reg.verify_all(&[att], cert_hash).unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
#[test]
fn payload_check_failure_rejects() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new();
reg.register(DemoVerifier);
// Sign an empty `a` so the signature verifies but the payload check fails.
let att = Attestation::sign(&sk, Demo { a: String::new() }, cert_hash).unwrap();
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// `RequireAll` with an empty allowlist degenerates to "no required types
/// to check" — the loop over `required_types` is a no-op, so an empty
/// batch verifies successfully. Pinned so operators relying on the empty
/// allowlist as a default-permissive RequireAll see consistent behavior.
#[test]
fn require_all_with_empty_allowlist_and_empty_batch_succeeds() {
let reg = Registry::new().with_policy(Policy::RequireAll);
let verified = reg.verify_all(&[], sample_hash()).unwrap();
assert!(verified.is_empty());
}
#[test]
fn register_returns_the_prior_verifier_on_double_register() {
let mut reg = Registry::new();
assert!(reg.register(DemoVerifier).is_none());
let prior = reg.register(DemoVerifierAlt);
let prior = prior.expect("second register must return the previous verifier");
assert_eq!(prior.type_(), "demo/v1");
// Last-wins semantics confirmed by behavior.
let sk = fresh();
let cert_hash = sample_hash();
let att = signed_demo(&sk, cert_hash, "ok");
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
}
@@ -0,0 +1,150 @@
//! Canonical test vectors.
//!
//! Ed25519 signatures are deterministic (RFC 8032). JCS canonicalization is
//! deterministic (RFC 8785). SHA-256 is deterministic. Given fixed inputs the
//! cert hash, signature, signer DID, and full wire JSON are therefore exactly
//! reproducible across implementations. This file pins those values so a
//! port to Go, JS, Python, or any other language has a concrete oracle to
//! check against without reading Rust.
//!
//! If a vector changes, that is a wire-protocol break. The change is either
//! intentional (bump `ATTEST_ENVELOPE_VERSION` and document the migration) or
//! a regression that broke interoperability with every other implementation.
use ed25519_dalek::SigningKey;
use gitlawb_attest::{Attestation, AttestationPayload, AttestedRefUpdateCert, Registry, CERT_TYPE};
use serde::{Deserialize, Serialize};
/// Fixed seed for the attestation signer. Real signers use a CSPRNG; this is a
/// test-vector convention only.
const SIGNER_SEED: [u8; 32] = [
0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef, 0xfe, 0xdc, 0xba, 0x98, 0x76, 0x54, 0x32, 0x10,
0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff,
];
/// Fixed cert body for the vectors. Mirrors the field layout of a
/// `gitlawb-core::cert::RefUpdateBody` plus a `signatures` array. The
/// signatures array is deliberately non-empty to test that it is stripped
/// before hashing.
fn fixture_cert_body() -> serde_json::Value {
serde_json::json!({
"type": CERT_TYPE,
"repo": "did:key:z6MkpYqTAt8jq3SExeUiVdyTrDdocy9j2NjGxFE6S6ZBh1Yt",
"ref_name": "refs/heads/main",
"from": "0000000000000000000000000000000000000000000000000000000000000000",
"to": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"seq": 1,
"timestamp": "2026-01-01T00:00:00Z",
"nonce": "vector-nonce-1",
"signatures": [
{"signer": "did:key:z6MkpYqTAt8jq3SExeUiVdyTrDdocy9j2NjGxFE6S6ZBh1Yt", "sig": "ignored"}
]
})
}
/// Fixed attestation payload. Shape mirrors what Covenant's runtime would
/// produce for a `covenant/exec/v1` attestation.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct ExecPayload {
agent: String,
capability: String,
sandbox_digest: String,
}
impl AttestationPayload for ExecPayload {
fn payload_type() -> &'static str {
"covenant/exec/v1"
}
}
fn fixture_payload() -> ExecPayload {
ExecPayload {
agent: "did:key:z6MkAgentTestVector".to_string(),
capability: "code.write+sandbox.exec".to_string(),
sandbox_digest: "sha256:c0ffee".to_string(),
}
}
/// The expected lowercase-hex SHA-256 of `JCS(cert_body \ {signatures})`.
/// Any port that produces a different value is not JCS-canonicalizing the
/// same way (likely culprits: key ordering, integer or string escaping).
const EXPECTED_CERT_HASH_HEX: &str =
"5898d7c7a7d32a5faaaf9a52bfc92e0eff98027554adeefba0c575816f43ce56";
/// The expected `did:key` of the signer derived from `SIGNER_SEED`. Any port
/// that produces a different value has a multibase or multicodec encoding bug.
const EXPECTED_SIGNER_DID: &str = "did:key:z6MkqkKY69DYo23W4YFDoCNgjr7cMvmJqHtJVoQkEzrbaopZ";
/// The expected base64url-no-pad Ed25519 signature over
/// `b"gitlawb-attest-sig/v1\n" || JCS({type, payload, cert_hash})`. Any port
/// that produces a different value is missing the domain tag, JCS-encoding
/// the input differently, or using a different Ed25519 implementation.
const EXPECTED_SIGNATURE_B64URL: &str =
"4L3ONbhrksB9pn--u86lMeAdePSiZZplAsxuN2Cy6vgI30ndle7gDUVuoGRJr4Ylhz7ipQvUzXaDTvZGFLPXBQ";
#[test]
fn cert_hash_is_deterministic_from_jcs() {
let env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let actual_hex = hex::encode(env.cert_hash().unwrap());
assert_eq!(
actual_hex, EXPECTED_CERT_HASH_HEX,
"cert hash drifted — JCS encoding or hash algorithm changed",
);
}
#[test]
fn signer_did_is_deterministic_from_seed() {
let sk = SigningKey::from_bytes(&SIGNER_SEED);
let env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let cert_hash = env.cert_hash().unwrap();
let att = Attestation::sign(&sk, fixture_payload(), cert_hash).unwrap();
assert_eq!(
att.signer, EXPECTED_SIGNER_DID,
"signer DID drifted — multibase or multicodec encoding changed",
);
}
#[test]
fn attestation_signature_is_deterministic() {
let sk = SigningKey::from_bytes(&SIGNER_SEED);
let env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let cert_hash = env.cert_hash().unwrap();
let att = Attestation::sign(&sk, fixture_payload(), cert_hash).unwrap();
assert_eq!(
att.sig, EXPECTED_SIGNATURE_B64URL,
"signature drifted — domain tag, JCS shape, or Ed25519 implementation changed",
);
}
#[test]
fn vector_attestation_verifies_through_the_public_api() {
// Round-trip the entire pipeline using only the public API on the fixed
// vectors. A non-Rust implementor should be able to:
// 1. Compute cert_hash and check against EXPECTED_CERT_HASH_HEX.
// 2. Derive signer DID and check against EXPECTED_SIGNER_DID.
// 3. Sign a payload identical to fixture_payload() and check against
// EXPECTED_SIGNATURE_B64URL.
// 4. Reconstruct the wire envelope and verify it round-trips.
let sk = SigningKey::from_bytes(&SIGNER_SEED);
let mut env = AttestedRefUpdateCert::from_cert(&fixture_cert_body()).unwrap();
let cert_hash = env.cert_hash().unwrap();
let att = Attestation::sign(&sk, fixture_payload(), cert_hash).unwrap();
env.attach(att);
let wire = serde_json::to_string(&env).unwrap();
let back: AttestedRefUpdateCert = serde_json::from_str(&wire).unwrap();
assert_eq!(back.attestations.len(), 1);
assert_eq!(back.attestations[0].sig, EXPECTED_SIGNATURE_B64URL);
let recovered = back.attestations[0]
.verify_signature(back.cert_hash().unwrap())
.unwrap();
assert_eq!(recovered.to_bytes(), sk.verifying_key().to_bytes());
// And through a registry, with the convenience method.
let reg = Registry::new();
let verified = back.verify_attestations(&reg).unwrap();
assert_eq!(verified.len(), 1);
assert_eq!(verified[0].cert_hash, EXPECTED_CERT_HASH_HEX);
assert_eq!(verified[0].signer, EXPECTED_SIGNER_DID);
}
@@ -0,0 +1,251 @@
//! End-to-end integration with `gitlawb-core`.
//!
//! Proves the attest envelope composes with the real `RefUpdateCert` from
//! `gitlawb-core`: a cert built and signed through the core API wraps cleanly,
//! survives signing and verifying an attestation, and round-trips back to a
//! `RefUpdateCert` that core still verifies. None of this is exercised by the
//! unit tests in `cert.rs`, which use hand-written JSON literals to keep the
//! crate's runtime deps minimal.
use ed25519_dalek::SigningKey;
use gitlawb_attest::{
Attestation, AttestationPayload, AttestationVerifier, AttestedRefUpdateCert, Error, Policy,
Registry, Result as AttestResult,
};
use gitlawb_core::cert::RefUpdateCert;
use gitlawb_core::identity::Keypair;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
fn zero_hash() -> String {
"0".repeat(64)
}
fn commit_hash(c: char) -> String {
c.to_string().repeat(64)
}
fn fresh_signing_key() -> SigningKey {
SigningKey::generate(&mut OsRng)
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
struct CovenantExecPayload {
agent: String,
capability: String,
sandbox_digest: String,
}
impl AttestationPayload for CovenantExecPayload {
fn payload_type() -> &'static str {
"covenant/exec/v1"
}
}
struct CovenantExecVerifier;
impl AttestationVerifier for CovenantExecVerifier {
fn type_(&self) -> &'static str {
"covenant/exec/v1"
}
fn verify_payload(&self, payload: &serde_json::Value) -> AttestResult<()> {
for field in ["agent", "capability", "sandbox_digest"] {
payload
.get(field)
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
.ok_or_else(|| Error::Payload(format!("missing or empty '{field}'")))?;
}
Ok(())
}
}
#[test]
fn wraps_real_cert_signs_and_verifies_attestation_then_unwraps_cleanly() {
// 1. Build a real cert through gitlawb-core. The pusher is the repo owner
// in this minimal setup; the cert is single-signature.
let pusher = Keypair::generate();
let repo_did = pusher.did();
let cert = RefUpdateCert::new(
repo_did,
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('a'),
1,
&pusher,
)
.unwrap();
cert.validate_structure().unwrap();
let valid_signers = cert.verify_all().unwrap();
assert_eq!(valid_signers.len(), 1);
// 2. Wrap. With no attestations attached the envelope is byte-identical to
// the underlying cert — that's the additive-on-the-wire promise.
let cert_json = serde_json::to_value(&cert).unwrap();
let mut env = AttestedRefUpdateCert::from_cert(&cert).unwrap();
assert_eq!(
serde_json::to_value(&env).unwrap(),
cert_json,
"empty envelope must serialize as the bare cert"
);
// 3. A separate party (here, an agent runtime) attests to how the commit
// was produced. The attestation signer is intentionally not the cert
// pusher: trust in the attestation is evaluated independently of trust
// in the push.
let agent_signer = fresh_signing_key();
let payload = CovenantExecPayload {
agent: "did:key:z6MkAgent".to_string(),
capability: "code.write+sandbox.exec".to_string(),
sandbox_digest: "sha256:c0ffee".to_string(),
};
let cert_hash = env.cert_hash().unwrap();
let attestation = Attestation::sign(&agent_signer, payload.clone(), cert_hash).unwrap();
env.attach(attestation);
// 4. Verify through a registry that knows `covenant/exec/v1`. Strict
// policy so an unknown discriminator would have been rejected.
let mut registry = Registry::new().with_policy(Policy::RejectUnknown);
registry.register(CovenantExecVerifier);
let verified = registry
.verify_all(&env.attestations, env.cert_hash().unwrap())
.unwrap();
assert_eq!(verified.len(), 1);
assert!(verified[0].fully_verified);
assert_eq!(verified[0].type_, "covenant/exec/v1");
assert_eq!(verified[0].cert_hash, hex::encode(cert_hash));
// 5. Round-trip through JSON and back. The wrapped cert with one
// attestation must parse back into the same envelope, and stripping
// the attestations must yield a `RefUpdateCert` core still verifies.
let on_wire = serde_json::to_string(&env).unwrap();
let back: AttestedRefUpdateCert = serde_json::from_str(&on_wire).unwrap();
assert_eq!(back.attestations.len(), 1);
assert_eq!(
back.attestations[0]
.payload_as::<CovenantExecPayload>()
.unwrap(),
payload
);
let mut unwrapped = back.cert.clone();
if let Some(obj) = unwrapped.as_object_mut() {
obj.remove("attestations");
}
let unwrapped_cert: RefUpdateCert = serde_json::from_value(unwrapped).unwrap();
let revalidated = unwrapped_cert.verify_all().unwrap();
assert_eq!(revalidated, valid_signers);
}
#[test]
fn cert_hash_is_stable_when_a_maintainer_countersigns() {
// The cert hash strips `signatures` before hashing, so countersignatures
// added after an attestation is signed must not invalidate the
// attestation's binding.
let pusher = Keypair::generate();
let maintainer = Keypair::generate();
let mut cert = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('b'),
7,
&pusher,
)
.unwrap();
let env_before = AttestedRefUpdateCert::from_cert(&cert).unwrap();
let hash_before = env_before.cert_hash().unwrap();
cert.countersign(&maintainer).unwrap();
let env_after = AttestedRefUpdateCert::from_cert(&cert).unwrap();
let hash_after = env_after.cert_hash().unwrap();
assert_eq!(
hash_before, hash_after,
"cert hash must be stable across countersignatures"
);
// An attestation signed against the pre-countersign hash still verifies
// against the post-countersign cert.
let agent = fresh_signing_key();
let payload = CovenantExecPayload {
agent: "did:key:z6MkAgent2".to_string(),
capability: "code.write".to_string(),
sandbox_digest: "sha256:deadbeef".to_string(),
};
let attestation = Attestation::sign(&agent, payload, hash_before).unwrap();
attestation.verify_signature(hash_after).unwrap();
}
#[test]
fn cross_cert_replay_is_rejected_against_a_real_cert() {
// Negative case: an attestation signed for one cert must not verify on a
// different cert built through the core API.
let pusher = Keypair::generate();
let cert_a = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('a'),
1,
&pusher,
)
.unwrap();
let cert_b = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
commit_hash('a'),
commit_hash('b'),
2,
&pusher,
)
.unwrap();
let env_a = AttestedRefUpdateCert::from_cert(&cert_a).unwrap();
let env_b = AttestedRefUpdateCert::from_cert(&cert_b).unwrap();
let hash_a = env_a.cert_hash().unwrap();
let hash_b = env_b.cert_hash().unwrap();
assert_ne!(hash_a, hash_b);
let agent = fresh_signing_key();
let payload = CovenantExecPayload {
agent: "did:key:z6MkAgent3".to_string(),
capability: "code.write".to_string(),
sandbox_digest: "sha256:cafe".to_string(),
};
let attestation = Attestation::sign(&agent, payload, hash_a).unwrap();
let err = attestation.verify_signature(hash_b).unwrap_err();
assert!(matches!(err, Error::CertHashMismatch));
}
/// A cert deserialized from JSON and a cert built through the core API must
/// hash identically. JCS canonicalization is what makes the additive-on-wire
/// promise hold across the serialize/deserialize boundary — without it, the
/// hash would depend on which side built the `serde_json::Value`.
#[test]
fn cert_hash_is_identical_across_the_deserialize_boundary() {
let pusher = Keypair::generate();
let cert = RefUpdateCert::new(
pusher.did(),
"refs/heads/main".to_string(),
zero_hash(),
commit_hash('c'),
9,
&pusher,
)
.unwrap();
let direct_hash = AttestedRefUpdateCert::from_cert(&cert)
.unwrap()
.cert_hash()
.unwrap();
let wire = serde_json::to_string(&cert).unwrap();
let parsed_value: serde_json::Value = serde_json::from_str(&wire).unwrap();
let parsed_env: AttestedRefUpdateCert = serde_json::from_value(parsed_value).unwrap();
let parsed_hash = parsed_env.cert_hash().unwrap();
assert_eq!(direct_hash, parsed_hash);
}
+4
View File
@@ -3,6 +3,7 @@ name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
@@ -23,6 +24,9 @@ chrono = { workspace = true }
uuid = { workspace = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
crypto_box = { version = "0.9", features = ["std", "chacha20"] }
chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
+101
View File
@@ -257,4 +257,105 @@ mod tests {
// Round-trip
let _: RefUpdateCert = serde_json::from_str(&json).unwrap();
}
#[test]
fn validate_structure_rejects_wrong_cert_type() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.body.type_ = "gitlawb/ref-update/v0".to_string();
assert!(cert.validate_structure().is_err());
}
#[test]
fn validate_structure_rejects_invalid_to_hash() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.body.to = "not-a-valid-sha256".to_string();
assert!(cert.validate_structure().is_err());
}
#[test]
fn validate_structure_rejects_empty_signatures() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures.clear();
assert!(cert.validate_structure().is_err());
}
#[test]
fn tampered_signature_fails_verify_all() {
let kp = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
// Replace kp's signature with a signature from kp2 over different bytes.
// The signer DID still points to kp, so verification must fail.
cert.signatures[0].sig = kp2.sign_b64(b"unrelated bytes");
assert!(cert.verify_all().is_err());
}
#[test]
fn threshold_not_met_when_signer_outside_maintainer_list() {
let kp_pusher = Keypair::generate();
let kp_maintainer = Keypair::generate();
let cert = RefUpdateCert::new(
kp_pusher.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp_pusher,
)
.unwrap();
let maintainers = vec![kp_maintainer.did()];
assert!(!cert.satisfies_threshold(&maintainers, 1).unwrap());
}
#[test]
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
assert!(cert.satisfies_threshold(&[], 0).unwrap());
}
}
+57
View File
@@ -129,4 +129,61 @@ mod tests {
let bytes = sha256_hex_to_bytes(&hex).unwrap();
assert_eq!(sha256_bytes(data), bytes);
}
#[test]
fn from_sha256_bytes_matches_object_bytes() {
// from_sha256_bytes must produce the same CID as from_git_object_bytes
// when given the SHA-256 of those bytes — the two construction paths
// must be equivalent for self-verifying content addressing to hold.
let data = b"blob 5\0hello";
let cid_from_object = Cid::from_git_object_bytes(data);
let hash = sha256_bytes(data);
let cid_from_hash = Cid::from_sha256_bytes(&hash);
assert_eq!(cid_from_object, cid_from_hash);
}
#[test]
fn from_str_parses_valid_cid() {
let data = b"tree content for test";
let cid = Cid::from_git_object_bytes(data);
let parsed = Cid::from_str(cid.as_str()).unwrap();
assert_eq!(cid, parsed);
}
#[test]
fn from_str_rejects_invalid_string() {
let result = Cid::from_str("not-a-valid-cid");
assert!(result.is_err());
}
#[test]
fn different_data_produces_different_cids() {
let c1 = Cid::from_git_object_bytes(b"blob 5\0hello");
let c2 = Cid::from_git_object_bytes(b"blob 5\0world");
assert_ne!(c1, c2);
}
#[test]
fn sha256_hex_to_bytes_rejects_non_hex() {
let non_hex = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz";
assert_eq!(non_hex.len(), 64);
let result = sha256_hex_to_bytes(non_hex);
assert!(result.is_err());
}
#[test]
fn sha256_hex_to_bytes_rejects_wrong_length() {
let result = sha256_hex_to_bytes("deadbeef");
assert!(result.is_err());
}
#[test]
fn sha256_hex_of_empty_input_is_well_known() {
// SHA-256("") is a fixed constant; verifies the hasher is wired correctly.
let h = sha256_hex(b"");
assert_eq!(
h,
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
);
}
}
+56
View File
@@ -252,4 +252,60 @@ mod tests {
let json = serde_json::to_string_pretty(&doc).unwrap();
assert!(json.contains("Ed25519VerificationKey2020"));
}
#[test]
fn to_verifying_key_fails_for_did_web() {
let web = Did::web("example.com");
let result = web.to_verifying_key();
assert!(
result.is_err(),
"did:web cannot resolve to a verifying key locally"
);
}
#[test]
fn to_verifying_key_fails_for_did_gitlawb() {
let gl = Did::gitlawb("z6MkSomeKey");
let result = gl.to_verifying_key();
assert!(
result.is_err(),
"did:gitlawb cannot resolve to a verifying key locally"
);
}
#[test]
fn from_str_rejects_missing_did_prefix() {
let result = "key:z6MkABCD".parse::<Did>();
assert!(result.is_err());
}
#[test]
fn from_str_rejects_unsupported_method() {
let result = "did:ethr:0x1234abcd".parse::<Did>();
assert!(result.is_err());
}
#[test]
fn validate_accepts_all_supported_methods() {
Did::web("example.com").validate().unwrap();
Did::gitlawb("z6MkSomeKey").validate().unwrap();
Keypair::generate().did().validate().unwrap();
}
#[test]
fn is_did_key_and_is_did_gitlawb_predicates() {
let key_did = Keypair::generate().did();
assert!(key_did.is_did_key());
assert!(!key_did.is_did_gitlawb());
let gl = Did::gitlawb("z6MkSomeKey");
assert!(!gl.is_did_key());
assert!(gl.is_did_gitlawb());
}
#[test]
fn method_id_extraction() {
assert_eq!(Did::web("example.com").method_id(), "example.com");
assert_eq!(Did::gitlawb("z6MkSomeKey").method_id(), "z6MkSomeKey");
}
}
+296
View File
@@ -0,0 +1,296 @@
//! Envelope encryption for withheld blobs (Option B). A random content key
//! encrypts the blob (XChaCha20-Poly1305); the content key is wrapped to each
//! recipient via an X25519 box keyed from their Ed25519 `did:key`. The node
//! seals with public keys only; readers open with their own private key.
use crate::identity::Keypair;
use anyhow::{Context, Result};
use ed25519_dalek::VerifyingKey;
use zeroize::Zeroizing;
/// X25519 public key (Montgomery u) for an Ed25519 verifying key.
fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
use curve25519_dalek::edwards::CompressedEdwardsY;
let edwards = CompressedEdwardsY::from_slice(vk.as_bytes())
.ok()
.and_then(|c| c.decompress())
.context("verifying key is not a valid edwards point")?;
Ok(edwards.to_montgomery().to_bytes())
}
/// X25519 secret scalar for an Ed25519 seed (SHA-512 of seed, lower 32, clamped).
/// Returns the scalar wrapped in `Zeroizing`, and scrubs the intermediate
/// SHA-512 digest, so no copy of this secret material lingers in freed memory.
fn x25519_secret_from_seed(seed: &[u8; 32]) -> Zeroizing<[u8; 32]> {
use sha2::{Digest, Sha512};
use zeroize::Zeroize;
let mut h = Sha512::digest(seed);
let mut s = Zeroizing::new([0u8; 32]);
s.copy_from_slice(&h[..32]);
s[0] &= 248;
s[31] &= 127;
s[31] |= 64;
h.as_mut_slice().zeroize();
s
}
use base64::{engine::general_purpose::STANDARD as B64, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit},
XChaCha20Poly1305, XNonce,
};
use crypto_box::{
aead::{AeadCore, OsRng},
ChaChaBox, PublicKey as XPublic, SecretKey as XSecret,
};
use rand::RngCore;
use serde::{Deserialize, Serialize};
const MAGIC: &[u8] = b"GLENC";
const VERSION: u8 = 2;
#[derive(Serialize, Deserialize)]
struct Recipient {
eph: String, // base64 ephemeral x25519 pubkey (32B)
nonce: String, // base64 box nonce (24B)
wrap: String, // base64 wrapped content key
}
#[derive(Serialize, Deserialize)]
struct Header {
alg: String,
nonce: String, // base64 body nonce (24B)
recipients: Vec<Recipient>,
}
/// Encrypt `plaintext` so any of `recipients` (Ed25519 keys) can decrypt.
pub fn seal_blob(plaintext: &[u8], recipients: &[VerifyingKey]) -> Result<Vec<u8>> {
if recipients.is_empty() {
return Err(anyhow::anyhow!("seal_blob: no recipients"));
}
let mut content_key = [0u8; 32];
OsRng.fill_bytes(&mut content_key);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
.map_err(|e| anyhow::anyhow!("content key: {e}"))?;
let mut body_nonce = [0u8; 24];
OsRng.fill_bytes(&mut body_nonce);
let body = body_cipher
.encrypt(XNonce::from_slice(&body_nonce), plaintext)
.map_err(|e| anyhow::anyhow!("body encrypt: {e}"))?;
let mut wrapped = Vec::with_capacity(recipients.len());
for vk in recipients {
let recip_x = XPublic::from(x25519_public(vk)?);
let eph = XSecret::generate(&mut OsRng);
let abox = ChaChaBox::new(&recip_x, &eph);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let ct = abox
.encrypt(&nonce, &content_key[..])
.map_err(|e| anyhow::anyhow!("wrap: {e}"))?;
wrapped.push(Recipient {
eph: B64.encode(eph.public_key().as_bytes()),
nonce: B64.encode(nonce),
wrap: B64.encode(ct),
});
}
let header = Header {
alg: "xchacha20poly1305".into(),
nonce: B64.encode(body_nonce),
recipients: wrapped,
};
let header_json = serde_json::to_vec(&header).context("encode header")?;
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
out.extend_from_slice(&header_json);
out.extend_from_slice(&body);
Ok(out)
}
/// Decrypt an envelope with `keypair`. Errors if not a recipient or on auth fail.
pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
let mut p = 0;
if envelope.len() < MAGIC.len() + 1 + 4 || &envelope[..MAGIC.len()] != MAGIC {
return Err(anyhow::anyhow!("bad envelope magic"));
}
p += MAGIC.len();
if envelope[p] != VERSION {
return Err(anyhow::anyhow!("unsupported envelope version"));
}
p += 1;
let hlen = u32::from_le_bytes(envelope[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header: Header =
serde_json::from_slice(envelope.get(p..p + hlen).context("truncated header")?)
.context("decode header")?;
let body = &envelope[p + hlen..];
let my_x = XSecret::from(*x25519_secret_from_seed(&keypair.to_seed()));
// Identities are blinded: no entry says which recipient it belongs to, so
// try each one. The ChaChaBox AEAD tag authenticates, so exactly the
// reader's own entry unwraps; every other entry fails cleanly.
let mut content_key: Option<Vec<u8>> = None;
for entry in &header.recipients {
let eph = match B64
.decode(&entry.eph)
.ok()
.and_then(|b| <[u8; 32]>::try_from(b.as_slice()).ok())
{
Some(b) => XPublic::from(b),
None => continue,
};
// from_slice panics on a wrong length, and the envelope is attacker
// controlled, so validate the 24-byte box nonce before using it.
let nonce = match B64
.decode(&entry.nonce)
.ok()
.and_then(|n| <[u8; 24]>::try_from(n.as_slice()).ok())
{
Some(n) => n,
None => continue,
};
let wrap = match B64.decode(&entry.wrap) {
Ok(w) => w,
Err(_) => continue,
};
let abox = ChaChaBox::new(&eph, &my_x);
if let Ok(ck) = abox.decrypt(
crypto_box::aead::generic_array::GenericArray::from_slice(&nonce),
wrap.as_slice(),
) {
content_key = Some(ck);
break;
}
}
let content_key = content_key.context("not a recipient of this envelope")?;
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
.map_err(|e| anyhow::anyhow!("content key: {e}"))?;
let body_nonce = B64
.decode(&header.nonce)
.ok()
.and_then(|n| <[u8; 24]>::try_from(n.as_slice()).ok())
.context("invalid body nonce")?;
body_cipher
.decrypt(XNonce::from_slice(&body_nonce), body)
.map_err(|_| anyhow::anyhow!("body decrypt failed"))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::identity::Keypair;
#[test]
fn ed25519_to_x25519_keypair_agrees() {
// The X25519 public derived from the Ed25519 public must equal the
// X25519 public of the X25519 secret derived from the same seed.
let kp = Keypair::generate();
let seed = kp.to_seed();
let xpub_from_public = x25519_public(&kp.verifying_key()).unwrap();
let xsec = x25519_secret_from_seed(&seed);
let xpub_from_secret = crypto_box::SecretKey::from(*xsec).public_key().to_bytes();
assert_eq!(xpub_from_public, xpub_from_secret);
}
#[test]
fn seal_open_round_trip_for_recipients() {
let owner = Keypair::generate();
let reader_a = Keypair::generate();
let reader_b = Keypair::generate();
let msg = b"private blob contents";
let env = seal_blob(msg, &[owner.verifying_key(), reader_a.verifying_key()]).unwrap();
assert_eq!(open_blob(&env, &owner).unwrap(), msg);
assert_eq!(open_blob(&env, &reader_a).unwrap(), msg);
assert!(
open_blob(&env, &reader_b).is_err(),
"non-recipient must fail"
);
}
#[test]
fn tampered_envelope_fails() {
let owner = Keypair::generate();
let mut env = seal_blob(b"hi", &[owner.verifying_key()]).unwrap();
let last = env.len() - 1;
env[last] ^= 0x01;
assert!(open_blob(&env, &owner).is_err());
}
#[test]
fn v2_header_contains_no_recipient_pubkey() {
// The blinded envelope header must not carry any recipient's public key.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Slice out the header bytes using the envelope framing:
// MAGIC | version(1B) | header_len(4B LE) | header_json | body
let mut p = MAGIC.len() + 1; // skip MAGIC + version byte
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header = &env[p..p + hlen];
let header_str = String::from_utf8_lossy(header);
let pubkey_b64 = B64.encode(reader.verifying_key().as_bytes());
assert!(
!header_str.contains(&pubkey_b64),
"recipient public key must not appear in the blinded header"
);
}
#[test]
fn v1_envelope_is_rejected() {
let reader = Keypair::generate();
let mut env = seal_blob(b"hi", &[reader.verifying_key()]).unwrap();
// Flip the version byte (immediately after MAGIC) from 2 to 1.
env[MAGIC.len()] = 1;
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("unsupported envelope version"),
"expected version-rejection error, got: {err}"
);
}
#[test]
fn malformed_nonce_returns_err_not_panic() {
// from_slice panics on wrong-length input; a crafted envelope on the
// public recovery path must surface an error, never panic.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Split the envelope framing into header JSON and body.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
let reframe = |header: &serde_json::Value| -> Vec<u8> {
let hj = serde_json::to_vec(header).unwrap();
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(hj.len() as u32).to_le_bytes());
out.extend_from_slice(&hj);
out.extend_from_slice(body);
out
};
let bad_nonce = serde_json::Value::String(B64.encode([0u8; 12]));
// Corrupted per-recipient nonce: entry is skipped, no match.
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
header["recipients"][0]["nonce"] = bad_nonce.clone();
assert!(open_blob(&reframe(&header), &reader).is_err());
// Corrupted body nonce: unwrap succeeds, body nonce is rejected.
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
header["nonce"] = bad_nonce;
assert!(open_blob(&reframe(&header), &reader).is_err());
}
}
+79 -1
View File
@@ -52,7 +52,9 @@ impl Keypair {
URL_SAFE_NO_PAD.encode(sig.to_bytes())
}
/// Export the signing key as raw 32-byte seed (wrapped in Zeroizing).
/// The raw 32-byte Ed25519 seed, wrapped in `Zeroizing` so the copy is
/// scrubbed on drop. This is the only seed accessor; it is used to derive
/// the X25519 secret for envelope decryption (see `crate::encrypt`).
pub fn to_seed(&self) -> Zeroizing<[u8; 32]> {
Zeroizing::new(self.signing_key.to_bytes())
}
@@ -157,4 +159,80 @@ mod tests {
let did = kp.did();
assert!(did.to_string().starts_with("did:key:z6Mk"));
}
#[test]
fn from_seed_round_trip() {
let kp = Keypair::generate();
let seed = kp.to_seed();
let kp2 = Keypair::from_seed(&seed).unwrap();
assert_eq!(kp.verifying_key(), kp2.verifying_key());
}
// `to_seed()` is the only seed accessor on `Keypair` (the raw, unzeroized
// `seed_bytes()` was removed in #41). Pin its return type so a bare-array
// accessor can't slip back in, and confirm it is deterministic. The
// from_seed -> verifying_key round-trip is already covered by
// `from_seed_round_trip`.
#[test]
fn to_seed_returns_zeroizing_and_is_deterministic() {
let kp = Keypair::generate();
// The type annotation is the regression guard: the seed must be handed
// out wrapped in `Zeroizing`, never as a bare `[u8; 32]`.
let seed: Zeroizing<[u8; 32]> = kp.to_seed();
assert_eq!(*kp.to_seed(), *seed); // stable across calls
}
#[test]
fn sign_b64_decodes_to_valid_signature() {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let kp = Keypair::generate();
let msg = b"test payload for b64 signing";
let b64 = kp.sign_b64(msg);
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&b64)
.expect("sign_b64 must produce valid base64url")
.try_into()
.expect("Ed25519 signature must be 64 bytes");
verify(&kp.verifying_key(), msg, &sig_bytes).unwrap();
}
#[test]
fn verify_rejects_wrong_key() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let msg = b"some message";
let sig = kp1.sign(msg);
let result = verify(&kp2.verifying_key(), msg, &sig.to_bytes());
assert!(
result.is_err(),
"signature from kp1 must not verify under kp2"
);
}
#[test]
fn signed_payload_round_trip() {
let kp = Keypair::generate();
let payload = serde_json::json!({"action": "push", "repo": "my-repo"});
let signed = Signed::new(payload, &kp).unwrap();
signed.verify(&kp.verifying_key()).unwrap();
}
#[test]
fn signed_payload_wrong_key_fails() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let payload = serde_json::json!({"action": "push"});
let signed = Signed::new(payload, &kp1).unwrap();
let result = signed.verify(&kp2.verifying_key());
assert!(
result.is_err(),
"Signed payload must not verify under a different key"
);
}
#[test]
fn from_pem_invalid_input_fails() {
let result = Keypair::from_pem("this is not valid PEM at all");
assert!(result.is_err());
}
}
+1
View File
@@ -1,6 +1,7 @@
pub mod cert;
pub mod cid;
pub mod did;
pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
+202
View File
@@ -41,6 +41,20 @@ impl Capability {
self.constraints = Some(constraints);
self
}
/// Returns `true` if `self` is a valid attenuation of `parent`.
///
/// A delegated capability is only valid if it is at most as permissive as
/// the parent capability backing it. `"*"` on the **parent**'s resource or
/// action field and `repo/admin` in the parent's action position act as
/// wildcards that cover any delegated value; wildcards on `self` carry no
/// special meaning.
pub fn is_attenuated_by(&self, parent: &Capability) -> bool {
let resource_ok = parent.with == self.with || parent.with == "*";
let action_ok =
parent.can == self.can || parent.can == "*" || parent.can == caps::REPO_ADMIN;
resource_ok && action_ok
}
}
/// Well-known gitlawb capability strings.
@@ -132,6 +146,26 @@ impl Ucan {
}
}
/// Check if this UCAN's not-before time is in the future (token not yet valid).
pub fn is_before_valid(&self) -> bool {
if let Some(nbf) = self.payload.nbf {
Utc::now().timestamp() < nbf
} else {
false
}
}
/// Verify this UCAN's audience matches `expected`.
pub fn verify_audience(&self, expected: &Did) -> Result<()> {
if &self.payload.aud != expected {
return Err(Error::Ucan(format!(
"audience mismatch: expected {expected}, got {}",
self.payload.aud
)));
}
Ok(())
}
/// Verify the signature on this UCAN.
pub fn verify_signature(&self) -> Result<()> {
use crate::identity::verify;
@@ -217,6 +251,10 @@ impl Ucan {
return Err(Error::Ucan("token is expired".to_string()));
}
if self.is_before_valid() {
return Err(Error::Ucan("token is not yet valid".to_string()));
}
for proof_token in &self.payload.prf {
let proof = Self::decode(proof_token)
.map_err(|e| Error::Ucan(format!("failed to decode proof: {e}")))?;
@@ -229,6 +267,17 @@ impl Ucan {
)));
}
// Every delegated capability must be covered by the proof (attenuation).
for cap in &self.payload.att {
let covered = proof.payload.att.iter().any(|p| cap.is_attenuated_by(p));
if !covered {
return Err(Error::Ucan(format!(
"capability attenuation violated: '{}' on '{}' not covered by proof",
cap.can, cap.with
)));
}
}
// Verify the proof's signature and chain recursively
proof.verify_chain()?;
}
@@ -399,4 +448,157 @@ mod tests {
let err = delegated.verify_chain().unwrap_err();
assert!(err.to_string().contains("expired"));
}
#[test]
fn is_before_valid_future_nbf() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let nbf_future = chrono::Utc::now() + chrono::Duration::hours(1);
let payload = UcanPayload {
ucan: "1.0.0".to_string(),
iss: issuer.did(),
aud: audience,
att: vec![],
exp: None,
nbf: Some(nbf_future.timestamp()),
prf: vec![],
};
let signing_bytes = serde_json::to_vec(&payload).unwrap();
let sig = issuer.sign_b64(&signing_bytes);
let ucan = Ucan { payload, s: sig };
assert!(ucan.is_before_valid());
let err = ucan.verify_chain().unwrap_err();
assert!(err.to_string().contains("not yet valid"));
}
#[test]
fn is_before_valid_past_nbf() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let nbf_past = chrono::Utc::now() - chrono::Duration::hours(1);
let payload = UcanPayload {
ucan: "1.0.0".to_string(),
iss: issuer.did(),
aud: audience,
att: vec![Capability::new("gitlawb://repos/test", caps::GIT_PUSH)],
exp: None,
nbf: Some(nbf_past.timestamp()),
prf: vec![],
};
let signing_bytes = serde_json::to_vec(&payload).unwrap();
let sig = issuer.sign_b64(&signing_bytes);
let ucan = Ucan { payload, s: sig };
assert!(!ucan.is_before_valid());
ucan.verify_chain().unwrap();
}
#[test]
fn verify_audience_matches() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(&issuer, audience.clone(), vec![], None).unwrap();
ucan.verify_audience(&audience).unwrap();
}
#[test]
fn verify_audience_mismatch() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let wrong = Keypair::generate().did();
let ucan = Ucan::issue(&issuer, audience, vec![], None).unwrap();
let err = ucan.verify_audience(&wrong).unwrap_err();
assert!(err.to_string().contains("audience mismatch"));
}
#[test]
fn attenuation_valid_subset() {
let alice = Keypair::generate();
let bob = Keypair::generate();
let charlie = Keypair::generate();
// Alice grants Bob push on a specific repo
let root = Ucan::issue(
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
)
.unwrap();
// Bob delegates the same capability (exact subset) to Charlie
let delegated = Ucan::delegate(
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
&root,
)
.unwrap();
delegated.verify_chain().unwrap();
}
#[test]
fn attenuation_exceeds_parent_is_rejected() {
let alice = Keypair::generate();
let bob = Keypair::generate();
let charlie = Keypair::generate();
// Alice grants Bob push on one repo only
let root = Ucan::issue(
&alice,
bob.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
)
.unwrap();
// Bob tries to delegate merge (not in the original grant) to Charlie
let delegated = Ucan::delegate(
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::PR_MERGE)],
None,
&root,
)
.unwrap();
let err = delegated.verify_chain().unwrap_err();
assert!(err.to_string().contains("attenuation violated"));
}
#[test]
fn attenuation_repo_admin_covers_all() {
let alice = Keypair::generate();
let bob = Keypair::generate();
let charlie = Keypair::generate();
// Alice grants Bob repo/admin (superpower)
let root = Ucan::issue(
&alice,
bob.did(),
vec![Capability::new(
"gitlawb://repos/org/repo",
caps::REPO_ADMIN,
)],
None,
)
.unwrap();
// Bob delegates a more specific capability — covered by repo/admin
let delegated = Ucan::delegate(
&bob,
charlie.did(),
vec![Capability::new("gitlawb://repos/org/repo", caps::GIT_PUSH)],
None,
&root,
)
.unwrap();
delegated.verify_chain().unwrap();
}
}
+16 -2
View File
@@ -3,6 +3,7 @@ name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
@@ -11,6 +12,7 @@ path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
ed25519-dalek = { workspace = true }
tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
@@ -37,13 +39,24 @@ http-body-util = "0.1"
tokio-util = { version = "0.7", features = ["io"] }
dirs-next = "2"
reqwest = { workspace = true }
libp2p = { version = "0.55", features = ["gossipsub", "kad", "mdns", "noise", "tcp", "tokio", "yamux", "identify", "macros"] }
libp2p-core = "0.43"
libp2p-gossipsub = "0.49.4"
libp2p-identify = "0.47"
libp2p-identity = { version = "0.2", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.48"
libp2p-quic = { version = "0.13", features = ["tokio"] }
libp2p-swarm = { version = "0.47", features = ["macros", "tokio"] }
multiaddr = "0.18"
futures = "0.3"
aws-sdk-s3 = "1"
aws-sdk-s3 = { version = "1", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
aws-config = { version = "1", features = ["behavior-version-latest"] }
async-compression = { version = "0.4", features = ["tokio", "zstd"] }
tar = "0.4"
zstd = "0.13"
# Prometheus metrics. Used to expose a /metrics endpoint for ops/observability
# on the opt-in GITLAWB_METRICS_ADDR listener. The crate is also the de-facto
# exposition format encoder in the Rust ecosystem.
prometheus = { version = "0.13", default-features = false }
alloy = { version = "1", default-features = false, features = [
"contract",
"provider-http",
@@ -54,6 +67,7 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
libp2p-dns = { version = "0.44.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
+2 -1
View File
@@ -316,6 +316,7 @@ pub async fn cancel_bounty(
/// POST /api/v1/bounties/{id}/dispute
pub async fn dispute_bounty(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let bounty = state
@@ -344,7 +345,7 @@ pub async fn dispute_bounty(
}
state.db.dispute_bounty(&id).await?;
tracing::info!(bounty_id = %id, "bounty disputed — reopened");
tracing::info!(bounty_id = %id, actor = %auth.0, "bounty disputed - reopened");
let updated = state
.db
+6 -6
View File
@@ -1,9 +1,10 @@
//! Changelog endpoint — unified timeline of commits, merged PRs, and closed issues.
use axum::extract::{Path, Query, State};
use axum::extract::{Extension, Path, Query, State};
use axum::Json;
use serde::Deserialize;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::git::store;
use crate::state::AppState;
@@ -27,12 +28,11 @@ pub async fn get_changelog(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(query): Query<ChangelogQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let limit = query.limit.min(100);
+123
View File
@@ -0,0 +1,123 @@
//! Authenticated discovery + fetch for encrypted withheld blobs (Option B1).
use axum::extract::{Extension, Path, State};
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs
/// Returns [{oid, cid}] for every encrypted blob in the repo, to any caller who
/// can read the repo. Not recipient-scoped: recipient identities are not stored,
/// so access control here is repo readability and decryption is gated by the
/// envelope crypto (only a real recipient can open an envelope).
pub async fn list_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
.map(|(oid, cid)| serde_json::json!({ "oid": oid, "cid": cid }))
.collect();
Ok(Json(serde_json::json!({ "blobs": blobs })))
}
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blob/{oid}
/// Returns raw envelope bytes to callers who can read the repo; the envelope
/// crypto still ensures only true recipients can decrypt.
pub async fn get_encrypted_blob(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo, oid)): Path<(String, String, String)>,
) -> Result<Vec<u8>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")));
}
let cid = state
.db
.encrypted_blob_cid(&record.id, &oid)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")))?;
let bytes = crate::ipfs_pin::cat(&state.config.ipfs_api, &cid)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
Ok(bytes)
}
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate
/// Returns [{oid, cid}] for every encrypted blob in the repo, for peer-mirror
/// replication (Option B2). Gated by repo readability, like discovery, so a
/// non-readable repo does not expose its blob index; for the intended case (a
/// public repo with withheld subtrees) the public root keeps this open to peers.
/// Recipient identities are deliberately withheld: the v2 envelopes no longer
/// carry recipient public keys, so peers must not learn the reader set either. A
/// mirror detects a re-seal by the CID changing (the OID is stable across
/// re-seals). Ciphertext metadata only, never plaintext.
pub async fn replicate_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
.map(|(oid, cid)| replicate_blob_json(oid, cid))
.collect();
Ok(Json(serde_json::json!({ "blobs": blobs })))
}
/// Serialize one blob for the replication wire. Recipient identities are
/// intentionally absent so a mirror never learns the reader set.
fn replicate_blob_json(oid: String, cid: String) -> serde_json::Value {
serde_json::json!({ "oid": oid, "cid": cid })
}
#[cfg(test)]
mod tests {
use super::replicate_blob_json;
#[test]
fn replicate_blob_json_omits_recipients() {
let v = replicate_blob_json("oid1".into(), "cidA".into());
assert_eq!(v["oid"], "oid1");
assert_eq!(v["cid"], "cidA");
assert!(
v.get("recipients").is_none(),
"replication wire must not carry recipient identities"
);
}
}
+4 -4
View File
@@ -70,8 +70,8 @@ pub async fn create_issue(
let create_result = git_issues::create_issue(&disk_path, &issue_id, &json_str);
// Always release the advisory lock — even on error
guard.release().await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(create_result.is_ok()).await;
create_result.map_err(|e| AppError::Git(e.to_string()))?;
@@ -225,8 +225,8 @@ pub async fn close_issue(
let close_result = git_issues::close_issue(&disk_path, &issue_id);
// Always release the advisory lock — even on error
guard.release().await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(close_result.is_ok()).await;
let updated = close_result
.map_err(|e| AppError::Git(e.to_string()))?
+37
View File
@@ -1,13 +1,20 @@
use crate::db::{RepoRecord, VisibilityRule};
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
pub mod agents;
pub mod arweave;
pub mod bounties;
pub mod certs;
pub mod changelog;
pub mod encrypted;
pub mod events;
pub mod ipfs;
pub mod issues;
pub mod labels;
pub mod peers;
pub mod profiles;
pub mod protect;
pub mod pulls;
pub mod register;
@@ -16,4 +23,34 @@ pub mod repos;
pub mod resolve;
pub mod stars;
pub mod tasks;
pub mod visibility;
pub mod webhooks;
/// Resolve a repo for a read request and enforce path-scoped visibility.
///
/// Returns 404 (`RepoNotFound`) if the repo does not exist or the caller may not
/// read `path`, using the same opaque response the git serve path returns so
/// existence is not confirmed. Returns the record and its visibility rules so a
/// content handler can apply an extra per-path check without a second DB query.
///
/// Callers pass `"/"` for repo-level reads (listings); content endpoints pass the
/// specific path so a withheld subtree is denied even on an otherwise-public repo.
pub(crate) async fn authorize_repo_read(
state: &AppState,
owner: &str,
name: &str,
caller: Option<&str>,
path: &str,
) -> Result<(RepoRecord, Vec<VisibilityRule>)> {
let record = state
.db
.get_repo(owner, name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, path) == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
Ok((record, rules))
}
+108 -10
View File
@@ -10,11 +10,13 @@
//! POST /api/v1/sync/notify — receive push notification from a peer node
//! POST /api/v1/sync/trigger — manually pull all repos from known peers
use axum::extract::{Path, State};
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use gitlawb_core::did::Did;
use serde::{Deserialize, Serialize};
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
@@ -61,8 +63,26 @@ pub async fn list_peers(State(state): State<AppState>) -> Result<Json<serde_json
/// The Authorization header's DID must match the `did` in the body.
pub async fn announce(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Json(req): Json<AnnounceRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let announced_did: Did = req
.did
.parse()
.map_err(|e: gitlawb_core::Error| AppError::BadRequest(e.to_string()))?;
if let Some(Extension(auth)) = auth {
if auth.0 != announced_did.to_string() {
return Err(AppError::BadRequest(
"Signature keyid must match announced DID".into(),
));
}
} else {
tracing::warn!(
did = %announced_did,
"accepted unsigned peer announce; set GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true after all peers upgrade"
);
}
// Validate the URL is HTTP/HTTPS
if !req.http_url.starts_with("http://") && !req.http_url.starts_with("https://") {
return Err(AppError::BadRequest(
@@ -70,6 +90,23 @@ pub async fn announce(
));
}
// Reject self-announcements: a peer row whose http_url is our own public
// URL makes the HTTP-notify path fan out to ourselves. Seen in prod when
// misconfigured dev nodes announce with their upstream's URL.
// prune_self_peers clears stale rows at boot; this stops new ones.
if let Some(self_url) = state.config.public_url.as_deref() {
if req.http_url.trim_end_matches('/') == self_url.trim_end_matches('/') {
return Err(AppError::BadRequest(
"http_url is this node's own public URL; refusing to register self as peer".into(),
));
}
}
if announced_did.to_string() == state.node_did.to_string() {
return Err(AppError::BadRequest(
"did is this node's own DID; refusing to register self as peer".into(),
));
}
state.db.upsert_peer(&req.did, &req.http_url).await?;
tracing::info!(did = %req.did, url = %req.http_url, "peer announced");
@@ -104,16 +141,30 @@ pub async fn trigger_sync(State(state): State<AppState>) -> Result<Json<serde_js
continue;
}
let url = format!("{}/api/v1/repos", peer.http_url.trim_end_matches('/'));
let result =
tokio::time::timeout(std::time::Duration::from_secs(5), client.get(&url).send()).await;
let repos: Vec<serde_json::Value> = match result {
Ok(Ok(resp)) if resp.status().is_success() => {
peers_reached += 1;
resp.json().await.unwrap_or_default()
// 30s with the body read inside the timeout: 5s only covered the
// response headers, so canonical nodes serving large unpaginated repo
// lists (and transpacific round trips) aborted mid-body.
let result = tokio::time::timeout(std::time::Duration::from_secs(30), async {
let resp = client.get(&url).send().await?;
if !resp.status().is_success() {
return Err(anyhow::anyhow!("peer returned status {}", resp.status()));
}
_ => {
tracing::warn!(peer = %peer.did, "trigger_sync: could not reach peer");
let repos: Vec<serde_json::Value> = resp.json().await?;
Ok::<_, anyhow::Error>(repos)
})
.await;
let repos = match result {
Ok(Ok(repos)) => {
peers_reached += 1;
repos
}
Ok(Err(e)) => {
tracing::warn!(peer = %peer.did, err = %e, "trigger_sync: peer fetch failed");
continue;
}
Err(_) => {
tracing::warn!(peer = %peer.did, "trigger_sync: peer timed out");
continue;
}
};
@@ -163,12 +214,38 @@ pub struct NotifyRequest {
pub ref_name: String,
pub new_sha: String,
pub node_did: String,
// Optional fields — older senders only included the four above. New
// senders include these so received_ref_updates has full provenance
// even when the libp2p mesh isn't delivering and the HTTP fallback
// is the only path that fired.
#[serde(default)]
pub pusher_did: Option<String>,
#[serde(default)]
pub old_sha: Option<String>,
#[serde(default)]
pub timestamp: Option<String>,
#[serde(default)]
pub cert_id: Option<String>,
}
pub async fn notify_sync(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Json(req): Json<NotifyRequest>,
) -> Result<Json<serde_json::Value>> {
if let Some(Extension(auth)) = auth {
if auth.0 != req.node_did {
return Err(AppError::BadRequest(
"Signature keyid must match notification node_did".into(),
));
}
} else {
tracing::warn!(
did = %req.node_did,
"accepted unsigned sync notify; set GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true after all peers upgrade"
);
}
// Only accept notifications from known peers
let peers = state.db.list_peers().await?;
let known = peers.iter().any(|p| p.did == req.node_did);
@@ -184,6 +261,27 @@ pub async fn notify_sync(
.enqueue_sync(&req.repo, &req.node_did, &req.ref_name, &req.new_sha, None)
.await?;
// Mirror the gossipsub-receive handler: insert the same record we'd
// get from the libp2p path, so /api/v1/events/ref-updates reflects
// pushes that arrive over either transport.
let now = chrono::Utc::now().to_rfc3339();
let update = crate::db::ReceivedRefUpdate {
id: uuid::Uuid::new_v4().to_string(),
node_did: req.node_did.clone(),
pusher_did: req.pusher_did.clone().unwrap_or_default(),
repo: req.repo.clone(),
ref_name: req.ref_name.clone(),
old_sha: req.old_sha.clone().unwrap_or_default(),
new_sha: req.new_sha.clone(),
timestamp: req.timestamp.clone().unwrap_or_else(|| now.clone()),
cert_id: req.cert_id.clone(),
received_at: now,
from_peer: format!("http:{}", req.node_did),
};
if let Err(e) = state.db.insert_ref_update(&update).await {
tracing::warn!(err = %e, repo = %req.repo, "failed to insert ref-update from sync notify");
}
tracing::info!(
repo = %req.repo,
peer = %req.node_did,
+171
View File
@@ -0,0 +1,171 @@
//! Agent profile API handlers.
//!
//! - `PUT /api/v1/profile` — upsert the caller's profile (requires HTTP Signature)
//! - `GET /api/v1/agents/{did}/profile` — read any agent's profile (public)
use axum::{
extract::{Path, State},
http::StatusCode,
Json,
};
use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use crate::auth::AuthenticatedDid;
use crate::pinata;
use crate::state::AppState;
#[derive(Debug, Deserialize)]
pub struct SetProfileRequest {
pub display_name: Option<String>,
pub bio: Option<String>,
pub avatar_url: Option<String>,
pub website: Option<String>,
pub socials: Option<SocialsInput>,
pub pin_to_ipfs: Option<bool>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct SocialsInput {
pub twitter: Option<String>,
pub github: Option<String>,
pub farcaster: Option<String>,
pub telegram: Option<String>,
}
pub async fn set_profile(
State(state): State<AppState>,
axum::Extension(auth): axum::Extension<AuthenticatedDid>,
Json(req): Json<SetProfileRequest>,
) -> Result<(StatusCode, Json<Value>), (StatusCode, Json<Value>)> {
let did = auth.0;
if let Some(ref bio) = req.bio {
if bio.len() > 280 {
return Err((
StatusCode::BAD_REQUEST,
Json(json!({ "message": "bio must be 280 characters or fewer" })),
));
}
}
if let Some(ref name) = req.display_name {
if name.len() > 50 {
return Err((
StatusCode::BAD_REQUEST,
Json(json!({ "message": "display_name must be 50 characters or fewer" })),
));
}
}
let socials_json = req
.socials
.as_ref()
.map(|s| serde_json::to_string(s).unwrap_or_else(|_| "{}".to_string()));
let result = state
.db
.upsert_profile(
&did,
req.display_name.as_deref(),
req.bio.as_deref(),
req.avatar_url.as_deref(),
req.website.as_deref(),
socials_json.as_deref(),
)
.await;
match result {
Ok(profile) => {
let mut resp = json!({
"did": profile.did,
"display_name": profile.display_name,
"bio": profile.bio,
"avatar_url": profile.avatar_url,
"website": profile.website,
"updated_at": profile.updated_at,
});
if let Some(ref socials_str) = profile.socials {
if let Ok(socials) = serde_json::from_str::<Value>(socials_str) {
resp["socials"] = socials;
}
}
if req.pin_to_ipfs.unwrap_or(false) && !state.config.pinata_jwt.is_empty() {
let profile_json = serde_json::to_vec(&resp).unwrap_or_default();
match pinata::pin_object(
&state.http_client,
&state.config.pinata_upload_url,
&state.config.pinata_jwt,
&format!("profile-{}", &did),
&profile_json,
)
.await
{
Ok(cid) if !cid.is_empty() => {
if let Err(e) = state.db.set_profile_cid(&did, &cid).await {
tracing::warn!(did = %did, err = %e, "failed to store profile CID");
}
resp["profile_cid"] = json!(cid);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(did = %did, err = %e, "IPFS pin failed — profile saved without CID");
}
}
} else if let Some(cid) = profile.profile_cid {
resp["profile_cid"] = json!(cid);
}
Ok((StatusCode::OK, Json(resp)))
}
Err(e) => {
tracing::error!(did = %did, error = %e, "failed to upsert profile");
Err((
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "message": "failed to save profile" })),
))
}
}
}
pub async fn get_profile(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<Json<Value>, (StatusCode, Json<Value>)> {
let profile = state.db.get_profile(&did).await.map_err(|e| {
tracing::error!(did = %did, error = %e, "failed to fetch profile");
(
StatusCode::INTERNAL_SERVER_ERROR,
Json(json!({ "message": "failed to fetch profile" })),
)
})?;
match profile {
Some(p) => {
let mut resp = json!({
"did": p.did,
"display_name": p.display_name,
"bio": p.bio,
"avatar_url": p.avatar_url,
"website": p.website,
"profile_cid": p.profile_cid,
"created_at": p.created_at,
"updated_at": p.updated_at,
});
if let Some(ref socials_str) = p.socials {
if let Ok(socials) = serde_json::from_str::<Value>(socials_str) {
resp["socials"] = socials;
}
}
Ok(Json(resp))
}
None => Err((
StatusCode::NOT_FOUND,
Json(json!({ "message": "profile not found" })),
)),
}
}
+41 -27
View File
@@ -101,12 +101,11 @@ pub async fn create_pr(
pub async fn list_prs(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let prs = state.db.list_prs(&record.id).await?;
Ok(Json(
@@ -118,12 +117,11 @@ pub async fn list_prs(
pub async fn get_pr(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<PullRequest>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
@@ -138,12 +136,11 @@ pub async fn get_pr(
pub async fn get_pr_diff(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
@@ -156,6 +153,25 @@ pub async fn get_pr_diff(
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
// Withhold the entire diff if it touches a path the caller cannot read, so a
// PR diff cannot leak private-subtree content of an otherwise-public repo.
let touched = store::branch_diff_names(&disk_path, &pr.target_branch, &pr.source_branch)
.map_err(|e| AppError::Git(e.to_string()))?;
for p in &touched {
let gate = format!("/{}", p.trim_start_matches('/'));
if crate::visibility::visibility_check(
&rules,
record.is_public,
&record.owner_did,
caller,
&gate,
) == crate::visibility::Decision::Deny
{
return Err(AppError::NotFound(format!("PR #{number} not found")));
}
}
let diff = store::branch_diff(&disk_path, &pr.target_branch, &pr.source_branch)
.map_err(|e| AppError::Git(e.to_string()))?;
@@ -203,8 +219,8 @@ pub async fn merge_pr(
&pr.title,
);
// Always release the advisory lock — even on error
guard.release().await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(merge_result.is_ok()).await;
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
@@ -324,12 +340,11 @@ pub async fn create_review(
pub async fn list_reviews(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
@@ -383,12 +398,11 @@ pub async fn create_comment(
pub async fn list_comments(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
+346 -105
View File
@@ -12,8 +12,9 @@ use uuid::Uuid;
use crate::cert;
use crate::error::{AppError, Result};
use crate::git::{smart_http, store};
use crate::git::{smart_http, store, visibility_pack};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
use crate::webhooks;
// ── Request / Response types ───────────────────────────────────────────────
@@ -185,12 +186,11 @@ pub async fn list_repos(
pub async fn get_repo(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<RepoResponse>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let count = state.db.count_stars(&record.id).await.unwrap_or(0);
Ok(Json(to_response(&record, &state, count)))
}
@@ -199,12 +199,11 @@ pub async fn get_repo(
pub async fn list_commits(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let disk_path = state
.repo_store
@@ -221,15 +220,27 @@ pub async fn list_commits(
pub async fn get_blob(
State(state): State<AppState>,
Path((owner, name, file_path)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Response> {
use axum::http::header;
use axum::response::IntoResponse;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Unnormalized paths ("../..", "./", "//") can't resolve in `git show`
// and crawlers combinatorially explode them from relative links — that's
// a client error, not a 500.
let file_path = file_path.trim_matches('/');
if file_path.is_empty()
|| file_path
.split('/')
.any(|seg| seg.is_empty() || seg == "." || seg == "..")
{
return Err(AppError::BadRequest("invalid file path".into()));
}
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let gate_path = format!("/{file_path}");
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, &gate_path).await?;
let disk_path = state
.repo_store
@@ -237,8 +248,19 @@ pub async fn get_blob(
.await
.map_err(|e| AppError::Git(e.to_string()))?;
let head_ref = store::resolve_head(&disk_path, &record.default_branch);
let content = store::read_file(&disk_path, &head_ref, &file_path)
.map_err(|e| AppError::Git(e.to_string()))?;
let content = store::read_file(&disk_path, &head_ref, file_path).map_err(|e| {
let msg = e.to_string();
// `git show ref:path` on a path absent from the tree is a 404,
// not a server error
if msg.contains("does not exist in")
|| msg.contains("invalid object name")
|| msg.contains("exists on disk, but not in")
{
AppError::NotFound(format!("file not found: {file_path}"))
} else {
AppError::Git(msg)
}
})?;
// Guess content type
let mime = match file_path.rsplit('.').next() {
@@ -259,12 +281,11 @@ pub async fn get_blob(
pub async fn get_tree_root(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let disk_path = state
.repo_store
@@ -281,12 +302,11 @@ pub async fn get_tree_root(
pub async fn get_tree(
State(state): State<AppState>,
Path((owner, name, tree_path)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let disk_path = state
.repo_store
@@ -308,6 +328,7 @@ pub async fn git_info_refs(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(query): Query<InfoRefsQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
tracing::info!(owner = %owner, repo = %name, "info/refs request");
@@ -322,6 +343,22 @@ pub async fn git_info_refs(
.ok_or_else(|| AppError::BadRequest("missing ?service= parameter".into()))?;
tracing::debug!(service = %service, repo = %name, "info/refs service");
// Enforce read (clone/fetch) visibility. The push advertisement
// (service=git-receive-pack) is authorized separately on the
// git-receive-pack POST, so leave it untouched here.
if service == "git-upload-pack" {
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
// Subtree (mode B) rules do not gate the advertisement: refs expose commit
// tips only, and blob withholding happens in the upload-pack pack build.
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/")
== Decision::Deny
{
tracing::debug!(repo = %name, caller = ?caller, "info/refs read denied by visibility");
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
}
// For receive-pack (push), download the latest from Tigris so the client
// sees the same refs that acquire_write() will operate on.
let disk_path = if service == "git-receive-pack" {
@@ -352,6 +389,7 @@ pub async fn git_info_refs(
pub async fn git_upload_pack(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
body: Bytes,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
@@ -361,23 +399,62 @@ pub async fn git_upload_pack(
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
tracing::debug!(repo = %name, caller = ?caller, "upload-pack read denied by visibility");
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
smart_http::upload_pack(&disk_path, body)
.await
.map_err(|e| {
let msg = e.to_string();
if msg.contains("bad line length") || msg.contains("protocol error") {
tracing::warn!(repo = %name, err = %msg, "git-upload-pack: bad client request");
AppError::BadRequest(msg)
} else {
tracing::error!(repo = %name, err = %msg, "git-upload-pack failed");
AppError::Git(msg)
}
let body_len = body.len();
// withheld_blob_oids walks every ref with blocking `git ls-tree`; keep that
// off the async worker thread.
let withheld = {
let path = disk_path.clone();
let rules = rules.clone();
let owner_did = record.owner_did.clone();
let caller_owned = caller.map(str::to_string);
let is_public = record.is_public;
tokio::task::spawn_blocking(move || {
visibility_pack::withheld_blob_oids(
&path,
&rules,
is_public,
&owner_did,
caller_owned.as_deref(),
)
})
.await
.map_err(|e| AppError::Git(e.to_string()))?
.map_err(|e| AppError::Git(e.to_string()))?
};
let resp = if withheld.is_empty() {
smart_http::upload_pack(&disk_path, body).await
} else {
tracing::info!(repo = %name, caller = ?caller, withheld = withheld.len(), "serving filtered pack");
smart_http::upload_pack_excluding(&disk_path, body, &withheld).await
}
.map_err(|e| {
let msg = e.to_string();
if msg.contains("bad line length") || msg.contains("protocol error") {
tracing::warn!(repo = %name, err = %msg, "git-upload-pack: bad client request");
AppError::BadRequest(msg)
} else {
tracing::error!(repo = %name, err = %msg, "git-upload-pack failed");
AppError::Git(msg)
}
})?;
crate::metrics::record_fetch(&format!("{owner}/{name}"));
crate::metrics::observe_pack_size(body_len as f64);
Ok(resp)
}
/// POST /:owner/:repo.git/git-receive-pack (AUTH REQUIRED — enforced by middleware)
@@ -451,11 +528,13 @@ pub async fn git_receive_pack(
})?;
let disk_path = guard.path().to_path_buf();
tracing::debug!(repo = %name, path = %disk_path.display(), "running git receive-pack");
let body_len = body.len();
let receive_result = smart_http::receive_pack(&disk_path, body).await;
// Always release the advisory lock — even on error — to prevent stale locks
// from blocking subsequent pushes to the same repo.
guard.release().await;
// from blocking subsequent pushes. Only upload to Tigris when the push
// succeeded; uploading a half-applied repo would propagate corruption.
guard.release(receive_result.is_ok()).await;
let result = receive_result.map_err(|e| {
tracing::error!(repo = %name, err = %e, "git receive-pack failed");
@@ -465,6 +544,11 @@ pub async fn git_receive_pack(
// Update the repo's updated_at timestamp after a successful push
let _ = state.db.touch_repo(&record.id).await;
// Record the successful push for metrics. The body has already been
// consumed by smart_http::receive_pack so we observe size up front.
crate::metrics::record_push(&record.id);
crate::metrics::observe_pack_size(body_len as f64);
// Record push event for trust score and issue a signed ref certificate
let pusher_did = extract_did_from_auth(&headers);
if let Some(ref did) = pusher_did {
@@ -544,20 +628,142 @@ pub async fn git_receive_pack(
}
}
// Pin new git objects to the local IPFS node (no-op if ipfs_api is empty)
{
// Replication enforcement (Phase 2): decide once per push whether the public
// may read this repo at all and, if so, which blob OIDs must not leave the
// node. `withheld == None` means replicate nothing (private / mode A /
// undetermined): skip every pin so even commit and tree objects (which
// withheld_blob_oids never lists) stay local. `announce` gates the
// network-facing announcements. Fail closed: a private or undetermined repo
// never leaks.
let rules_opt = state.db.list_visibility_rules(&record.id).await.ok();
let announce = match &rules_opt {
Some(rules) => {
visibility_check(rules, record.is_public, &record.owner_did, None, "/")
== Decision::Allow
}
None => false,
};
let withheld: Option<std::collections::HashSet<String>> = if !announce {
None
} else {
match &rules_opt {
Some(rules) if rules.is_empty() => Some(std::collections::HashSet::new()),
// withheld_blob_oids walks every ref with blocking `git ls-tree`;
// keep that off the async worker thread.
Some(rules) => {
let path = disk_path.clone();
let rules = rules.clone();
let owner_did = record.owner_did.clone();
let is_public = record.is_public;
tokio::task::spawn_blocking(move || {
crate::git::visibility_pack::withheld_blob_oids(
&path, &rules, is_public, &owner_did, None,
)
})
.await
.map_err(|e| {
tracing::warn!(err = %e, "withheld_blob_oids task panicked; skipping replication for this push")
})
.ok()
.and_then(|r| {
r.map_err(|e| {
tracing::warn!(err = %e, "withheld_blob_oids failed; skipping replication for this push")
})
.ok()
})
}
None => None,
}
};
// Pin new git objects to the local IPFS node (no-op if ipfs_api is empty).
// Skipped entirely when the public cannot read the repo (withheld == None).
if let Some(withheld_ipfs) = withheld.clone() {
let ipfs_api = state.config.ipfs_api.clone();
let repo_path_clone = disk_path.clone();
let db_clone = state.db.clone();
let rules_for_enc = rules_opt.clone();
let repo_id = record.id.clone();
let owner_did = record.owner_did.clone();
let is_public = record.is_public;
let irys_url = state.config.irys_url.clone();
let http_client = std::sync::Arc::clone(&state.http_client);
let node_did_str = state.node_did.to_string();
let node_seed = state.node_keypair.to_seed();
let repo_name = record.name.clone();
tokio::spawn(async move {
let pinned =
crate::ipfs_pin::pin_new_objects(&ipfs_api, &repo_path_clone, &db_clone).await;
let pinned = crate::ipfs_pin::pin_new_objects(
&ipfs_api,
&repo_path_clone,
&db_clone,
&withheld_ipfs,
)
.await;
if !pinned.is_empty() {
tracing::info!(count = pinned.len(), "pinned git objects to IPFS");
for (sha, cid) in &pinned {
tracing::info!(sha = %sha, %cid, "pinned");
}
}
// Option B1: encrypt-then-pin the withheld blobs so authorized
// readers can recover them when the origin cannot serve them.
if let Some(rules) = rules_for_enc.filter(|r| !r.is_empty()) {
let p = repo_path_clone.clone();
let owner = owner_did.clone();
let recip = tokio::task::spawn_blocking(move || {
crate::git::visibility_pack::withheld_blob_recipients(
&p, &rules, is_public, &owner,
)
})
.await;
if let Ok(Ok(recipients)) = recip {
let delta = crate::encrypted_pin::encrypt_and_pin(
&ipfs_api,
&repo_path_clone,
&db_clone,
&repo_id,
&node_seed,
&recipients,
)
.await;
// Option B3: anchor a per-push manifest of the blobs sealed
// this push to Arweave, so the oid->cid index survives total
// node loss. Best-effort; never fails the push.
if !delta.is_empty() && !irys_url.is_empty() {
let owner_short = owner_did.split(':').next_back().unwrap_or(&owner_did);
let repo_slug = format!("{owner_short}/{repo_name}");
let ts = chrono::Utc::now().to_rfc3339();
let manifest = crate::arweave::EncryptedManifest {
repo: &repo_slug,
owner_did: &owner_did,
node_did: &node_did_str,
timestamp: &ts,
blobs: &delta,
};
match crate::arweave::anchor_encrypted_manifest(
&http_client,
&irys_url,
&manifest,
)
.await
{
Ok(tx) if !tx.is_empty() => tracing::info!(
repo = %repo_slug,
tx_id = %tx,
"anchored encrypted manifest to Arweave"
),
Ok(_) => {}
Err(e) => tracing::warn!(
repo = %repo_slug,
err = %e,
"encrypted manifest anchor failed"
),
}
}
}
}
});
}
@@ -589,15 +795,23 @@ pub async fn git_receive_pack(
let irys_url = state.config.irys_url.clone();
let owner_did_for_arweave = record.owner_did.clone();
let self_public_url = state.config.public_url.clone();
let node_keypair = Arc::clone(&state.node_keypair);
let withheld_pinata = withheld;
tokio::spawn(async move {
let pinned = crate::pinata::pin_new_objects(
&http_client,
&pinata_upload_url,
&pinata_jwt,
&repo_path_clone,
&db_clone,
)
.await;
let pinned = match &withheld_pinata {
Some(withheld) => {
crate::pinata::pin_new_objects(
&http_client,
&pinata_upload_url,
&pinata_jwt,
&repo_path_clone,
&db_clone,
withheld,
)
.await
}
None => Vec::new(),
};
if !pinned.is_empty() {
tracing::info!(count = pinned.len(), "pinned git objects to Pinata");
@@ -616,54 +830,82 @@ pub async fn git_receive_pack(
.await;
}
if let Some(p2p) = &p2p_handle {
p2p.publish_ref_update(crate::p2p::RefUpdateEvent {
node_did: node_did_str.clone(),
pusher_did: pusher_did_clone.clone(),
repo: repo_slug.clone(),
ref_name: ref_name.clone(),
old_sha: "".to_string(),
new_sha: new_sha.clone(),
timestamp: chrono::Utc::now().to_rfc3339(),
cert_id: None,
cid: cid.map(|s| s.to_string()),
})
.await;
if announce {
if let Some(p2p) = &p2p_handle {
p2p.publish_ref_update(crate::p2p::RefUpdateEvent {
node_did: node_did_str.clone(),
pusher_did: pusher_did_clone.clone(),
repo: repo_slug.clone(),
ref_name: ref_name.clone(),
old_sha: "".to_string(),
new_sha: new_sha.clone(),
timestamp: chrono::Utc::now().to_rfc3339(),
cert_id: None,
cid: cid.map(|s| s.to_string()),
})
.await;
}
}
}
// HTTP peer notification — notify all known peers to pull from us.
// This is the reliable fallback when Gossipsub p2p is not yet connected.
if let Ok(peers) = db_for_peers.list_peers().await {
for peer in peers {
if peer.http_url.is_empty() {
continue;
}
let peer_url = peer.http_url.trim_end_matches('/');
if let Some(self_url) = self_public_url.as_deref() {
if peer_url == self_url.trim_end_matches('/') {
// Suppressed for repos the public cannot read.
if announce {
if let Ok(peers) = db_for_peers.list_peers().await {
for peer in peers {
if peer.http_url.is_empty() {
continue;
}
}
let notify_url = format!("{peer_url}/api/v1/sync/notify");
let body = serde_json::json!({
"repo": repo_slug,
"ref_name": ref_updates_clone.first().map(|(r, _)| r).unwrap_or(&String::new()),
"new_sha": ref_updates_clone.first().map(|(_, s)| s).unwrap_or(&String::new()),
"node_did": node_did_str,
"pusher_did": pusher_did_clone,
"old_sha": "0000000000000000000000000000000000000000",
"timestamp": chrono::Utc::now().to_rfc3339(),
});
match http_client.post(&notify_url).json(&body).send().await {
Ok(r) if r.status().is_success() => {
tracing::info!(peer = %peer.did, repo = %repo_slug, "notified peer to sync")
let peer_url = peer.http_url.trim_end_matches('/');
if let Some(self_url) = self_public_url.as_deref() {
if peer_url == self_url.trim_end_matches('/') {
continue;
}
}
Ok(r) => {
tracing::warn!(peer = %peer.did, status = %r.status(), "peer sync notify returned error")
}
Err(e) => {
tracing::warn!(peer = %peer.did, err = %e, "failed to notify peer")
let path = "/api/v1/sync/notify";
let notify_url = format!("{peer_url}{path}");
let body = serde_json::json!({
"repo": repo_slug.clone(),
"ref_name": ref_updates_clone.first().map(|(r, _)| r).unwrap_or(&String::new()),
"new_sha": ref_updates_clone.first().map(|(_, s)| s).unwrap_or(&String::new()),
"node_did": node_did_str.clone(),
"pusher_did": pusher_did_clone.clone(),
"old_sha": "0000000000000000000000000000000000000000",
"timestamp": chrono::Utc::now().to_rfc3339(),
});
let body_bytes = match serde_json::to_vec(&body) {
Ok(bytes) => bytes,
Err(e) => {
tracing::warn!(peer = %peer.did, err = %e, "failed to serialize peer sync notify");
continue;
}
};
let signed = gitlawb_core::http_sig::sign_request(
node_keypair.as_ref(),
"POST",
path,
&body_bytes,
);
match http_client
.post(&notify_url)
.header("Content-Type", "application/json")
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature)
.body(body_bytes)
.send()
.await
{
Ok(r) if r.status().is_success() => {
tracing::info!(peer = %peer.did, repo = %repo_slug, "notified peer to sync")
}
Ok(r) => {
tracing::warn!(peer = %peer.did, status = %r.status(), "peer sync notify returned error")
}
Err(e) => {
tracing::warn!(peer = %peer.did, err = %e, "failed to notify peer")
}
}
}
}
@@ -687,8 +929,9 @@ pub async fn git_receive_pack(
timestamp: now_ts.clone(),
});
// Arweave permanent anchoring — fire for each ref update
if !irys_url.is_empty() {
// Arweave permanent anchoring — fire for each ref update.
// Suppressed for repos the public cannot read (public permanent ledger).
if announce && !irys_url.is_empty() {
for (ref_name, new_sha) in &ref_updates_clone {
let cid = cid_map.get(new_sha).cloned();
let anchor = crate::arweave::RefAnchor {
@@ -737,12 +980,11 @@ pub async fn git_receive_pack(
pub async fn list_refs(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let _record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (_record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let repo_slug = format!("{owner}/{repo}");
let refs = state.db.list_branch_cids(&repo_slug).await?;
@@ -845,11 +1087,10 @@ pub async fn fork_repo(
Path((owner, name)): Path<(String, String)>,
Json(req): Json<ForkRepoRequest>,
) -> Result<(StatusCode, Json<RepoResponse>)> {
let source = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Enforce read visibility on the source before cloning: an unauthorized
// caller must not be able to fork (full mirror) a repo they cannot read.
let (source, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let fork_name = req.name.unwrap_or_else(|| source.name.clone());
let forker_did = auth.0;
+249
View File
@@ -0,0 +1,249 @@
//! Path-scoped visibility management API. Owner-only, mirrors `api/protect.rs`.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use serde::Deserialize;
use crate::auth::AuthenticatedDid;
use crate::db::VisibilityMode;
use crate::error::{AppError, Result};
use crate::state::AppState;
#[derive(Deserialize)]
pub struct SetVisibilityRequest {
pub path_glob: String,
/// "a" or "b"; defaults to "b" if absent or unrecognized.
#[serde(default)]
pub mode: Option<String>,
#[serde(default)]
pub reader_dids: Vec<String>,
}
#[derive(Deserialize)]
pub struct RemoveVisibilityRequest {
pub path_glob: String,
}
fn require_owner(record: &crate::db::RepoRecord, caller: &str) -> Result<()> {
let owner_short = record
.owner_did
.split(':')
.next_back()
.unwrap_or(&record.owner_did);
if caller != record.owner_did && caller != owner_short {
return Err(AppError::BadRequest(
"only the repo owner can manage visibility".into(),
));
}
Ok(())
}
/// Reject malformed globs before they reach the store, where they would
/// silently misconfigure access (an empty glob behaves like "/", and a glob
/// without a leading "/" never matches a real repo path). The accepted forms
/// match what `visibility_check` understands: "/", "/prefix", or "/prefix/**".
fn validate_path_glob(path_glob: &str) -> Result<()> {
if !path_glob.starts_with('/') {
return Err(AppError::BadRequest("path_glob must start with '/'".into()));
}
if path_glob == "/**" {
return Err(AppError::BadRequest(
"use '/' for whole-repo scope, not '/**'".into(),
));
}
if path_glob != "/" && path_glob.ends_with('/') {
return Err(AppError::BadRequest(
"path_glob must not end with '/'".into(),
));
}
if path_glob.contains('*') && !path_glob.ends_with("/**") {
return Err(AppError::BadRequest(
"the only supported wildcard is a trailing '/**'".into(),
));
}
Ok(())
}
/// PUT /api/v1/repos/{owner}/{repo}/visibility
pub async fn set_visibility(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<SetVisibilityRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
validate_path_glob(&req.path_glob)?;
let mode = match req.mode.as_deref() {
Some("a") => VisibilityMode::A,
_ => VisibilityMode::B,
};
// Mode A hides existence and is only coherent for the whole repo; a subtree
// cannot hide its existence without rewriting git history (see spec).
if mode == VisibilityMode::A && req.path_glob != "/" {
return Err(AppError::BadRequest(
"mode 'a' (hide) is only allowed for the whole repo (path_glob '/'); use mode 'b' for subtrees".into(),
));
}
// An empty reader_dids list is valid and intentional: the owner is always
// allowed by visibility_check, so a "/" rule with no readers is exactly the
// whole-repo "private to owner only" case.
state
.db
.set_visibility_rule(&record.id, &req.path_glob, mode, &req.reader_dids, &auth.0)
.await?;
let subtree_warning = req.path_glob != "/";
tracing::info!(
repo = %repo, caller = %auth.0, path_glob = %req.path_glob, mode = %mode.as_str(),
subtree_pending = subtree_warning,
"visibility rule set"
);
Ok((
StatusCode::CREATED,
Json(serde_json::json!({
"status": "set",
"repo": format!("{owner}/{repo}"),
"path_glob": req.path_glob,
"mode": mode.as_str(),
"reader_dids": req.reader_dids,
"subtree_clone_enforcement_pending": subtree_warning,
})),
))
}
/// DELETE /api/v1/repos/{owner}/{repo}/visibility
pub async fn remove_visibility(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
Json(req): Json<RemoveVisibilityRequest>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
state
.db
.remove_visibility_rule(&record.id, &req.path_glob)
.await?;
tracing::info!(
repo = %repo, caller = %auth.0, path_glob = %req.path_glob,
"visibility rule removed"
);
Ok(Json(serde_json::json!({
"status": "removed",
"repo": format!("{owner}/{repo}"),
"path_glob": req.path_glob,
})))
}
/// GET /api/v1/repos/{owner}/{repo}/visibility
pub async fn list_visibility(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let rules_json: Vec<_> = rules
.into_iter()
.map(|r| {
serde_json::json!({
"path_glob": r.path_glob,
"mode": r.mode.as_str(),
"reader_dids": r.reader_dids,
"created_by": r.created_by,
})
})
.collect();
Ok(Json(serde_json::json!({
"repo": format!("{owner}/{repo}"),
"count": rules_json.len(),
"rules": rules_json,
})))
}
/// GET /api/v1/repos/{owner}/{repo}/withheld-paths
///
/// Returns the path globs the (optionally authenticated) caller is denied
/// (`withheld`) plus any more-specific globs that are allowed underneath a
/// denied one (`reinclude`), so a clean-clone client can sparse-exclude the
/// denied subtrees while re-including the allowed nested paths. Unlike
/// `list_visibility` this is not owner-gated and never exposes reader_dids.
pub async fn withheld_paths(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
// Whole-repo read gate: a caller who cannot read "/" gets repo-not-found,
// matching the git read endpoints, so this never discloses a private repo's
// existence or its path layout to an unauthorized caller.
if crate::visibility::visibility_check(&rules, record.is_public, &record.owner_did, caller, "/")
== crate::visibility::Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let withheld =
crate::visibility::withheld_globs(&rules, record.is_public, &record.owner_did, caller);
let reinclude =
crate::visibility::reincluded_globs(&rules, record.is_public, &record.owner_did, caller);
Ok(Json(serde_json::json!({
"repo": format!("{owner}/{repo}"),
"withheld": withheld,
"reinclude": reinclude,
})))
}
#[cfg(test)]
mod tests {
use super::validate_path_glob;
#[test]
fn accepts_supported_globs() {
for g in ["/", "/secret", "/secret/**", "/a/b/c", "/a/b/**"] {
assert!(validate_path_glob(g).is_ok(), "{g} should be valid");
}
}
#[test]
fn rejects_malformed_globs() {
// empty, no leading slash, whole-repo via "/**", trailing slash, and
// non-trailing wildcards are all rejected.
for g in ["", "secret/**", "/**", "/secret/", "/a*b", "/*/x"] {
assert!(validate_path_glob(g).is_err(), "{g} should be rejected");
}
}
}
+175
View File
@@ -103,6 +103,108 @@ pub async fn anchor_ref_update(
Ok(tx_id)
}
/// A per-push manifest of the blobs encrypted this push (Option B3). The
/// `blobs` slice is `(oid, cid)` tuples. Anchored directly to Arweave as its JSON
/// body so the discovery index survives total node loss. Recipient identities are
/// never part of the manifest.
pub struct EncryptedManifest<'a> {
pub repo: &'a str,
pub owner_did: &'a str,
pub node_did: &'a str,
pub timestamp: &'a str,
pub blobs: &'a [(String, String)],
}
/// Anchor a per-push encrypted-blob manifest to Arweave via Irys. The manifest
/// JSON body is the payload (not a CID pointer to IPFS), so the index is
/// permanent and self-contained. Recipient identities are deliberately omitted:
/// the anchor is permanent and public, and the v2 envelopes no longer expose
/// recipients, so the reader set must not be written to Arweave either.
///
/// Returns the Irys/Arweave transaction ID, or `Ok("")` when `irys_url` is empty
/// (anchoring disabled) or there are no blobs to anchor.
pub async fn anchor_encrypted_manifest(
client: &reqwest::Client,
irys_url: &str,
manifest: &EncryptedManifest<'_>,
) -> Result<String> {
if irys_url.is_empty() || manifest.blobs.is_empty() {
return Ok(String::new());
}
let blobs_json: Vec<serde_json::Value> = manifest
.blobs
.iter()
.map(|(oid, cid)| manifest_blob_json(oid, cid))
.collect();
let payload = json!({
"schema": "gitlawb/encrypted-manifest/v1",
"repo": manifest.repo,
"owner_did": manifest.owner_did,
"node_did": manifest.node_did,
"timestamp": manifest.timestamp,
"blobs": blobs_json,
});
let body = serde_json::to_vec(&payload)?;
let url = format!("{}/upload", irys_url.trim_end_matches('/'));
let resp = client
.post(&url)
.header("Content-Type", "application/json")
.header("x-irys-tags", build_manifest_tags_header(manifest))
.body(body)
.send()
.await
.map_err(|e| anyhow::anyhow!("Irys upload failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(anyhow::anyhow!("Irys returned {status}: {body}"));
}
let json: serde_json::Value = resp
.json()
.await
.map_err(|e| anyhow::anyhow!("failed to parse Irys response: {e}"))?;
let tx_id = json["id"]
.as_str()
.ok_or_else(|| anyhow::anyhow!("no 'id' in Irys response: {json}"))?
.to_string();
tracing::info!(
repo = %manifest.repo,
tx_id = %tx_id,
blobs = manifest.blobs.len(),
"anchored encrypted manifest to Arweave"
);
Ok(tx_id)
}
/// Serialize one blob for the Arweave manifest. Recipient identities are
/// intentionally absent so the permanent public anchor never records who can
/// read a blob.
fn manifest_blob_json(oid: &str, cid: &str) -> serde_json::Value {
json!({ "oid": oid, "cid": cid })
}
/// Build the Irys tag header for an encrypted-blob manifest. `Repo` and `Schema`
/// are the tags the `gl` recovery query filters on.
fn build_manifest_tags_header(manifest: &EncryptedManifest<'_>) -> String {
[
"App-Name:gitlawb".to_string(),
"Schema:gitlawb/encrypted-manifest/v1".to_string(),
format!("Repo:{}", sanitize_tag(manifest.repo)),
format!("Owner-DID:{}", sanitize_tag(manifest.owner_did)),
format!("Node-DID:{}", sanitize_tag(manifest.node_did)),
]
.join(",")
}
/// Arweave permanent URL for a given Irys transaction ID.
pub fn arweave_url(tx_id: &str) -> String {
format!("https://arweave.net/{tx_id}")
@@ -193,6 +295,79 @@ mod tests {
);
}
#[tokio::test]
async fn test_manifest_anchor_noop_when_url_empty() {
let client = reqwest::Client::new();
let blobs = vec![("oid1".to_string(), "cid1".to_string())];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
assert_eq!(
anchor_encrypted_manifest(&client, "", &m).await.unwrap(),
""
);
}
#[tokio::test]
async fn test_manifest_anchor_noop_when_no_blobs() {
let client = reqwest::Client::new();
let blobs: Vec<(String, String)> = vec![];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
// Non-empty URL, but no blobs: still a no-op.
assert_eq!(
anchor_encrypted_manifest(&client, "https://example.invalid", &m)
.await
.unwrap(),
""
);
}
#[tokio::test]
async fn test_manifest_anchor_success() {
let mut server = mockito::Server::new_async().await;
let _mock = server
.mock("POST", "/upload")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"id":"MANIFESTTX123","timestamp":1710000000000,"version":"1.0.0"}"#)
.create_async()
.await;
let client = reqwest::Client::new();
let blobs = vec![("oid1".to_string(), "cid1".to_string())];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
let r = anchor_encrypted_manifest(&client, &server.url(), &m).await;
assert_eq!(r.unwrap(), "MANIFESTTX123");
_mock.assert_async().await;
}
#[test]
fn manifest_blob_json_omits_recipients() {
let v = manifest_blob_json("oid1", "cidA");
assert_eq!(v["oid"], "oid1");
assert_eq!(v["cid"], "cidA");
assert!(
v.get("recipients").is_none(),
"Arweave manifest must not anchor recipient identities"
);
}
#[test]
fn test_sanitize_tag() {
assert_eq!(sanitize_tag("alice/myrepo"), "alice/myrepo");
+349 -1
View File
@@ -1,5 +1,5 @@
use axum::body::Body;
use axum::extract::Request;
use axum::extract::{Request, State};
use axum::http::StatusCode;
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
@@ -8,6 +8,11 @@ use http_body_util::BodyExt;
use serde_json::json;
use std::collections::HashMap;
use gitlawb_core::did::Did;
use gitlawb_core::ucan::Ucan;
use crate::state::AppState;
/// The authenticated agent's DID, injected into request extensions by `require_signature`.
#[derive(Clone, Debug)]
pub struct AuthenticatedDid(pub String);
@@ -228,6 +233,127 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
next.run(request).await
}
/// Optional variant for rolling upgrades: verify and inject `AuthenticatedDid` when
/// RFC 9421 signature headers are present, but allow legacy unsigned requests to
/// continue when no signature attempt was made.
pub async fn optional_signature(request: Request, next: Next) -> Response {
let has_signature_headers = request.headers().contains_key("signature-input")
|| request.headers().contains_key("signature");
if has_signature_headers {
return require_signature(request, next).await;
}
next.run(request).await
}
/// Validate a raw UCAN token string supplied in `X-Ucan`.
///
/// Checks performed:
/// 1. The token decodes to a valid [`Ucan`] structure.
/// 2. The UCAN issuer (`iss`) matches `signer_did` — the DID that signed the
/// HTTP request — preventing replay of another agent's UCAN.
/// 3. The UCAN audience (`aud`) matches `expected_aud` — the node's own DID.
/// 4. The full proof chain is cryptographically valid (signatures, expiry,
/// not-before, chain linkage, and capability attenuation).
fn validate_ucan_chain(
token: &str,
expected_aud: &Did,
signer_did: &Did,
) -> Result<(), (StatusCode, Json<serde_json::Value>)> {
let ucan = Ucan::decode(token).map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
if &ucan.payload.iss != signer_did {
return Err((
StatusCode::UNAUTHORIZED,
Json(json!({
"error": "invalid_ucan",
"message": format!(
"UCAN issuer {} does not match request signer {}",
ucan.payload.iss, signer_did
),
})),
));
}
ucan.verify_audience(expected_aud).map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
ucan.verify_chain().map_err(|e| {
(
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
)
})?;
Ok(())
}
/// Axum middleware that validates a UCAN chain when `X-Ucan` is present.
///
/// Must be layered so that it runs after [`require_signature`], which sets the
/// [`AuthenticatedDid`] extension consumed here.
///
/// When `X-Ucan` is absent the request passes through unchanged, preserving
/// backward compatibility for agents that pre-date UCAN delegation. When the
/// header is present the full chain is validated: the UCAN issuer must match
/// the HTTP Signature identity, the audience must be this node's DID, and
/// every proof in the chain must be cryptographically sound with no capability
/// escalation.
pub async fn require_ucan_chain(
State(state): State<AppState>,
request: Request,
next: Next,
) -> Response {
let token = match request
.headers()
.get("x-ucan")
.and_then(|v| v.to_str().ok())
.map(str::to_owned)
{
Some(t) => t,
None => return next.run(request).await,
};
let signer_did: Did = match request.extensions().get::<AuthenticatedDid>() {
Some(a) => match a.0.parse() {
Ok(did) => did,
Err(e) => {
tracing::warn!(raw_did = %a.0, err = %e, "failed to parse DID from authenticated identity");
return (
StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_identity", "message": "invalid DID in token" })),
)
.into_response();
}
},
None => {
return (
StatusCode::UNAUTHORIZED,
Json(json!({
"error": "invalid_ucan",
"message": "UCAN validation requires a valid HTTP Signature",
})),
)
.into_response()
}
};
if let Err((status, body)) = validate_ucan_chain(&token, &state.node_did, &signer_did) {
return (status, body).into_response();
}
tracing::debug!(did = %signer_did, "UCAN chain validated");
next.run(request).await
}
fn human_detected(message: &str) -> impl IntoResponse {
(
StatusCode::UNAUTHORIZED,
@@ -246,3 +372,225 @@ fn human_detected(message: &str) -> impl IntoResponse {
})),
)
}
#[cfg(test)]
mod tests {
use super::*;
use axum::{middleware, Router};
use gitlawb_core::identity::Keypair;
use gitlawb_core::ucan::{caps, Capability, Ucan};
use std::{path::PathBuf, sync::Arc, time::Duration};
use tower::ServiceExt;
fn bootstrap_ucan(node: &Keypair, agent_did: Did) -> Ucan {
Ucan::bootstrap(node, agent_did).unwrap()
}
fn delegation_ucan(agent: &Keypair, node_did: Did, proof: &Ucan) -> Ucan {
Ucan::delegate(
agent,
node_did,
vec![Capability::new("gitlawb://alpha", caps::NETWORK_JOIN)],
None,
proof,
)
.unwrap()
}
#[test]
fn validate_ucan_chain_valid() {
let node = Keypair::generate();
let agent = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let proof = bootstrap_ucan(&node, agent_did.clone());
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
assert!(validate_ucan_chain(&token, &node_did, &agent_did).is_ok());
}
#[test]
fn validate_ucan_chain_wrong_issuer() {
let node = Keypair::generate();
let agent = Keypair::generate();
let other = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let proof = bootstrap_ucan(&node, agent_did.clone());
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
// signer_did is `other` but UCAN iss is `agent` — must be rejected
let err = validate_ucan_chain(&token, &node_did, &other.did()).unwrap_err();
assert_eq!(err.0, StatusCode::UNAUTHORIZED);
let body = err.1 .0.to_string();
assert!(body.contains("does not match request signer"));
}
#[test]
fn validate_ucan_chain_wrong_audience() {
let node = Keypair::generate();
let agent = Keypair::generate();
let other_node = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let proof = bootstrap_ucan(&node, agent_did.clone());
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
// expected_aud is a different node — must be rejected
let err = validate_ucan_chain(&token, &other_node.did(), &agent_did).unwrap_err();
assert_eq!(err.0, StatusCode::UNAUTHORIZED);
let body = err.1 .0.to_string();
assert!(body.contains("audience mismatch"));
}
#[test]
fn validate_ucan_chain_expired_proof() {
let node = Keypair::generate();
let agent = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
let exp = chrono::Utc::now() - chrono::Duration::hours(1);
let proof = Ucan::issue(
&node,
agent_did.clone(),
vec![Capability::new("gitlawb://alpha", caps::NETWORK_JOIN)],
Some(exp),
)
.unwrap();
let delegation = delegation_ucan(&agent, node_did.clone(), &proof);
let token = delegation.encode().unwrap();
let err = validate_ucan_chain(&token, &node_did, &agent_did).unwrap_err();
assert_eq!(err.0, StatusCode::UNAUTHORIZED);
let body = err.1 .0.to_string();
assert!(body.contains("expired"));
}
fn make_test_state(node_did: gitlawb_core::did::Did) -> crate::state::AppState {
use crate::{config::Config, graphql, rate_limit::RateLimiter};
use clap::Parser;
let keypair = Keypair::generate();
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/gitlawb_test_placeholder")
.expect("lazy pool creation should not fail");
let db = Arc::new(crate::db::Db::for_testing(pool.clone()));
let schema = Arc::new(graphql::build_schema(
db.clone(),
ref_tx.clone(),
task_tx.clone(),
));
crate::state::AppState {
config: Arc::new(Config::parse_from(["gitlawb-node"])),
db,
node_did,
node_keypair: Arc::new(keypair),
p2p: None,
http_client: Arc::new(reqwest::Client::new()),
ref_update_tx: ref_tx,
task_event_tx: task_tx,
graphql_schema: schema,
machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
shutdown_tx: tokio::sync::watch::channel(false).0,
}
}
#[tokio::test]
async fn require_ucan_chain_no_header_passes_through() {
let state = make_test_state(Keypair::generate().did());
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
let req = Request::builder()
.uri("/")
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn require_ucan_chain_missing_did_returns_401() {
let state = make_test_state(Keypair::generate().did());
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
// x-ucan present but no AuthenticatedDid extension → 401
let req = Request::builder()
.uri("/")
.header("x-ucan", "any-token")
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn require_ucan_chain_wrong_issuer_returns_401() {
let node = Keypair::generate();
let agent = Keypair::generate();
let other = Keypair::generate();
let node_did = node.did();
let agent_did = agent.did();
// Build a valid token where iss = agent, but supply `other` as the signer.
let proof = bootstrap_ucan(&node, agent_did.clone());
let token = delegation_ucan(&agent, node_did.clone(), &proof)
.encode()
.unwrap();
let state = make_test_state(node_did);
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
// AuthenticatedDid is `other`, UCAN iss is `agent` → issuer mismatch → 401
let req = Request::builder()
.uri("/")
.header("x-ucan", token)
.extension(AuthenticatedDid(other.did().to_string()))
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn require_ucan_chain_malformed_token_returns_401() {
let state = make_test_state(Keypair::generate().did());
let app = Router::new()
.route("/", axum::routing::get(|| async { StatusCode::OK }))
.layer(middleware::from_fn_with_state(state, require_ucan_chain));
// Malformed x-ucan (invalid JSON)
let req = Request::builder()
.uri("/")
.header("x-ucan", "invalid-token-structure")
.extension(AuthenticatedDid(Keypair::generate().did().to_string()))
.body(axum::body::Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
let body_bytes = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
let body_json: serde_json::Value = serde_json::from_slice(&body_bytes).unwrap();
assert_eq!(body_json["error"], "invalid_ucan");
}
}
+1 -1
View File
@@ -11,7 +11,7 @@
use std::str::FromStr;
use libp2p::Multiaddr;
use libp2p_core::Multiaddr;
use serde::Deserialize;
use tracing::{info, warn};
+25 -3
View File
@@ -28,7 +28,7 @@ pub struct Config {
#[arg(long, env = "GITLAWB_KEY", default_value = "~/.gitlawb/identity.pem")]
pub key_path: String,
/// Allow unauthenticated reads (default: true for public repos)
/// Reserved for private-read mode; per-repo read enforcement is not wired in alpha
#[arg(long, env = "GITLAWB_PUBLIC_READ", default_value_t = true)]
pub public_read: bool,
@@ -40,6 +40,15 @@ pub struct Config {
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>,
/// Require RFC 9421 signatures on peer announce/sync write routes.
/// Keep false during rolling upgrades so existing live nodes can still gossip.
#[arg(
long,
env = "GITLAWB_REQUIRE_SIGNED_PEER_WRITES",
default_value_t = false
)]
pub require_signed_peer_writes: bool,
/// URL of local IPFS/Kubo node HTTP API (e.g. http://127.0.0.1:5001)
#[arg(long, env = "GITLAWB_IPFS_API", default_value = "")]
pub ipfs_api: String,
@@ -56,12 +65,12 @@ pub struct Config {
)]
pub pinata_upload_url: String,
/// libp2p TCP port (0 = disabled)
/// libp2p QUIC/UDP port (0 = disabled)
#[arg(long, env = "GITLAWB_P2P_PORT", default_value_t = 7546)]
pub p2p_port: u16,
/// libp2p bootstrap multiaddrs (comma-separated)
/// Example: /ip4/1.2.3.4/tcp/7546/p2p/12D3KooW...
/// Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
#[arg(long, env = "GITLAWB_P2P_BOOTSTRAP", value_delimiter = ',')]
pub p2p_bootstrap: Vec<String>,
@@ -122,6 +131,19 @@ pub struct Config {
/// Default: 2 GB. Set lower on resource-constrained nodes.
#[arg(long, env = "GITLAWB_MAX_PACK_BYTES", default_value_t = 2_147_483_648)]
pub max_pack_bytes: usize,
/// Optional address to bind a Prometheus `/metrics` exposition endpoint on.
/// Example: `127.0.0.1:9091`. Leave empty (default) to disable.
/// Bind to localhost or a private interface — the metrics endpoint is
/// unauthenticated.
#[arg(long, env = "GITLAWB_METRICS_ADDR", default_value = "")]
pub metrics_addr: String,
/// Maximum time to wait for in-flight requests to drain on shutdown, in
/// seconds. After this elapses, the server returns 503 to anything still
/// in flight and exits. Default: 30s.
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
pub shutdown_grace_secs: u64,
}
impl Config {
+652 -11
View File
@@ -1,7 +1,8 @@
use anyhow::Result;
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::{PgPool, Row};
use tracing::info;
use uuid::Uuid;
// ── Public data types ─────────────────────────────────────────────────────────
@@ -21,6 +22,51 @@ pub struct RepoRecord {
pub machine_id: Option<String>,
}
/// Per-rule replication mode for a visibility rule.
/// `A` hides existence entirely (only valid at whole-repo scope `/`).
/// `B` keeps object SHAs and the path visible but withholds content
/// (the only mode allowed for subtrees; enforced on clones in Phase 3).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum VisibilityMode {
A,
B,
}
impl VisibilityMode {
pub fn as_str(&self) -> &'static str {
match self {
VisibilityMode::A => "a",
VisibilityMode::B => "b",
}
}
pub fn from_db(s: &str) -> Self {
match s {
"a" => VisibilityMode::A,
"b" => VisibilityMode::B,
other => {
tracing::warn!("unknown visibility mode in DB: {other:?}, defaulting to B");
VisibilityMode::B
}
}
}
}
/// A path-scoped visibility rule. `path_glob` is "/" for whole-repo, or a
/// subtree pattern such as "/secret-pkg/**". The repo owner is always an
/// implicit reader regardless of `reader_dids`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VisibilityRule {
pub id: String,
pub repo_id: String,
pub path_glob: String,
pub mode: VisibilityMode,
pub reader_dids: Vec<String>,
pub created_by: String,
pub created_at: DateTime<Utc>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PullRequest {
pub id: String,
@@ -177,6 +223,19 @@ pub struct AgentRow {
pub last_seen: Option<String>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProfileRecord {
pub did: String,
pub display_name: Option<String>,
pub bio: Option<String>,
pub avatar_url: Option<String>,
pub website: Option<String>,
pub socials: Option<String>,
pub profile_cid: Option<String>,
pub created_at: String,
pub updated_at: String,
}
// ── Db ────────────────────────────────────────────────────────────────────────
#[derive(Clone)]
@@ -190,6 +249,11 @@ impl Db {
&self.pool
}
#[cfg(test)]
pub fn for_testing(pool: PgPool) -> Self {
Self { pool }
}
pub async fn connect(database_url: &str) -> Result<Self> {
let pool = PgPool::connect(database_url).await?;
let db = Self { pool };
@@ -197,8 +261,175 @@ impl Db {
Ok(db)
}
/// Run all pending versioned migrations in order, inside a single
/// transaction per migration. Idempotent — migrations whose version is
/// already recorded in `schema_migrations` are skipped.
///
/// Concurrency: the whole routine is guarded by a Postgres advisory lock so
/// two node instances pointed at the same database (e.g. during a
/// blue/green or rolling deploy) cannot race to apply the same migration
/// and trip the `schema_migrations` primary key.
///
/// Legacy installs: v1 bundles the entire pre-versioning schema, and every
/// statement in it is idempotent (`CREATE TABLE IF NOT EXISTS`,
/// `CREATE INDEX IF NOT EXISTS`, `ADD COLUMN IF NOT EXISTS`). So an existing
/// node that predates this system just runs v1 once: existing objects are
/// no-ops, and any objects it was missing are created. We deliberately do
/// *not* short-circuit on the presence of a single canonical table — a node
/// that was behind on schema would then be marked complete while still
/// missing newer objects.
async fn migrate(&self) -> Result<()> {
let stmts = [
// Bootstrap: ensure the `schema_migrations` table itself exists.
sqlx::query(
r#"CREATE TABLE IF NOT EXISTS schema_migrations (
version BIGINT NOT NULL PRIMARY KEY,
name TEXT NOT NULL,
applied_at TEXT NOT NULL
)"#,
)
.execute(&self.pool)
.await
.context("creating schema_migrations table")?;
// Serialize migrations across processes: hold a session-level advisory
// lock on a dedicated connection for the whole run. Another instance
// starting up blocks here until we finish. The lock is released when we
// explicitly unlock below, or automatically if the connection is
// dropped (e.g. on panic), so a crash can't wedge future restarts.
let mut lock_conn = self
.pool
.acquire()
.await
.context("acquiring connection for migration advisory lock")?;
sqlx::query("SELECT pg_advisory_lock($1)")
.bind(MIGRATION_ADVISORY_LOCK)
.execute(&mut *lock_conn)
.await
.context("acquiring migration advisory lock")?;
let result = self.run_pending_migrations().await;
let _ = sqlx::query("SELECT pg_advisory_unlock($1)")
.bind(MIGRATION_ADVISORY_LOCK)
.execute(&mut *lock_conn)
.await;
result
}
/// Apply every migration whose version isn't yet recorded, in order.
/// Must be called while holding the migration advisory lock.
async fn run_pending_migrations(&self) -> Result<()> {
for m in MIGRATIONS {
let already: bool = sqlx::query(
"SELECT EXISTS(SELECT 1 FROM schema_migrations WHERE version = $1) AS applied",
)
.bind(m.version)
.fetch_one(&self.pool)
.await?
.get::<bool, _>("applied");
if already {
continue;
}
let started = std::time::Instant::now();
info!(
version = m.version,
name = m.name,
statements = m.stmts.len(),
"applying migration"
);
// Run the migration body in a single transaction so a failure
// mid-way leaves the database in its prior state rather than
// partially mutated.
let mut tx = self.pool.begin().await?;
for stmt in m.stmts {
sqlx::query(stmt).execute(&mut *tx).await.with_context(|| {
format!(
"migration v{} ({}) failed on statement: {}",
m.version, m.name, stmt
)
})?;
}
sqlx::query(
"INSERT INTO schema_migrations (version, name, applied_at)
VALUES ($1, $2, $3)",
)
.bind(m.version)
.bind(m.name)
.bind(Utc::now().to_rfc3339())
.execute(&mut *tx)
.await
.context("recording migration as applied")?;
tx.commit()
.await
.with_context(|| format!("committing migration v{}", m.version))?;
info!(
version = m.version,
name = m.name,
elapsed_ms = started.elapsed().as_millis() as u64,
"migration applied"
);
}
Ok(())
}
/// Returns `(version, name, applied_at)` for every applied migration,
/// oldest first. Useful for ops/observability — surface via `gl status`
/// or `/api/v1/stats` in a follow-up.
#[allow(dead_code)]
pub async fn migration_status(&self) -> Result<Vec<(i64, String, String)>> {
let rows = sqlx::query(
"SELECT version, name, applied_at FROM schema_migrations ORDER BY version ASC",
)
.fetch_all(&self.pool)
.await?;
Ok(rows
.into_iter()
.map(|r| {
(
r.get::<i64, _>("version"),
r.get("name"),
r.get("applied_at"),
)
})
.collect())
}
}
// ── Migration catalogue ──────────────────────────────────────────────────────
//
// All schema statements are bundled into a single v1 migration so we can ship
// versioned migrations on a live network without breaking the existing
// install base. Future schema changes MUST be added as v2, v3, … — never
// appended to v1. Operators can read `schema_migrations` to confirm a node
// is at the expected version.
//
// Each migration runs in a single transaction, so statements that Postgres
// forbids inside a transaction (notably `CREATE INDEX CONCURRENTLY`) cannot be
// used here. Build such indexes the ordinary, transaction-safe way, or stage
// them as a dedicated out-of-band operational step.
// Arbitrary but stable key for the migration advisory lock ("gitlawb_" bytes).
const MIGRATION_ADVISORY_LOCK: i64 = 0x6769_746C_6177_625F;
const MIGRATION_V1_NAME: &str = "initial_schema";
struct Migration {
version: i64,
name: &'static str,
stmts: &'static [&'static str],
}
const MIGRATIONS: &[Migration] = &[
Migration {
version: 1,
name: MIGRATION_V1_NAME,
stmts: &[
r#"CREATE TABLE IF NOT EXISTS repos (
id TEXT NOT NULL PRIMARY KEY,
name TEXT NOT NULL,
@@ -453,14 +684,69 @@ impl Db {
"CREATE INDEX IF NOT EXISTS idx_bounties_status ON bounties(status)",
"CREATE INDEX IF NOT EXISTS idx_bounties_repo ON bounties(repo_owner, repo_name)",
"CREATE INDEX IF NOT EXISTS idx_bounties_claimant ON bounties(claimant_did)",
];
for stmt in &stmts {
sqlx::query(stmt).execute(&self.pool).await?;
}
Ok(())
}
}
],
},
Migration {
version: 2,
name: "agent_profiles",
stmts: &[
r#"CREATE TABLE IF NOT EXISTS agent_profiles (
did TEXT NOT NULL PRIMARY KEY,
display_name TEXT,
bio TEXT,
avatar_url TEXT,
website TEXT,
socials TEXT,
profile_cid TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)"#,
],
},
Migration {
version: 3,
name: "visibility_rules",
stmts: &[
r#"CREATE TABLE IF NOT EXISTS visibility_rules (
id TEXT NOT NULL PRIMARY KEY,
repo_id TEXT NOT NULL,
path_glob TEXT NOT NULL,
mode TEXT NOT NULL,
reader_dids TEXT NOT NULL,
created_by TEXT NOT NULL,
created_at TEXT NOT NULL,
UNIQUE(repo_id, path_glob)
)"#,
"CREATE INDEX IF NOT EXISTS idx_visibility_rules_repo ON visibility_rules(repo_id)",
],
},
Migration {
version: 4,
name: "encrypted_blobs",
stmts: &[
r#"CREATE TABLE IF NOT EXISTS encrypted_blobs (
repo_id TEXT NOT NULL,
oid TEXT NOT NULL,
cid TEXT NOT NULL,
recipients TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (repo_id, oid)
)"#,
"CREATE INDEX IF NOT EXISTS idx_encrypted_blobs_repo ON encrypted_blobs(repo_id)",
],
},
Migration {
version: 5,
name: "encrypted_blobs_blind_recipients",
stmts: &[
// Replace the cleartext recipient DID list with an opaque, node-keyed
// tag used only to detect a recipient-set change. Existing rows get an
// empty tag and are re-sealed on the next push.
"ALTER TABLE encrypted_blobs DROP COLUMN IF EXISTS recipients",
"ALTER TABLE encrypted_blobs ADD COLUMN IF NOT EXISTS recipients_tag TEXT NOT NULL DEFAULT ''",
],
},
];
// ── Repos ─────────────────────────────────────────────────────────────────────
@@ -592,7 +878,14 @@ impl Db {
"WITH deduped AS (
SELECT DISTINCT ON (split_part(owner_did, ':', -1), name)
id, name, owner_did, description, is_public, default_branch,
created_at, updated_at, disk_path, forked_from, machine_id
created_at,
-- group MAX, not the canonical row's own value: pushes that
-- arrive via gossip touch only the mirror row, so the
-- canonical updated_at goes stale
MAX(updated_at) OVER (
PARTITION BY split_part(owner_did, ':', -1), name
) AS updated_at,
disk_path, forked_from, machine_id
FROM repos
WHERE ($1::text IS NULL OR owner_did = $1 OR owner_did LIKE '%:' || $1)
ORDER BY split_part(owner_did, ':', -1), name,
@@ -1361,6 +1654,76 @@ impl Db {
Ok(())
}
pub async fn record_encrypted_blob(
&self,
repo_id: &str,
oid: &str,
cid: &str,
recipients_tag: &str,
) -> Result<()> {
sqlx::query(
"INSERT INTO encrypted_blobs (repo_id, oid, cid, recipients_tag, created_at)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (repo_id, oid) DO UPDATE SET cid = EXCLUDED.cid, recipients_tag = EXCLUDED.recipients_tag",
)
.bind(repo_id)
.bind(oid)
.bind(cid)
.bind(recipients_tag)
.bind(Utc::now().to_rfc3339())
.execute(&self.pool)
.await?;
Ok(())
}
/// (oid, cid) for every encrypted blob in the repo, unscoped by caller. Used
/// by both the B2 replication view and B1 discovery. Recipient identities are
/// not stored, so authorization is the caller's repo readability, not a per
/// recipient check. Ciphertext metadata only.
pub async fn list_all_encrypted_blobs(&self, repo_id: &str) -> Result<Vec<(String, String)>> {
let rows = sqlx::query("SELECT oid, cid FROM encrypted_blobs WHERE repo_id = $1")
.bind(repo_id)
.fetch_all(&self.pool)
.await?;
let mut out = Vec::new();
for row in rows {
let oid: String = row.get("oid");
let cid: String = row.get("cid");
out.push((oid, cid));
}
Ok(out)
}
/// The CID of one encrypted blob, or None if there is no such row. Recipient
/// authorization is not enforced here: the handler checks repo readability and
/// the envelope crypto gates decryption.
pub async fn encrypted_blob_cid(&self, repo_id: &str, oid: &str) -> Result<Option<String>> {
let row = sqlx::query("SELECT cid FROM encrypted_blobs WHERE repo_id = $1 AND oid = $2")
.bind(repo_id)
.bind(oid)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|r| r.get("cid")))
}
/// The opaque recipients tag stored for an encrypted blob, or None if there is
/// no row. Used only to decide whether a re-seal is needed (the recipient set
/// changed); the tag is a node-keyed fingerprint, not the DID list.
pub async fn encrypted_blob_recipients_tag(
&self,
repo_id: &str,
oid: &str,
) -> Result<Option<String>> {
let row = sqlx::query(
"SELECT recipients_tag FROM encrypted_blobs WHERE repo_id = $1 AND oid = $2",
)
.bind(repo_id)
.bind(oid)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|r| r.get("recipients_tag")))
}
pub async fn list_pinned_cids(&self) -> Result<Vec<PinnedCidRecord>> {
let rows = sqlx::query(
"SELECT sha256_hex, cid, pinned_at, pinata_cid FROM pinned_cids ORDER BY pinned_at DESC",
@@ -1874,6 +2237,80 @@ impl Db {
}
}
// ── Path-scoped Visibility ────────────────────────────────────────────────────
impl Db {
pub async fn set_visibility_rule(
&self,
repo_id: &str,
path_glob: &str,
mode: VisibilityMode,
reader_dids: &[String],
created_by: &str,
) -> Result<()> {
let id = Uuid::new_v4().to_string();
let now = Utc::now().to_rfc3339();
let readers = serde_json::to_string(reader_dids).unwrap_or_else(|_| "[]".to_string());
sqlx::query(
"INSERT INTO visibility_rules
(id, repo_id, path_glob, mode, reader_dids, created_by, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7)
ON CONFLICT (repo_id, path_glob) DO UPDATE
SET mode = EXCLUDED.mode,
reader_dids = EXCLUDED.reader_dids,
created_by = EXCLUDED.created_by,
created_at = EXCLUDED.created_at",
)
.bind(&id)
.bind(repo_id)
.bind(path_glob)
.bind(mode.as_str())
.bind(&readers)
.bind(created_by)
.bind(&now)
.execute(&self.pool)
.await?;
Ok(())
}
pub async fn remove_visibility_rule(&self, repo_id: &str, path_glob: &str) -> Result<()> {
sqlx::query("DELETE FROM visibility_rules WHERE repo_id = $1 AND path_glob = $2")
.bind(repo_id)
.bind(path_glob)
.execute(&self.pool)
.await?;
Ok(())
}
pub async fn list_visibility_rules(&self, repo_id: &str) -> Result<Vec<VisibilityRule>> {
let rows = sqlx::query(
"SELECT id, repo_id, path_glob, mode, reader_dids, created_by, created_at
FROM visibility_rules WHERE repo_id = $1 ORDER BY path_glob",
)
.bind(repo_id)
.fetch_all(&self.pool)
.await?;
Ok(rows
.into_iter()
.map(|r| {
let readers: String = r.get("reader_dids");
let created_at: String = r.get("created_at");
VisibilityRule {
id: r.get("id"),
repo_id: r.get("repo_id"),
path_glob: r.get("path_glob"),
mode: VisibilityMode::from_db(&r.get::<String, _>("mode")),
reader_dids: serde_json::from_str(&readers).unwrap_or_default(),
created_by: r.get("created_by"),
created_at: created_at
.parse::<DateTime<Utc>>()
.unwrap_or_else(|_| Utc::now()),
}
})
.collect())
}
}
// ── Repo Stars ────────────────────────────────────────────────────────────────
impl Db {
@@ -2189,3 +2626,207 @@ impl Db {
}
}
}
// ── Agent Profiles ───────────────────────────────────────────────────────────
impl Db {
pub async fn upsert_profile(
&self,
did: &str,
display_name: Option<&str>,
bio: Option<&str>,
avatar_url: Option<&str>,
website: Option<&str>,
socials: Option<&str>,
) -> Result<ProfileRecord> {
let now = Utc::now().to_rfc3339();
// Try update first for existing profiles (merge fields)
let existing = self.get_profile(did).await?;
if let Some(existing) = existing {
let new_name = display_name.or(existing.display_name.as_deref());
let new_bio = bio.or(existing.bio.as_deref());
let new_avatar = avatar_url.or(existing.avatar_url.as_deref());
let new_website = website.or(existing.website.as_deref());
let new_socials = socials.or(existing.socials.as_deref());
sqlx::query(
"UPDATE agent_profiles
SET display_name=$1, bio=$2, avatar_url=$3, website=$4, socials=$5, updated_at=$6
WHERE did=$7",
)
.bind(new_name)
.bind(new_bio)
.bind(new_avatar)
.bind(new_website)
.bind(new_socials)
.bind(&now)
.bind(did)
.execute(&self.pool)
.await?;
Ok(ProfileRecord {
did: did.to_string(),
display_name: new_name.map(String::from),
bio: new_bio.map(String::from),
avatar_url: new_avatar.map(String::from),
website: new_website.map(String::from),
socials: new_socials.map(String::from),
profile_cid: existing.profile_cid,
created_at: existing.created_at,
updated_at: now,
})
} else {
sqlx::query(
"INSERT INTO agent_profiles (did, display_name, bio, avatar_url, website, socials, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)",
)
.bind(did)
.bind(display_name)
.bind(bio)
.bind(avatar_url)
.bind(website)
.bind(socials)
.bind(&now)
.bind(&now)
.execute(&self.pool)
.await?;
Ok(ProfileRecord {
did: did.to_string(),
display_name: display_name.map(String::from),
bio: bio.map(String::from),
avatar_url: avatar_url.map(String::from),
website: website.map(String::from),
socials: socials.map(String::from),
profile_cid: None,
created_at: now.clone(),
updated_at: now,
})
}
}
pub async fn get_profile(&self, did: &str) -> Result<Option<ProfileRecord>> {
let row = sqlx::query(
"SELECT did, display_name, bio, avatar_url, website, socials, profile_cid, created_at, updated_at
FROM agent_profiles
WHERE did = $1 OR did LIKE '%:' || $1",
)
.bind(did)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|r| ProfileRecord {
did: r.get("did"),
display_name: r.get("display_name"),
bio: r.get("bio"),
avatar_url: r.get("avatar_url"),
website: r.get("website"),
socials: r.get("socials"),
profile_cid: r.get("profile_cid"),
created_at: r.get("created_at"),
updated_at: r.get("updated_at"),
}))
}
pub async fn set_profile_cid(&self, did: &str, cid: &str) -> Result<()> {
sqlx::query("UPDATE agent_profiles SET profile_cid = $1, updated_at = $2 WHERE did = $3")
.bind(cid)
.bind(Utc::now().to_rfc3339())
.bind(did)
.execute(&self.pool)
.await?;
Ok(())
}
}
// ── Tests ─────────────────────────────────────────────────────────────────────
//
// These tests don't require a live Postgres connection. They validate the
// static migration catalogue is well-formed so a future maintainer can't
// ship a regression like duplicate versions, negative versions, or empty
// migration bodies. The actual SQL execution is exercised by integration
// tests / first-run on a real node.
#[cfg(test)]
mod migration_tests {
use super::{MIGRATIONS, MIGRATION_V1_NAME};
#[test]
fn migrations_are_non_empty() {
assert!(
!MIGRATIONS.is_empty(),
"MIGRATIONS must contain at least the initial v1 schema"
);
}
#[test]
fn migration_versions_are_strictly_increasing() {
let mut last = i64::MIN;
for m in MIGRATIONS {
assert!(
m.version > last,
"migration versions must be strictly increasing; \
found {} after {}",
m.version,
last
);
last = m.version;
}
}
#[test]
fn migration_versions_start_at_one() {
// A version of 0 (or negative) would be a footgun: any future
// `WHERE version > current_max` style query would skip it.
assert_eq!(
MIGRATIONS.first().map(|m| m.version),
Some(1),
"the first migration must have version 1"
);
}
#[test]
fn migration_names_are_non_empty_and_distinct() {
let mut seen = std::collections::HashSet::new();
for m in MIGRATIONS {
assert!(
!m.name.is_empty(),
"migration v{} has empty name",
m.version
);
assert!(
!m.name.contains(char::is_whitespace),
"migration v{} name {:?} contains whitespace",
m.version,
m.name
);
assert!(
seen.insert(m.name),
"duplicate migration name: {:?}",
m.name
);
}
}
#[test]
fn migration_bodies_are_non_empty() {
for m in MIGRATIONS {
assert!(
!m.stmts.is_empty(),
"migration v{} ({}) has no SQL statements",
m.version,
m.name
);
}
}
#[test]
fn v1_name_is_the_initial_schema() {
// This is what the legacy-install backfill writes to
// `schema_migrations` when an existing node upgrades. If you rename
// it, you must also update the backfill.
assert_eq!(MIGRATIONS[0].name, MIGRATION_V1_NAME);
}
}
+136
View File
@@ -0,0 +1,136 @@
//! Encrypt-then-pin for withheld blobs (Option B1). Each withheld blob is sealed
//! to its recipient DIDs and the envelope pinned to IPFS, recorded in
//! `encrypted_blobs`. Best-effort per blob: a failure is logged and skipped,
//! never pinned in plaintext.
use std::collections::{BTreeSet, HashMap};
use std::path::Path;
use std::str::FromStr;
use ed25519_dalek::VerifyingKey;
use gitlawb_core::did::Did;
use gitlawb_core::encrypt::seal_blob;
use crate::db::Db;
use hmac::{Hmac, Mac};
use sha2::Sha256;
type HmacSha256 = Hmac<Sha256>;
/// Opaque, node-keyed fingerprint of a blob's recipient set. Stored in place of
/// the cleartext DID list so a DB compromise cannot reveal the reader set; used
/// only to detect a recipient-set change so an unchanged blob is not re-sealed.
/// Order-insensitive (the input `BTreeSet` is already sorted).
pub fn recipients_tag(node_seed: &[u8; 32], dids: &BTreeSet<String>) -> String {
let mut mac = HmacSha256::new_from_slice(node_seed).expect("HMAC accepts any key length");
mac.update(b"gitlawb/recipients-tag/v1");
for did in dids {
mac.update(b"\n");
mac.update(did.as_bytes());
}
hex::encode(mac.finalize().into_bytes())
}
/// Resolve a DID string to its Ed25519 verifying key, or None if it carries no
/// inline key (e.g. did:web / did:gitlawb).
fn did_to_key(did: &str) -> Option<VerifyingKey> {
Did::from_str(did).ok()?.to_verifying_key().ok()
}
/// Encrypt and pin every withheld blob. `recipients` maps blob oid -> DID set;
/// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob
/// actually sealed and recorded this call (the per-push delta), used by Option B3
/// to anchor a manifest. Recipient identities are never stored or returned.
pub async fn encrypt_and_pin(
ipfs_api: &str,
repo_path: &Path,
db: &Db,
repo_id: &str,
node_seed: &[u8; 32],
recipients: &HashMap<String, BTreeSet<String>>,
) -> Vec<(String, String)> {
let mut sealed = Vec::new();
for (oid, dids) in recipients {
// Skip only if an existing envelope already covers exactly these
// recipients. If the recipient set changed (e.g. a reader was added to
// the rule), re-seal so the new reader can recover the blob. Reader
// removal is not retroactive: the old envelope is already public. The
// comparison is on the opaque node-keyed tag, never the DID list.
let tag = recipients_tag(node_seed, dids);
match db.encrypted_blob_recipients_tag(repo_id, oid).await {
Ok(Some(stored_tag)) if stored_tag == tag => continue,
Ok(_) => {}
Err(e) => {
// A DB read failure is not a cache miss: re-sealing here would do
// an avoidable IPFS write during a partial outage. Skip and retry
// on the next push.
tracing::warn!(oid = %oid, err = %e, "recipients_tag lookup failed; skipping reseal");
continue;
}
}
let keys: Vec<VerifyingKey> = dids.iter().filter_map(|d| did_to_key(d)).collect();
if keys.is_empty() {
tracing::warn!(oid = %oid, "no resolvable recipient keys; skipping encrypted pin");
continue;
}
let data = match crate::git::store::read_object(repo_path, oid) {
Ok(Some((_t, bytes))) => bytes,
_ => continue,
};
let envelope = match seal_blob(&data, &keys) {
Ok(e) => e,
Err(e) => {
tracing::warn!(oid = %oid, err = %e, "seal_blob failed; skipping");
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
Ok(c) if !c.is_empty() => c,
_ => continue,
};
if let Err(e) = db.record_encrypted_blob(repo_id, oid, &cid, &tag).await {
tracing::warn!(oid = %oid, err = %e, "record_encrypted_blob failed");
continue;
}
sealed.push((oid.clone(), cid.clone()));
}
sealed
}
#[cfg(test)]
mod tests {
use super::recipients_tag;
use std::collections::BTreeSet;
fn set(dids: &[&str]) -> BTreeSet<String> {
dids.iter().map(|s| s.to_string()).collect()
}
#[test]
fn tag_is_order_insensitive() {
let seed = [7u8; 32];
let a = recipients_tag(&seed, &set(&["did:key:zA", "did:key:zB"]));
let b = recipients_tag(&seed, &set(&["did:key:zB", "did:key:zA"]));
assert_eq!(a, b);
}
#[test]
fn tag_differs_for_different_sets() {
let seed = [7u8; 32];
let a = recipients_tag(&seed, &set(&["did:key:zA"]));
let b = recipients_tag(&seed, &set(&["did:key:zA", "did:key:zB"]));
assert_ne!(a, b);
}
#[test]
fn tag_is_keyed_by_node_seed() {
let dids = set(&["did:key:zA", "did:key:zB"]);
let a = recipients_tag(&[1u8; 32], &dids);
let b = recipients_tag(&[2u8; 32], &dids);
assert_ne!(
a, b,
"tag must depend on the node seed, not be a plain hash"
);
}
}
+1
View File
@@ -3,3 +3,4 @@ pub mod repo_store;
pub mod smart_http;
pub mod store;
pub mod tigris;
pub mod visibility_pack;
+51 -17
View File
@@ -33,6 +33,16 @@ pub struct RepoStore {
}
impl RepoStore {
#[cfg(test)]
pub fn for_testing(repos_dir: PathBuf, pool: PgPool) -> Self {
Self {
repos_dir,
tigris: None,
pool,
migrated: Arc::new(tokio::sync::Mutex::new(std::collections::HashSet::new())),
}
}
pub fn new(repos_dir: PathBuf, tigris: Option<TigrisClient>, pool: PgPool) -> Self {
Self {
repos_dir,
@@ -120,10 +130,19 @@ impl RepoStore {
if let Some(ref tigris) = self.tigris {
if tigris.exists(&owner_slug, repo_name).await.unwrap_or(false) {
debug!(repo = %repo_name, "acquire_fresh: downloading latest from tigris");
tigris
.download(&owner_slug, repo_name, &local_path)
.await
.context("downloading repo from tigris (fresh)")?;
if let Err(e) = tigris.download(&owner_slug, repo_name, &local_path).await {
// The Tigris archive is present (HEAD ok) but unreadable — a
// corrupt/partial upload, or a transient GET failure. If we have a
// valid local copy, proceed with it rather than blocking the write;
// the post-write upload re-syncs (self-heals) Tigris. Only hard-fail
// when there is no local copy to fall back to.
if local_path.exists() {
warn!(repo = %repo_name, err = %e,
"acquire_fresh: tigris download failed — falling back to local copy");
return Ok(local_path);
}
return Err(e).context("downloading repo from tigris (fresh)");
}
return Ok(local_path);
}
}
@@ -164,10 +183,17 @@ impl RepoStore {
if let Some(ref tigris) = self.tigris {
if tigris.exists(&owner_slug, repo_name).await.unwrap_or(false) {
debug!(repo = %repo_name, "write acquire: downloading latest from tigris");
tigris
.download(&owner_slug, repo_name, &local_path)
.await
.context("downloading repo from tigris for write")?;
if let Err(e) = tigris.download(&owner_slug, repo_name, &local_path).await {
// Same self-healing fallback as acquire_fresh: a corrupt/unreadable
// Tigris archive must not block a write when a valid local copy
// exists — release(success) will re-upload a good archive.
if local_path.exists() {
warn!(repo = %repo_name, err = %e,
"write acquire: tigris download failed — falling back to local copy");
} else {
return Err(e).context("downloading repo from tigris for write");
}
}
}
}
@@ -338,16 +364,24 @@ impl RepoWriteGuard {
&self.local_path
}
/// Upload to Tigris and release the advisory lock. Call this when the write is done.
pub async fn release(self) {
// Upload to Tigris
if let Some(ref tigris) = self.tigris {
if let Err(e) = tigris
.upload(&self.owner_slug, &self.repo_name, &self.local_path)
.await
{
warn!(repo = %self.repo_name, err = %e, "failed to upload repo to tigris after write");
/// Upload to Tigris (only when the write succeeded) and release the advisory
/// lock. Pass `success = false` when the write operation failed — uploading a
/// half-applied or otherwise inconsistent repo would propagate corruption to
/// Tigris (and to every node that later downloads it). The lock is always
/// released regardless, to avoid stale locks blocking future writes.
pub async fn release(self, success: bool) {
// Upload to Tigris only on success.
if success {
if let Some(ref tigris) = self.tigris {
if let Err(e) = tigris
.upload(&self.owner_slug, &self.repo_name, &self.local_path)
.await
{
warn!(repo = %self.repo_name, err = %e, "failed to upload repo to tigris after write");
}
}
} else {
warn!(repo = %self.repo_name, "write failed — skipping tigris upload to avoid propagating an inconsistent repo");
}
// Release advisory lock
+570 -1
View File
@@ -1,8 +1,9 @@
use anyhow::{bail, Result};
use anyhow::{bail, Context, Result};
use axum::body::Body;
use axum::http::StatusCode;
use axum::response::Response;
use bytes::Bytes;
use std::collections::HashSet;
use std::path::Path;
use std::process::Stdio;
use tokio::io::AsyncWriteExt;
@@ -120,3 +121,571 @@ fn pkt_line(data: &str) -> Vec<u8> {
let len = data.len() + 4;
format!("{len:04x}{data}").into_bytes()
}
/// Build a packfile containing every object reachable from all refs EXCEPT the
/// given blob OIDs. Commits and trees are always included, so SHAs stay intact;
/// only the named blobs are dropped.
pub fn build_filtered_pack(repo_path: &Path, withheld: &HashSet<String>) -> Result<Vec<u8>> {
// All reachable objects as "oid [path]" lines.
let rev = std::process::Command::new("git")
.args(["rev-list", "--objects", "--all"])
.current_dir(repo_path)
.output()?;
if !rev.status.success() {
bail!(
"git rev-list failed: {}",
String::from_utf8_lossy(&rev.stderr)
);
}
let mut keep = Vec::new();
for line in String::from_utf8_lossy(&rev.stdout).lines() {
let oid = line.split_whitespace().next().unwrap_or("");
if oid.is_empty() || withheld.contains(oid) {
continue;
}
keep.push(oid.to_string());
}
let mut child = std::process::Command::new("git")
.args(["pack-objects", "--stdout"])
.current_dir(repo_path)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?;
{
use std::io::Write as _;
let mut stdin = child.stdin.take().expect("stdin");
stdin.write_all(keep.join("\n").as_bytes())?;
stdin.write_all(b"\n")?;
}
let out = child.wait_with_output()?;
if !out.status.success() {
bail!(
"git pack-objects failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
Ok(out.stdout)
}
/// Serve a clone/fetch with the withheld blobs removed from the response pack.
///
/// The framing is git protocol v0 (`NAK` then the pack), matching the v0 ref
/// advertisement that `info_refs` emits (it runs `git upload-pack
/// --advertise-refs` without `GIT_PROTOCOL=version=2`, so clients negotiate v0).
/// If `info_refs` ever advertises v2, this serve path must learn v2 framing too.
///
/// Because the pack deliberately omits blobs that the sent trees still
/// reference, the pack is not closed under reachability. A stock full clone
/// rejects it at fetch time ("remote did not send all necessary objects"); only
/// a partial clone (the client passes `--filter`, marking a promisor remote)
/// accepts the pack with the private blobs absent. Tree and commit SHAs stay
/// intact either way. The clean partial-clone client UX is a separate follow-up
/// (git-remote-gitlawb); the security guarantee (private bytes never leave the
/// node) holds regardless of client.
///
/// Negotiation is intentionally ignored: rather than honoring the client's
/// `want`/`have` lines, this always sends a self-contained pack of every object
/// across all refs minus the withheld blobs, and replies `NAK`. A fresh clone
/// and an incremental fetch are both correct (the client de-duplicates objects
/// it already has); the cost is that a fetch re-sends the full object set
/// instead of a thin delta. Honoring negotiation for smaller fetch packs is an
/// optimization follow-up, not a correctness requirement.
pub async fn upload_pack_excluding(
repo_path: &Path,
request_body: Bytes,
withheld: &HashSet<String>,
) -> Result<Response> {
// build_filtered_pack shells out to git (rev-list, pack-objects) with
// blocking std::process I/O; run it off the async worker so a large repo's
// pack build does not stall the tokio runtime.
let pack = {
let repo_path = repo_path.to_path_buf();
let withheld = withheld.clone();
tokio::task::spawn_blocking(move || build_filtered_pack(&repo_path, &withheld))
.await
.context("filtered-pack build task panicked")??
};
// The client lists its capabilities on the first `want` line. Honor
// side-band-64k when offered (every modern smart-HTTP client offers it);
// otherwise stream the raw pack after NAK.
let sideband = memmem(&request_body, b"side-band-64k");
let mut body = Vec::new();
body.extend_from_slice(&pkt_line("NAK\n"));
if sideband {
// Band 1 carries pack data, chunked under the pkt-line size limit.
for chunk in pack.chunks(65515) {
let mut framed = Vec::with_capacity(chunk.len() + 1);
framed.push(0x01);
framed.extend_from_slice(chunk);
let len = framed.len() + 4;
body.extend_from_slice(format!("{len:04x}").as_bytes());
body.extend_from_slice(&framed);
}
body.extend_from_slice(b"0000");
} else {
body.extend_from_slice(&pack);
}
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/x-git-upload-pack-result")
.header("Cache-Control", "no-cache")
.body(Body::from(body))?)
}
/// True if `needle` occurs anywhere in `haystack`. Small substring scan used to
/// detect a client capability token in the upload-pack request body.
fn memmem(haystack: &[u8], needle: &[u8]) -> bool {
if needle.is_empty() || haystack.len() < needle.len() {
return needle.is_empty();
}
haystack
.windows(needle.len())
.any(|window| window == needle)
}
#[cfg(test)]
mod tests {
use super::*;
use std::process::Command;
use tempfile::TempDir;
/// List OIDs in a pack by writing it to a temp dir and running verify-pack.
pub(super) fn pack_object_ids(pack: &[u8]) -> std::collections::HashSet<String> {
let dir = TempDir::new().unwrap();
let path = dir.path().join("test.pack");
std::fs::write(&path, pack).unwrap();
// index-pack creates the matching .idx next to the pack.
let ok = Command::new("git")
.args(["index-pack", path.to_str().unwrap()])
.status()
.unwrap()
.success();
assert!(ok, "index-pack failed");
let out = Command::new("git")
.args(["verify-pack", "-v", path.to_str().unwrap()])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout)
.lines()
.filter_map(|l| l.split_whitespace().next())
.filter(|t| t.len() == 40 && t.chars().all(|c| c.is_ascii_hexdigit()))
.map(|s| s.to_string())
.collect()
}
#[tokio::test]
async fn filtered_serve_excludes_withheld_blob() {
// Build a bare repo, capture the secret + public blob OIDs.
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("bare.git");
let g = |args: &[&str], dir: &std::path::Path| {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
};
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"pub\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"SECRET\n").unwrap();
g(&["init", "-q"], &work);
g(&["config", "user.email", "t@t"], &work);
g(&["config", "user.name", "t"], &work);
g(&["add", "."], &work);
g(&["commit", "-qm", "init"], &work);
let oid = |p: &str| {
let o = Command::new("git")
.args(["rev-parse", &format!("HEAD:{p}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let secret = oid("secret/b.txt");
let public = oid("public/a.txt");
g(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
let mut withheld = std::collections::HashSet::new();
withheld.insert(secret.clone());
let pack = build_filtered_pack(&bare, &withheld).unwrap();
let ids = pack_object_ids(&pack);
assert!(ids.contains(&public), "public blob must be in the pack");
assert!(
!ids.contains(&secret),
"secret blob must NOT be in the pack"
);
}
#[tokio::test]
async fn client_clone_lacks_withheld_blob_bytes() {
use axum::body::to_bytes;
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("bare.git");
let g = |args: &[&str], dir: &std::path::Path| {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
};
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"pub\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"SECRET\n").unwrap();
g(&["init", "-q"], &work);
g(&["config", "user.email", "t@t"], &work);
g(&["config", "user.name", "t"], &work);
g(&["add", "."], &work);
g(&["commit", "-qm", "init"], &work);
let oid = |p: &str| {
let o = Command::new("git")
.args(["rev-parse", &format!("HEAD:{p}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let secret_oid = oid("secret/b.txt");
let public_oid = oid("public/a.txt");
g(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
let mut withheld = std::collections::HashSet::new();
withheld.insert(secret_oid.clone());
// A realistic v0 request advertises side-band-64k, so the serve frames
// the pack in band 1 (the path real clients exercise).
let req = Bytes::from_static(
b"0098want 0000000000000000000000000000000000000000 \
side-band-64k ofs-delta agent=git/2\n00000009done\n",
);
let resp = upload_pack_excluding(&bare, req, &withheld).await.unwrap();
let body = to_bytes(resp.into_body(), usize::MAX).await.unwrap();
let ids = pack_object_ids(&extract_pack(&body));
assert!(
ids.contains(&public_oid),
"public blob must be present in served pack"
);
assert!(
!ids.contains(&secret_oid),
"withheld blob must be absent from served pack"
);
}
/// Strip the v0 upload-pack framing (NAK line + sideband-64k bands),
/// returning the raw pack. Mirrors how a client de-frames the band-1 stream.
fn extract_pack(body: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
let mut i = 0;
while i + 4 <= body.len() {
let len =
usize::from_str_radix(std::str::from_utf8(&body[i..i + 4]).unwrap_or("0000"), 16)
.unwrap_or(0);
if len == 0 {
i += 4;
continue;
}
let chunk = &body[i + 4..i + len];
// band 1 = pack data; skip the NAK line and any other bands.
if chunk.first() == Some(&0x01) {
out.extend_from_slice(&chunk[1..]);
}
i += len;
}
out
}
// Shared harness for the real-git server tests: a minimal smart-HTTP server
// backed by the real info_refs + upload_pack_excluding.
#[derive(Clone)]
struct FilterState {
repo: std::path::PathBuf,
withheld: HashSet<String>,
}
async fn refs_handler(
axum::extract::State(st): axum::extract::State<std::sync::Arc<FilterState>>,
axum::extract::Query(q): axum::extract::Query<std::collections::HashMap<String, String>>,
) -> Response {
let service = q.get("service").cloned().unwrap_or_default();
info_refs(&st.repo, &service).await.unwrap()
}
async fn pack_handler(
axum::extract::State(st): axum::extract::State<std::sync::Arc<FilterState>>,
body: Bytes,
) -> Response {
upload_pack_excluding(&st.repo, body, &st.withheld)
.await
.unwrap()
}
/// Spawn the server for `bare`, withholding `withheld`. Returns the clone URL
/// and the server task (abort it when done).
async fn spawn_filter_server(
bare: std::path::PathBuf,
withheld: HashSet<String>,
) -> (String, tokio::task::JoinHandle<()>) {
use axum::routing::{get, post};
let state = std::sync::Arc::new(FilterState {
repo: bare,
withheld,
});
let app = axum::Router::new()
.route("/repo.git/info/refs", get(refs_handler))
.route("/repo.git/git-upload-pack", post(pack_handler))
.with_state(state);
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
let handle = tokio::spawn(async move {
axum::serve(listener, app).await.unwrap();
});
(format!("http://127.0.0.1:{port}/repo.git"), handle)
}
fn run_git(args: &[&str], dir: &std::path::Path) {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
}
/// Build a work repo (public/a.txt, secret/b.txt) and a bare clone of it.
/// Returns (work, bare, secret_blob_oid, public_blob_oid).
fn fixture_with_secret(
td: &TempDir,
) -> (std::path::PathBuf, std::path::PathBuf, String, String) {
let work = td.path().join("work");
let bare = td.path().join("bare.git");
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"pub\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"SECRET\n").unwrap();
run_git(&["init", "-q"], &work);
run_git(&["config", "user.email", "t@t"], &work);
run_git(&["config", "user.name", "t"], &work);
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "init"], &work);
let oid = |p: &str| {
let o = Command::new("git")
.args(["rev-parse", &format!("HEAD:{p}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let secret_oid = oid("secret/b.txt");
let public_oid = oid("public/a.txt");
run_git(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
(work, bare, secret_oid, public_oid)
}
/// Enumerate exactly the objects a repo physically has (no promisor lazy
/// fetch), so tests assert on what bytes actually crossed the wire.
fn local_object_ids(repo: &std::path::Path) -> String {
let out = Command::new("git")
.args(["cat-file", "--batch-all-objects", "--batch-check"])
.current_dir(repo)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).into_owned()
}
/// End-to-end: a real `git` client clones through `info_refs` +
/// `upload_pack_excluding` and ends up without the withheld blob's bytes
/// while still seeing its tree entry (SHA). Uses a partial clone
/// (`--filter`) because a pack that omits a referenced blob is only
/// accepted by a promisor-aware client; a stock full clone is refused at
/// fetch time by the connectivity check.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn real_git_partial_clone_omits_withheld_blob() {
let td = TempDir::new().unwrap();
let (_work, bare, secret_oid, public_oid) = fixture_with_secret(&td);
let (url, server) = spawn_filter_server(bare, HashSet::from([secret_oid.clone()])).await;
let dest = td.path().join("clone");
let dest_s = dest.to_str().unwrap().to_string();
let out = tokio::task::spawn_blocking(move || {
Command::new("git")
.args([
"-c",
"protocol.version=2",
"clone",
"--filter=blob:none",
"--no-checkout",
"-q",
&url,
&dest_s,
])
.output()
.unwrap()
})
.await
.unwrap();
assert!(
out.status.success(),
"clone failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// The public blob is present in the clone, the withheld blob is not.
let local = local_object_ids(&dest);
assert!(
local.contains(&public_oid),
"public blob should be present in the clone"
);
assert!(
!local.contains(&secret_oid),
"withheld blob bytes must be absent from the clone"
);
// The tree entry (and SHA) for the private file is still visible.
let tree = Command::new("git")
.args(["ls-tree", "-r", "HEAD"])
.current_dir(&dest)
.output()
.unwrap();
let tree = String::from_utf8_lossy(&tree.stdout);
assert!(
tree.contains(&secret_oid) && tree.contains("secret/b.txt"),
"the private path and its blob SHA must remain visible: {tree}"
);
server.abort();
}
/// End-to-end: an incremental `git fetch` after a partial clone still works
/// and still withholds the private blob. The serve path ignores the client's
/// have/want negotiation and always sends a self-contained pack of all refs
/// minus the withheld blobs (it replies NAK, so the client treats it as "no
/// common commits" and accepts the full set). This is correct, just not
/// bandwidth-optimal; thin-pack/negotiation is an optimization follow-up.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn real_git_fetch_after_partial_clone_still_withholds() {
let td = TempDir::new().unwrap();
let (work, bare, secret_oid, _public_oid) = fixture_with_secret(&td);
let branch = {
let o = Command::new("git")
.args(["symbolic-ref", "--short", "HEAD"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let (url, server) =
spawn_filter_server(bare.clone(), HashSet::from([secret_oid.clone()])).await;
// Partial-clone the initial state.
let dest = td.path().join("clone");
let dest_s = dest.to_str().unwrap().to_string();
let url_c = url.clone();
let out = tokio::task::spawn_blocking(move || {
Command::new("git")
.args([
"-c",
"protocol.version=2",
"clone",
"--filter=blob:none",
"--no-checkout",
"-q",
&url_c,
&dest_s,
])
.output()
.unwrap()
})
.await
.unwrap();
assert!(
out.status.success(),
"clone failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// Add a new public commit on the server side.
std::fs::write(work.join("public/c.txt"), b"v2\n").unwrap();
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "c2"], &work);
let new_oid = {
let o = Command::new("git")
.args(["rev-parse", "HEAD:public/c.txt"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
run_git(&["push", "-q", bare.to_str().unwrap(), &branch], &work);
// Incremental fetch: the client has c1 and asks for the update.
let dest_f = dest.clone();
let out = tokio::task::spawn_blocking(move || {
Command::new("git")
.args(["-c", "protocol.version=2", "fetch", "-q", "origin"])
.current_dir(&dest_f)
.output()
.unwrap()
})
.await
.unwrap();
assert!(
out.status.success(),
"fetch failed: {}",
String::from_utf8_lossy(&out.stderr)
);
// The new commit's blob arrived; the withheld blob is still absent.
let local = local_object_ids(&dest);
assert!(
local.contains(&new_oid),
"the new commit's blob must be fetched"
);
assert!(
!local.contains(&secret_oid),
"withheld blob must remain absent after fetch"
);
server.abort();
}
}
+83
View File
@@ -314,6 +314,39 @@ pub fn branch_diff(repo_path: &Path, target_branch: &str, source_branch: &str) -
Ok(String::from_utf8_lossy(&output.stdout).to_string())
}
/// The repo-relative paths changed by `git diff target...source` (the same range
/// as `branch_diff`). Used to enforce per-path visibility on a PR diff: if the
/// caller cannot read one of these paths, the diff is withheld.
pub fn branch_diff_names(
repo_path: &Path,
target_branch: &str,
source_branch: &str,
) -> Result<Vec<String>> {
let output = Command::new("git")
.args([
"diff",
"--name-only",
"-z",
&format!("{target_branch}...{source_branch}"),
])
.current_dir(repo_path)
.output()
.context("failed to run git diff --name-only")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git diff --name-only failed: {stderr}");
}
// Split on NUL (`-z`) so paths containing newlines keep their exact bytes;
// `--name-only` without `-z` would quote/escape such paths and they would no
// longer match the visibility globs in get_pr_diff, leaking the diff.
Ok(output
.stdout
.split(|b| *b == b'\0')
.filter(|s| !s.is_empty())
.map(|s| String::from_utf8_lossy(s).into_owned())
.collect())
}
/// Merge source_branch into target_branch in a bare repo using a temporary worktree.
/// Returns the new merge commit hash.
pub fn merge_branch(
@@ -399,3 +432,53 @@ pub fn repo_disk_path(repos_dir: &Path, owner_did: &str, repo_name: &str) -> Pat
let owner_slug = owner_did.replace([':', '/'], "_");
repos_dir.join(owner_slug).join(format!("{repo_name}.git"))
}
#[cfg(test)]
mod tests {
use super::branch_diff_names;
use std::path::Path;
use std::process::Command;
#[test]
fn branch_diff_names_lists_changed_paths() {
let td = tempfile::TempDir::new().unwrap();
let work: &Path = td.path();
let g = |args: &[&str]| {
assert!(Command::new("git")
.args(args)
.current_dir(work)
.status()
.unwrap()
.success());
};
g(&["init", "-q"]);
g(&["config", "user.email", "t@t"]);
g(&["config", "user.name", "t"]);
std::fs::write(work.join("base.txt"), b"base\n").unwrap();
g(&["add", "."]);
g(&["commit", "-qm", "base"]);
let main = {
let o = Command::new("git")
.args(["symbolic-ref", "--short", "HEAD"])
.current_dir(work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
g(&["checkout", "-q", "-b", "feature"]);
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::write(work.join("secret/x.txt"), b"secret\n").unwrap();
g(&["add", "."]);
g(&["commit", "-qm", "feat"]);
let names = branch_diff_names(work, &main, "feature").unwrap();
assert!(
names.iter().any(|p| p == "secret/x.txt"),
"expected secret/x.txt in changed paths, got {names:?}"
);
assert!(
!names.iter().any(|p| p == "base.txt"),
"unchanged file must not appear: {names:?}"
);
}
}
+38 -8
View File
@@ -157,16 +157,46 @@ fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
}
/// Decompress a tar.zst byte vector into a local directory.
///
/// Extraction is atomic with respect to `local_path`: the archive is unpacked
/// into a sibling temp directory first, and only swapped into place once it
/// fully succeeds. A corrupt or truncated archive therefore can never clobber a
/// good existing copy at `local_path` — on failure we discard the temp dir and
/// leave `local_path` exactly as it was.
fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
// Ensure parent directory exists
if let Some(parent) = local_path.parent() {
std::fs::create_dir_all(parent).context("creating parent dir")?;
}
std::fs::create_dir_all(local_path).context("creating repo dir")?;
let parent = local_path.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let decoder = zstd::stream::Decoder::new(data)?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(local_path).context("unpacking tar.zst")?;
let file_name = local_path
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
let tmp_dir = parent.join(format!(".{file_name}.tmp-extract"));
// Clear any leftover temp dir from a previously-interrupted extraction.
let _ = std::fs::remove_dir_all(&tmp_dir);
std::fs::create_dir_all(&tmp_dir).context("creating temp extract dir")?;
// Unpack into the temp dir; on any failure, clean up and bail without
// touching local_path.
let unpack = (|| -> Result<()> {
let decoder = zstd::stream::Decoder::new(data)?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(&tmp_dir).context("unpacking tar.zst")?;
Ok(())
})();
if let Err(e) = unpack {
let _ = std::fs::remove_dir_all(&tmp_dir);
return Err(e);
}
// Swap the freshly-extracted repo into place. rename within the same parent
// is effectively atomic, but most platforms refuse to rename onto a
// non-empty dir, so remove the old copy first.
if local_path.exists() {
std::fs::remove_dir_all(local_path).context("removing stale repo dir")?;
}
std::fs::rename(&tmp_dir, local_path).context("swapping extracted repo into place")?;
Ok(())
}
@@ -0,0 +1,298 @@
//! Resolve which blob OIDs must be withheld from a caller because every path
//! at which the blob appears is denied by the repo's visibility rules. Trees
//! and commits are never withheld (mode B keeps SHAs intact); only blob
//! content is held back.
use crate::db::VisibilityRule;
use crate::git::store;
use crate::visibility::{visibility_check, Decision};
use anyhow::{Context, Result};
use std::collections::{BTreeSet, HashMap, HashSet};
use std::path::Path;
/// List every (blob_oid, "/repo/relative/path") pair reachable from any branch
/// ref in `repo_path`. Uses `git ls-tree -r` per ref so each path a blob lives
/// at is represented (the same blob content can appear at several paths). Paths
/// are returned with a leading "/" to match the glob form used by visibility
/// rules ("/secret/**").
fn blob_paths(repo_path: &Path) -> Result<Vec<(String, String)>> {
let refs = store::list_refs(repo_path).context("list_refs failed")?;
let mut out = Vec::new();
for (refname, _oid) in refs {
if !refname.starts_with("refs/heads/") && !refname.starts_with("refs/tags/") {
continue;
}
let listing = std::process::Command::new("git")
.args(["ls-tree", "-r", &refname])
.current_dir(repo_path)
.output()
.context("git ls-tree -r failed")?;
if !listing.status.success() {
continue;
}
for line in String::from_utf8_lossy(&listing.stdout).lines() {
// "<mode> blob <oid>\t<path>"
let Some((meta, path)) = line.split_once('\t') else {
continue;
};
let mut parts = meta.split_whitespace();
let _mode = parts.next();
let kind = parts.next();
let oid = parts.next();
if kind == Some("blob") {
if let Some(oid) = oid {
out.push((oid.to_string(), format!("/{path}")));
}
}
}
}
Ok(out)
}
/// Blob OIDs the caller may not read. A blob is withheld only if visibility
/// denies the caller at *every* path the blob appears at; a blob that is also
/// reachable through an allowed path is sent (its content is public elsewhere).
///
/// The whole-repo "/" gate is handled by the caller before this function runs:
/// if "/" denies, the caller gets a 404 and never reaches the filtered serve.
pub fn withheld_blob_oids(
repo_path: &Path,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Result<HashSet<String>> {
let mut denied: HashSet<String> = HashSet::new();
let mut allowed: HashSet<String> = HashSet::new();
for (oid, path) in blob_paths(repo_path)? {
match visibility_check(rules, is_public, owner_did, caller, &path) {
Decision::Deny => {
denied.insert(oid);
}
Decision::Allow => {
allowed.insert(oid);
}
}
}
Ok(denied.difference(&allowed).cloned().collect())
}
/// Objects that may replicate to the public: everything not in `withheld`.
/// Order-preserving. The single seam every replication site (IPFS, Pinata)
/// passes its object list through; option B would later reroute the withheld
/// ones through encrypt-then-pin instead of dropping them.
pub fn replicable_objects(all: Vec<String>, withheld: &HashSet<String>) -> Vec<String> {
all.into_iter()
.filter(|oid| !withheld.contains(oid))
.collect()
}
/// For every blob withheld from anonymous, the DIDs allowed to read it: the
/// owner plus any reader DID that `visibility_check` Allows at some path the
/// blob appears at. Least-privilege: a reader of one private subtree is not a
/// recipient of a blob that only lives in another.
pub fn withheld_blob_recipients(
repo_path: &Path,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
) -> Result<HashMap<String, BTreeSet<String>>> {
let withheld = withheld_blob_oids(repo_path, rules, is_public, owner_did, None)?;
if withheld.is_empty() {
return Ok(HashMap::new());
}
let mut candidates: BTreeSet<String> = BTreeSet::new();
for r in rules {
for d in &r.reader_dids {
candidates.insert(d.clone());
}
}
let mut out: HashMap<String, BTreeSet<String>> = HashMap::new();
for (oid, path) in blob_paths(repo_path)? {
if !withheld.contains(&oid) {
continue;
}
let entry = out.entry(oid).or_default();
entry.insert(owner_did.to_string());
for did in &candidates {
if visibility_check(rules, is_public, owner_did, Some(did), &path) == Decision::Allow {
entry.insert(did.clone());
}
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::VisibilityMode;
use chrono::Utc;
use std::process::Command;
use tempfile::TempDir;
fn rule(path_glob: &str, readers: &[&str]) -> VisibilityRule {
VisibilityRule {
id: "x".into(),
repo_id: "r1".into(),
path_glob: path_glob.into(),
mode: VisibilityMode::B,
reader_dids: readers.iter().map(|s| s.to_string()).collect(),
created_by: "did:key:zOwner".into(),
created_at: Utc::now(),
}
}
const OWNER: &str = "did:key:zOwner";
/// Build a bare repo with public/a.txt and secret/b.txt at one commit.
/// Returns (tempdir, bare_path, secret_blob_oid, public_blob_oid).
fn fixture() -> (TempDir, std::path::PathBuf, String, String) {
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("bare.git");
let run = |args: &[&str], dir: &Path| {
let ok = Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success();
assert!(ok, "git {args:?} failed");
};
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::write(work.join("public/a.txt"), b"public bytes\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"TOP SECRET\n").unwrap();
run(&["init", "-q"], &work);
run(&["config", "user.email", "t@t"], &work);
run(&["config", "user.name", "t"], &work);
run(&["add", "."], &work);
run(&["commit", "-qm", "init"], &work);
let oid = |path: &str| {
let out = Command::new("git")
.args(["rev-parse", &format!("HEAD:{path}")])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let secret = oid("secret/b.txt");
let public = oid("public/a.txt");
run(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
(td, bare, secret, public)
}
#[test]
fn anonymous_caller_withholds_only_private_blob() {
let (_td, bare, secret_oid, public_oid) = fixture();
let rules = [rule("/secret/**", &[])];
// caller = None models the public / any peer: what must not replicate.
let withheld = withheld_blob_oids(&bare, &rules, true, OWNER, None).unwrap();
assert!(
withheld.contains(&secret_oid),
"secret blob must be withheld"
);
assert!(
!withheld.contains(&public_oid),
"public blob must replicate"
);
// Trees and commits are never withheld; the set holds only the secret blob.
assert_eq!(withheld.len(), 1, "only the secret blob OID is withheld");
}
#[test]
fn non_reader_withholds_only_the_private_blob() {
let (_td, bare, secret, public) = fixture();
let rules = [rule("/secret/**", &["did:key:zFriend"])];
let withheld =
withheld_blob_oids(&bare, &rules, true, OWNER, Some("did:key:zStranger")).unwrap();
assert!(withheld.contains(&secret), "secret blob must be withheld");
assert!(
!withheld.contains(&public),
"public blob must NOT be withheld"
);
}
#[test]
fn owner_withholds_nothing() {
let (_td, bare, secret, public) = fixture();
let rules = [rule("/secret/**", &["did:key:zFriend"])];
let withheld = withheld_blob_oids(&bare, &rules, true, OWNER, Some(OWNER)).unwrap();
assert!(withheld.is_empty(), "owner sees everything");
let _ = (secret, public);
}
#[test]
fn listed_reader_withholds_nothing() {
let (_td, bare, _secret, _public) = fixture();
let rules = [rule("/secret/**", &["did:key:zFriend"])];
let withheld =
withheld_blob_oids(&bare, &rules, true, OWNER, Some("did:key:zFriend")).unwrap();
assert!(withheld.is_empty(), "listed reader sees the subtree");
}
#[test]
fn no_subtree_rules_withholds_nothing() {
let (_td, bare, _secret, _public) = fixture();
let withheld = withheld_blob_oids(&bare, &[], true, OWNER, None).unwrap();
assert!(
withheld.is_empty(),
"public repo, no rules, nothing withheld"
);
}
#[test]
fn replicable_objects_drops_withheld_keeps_rest() {
let all = vec!["aaa".to_string(), "bbb".to_string(), "ccc".to_string()];
let withheld: HashSet<String> = ["bbb".to_string()].into_iter().collect();
let got = replicable_objects(all, &withheld);
assert_eq!(got, vec!["aaa".to_string(), "ccc".to_string()]);
}
#[test]
fn replicable_objects_empty_withheld_keeps_all() {
let all = vec!["aaa".to_string(), "bbb".to_string()];
let withheld: HashSet<String> = HashSet::new();
let got = replicable_objects(all.clone(), &withheld);
assert_eq!(got, all);
}
#[test]
fn recipients_are_owner_plus_allowed_readers_only() {
let (_td, repo, secret_oid, public_oid) = fixture();
let reader = "did:key:zReader";
let rules = vec![rule("/secret/**", &[reader])];
let map = withheld_blob_recipients(&repo, &rules, true, OWNER).unwrap();
let recips = map.get(&secret_oid).expect("secret blob has recipients");
assert!(recips.contains(OWNER));
assert!(recips.contains(reader));
assert!(
!map.contains_key(&public_oid),
"public blob is not encrypted"
);
}
#[test]
fn node_seal_open_round_trip() {
use gitlawb_core::encrypt::{open_blob, seal_blob};
use gitlawb_core::identity::Keypair;
let (_td, repo, secret_oid, _public) = fixture();
let (_t, bytes) = crate::git::store::read_object(&repo, &secret_oid)
.unwrap()
.unwrap();
let reader = Keypair::generate();
let env = seal_blob(&bytes, &[reader.verifying_key()]).unwrap();
assert_eq!(open_blob(&env, &reader).unwrap(), bytes);
}
}
+18
View File
@@ -7,6 +7,8 @@
//! If `ipfs_api` is empty the functions are no-ops, so the node works fine
//! without a local IPFS daemon.
use std::collections::HashSet;
use anyhow::Result;
use gitlawb_core::cid::Cid;
@@ -70,6 +72,19 @@ pub async fn pin_git_object(ipfs_api: &str, sha256_hex: &str, data: &[u8]) -> Re
Ok(cid)
}
/// Fetch raw bytes for a CID from the local Kubo node (`/api/v0/cat`).
pub async fn cat(ipfs_api: &str, cid: &str) -> Result<Vec<u8>> {
if ipfs_api.is_empty() {
return Err(anyhow::anyhow!("IPFS not configured"));
}
let url = format!("{}/api/v0/cat?arg={}", ipfs_api.trim_end_matches('/'), cid);
let resp = reqwest::Client::new().post(&url).send().await?;
if !resp.status().is_success() {
return Err(anyhow::anyhow!("ipfs cat {cid}: {}", resp.status()));
}
Ok(resp.bytes().await?.to_vec())
}
/// List all git objects in the given bare repo and pin any that are not yet
/// recorded in `pinned_cids`.
///
@@ -78,6 +93,7 @@ pub async fn pin_new_objects(
ipfs_api: &str,
repo_path: &std::path::Path,
db: &crate::db::Db,
withheld: &HashSet<String>,
) -> Vec<(String, String)> {
if ipfs_api.is_empty() {
return vec![];
@@ -92,6 +108,8 @@ pub async fn pin_new_objects(
}
};
let object_list = crate::git::visibility_pack::replicable_objects(object_list, withheld);
let mut pinned = Vec::new();
for sha in object_list {
+292 -29
View File
@@ -5,24 +5,30 @@ mod bootstrap;
mod cert;
mod config;
mod db;
mod encrypted_pin;
mod error;
mod git;
mod graphql;
mod ipfs_pin;
mod metrics;
mod operator;
mod p2p;
mod pinata;
mod rate_limit;
mod server;
mod state;
mod sync;
mod visibility;
mod webhooks;
use anyhow::{Context, Result};
use clap::Parser;
use std::sync::Arc;
use tokio::net::TcpListener;
use tokio::sync::watch;
use tracing::{info, warn};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use config::Config;
@@ -45,10 +51,22 @@ async fn main() -> Result<()> {
// bootstrap peers. Operators can opt out via GITLAWB_BOOTSTRAP_DISABLE_SEEDS.
bootstrap::merge_seeds(&mut config);
if !config.public_read {
warn!(
"GITLAWB_PUBLIC_READ=false is reserved; per-repository private-read enforcement is not wired in alpha"
);
}
// Load or generate the node's identity keypair
let keypair = load_or_create_keypair(&config)?;
let node_did = keypair.did();
// One-time metrics init. Must run before any handler that calls into
// `metrics::record_*` so the registry exists when the first event fires.
// Safe to call even when GITLAWB_METRICS_ADDR is unset — those helpers
// are simply no-ops until something reads from the registry.
metrics::init(env!("CARGO_PKG_VERSION"), &node_did.to_string());
info!("╔══════════════════════════════════════════╗");
info!(
"║ gitlawb node v{} ║",
@@ -58,6 +76,12 @@ async fn main() -> Result<()> {
info!(did = %node_did, "node identity");
info!(addr = %config.bind_addr(), "listening");
// Process-wide shutdown signal. One sender lives in AppState (cloned
// into every handler); main() keeps a clone and flips it on SIGINT
// or SIGTERM. Tasks that hold a watch::Receiver get notified at
// their next await point.
let (shutdown_tx, _shutdown_rx_for_main) = watch::channel(false);
// Connect to PostgreSQL database
let db = Arc::new(
Db::connect(&config.database_url)
@@ -84,12 +108,14 @@ async fn main() -> Result<()> {
.iter()
.filter_map(|s| s.parse().ok())
.collect();
let shutdown_rx = shutdown_tx.subscribe();
match p2p::start(
&node_did.to_string(),
config.p2p_port,
bootstrap_addrs,
Arc::clone(&db),
config.auto_sync,
shutdown_rx,
)
.await
{
@@ -147,6 +173,8 @@ async fn main() -> Result<()> {
let repo_store =
git::repo_store::RepoStore::new(config.repos_dir.clone(), tigris, db.pool().clone());
let rate_limiter = rate_limit::RateLimiter::new(10, std::time::Duration::from_secs(3600));
let state = AppState {
config: Arc::new(config.clone()),
db,
@@ -159,8 +187,84 @@ async fn main() -> Result<()> {
graphql_schema,
machine_id,
repo_store,
rate_limiter,
shutdown_tx: shutdown_tx.clone(),
};
// Periodic peer-count poll for the metrics gauge. If p2p is disabled
// we still set the gauge to 0 so dashboards don't show "no data".
{
let p2p_for_metrics = state.p2p.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(15));
loop {
tokio::select! {
_ = interval.tick() => {
let count = match &p2p_for_metrics {
Some(h) => h.status().await.map(|s| s.connected_peers).unwrap_or(0),
None => 0,
};
metrics::set_peers_connected(count as i64);
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
return;
}
}
}
}
});
}
// Spawn a task that flips the shutdown signal on SIGINT or SIGTERM.
// On Unix, both signals are handled. On Windows, only Ctrl-C is
// supported by tokio::signal::ctrl_c.
{
let tx = shutdown_tx.clone();
tokio::spawn(async move {
#[cfg(unix)]
{
use tokio::signal::unix::{signal as unix_signal, SignalKind};
let mut sigterm =
unix_signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut sigint =
unix_signal(SignalKind::interrupt()).expect("install SIGINT handler");
tokio::select! {
_ = sigterm.recv() => info!("SIGTERM received, shutting down"),
_ = sigint.recv() => info!("SIGINT received, shutting down"),
}
}
#[cfg(not(unix))]
{
use tokio::signal;
let _ = signal::ctrl_c().await;
info!("Ctrl-C received, shutting down");
}
tx.send(true).ok();
});
}
// Periodic cleanup of expired rate limit entries
{
let rl = state.rate_limiter.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
loop {
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(300)) => {
rl.cleanup().await;
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
break;
}
}
}
}
});
}
let router = server::build_router(state.clone());
let listener = TcpListener::bind(config.bind_addr())
.await
@@ -173,6 +277,23 @@ async fn main() -> Result<()> {
&config.database_url.split('@').next_back().unwrap_or("?")
);
// Optional Prometheus metrics listener on a separate port.
let metrics_handle = if !config.metrics_addr.is_empty() {
match spawn_metrics_server(&config.metrics_addr, state.clone()).await {
Ok(handle) => {
info!(addr = %config.metrics_addr, "metrics endpoint listening");
Some(handle)
}
Err(e) => {
warn!(err = %e, addr = %config.metrics_addr, "failed to start metrics endpoint — continuing without");
None
}
}
} else {
info!("metrics endpoint disabled (GITLAWB_METRICS_ADDR not set)");
None
};
// Publish our DID record to the Kademlia DHT shortly after startup
if let Some(p2p) = &state.p2p {
let did_record = p2p::DidRecord {
@@ -183,9 +304,13 @@ async fn main() -> Result<()> {
timestamp: chrono::Utc::now().to_rfc3339(),
};
let p2p_clone = Arc::clone(p2p);
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
// Small delay so Kademlia can find peers first
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(5)) => {}
_ = shutdown_rx.changed() => return,
}
p2p_clone.put_did(did_record).await;
info!("DID record published to Kademlia DHT");
});
@@ -195,14 +320,20 @@ async fn main() -> Result<()> {
{
let gossip_state = state.clone();
let bootstrap_peers = config.bootstrap_peers.clone();
let shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
gossip_task(gossip_state, bootstrap_peers).await;
gossip_task(gossip_state, bootstrap_peers, shutdown_rx).await;
});
}
// Start multi-node sync worker if auto_sync is enabled
if config.auto_sync {
sync::start(Arc::clone(&state.db), Arc::clone(&state.config));
sync::start(
Arc::clone(&state.db),
Arc::clone(&state.config),
Arc::clone(&state.node_keypair),
state.subscribe_shutdown(),
);
info!("auto-sync worker started");
}
@@ -214,7 +345,7 @@ async fn main() -> Result<()> {
Ok(client) => match operator::startup_check(&client).await {
Ok(_) => {
let arc_client = Arc::new(client);
arc_client.spawn_heartbeat_loop();
arc_client.spawn_heartbeat_loop(state.subscribe_shutdown());
}
Err(e) => {
if state.config.operator_strict_mode {
@@ -234,10 +365,90 @@ async fn main() -> Result<()> {
info!("on-chain PoS disabled (GITLAWB_CONTRACT_NODE_STAKING or GITLAWB_OPERATOR_PRIVATE_KEY unset)");
}
axum::serve(listener, router).await?;
// axum's `with_graceful_shutdown` waits for in-flight requests to
// complete (up to the configured grace) once the future resolves.
let shutdown_signal_for_axum = state.subscribe_shutdown();
let grace = std::time::Duration::from_secs(config.shutdown_grace_secs);
info!(grace_secs = config.shutdown_grace_secs, "axum server ready");
let serve_result = axum::serve(listener, router)
.with_graceful_shutdown(async move {
let mut rx = shutdown_signal_for_axum;
// Wait until the watcher flips to true, then return so axum
// can begin draining.
while !*rx.borrow_and_update() {
if rx.changed().await.is_err() {
// Sender dropped — treat as shutdown.
break;
}
}
})
.await;
// Server has stopped accepting new connections and drained in-flight
// requests. Tear the rest of the system down.
info!("HTTP server stopped, beginning process shutdown");
if let Some(h) = metrics_handle {
h.abort();
}
let _ = grace; // recorded for operators in the log above; not enforced
serve_result?;
info!("clean exit");
Ok(())
}
/// Spawn a small axum router that exposes only `GET /metrics` on its own
/// listener. Returns the JoinHandle so `main()` can abort it on shutdown.
/// This is deliberately separate from the main router so the metrics port
/// can be firewalled differently from the API port — bind to localhost
/// or a private interface only.
async fn spawn_metrics_server(addr: &str, state: AppState) -> Result<tokio::task::JoinHandle<()>> {
use axum::{response::IntoResponse, routing::get, Router};
async fn metrics_handler() -> impl IntoResponse {
match metrics::encode() {
Ok(body) => (
axum::http::StatusCode::OK,
[(
axum::http::header::CONTENT_TYPE,
"text/plain; version=0.0.4; charset=utf-8",
)],
body,
),
Err(e) => (
axum::http::StatusCode::INTERNAL_SERVER_ERROR,
[(
axum::http::header::CONTENT_TYPE,
"text/plain; charset=utf-8",
)],
format!("metrics encode error: {e}"),
),
}
}
let mut shutdown_rx = state.subscribe_shutdown();
let listener = TcpListener::bind(addr)
.await
.with_context(|| format!("failed to bind metrics listener to {addr}"))?;
let app = Router::new().route("/metrics", get(metrics_handler));
let handle = tokio::spawn(async move {
if let Err(e) = axum::serve(listener, app)
.with_graceful_shutdown(async move {
while !*shutdown_rx.borrow_and_update() {
if shutdown_rx.changed().await.is_err() {
break;
}
}
})
.await
{
warn!(err = %e, "metrics server exited with error");
}
});
Ok(handle)
}
fn build_operator_client(
config: &config::Config,
node_did: &str,
@@ -260,9 +471,21 @@ fn build_operator_client(
}
/// Announce to bootstrap peers on startup, then periodically ping all known peers.
async fn gossip_task(state: AppState, bootstrap_peers: Vec<String>) {
// Small delay to let the HTTP server come up before we try to announce
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
async fn gossip_task(
state: AppState,
bootstrap_peers: Vec<String>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) {
// If shutdown arrives during the initial delay, exit before announcing.
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(2)) => {}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("gossip: shutdown during startup delay, exiting");
return;
}
}
}
let client = reqwest::Client::new();
let my_did = state.node_did.to_string();
@@ -270,13 +493,43 @@ async fn gossip_task(state: AppState, bootstrap_peers: Vec<String>) {
// Announce ourselves to each bootstrap peer
for peer_url in &bootstrap_peers {
let announce_url = format!("{}/api/v1/peers/announce", peer_url.trim_end_matches('/'));
// Cooperative shutdown between peers — a slow peer shouldn't
// block the node exiting.
if *shutdown_rx.borrow() {
info!("gossip: shutdown signalled during peer announce, exiting");
return;
}
let path = "/api/v1/peers/announce";
let announce_url = format!("{}{}", peer_url.trim_end_matches('/'), path);
let body = serde_json::json!({
"did": my_did,
"http_url": my_url,
"did": my_did.clone(),
"http_url": my_url.clone(),
});
match client.post(&announce_url).json(&body).send().await {
Ok(resp) => {
let body_bytes = match serde_json::to_vec(&body) {
Ok(bytes) => bytes,
Err(e) => {
tracing::warn!(err = %e, "failed to serialize peer announce body");
continue;
}
};
let signed = sign_request(state.node_keypair.as_ref(), "POST", path, &body_bytes);
// Per-request timeout inside the loop; do not let one hung peer
// block others. The request itself is a normal tokio future so
// it's cancel-safe on shutdown.
match tokio::time::timeout(
std::time::Duration::from_secs(5),
client
.post(&announce_url)
.header("Content-Type", "application/json")
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature)
.body(body_bytes)
.send(),
)
.await
{
Ok(Ok(resp)) => {
if resp.status().is_success() {
if let Ok(json) = resp.json::<serde_json::Value>().await {
// Add them back to our peer list
@@ -292,29 +545,39 @@ async fn gossip_task(state: AppState, bootstrap_peers: Vec<String>) {
}
}
}
Err(e) => {
Ok(Err(e)) => {
tracing::warn!(url = %announce_url, err = %e, "failed to announce to bootstrap peer")
}
Err(_) => tracing::warn!(url = %announce_url, "bootstrap peer announce timed out (5s)"),
}
}
// Periodic ping every 5 minutes
// Periodic ping every 5 minutes — exit on shutdown.
let mut interval = tokio::time::interval(std::time::Duration::from_secs(300));
loop {
interval.tick().await;
let peers = match state.db.list_peers().await {
Ok(p) => p,
Err(_) => continue,
};
for peer in peers {
let url = format!("{}/health", peer.http_url.trim_end_matches('/'));
let ok = client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false);
let _ = state.db.mark_peer_ping(&peer.did, ok).await;
tokio::select! {
_ = interval.tick() => {
let peers = match state.db.list_peers().await {
Ok(p) => p,
Err(_) => continue,
};
for peer in peers {
let url = format!("{}/health", peer.http_url.trim_end_matches('/'));
let ok = client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false);
let _ = state.db.mark_peer_ping(&peer.did, ok).await;
}
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("gossip task: shutdown signal received, exiting");
return;
}
}
}
}
}
+352
View File
@@ -0,0 +1,352 @@
//! Prometheus metrics for the gitlawb node.
//!
//! Exposes a small, deliberately conservative set of counters and histograms
//! that cover the questions an operator actually asks of a live node:
//!
//! * is push traffic flowing? — `gitlawb_pushes_total`
//! * is fetch traffic flowing? — `gitlawb_fetches_total`
//! * are signature checks passing or failing? —
//! `gitlawb_auth_successes_total` / `gitlawb_auth_failures_total`
//! * is the sync worker making progress? —
//! `gitlawb_sync_queue_processed_total{status}`
//! * are webhooks reaching their endpoints? —
//! `gitlawb_webhook_deliveries_total{result}`
//! * how big are the packs we're sending and receiving? —
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
//! * currently-connected peer count — `gitlawb_peers_connected`
//!
//! All metrics live in a single process-wide registry initialized by
//! [`init`]. Increment helpers (`record_push`, `record_auth_failure`, ...)
//! are no-ops until [`init`] has been called, so unit tests and benchmarks
//! that don't go through `main()` don't need to do anything special.
//!
//! The endpoint is opt-in via `GITLAWB_METRICS_ADDR`; if unset, no listener
//! is bound and no exposition happens. Increments are atomic and effectively
//! free.
//!
//! Follow-ups (not in this module):
//! * per-route latency histograms (TraceLayer already gives us spans)
//! * per-peer p2p counters
//! * ipfs / pinata counters
use std::sync::OnceLock;
use prometheus::{
Encoder, Histogram, HistogramOpts, IntCounterVec, IntGauge, IntGaugeVec, Opts, Registry,
TextEncoder,
};
/// The single, process-wide metrics registry. Initialized by [`init`].
/// Wrapped in `OnceLock` so unit tests can run without going through
/// `main()` — the public `record_*` helpers treat an uninitialized
/// registry as a silent no-op.
static REGISTRY: OnceLock<Registry> = OnceLock::new();
static INFO: OnceLock<IntGaugeVec> = OnceLock::new();
static PUSHES: OnceLock<IntCounterVec> = OnceLock::new();
static FETCHES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_SUCCESSES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_FAILURES: OnceLock<IntCounterVec> = OnceLock::new();
static SYNC_PROCESSED: OnceLock<IntCounterVec> = OnceLock::new();
static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
/// One-time initializer. Builds the registry, registers every metric,
/// and sets the constant `gitlawb_info` gauge. Idempotent — calling
/// more than once is a silent no-op. MUST be called from `main()` after
/// the node DID is known.
pub fn init(version: &str, node_did: &str) {
if REGISTRY.get().is_some() {
return;
}
let registry = Registry::new();
let info = IntGaugeVec::new(
Opts::new(
"gitlawb_info",
"Constant 1 labelled with version and node DID",
),
&["version", "did"],
)
.expect("gitlawb_info metric definition");
registry
.register(Box::new(info.clone()))
.expect("register gitlawb_info");
info.with_label_values(&[version, node_did]).set(1);
INFO.set(info).expect("set INFO once");
let pushes = IntCounterVec::new(
Opts::new(
"gitlawb_pushes_total",
"Total successful git push (receive-pack) completions",
),
&["repo"],
)
.expect("gitlawb_pushes_total definition");
registry
.register(Box::new(pushes.clone()))
.expect("register gitlawb_pushes_total");
PUSHES.set(pushes).expect("set PUSHES once");
let fetches = IntCounterVec::new(
Opts::new(
"gitlawb_fetches_total",
"Total successful git fetch (upload-pack) completions",
),
&["repo"],
)
.expect("gitlawb_fetches_total definition");
registry
.register(Box::new(fetches.clone()))
.expect("register gitlawb_fetches_total");
FETCHES.set(fetches).expect("set FETCHES once");
let auth_successes = IntCounterVec::new(
Opts::new(
"gitlawb_auth_successes_total",
"Total HTTP signature checks that passed",
),
&["route"],
)
.expect("gitlawb_auth_successes_total definition");
registry
.register(Box::new(auth_successes.clone()))
.expect("register gitlawb_auth_successes_total");
AUTH_SUCCESSES
.set(auth_successes)
.expect("set AUTH_SUCCESSES once");
let auth_failures = IntCounterVec::new(
Opts::new(
"gitlawb_auth_failures_total",
"Total HTTP signature checks that failed",
),
&["route", "reason"],
)
.expect("gitlawb_auth_failures_total definition");
registry
.register(Box::new(auth_failures.clone()))
.expect("register gitlawb_auth_failures_total");
AUTH_FAILURES
.set(auth_failures)
.expect("set AUTH_FAILURES once");
let sync_processed = IntCounterVec::new(
Opts::new(
"gitlawb_sync_queue_processed_total",
"Total sync_queue items the worker has processed",
),
&["status"],
)
.expect("gitlawb_sync_queue_processed_total definition");
registry
.register(Box::new(sync_processed.clone()))
.expect("register gitlawb_sync_queue_processed_total");
SYNC_PROCESSED
.set(sync_processed)
.expect("set SYNC_PROCESSED once");
let webhook_deliveries = IntCounterVec::new(
Opts::new(
"gitlawb_webhook_deliveries_total",
"Total webhook delivery attempts and their outcomes",
),
&["result"],
)
.expect("gitlawb_webhook_deliveries_total definition");
registry
.register(Box::new(webhook_deliveries.clone()))
.expect("register gitlawb_webhook_deliveries_total");
WEBHOOK_DELIVERIES
.set(webhook_deliveries)
.expect("set WEBHOOK_DELIVERIES once");
let pack_size = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_pack_size_bytes",
"Distribution of git pack body sizes processed (bytes)",
)
.buckets(vec![
1_024.0, // 1 KB
64_000.0, // ~64 KB
1_000_000.0, // 1 MB
16_000_000.0, // 16 MB
64_000_000.0, // 64 MB
256_000_000.0, // 256 MB
1_073_741_824.0, // 1 GiB
2_147_483_648.0, // 2 GiB (current cap)
]),
)
.expect("gitlawb_pack_size_bytes definition");
registry
.register(Box::new(pack_size.clone()))
.expect("register gitlawb_pack_size_bytes");
PACK_SIZE.set(pack_size).expect("set PACK_SIZE once");
let peers_connected = IntGauge::with_opts(Opts::new(
"gitlawb_peers_connected",
"Currently connected libp2p peers",
))
.expect("gitlawb_peers_connected definition");
registry
.register(Box::new(peers_connected.clone()))
.expect("register gitlawb_peers_connected");
PEERS_CONNECTED
.set(peers_connected)
.expect("set PEERS_CONNECTED once");
REGISTRY
.set(registry)
.expect("set REGISTRY once (init must be called exactly once)");
}
/// Record one successful push (receive-pack completion). Labelled with
/// the repo in the form `owner_short/repo_name`. No-op if `init` was
/// never called (e.g. inside a unit test that doesn't go through main).
pub fn record_push(repo: &str) {
if let Some(c) = PUSHES.get() {
c.with_label_values(&[repo]).inc();
}
}
/// Record one successful fetch (upload-pack completion).
pub fn record_fetch(repo: &str) {
if let Some(c) = FETCHES.get() {
c.with_label_values(&[repo]).inc();
}
}
/// Record one HTTP signature check that passed.
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_success(route: &str) {
if let Some(c) = AUTH_SUCCESSES.get() {
c.with_label_values(&[route]).inc();
}
}
/// Record one HTTP signature check that failed. `reason` is a short,
/// machine-friendly token (e.g. `expired`, `bad_signature`, `missing_header`).
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_failure(route: &str, reason: &str) {
if let Some(c) = AUTH_FAILURES.get() {
c.with_label_values(&[route, reason]).inc();
}
}
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped}.
pub fn record_sync_processed(status: &str) {
if let Some(c) = SYNC_PROCESSED.get() {
c.with_label_values(&[status]).inc();
}
}
/// Record one webhook delivery attempt outcome. `result` ∈
/// {ok, http_error, network_error, skipped}.
pub fn record_webhook_delivery(result: &str) {
if let Some(c) = WEBHOOK_DELIVERIES.get() {
c.with_label_values(&[result]).inc();
}
}
/// Record a pack body size observation (bytes).
pub fn observe_pack_size(bytes: f64) {
if let Some(h) = PACK_SIZE.get() {
h.observe(bytes);
}
}
/// Update the currently-connected peer count gauge.
pub fn set_peers_connected(count: i64) {
if let Some(g) = PEERS_CONNECTED.get() {
g.set(count);
}
}
/// Encode the registry as the standard Prometheus text exposition format.
/// Returns an error if `init` was never called.
pub fn encode() -> Result<String, prometheus::Error> {
let registry = REGISTRY
.get()
.ok_or_else(|| prometheus::Error::Msg("metrics::init() was never called".into()))?;
let metric_families = registry.gather();
let mut buf = Vec::new();
let encoder = TextEncoder::new();
encoder.encode(&metric_families, &mut buf)?;
String::from_utf8(buf)
.map_err(|e| prometheus::Error::Msg(format!("non-UTF8 in metric buffer: {e}")))
}
#[cfg(test)]
mod tests {
use super::*;
// Note: these tests are not run in parallel by default (cargo runs
// tests in a binary in parallel via threads but the OnceLock guard
// means only the first init() call succeeds; subsequent ones are
// no-ops). The encode test below is structured so it's safe to run
// alongside other tests in the same binary.
#[test]
fn encode_after_init_returns_prometheus_text() {
// Reset is not supported by OnceLock, so this test relies on
// either being the first to run, or `init` being a no-op on
// second call. Both paths are exercised: if init() was already
// called by another test, the no-op branch still leaves the
// registry usable.
init("0.0.0-test", "did:key:test");
PUSHES
.get()
.expect("PUSHES set after init")
.with_label_values(&["alice/repo"])
.inc();
let body = encode().expect("encode should succeed after init");
assert!(
body.contains("# HELP gitlawb_info"),
"expected gitlawb_info HELP line in: {body}"
);
assert!(
body.contains("# TYPE gitlawb_pushes_total counter"),
"expected gitlawb_pushes_total TYPE line in: {body}"
);
assert!(
body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"),
"expected the incremented counter to be visible in: {body}"
);
}
#[test]
fn record_helpers_are_noops_before_init() {
// These don't panic. They also don't show up in encode() output
// because the registry doesn't exist yet.
record_push("test/repo");
record_fetch("test/repo");
record_auth_success("test/route");
record_auth_failure("test/route", "test_reason");
record_sync_processed("done");
record_webhook_delivery("ok");
observe_pack_size(1024.0);
set_peers_connected(0);
}
#[test]
fn encode_before_init_returns_error() {
// encode() must error rather than panic if init() was never called.
// This test is meaningful only if NO other test in this binary
// ran first. cargo runs tests on multiple threads within a
// single binary, so this is racy — but worst case the assertion
// below is "encode succeeded with a non-empty body", which is
// still a valid state.
let result = encode();
match result {
Err(_) => { /* expected when init wasn't called */ }
Ok(body) => {
// Some other test ran init() first; that's fine, the
// body should still be a valid exposition payload.
assert!(!body.is_empty(), "non-empty exposition body");
}
}
}
}
+19 -6
View File
@@ -145,8 +145,12 @@ impl OperatorClient {
}
/// Spawn a background task that posts a heartbeat on `heartbeat_interval` cadence.
/// Errors are logged and retried on the next tick.
pub fn spawn_heartbeat_loop(self: Arc<Self>) {
/// Errors are logged and retried on the next tick. The task exits cleanly
/// when `shutdown_rx` flips to `true`.
pub fn spawn_heartbeat_loop(
self: Arc<Self>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) {
let interval_secs = self.cfg.heartbeat_interval.as_secs();
info!(
interval_hours = interval_secs / 3600,
@@ -158,11 +162,20 @@ impl OperatorClient {
// First tick fires immediately — use it to send an initial heartbeat
interval.tick().await;
loop {
match self.send_heartbeat().await {
Ok(tx) => info!(tx = %tx, "operator heartbeat sent"),
Err(e) => warn!(err = %e, "operator heartbeat failed — will retry"),
tokio::select! {
_ = interval.tick() => {
match self.send_heartbeat().await {
Ok(tx) => info!(tx = %tx, "operator heartbeat sent"),
Err(e) => warn!(err = %e, "operator heartbeat failed — will retry"),
}
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("operator heartbeat loop: shutdown signal received, exiting");
return;
}
}
}
interval.tick().await;
}
});
}
+74 -70
View File
@@ -3,7 +3,6 @@
//! Provides:
//! - Peer discovery via Kademlia DHT (DID → multiaddr mapping)
//! - Real-time ref-update events via Gossipsub
//! - Local network discovery via mDNS
//!
//! The node's PeerId is derived from its Ed25519 identity keypair,
//! so the gitlawb DID and libp2p PeerId share the same key.
@@ -15,12 +14,13 @@ use std::time::Duration;
use anyhow::Result;
use chrono::Utc;
use libp2p::futures::StreamExt;
use libp2p::{
gossipsub, identify, kad, mdns, noise,
swarm::{NetworkBehaviour, SwarmEvent},
tcp, yamux, Multiaddr, PeerId,
};
use futures::StreamExt;
use libp2p_core::{muxing::StreamMuxerBox, Multiaddr, PeerId, Transport};
use libp2p_gossipsub as gossipsub;
use libp2p_identify as identify;
use libp2p_identity as identity;
use libp2p_kad as kad;
use libp2p_swarm::{NetworkBehaviour, Swarm, SwarmEvent};
use tokio::sync::{mpsc, oneshot};
use tracing::{debug, info, warn};
use uuid::Uuid;
@@ -152,21 +152,23 @@ fn did_to_kad_key(did: &str) -> kad::RecordKey {
/// Combined libp2p behaviour.
#[derive(NetworkBehaviour)]
#[behaviour(prelude = "libp2p_swarm::derive_prelude")]
struct GitlawbBehaviour {
kademlia: kad::Behaviour<kad::store::MemoryStore>,
gossipsub: gossipsub::Behaviour,
identify: identify::Behaviour,
mdns: mdns::tokio::Behaviour,
}
/// Start the libp2p swarm. Returns a handle for sending commands and the
/// listening multiaddrs. Runs the event loop as a background tokio task.
/// listening multiaddrs. Runs the event loop as a background tokio task
/// that exits cleanly when `shutdown_rx` flips to `true`.
pub async fn start(
node_did: &str,
listen_port: u16,
bootstrap_addrs: Vec<Multiaddr>,
db: Arc<Db>,
auto_sync: bool,
shutdown_rx: tokio::sync::watch::Receiver<bool>,
) -> Result<P2pHandle> {
// Derive a stable libp2p Ed25519 key from a seed based on the node DID.
// In production you'd load/persist this key alongside the identity PEM.
@@ -183,7 +185,7 @@ pub async fn start(
seed_bytes[i * 8..(i + 1) * 8].copy_from_slice(&seed.wrapping_add(i as u64).to_le_bytes());
}
let local_key = libp2p::identity::Keypair::ed25519_from_bytes(seed_bytes)
let local_key = identity::Keypair::ed25519_from_bytes(seed_bytes)
.map_err(|e| anyhow::anyhow!("failed to create p2p keypair: {e}"))?;
let local_peer_id = PeerId::from(local_key.public());
@@ -196,65 +198,66 @@ pub async fn start(
local_peer_id,
};
// Build the swarm
let mut swarm = libp2p::SwarmBuilder::with_existing_identity(local_key)
.with_tokio()
.with_tcp(
tcp::Config::default(),
noise::Config::new,
yamux::Config::default,
)?
.with_behaviour(|key| {
let peer_id = PeerId::from(key.public());
let kad_store = kad::store::MemoryStore::new(local_peer_id);
let mut kademlia = kad::Behaviour::new(local_peer_id, kad_store);
kademlia.set_mode(Some(kad::Mode::Server));
// Kademlia
let kad_store = kad::store::MemoryStore::new(peer_id);
let mut kademlia = kad::Behaviour::new(peer_id, kad_store);
kademlia.set_mode(Some(kad::Mode::Server));
let gossipsub_config = gossipsub::ConfigBuilder::default()
.heartbeat_interval(Duration::from_secs(10))
.validation_mode(gossipsub::ValidationMode::Permissive)
.message_id_fn(|msg: &gossipsub::Message| {
let mut h = DefaultHasher::new();
msg.data.hash(&mut h);
gossipsub::MessageId::from(h.finish().to_string())
})
.build()
.expect("gossipsub config");
let gossipsub = gossipsub::Behaviour::new(
gossipsub::MessageAuthenticity::Signed(local_key.clone()),
gossipsub_config,
)
.expect("gossipsub behaviour");
// Gossipsub — message_id based on content hash to deduplicate
let gossipsub_config = gossipsub::ConfigBuilder::default()
.heartbeat_interval(Duration::from_secs(10))
.validation_mode(gossipsub::ValidationMode::Permissive)
.message_id_fn(|msg: &gossipsub::Message| {
let mut h = DefaultHasher::new();
msg.data.hash(&mut h);
gossipsub::MessageId::from(h.finish().to_string())
})
.build()
.expect("gossipsub config");
let gossipsub = gossipsub::Behaviour::new(
gossipsub::MessageAuthenticity::Signed(key.clone()),
gossipsub_config,
)
.expect("gossipsub behaviour");
let identify = identify::Behaviour::new(identify::Config::new(
"/gitlawb/1.0.0".to_string(),
local_key.public(),
));
// Identify — so peers can exchange their observed addresses
let identify = identify::Behaviour::new(identify::Config::new(
"/gitlawb/1.0.0".to_string(),
key.public(),
));
// mDNS — local network peer discovery
let mdns = mdns::tokio::Behaviour::new(mdns::Config::default(), peer_id)
.expect("mdns behaviour");
GitlawbBehaviour {
kademlia,
gossipsub,
identify,
mdns,
}
})?
.build();
let behaviour = GitlawbBehaviour {
kademlia,
gossipsub,
identify,
};
// DNS wraps QUIC so multiaddrs like /dns6/<app>.internal/udp/…/quic-v1
// resolve at dial time. On Fly, peer nodes must dial each other over the
// private 6PN network via <app>.internal hostnames — dialing through the
// public anycast edge breaks the handshake (the proxy closes the
// connection mid-stream).
let quic = libp2p_quic::tokio::Transport::new(libp2p_quic::Config::new(&local_key))
.map(|(peer_id, muxer), _| (peer_id, StreamMuxerBox::new(muxer)));
let transport = libp2p_dns::tokio::Transport::system(quic)?.boxed();
let mut swarm = Swarm::new(
transport,
behaviour,
local_peer_id,
libp2p_swarm::Config::with_tokio_executor(),
);
// Subscribe to the ref-updates topic
let topic = gossipsub::IdentTopic::new(REF_UPDATES_TOPIC);
swarm.behaviour_mut().gossipsub.subscribe(&topic)?;
// Listen
let listen_addr: Multiaddr = format!("/ip4/0.0.0.0/tcp/{listen_port}").parse()?;
swarm.listen_on(listen_addr)?;
// Listen on both IPv4 (local/mDNS + any IPv4 dials) and IPv6 (required
// for Fly's 6PN inter-app network — <app>.internal DNS only returns AAAA
// records, so peers dial us via IPv6 and need a matching IPv6 socket).
let v4: Multiaddr = format!("/ip4/0.0.0.0/udp/{listen_port}/quic-v1").parse()?;
if let Err(e) = swarm.listen_on(v4) {
warn!(err = %e, "failed to listen on IPv4");
}
let v6: Multiaddr = format!("/ip6/::/udp/{listen_port}/quic-v1").parse()?;
if let Err(e) = swarm.listen_on(v6) {
warn!(err = %e, "failed to listen on IPv6");
}
// Bootstrap Kademlia with known peers
for addr in bootstrap_addrs {
@@ -270,8 +273,18 @@ pub async fn start(
// Start the event loop as a background task
tokio::spawn(async move {
let mut shutdown_rx = shutdown_rx;
loop {
tokio::select! {
// Graceful shutdown: exit the swarm loop when the
// process-wide signal flips. This drops the Swarm
// which closes all libp2p connections cleanly.
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("p2p swarm: shutdown signal received, exiting event loop");
break;
}
}
// Handle swarm events
event = swarm.select_next_some() => {
match event {
@@ -343,15 +356,6 @@ pub async fn start(
}
}
SwarmEvent::Behaviour(GitlawbBehaviourEvent::Mdns(
mdns::Event::Discovered(peers)
)) => {
for (peer_id, addr) in peers {
debug!(peer = %peer_id, addr = %addr, "mDNS discovered peer");
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr.clone());
let _ = swarm.dial(addr);
}
}
SwarmEvent::Behaviour(GitlawbBehaviourEvent::Identify(
identify::Event::Received { peer_id, info, .. }
)) => {
+3
View File
@@ -7,6 +7,7 @@
//! no-op, so nodes without Pinata backing work fine.
use anyhow::Result;
use std::collections::HashSet;
/// Pin a single git object's raw bytes on Pinata (v3 API).
///
@@ -76,6 +77,7 @@ pub async fn pin_new_objects(
jwt: &str,
repo_path: &std::path::Path,
db: &crate::db::Db,
withheld: &HashSet<String>,
) -> Vec<(String, String)> {
if jwt.is_empty() {
return vec![];
@@ -92,6 +94,7 @@ pub async fn pin_new_objects(
return vec![];
}
};
let object_list = crate::git::visibility_pack::replicable_objects(object_list, withheld);
let mut pinned = Vec::new();
+129
View File
@@ -0,0 +1,129 @@
use std::collections::HashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};
use axum::extract::Request;
use axum::http::StatusCode;
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
use tokio::sync::Mutex;
use crate::auth::AuthenticatedDid;
#[derive(Clone)]
struct Window {
timestamps: Vec<Instant>,
}
#[derive(Clone)]
pub struct RateLimiter {
state: Arc<Mutex<HashMap<String, Window>>>,
max_requests: usize,
window: Duration,
}
impl RateLimiter {
pub fn new(max_requests: usize, window: Duration) -> Self {
Self {
state: Arc::new(Mutex::new(HashMap::new())),
max_requests,
window,
}
}
async fn check(&self, key: &str) -> bool {
let now = Instant::now();
let mut state = self.state.lock().await;
let window = state.entry(key.to_string()).or_insert_with(|| Window {
timestamps: Vec::new(),
});
window
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
if window.timestamps.len() >= self.max_requests {
return false;
}
window.timestamps.push(now);
true
}
pub async fn cleanup(&self) {
let now = Instant::now();
let mut state = self.state.lock().await;
state.retain(|_, w| {
w.timestamps
.retain(|t| now.duration_since(*t) < self.window);
!w.timestamps.is_empty()
});
}
}
pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
let limiter = request.extensions().get::<RateLimiter>().cloned();
let did = request
.extensions()
.get::<AuthenticatedDid>()
.map(|a| a.0.clone());
if let (Some(limiter), Some(did)) = (limiter, did) {
if !limiter.check(&did).await {
return (
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response();
}
}
next.run(request).await
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn allows_within_limit() {
let limiter = RateLimiter::new(3, Duration::from_secs(60));
assert!(limiter.check("did:key:test1").await);
assert!(limiter.check("did:key:test1").await);
assert!(limiter.check("did:key:test1").await);
}
#[tokio::test]
async fn blocks_over_limit() {
let limiter = RateLimiter::new(2, Duration::from_secs(60));
assert!(limiter.check("did:key:test2").await);
assert!(limiter.check("did:key:test2").await);
assert!(!limiter.check("did:key:test2").await);
}
#[tokio::test]
async fn separate_keys_independent() {
let limiter = RateLimiter::new(1, Duration::from_secs(60));
assert!(limiter.check("did:key:alice").await);
assert!(limiter.check("did:key:bob").await);
assert!(!limiter.check("did:key:alice").await);
}
#[tokio::test]
async fn window_expires() {
let limiter = RateLimiter::new(1, Duration::from_millis(50));
assert!(limiter.check("did:key:test3").await);
assert!(!limiter.check("did:key:test3").await);
tokio::time::sleep(Duration::from_millis(60)).await;
assert!(limiter.check("did:key:test3").await);
}
#[tokio::test]
async fn cleanup_removes_expired() {
let limiter = RateLimiter::new(1, Duration::from_millis(50));
limiter.check("did:key:stale").await;
tokio::time::sleep(Duration::from_millis(60)).await;
limiter.cleanup().await;
let state = limiter.state.lock().await;
assert!(state.is_empty());
}
}
+196 -122
View File
@@ -13,10 +13,11 @@ use tower_http::trace::{DefaultOnFailure, DefaultOnResponse, TraceLayer};
use tracing::Level;
use crate::api::{
agents, arweave, bounties, certs, changelog, events, ipfs, issues, labels, peers, protect,
pulls, register, replicas, repos, resolve, stars, tasks, webhooks,
agents, arweave, bounties, certs, changelog, events, ipfs, issues, labels, peers, profiles,
protect, pulls, register, replicas, repos, resolve, stars, tasks, visibility, webhooks,
};
use crate::auth;
use crate::rate_limit;
use crate::state::AppState;
async fn graphql_handler(State(state): State<AppState>, req: GraphQLRequest) -> GraphQLResponse {
@@ -30,6 +31,19 @@ async fn graphql_playground() -> impl IntoResponse {
))
}
/// Applies the standard auth middleware pair to a router: HTTP Signature verification
/// followed by UCAN chain validation. The two layers run in this order for every
/// matched request: `require_signature` first (sets `AuthenticatedDid`), then
/// `require_ucan_chain` (reads it).
fn add_auth_layers(router: Router<AppState>, state: AppState) -> Router<AppState> {
router
.layer(middleware::from_fn_with_state(
state,
auth::require_ucan_chain,
))
.layer(middleware::from_fn(auth::require_signature))
}
pub fn build_router(state: AppState) -> Router {
// ── GraphQL routes ─────────────────────────────────────────────────────
let schema = state.graphql_schema.as_ref().clone();
@@ -38,95 +52,120 @@ pub fn build_router(state: AppState) -> Router {
.route_service("/graphql/ws", GraphQLSubscription::new(schema));
// ── Task routes (write — require HTTP Signature) ───────────────────────
let task_write_routes = Router::new()
.route("/api/v1/tasks", post(tasks::create_task))
.route("/api/v1/tasks/{id}/claim", post(tasks::claim_task))
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task))
.layer(middleware::from_fn(auth::require_signature));
let task_write_routes = add_auth_layers(
Router::new()
.route("/api/v1/tasks", post(tasks::create_task))
.route("/api/v1/tasks/{id}/claim", post(tasks::claim_task))
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task)),
state.clone(),
);
// ── Task routes (read — open) ──────────────────────────────────────────
let task_read_routes = Router::new()
.route("/api/v1/tasks", get(tasks::list_tasks))
.route("/api/v1/tasks/{id}", get(tasks::get_task));
// ── Write routes — require HTTP Signature ──────────────────────────────
let write_routes = Router::new()
.route("/api/v1/repos", post(repos::create_repo))
.route("/api/register", post(register::register))
.route("/api/v1/repos/{owner}/{repo}/pulls", post(pulls::create_pr))
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/merge",
post(pulls::merge_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/close",
post(pulls::close_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/reviews",
post(pulls::create_review),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/comments",
post(pulls::create_comment),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks",
post(webhooks::create_webhook),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks/{id}",
axum::routing::delete(webhooks::delete_webhook),
)
.route(
"/api/v1/repos/{owner}/{repo}/branches/{branch}/protect",
post(protect::protect_branch),
)
.route(
"/api/v1/repos/{owner}/{repo}/branches/{branch}/protect",
axum::routing::delete(protect::unprotect_branch),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
axum::routing::put(stars::star_repo),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
axum::routing::delete(stars::unstar_repo),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
axum::routing::put(replicas::register_replica),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
axum::routing::delete(replicas::unregister_replica),
)
.route("/api/v1/repos/{owner}/{repo}/fork", post(repos::fork_repo))
.route(
"/api/v1/repos/{owner}/{repo}/labels",
post(labels::add_label),
)
.route(
"/api/v1/repos/{owner}/{repo}/labels/{label}",
axum::routing::delete(labels::remove_label),
)
.layer(middleware::from_fn(auth::require_signature));
// ── Rate-limited creation routes — require HTTP Signature + per-DID throttle
let limiter = state.rate_limiter.clone();
let creation_routes = add_auth_layers(
Router::new()
.route("/api/v1/repos", post(repos::create_repo))
.route("/api/register", post(register::register))
.route("/api/v1/repos/{owner}/{repo}/fork", post(repos::fork_repo))
.route(
"/api/v1/repos/{owner}/{repo}/issues",
post(issues::create_issue),
)
.route("/api/v1/repos/{owner}/{repo}/pulls", post(pulls::create_pr))
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(limiter)),
state.clone(),
);
// ── Write routes — require HTTP Signature (no rate limit) ─────────────
let write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/merge",
post(pulls::merge_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/close",
post(pulls::close_pr),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/reviews",
post(pulls::create_review),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/comments",
post(pulls::create_comment),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks",
post(webhooks::create_webhook),
)
.route(
"/api/v1/repos/{owner}/{repo}/hooks/{id}",
axum::routing::delete(webhooks::delete_webhook),
)
.route(
"/api/v1/repos/{owner}/{repo}/branches/{branch}/protect",
post(protect::protect_branch),
)
.route(
"/api/v1/repos/{owner}/{repo}/branches/{branch}/protect",
axum::routing::delete(protect::unprotect_branch),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
axum::routing::put(stars::star_repo),
)
.route(
"/api/v1/repos/{owner}/{repo}/star",
axum::routing::delete(stars::unstar_repo),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
axum::routing::put(replicas::register_replica),
)
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
axum::routing::delete(replicas::unregister_replica),
)
.route(
"/api/v1/repos/{owner}/{repo}/labels",
post(labels::add_label),
)
.route(
"/api/v1/repos/{owner}/{repo}/labels/{label}",
axum::routing::delete(labels::remove_label),
)
.route(
"/api/v1/repos/{owner}/{repo}/visibility",
axum::routing::put(visibility::set_visibility)
.delete(visibility::remove_visibility)
.get(visibility::list_visibility),
),
state.clone(),
);
// Body limit is raised to GITLAWB_MAX_PACK_BYTES (default 2 GB) for git
// routes only — all other API routes keep axum's default 2 MB cap.
// HTTP Signature is enforced on receive-pack (push) — the git-remote-gitlawb
// helper signs requests with RFC 9421 signatures using the agent's keypair.
let pack_limit = state.config.max_pack_bytes;
let git_write_routes = Router::new()
.route(
"/{owner}/{repo}/git-receive-pack",
post(repos::git_receive_pack),
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit))
.layer(middleware::from_fn(auth::require_signature));
let git_write_routes = add_auth_layers(
Router::new()
.route(
"/{owner}/{repo}/git-receive-pack",
post(repos::git_receive_pack),
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit)),
state.clone(),
);
// ── IPFS content-addressed retrieval and pin listing ──────────────────
let ipfs_routes = Router::new()
@@ -137,25 +176,31 @@ pub fn build_router(state: AppState) -> Router {
let arweave_routes = Router::new().route("/api/v1/arweave/anchors", get(arweave::list_anchors));
// ── Bounty routes (write — require HTTP Signature) ─────────────────
let bounty_write_routes = Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/bounties",
post(bounties::create_bounty),
)
.route("/api/v1/bounties/{id}/claim", post(bounties::claim_bounty))
.route(
"/api/v1/bounties/{id}/submit",
post(bounties::submit_bounty),
)
.route(
"/api/v1/bounties/{id}/approve",
post(bounties::approve_bounty),
)
.route(
"/api/v1/bounties/{id}/cancel",
post(bounties::cancel_bounty),
)
.layer(middleware::from_fn(auth::require_signature));
let bounty_write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/bounties",
post(bounties::create_bounty),
)
.route("/api/v1/bounties/{id}/claim", post(bounties::claim_bounty))
.route(
"/api/v1/bounties/{id}/submit",
post(bounties::submit_bounty),
)
.route(
"/api/v1/bounties/{id}/approve",
post(bounties::approve_bounty),
)
.route(
"/api/v1/bounties/{id}/cancel",
post(bounties::cancel_bounty),
)
.route(
"/api/v1/bounties/{id}/dispute",
post(bounties::dispute_bounty),
),
state.clone(),
);
// ── Bounty routes (read — open) ──────────────────────────────────────
let bounty_read_routes = Router::new()
@@ -165,41 +210,48 @@ pub fn build_router(state: AppState) -> Router {
)
.route("/api/v1/bounties", get(bounties::list_all_bounties))
.route("/api/v1/bounties/{id}", get(bounties::get_bounty))
.route(
"/api/v1/bounties/{id}/dispute",
post(bounties::dispute_bounty),
)
.route("/api/v1/bounties/stats", get(bounties::bounty_stats))
.route(
"/api/v1/agents/{did}/bounties",
get(bounties::agent_bounty_stats),
);
// ── Issue routes ──────────────────────────────────────────────────────
let issue_write_routes = Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/issues",
post(issues::create_issue),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/close",
post(issues::close_issue),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/comments",
post(issues::create_issue_comment),
)
.layer(middleware::from_fn(auth::require_signature));
// ── Profile routes (write — require HTTP Signature) ─────────────────
let profile_write_routes = add_auth_layers(
Router::new().route("/api/v1/profile", axum::routing::put(profiles::set_profile)),
state.clone(),
);
// ── Issue routes (write — require HTTP Signature, no rate limit) ─────
let issue_write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/close",
post(issues::close_issue),
)
.route(
"/api/v1/repos/{owner}/{repo}/issues/{id}/comments",
post(issues::create_issue_comment),
),
state.clone(),
);
// ── Peer discovery routes ─────────────────────────────────────────────
// announce is intentionally unauthenticated — nodes announce by DID which
// is self-proving; rate-limiting and blocklisting are handled elsewhere.
// Peer writes accept signatures when present and can require them after a
// coordinated live-network upgrade.
let peer_read_routes = Router::new()
.route("/api/v1/peers", get(peers::list_peers))
.route("/api/v1/peers/{did}/ping", get(peers::ping_peer));
let mut peer_write_routes = Router::new()
.route("/api/v1/peers/announce", post(peers::announce))
.route("/api/v1/peers/{did}/ping", get(peers::ping_peer))
.route("/api/v1/sync/trigger", post(peers::trigger_sync))
.route("/api/v1/sync/notify", post(peers::notify_sync));
peer_write_routes = if state.config.require_signed_peer_writes {
add_auth_layers(peer_write_routes, state.clone())
} else {
peer_write_routes.layer(middleware::from_fn(auth::optional_signature))
};
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
@@ -250,6 +302,7 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/agents", get(agents::list_agents))
.route("/api/v1/agents/{did}", get(agents::show_agent))
.route("/api/v1/agents/{did}/trust", get(agents::get_trust))
.route("/api/v1/agents/{did}/profile", get(profiles::get_profile))
.route("/api/v1/events/ref-updates", get(events::list_ref_updates))
.route("/api/v1/resolve/{did}", get(resolve::resolve_did))
.route("/api/v1/repos/{owner}/{repo}/pulls", get(pulls::list_prs))
@@ -290,17 +343,35 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/repos/{owner}/{repo}/replicas",
get(replicas::list_replicas),
)
.route("/{owner}/{repo}/info/refs", get(repos::git_info_refs));
.layer(middleware::from_fn(auth::optional_signature));
// git-upload-pack (clone/fetch) — same raised body limit as receive-pack so
// large pack responses from the server don't get truncated on the client side.
let git_read_routes = Router::new()
.route("/{owner}/{repo}/info/refs", get(repos::git_info_refs))
.route(
"/{owner}/{repo}/git-upload-pack",
post(repos::git_upload_pack),
)
.route(
"/api/v1/repos/{owner}/{repo}/withheld-paths",
axum::routing::get(visibility::withheld_paths),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blobs",
axum::routing::get(crate::api::encrypted::list_encrypted_blobs),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blob/{oid}",
axum::routing::get(crate::api::encrypted::get_encrypted_blob),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate",
axum::routing::get(crate::api::encrypted::replicate_encrypted_blobs),
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit));
.layer(RequestBodyLimitLayer::new(pack_limit))
.layer(middleware::from_fn(auth::optional_signature));
// ── Meta ──────────────────────────────────────────────────────────────
let meta_routes = Router::new()
@@ -316,12 +387,15 @@ pub fn build_router(state: AppState) -> Router {
.merge(task_read_routes)
.merge(bounty_write_routes)
.merge(bounty_read_routes)
.merge(profile_write_routes)
.merge(creation_routes)
.merge(write_routes)
.merge(git_write_routes)
.merge(git_read_routes)
.merge(issue_write_routes)
.merge(read_routes)
.merge(peer_read_routes)
.merge(peer_write_routes)
.merge(ipfs_routes)
.merge(arweave_routes)
.merge(meta_routes)
+40
View File
@@ -6,6 +6,7 @@ use crate::config::Config;
use crate::db::Db;
use crate::git::repo_store::RepoStore;
use crate::p2p::P2pHandle;
use crate::rate_limit::RateLimiter;
#[derive(Clone, Debug)]
pub struct RefUpdateBroadcast {
@@ -48,4 +49,43 @@ pub struct AppState {
pub machine_id: Option<String>,
/// Centralized repo storage: local disk cache + optional Tigris backend
pub repo_store: RepoStore,
/// Per-DID rate limiter for creation endpoints (repos, issues, PRs)
pub rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
/// * the SIGINT/SIGTERM handler in `main()`
/// * axum's `with_graceful_shutdown` to drain in-flight HTTP requests
/// * the libp2p swarm task
/// * the gossip, sync, operator heartbeat, and rate-limit cleanup loops
pub shutdown_tx: tokio::sync::watch::Sender<bool>,
}
impl AppState {
/// Subscribe to the shutdown signal. Returns a fresh receiver whose
/// initial value matches the current state.
pub fn subscribe_shutdown(&self) -> tokio::sync::watch::Receiver<bool> {
self.shutdown_tx.subscribe()
}
/// Trigger graceful shutdown. Idempotent — calling more than once
/// has no effect. Returns `true` if this call was the one that
/// flipped the signal.
#[allow(dead_code)] // used by tests; main() drives the signal directly
pub fn shutdown(&self) -> bool {
self.shutdown_tx.send_if_modified(|v| {
if *v {
false
} else {
*v = true;
true
}
})
}
/// `true` once shutdown has been signalled.
#[allow(dead_code)] // used by tests and any future handler that wants to short-circuit
pub fn is_shutting_down(&self) -> bool {
*self.shutdown_tx.borrow()
}
}
+644 -42
View File
@@ -9,34 +9,134 @@
//! 2. If the repo doesn't exist locally → `git clone --mirror`.
//! 3. If it exists → `git fetch --prune` from the origin.
//! 4. Mark done or failed.
//! 5. On success, register ourselves as a replica with the origin node so
//! its `replica_count` reflects reality (best-effort, idempotent).
use std::collections::HashMap;
use std::path::Path;
use std::sync::Arc;
use gitlawb_core::identity::Keypair;
use tracing::{info, warn};
use crate::config::Config;
use crate::db::Db;
/// Start the background sync worker. Returns immediately; the worker runs
/// as a detached tokio task.
pub fn start(db: Arc<Db>, config: Arc<Config>) {
tokio::spawn(async move {
run(db, config).await;
});
/// How to mirror a repo, decided from the origin's `withheld-paths` answer.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum MirrorMode {
/// No withheld content: a normal full mirror.
Plain,
/// Withheld content present: a promisor mirror that tolerates the blobs the
/// origin omits for an anonymous caller.
Promisor,
}
async fn run(db: Arc<Db>, config: Arc<Config>) {
let machine_id = std::env::var("FLY_MACHINE_ID").ok();
info!("sync worker started (auto_sync=true)");
let mut interval = tokio::time::interval(std::time::Duration::from_secs(30));
loop {
interval.tick().await;
process_batch(&db, &config, machine_id.as_deref()).await;
/// Decide the mirror mode from the origin's `withheld-paths` response.
///
/// `Some(non-empty)` → the repo has a private subtree → `Promisor`.
/// `Some(empty)` → fully public → `Plain`.
/// `None` → the lookup 404'd or failed. Attempt a `Plain` mirror; a
/// mode-A repo also 404s the git read endpoint, so the clone
/// fails and nothing is mirrored (fail-closed at the git
/// layer), while a public repo on a peer that predates the
/// `withheld-paths` route still gets mirrored.
fn classify_mirror(withheld: Option<Vec<String>>) -> MirrorMode {
match withheld {
Some(globs) if !globs.is_empty() => MirrorMode::Promisor,
_ => MirrorMode::Plain,
}
}
async fn process_batch(db: &Db, config: &Config, machine_id: Option<&str>) {
/// One encrypted blob as advertised by an origin's `encrypted-blobs/replicate`
/// endpoint (Option B2). Ciphertext metadata only; recipient identities are
/// withheld from peers, so a re-seal is detected by the CID changing.
#[derive(Debug, Clone, PartialEq, Eq, serde::Deserialize)]
struct ReplicaBlob {
oid: String,
cid: String,
}
/// The shape of the `encrypted-blobs/replicate` JSON response.
#[derive(Debug, serde::Deserialize)]
struct ReplicateResponse {
#[serde(default)]
blobs: Vec<ReplicaBlob>,
}
/// Decide which of the origin's encrypted blobs this mirror must (re)replicate.
///
/// `have` maps each already-stored blob's oid to the CID the mirror pinned. A
/// remote blob is returned when the mirror has no row for that oid, or when the
/// stored CID differs from the advertised one. A re-seal regenerates the
/// envelope (new content key, nonce, and per-recipient wraps), so the CID
/// changes while the OID stays stable; comparing CIDs detects a re-seal without
/// the mirror ever holding recipient identities.
fn blobs_needing_replication(
remote: &[ReplicaBlob],
have: &HashMap<String, String>,
) -> Vec<ReplicaBlob> {
remote
.iter()
.filter(|b| match have.get(&b.oid) {
None => true,
Some(stored_cid) => stored_cid != &b.cid,
})
.cloned()
.collect()
}
/// Start the background sync worker. Returns immediately; the worker runs
/// as a detached tokio task that exits cleanly when `shutdown_rx` flips
/// to `true`.
pub fn start(
db: Arc<Db>,
config: Arc<Config>,
keypair: Arc<Keypair>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) {
tokio::spawn(async move {
run(db, config, keypair, &mut shutdown_rx).await;
});
}
async fn run(
db: Arc<Db>,
config: Arc<Config>,
keypair: Arc<Keypair>,
shutdown_rx: &mut tokio::sync::watch::Receiver<bool>,
) {
let machine_id = std::env::var("FLY_MACHINE_ID").ok();
// Bound each peer HTTP call (withheld-paths lookup + replica registration)
// so a stalled peer cannot hang the worker.
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.unwrap_or_else(|_| reqwest::Client::new());
info!("sync worker started (auto_sync=true)");
let mut interval = tokio::time::interval(std::time::Duration::from_secs(30));
loop {
tokio::select! {
_ = interval.tick() => {
process_batch(&db, &config, &keypair, machine_id.as_deref(), &client).await;
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
info!("sync worker: shutdown signal received, exiting");
return;
}
}
}
}
}
async fn process_batch(
db: &Db,
config: &Config,
keypair: &Keypair,
machine_id: Option<&str>,
client: &reqwest::Client,
) {
let items = match db.dequeue_pending_syncs(10).await {
Ok(v) => v,
Err(e) => {
@@ -85,10 +185,13 @@ async fn process_batch(db: &Db, config: &Config, machine_id: Option<&str>) {
// (no .git suffix — the server routes don't include it)
let remote_url = format!("{}/{}", origin_url, item.repo);
let withheld = fetch_withheld(client, &origin_url, owner_short, repo_name).await;
let mode = classify_mirror(withheld);
let result = if local_path.exists() {
fetch_repo(&local_path, &remote_url).await
fetch_repo(&local_path, &remote_url, mode).await
} else {
clone_repo(&remote_url, &local_path).await
clone_repo(&remote_url, &local_path, mode).await
};
match result {
@@ -103,25 +206,257 @@ async fn process_batch(db: &Db, config: &Config, machine_id: Option<&str>) {
machine_id,
)
.await;
// Option B2: carry the encrypted withheld-blob envelopes too, so an
// authorized reader can recover private content from this mirror if
// the origin dies. `item.repo` is the slug "{owner_short}/{name}",
// which is the id upsert_mirror_repo wrote (the local repo_id).
replicate_encrypted_blobs(
client,
&origin_url,
owner_short,
repo_name,
db,
&item.repo,
&config.ipfs_api,
)
.await;
let _ = db.mark_sync_done(&item.id).await;
crate::metrics::record_sync_processed("done");
// Tell the origin we now host a replica so its replica_count
// reflects reality. Best-effort: idempotent on the origin and
// never fails the sync.
register_replica_with_origin(
client,
keypair,
config.public_url.as_deref(),
&origin_url,
owner_short,
repo_name,
)
.await;
}
Err(e) => {
warn!(repo = %item.repo, origin = %origin_url, err = %e, "repo sync failed");
let _ = db.mark_sync_failed(&item.id).await;
crate::metrics::record_sync_processed("failed");
}
}
}
}
/// Mirror-clone a repo from a remote URL into a local bare repo.
async fn clone_repo(remote_url: &str, local_path: &Path) -> anyhow::Result<()> {
/// Query the origin's anonymous `withheld-paths` endpoint. Returns the withheld
/// glob list on a 2xx, or `None` on any non-success / network / parse error
/// (treated as "unknown" by `classify_mirror`).
async fn fetch_withheld(
client: &reqwest::Client,
origin_url: &str,
owner: &str,
repo: &str,
) -> Option<Vec<String>> {
let url = format!("{origin_url}/api/v1/repos/{owner}/{repo}/withheld-paths");
let resp = client.get(&url).send().await.ok()?;
if !resp.status().is_success() {
return None;
}
let body: serde_json::Value = resp.json().await.ok()?;
let globs = body
.get("withheld")?
.as_array()?
.iter()
.filter_map(|v| v.as_str().map(str::to_string))
.collect();
Some(globs)
}
/// Signed request path for replica registration on the origin node.
fn replica_registration_path(owner: &str, repo: &str) -> String {
format!("/api/v1/repos/{owner}/{repo}/replicas")
}
/// Best-effort `PUT /api/v1/repos/{owner}/{repo}/replicas` against the origin
/// node after a successful mirror, signed with our node keypair. The origin
/// records (our DID, our public URL) and exposes it via its public replica
/// list. PUT is idempotent there (201 on first registration, 200 after), so
/// re-registering on every successful sync is safe and self-healing.
///
/// Skipped when we have no public URL to advertise. Failures are logged and
/// never affect the sync result. Reuses the worker's shared `client` (30s
/// timeout) with a tighter per-request timeout.
async fn register_replica_with_origin(
client: &reqwest::Client,
keypair: &Keypair,
public_url: Option<&str>,
origin_url: &str,
owner: &str,
repo: &str,
) {
let self_url = match public_url {
Some(u) if !u.is_empty() => u,
_ => return,
};
let path = replica_registration_path(owner, repo);
let body = serde_json::json!({ "url": self_url });
let body_bytes = match serde_json::to_vec(&body) {
Ok(b) => b,
Err(e) => {
warn!(owner, repo, err = %e, "failed to serialize replica registration");
return;
}
};
let signed = gitlawb_core::http_sig::sign_request(keypair, "PUT", &path, &body_bytes);
match client
.put(format!("{origin_url}{path}"))
.header("Content-Type", "application/json")
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature)
.body(body_bytes)
.timeout(std::time::Duration::from_secs(10))
.send()
.await
{
Ok(r) if r.status().is_success() => {
info!(owner, repo, origin = %origin_url, "registered as replica with origin");
}
Ok(r) => {
warn!(owner, repo, origin = %origin_url, status = %r.status(), "replica registration rejected by origin");
}
Err(e) => {
warn!(owner, repo, origin = %origin_url, err = %e, "replica registration request failed");
}
}
}
/// Replicate the origin's encrypted withheld blobs onto this mirror (Option B2).
///
/// After the git objects are mirrored, fetch the origin's replication listing,
/// then for each blob the mirror does not already hold (or whose CID changed,
/// i.e. the origin re-sealed) pull the ciphertext envelope over IPFS, pin it
/// locally, and record the `encrypted_blobs` row keyed by this mirror's local
/// `repo_id`. The mirror stores no recipient identities.
///
/// Best-effort and idempotent: any per-blob failure is logged and skipped, to be
/// retried on the next sync. Confidentiality is never at risk; the mirror only
/// ever handles ciphertext and never decrypts. Cleanly a no-op when IPFS is
/// unconfigured, the origin reports no encrypted blobs, or the replicate endpoint
/// is absent (older peer) or unreachable.
async fn replicate_encrypted_blobs(
client: &reqwest::Client,
origin_url: &str,
owner: &str,
repo: &str,
db: &Db,
repo_id: &str,
ipfs_api: &str,
) {
if ipfs_api.is_empty() {
return;
}
let url = format!("{origin_url}/api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate");
let resp = match client.get(&url).send().await {
Ok(r) if r.status().is_success() => r,
_ => return,
};
let parsed: ReplicateResponse = match resp.json().await {
Ok(p) => p,
Err(e) => {
warn!(repo = %repo, err = %e, "failed to parse encrypted-blobs/replicate response");
return;
}
};
if parsed.blobs.is_empty() {
return;
}
let have: HashMap<String, String> = match db.list_all_encrypted_blobs(repo_id).await {
Ok(rows) => rows.into_iter().collect(),
Err(e) => {
warn!(repo = %repo, err = %e, "failed to list local encrypted blobs for replication");
return;
}
};
for blob in blobs_needing_replication(&parsed.blobs, &have) {
let envelope = match crate::ipfs_pin::cat(ipfs_api, &blob.cid).await {
Ok(bytes) => bytes,
Err(e) => {
warn!(oid = %blob.oid, cid = %blob.cid, err = %e, "failed to fetch encrypted envelope over IPFS; will retry next sync");
continue;
}
};
match crate::ipfs_pin::pin_git_object(ipfs_api, &blob.oid, &envelope).await {
Ok(cid) if !cid.is_empty() => {
if cid != blob.cid {
warn!(oid = %blob.oid, expected = %blob.cid, got = %cid, "replicated envelope CID mismatch; skipping record");
continue;
}
if let Err(e) = db.record_encrypted_blob(repo_id, &blob.oid, &cid, "").await {
warn!(oid = %blob.oid, err = %e, "failed to record replicated encrypted blob");
}
}
_ => {
warn!(oid = %blob.oid, "failed to pin replicated encrypted envelope; will retry next sync");
}
}
}
}
/// Run a git subprocess, returning an error with stderr on non-zero exit.
async fn git_run(args: &[&str]) -> anyhow::Result<()> {
let out = tokio::process::Command::new("git")
.args([
"clone",
"--mirror",
remote_url,
local_path.to_str().unwrap_or("."),
])
.args(args)
.output()
.await
.map_err(|e| anyhow::anyhow!("git failed to spawn: {e}"))?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!("git {args:?} failed: {stderr}"));
}
Ok(())
}
/// Run a git subprocess, ignoring a non-zero exit. Used for idempotent
/// `config --unset`, which exits non-zero when the key is already absent.
async fn git_run_lenient(args: &[&str]) {
let _ = tokio::process::Command::new("git")
.args(args)
.output()
.await;
}
/// Read a single git config value; `None` if unset or on error.
async fn git_config_get(repo: &str, key: &str) -> Option<String> {
let out = tokio::process::Command::new("git")
.args(["-C", repo, "config", "--get", key])
.output()
.await
.ok()?;
if !out.status.success() {
return None;
}
let value = String::from_utf8_lossy(&out.stdout).trim().to_string();
(!value.is_empty()).then_some(value)
}
/// Mirror-clone a repo from a remote URL into a local bare repo.
/// `Promisor` mode adds `--filter=blob:limit=10g`, which marks the repo a git
/// promisor (so a pack with origin-omitted withheld blobs is accepted) while
/// the huge size limit means every blob the origin *does* send is kept.
async fn clone_repo(remote_url: &str, local_path: &Path, mode: MirrorMode) -> anyhow::Result<()> {
let local_str = local_path.to_str().unwrap_or(".");
let mut args = vec!["clone", "--mirror"];
if mode == MirrorMode::Promisor {
args.push("--filter=blob:limit=10g");
}
args.push(remote_url);
args.push(local_str);
let out = tokio::process::Command::new("git")
.args(&args)
.output()
.await
.map_err(|e| anyhow::anyhow!("git clone failed to spawn: {e}"))?;
@@ -133,24 +468,291 @@ async fn clone_repo(remote_url: &str, local_path: &Path) -> anyhow::Result<()> {
Ok(())
}
/// Fetch all refs from the remote into an existing mirror repo.
async fn fetch_repo(local_path: &Path, remote_url: &str) -> anyhow::Result<()> {
let out = tokio::process::Command::new("git")
.args([
"-C",
local_path.to_str().unwrap_or("."),
"fetch",
"--prune",
remote_url,
"+refs/*:refs/*",
])
.output()
.await
.map_err(|e| anyhow::anyhow!("git fetch failed to spawn: {e}"))?;
/// Fetch all refs from the remote into an existing mirror repo. Refreshes the
/// stored `origin` URL (the peer's URL may have changed) and fetches via the
/// `origin` remote so any stored promisor settings are honored.
///
/// `Promisor` applies the promisor config first (covers a repo that became
/// mode-B after a plain initial mirror). `Plain` on a mirror that was previously
/// a promisor (the repo went private -> public) clears the partial-clone config
/// and `--refetch`es, so the once-withheld, now-public blobs are backfilled
/// rather than left permanently missing.
async fn fetch_repo(local_path: &Path, remote_url: &str, mode: MirrorMode) -> anyhow::Result<()> {
let local_str = local_path.to_str().unwrap_or(".");
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr);
return Err(anyhow::anyhow!("git fetch failed: {stderr}"));
git_run(&["-C", local_str, "remote", "set-url", "origin", remote_url]).await?;
match mode {
MirrorMode::Promisor => {
git_run(&["-C", local_str, "config", "remote.origin.promisor", "true"]).await?;
git_run(&[
"-C",
local_str,
"config",
"remote.origin.partialclonefilter",
"blob:limit=10g",
])
.await?;
git_run(&["-C", local_str, "fetch", "--prune", "origin"]).await
}
MirrorMode::Plain => {
let was_promisor = git_config_get(local_str, "remote.origin.promisor")
.await
.as_deref()
== Some("true");
if was_promisor {
git_run_lenient(&[
"-C",
local_str,
"config",
"--unset",
"remote.origin.promisor",
])
.await;
git_run_lenient(&[
"-C",
local_str,
"config",
"--unset",
"remote.origin.partialclonefilter",
])
.await;
git_run(&["-C", local_str, "fetch", "--refetch", "--prune", "origin"]).await
} else {
git_run(&["-C", local_str, "fetch", "--prune", "origin"]).await
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::process::Command;
use tempfile::TempDir;
#[test]
fn classify_promisor_when_withheld_nonempty() {
let mode = classify_mirror(Some(vec!["/secret/**".to_string()]));
assert!(matches!(mode, MirrorMode::Promisor));
}
#[test]
fn classify_plain_when_withheld_empty() {
let mode = classify_mirror(Some(vec![]));
assert!(matches!(mode, MirrorMode::Plain));
}
#[test]
fn classify_plain_when_lookup_failed() {
// None == 404 / network error / parse failure: attempt a plain mirror
// and let the git read endpoint fail-close a mode-A repo.
let mode = classify_mirror(None);
assert!(matches!(mode, MirrorMode::Plain));
}
fn rb(oid: &str, cid: &str) -> ReplicaBlob {
ReplicaBlob {
oid: oid.to_string(),
cid: cid.to_string(),
}
}
#[test]
fn replicate_stores_new_blob() {
let remote = vec![rb("oid1", "cidA")];
let have = HashMap::new();
assert_eq!(blobs_needing_replication(&remote, &have), remote);
}
#[test]
fn replicate_skips_already_present_same_cid() {
let remote = vec![rb("oid1", "cidA")];
let mut have = HashMap::new();
have.insert("oid1".to_string(), "cidA".to_string());
assert!(blobs_needing_replication(&remote, &have).is_empty());
}
#[test]
fn replicate_restores_on_cid_change() {
// The origin re-sealed: same oid, new envelope, new cid.
let remote = vec![rb("oid1", "cidB")];
let mut have = HashMap::new();
have.insert("oid1".to_string(), "cidA".to_string());
assert_eq!(blobs_needing_replication(&remote, &have), remote);
}
#[test]
fn replicate_empty_remote_is_noop() {
assert!(blobs_needing_replication(&[], &HashMap::new()).is_empty());
}
#[test]
fn replicate_response_parses() {
// An older origin may still send a recipients field; it must be ignored.
let json = r#"{"blobs":[{"oid":"o1","cid":"c1","recipients":["did:key:zA"]}]}"#;
let parsed: ReplicateResponse = serde_json::from_str(json).unwrap();
assert_eq!(parsed.blobs.len(), 1);
assert_eq!(parsed.blobs[0].oid, "o1");
assert_eq!(parsed.blobs[0].cid, "c1");
}
#[test]
fn replicate_response_empty_blobs_parses() {
let parsed: ReplicateResponse = serde_json::from_str(r#"{"blobs":[]}"#).unwrap();
assert!(parsed.blobs.is_empty());
}
fn g(args: &[&str], dir: &Path) {
assert!(Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success());
}
/// Build a bare remote containing `files`, committed on one branch.
/// Returns (tempdir, file:// url). file:// makes git honor --filter.
fn bare_remote(files: &[(&str, &[u8])]) -> (TempDir, String) {
let td = TempDir::new().unwrap();
let origin = td.path().join("origin");
let bare = td.path().join("bare.git");
for (path, contents) in files {
let full = origin.join(path);
std::fs::create_dir_all(full.parent().unwrap()).unwrap();
std::fs::write(full, contents).unwrap();
}
g(&["init", "-q"], &origin);
g(&["config", "user.email", "t@t"], &origin);
g(&["config", "user.name", "t"], &origin);
g(&["add", "."], &origin);
g(&["commit", "-qm", "init"], &origin);
g(
&[
"clone",
"-q",
"--bare",
origin.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
let url = format!("file://{}", bare.display());
(td, url)
}
fn git_config(repo: &Path, key: &str) -> String {
let out = Command::new("git")
.args(["-C", repo.to_str().unwrap(), "config", "--get", key])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
fn object_count(repo: &Path) -> usize {
let out = Command::new("git")
.args([
"-C",
repo.to_str().unwrap(),
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname)",
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout)
.lines()
.filter(|l| !l.trim().is_empty())
.count()
}
#[tokio::test]
async fn promisor_clone_marks_promisor_and_keeps_objects() {
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n"), ("secret/b.txt", b"SECRET\n")]);
let dest = td.path().join("mirror.git");
clone_repo(&url, &dest, MirrorMode::Promisor).await.unwrap();
assert_eq!(git_config(&dest, "remote.origin.promisor"), "true");
assert_eq!(git_config(&dest, "remote.origin.mirror"), "true");
// No withholding on a plain bare origin, so every object is present:
// 1 commit + 1 root tree + 2 subtrees + 2 blobs = 6.
assert_eq!(object_count(&dest), 6);
}
#[tokio::test]
async fn plain_clone_is_not_promisor() {
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n")]);
let dest = td.path().join("mirror.git");
clone_repo(&url, &dest, MirrorMode::Plain).await.unwrap();
assert_eq!(git_config(&dest, "remote.origin.promisor"), "");
assert_eq!(git_config(&dest, "remote.origin.mirror"), "true");
}
#[tokio::test]
async fn promisor_fetch_updates_existing_mirror() {
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n")]);
let dest = td.path().join("mirror.git");
clone_repo(&url, &dest, MirrorMode::Promisor).await.unwrap();
let before = object_count(&dest);
// Add a second commit to the origin working tree and push to the bare
// (the working repo has no named remote, so push via the file:// URL).
let origin = td.path().join("origin");
std::fs::write(origin.join("public/c.txt"), b"more\n").unwrap();
g(&["add", "."], &origin);
g(&["commit", "-qm", "second"], &origin);
g(&["push", "-q", &url, "HEAD"], &origin);
fetch_repo(&dest, &url, MirrorMode::Promisor).await.unwrap();
assert_eq!(git_config(&dest, "remote.origin.promisor"), "true");
assert!(object_count(&dest) > before, "fetch pulled the new commit");
}
#[tokio::test]
async fn plain_fetch_clears_promisor_config_on_transition() {
// Repo started mode-B (promisor mirror), then went fully public, so the
// next sync classifies Plain. fetch_repo must drop the partial-clone
// config and refetch instead of leaving the mirror a promisor forever.
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n")]);
let dest = td.path().join("mirror.git");
clone_repo(&url, &dest, MirrorMode::Promisor).await.unwrap();
assert_eq!(git_config(&dest, "remote.origin.promisor"), "true");
fetch_repo(&dest, &url, MirrorMode::Plain).await.unwrap();
assert_eq!(git_config(&dest, "remote.origin.promisor"), "");
assert_eq!(git_config(&dest, "remote.origin.partialclonefilter"), "");
}
#[test]
fn registration_path_matches_replicas_route() {
// Must stay in sync with the route in api/mod.rs:
// PUT /api/v1/repos/:owner/:repo/replicas
assert_eq!(
replica_registration_path("z6MkOwner", "my-repo"),
"/api/v1/repos/z6MkOwner/my-repo/replicas"
);
}
#[tokio::test]
async fn registration_skipped_without_public_url() {
// No public URL to advertise → must return without sending anything.
// An unroutable origin URL would otherwise surface as a warn + delay.
let client = reqwest::Client::new();
let keypair = Keypair::generate();
register_replica_with_origin(
&client,
&keypair,
None,
"http://127.0.0.1:1", // would fail instantly if contacted
"owner",
"repo",
)
.await;
register_replica_with_origin(&client, &keypair, Some(""), "http://127.0.0.1:1", "o", "r")
.await;
}
Ok(())
}
+374
View File
@@ -0,0 +1,374 @@
//! Pure read-authorization logic for path-scoped visibility.
//!
//! `visibility_check` decides whether a caller may read a given path in a repo,
//! based on the repo's visibility rules with a fallback to the legacy
//! `is_public` flag. It performs no I/O so it is exhaustively unit tested.
use crate::db::VisibilityRule;
#[derive(Debug, PartialEq, Eq)]
pub enum Decision {
Allow,
Deny,
}
/// True if `caller` is the repo owner (matches full did:key or its short form),
/// mirroring the owner-match idiom in `api/protect.rs`.
fn is_owner(owner_did: &str, caller: &str) -> bool {
let owner_short = owner_did.split(':').next_back().unwrap_or(owner_did);
caller == owner_did || caller == owner_short
}
/// The match prefix for a glob: "/" stays "/", "/secret/**" becomes "/secret".
fn glob_prefix(glob: &str) -> &str {
let p = glob.trim_end_matches("**").trim_end_matches('/');
if p.is_empty() {
"/"
} else {
p
}
}
/// Does `glob` match `path`? "/" matches everything; "/secret" matches
/// "/secret" and any "/secret/..." descendant.
fn glob_matches(glob: &str, path: &str) -> bool {
let prefix = glob_prefix(glob);
if prefix == "/" {
return true;
}
path == prefix || path.starts_with(&format!("{prefix}/"))
}
/// Specificity = length of the match prefix; longer is more specific.
fn specificity(glob: &str) -> usize {
glob_prefix(glob).len()
}
/// Decide whether `caller` (None = anonymous) may read `path` in a repo.
/// `path` is "/" for a whole-repo clone/fetch.
///
/// Reader DIDs in a rule are matched exactly, so they must be stored in full
/// `did:key:...` form. The owner is the only identity matched in both full and
/// short form.
pub fn visibility_check(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
path: &str,
) -> Decision {
// The owner can always read everything.
if let Some(c) = caller {
if is_owner(owner_did, c) {
return Decision::Allow;
}
}
// Most-specific matching rule wins. On equal specificity the last rule in
// DB order is chosen; `list_visibility_rules` orders by `path_glob`, so this
// is deterministic.
let best = rules
.iter()
.filter(|r| glob_matches(&r.path_glob, path))
.max_by_key(|r| specificity(&r.path_glob));
match best {
Some(rule) => {
// Phase 1 treats every matching rule as an allow-list keyed by
// `reader_dids`. `rule.mode` (A vs B) is stored from day one but not
// acted on here; it governs replication behavior in Phases 2-3.
let allowed = caller
.map(|c| rule.reader_dids.iter().any(|d| d == c))
.unwrap_or(false);
if allowed {
Decision::Allow
} else {
Decision::Deny
}
}
None => {
if is_public {
Decision::Allow
} else {
Decision::Deny
}
}
}
}
/// The subtree path globs that `caller` (None = anonymous) may NOT read, given
/// the repo's rules. Whole-repo ("/") rules are excluded: a denied whole-repo
/// read is handled by the 404 gate before a clone ever starts. Each remaining
/// rule is reported when `visibility_check` denies the caller at the glob's
/// representative path. Used by the clean-clone client to sparse-exclude the
/// private paths from checkout.
pub fn withheld_globs(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Vec<String> {
rules
.iter()
.filter(|r| r.path_glob != "/")
.filter(|r| {
let probe = glob_prefix(&r.path_glob);
visibility_check(rules, is_public, owner_did, caller, probe) == Decision::Deny
})
.map(|r| r.path_glob.clone())
.collect()
}
/// The allowed globs that sit strictly underneath a denied glob. A clean-clone
/// client sparse-excludes everything in `withheld_globs`, which would also hide
/// these nested allowed paths; re-including them restores the caller's access.
/// Example: with `/secret/**` denied and `/secret/public/**` allowed for the
/// same caller, `/secret/public/**` is returned here so the client re-includes
/// it after excluding `/secret/`.
pub fn reincluded_globs(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Vec<String> {
let denied: Vec<&str> = rules
.iter()
.filter(|r| r.path_glob != "/")
.filter(|r| {
visibility_check(
rules,
is_public,
owner_did,
caller,
glob_prefix(&r.path_glob),
) == Decision::Deny
})
.map(|r| glob_prefix(&r.path_glob))
.collect();
rules
.iter()
.filter(|r| r.path_glob != "/")
.filter(|r| {
visibility_check(
rules,
is_public,
owner_did,
caller,
glob_prefix(&r.path_glob),
) == Decision::Allow
})
.filter(|r| {
let p = glob_prefix(&r.path_glob);
denied
.iter()
.any(|d| *d != p && p.starts_with(&format!("{d}/")))
})
.map(|r| r.path_glob.clone())
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::VisibilityMode;
use chrono::Utc;
fn rule(path_glob: &str, mode: VisibilityMode, readers: &[&str]) -> VisibilityRule {
VisibilityRule {
id: "x".into(),
repo_id: "r1".into(),
path_glob: path_glob.into(),
mode,
reader_dids: readers.iter().map(|s| s.to_string()).collect(),
created_by: "did:key:z6MkOwner".into(),
created_at: Utc::now(),
}
}
const OWNER: &str = "did:key:z6MkOwner";
#[test]
fn withheld_globs_lists_only_denied_subtrees() {
let rules = [
rule("/secret/**", VisibilityMode::B, &["did:key:z6MkFriend"]),
rule("/docs/**", VisibilityMode::B, &["did:key:z6MkStranger"]),
];
// Stranger is denied /secret but allowed /docs.
let mut got = withheld_globs(&rules, true, OWNER, Some("did:key:z6MkStranger"));
got.sort();
assert_eq!(got, vec!["/secret/**".to_string()]);
// Owner is denied nothing.
assert!(withheld_globs(&rules, true, OWNER, Some(OWNER)).is_empty());
// Anonymous is denied both.
let mut anon = withheld_globs(&rules, true, OWNER, None);
anon.sort();
assert_eq!(anon, vec!["/docs/**".to_string(), "/secret/**".to_string()]);
}
#[test]
fn reincluded_globs_restores_allowed_nested_path() {
let rules = [
rule("/secret/**", VisibilityMode::B, &["did:key:z6MkFriend"]),
rule(
"/secret/public/**",
VisibilityMode::B,
&["did:key:z6MkFriend", "did:key:z6MkStranger"],
),
];
// Stranger is denied /secret/** but allowed the nested /secret/public/**.
let withheld = withheld_globs(&rules, true, OWNER, Some("did:key:z6MkStranger"));
assert_eq!(withheld, vec!["/secret/**".to_string()]);
let reinc = reincluded_globs(&rules, true, OWNER, Some("did:key:z6MkStranger"));
assert_eq!(reinc, vec!["/secret/public/**".to_string()]);
// Owner is denied nothing, so there is nothing to re-include.
assert!(reincluded_globs(&rules, true, OWNER, Some(OWNER)).is_empty());
}
#[test]
fn no_rules_public_allows_anonymous() {
assert_eq!(
visibility_check(&[], true, OWNER, None, "/"),
Decision::Allow
);
}
#[test]
fn no_rules_private_denies_anonymous() {
assert_eq!(
visibility_check(&[], false, OWNER, None, "/"),
Decision::Deny
);
}
#[test]
fn root_rule_denies_anonymous() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/"),
Decision::Deny
);
}
#[test]
fn root_rule_allows_owner() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some(OWNER), "/"),
Decision::Allow
);
}
#[test]
fn root_rule_allows_owner_short_form() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some("z6MkOwner"), "/"),
Decision::Allow
);
}
#[test]
fn root_rule_allows_listed_reader() {
let rules = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some("did:key:z6MkFriend"), "/"),
Decision::Allow
);
}
#[test]
fn root_rule_denies_unlisted_reader() {
let rules = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
assert_eq!(
visibility_check(&rules, true, OWNER, Some("did:key:z6MkStranger"), "/"),
Decision::Deny
);
}
#[test]
fn subtree_rule_matches_descendant_path() {
let rules = [rule(
"/secret/**",
VisibilityMode::B,
&["did:key:z6MkFriend"],
)];
assert_eq!(
visibility_check(
&rules,
true,
OWNER,
Some("did:key:z6MkStranger"),
"/secret/a.rs"
),
Decision::Deny
);
assert_eq!(
visibility_check(
&rules,
true,
OWNER,
Some("did:key:z6MkFriend"),
"/secret/a.rs"
),
Decision::Allow
);
}
#[test]
fn subtree_rule_does_not_affect_root_clone() {
// A subtree rule must not gate a whole-repo (path "/") read: the public
// fallback applies because the subtree glob does not match "/".
let rules = [rule("/secret/**", VisibilityMode::B, &[])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/"),
Decision::Allow
);
}
#[test]
fn most_specific_rule_wins() {
// Public repo, but /secret is locked. A stranger reading /secret is denied
// by the more specific rule even though "/" would allow.
let rules = [
rule("/", VisibilityMode::A, &["did:key:z6MkStranger"]),
rule("/secret/**", VisibilityMode::B, &["did:key:z6MkFriend"]),
];
// stranger is a root reader but not a /secret reader
assert_eq!(
visibility_check(
&rules,
true,
OWNER,
Some("did:key:z6MkStranger"),
"/secret/a.rs"
),
Decision::Deny
);
// stranger can still read root
assert_eq!(
visibility_check(&rules, true, OWNER, Some("did:key:z6MkStranger"), "/"),
Decision::Allow
);
}
// Mirrors the gossip-announce gate in git_receive_pack: announce iff an
// anonymous caller can read "/".
#[test]
fn announce_gate_matches_public_readability() {
let announce = |rules: &[VisibilityRule], is_public: bool| {
visibility_check(rules, is_public, OWNER, None, "/") == Decision::Allow
};
// Public repo, no rules → announce.
assert!(announce(&[], true));
// Legacy private repo (is_public false, no rules) → silent.
assert!(!announce(&[], false));
// Mode A whole-repo rule with no public readers → silent.
assert!(!announce(&[rule("/", VisibilityMode::A, &[])], true));
// Mode B public repo with a private subtree → still announce.
assert!(announce(
&[rule("/secret/**", VisibilityMode::B, &[])],
true
));
}
}
+23 -12
View File
@@ -93,18 +93,29 @@ async fn fire_event_async(
}
match req.send().await {
Ok(resp) => tracing::info!(
url = %hook.url,
event = %event_name,
status = %resp.status(),
"webhook delivered"
),
Err(e) => tracing::warn!(
url = %hook.url,
event = %event_name,
err = %e,
"webhook delivery failed"
),
Ok(resp) => {
let result_label = if resp.status().is_success() {
"ok"
} else {
"http_error"
};
crate::metrics::record_webhook_delivery(result_label);
tracing::info!(
url = %hook.url,
event = %event_name,
status = %resp.status(),
"webhook delivered"
)
}
Err(e) => {
crate::metrics::record_webhook_delivery("network_error");
tracing::warn!(
url = %hook.url,
event = %event_name,
err = %e,
"webhook delivery failed"
)
}
}
});
}
+1
View File
@@ -3,6 +3,7 @@ name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
File diff suppressed because it is too large. Load diff
+19 -1
View File
@@ -14,7 +14,7 @@ impl NodeClient {
pub fn new(node_url: impl Into<String>, keypair: Option<Keypair>) -> Self {
let inner = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.user_agent("gl/0.2.0 gitlawb-cli")
.user_agent(format!("gl/{} gitlawb-cli", env!("CARGO_PKG_VERSION")))
.build()
.expect("failed to build HTTP client");
Self {
@@ -34,6 +34,24 @@ impl NodeClient {
.with_context(|| format!("GET {url}"))
}
/// GET with RFC 9421 HTTP Signature auth, for owner-only read endpoints.
/// Signs over the empty body (same shape the node verifies for signed reads).
pub async fn get_signed(&self, path: &str) -> Result<reqwest::Response> {
let url = format!("{}{}", self.node_url, path);
let kp = self
.keypair
.as_ref()
.context("get_signed requires an identity keypair")?;
let signed = sign_request(kp, "GET", path, b"");
let req = self
.inner
.get(&url)
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
req.send().await.with_context(|| format!("GET {url}"))
}
/// POST with JSON body + RFC 9421 HTTP Signature auth.
pub async fn post(&self, path: &str, body: &[u8]) -> Result<reqwest::Response> {
let url = format!("{}{}", self.node_url, path);
+15
View File
@@ -7,6 +7,7 @@ mod agent;
mod bounty;
mod cert;
mod changelog;
mod clone;
mod doctor;
mod http;
mod identity;
@@ -20,6 +21,7 @@ mod node;
mod node_stake;
mod peer;
mod pr;
mod profile;
mod protect;
mod quickstart;
mod register;
@@ -29,6 +31,7 @@ mod status;
mod sync;
mod task;
mod ucan_cmd;
mod visibility;
mod webhook;
mod whoami;
@@ -55,6 +58,9 @@ enum Commands {
/// Register this agent with a gitlawb node
Register(register::RegisterArgs),
/// Clone a gitlawb repo, handling private subtrees cleanly
Clone(clone::CloneArgs),
/// Manage repositories
Repo(repo::RepoArgs),
@@ -112,9 +118,15 @@ enum Commands {
/// List and inspect registered agents on a node
Agent(agent::AgentArgs),
/// Manage your agent profile (name, bio, avatar, social links)
Profile(profile::ProfileArgs),
/// Manage branch protection rules
Protect(protect::ProtectArgs),
/// Manage path-scoped read visibility rules
Visibility(visibility::VisibilityArgs),
/// Show unified activity changelog for a repository
Changelog(changelog::ChangelogArgs),
@@ -142,6 +154,7 @@ async fn main() -> Result<()> {
match cli.command {
Commands::Identity { cmd } => identity::run(cmd).await,
Commands::Register(args) => register::run(args).await,
Commands::Clone(args) => clone::run(args).await,
Commands::Repo(args) => repo::run(args).await,
Commands::Issue(args) => issue::run(args).await,
Commands::Pr(args) => pr::run(args).await,
@@ -161,7 +174,9 @@ async fn main() -> Result<()> {
Commands::Star(args) => star::run(args).await,
Commands::Status(args) => status::run(args).await,
Commands::Agent(args) => agent::run(args).await,
Commands::Profile(args) => profile::run(args).await,
Commands::Protect(args) => protect::run(args).await,
Commands::Visibility(args) => visibility::run(args).await,
Commands::Changelog(args) => changelog::run(args).await,
Commands::Bounty(args) => bounty::run(args).await,
Commands::Ucan(args) => ucan_cmd::run(args).await,
+510
View File
@@ -0,0 +1,510 @@
//! `gl profile` — manage your agent profile (display name, bio, avatar, socials).
//!
//! Profile metadata is stored on the gitlawb node and optionally pinned to IPFS
//! for decentralized resolution. All writes are signed with your DID key.
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
pub struct ProfileArgs {
#[command(subcommand)]
pub cmd: ProfileCmd,
}
#[derive(Subcommand)]
pub enum ProfileCmd {
/// Set your profile metadata (display name, bio, avatar, social links)
Set {
/// Display name (e.g. "Axiom")
#[arg(long)]
name: Option<String>,
/// Short bio (max 280 characters)
#[arg(long)]
bio: Option<String>,
/// Avatar URL or IPFS CID (e.g. "ipfs://bafkrei..." or "https://...")
#[arg(long)]
avatar: Option<String>,
/// Website URL
#[arg(long)]
website: Option<String>,
/// Twitter/X handle (without @)
#[arg(long)]
twitter: Option<String>,
/// GitHub username
#[arg(long)]
github: Option<String>,
/// Farcaster username
#[arg(long)]
farcaster: Option<String>,
/// Telegram username
#[arg(long)]
telegram: Option<String>,
/// Pin profile JSON to IPFS for decentralized resolution
#[arg(long)]
pin: bool,
/// Node URL
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Identity directory
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show your own profile
Show {
/// Node URL
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Identity directory
#[arg(long)]
dir: Option<PathBuf>,
},
/// Get another agent's profile by DID
Get {
/// DID or short DID prefix of the agent
did: String,
/// Node URL
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
},
/// Import profile from a JSON file
Import {
/// Path to profile JSON file
path: PathBuf,
/// Pin profile JSON to IPFS
#[arg(long)]
pin: bool,
/// Node URL
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Identity directory
#[arg(long)]
dir: Option<PathBuf>,
},
/// Export your profile as JSON
Export {
/// Node URL
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Identity directory
#[arg(long)]
dir: Option<PathBuf>,
},
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ProfilePayload {
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub bio: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub avatar_url: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub website: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub socials: Option<Socials>,
#[serde(skip_serializing_if = "Option::is_none")]
pub pin_to_ipfs: Option<bool>,
}
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct Socials {
#[serde(skip_serializing_if = "Option::is_none")]
pub twitter: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub github: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub farcaster: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub telegram: Option<String>,
}
pub async fn run(args: ProfileArgs) -> Result<()> {
match args.cmd {
ProfileCmd::Set {
name,
bio,
avatar,
website,
twitter,
github,
farcaster,
telegram,
pin,
node,
dir,
} => {
cmd_set(
name, bio, avatar, website, twitter, github, farcaster, telegram, pin, node, dir,
)
.await
}
ProfileCmd::Show { node, dir } => cmd_show(node, dir).await,
ProfileCmd::Get { did, node } => cmd_get(did, node).await,
ProfileCmd::Import {
path,
pin,
node,
dir,
} => cmd_import(path, pin, node, dir).await,
ProfileCmd::Export { node, dir } => cmd_export(node, dir).await,
}
}
#[allow(clippy::too_many_arguments)]
async fn cmd_set(
name: Option<String>,
bio: Option<String>,
avatar: Option<String>,
website: Option<String>,
twitter: Option<String>,
github: Option<String>,
farcaster: Option<String>,
telegram: Option<String>,
pin: bool,
node: String,
dir: Option<PathBuf>,
) -> Result<()> {
if name.is_none()
&& bio.is_none()
&& avatar.is_none()
&& website.is_none()
&& twitter.is_none()
&& github.is_none()
&& farcaster.is_none()
&& telegram.is_none()
{
anyhow::bail!(
"nothing to set — provide at least one of --name, --bio, --avatar, --website, --twitter, --github, --farcaster, --telegram"
);
}
if let Some(ref b) = bio {
if b.len() > 280 {
anyhow::bail!("bio must be 280 characters or fewer (got {})", b.len());
}
}
let keypair = load_keypair_from_dir(dir.as_deref())?;
let did = keypair.did();
let has_socials =
twitter.is_some() || github.is_some() || farcaster.is_some() || telegram.is_some();
let socials = if has_socials {
Some(Socials {
twitter,
github,
farcaster,
telegram,
})
} else {
None
};
let payload = ProfilePayload {
display_name: name,
bio,
avatar_url: avatar,
website,
socials,
pin_to_ipfs: if pin { Some(true) } else { None },
};
let client = NodeClient::new(&node, Some(keypair));
let body = serde_json::to_vec(&payload)?;
println!("Updating profile for {did}...");
let resp = client
.put("/api/v1/profile", &body)
.await
.context("failed to update profile")?;
let status = resp.status();
let resp_body: serde_json::Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = resp_body
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("unknown error");
anyhow::bail!("profile update failed ({status}): {msg}");
}
println!();
println!("✓ Profile updated");
if let Some(name) = resp_body.get("display_name").and_then(|v| v.as_str()) {
println!(" Name: {name}");
}
if let Some(bio) = resp_body.get("bio").and_then(|v| v.as_str()) {
println!(" Bio: {bio}");
}
if let Some(avatar) = resp_body.get("avatar_url").and_then(|v| v.as_str()) {
println!(" Avatar: {avatar}");
}
if let Some(cid) = resp_body.get("profile_cid").and_then(|v| v.as_str()) {
println!(" IPFS: ipfs://{cid}");
}
Ok(())
}
async fn cmd_show(node: String, dir: Option<PathBuf>) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let did_str = keypair.did().to_string();
let short = did_short(&did_str).to_string();
cmd_get(short, node).await
}
async fn cmd_get(did: String, node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
let path = format!("/api/v1/agents/{did}/profile");
let resp = client.get(&path).await.context("failed to fetch profile")?;
let status = resp.status();
if status.as_u16() == 404 {
println!("No profile found for {did}");
println!("Set one with: gl profile set --name \"Your Name\" --bio \"About you\"");
return Ok(());
}
let body: serde_json::Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = body
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("unknown error");
anyhow::bail!("failed to get profile ({status}): {msg}");
}
let did_str = body["did"].as_str().unwrap_or(&did);
let name = body["display_name"].as_str().unwrap_or("(not set)");
let bio = body["bio"].as_str().unwrap_or("(not set)");
let avatar = body["avatar_url"].as_str();
let website = body["website"].as_str();
let cid = body["profile_cid"].as_str();
println!("Agent Profile");
println!(" DID: {did_str}");
println!(" Name: {name}");
println!(" Bio: {bio}");
if let Some(a) = avatar {
println!(" Avatar: {a}");
}
if let Some(w) = website {
println!(" Website: {w}");
}
if let Some(socials) = body.get("socials") {
let mut has_any = false;
if let Some(t) = socials["twitter"].as_str() {
if !has_any {
println!(" Socials:");
has_any = true;
}
println!(" Twitter: @{t}");
}
if let Some(g) = socials["github"].as_str() {
if !has_any {
println!(" Socials:");
has_any = true;
}
println!(" GitHub: {g}");
}
if let Some(f) = socials["farcaster"].as_str() {
if !has_any {
println!(" Socials:");
has_any = true;
}
println!(" Farcaster: {f}");
}
if let Some(tg) = socials["telegram"].as_str() {
if !has_any {
println!(" Socials:");
// suppress unused assignment warning
let _ = has_any;
}
println!(" Telegram: {tg}");
}
}
if let Some(c) = cid {
println!(" IPFS: ipfs://{c}");
}
let updated = body["updated_at"].as_str().unwrap_or("unknown");
println!(" Updated: {updated}");
Ok(())
}
async fn cmd_import(path: PathBuf, pin: bool, node: String, dir: Option<PathBuf>) -> Result<()> {
let content = std::fs::read_to_string(&path)
.with_context(|| format!("could not read {}", path.display()))?;
let mut payload: ProfilePayload =
serde_json::from_str(&content).context("invalid profile JSON")?;
if pin {
payload.pin_to_ipfs = Some(true);
}
if let Some(ref b) = payload.bio {
if b.len() > 280 {
anyhow::bail!("bio must be 280 characters or fewer (got {})", b.len());
}
}
let keypair = load_keypair_from_dir(dir.as_deref())?;
let did = keypair.did();
let client = NodeClient::new(&node, Some(keypair));
let body = serde_json::to_vec(&payload)?;
println!("Importing profile from {} for {did}...", path.display());
let resp = client
.put("/api/v1/profile", &body)
.await
.context("failed to import profile")?;
let status = resp.status();
let resp_body: serde_json::Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = resp_body
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("unknown error");
anyhow::bail!("profile import failed ({status}): {msg}");
}
println!("✓ Profile imported successfully");
Ok(())
}
async fn cmd_export(node: String, dir: Option<PathBuf>) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let did_str = keypair.did().to_string();
let short = did_short(&did_str);
let client = NodeClient::new(&node, None);
let path = format!("/api/v1/agents/{short}/profile");
let resp = client.get(&path).await.context("failed to fetch profile")?;
let status = resp.status();
if status.as_u16() == 404 {
anyhow::bail!("no profile found — set one first with `gl profile set`");
}
let body: serde_json::Value = resp.json().await.context("invalid JSON response")?;
if !status.is_success() {
let msg = body
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("unknown error");
anyhow::bail!("failed to export profile ({status}): {msg}");
}
let export = json!({
"display_name": body.get("display_name"),
"bio": body.get("bio"),
"avatar_url": body.get("avatar_url"),
"website": body.get("website"),
"socials": body.get("socials"),
});
println!("{}", serde_json::to_string_pretty(&export)?);
Ok(())
}
fn did_short(did: &str) -> &str {
did.rsplit(':').next().unwrap_or(did)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_did_short_extracts_suffix() {
assert_eq!(did_short("did:key:z6MkfP9F7z"), "z6MkfP9F7z");
assert_eq!(did_short("z6MkfP9F7z"), "z6MkfP9F7z");
}
#[test]
fn test_profile_payload_serialization() {
let payload = ProfilePayload {
display_name: Some("Axiom".to_string()),
bio: Some("AI builder".to_string()),
avatar_url: None,
website: None,
socials: Some(Socials {
twitter: Some("AxiomBot".to_string()),
github: Some("0xAxiom".to_string()),
farcaster: None,
telegram: None,
}),
pin_to_ipfs: None,
};
let json = serde_json::to_string(&payload).unwrap();
assert!(json.contains("Axiom"));
assert!(json.contains("AxiomBot"));
assert!(!json.contains("avatar_url"));
}
#[test]
fn test_profile_payload_deserialization() {
let json = r#"{
"display_name": "Test Agent",
"bio": "I test things",
"socials": {
"twitter": "testbot",
"github": "test-org"
}
}"#;
let payload: ProfilePayload = serde_json::from_str(json).unwrap();
assert_eq!(payload.display_name.as_deref(), Some("Test Agent"));
assert_eq!(
payload.socials.as_ref().unwrap().twitter.as_deref(),
Some("testbot")
);
}
#[test]
fn test_bio_length_validation() {
let long_bio = "a".repeat(281);
assert!(long_bio.len() > 280);
}
}
+9 -2
View File
@@ -1,7 +1,9 @@
use anyhow::Result;
use clap::{Args, Subcommand};
use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
pub struct SyncArgs {
@@ -11,6 +13,10 @@ pub struct SyncArgs {
/// Node URL
#[arg(long, env = "GITLAWB_NODE", default_value = "https://node.gitlawb.com")]
pub node: String,
/// Identity directory for signed sync trigger requests
#[arg(long)]
pub dir: Option<PathBuf>,
}
#[derive(Subcommand)]
@@ -22,10 +28,10 @@ pub enum SyncCmd {
}
pub async fn run(args: SyncArgs) -> Result<()> {
let client = NodeClient::new(&args.node, None);
match args.cmd {
SyncCmd::Trigger => {
let keypair = load_keypair_from_dir(args.dir.as_deref()).ok();
let client = NodeClient::new(&args.node, keypair);
let resp: serde_json::Value = client
.post("/api/v1/sync/trigger", b"{}")
.await?
@@ -39,6 +45,7 @@ pub async fn run(args: SyncArgs) -> Result<()> {
println!(" worker picks up within 30s");
}
SyncCmd::Status => {
let client = NodeClient::new(&args.node, None);
// Just show peer list and node stats for now
let stats: serde_json::Value = client.get("/api/v1/stats").await?.json().await?;
let peers: serde_json::Value = client.get("/api/v1/peers").await?.json().await?;
+310
View File
@@ -0,0 +1,310 @@
//! `gl visibility`: manage path-scoped read visibility rules.
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::{Path, PathBuf};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
pub struct VisibilityArgs {
#[command(subcommand)]
pub cmd: VisibilityCmd,
}
#[derive(Subcommand)]
pub enum VisibilityCmd {
/// Set a visibility rule. Use "/" for the whole repo.
Set {
/// Path glob, e.g. "/" or "/secret-pkg/**"
path_glob: String,
#[arg(long)]
repo: String,
/// Comma-separated reader DIDs allowed to read this path
#[arg(long, value_delimiter = ',')]
readers: Vec<String>,
/// Replication mode: "a" (hide, whole-repo only) or "b" (lock content)
#[arg(long, default_value = "b")]
mode: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Remove a visibility rule.
Remove {
path_glob: String,
#[arg(long)]
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// List visibility rules for a repo (owner only).
List {
#[arg(long)]
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
}
pub async fn run(args: VisibilityArgs) -> Result<()> {
match args.cmd {
VisibilityCmd::Set {
path_glob,
repo,
readers,
mode,
node,
dir,
} => cmd_set(path_glob, repo, readers, mode, node, dir).await,
VisibilityCmd::Remove {
path_glob,
repo,
node,
dir,
} => cmd_remove(path_glob, repo, node, dir).await,
VisibilityCmd::List { repo, node, dir } => cmd_list(repo, node, dir).await,
}
}
async fn resolve_owner_repo(
repo: &str,
node: &str,
dir: Option<&Path>,
) -> Result<(String, String)> {
if let Some((owner, name)) = repo.split_once('/') {
return Ok((owner.to_string(), name.to_string()));
}
let short = if let Ok(kp) = load_keypair_from_dir(dir) {
let did = kp.did().to_string();
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(node, None);
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let did = info["did"].as_str().context("node missing DID")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
Ok((short, repo.to_string()))
}
async fn cmd_set(
path_glob: String,
repo: String,
readers: Vec<String>,
mode: String,
node: String,
dir: Option<PathBuf>,
) -> Result<()> {
let kp = load_keypair_from_dir(dir.as_deref())
.context("identity not found, run `gl identity new` first")?;
let (owner, name) = resolve_owner_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, Some(kp));
let body = serde_json::to_vec(&serde_json::json!({
"path_glob": path_glob,
"mode": mode,
"reader_dids": readers,
}))?;
let resp = client
.put(&format!("/api/v1/repos/{owner}/{name}/visibility"), &body)
.await
.context("failed to connect to node")?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("visibility set failed ({status}): {msg}");
}
println!("✓ Visibility rule set on {owner}/{name}: {path_glob} (mode {mode})");
if path_glob != "/" {
println!(
" Note: subtree content is NOT withheld from clones yet (Phase 3). Only whole-repo (\"/\") rules are enforced today. This rule is stored and will take effect when subtree enforcement ships."
);
}
Ok(())
}
async fn cmd_remove(
path_glob: String,
repo: String,
node: String,
dir: Option<PathBuf>,
) -> Result<()> {
let kp = load_keypair_from_dir(dir.as_deref())
.context("identity not found, run `gl identity new` first")?;
let (owner, name) = resolve_owner_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, Some(kp));
let body = serde_json::to_vec(&serde_json::json!({ "path_glob": path_glob }))?;
let resp = client
.delete(&format!("/api/v1/repos/{owner}/{name}/visibility"), &body)
.await
.context("failed to connect to node")?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("visibility remove failed ({status}): {msg}");
}
println!("✓ Visibility rule removed from {owner}/{name}: {path_glob}");
Ok(())
}
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let kp = load_keypair_from_dir(dir.as_deref())
.context("identity not found, run `gl identity new` first")?;
let (owner, name) = resolve_owner_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, Some(kp));
// owner-only endpoint: must send a signed request
let resp = client
.get_signed(&format!("/api/v1/repos/{owner}/{name}/visibility"))
.await
.context("failed to connect to node")?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("visibility list failed ({status}): {msg}");
}
let rules = body["rules"].as_array().cloned().unwrap_or_default();
if rules.is_empty() {
println!("No visibility rules on {owner}/{name} (repo follows its is_public flag).");
} else {
println!("Visibility rules for {owner}/{name}:");
for r in rules {
let glob = r["path_glob"].as_str().unwrap_or("?");
let mode = r["mode"].as_str().unwrap_or("?");
let readers = r["reader_dids"].as_array().cloned().unwrap_or_default();
let readers: Vec<&str> = readers.iter().filter_map(|d| d.as_str()).collect();
let readers_str = if readers.is_empty() {
"none".to_string()
} else {
readers.join(", ")
};
println!(" {glob} (mode {mode}) readers: {readers_str}");
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_cmd_set_success() {
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _m = server
.mock(
"PUT",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/visibility".to_string()),
)
.with_status(201)
.with_header("content-type", "application/json")
.with_body(r#"{"status":"set"}"#)
.create_async()
.await;
cmd_set(
"/".to_string(),
"myrepo".to_string(),
vec!["did:key:abc".to_string()],
"b".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_list_success() {
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/visibility".to_string()),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"rules":[{"path_glob":"/","mode":"b","reader_dids":["did:key:abc"]}]}"#)
.create_async()
.await;
cmd_list(
"myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_remove_success() {
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _m = server
.mock(
"DELETE",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/visibility".to_string()),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"status":"removed"}"#)
.create_async()
.await;
cmd_remove(
"/".to_string(),
"myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
}
+7 -1
View File
@@ -24,7 +24,7 @@ services:
condition: service_healthy
ports:
- "${GITLAWB_PORT:-7545}:${GITLAWB_PORT:-7545}" # HTTP API + git smart-HTTP
- "${GITLAWB_P2P_PORT:-7546}:${GITLAWB_P2P_PORT:-7546}" # libp2p
- "${GITLAWB_P2P_PORT:-7546}:${GITLAWB_P2P_PORT:-7546}/udp" # libp2p QUIC
volumes:
- gitlawb-data:/data
environment:
@@ -33,6 +33,12 @@ services:
GITLAWB_PORT: ${GITLAWB_PORT:-7545}
GITLAWB_PUBLIC_URL: ${GITLAWB_PUBLIC_URL:-http://localhost:7545}
GITLAWB_P2P_PORT: ${GITLAWB_P2P_PORT:-7546}
GITLAWB_BOOTSTRAP_PEERS: ${GITLAWB_BOOTSTRAP_PEERS:-}
GITLAWB_BOOTSTRAP_DISABLE_SEEDS: ${GITLAWB_BOOTSTRAP_DISABLE_SEEDS:-false}
GITLAWB_REQUIRE_SIGNED_PEER_WRITES: ${GITLAWB_REQUIRE_SIGNED_PEER_WRITES:-false}
GITLAWB_P2P_BOOTSTRAP: ${GITLAWB_P2P_BOOTSTRAP:-}
GITLAWB_PUBLIC_READ: ${GITLAWB_PUBLIC_READ:-true}
GITLAWB_MAX_PACK_BYTES: ${GITLAWB_MAX_PACK_BYTES:-2147483648}
# Sync
GITLAWB_AUTO_SYNC: ${GITLAWB_AUTO_SYNC:-false}
# On-chain PoS (optional — leave unset for local/dev)
+1 -1
View File
@@ -114,7 +114,7 @@ The caller of `distribute()` receives 1% of the pot. At 100k weekly, that's 1,00
Why: removes operational burden. Even if the official keeper misses a run, any staker or searcher has financial incentive to call it.
Repo ships a GitHub Actions workflow (`keeper-distribute.yml`) that runs Sundays as a default keeper — the wallet it uses self-funds its gas from the 1% reward.
Operators can run a weekly keeper job that calls `distribute()`; the caller wallet self-funds its gas from the 1% reward.
---
+82
View File
@@ -0,0 +1,82 @@
# Maintainer roadmap
Gitlawb node is live software. People are already using the network, so maintainer work should optimize for uptime, compatibility, and predictable rollouts before feature velocity.
Mission: when code is pushed to a node, the node should not go down, and the network should be able to serve git content like a resilient delivery layer.
## Operating principles
- Protect live nodes first. Do not ship breaking protocol or API changes without a staged compatibility window.
- Prefer opt-in hardening flags before making behavior mandatory across the network.
- Make every public claim in docs match current behavior.
- Treat every node operator as a production user, even when the feature is early.
- Keep new contributor tasks small and testable.
## Now: stabilize the live network
Owner focus: maintainers.
- Keep CI green on `main`: format, clippy, workspace tests, release build.
- Add PR Docker smoke testing so quickstart regressions are caught before merge.
- Add installer smoke tests for release archive names and extraction layout.
- Publish a short operator notice for `GITLAWB_REQUIRE_SIGNED_PEER_WRITES`: leave it `false` until all known peers upgrade, then flip it.
- Add a live-node checklist: health endpoint, database backup, data volume backup, public URL, reverse proxy/TLS, peer reachability.
- Track known public nodes and their version so coordinated upgrades are possible.
## Next: stop avoidable downtime
Owner focus: node reliability.
- Add versioned database migrations instead of implicit startup schema changes.
- Add graceful shutdown and startup logging that clearly distinguishes database, repos, P2P, and operator-loop failures.
- Add bounded request/concurrency settings for git pack operations.
- Add operator docs for lowering `GITLAWB_MAX_PACK_BYTES` on small nodes.
- Add recovery docs for a broken repo cache, missing data volume, or failed Tigris/S3 sync.
- Add basic metrics for pushes, fetches, pack sizes, peer sync queue, failed auth, and webhook failures.
## Security hardening
Owner focus: protocol/auth.
- Implement repo write authorization: repo owner checks, protected branches, UCAN capability checks, and clear delegation semantics.
- Implement private-read enforcement or remove private-repo affordances until it exists.
- Add UCAN revocation or blocklisting, with an emergency compromised-key runbook.
- Add mutation-aware GraphQL auth before GraphQL becomes a public write API surface.
- Harden peer registration and outbound peer calls against SSRF and peer-list poisoning.
- After all live peers upgrade, enable signed peer writes by default.
## Product and CLI
Owner focus: developer experience.
- Make public-node vs localhost behavior explicit across `gl`, `git-remote-gitlawb`, README, and quickstart docs.
- Add `--version` support to `git-remote-gitlawb` so release smoke tests can validate it directly.
- Improve `gl doctor` for live operators: check Docker, Postgres, public URL, node health, p2p port, and binary versions.
- Add a compact `gl node doctor` command for self-hosted nodes.
- Make errors from JSON API responses more consistent and actionable.
## UI direction
Owner focus: desktop/operator UI.
- Start with an operator dashboard, not a broad GitHub clone UI.
- Show node status, version, DID, public URL, database status, repo count, peer count, recent pushes, sync queue, and storage mode.
- Include safe controls: start/stop local stack, copy config, open logs, run health checks.
- Avoid making PoS/operator wallet actions one-click until signing, key storage, and failure states are reviewed.
## Release rhythm
- Patch releases: bug fixes, docs corrections, backward-compatible hardening.
- Minor releases: new CLI/API features, opt-in hardening flags, new operator UI capabilities.
- Breaking protocol changes: require a migration note, rollout plan, and compatibility window.
## Suggested issue labels
- `live-network`
- `good-first-issue`
- `operator-experience`
- `security`
- `protocol`
- `ci-release`
- `docs`
- `ui`
+148
View File
@@ -0,0 +1,148 @@
# OSS readiness audit
Date: 2026-05-28
Repo state inspected: `main` tracking `origin/main`, starting at `b12c6bc feat: per-DID rate limiting on creation endpoints (10/hour) (#13)`.
## Commands run
```sh
git status --short --branch
git remote -v
git branch --show-current
git rev-parse --abbrev-ref --symbolic-full-name '@{u}'
git log -1 --oneline
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo run -q -p gl -- --help
cargo run -q -p gitlawb-node -- --help
docker --version
docker compose config
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb
docker build -t gitlawb-node:oss-audit .
```
## Build and test health
- `cargo fmt --all -- --check` passed.
- `cargo clippy --workspace --all-targets -- -D warnings` passed.
- `cargo test --workspace` passed: `git-remote-gitlawb` 6 tests, `gitlawb-core` 38 tests, `gitlawb-node` 59 tests, `gl` 188 tests, plus doc tests.
- `gl --help` and `gitlawb-node --help` both render successfully.
- `docker compose config` parses successfully.
- `cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb` passed.
- `target/release/gitlawb-node --version` and `target/release/gl --version` report `0.3.9`.
- `git-remote-gitlawb` has no `--version` flag; release smoke tests should use a helper-specific invocation.
- Open Rust Dependabot alerts were removed from the active dependency graph by upgrading vulnerable crates and switching P2P from TCP/Yamux to QUIC/UDP.
- `cargo-audit` is not installed in this environment, so advisory validation used GitHub Dependabot API output plus `cargo tree`/`Cargo.lock` checks confirming the alerted vulnerable package versions are no longer present.
- Full Docker image build could not run in this environment because the Docker CLI is installed but the Docker Desktop Linux engine pipe is not available.
- `bash -n install.sh` could not run in this Windows environment because `bash` is routed through WSL and WSL has no `/bin/bash` installed.
Recommended next CI additions:
- Add a PR Docker image smoke test (`docker build` plus `gitlawb-node --version`).
- Add installer smoke tests for Linux/macOS archive names and extraction layout.
- Add `cargo audit` or equivalent advisory reporting, with documented ignores for accepted advisories.
- Add an MSRV check so Rust 1.91+ remains an explicit supported contract.
## Install and docs accuracy
Fixed in this pass:
- `install.sh` now downloads from `Gitlawb/node`, matches release asset names (`gitlawb-node-<version>-<target>.tar.gz`), handles `--version vX.Y.Z`, extracts the packaged directory, verifies checksums, and installs `gl`, `git-remote-gitlawb`, and `gitlawb-node` when present.
- `docs/RUN-A-NODE.md` now matches the README Rust requirement (`1.91+`) and the release workflow's GHCR image path (`ghcr.io/gitlawb/node:latest`).
- `docs/ECONOMICS.md` no longer claims this repo ships a `keeper-distribute.yml` workflow that is not present.
- `.env.example` now distinguishes HTTP bootstrap peers from libp2p multiaddrs and documents seed-list opt-out.
- `scripts/build-bins.sh` now writes to `dist/bin` instead of a missing `web/public/bin` path.
- P2P docs/config now describe QUIC/UDP on `GITLAWB_P2P_PORT`; Docker and Fly configs expose that port as UDP.
Remaining doc caveats:
- Public docs URLs (`gitlawb.com/install.sh`, `docs.gitlawb.com`) were not verified in this local pass.
- `git-remote-gitlawb` defaults to `http://127.0.0.1:7545`, while most `gl` commands default to `https://node.gitlawb.com`; the docs should be explicit about setting `GITLAWB_NODE` for public-network cloning/fetching.
## Docker and self-hosting readiness
Positive:
- Runtime image runs as a non-root `gitlawb` user.
- Compose includes Postgres health checks and persistent volumes for database and node data.
- Node health check uses `/health`.
- Source default bind is `127.0.0.1`; Compose intentionally exposes `0.0.0.0`.
Risks:
- Compose defaults `POSTGRES_PASSWORD` to `changeme`; production docs should require changing it before public exposure.
- Compose publishes HTTP and libp2p ports directly. Operators should put HTTP behind TLS/reverse proxy and decide explicitly whether P2P is public.
- The node auto-merges `bootstrap-peers.json` unless `GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true`; good for public discovery, surprising for isolated dev/test networks.
- There is no versioned database migration system; schema is created/altered from code at startup, which is convenient early on but risky for live upgrades.
## Security and trust boundaries
Fixed or staged in this pass:
- `POST /api/v1/bounties/{id}/dispute` moved behind HTTP Signature auth.
- Peer announce/sync notifications are now signed by upgraded nodes.
- Peer announce and sync notify handlers verify the signature keyid when a signature is present.
- `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false` keeps live nodes backward-compatible during rolling upgrades; operators can flip it after all known peers upgrade.
Live-network blockers to prioritize:
- GraphQL POST is still open for compatibility; GraphQL mutations should get mutation-aware auth before it becomes a public write API surface.
- Push authorization is still not capability-complete. A valid DID signature is authentication, not authorization; unprotected repo branches do not yet enforce owner/UCAN capability checks.
- UCAN chain validation is incomplete and UCAN revocation/blocklisting is not implemented as an operator feature.
- Private repository reads are not enforced. `is_public` and `GITLAWB_PUBLIC_READ` exist, but per-repository private-read behavior is not wired.
- Peer URLs are self-asserted by DIDs. Signatures prove control of the DID key when present, not ownership/safety of the announced URL.
- Outbound peer fetch/ping/sync paths should be reviewed for SSRF protections before accepting arbitrary public peer registrations.
## Keys, identity, and auth handling
- Node identity keys are generated as Ed25519 PKCS#8 PEM files; Unix builds set `0600` on newly generated node keys.
- Windows builds do not apply equivalent ACL hardening.
- Operator wallet private keys are read from `GITLAWB_OPERATOR_PRIVATE_KEY`; operators should use a dedicated low-balance wallet and avoid process managers that expose env vars broadly.
- The CLI stores identity in a PEM file and signs API writes with RFC 9421 headers.
- HTTP Signature clock skew is limited to 5 minutes.
## Network exposure and defaults
- `gitlawb-node` defaults to `127.0.0.1:7545` from source, but Docker Compose binds and publishes `7545` and `7546`.
- `/health`, `/`, `/api/v1/stats`, `/api/v1/contracts`, and read APIs are public by default.
- `GITLAWB_MAX_PACK_BYTES` defaults to 2 GiB for git smart-HTTP. Operators on small nodes should lower it.
- `GITLAWB_AUTO_SYNC=false` by default, which is a good conservative default.
## CLI usability
Positive:
- The top-level `gl --help` surface is broad and discoverable.
- `gl doctor`, `gl quickstart`, `gl init`, and `gl status` are good OSS onboarding affordances.
- The CLI now reports its real package version in the HTTP user agent.
Risks:
- Many commands default to the public node; `git-remote-gitlawb` defaults to localhost. This split should be called out in README examples.
- `gl sync trigger` signs the request when a local identity is available and falls back to the legacy unsigned request for live compatibility.
- Several CLI commands parse dynamic JSON responses permissively; good for compatibility, but error messages can hide response-shape regressions.
## CI and release readiness
Positive:
- PR CI pins third-party actions by commit SHA.
- CI runs format, clippy with `-D warnings`, and workspace tests.
- Release workflow uses release-please, builds a Docker image, runs a Docker `--version` smoke test, and attaches multi-target binary archives.
Risks:
- PR CI does not build the Docker image, so Dockerfile regressions can reach `main`.
- Release binaries are not smoke-tested after packaging.
- The installer was not covered by CI; this pass fixed current asset-name/layout mismatches, but tests should lock it down.
- No automated dependency advisory job is present.
## Obvious live-network priorities
1. Implement repo write authorization: owner checks, UCAN capability checks, and clear delegation semantics for push/PR/issue/bounty operations.
2. Implement private-read enforcement or remove private repo affordances until it exists.
3. Add UCAN revocation/blocklisting and operator docs for emergency key compromise.
4. Harden peer registration and outbound fetch behavior against SSRF and peer-list poisoning.
5. Add Docker/installer/release smoke tests to CI.
6. Label PoS/economics docs consistently with the current live contract and rewards status.
+4 -4
View File
@@ -7,8 +7,8 @@ Step-by-step guide to staking $GITLAWB, registering your node on-chain, and earn
## Prerequisites
- A wallet with at least **10,000 $GITLAWB** (minimum stake) plus a small amount of ETH on Base for gas
- Docker or Rust 1.80+ (for running the node process)
- A public HTTP URL (your-host.com) — can be a VPS, Fly.io app, or anything reachable
- Docker or Rust 1.91+ (for running the node process)
- A public HTTP URL (your-host.com) — can be a VPS, Fly.io app, or anything reachable. A Fly.io config is provided at `infra/fly/fly.toml` (deploy from the repo root with `fly deploy -c infra/fly/fly.toml`)
---
@@ -61,7 +61,7 @@ What this does:
docker run -d \
--name gitlawb-node \
-p 7545:7545 \
-p 7546:7546 \
-p 7546:7546/udp \
-v gitlawb-data:/data \
-e DATABASE_URL=postgresql://user:pass@host/gitlawb \
-e GITLAWB_PUBLIC_URL=https://my-node.example.com \
@@ -69,7 +69,7 @@ docker run -d \
-e GITLAWB_CONTRACT_NODE_STAKING=$GITLAWB_CONTRACT_NODE_STAKING \
-e GITLAWB_CHAIN_RPC_URL=$GITLAWB_CHAIN_RPC_URL \
-e GITLAWB_OPERATOR_STRICT_MODE=true \
ghcr.io/gitlawb/gitlawb-node:latest
ghcr.io/gitlawb/node:latest
```
### Option B — docker-compose
+36
View File
@@ -0,0 +1,36 @@
# infra/
Deployment configuration, organized by target — one subdirectory per platform.
```text
infra/
├── fly/
│ └── fly.toml # Fly.io app config (gitlawb-node-test)
└── aws/ # Terraform: single EC2 + Docker (see aws/README.md)
```
## Deploying to Fly.io
Run from the **repo root** so the Docker build context includes `crates/`,
`Cargo.toml`, and `bootstrap-peers.json`:
```sh
fly deploy -c infra/fly/fly.toml
```
The `dockerfile` path inside `fly.toml` is resolved relative to the config
file, so it points to `../../Dockerfile`.
## Deploying to AWS
See [`aws/README.md`](aws/README.md) — Terraform for a single EC2 instance
running the published `ghcr.io/gitlawb/node` image with Docker compose.
## What intentionally stays at the repo root
- `Dockerfile` / `Dockerfile.bins` — shared by the release CI workflow
(`.github/workflows/release.yml`), `scripts/build-bins.sh`, and Fly builds.
- `docker-compose.yml` — local dev stack; bundled into the macOS app by
`scripts/build-macos-app.sh` and used for repo detection by the app.
Future targets should follow the same per-platform layout.
+17
View File
@@ -0,0 +1,17 @@
# Terraform state and workdir — never commit (state contains secrets)
*.tfstate
*.tfstate.*
.terraform/
crash.log
crash.*.log
# Local variable files may contain secrets; keep the example only
*.tfvars
*.tfvars.json
!terraform.tfvars.example
# Note: .terraform.lock.hcl IS committed (reproducible provider versions)
# Saved plan files can embed sensitive values
tfplan
*.tfplan
+46
View File
@@ -0,0 +1,46 @@
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.
provider "registry.terraform.io/hashicorp/aws" {
version = "5.100.0"
constraints = "~> 5.0"
hashes = [
"h1:wOhTPz6apLBuF7/FYZuCoXRK/MLgrNprZ3vXmq83g5k=",
"zh:054b8dd49f0549c9a7cc27d159e45327b7b65cf404da5e5a20da154b90b8a644",
"zh:0b97bf8d5e03d15d83cc40b0530a1f84b459354939ba6f135a0086c20ebbe6b2",
"zh:1589a2266af699cbd5d80737a0fe02e54ec9cf2ca54e7e00ac51c7359056f274",
"zh:6330766f1d85f01ae6ea90d1b214b8b74cc8c1badc4696b165b36ddd4cc15f7b",
"zh:7c8c2e30d8e55291b86fcb64bdf6c25489d538688545eb48fd74ad622e5d3862",
"zh:99b1003bd9bd32ee323544da897148f46a527f622dc3971af63ea3e251596342",
"zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425",
"zh:9f8b909d3ec50ade83c8062290378b1ec553edef6a447c56dadc01a99f4eaa93",
"zh:aaef921ff9aabaf8b1869a86d692ebd24fbd4e12c21205034bb679b9caf883a2",
"zh:ac882313207aba00dd5a76dbd572a0ddc818bb9cbf5c9d61b28fe30efaec951e",
"zh:bb64e8aff37becab373a1a0cc1080990785304141af42ed6aa3dd4913b000421",
"zh:dfe495f6621df5540d9c92ad40b8067376350b005c637ea6efac5dc15028add4",
"zh:f0ddf0eaf052766cfe09dea8200a946519f653c384ab4336e2a4a64fdd6310e9",
"zh:f1b7e684f4c7ae1eed272b6de7d2049bb87a0275cb04dbb7cda6636f600699c9",
"zh:ff461571e3f233699bf690db319dfe46aec75e58726636a0d97dd9ac6e32fb70",
]
}
provider "registry.terraform.io/hashicorp/random" {
version = "3.9.0"
constraints = "~> 3.6"
hashes = [
"h1:lVDv+0AjDjrLfpmaJbWqUmIw/k3/AHXLc3N4m55SNdo=",
"zh:161ad0bd9a75768c82f53fb6e7172a9d8be2d4889b012645a34795031aaf1bf1",
"zh:19dc9a5b17729725ccfc4f45b0500af0ee5bc6b6b160c7adb8f2bf617d2c80ea",
"zh:269eda8fe42daa7974d5a34d166c3ba9defe80cde86c01e4dadcfdf2e1f05e5f",
"zh:373f7c65566f8f2cc7f45d698654feb9d988996957e1266a69ca00c52d6d16d0",
"zh:5599d16804c41c83009ec621b6d6b6f74e102f5827678a4750f8809055546b61",
"zh:583be0440469a22bff70dcfa56593b01566860b29607437264adb51060cf46fc",
"zh:5f211d8ec3f2e1f414870d9584bfe26e6995560ef81c748f8447a48164767398",
"zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3",
"zh:7b547fd16216761ef86efc3ed516ac5ac0c5c42b7c7eb24a08cef2d93f69ed5e",
"zh:7e7c0679daf2a382151d05068c8c3f0dae6b7b7dccf818827b73dd08638df2ef",
"zh:8089dec888a8038b9b4fb23b3df7e1057293dbc5b60b42cc47ff690d69d4b61b",
"zh:c51f15a031edfd6f23ce8ced3446ca7f8d8d647e2499890d7d5d10d5016d7257",
"zh:c94784f005708890dc6895afd53636ec00ec1e430b15d41e5aebfb1d4b39bd04",
]
}
+157
View File
@@ -0,0 +1,157 @@
# gitlawb node on AWS (Terraform)
Single EC2 instance running the published node image + Postgres via Docker
compose, with a persistent encrypted EBS volume, Elastic IP, SSM access, and
daily snapshots.
```text
Elastic IP ──► EC2 t4g.small (Amazon Linux 2023, arm64)
7545/tcp docker compose:
7546/udp ├─ node (ghcr.io/gitlawb/node, pulled — not built)
└─ postgres:16-alpine
EBS gp3 volume mounted at /mnt/data
├─ node/ → container /data (repos + identity key)
└─ postgres/ → postgres data dir
```
## Prerequisites
- Terraform ≥ 1.6
- AWS credentials configured (`aws sts get-caller-identity` works)
- AWS CLI + [Session Manager plugin](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html) (for shell access)
- A default VPC in the target region (or pass `subnet_id`)
## Quick start
```sh
cd infra/aws
cp terraform.tfvars.example terraform.tfvars # edit: public_url at minimum
terraform init
terraform plan
terraform apply # ⚠ creates billable resources (~$25/mo: EC2 + EBS + snapshots)
```
After apply (~3-5 min for first boot to pull images and start):
```sh
curl "$(terraform output -raw api_url)/health"
```
## ⚠ First boot: back up the identity key
The node generates `/data/keys/identity.pem` on first start — it defines the
node's DID. **Losing it permanently changes the node's identity.** Back it up
immediately:
```sh
$(terraform output -raw ssm_session_command)
# in the session:
sudo cat /mnt/data/node/keys/identity.pem
```
Store the key somewhere safe (password manager / offline). The volume's
`prevent_destroy` guard and daily DLM snapshots protect against accidents, but
are not a substitute for an offline backup.
## Shell access
SSM Session Manager — no SSH port, no keys to manage:
```sh
$(terraform output -raw ssm_session_command)
```
Bootstrap log: `/var/log/gitlawb-bootstrap.log`. Stack lives in `/opt/gitlawb`
(`docker compose ps`, `docker compose logs node`).
SSH is off by default; set `ssh_ingress_cidr` + `ssh_key_name` if you need it.
## Building and pushing the image (ECR mode)
With `create_ecr_repo = true` the instance pulls from a private ECR repo in
this account instead of an external registry (the instance role gets pull
rights via the amazon-ecr-credential-helper). Build from the repo root —
arm64 to match t4g (native on Apple Silicon):
```sh
ECR_URL=$(terraform -chdir=infra/aws output -raw ecr_repository_url)
aws ecr get-login-password --region us-east-1 | \
docker login --username AWS --password-stdin "${ECR_URL%%/*}"
docker build --platform linux/arm64 -t "$ECR_URL:latest" .
docker push "$ECR_URL:latest"
```
Then run the upgrade command (below) to roll it out.
## Upgrading the node
User-data only runs at first boot, so upgrades go through SSM:
```sh
$(terraform output -raw upgrade_command)
```
This runs `docker compose pull && docker compose up -d` on the instance.
- With `image_tag = "latest"` (default) that picks up the newest release.
- With a **pinned tag**, first edit the tag in `/opt/gitlawb/compose.yaml` on
the instance (via SSM session), then run the upgrade command — and keep
`image_tag` in terraform.tfvars in sync so a future instance replacement
boots the same version.
Replace the instance itself (OS/AMI/instance-type changes) with
`terraform apply -replace=aws_instance.node` — the data volume reattaches and
`/data` (including the identity key) survives.
## Changing configuration
User-data only runs at first boot, and the instance ignores `user_data` drift
(`ignore_changes`), so editing terraform.tfvars values that feed the bootstrap
(`bootstrap_peers`, `public_url`, integrations, `image_tag`) does **not**
affect a running instance on `terraform apply`. To roll out such changes,
either edit `/opt/gitlawb/.env` on the instance (SSM session, then
`docker compose up -d`), or replace the instance:
```sh
terraform apply -replace=aws_instance.node
```
The data volume reattaches; repos, postgres data, and the identity key survive.
## Remote state (optional)
Local state is the default. To move state to S3: create a versioned bucket,
uncomment the `backend "s3"` block in `versions.tf`, then:
```sh
terraform init -migrate-state
```
## Teardown
`terraform destroy` will **fail on the data volume by design**
(`prevent_destroy`). To tear everything down:
1. Back up the identity key (above) and take a final snapshot if you may return.
2. Remove the `prevent_destroy` line from `aws_ebs_volume.data` in `main.tf`.
3. `terraform destroy`.
Note: DLM snapshots created by the policy are not deleted by destroy — clean
them up in the EC2 console if unwanted. The Elastic IP is released on destroy.
## Security notes
- Postgres password: generated by Terraform, stored as an SSM SecureString,
fetched at boot via the instance profile — never in user-data or state-free
files on disk (only in `/opt/gitlawb/.env`, mode 600). It IS in Terraform
state — treat state as sensitive (another reason for the S3 backend).
- Sensitive optional vars (`operator_private_key`, `pinata_jwt`,
`s3_access_key_id`, `s3_secret_access_key`) follow the same SSM path.
- SSM secrets use the AWS-managed `aws/ssm` key by default; set
`ssm_kms_key_id` to encrypt with a customer-managed KMS key instead (the
instance role is granted `kms:Decrypt` on that key automatically).
- IMDSv2 is required; metrics port is closed unless `metrics_ingress_cidr` is set.
- The node itself serves plain HTTP on 7545. Set `domain_name` (with DNS
pointing at the Elastic IP) to run a Caddy sidecar with automatic Let's
Encrypt TLS on 443 (+ http→https redirect on 80) — matching the Fly nodes.
Certs persist on the data volume (`/mnt/data/caddy`).
+82
View File
@@ -0,0 +1,82 @@
# Rendered by Terraform (compose.yaml.tftpl) and written to /opt/gitlawb/compose.yaml
# by user-data. Adapted from the repo-root docker-compose.yml: pulls the published
# image instead of building, and binds data dirs to the dedicated EBS volume.
# `$${VAR}` entries are resolved by docker compose from /opt/gitlawb/.env at runtime.
services:
%{ if domain_name != "" ~}
caddy:
image: caddy:2-alpine
depends_on:
node:
condition: service_healthy
ports:
- "80:80" # redirects to https
- "443:443" # Let's Encrypt cert, auto-renewed; certs persist on the EBS volume
command: caddy reverse-proxy --from ${domain_name} --to node:${gitlawb_port}
volumes:
- /mnt/data/caddy:/data
restart: unless-stopped
%{ endif ~}
%{ if db_host == "" ~}
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: ${pg_db}
POSTGRES_USER: ${pg_user}
POSTGRES_PASSWORD: $${POSTGRES_PASSWORD}
volumes:
- /mnt/data/postgres:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${pg_user}"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
%{ endif ~}
node:
image: ${image_repo}:${image_tag}
%{ if db_host == "" ~}
depends_on:
postgres:
condition: service_healthy
%{ endif ~}
ports:
- "${gitlawb_port}:${gitlawb_port}" # HTTP API + git smart-HTTP
- "${p2p_port}:${p2p_port}/udp" # libp2p QUIC
%{ if expose_metrics ~}
- "${metrics_port}:${metrics_port}" # Prometheus /metrics
%{ endif ~}
volumes:
- /mnt/data/node:/data
environment:
DATABASE_URL: postgresql://${pg_user}:$${POSTGRES_PASSWORD}@${db_host != "" ? db_host : "postgres"}:5432/${pg_db}
GITLAWB_HOST: 0.0.0.0
GITLAWB_PORT: "${gitlawb_port}"
GITLAWB_P2P_PORT: "${p2p_port}"
GITLAWB_REPOS_DIR: /data/repos
GITLAWB_KEY: /data/keys/identity.pem
GITLAWB_PUBLIC_URL: $${GITLAWB_PUBLIC_URL}
GITLAWB_BOOTSTRAP_PEERS: $${GITLAWB_BOOTSTRAP_PEERS}
GITLAWB_AUTO_SYNC: $${GITLAWB_AUTO_SYNC}
GITLAWB_MAX_PACK_BYTES: $${GITLAWB_MAX_PACK_BYTES}
# On-chain PoS (optional — empty unless set in terraform.tfvars)
GITLAWB_CHAIN_RPC_URL: $${GITLAWB_CHAIN_RPC_URL}
GITLAWB_CONTRACT_NODE_STAKING: $${GITLAWB_CONTRACT_NODE_STAKING}
GITLAWB_OPERATOR_PRIVATE_KEY: $${GITLAWB_OPERATOR_PRIVATE_KEY}
# IPFS pinning (optional)
GITLAWB_PINATA_JWT: $${GITLAWB_PINATA_JWT}
# Shared S3-compatible pack storage (optional)
GITLAWB_TIGRIS_BUCKET: $${GITLAWB_TIGRIS_BUCKET}
AWS_ACCESS_KEY_ID: $${S3_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: $${S3_SECRET_ACCESS_KEY}
AWS_ENDPOINT_URL_S3: $${S3_ENDPOINT_URL}
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:${gitlawb_port}/health || exit 1"]
interval: 30s
timeout: 5s
retries: 3
start_period: 30s
restart: unless-stopped
+448
View File
@@ -0,0 +1,448 @@
# Single-EC2 gitlawb node: Docker compose (node + postgres) on Amazon Linux 2023,
# persistent EBS data volume, Elastic IP, SSM access, daily snapshots.
# ---------------------------------------------------------------------------
# Network placement (default VPC unless subnet_id is set)
# ---------------------------------------------------------------------------
data "aws_vpc" "default" {
default = true
}
data "aws_subnets" "default" {
filter {
name = "vpc-id"
values = [data.aws_vpc.default.id]
}
}
locals {
subnet_id = coalesce(var.subnet_id, sort(data.aws_subnets.default.ids)[0])
expose_metrics = var.metrics_ingress_cidr != null
common_tags = merge({ Project = "gitlawb-node", ManagedBy = "terraform" }, var.tags)
}
data "aws_subnet" "selected" {
id = local.subnet_id
}
# Latest Amazon Linux 2023 arm64 AMI (pairs with t4g/Graviton; node image is
# multi-arch). ignore_changes on the instance AMI avoids churn on AL releases.
data "aws_ssm_parameter" "al2023_arm64" {
name = "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64"
}
# ---------------------------------------------------------------------------
# Secrets (SSM Parameter Store — never baked into user-data)
# ---------------------------------------------------------------------------
resource "random_password" "postgres" {
length = 32
special = false
}
resource "aws_ssm_parameter" "postgres_password" {
name = "/${var.name_prefix}/postgres_password"
type = "SecureString"
key_id = var.ssm_kms_key_id
value = random_password.postgres.result
tags = local.common_tags
}
resource "aws_ssm_parameter" "operator_key" {
count = var.operator_private_key != "" ? 1 : 0
name = "/${var.name_prefix}/operator_private_key"
type = "SecureString"
key_id = var.ssm_kms_key_id
value = var.operator_private_key
tags = local.common_tags
}
resource "aws_ssm_parameter" "pinata_jwt" {
count = var.pinata_jwt != "" ? 1 : 0
name = "/${var.name_prefix}/pinata_jwt"
type = "SecureString"
key_id = var.ssm_kms_key_id
value = var.pinata_jwt
tags = local.common_tags
}
resource "aws_ssm_parameter" "s3_secret" {
count = var.s3_secret_access_key != "" ? 1 : 0
name = "/${var.name_prefix}/s3_secret_access_key"
type = "SecureString"
key_id = var.ssm_kms_key_id
value = var.s3_secret_access_key
tags = local.common_tags
}
resource "aws_ssm_parameter" "s3_access_key" {
count = var.s3_access_key_id != "" ? 1 : 0
name = "/${var.name_prefix}/s3_access_key_id"
type = "SecureString"
key_id = var.ssm_kms_key_id
value = var.s3_access_key_id
tags = local.common_tags
}
locals {
secret_param_arns = concat(
[aws_ssm_parameter.postgres_password.arn],
aws_ssm_parameter.operator_key[*].arn,
aws_ssm_parameter.pinata_jwt[*].arn,
aws_ssm_parameter.s3_secret[*].arn,
aws_ssm_parameter.s3_access_key[*].arn,
)
}
# ---------------------------------------------------------------------------
# IAM: SSM Session Manager access + least-privilege read of our parameters.
# The AWS-managed aws/ssm KMS key needs no explicit kms:Decrypt grant; a
# customer-managed key (ssm_kms_key_id) gets one scoped to that key.
# ---------------------------------------------------------------------------
data "aws_kms_key" "ssm" {
count = var.ssm_kms_key_id != null ? 1 : 0
key_id = var.ssm_kms_key_id
}
resource "aws_iam_role" "node" {
name = "${var.name_prefix}-instance"
tags = local.common_tags
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "ec2.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
}
resource "aws_iam_role_policy_attachment" "ssm_core" {
role = aws_iam_role.node.name
policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}
resource "aws_iam_role_policy" "ssm_params_read" {
name = "read-gitlawb-secrets"
role = aws_iam_role.node.id
policy = jsonencode({
Version = "2012-10-17"
Statement = concat(
[{
Effect = "Allow"
Action = ["ssm:GetParameter"]
Resource = local.secret_param_arns
}],
var.ssm_kms_key_id != null ? [{
Effect = "Allow"
Action = ["kms:Decrypt"]
Resource = [data.aws_kms_key.ssm[0].arn]
# Only via Parameter Store, and only for this stack's parameters —
# not arbitrary ciphertext encrypted under the same CMK.
Condition = {
StringEquals = {
"kms:ViaService" = "ssm.${var.region}.amazonaws.com"
"kms:EncryptionContext:PARAMETER_ARN" = local.secret_param_arns
}
}
}] : []
)
})
}
resource "aws_iam_instance_profile" "node" {
name = "${var.name_prefix}-instance"
role = aws_iam_role.node.name
}
# ---------------------------------------------------------------------------
# Security group
# ---------------------------------------------------------------------------
resource "aws_security_group" "node" {
name = "${var.name_prefix}-sg"
description = "gitlawb node: HTTP API + libp2p UDP"
vpc_id = data.aws_subnet.selected.vpc_id # follows subnet_id overrides into non-default VPCs
tags = local.common_tags
ingress {
description = "HTTP API + git smart-HTTP"
from_port = var.gitlawb_port
to_port = var.gitlawb_port
protocol = "tcp"
cidr_blocks = [var.api_ingress_cidr]
}
ingress {
description = "libp2p QUIC"
from_port = var.gitlawb_p2p_port
to_port = var.gitlawb_p2p_port
protocol = "udp"
cidr_blocks = [var.p2p_ingress_cidr]
}
dynamic "ingress" {
for_each = var.domain_name != "" ? [80, 443] : []
content {
description = "Caddy HTTP/HTTPS (TLS for ${var.domain_name})"
from_port = ingress.value
to_port = ingress.value
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}
dynamic "ingress" {
for_each = local.expose_metrics ? [1] : []
content {
description = "Prometheus metrics"
from_port = var.metrics_port
to_port = var.metrics_port
protocol = "tcp"
cidr_blocks = [var.metrics_ingress_cidr]
}
}
dynamic "ingress" {
for_each = var.ssh_ingress_cidr != null ? [1] : []
content {
description = "SSH (prefer SSM Session Manager)"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [var.ssh_ingress_cidr]
}
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
# ---------------------------------------------------------------------------
# Persistent data volume — survives instance replacement. prevent_destroy
# guards repos, postgres data, and the node identity key (/data/keys).
# ---------------------------------------------------------------------------
resource "aws_ebs_volume" "data" {
availability_zone = data.aws_subnet.selected.availability_zone
size = var.data_volume_size_gb
type = var.data_volume_type
encrypted = true
tags = merge(local.common_tags, {
Name = "${var.name_prefix}-data"
Snapshot = "true" # targeted by the DLM snapshot policy
})
lifecycle {
prevent_destroy = true
}
}
# ---------------------------------------------------------------------------
# Instance + Elastic IP
# ---------------------------------------------------------------------------
resource "aws_eip" "node" {
domain = "vpc"
tags = merge(local.common_tags, { Name = var.name_prefix })
}
# ---------------------------------------------------------------------------
# Optional private ECR repository — self-contained image hosting in this
# account instead of an external registry. Push with: see README "Building
# and pushing the image".
# ---------------------------------------------------------------------------
resource "aws_ecr_repository" "node" {
count = var.create_ecr_repo ? 1 : 0
name = var.name_prefix
tags = local.common_tags
image_scanning_configuration {
scan_on_push = true
}
}
resource "aws_iam_role_policy_attachment" "ecr_read" {
count = var.create_ecr_repo ? 1 : 0
role = aws_iam_role.node.name
policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
}
locals {
image_repo = var.create_ecr_repo ? aws_ecr_repository.node[0].repository_url : var.image_repo
ecr_registry = var.create_ecr_repo ? split("/", aws_ecr_repository.node[0].repository_url)[0] : ""
}
locals {
public_url = (
var.public_url != "" ? var.public_url :
var.domain_name != "" ? "https://${var.domain_name}" :
"http://${aws_eip.node.public_ip}:${var.gitlawb_port}"
)
compose_yaml = templatefile("${path.module}/compose.yaml.tftpl", {
image_repo = local.image_repo
image_tag = var.image_tag
domain_name = var.domain_name
db_host = var.use_rds ? aws_db_instance.node[0].address : ""
gitlawb_port = var.gitlawb_port
p2p_port = var.gitlawb_p2p_port
metrics_port = var.metrics_port
expose_metrics = local.expose_metrics
pg_user = var.postgres_user
pg_db = var.postgres_db
})
user_data = templatefile("${path.module}/user-data.sh.tftpl", {
region = var.region
pg_password_param = aws_ssm_parameter.postgres_password.name
operator_key_param = try(aws_ssm_parameter.operator_key[0].name, "")
pinata_jwt_param = try(aws_ssm_parameter.pinata_jwt[0].name, "")
s3_secret_param = try(aws_ssm_parameter.s3_secret[0].name, "")
s3_access_key_param = try(aws_ssm_parameter.s3_access_key[0].name, "")
public_url = local.public_url
bootstrap_peers = var.bootstrap_peers
auto_sync = var.auto_sync
max_pack_bytes = var.max_pack_bytes
chain_rpc_url = var.chain_rpc_url
contract_node_staking = var.contract_node_staking
tigris_bucket = var.tigris_bucket
s3_endpoint_url = var.s3_endpoint_url
ecr_registry = local.ecr_registry
compose_yaml = local.compose_yaml
})
}
resource "aws_instance" "node" {
ami = nonsensitive(data.aws_ssm_parameter.al2023_arm64.value)
instance_type = var.instance_type
subnet_id = local.subnet_id
vpc_security_group_ids = [aws_security_group.node.id]
iam_instance_profile = aws_iam_instance_profile.node.name
key_name = var.ssh_key_name
user_data = local.user_data
tags = merge(local.common_tags, { Name = var.name_prefix })
metadata_options {
http_endpoint = "enabled"
http_tokens = "required" # IMDSv2 only
}
root_block_device {
volume_size = 8
volume_type = "gp3"
encrypted = true
}
lifecycle {
# ami: new AL2023 releases shouldn't churn the instance; replace
# deliberately for OS upgrades.
# user_data: only runs at first boot, so re-rendering it on a live
# instance is a pointless stop/start. Config changes that feed user-data
# (bootstrap peers, integrations, image tag) require a deliberate
# `terraform apply -replace=aws_instance.node` — see README "Changing
# configuration".
ignore_changes = [ami, user_data]
}
}
resource "aws_volume_attachment" "data" {
device_name = "/dev/sdf" # presented as /dev/nvme1n1 on Nitro; user-data discovers it
volume_id = aws_ebs_volume.data.id
instance_id = aws_instance.node.id
}
resource "aws_eip_association" "node" {
instance_id = aws_instance.node.id
allocation_id = aws_eip.node.id
}
# ---------------------------------------------------------------------------
# Daily EBS snapshots of the data volume (DLM)
# ---------------------------------------------------------------------------
resource "aws_iam_role" "dlm" {
name = "${var.name_prefix}-dlm"
tags = local.common_tags
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "dlm.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
}
resource "aws_iam_role_policy_attachment" "dlm" {
role = aws_iam_role.dlm.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSDataLifecycleManagerServiceRole"
}
resource "aws_dlm_lifecycle_policy" "data" {
description = "Daily snapshots of the gitlawb data volume"
execution_role_arn = aws_iam_role.dlm.arn
state = "ENABLED"
tags = local.common_tags
policy_details {
resource_types = ["VOLUME"]
# Name makes the target stack-specific — a bare Snapshot=true would also
# match unrelated tagged volumes in a shared account.
target_tags = {
Snapshot = "true"
Name = "${var.name_prefix}-data"
}
schedule {
name = "daily"
create_rule {
interval = 24
interval_unit = "HOURS"
times = ["05:00"]
}
retain_rule {
count = var.snapshot_retain_count
}
copy_tags = true
}
}
}
# ---------------------------------------------------------------------------
# Upgrade runbook as code: pull + restart the compose stack via SSM
# ---------------------------------------------------------------------------
resource "aws_ssm_document" "upgrade" {
name = "${var.name_prefix}-upgrade"
document_type = "Command"
document_format = "JSON"
tags = local.common_tags
content = jsonencode({
schemaVersion = "2.2"
description = "Pull the latest gitlawb node image and restart the compose stack"
mainSteps = [{
action = "aws:runShellScript"
name = "upgrade"
inputs = {
runCommand = [
"cd /opt/gitlawb && docker compose pull && docker compose up -d --remove-orphans && docker image prune -f"
]
}
}]
})
}
+105
View File
@@ -0,0 +1,105 @@
# Alerting (enabled when alert_email is set): SNS email + CloudWatch alarms
# on external API health (Route 53 health check), EC2 status, CPU, and RDS
# free storage. The email subscription must be confirmed once via the link
# SNS sends.
locals {
alarms_enabled = var.alert_email != ""
}
resource "aws_sns_topic" "alerts" {
count = local.alarms_enabled ? 1 : 0
name = "${var.name_prefix}-alerts"
tags = local.common_tags
}
resource "aws_sns_topic_subscription" "email" {
count = local.alarms_enabled ? 1 : 0
topic_arn = aws_sns_topic.alerts[0].arn
protocol = "email"
endpoint = var.alert_email
}
# External HTTPS /health probe — catches everything the EC2-level checks
# can't (container down, caddy broken, cert expired, DNS wrong).
resource "aws_route53_health_check" "api" {
count = local.alarms_enabled && var.domain_name != "" ? 1 : 0
fqdn = var.domain_name
port = 443
type = "HTTPS"
resource_path = "/health"
request_interval = 30
failure_threshold = 3
tags = merge(local.common_tags, { Name = "${var.name_prefix}-api" })
}
# Route 53 health check metrics are only published in us-east-1; this module
# already runs there by default.
resource "aws_cloudwatch_metric_alarm" "api_health" {
count = local.alarms_enabled && var.domain_name != "" ? 1 : 0
alarm_name = "${var.name_prefix}-api-health"
alarm_description = "https://${var.domain_name}/health is failing"
namespace = "AWS/Route53"
metric_name = "HealthCheckStatus"
dimensions = { HealthCheckId = aws_route53_health_check.api[0].id }
statistic = "Minimum"
period = 60
evaluation_periods = 2
threshold = 1
comparison_operator = "LessThanThreshold"
treat_missing_data = "breaching"
alarm_actions = [aws_sns_topic.alerts[0].arn]
ok_actions = [aws_sns_topic.alerts[0].arn]
tags = local.common_tags
}
resource "aws_cloudwatch_metric_alarm" "status_check" {
count = local.alarms_enabled ? 1 : 0
alarm_name = "${var.name_prefix}-status-check"
alarm_description = "EC2 status checks failing on the node instance"
namespace = "AWS/EC2"
metric_name = "StatusCheckFailed"
dimensions = { InstanceId = aws_instance.node.id }
statistic = "Maximum"
period = 60
evaluation_periods = 2
threshold = 1
comparison_operator = "GreaterThanOrEqualToThreshold"
alarm_actions = [aws_sns_topic.alerts[0].arn]
ok_actions = [aws_sns_topic.alerts[0].arn]
tags = local.common_tags
}
resource "aws_cloudwatch_metric_alarm" "cpu_high" {
count = local.alarms_enabled ? 1 : 0
alarm_name = "${var.name_prefix}-cpu-high"
alarm_description = "Node instance CPU above 80% for 15 minutes"
namespace = "AWS/EC2"
metric_name = "CPUUtilization"
dimensions = { InstanceId = aws_instance.node.id }
statistic = "Average"
period = 300
evaluation_periods = 3
threshold = 80
comparison_operator = "GreaterThanThreshold"
alarm_actions = [aws_sns_topic.alerts[0].arn]
ok_actions = [aws_sns_topic.alerts[0].arn]
tags = local.common_tags
}
resource "aws_cloudwatch_metric_alarm" "rds_storage_low" {
count = local.alarms_enabled && var.use_rds ? 1 : 0
alarm_name = "${var.name_prefix}-rds-storage-low"
alarm_description = "RDS free storage below 2 GB"
namespace = "AWS/RDS"
metric_name = "FreeStorageSpace"
dimensions = { DBInstanceIdentifier = aws_db_instance.node[0].identifier }
statistic = "Minimum"
period = 300
evaluation_periods = 2
threshold = 2147483648
comparison_operator = "LessThanThreshold"
alarm_actions = [aws_sns_topic.alerts[0].arn]
ok_actions = [aws_sns_topic.alerts[0].arn]
tags = local.common_tags
}
+44
View File
@@ -0,0 +1,44 @@
output "elastic_ip" {
description = "Public IP of the node"
value = aws_eip.node.public_ip
}
output "api_url" {
description = "HTTP API endpoint (honors the public_url override)"
value = local.public_url
}
output "instance_id" {
description = "EC2 instance ID"
value = aws_instance.node.id
}
output "data_volume_id" {
description = "Persistent EBS data volume (protected by prevent_destroy)"
value = aws_ebs_volume.data.id
}
output "ecr_repository_url" {
description = "Private ECR repo for the node image (null unless create_ecr_repo)"
value = try(aws_ecr_repository.node[0].repository_url, null)
}
output "rds_endpoint" {
description = "RDS Postgres endpoint (null unless use_rds)"
value = try(aws_db_instance.node[0].address, null)
}
output "postgres_password_ssm_param" {
description = "SSM parameter holding the postgres password (value not shown)"
value = aws_ssm_parameter.postgres_password.name
}
output "ssm_session_command" {
description = "Open a shell on the instance"
value = "aws ssm start-session --target ${aws_instance.node.id} --region ${var.region}"
}
output "upgrade_command" {
description = "Pull the latest node image and restart the stack"
value = "aws ssm send-command --document-name ${aws_ssm_document.upgrade.name} --targets Key=InstanceIds,Values=${aws_instance.node.id} --region ${var.region}"
}
+58
View File
@@ -0,0 +1,58 @@
# Optional managed Postgres (use_rds = true): the database survives instance
# loss independently, with automated backups / point-in-time recovery. The
# in-Docker postgres service is dropped from the compose stack when enabled.
data "aws_subnets" "node_vpc" {
filter {
name = "vpc-id"
values = [data.aws_subnet.selected.vpc_id]
}
}
resource "aws_db_subnet_group" "node" {
count = var.use_rds ? 1 : 0
name = "${var.name_prefix}-db"
subnet_ids = data.aws_subnets.node_vpc.ids
tags = local.common_tags
}
resource "aws_security_group" "rds" {
count = var.use_rds ? 1 : 0
name = "${var.name_prefix}-rds"
description = "Postgres, reachable only from the node instance"
vpc_id = data.aws_subnet.selected.vpc_id
tags = local.common_tags
ingress {
description = "Postgres from the node"
from_port = 5432
to_port = 5432
protocol = "tcp"
security_groups = [aws_security_group.node.id]
}
}
resource "aws_db_instance" "node" {
count = var.use_rds ? 1 : 0
identifier = "${var.name_prefix}-db"
engine = "postgres"
engine_version = "16"
instance_class = var.rds_instance_class
allocated_storage = var.rds_allocated_storage_gb
storage_type = "gp3"
storage_encrypted = true
db_name = var.postgres_db
username = var.postgres_user
password = random_password.postgres.result
db_subnet_group_name = aws_db_subnet_group.node[0].name
vpc_security_group_ids = [aws_security_group.rds[0].id]
backup_retention_period = var.rds_backup_retention_days
deletion_protection = true
skip_final_snapshot = false
final_snapshot_identifier = "${var.name_prefix}-db-final"
apply_immediately = true
tags = local.common_tags
}
+53
View File
@@ -0,0 +1,53 @@
# Copy to terraform.tfvars and adjust. terraform.tfvars is gitignored.
region = "us-east-1"
name_prefix = "gitlawb-node"
instance_type = "t4g.small" # ARM/Graviton; node image is multi-arch
# --- Image -----------------------------------------------------------------
# image_repo = "ghcr.io/gitlawb/node"
# image_tag = "latest" # pin a version for production, e.g. "0.9.2"
# Or host the image in a private ECR repo in this account (image_repo is then
# ignored; build & push per README "Building and pushing the image"):
# create_ecr_repo = true
# --- Storage ---------------------------------------------------------------
# data_volume_size_gb = 20
# snapshot_retain_count = 7
# --- Node configuration (defaults mirror infra/fly/fly.toml) ---------------
# DNS name pointing at the Elastic IP: enables a Caddy sidecar with automatic
# HTTPS (Let's Encrypt) on 80/443, and public_url becomes https://<domain>
# domain_name = "node.example.com"
# Explicit override; rarely needed when domain_name is set.
# Defaults to https://<domain_name>, else http://<elastic-ip>:7545
# public_url = "https://node.example.com"
# bootstrap_peers = "https://node.gitlawb.com,https://node2.gitlawb.com,https://node3.gitlawb.com"
# auto_sync = "true"
# max_pack_bytes = "524288000"
# --- Ingress ---------------------------------------------------------------
# api_ingress_cidr = "0.0.0.0/0" # public node
# p2p_ingress_cidr = "0.0.0.0/0"
# metrics_ingress_cidr = "203.0.113.10/32" # your Prometheus scraper; null = closed
# ssh_ingress_cidr = null # keep null — use SSM Session Manager
# ssh_key_name = null
# --- Optional: on-chain PoS operator ----------------------------------------
# chain_rpc_url = "https://mainnet.base.org"
# contract_node_staking = "0x..."
# operator_private_key = "0x..." # stored in SSM SecureString, not user-data
# --- Optional: IPFS / shared storage ----------------------------------------
# pinata_jwt = "..."
# tigris_bucket = "my-bucket"
# s3_access_key_id = "..." # stored in SSM SecureString, not user-data
# s3_secret_access_key = "..." # stored in SSM SecureString, not user-data
# s3_endpoint_url = "https://fly.storage.tigris.dev"
# Encrypt SSM secrets with a customer-managed KMS key instead of aws/ssm
# ssm_kms_key_id = "alias/my-key"
# tags = { Environment = "production" }
+116
View File
@@ -0,0 +1,116 @@
#!/bin/bash
# Rendered by Terraform (user-data.sh.tftpl). Runs once at first boot via cloud-init.
set -euxo pipefail
exec > >(tee -a /var/log/gitlawb-bootstrap.log) 2>&1
COMPOSE_VERSION="v2.32.4"
# ---------------------------------------------------------------------------
# Docker + compose plugin (Amazon Linux 2023)
# ---------------------------------------------------------------------------
dnf install -y docker
systemctl enable --now docker
mkdir -p /usr/local/lib/docker/cli-plugins
curl -fsSL --retry 5 --retry-delay 3 --retry-all-errors \
--connect-timeout 10 --max-time 300 \
"https://github.com/docker/compose/releases/download/$COMPOSE_VERSION/docker-compose-linux-$(uname -m)" \
-o /usr/local/lib/docker/cli-plugins/docker-compose
chmod +x /usr/local/lib/docker/cli-plugins/docker-compose
%{ if ecr_registry != "" ~}
# Pull from the account's private ECR via the instance role — the credential
# helper avoids expiring `docker login` tokens (works for upgrades too).
dnf install -y amazon-ecr-credential-helper
mkdir -p /root/.docker
cat > /root/.docker/config.json <<'DOCKER_EOF'
{ "credHelpers": { "${ecr_registry}": "ecr-login" } }
DOCKER_EOF
%{ endif ~}
# ---------------------------------------------------------------------------
# Data volume: find the attached EBS volume (Nitro presents /dev/sdf as NVMe;
# attachment can lag first boot, so retry). The data disk is the one that is
# not the root disk (no mounted partitions).
# ---------------------------------------------------------------------------
DATA_DEV=""
for i in $(seq 1 60); do
for dev in $(lsblk -dpno NAME,TYPE | awk '$2 == "disk" { print $1 }'); do
if ! lsblk -no MOUNTPOINTS "$dev" | grep -q .; then
DATA_DEV="$dev"
break
fi
done
[ -n "$DATA_DEV" ] && break
sleep 5
done
if [ -z "$DATA_DEV" ]; then
echo "FATAL: data EBS volume never attached" >&2
exit 1
fi
# Format ONLY on first boot (no existing filesystem). Never reformat on
# reattach — this is what preserves repos + the node identity key.
if ! blkid "$DATA_DEV"; then
mkfs.ext4 -L gitlawb-data "$DATA_DEV"
fi
mkdir -p /mnt/data
grep -q 'LABEL=gitlawb-data' /etc/fstab || \
echo 'LABEL=gitlawb-data /mnt/data ext4 defaults,nofail 0 2' >> /etc/fstab
mount -a
# Grow the filesystem if the EBS volume was enlarged (no-op otherwise)
resize2fs "$DATA_DEV" || true
# Node container runs as uid/gid 1000 (gitlawb). Postgres manages its own dir.
# caddy/ holds TLS certs (only used when a domain is configured).
mkdir -p /mnt/data/node/repos /mnt/data/node/keys /mnt/data/postgres /mnt/data/caddy
chown -R 1000:1000 /mnt/data/node
# ---------------------------------------------------------------------------
# Secrets from SSM Parameter Store (instance profile grants read on these only)
# xtrace is disabled through the .env write so secret values never hit the log.
# ---------------------------------------------------------------------------
set +x
fetch_param() {
if [ -z "$1" ]; then echo ""; return; fi
aws ssm get-parameter --name "$1" --with-decryption \
--region '${region}' --query Parameter.Value --output text
}
PGPASS="$(fetch_param '${pg_password_param}')"
OPERATOR_KEY="$(fetch_param '${operator_key_param}')"
PINATA_JWT="$(fetch_param '${pinata_jwt_param}')"
S3_SECRET="$(fetch_param '${s3_secret_param}')"
S3_KEY_ID="$(fetch_param '${s3_access_key_param}')"
# ---------------------------------------------------------------------------
# Compose stack
# ---------------------------------------------------------------------------
mkdir -p /opt/gitlawb
cat > /opt/gitlawb/.env <<ENV_EOF
POSTGRES_PASSWORD=$PGPASS
GITLAWB_PUBLIC_URL=${public_url}
GITLAWB_BOOTSTRAP_PEERS=${bootstrap_peers}
GITLAWB_AUTO_SYNC=${auto_sync}
GITLAWB_MAX_PACK_BYTES=${max_pack_bytes}
GITLAWB_CHAIN_RPC_URL=${chain_rpc_url}
GITLAWB_CONTRACT_NODE_STAKING=${contract_node_staking}
GITLAWB_OPERATOR_PRIVATE_KEY=$OPERATOR_KEY
GITLAWB_PINATA_JWT=$PINATA_JWT
GITLAWB_TIGRIS_BUCKET=${tigris_bucket}
S3_ACCESS_KEY_ID=$S3_KEY_ID
S3_SECRET_ACCESS_KEY=$S3_SECRET
S3_ENDPOINT_URL=${s3_endpoint_url}
ENV_EOF
chmod 600 /opt/gitlawb/.env
set -x
# Quoted heredoc: $${VAR} placeholders inside stay literal for docker compose.
cat > /opt/gitlawb/compose.yaml <<'COMPOSE_EOF'
${compose_yaml}
COMPOSE_EOF
cd /opt/gitlawb
docker compose pull
docker compose up -d --remove-orphans
echo "gitlawb-node bootstrap complete"
+276
View File
@@ -0,0 +1,276 @@
# ---------------------------------------------------------------------------
# General
# ---------------------------------------------------------------------------
variable "region" {
description = "AWS region to deploy into"
type = string
default = "us-east-1"
}
variable "name_prefix" {
description = "Prefix for resource names and tags"
type = string
default = "gitlawb-node"
}
variable "tags" {
description = "Extra tags applied to all resources"
type = map(string)
default = {}
}
variable "subnet_id" {
description = "Subnet to launch into. Defaults to the first subnet of the default VPC."
type = string
default = null
}
# ---------------------------------------------------------------------------
# Compute & storage
# ---------------------------------------------------------------------------
variable "instance_type" {
description = "EC2 instance type (ARM/Graviton — the node image is multi-arch)"
type = string
default = "t4g.small"
}
variable "data_volume_size_gb" {
description = "Size of the persistent /data EBS volume in GB (repos, identity key, postgres)"
type = number
default = 20
}
variable "data_volume_type" {
description = "EBS volume type for the data volume"
type = string
default = "gp3"
}
variable "snapshot_retain_count" {
description = "How many daily EBS snapshots of the data volume to retain"
type = number
default = 7
validation {
condition = var.snapshot_retain_count >= 1 && var.snapshot_retain_count <= 1000
error_message = "snapshot_retain_count must be between 1 and 1000 (DLM retain rule limits)."
}
}
# ---------------------------------------------------------------------------
# Node image
# ---------------------------------------------------------------------------
variable "image_repo" {
description = "Container image repository for the node. Ignored when create_ecr_repo is true (the module-managed ECR repo is used instead)."
type = string
default = "ghcr.io/gitlawb/node"
}
variable "create_ecr_repo" {
description = "Create a private ECR repository in this account and pull the node image from it (instance role gets pull rights via a credential helper). Push images with scripts in the README."
type = bool
default = false
}
variable "image_tag" {
description = "Image tag to run. After changing, run the upgrade SSM command (see outputs) — user-data only runs at first boot."
type = string
default = "latest"
}
# ---------------------------------------------------------------------------
# Networking / ingress
# ---------------------------------------------------------------------------
variable "gitlawb_port" {
description = "HTTP API port"
type = number
default = 7545
}
variable "gitlawb_p2p_port" {
description = "libp2p UDP port"
type = number
default = 7546
}
variable "metrics_port" {
description = "Prometheus metrics port"
type = number
default = 9091
}
variable "api_ingress_cidr" {
description = "CIDR allowed to reach the HTTP API (public node by default)"
type = string
default = "0.0.0.0/0"
}
variable "p2p_ingress_cidr" {
description = "CIDR allowed to reach the p2p UDP port"
type = string
default = "0.0.0.0/0"
}
variable "metrics_ingress_cidr" {
description = "CIDR allowed to scrape /metrics. null = metrics port not exposed."
type = string
default = null
}
variable "ssh_ingress_cidr" {
description = "CIDR allowed to SSH. null = no SSH (use SSM Session Manager)."
type = string
default = null
}
variable "ssh_key_name" {
description = "Existing EC2 key pair name for SSH. Only used if ssh_ingress_cidr is set."
type = string
default = null
}
# ---------------------------------------------------------------------------
# Node configuration (defaults mirror infra/fly/fly.toml)
# ---------------------------------------------------------------------------
variable "domain_name" {
description = "DNS name pointing at the Elastic IP. When set, a Caddy sidecar terminates TLS on 80/443 with an auto-renewed Let's Encrypt cert, and public_url defaults to https://<domain_name>."
type = string
default = ""
}
variable "public_url" {
description = "Public URL of this node (GITLAWB_PUBLIC_URL). Leave empty to default to http://<elastic-ip>:<port> after apply — set a real DNS name for production."
type = string
default = ""
}
variable "bootstrap_peers" {
description = "Comma-separated bootstrap peer URLs (GITLAWB_BOOTSTRAP_PEERS)"
type = string
default = "https://node.gitlawb.com,https://node2.gitlawb.com,https://node3.gitlawb.com"
}
variable "auto_sync" {
description = "GITLAWB_AUTO_SYNC"
type = string
default = "true"
}
variable "max_pack_bytes" {
description = "GITLAWB_MAX_PACK_BYTES (500MB, matching the Fly deployment)"
type = string
default = "524288000"
}
variable "ssm_kms_key_id" {
description = "Customer-managed KMS key (ID, alias, or ARN) for encrypting the SSM SecureString secrets. null = AWS-managed aws/ssm key."
type = string
default = null
}
# ---------------------------------------------------------------------------
# Postgres
# ---------------------------------------------------------------------------
variable "use_rds" {
description = "Run Postgres in RDS instead of a container on the instance. The DB then survives instance loss with automated backups."
type = bool
default = false
}
variable "rds_instance_class" {
description = "RDS instance class (only used when use_rds)"
type = string
default = "db.t4g.micro"
}
variable "rds_allocated_storage_gb" {
description = "RDS storage in GB (only used when use_rds)"
type = number
default = 20
}
variable "rds_backup_retention_days" {
description = "RDS automated backup retention (only used when use_rds)"
type = number
default = 7
}
variable "alert_email" {
description = "Email for CloudWatch alarm notifications via SNS (confirm the subscription email once). Empty = no alarms."
type = string
default = ""
}
variable "postgres_user" {
description = "Postgres user for the node database"
type = string
default = "gitlawb"
}
variable "postgres_db" {
description = "Postgres database name"
type = string
default = "gitlawb"
}
# ---------------------------------------------------------------------------
# Optional integrations (all off by default; see .env.example at repo root)
# ---------------------------------------------------------------------------
variable "chain_rpc_url" {
description = "Base L2 RPC URL (optional, PoS operator)"
type = string
default = ""
}
variable "contract_node_staking" {
description = "Node staking contract address (optional)"
type = string
default = ""
}
variable "operator_private_key" {
description = "PoS operator private key (optional). Stored in SSM, not in user-data."
type = string
default = ""
sensitive = true
}
variable "pinata_jwt" {
description = "Pinata JWT for IPFS pinning (optional). Stored in SSM, not in user-data."
type = string
default = ""
sensitive = true
}
variable "tigris_bucket" {
description = "S3-compatible bucket for pack storage (optional, GITLAWB_TIGRIS_BUCKET)"
type = string
default = ""
}
variable "s3_access_key_id" {
description = "Access key for the S3-compatible bucket (optional)"
type = string
default = ""
sensitive = true
}
variable "s3_secret_access_key" {
description = "Secret key for the S3-compatible bucket (optional). Stored in SSM, not in user-data."
type = string
default = ""
sensitive = true
}
variable "s3_endpoint_url" {
description = "Custom S3 endpoint URL (optional, AWS_ENDPOINT_URL_S3)"
type = string
default = ""
}
+25
View File
@@ -0,0 +1,25 @@
terraform {
required_version = ">= 1.6"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
random = {
source = "hashicorp/random"
version = "~> 3.6"
}
}
backend "s3" {
bucket = "gitlawb-tfstate-874373491455"
key = "infra/aws/terraform.tfstate"
region = "us-east-1"
use_lockfile = true
}
}
provider "aws" {
region = var.region
}
+7 -3
View File
@@ -1,8 +1,10 @@
# NOTE: this config targets the shared TEST instance. To deploy your own
# node, change `app` (and the volume/URL below) or use `fly deploy -a <app>`.
app = "gitlawb-node-test"
primary_region = "iad"
[build]
dockerfile = "Dockerfile"
dockerfile = "../../Dockerfile"
[env]
GITLAWB_HOST = "0.0.0.0"
@@ -28,10 +30,12 @@ primary_region = "iad"
soft_limit = 400
[http_service.http_options]
idle_timeout = 600
# 120s, not 600: long idle windows let hung clients pin Fly's connection
# slots (hard_limit) — root factor in the 2026-06-12 prod outage
idle_timeout = 120
[[services]]
protocol = "tcp"
protocol = "udp"
internal_port = 7546
[[services.ports]]
+97 -45
View File
@@ -1,14 +1,50 @@
#!/usr/bin/env bash
# gitlawb installer — downloads pre-built binaries from GitHub Releases
# Usage: curl -sSf https://gitlawb.com/install.sh | sh
# curl -sSf https://gitlawb.com/install.sh | sh -s -- --version v0.1.2
# gitlawb installer - downloads pre-built binaries from GitHub Releases.
#
# Usage:
# curl -fsSL https://gitlawb.com/install.sh | sh
# curl -fsSL https://gitlawb.com/install.sh | sh -s -- --version v0.3.9
set -euo pipefail
REPO="gitlawb/releases"
REPO="${GITLAWB_RELEASE_REPO:-Gitlawb/node}"
INSTALL_DIR="${GITLAWB_INSTALL_DIR:-$HOME/.local/bin}"
VERSION="${1:-latest}" # pass --version vX.Y.Z or leave empty for latest
VERSION_ARG="latest"
usage() {
cat <<EOF
gitlawb installer
Options:
--version <tag> Install a specific release tag, for example v0.3.9.
-h, --help Show this help.
Environment:
GITLAWB_INSTALL_DIR Install directory (default: $HOME/.local/bin)
GITLAWB_RELEASE_REPO GitHub repo to download from (default: Gitlawb/node)
EOF
}
while [ "$#" -gt 0 ]; do
case "$1" in
--version)
if [ "$#" -lt 2 ]; then
echo "error: --version requires a value" >&2
exit 1
fi
VERSION_ARG="$2"
shift 2
;;
-h|--help)
usage
exit 0
;;
*)
VERSION_ARG="$1"
shift
;;
esac
done
# ── Detect OS and arch ────────────────────────────────────────────────────
OS=$(uname -s)
ARCH=$(uname -m)
@@ -16,8 +52,8 @@ case "$OS" in
Linux) OS_NAME="linux" ;;
Darwin) OS_NAME="darwin" ;;
*)
echo "error: unsupported OS: $OS"
echo " please build from source: cargo install --git https://github.com/$REPO gl"
echo "error: unsupported OS: $OS" >&2
echo " please build from source: cargo install --git https://github.com/${REPO} gl" >&2
exit 1
;;
esac
@@ -26,12 +62,11 @@ case "$ARCH" in
x86_64) ARCH_NAME="x86_64" ;;
aarch64 | arm64) ARCH_NAME="aarch64" ;;
*)
echo "error: unsupported architecture: $ARCH"
echo "error: unsupported architecture: $ARCH" >&2
exit 1
;;
esac
# Rust target triple
case "${OS_NAME}-${ARCH_NAME}" in
linux-x86_64) TARGET="x86_64-unknown-linux-musl" ;;
linux-aarch64) TARGET="aarch64-unknown-linux-musl" ;;
@@ -39,86 +74,103 @@ case "${OS_NAME}-${ARCH_NAME}" in
darwin-aarch64) TARGET="aarch64-apple-darwin" ;;
esac
# ── Resolve version ───────────────────────────────────────────────────────
if [ "$VERSION" = "latest" ]; then
if [ "$VERSION_ARG" = "latest" ]; then
echo "Fetching latest release version..."
VERSION=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" \
TAG=$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" \
| grep '"tag_name"' \
| sed -E 's/.*"tag_name": *"([^"]+)".*/\1/')
if [ -z "$VERSION" ]; then
echo "error: could not determine latest release. Check https://github.com/${REPO}/releases"
if [ -z "$TAG" ]; then
echo "error: could not determine latest release. Check https://github.com/${REPO}/releases" >&2
exit 1
fi
else
TAG="$VERSION_ARG"
fi
ARCHIVE="gitlawb-${VERSION}-${TARGET}.tar.gz"
DOWNLOAD_URL="https://github.com/${REPO}/releases/download/${VERSION}/${ARCHIVE}"
case "$TAG" in
v*) VERSION="${TAG#v}" ;;
*) VERSION="$TAG"; TAG="v$TAG" ;;
esac
PACKAGE_NAME="gitlawb-node-${VERSION}-${TARGET}"
ARCHIVE="${PACKAGE_NAME}.tar.gz"
DOWNLOAD_URL="https://github.com/${REPO}/releases/download/${TAG}/${ARCHIVE}"
CHECKSUM_URL="${DOWNLOAD_URL}.sha256"
echo "Installing gitlawb ${VERSION} for ${OS_NAME}/${ARCH_NAME}"
echo "Installing gitlawb ${TAG} for ${OS_NAME}/${ARCH_NAME}"
echo " Archive: ${ARCHIVE}"
echo " Into: ${INSTALL_DIR}"
echo ""
# ── Download ──────────────────────────────────────────────────────────────
TMP_DIR=$(mktemp -d)
trap 'rm -rf "$TMP_DIR"' EXIT
echo "Downloading..."
if ! curl -fSL --progress-bar -o "$TMP_DIR/$ARCHIVE" "$DOWNLOAD_URL"; then
echo ""
echo "error: download failed: $DOWNLOAD_URL"
echo " Check https://github.com/${REPO}/releases for available builds."
echo "error: download failed: $DOWNLOAD_URL" >&2
echo " Check https://github.com/${REPO}/releases for available builds." >&2
exit 1
fi
# ── Verify checksum ───────────────────────────────────────────────────────
if curl -fsSL -o "$TMP_DIR/$ARCHIVE.sha256" "$CHECKSUM_URL" 2>/dev/null; then
echo "Verifying checksum..."
EXPECTED=$(awk '{print $1}' "$TMP_DIR/$ARCHIVE.sha256")
if command -v sha256sum &>/dev/null; then
if command -v sha256sum >/dev/null 2>&1; then
ACTUAL=$(sha256sum "$TMP_DIR/$ARCHIVE" | awk '{print $1}')
elif command -v shasum &>/dev/null; then
elif command -v shasum >/dev/null 2>&1; then
ACTUAL=$(shasum -a 256 "$TMP_DIR/$ARCHIVE" | awk '{print $1}')
else
echo "warning: no sha256 tool found, skipping checksum verification"
ACTUAL="$EXPECTED"
fi
if [ "$EXPECTED" != "$ACTUAL" ]; then
echo "error: checksum mismatch!"
echo " expected: $EXPECTED"
echo " actual: $ACTUAL"
echo "error: checksum mismatch!" >&2
echo " expected: $EXPECTED" >&2
echo " actual: $ACTUAL" >&2
exit 1
fi
echo " ✓ checksum OK"
echo " checksum OK"
fi
# ── Extract and install ───────────────────────────────────────────────────
echo "Extracting..."
tar -xzf "$TMP_DIR/$ARCHIVE" -C "$TMP_DIR"
PACKAGE_DIR="$TMP_DIR/$PACKAGE_NAME"
if [ ! -d "$PACKAGE_DIR" ]; then
PACKAGE_DIR=$(find "$TMP_DIR" -mindepth 1 -maxdepth 1 -type d | head -n 1)
fi
if [ ! -d "$PACKAGE_DIR" ]; then
echo "error: could not find extracted package directory" >&2
exit 1
fi
mkdir -p "$INSTALL_DIR"
# Write to a staging file then rename — rename(2) is atomic on POSIX so
# a concurrent `gl doctor` (or any running gl process) never sees a
# partially-written binary.
install -m 755 "$TMP_DIR/gl" "$INSTALL_DIR/gl.new"
mv -f "$INSTALL_DIR/gl.new" "$INSTALL_DIR/gl"
install -m 755 "$TMP_DIR/git-remote-gitlawb" "$INSTALL_DIR/git-remote-gitlawb.new"
mv -f "$INSTALL_DIR/git-remote-gitlawb.new" "$INSTALL_DIR/git-remote-gitlawb"
INSTALLED=()
for BIN in gl git-remote-gitlawb gitlawb-node; do
if [ -f "$PACKAGE_DIR/$BIN" ]; then
install -m 755 "$PACKAGE_DIR/$BIN" "$INSTALL_DIR/$BIN.new"
mv -f "$INSTALL_DIR/$BIN.new" "$INSTALL_DIR/$BIN"
INSTALLED+=("$BIN")
fi
done
if [ "${#INSTALLED[@]}" -eq 0 ]; then
echo "error: no installable binaries found in $ARCHIVE" >&2
exit 1
fi
# ── PATH check ────────────────────────────────────────────────────────────
echo ""
echo "✓ Installed gitlawb ${VERSION}"
echo " gl → ${INSTALL_DIR}/gl"
echo " git-remote-gitlawb → ${INSTALL_DIR}/git-remote-gitlawb"
echo "Installed gitlawb ${TAG}"
for BIN in "${INSTALLED[@]}"; do
echo " ${BIN} -> ${INSTALL_DIR}/${BIN}"
done
echo ""
# Check if install dir is already on PATH
if echo ":$PATH:" | grep -q ":${INSTALL_DIR}:"; then
echo "Run:"
echo " gl doctor check your setup"
echo " gl quickstart create your identity and first repo"
echo " gl doctor"
echo " gl quickstart"
else
SHELL_NAME=$(basename "${SHELL:-bash}")
case "$SHELL_NAME" in
@@ -137,8 +189,8 @@ else
fi
echo ""
echo "Then run:"
echo " gl doctor check your setup"
echo " gl quickstart create your identity and first repo"
echo " gl doctor"
echo " gl quickstart"
fi
echo ""
+2 -2
View File
@@ -1,6 +1,6 @@
#!/bin/bash
# Build gl and git-remote-gitlawb binaries for all platforms.
# Output goes to web/public/bin/
# Output goes to dist/bin/
#
# Usage:
# ./scripts/build-bins.sh # linux-x86_64 + linux-arm64
@@ -11,7 +11,7 @@ set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
ROOT="$SCRIPT_DIR/.."
OUT="$ROOT/web/public/bin"
OUT="$ROOT/dist/bin"
mkdir -p "$OUT"