Compare commits

...
Author SHA1 Message Date
beardthelion 9a46e8cbb7 fix(core): zeroize the derived X25519 secret (#65) (#66)
x25519_secret_from_seed derived the X25519 private scalar from the Ed25519
seed and returned it as a bare [u8; 32]. Because [u8; 32] is Copy with no
Drop (and sha2's digest output likewise has no zeroize), the scalar, the
SHA-512 digest, and the returned temporary were released without being
scrubbed, leaving secret-derived material in freed memory. This is the same
Copy-no-Drop class fixed for the raw seed in #41, on the derived secret.

Return Zeroizing<[u8; 32]>, build the scalar directly into a zeroizing
buffer so no bare secret local persists, and explicitly wipe the SHA-512
digest before it drops. Callers deref the result into crypto_box::SecretKey,
which already scrubs its own copy. No behavior change: the derived scalar and
all decryption output are byte-identical, covered by the existing
ed25519_to_x25519_keypair_agrees and seal_open_round_trip_for_recipients tests.

Stacked on #41 (#64).
2026-06-23 22:05:56 +08:00
beardthelion e4f5e87e0b fix(node): migrate node_seed pin caller to to_seed()
main grew a third seed_bytes() caller (repos.rs, from #40) after this
branch was cut. Route it through to_seed() too: node_seed is moved into
a long-lived tokio::spawn pin task, so the Zeroizing wrapper scrubs the
copy when that task ends. encrypt_and_pin takes &[u8; 32]; the wrapper
deref-coerces, so the call site is unchanged.
2026-06-22 12:14:40 -05:00
beardthelion 38e654a289 Merge remote-tracking branch 'origin/main' into fix/zeroize-seed-bytes 2026-06-22 11:59:42 -05:00
beardthelion 4f814459cc test(core): pin to_seed's Zeroizing return type, drop duplicate round-trip
The to_seed_is_sole_seed_accessor test was named for a compile-time
property its body couldn't assert, and its from_seed -> DID check
duplicated from_seed_round_trip. Replace it with a type-annotated
binding that fails to compile if to_seed ever returns a bare [u8; 32]
again, plus the determinism check, which is the only behavior it
uniquely covered.
2026-06-22 11:52:38 -05:00
beardthelion 7533b74f79 ci: harden PR pipeline (audit, MSRV, beta matrix, release + docker smoke) (#57)
* ci: harden PR pipeline (audit, MSRV, release build, beta matrix, docker smoke)

Restructure the single stable-only fmt+clippy+test job into separate jobs:
fmt+clippy (stable), a test matrix on stable+beta (beta non-blocking),
build --release, cargo audit, an MSRV check pinned to Rust 1.91, and a
Docker build smoke test mirroring the release workflow. Add concurrency
cancellation for superseded PR runs and per-job timeouts.

Declare rust-version = "1.91" in [workspace.package] and inherit it in
every member crate so the stated MSRV is a real, CI-enforced contract.

* ci: disable credential persistence on checkout steps

Set persist-credentials: false on all six actions/checkout steps. These
jobs run untrusted PR code via cargo and docker build, and checkout's
default leaves the GITHUB_TOKEN in the local git config where that code
could read it. None of the jobs push or otherwise need git credentials.

* fix(deps): bump rustls-webpki to 0.103.13

Clears three advisories flagged by the new cargo audit job: a reachable
panic in CRL parsing and two name-constraint bypasses (wildcard and URI).
Semver-compatible patch, lockfile only, no manifest or code change.

cargo also corrects a stale lockfile reference while rewriting Cargo.lock:
data-encoding-macro-internal 0.1.17 (a build-time macro crate) requires
syn 1.x, and its dependency entry is re-pinned from syn 2.0.117 to the
1.0.109 it actually resolves to. Unrelated to the bump, unavoidable when
cargo touches the lock, and the more-correct state.

* ci(audit): suppress no-fix advisories with justification and a re-check

The cargo audit job stays hard-fail. The remaining three advisories have
no available upstream fix, so they are suppressed in .cargo/audit.toml
(the location cargo-audit actually reads; a repo-root audit.toml is not
read), each with an inline rationale and removal condition:

- RUSTSEC-2026-0118/0119 (hickory-proto 0.25.2, via libp2p-dns 0.44):
  reachable, since the node wraps QUIC in the system DNS resolver. No fix
  until libp2p-dns ships hickory 0.26. Accepted, reachable risk.
- RUSTSEC-2023-0071 (rsa Marvin): in the lockfile via sqlx's unconditional
  sqlx-mysql, but not linked (sqlx is postgres-only), so no RSA oracle
  exists; no upstream fix regardless.

Two controls keep the suppression honest. The PR audit job echoes the
active ignore list to the job summary, so a green check is not read as
advisory-clean. A weekly scheduled job re-runs cargo audit without the
ignores for visibility and hard-fails if hickory-proto moves off 0.25.x
while the ignores remain, so the suppression cannot silently outlive the
upstream fix.

* ci(audit): reference tracking issue #76 in the hickory ignores

* ci(audit): treat hickory-proto removal as drift too

The scheduled drift guard only failed when hickory-proto was present and
off 0.25.x, so removing it from Cargo.lock entirely left the RUSTSEC-2026-0118
/0119 ignores in place with the guard still passing. Treat an absent version
as drift as well, so dead ignore entries are caught whether the dependency is
bumped past 0.25.x or removed.
2026-06-22 19:02:38 +08:00
beardthelion abdc775770 feat(node): blind recipient identities at rest and gate B1 by repo readability (#40)
* feat(node): blind recipient identities at rest and gate B1 by repo readability

The origin no longer stores recipient DIDs. Migration v5 replaces the
encrypted_blobs.recipients column with an opaque, node-keyed recipients_tag
used only to detect a recipient-set change for re-seal. B1 discovery and fetch
are now gated by the same repo-readability check the git read path uses, not by
per-recipient matching; decryption is gated by the envelope crypto, so a
non-recipient who can read the repo sees a blob's {oid, cid} but cannot open it.
encrypt_and_pin keys the tag from the node seed and returns {oid, cid}; the
Arweave manifest tuple drops the now-unused recipient vec.

A DB compromise no longer reveals the reader set; recovering it would require
brute-forcing candidate DID sets against the keyed tag with the node key.

* chore: remove redundant .gitignore entry (covered by local exclude)

* fix(node): gate /replicate by repo readability; harden reseal on DB error

Address review on the at-rest blinding change:
- The encrypted-blobs/replicate listing returned {oid, cid} with no visibility
  check, so a non-readable repo's blob index was reachable by an unauthenticated
  caller who guessed {owner}/{repo}. Gate it by the same repo-readability check
  discovery and fetch use. For the intended case (a public repo with withheld
  subtrees) the public root keeps this open to peers; only fully non-readable
  repos are withheld, which is the desired behavior.
- encrypt_and_pin treated a recipients_tag DB read error as a cache miss and
  resealed, causing avoidable IPFS writes during a partial outage; skip and retry
  on the next push instead.
- Correct the get_encrypted_blob doc comment to describe repo-readability access.
2026-06-22 19:01:19 +08:00
beardthelion d45318281c fix(core): route seed access through the zeroizing wrapper (#41)
Remove Keypair::seed_bytes(), which returned a bare [u8; 32] copy of the
Ed25519 private seed. Because [u8; 32] is Copy with no Drop, each caller's
copy was released without being scrubbed, leaving secret key material in
freed memory. The zeroizing accessor to_seed() -> Zeroizing<[u8; 32]>
already existed beside it; this makes it the sole seed accessor and migrates
the two in-crate callers (envelope open path and a crypto test) to it via
deref coercion. No behavior change to envelope encryption: the derived
X25519 secret is byte-identical, covered by the existing round-trip tests.

Adds an identity test pinning to_seed() as the only seed accessor.
2026-06-19 22:01:08 -05:00
beardthelion e37ea7fec6 fix(node): enforce path-scoped visibility on the REST read API (#52)
* fix(node): enforce path-scoped visibility on the REST read API

The repo-scoped REST read endpoints (blob, tree, commits, refs, changelog,
pulls, pr diff, repo metadata) read straight from disk after a bare repo
lookup, with no visibility check, so private repositories and private
subtrees of public repositories were served to unauthenticated callers,
bypassing the withholding the git and replication paths enforce.

Add a shared authorize_repo_read gate (resolve repo, load rules,
visibility_check, 404 on deny) and call it from every repo-scoped read
handler. get_blob gates at the requested file path; get_pr_diff withholds
the whole diff when it touches a path the caller cannot read. Attach
optional_signature to the read routes so authorized readers authenticate.
Listings stay gated at the repo root, consistent with mode B exposing tree
and commit SHAs over git.

Closes #51

* fix(node): gate PR reviews/comments and fork on repo read visibility

The REST visibility gate added in this PR missed three read surfaces:
list_reviews and list_comments served a private repo's PR reviews and
comments to unauthenticated callers, and fork_repo cloned a full mirror of
the source without checking the caller could read it. Route all three through
authorize_repo_read at the repo root, matching the git-protocol gate.

Also make branch_diff_names fail closed: it ignored git's exit status and
returned an empty path list on error, which would let get_pr_diff serve a diff
it should have withheld. Fix changelog.rs import order to satisfy cargo fmt.

* fix(node): NUL-delimit branch_diff_names to keep path-gate enforcement exact

git diff --name-only quotes and escapes paths containing newlines, so they
no longer match the visibility globs in get_pr_diff. On a public repo where /
is allowed but a subtree is denied, a touched restricted path could be missed
by the deny check and the full PR diff disclosed. Use -z and split on NUL to
preserve exact path bytes.
2026-06-19 22:53:18 +08:00
beardthelion 197c4e1cf1 Add contributor templates, issue forms, and PR triage automation (#58)
* docs: add pull request template

* docs: add structured issue forms and template chooser config

* chore: add CODEOWNERS for CI, release, and security paths

* ci: add quality-signal PR triage workflow

* ci: add label-gated auto-stale workflow for ignored PRs

* docs: document PR quality bar and triage/stale policy in CONTRIBUTING

* ci: recognize inline #[cfg(test)] tests and fix triage comment grammar
2026-06-19 22:52:32 +08:00
a5d2264cc0 ci(release): smoke-test packaged binaries' --version before attaching (#59)
The release-binaries job packaged gl, git-remote-gitlawb, and gitlawb-node
into the release tarball but never ran them, so a broken artifact could ship.
This is the follow-up promised in #30 and the gap noted in OSS-READINESS-AUDIT
("release binaries are not smoke-tested after packaging"), now unblocked by
git-remote-gitlawb gaining --version.

Run each binary's --version after build and assert the released version string
is present. Skipped for aarch64-unknown-linux-musl, which is cross-compiled and
can't execute on the x86_64 runner.

Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-06-19 22:50:35 +08:00
beardthelionandCommandCodeBot 5ff7af84fa feat: encrypted replication for private subtrees (B1/B2/B3) for #18 (#36)
* feat(core): ed25519 to x25519 key conversion for envelope crypto

* feat(core): seal/open per-blob encryption envelopes

* feat(node): resolve per-blob encryption recipients (least-privilege)

* feat(node): encrypted_blobs table and recipient-scoped queries

* feat(node): encrypt-then-pin withheld blobs at push (IPFS)

At the push chokepoint, after pinning withheld objects, resolve each
withheld blob's recipient DIDs and seal it to their Ed25519 keys with
seal_blob, pinning the ciphertext to IPFS and recording it in
encrypted_blobs. Best-effort per blob: failures are logged and skipped,
never pinned in plaintext. Pinata replication is unchanged; B1
encrypts to IPFS only.

Adds ed25519-dalek as a direct dependency of gitlawb-node (it was only
declared in the workspace Cargo.toml).

* feat(node): authenticated discovery and fetch for encrypted blobs

* feat(gl): transparent recovery of authorized encrypted blobs on clone

* style: cargo fmt

* fix(node): re-seal encrypted blob when recipient set changes

Skip re-pinning only when an existing envelope already covers exactly the
current recipients. A reader added to a rule after the first pin now gets
a re-seal on the next push instead of being permanently locked out.
Reader removal stays non-retroactive (the old envelope is already public).

* feat(node): unscoped encrypted-blob listing for replication

* feat(node): encrypted-blobs/replicate endpoint for peer mirrors

* feat(node): route encrypted-blobs/replicate under git_read_routes

* feat(node): peer mirrors replicate encrypted withheld blobs (Option B2)

* feat(node): anchor_encrypted_manifest for Option B3 Arweave manifests

* feat(node): encrypt_and_pin returns the per-push sealed delta

* feat(node): anchor encrypted-blob manifest on push (Option B3)

* feat(gl): Arweave/IPFS gateway recovery for encrypted blobs (Option B3)

* feat(gl): transparent Arweave fallback recovery on clone (Option B3)

* style: cargo fmt

* fix(gl): bound Arweave recovery gateway requests with a 30s timeout

* test(gl): read-path recovery tests for Arweave fallback; quiet promisor present-check

Add two hermetic integration tests for recover_from_arweave that drive the
full read path over mocked Arweave GraphQL + IPFS gateways: discover the
manifest, fetch it, fetch the envelope, decrypt, and install the withheld
blob. One covers an authorized recipient (blob installed), the other a
non-recipient (nothing recovered). Both simulate origin death by removing
the promisor remote and enable uploadpack.allowFilter so the blob is truly
withheld over file://.

Also harden the local presence check in recover_from_arweave with
GIT_NO_LAZY_FETCH=1 and .output() so the expected 'missing object' case does
not trigger a wasted promisor fetch or leak git stderr to the user.

* fix: surface corrupt recipients JSON and silent recovery I/O failures

db: parse_recipients now propagates a descriptive error instead of
defaulting corrupt recipients JSON to an empty list, which would have
denied authorized readers and handed peers incomplete metadata.

gl: clone recovery now warns when the sparse-checkout file cannot be read
or written, or when the post-recovery checkout fails, instead of silently
discarding those errors and claiming files were recovered.

* feat(core): blind recipient identities in withheld-blob envelopes

Envelope v2 drops the cleartext recipient public key (kid) from each
wrapped-key header entry, so a party holding the public IPFS/Arweave copy
can no longer enumerate who is authorized to decrypt a withheld blob. The
version is bumped to 2 and v1 envelopes are rejected. open_blob now selects
the reader's entry by trial decryption (the AEAD tag authenticates exactly
one entry) instead of matching on the public key.

Recipient count is still visible; identity is not. Scope is envelope-only,
the node DB recipients column and peer replication metadata are unchanged.

Co-authored-by: CommandCodeBot <[email protected]>

* fix(core): reject malformed envelope nonces instead of panicking

open_blob fed attacker-controlled envelopes to GenericArray::from_slice and
XNonce::from_slice, which panic on a wrong-length input. Validate both the
per-recipient box nonce and the body nonce to 24 bytes before use: skip a
recipient entry whose nonce is malformed, and return an error for a malformed
body nonce, so the public recovery path surfaces an error rather than panicking.

* feat(node): blind recipient identities on the B2 replication surface

The encrypted-blobs/replicate endpoint shipped every blob's full recipient
DID list to mirrors, which persisted it, so any mirroring peer learned the
reader set. The v2 envelope blinding already removed recipient public keys
from the pinned envelopes, so the comment justifying this (DIDs are already
public) was no longer true.

/replicate now returns {oid, cid} only. Mirrors detect a re-seal by the CID
changing (the OID is stable across re-seals) instead of comparing recipient
sets, and store no recipient identities. Origin-side authz and the at-rest
recipients column are unchanged; this blinds only the peer-facing surface.

* feat(node): stop anchoring recipient identities to Arweave

The B3 encrypted-blob manifest anchored {oid, cid, recipients} to Arweave, a
permanent public record of every blob's reader set. The v2 envelope blinding
already removed recipient keys from the pinned envelopes, so the comment
justifying this (recipient DIDs are already public) was no longer true. The
manifest now anchors {oid, cid} only. The gl reader already ignores the
recipients field (it recovers by trial decryption), so no reader changes.

---------

Co-authored-by: CommandCodeBot <[email protected]>
2026-06-19 22:48:33 +08:00
38 changed files with 2813 additions and 78 deletions

No files matched your search

+38
View File
@@ -0,0 +1,38 @@
# cargo-audit configuration.
#
# cargo-audit reads its config from .cargo/audit.toml (relative to the working
# directory) or ~/.cargo/audit.toml. It does NOT read a repo-root audit.toml,
# and there is no CLI flag to point at a config file (`-f` is the lockfile path).
# This file lives at .cargo/audit.toml for that reason.
#
# The audit job stays hard-fail: only the advisories explicitly listed below are
# suppressed, so any NEW advisory still breaks CI. Each entry below has no
# available upstream fix; everything fixable is fixed in Cargo.lock instead.
#
# Tracking: the hickory ignores are reachable accepted risk. A scheduled re-check
# (.github/workflows/audit-schedule.yml) runs without these ignores and fails if
# the lockfile moves off the pinned vulnerable versions while the ignores remain.
[advisories]
ignore = [
# hickory-proto 0.25.2 (pulled by libp2p-dns 0.44 -> hickory-resolver 0.25).
# REACHABLE: the node wraps its QUIC transport in the system DNS resolver
# (crates/gitlawb-node/src/p2p/mod.rs:238), so a malicious DNS response can
# trigger these DoS paths. No fix is available: the latest libp2p-dns on
# crates.io is 0.44 and pins hickory 0.25; the fix requires hickory >=0.26.1.
# REMOVE both once libp2p-dns ships a release using hickory 0.26 (verified
# 2026-06-21: cargo update -p hickory-proto locks 0 packages; still 0.25.2).
# Tracking: https://github.com/Gitlawb/node/issues/76
"RUSTSEC-2026-0118", # hickory NSEC3 closest-encloser unbounded loop
"RUSTSEC-2026-0119", # hickory O(n^2) name-compression CPU exhaustion
# rsa 0.9.10 (Marvin timing sidechannel). Present in Cargo.lock because sqlx
# resolves sqlx-mysql unconditionally, and sqlx-mysql depends on rsa. cargo
# audit scans the lockfile, so it flags rsa even though it is NOT linked into
# our binary: sqlx is built postgres-only (crates/gitlawb-node/Cargo.toml),
# so the MySQL RSA auth path is never compiled and there is no RSA oracle to
# attack. No upstream fix exists regardless. REMOVE this ignore if sqlx is
# ever built with the `mysql` feature, or any other consumer of rsa enters
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
]
+11
View File
@@ -0,0 +1,11 @@
# Ownership for CI, release, and security-sensitive infrastructure.
# A PR that edits these paths requires review from a code owner once
# branch protection's "Require review from Code Owners" is enabled.
/.github/ @beardthelion
/Dockerfile @beardthelion
/Dockerfile.bins @beardthelion
/docker-compose.yml @beardthelion
/release-please-config.json @beardthelion
/.release-please-manifest.json @beardthelion
/SECURITY.md @beardthelion
+66
View File
@@ -0,0 +1,66 @@
name: Bug report
description: Report a defect in a Gitlawb crate or node behavior
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for the report. Please fill in the fields below so we can reproduce the issue.
For security vulnerabilities, do NOT open an issue — follow SECURITY.md instead.
- type: textarea
id: what-happened
attributes:
label: What happened?
description: A clear description of the bug.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: repro
attributes:
label: Steps to reproduce
placeholder: |
1. ...
2. ...
3. ...
validations:
required: true
- type: dropdown
id: crate
attributes:
label: Affected crate
options:
- gitlawb-node
- gl
- git-remote-gitlawb
- gitlawb-core
- not sure / other
validations:
required: true
- type: input
id: version
attributes:
label: Version or commit
description: Output of the binary's `--version`, or the git commit you built.
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
description: OS, Rust version (`rustc --version`), and anything else relevant.
validations:
required: false
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste logs or errors. This will be rendered as code, no need for backticks.
render: shell
validations:
required: false
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/Gitlawb/node/blob/main/SECURITY.md
about: Please report security issues privately per our security policy, not as a public issue.
- name: Questions & discussion
url: https://github.com/Gitlawb/node/discussions
about: For usage questions and open-ended discussion.
@@ -0,0 +1,34 @@
name: Feature request
description: Suggest an improvement or new capability
labels: ["enhancement"]
body:
- type: textarea
id: problem
attributes:
label: Problem / use case
description: What are you trying to do that's hard or impossible today?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
validations:
required: false
- type: dropdown
id: protocol
attributes:
label: Does this touch the protocol?
description: Identity, signatures, UCAN, ref certs, or P2P wire formats. Protocol changes need an issue discussion before any PR.
options:
- "No"
- "Yes"
- "Not sure"
validations:
required: true
+63
View File
@@ -0,0 +1,63 @@
<!--
Thanks for contributing to Gitlawb. Keep PRs focused on one change.
Protocol-level changes (identity, signatures, wire format) should start
as an issue before code — see CONTRIBUTING.md.
-->
## Summary
<!-- One or two sentences: what changes and why it matters. -->
## Motivation & context
<!-- What problem does this solve? Link prior discussion if any. -->
Closes #
## Kind of change
- [ ] Bug fix
- [ ] Feature
- [ ] Security fix
- [ ] Docs
- [ ] Tests / CI
- [ ] Refactor (no behavior change)
- [ ] Breaking or protocol change (issue required first)
## What changed
<!-- Bullet the concrete changes. Note the crate(s) touched:
gitlawb-core / gitlawb-node / gl / git-remote-gitlawb -->
-
## How a reviewer can verify
<!-- Commands to run, or repro steps for a bug plus proof it's fixed. -->
```sh
```
## Before you request review
- [ ] Scope is one logical change; no unrelated churn
- [ ] `cargo test --workspace` passes locally
- [ ] New behavior is covered by tests (required for fixes)
- [ ] `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` are clean
- [ ] Commit titles use Conventional Commits (`feat(...)`, `fix(...)`, `docs(...)`)
- [ ] Docs / `.env.example` updated if behavior or config changed (or N/A)
- [ ] Checked existing PRs so this isn't a duplicate
## Protocol & signing impact
<!-- Delete this block if your change doesn't touch the protocol. -->
- [ ] Touches DID / `did:key`, Ed25519 / RFC 9421 signatures, UCAN, ref certs, or P2P wire formats
- [ ] Discussed in an issue before implementation
- [ ] Backward-compatible with existing nodes and previously signed history
## Notes for reviewers
<!-- Anything out of scope, follow-ups, or known limitations. Optional. -->
+77
View File
@@ -0,0 +1,77 @@
name: Scheduled Audit
# Self-expiring suppression check (companion to the PR-blocking audit gate in
# pr-checks.yml). Runs weekly. It re-runs cargo audit WITHOUT the .cargo/audit.toml
# ignore list to surface the full advisory set for visibility, and it hard-fails
# if the lockfile has moved off the pinned vulnerable versions while the ignores
# are still present, i.e. the upstream fix landed but the ignore was not dropped.
on:
schedule:
- cron: "0 6 * * 1" # Mondays 06:00 UTC
workflow_dispatch:
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
jobs:
scheduled-audit:
name: scheduled audit (no ignores)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit-schedule
- name: Install cargo-audit
run: cargo install --locked cargo-audit
# Full advisory report with NO suppressions. Run from a scratch dir that
# has no .cargo/audit.toml so the ignore list does not apply; never fail
# the build on this step, it is visibility only.
- name: Full audit report (ignores not applied)
run: |
mkdir -p "$RUNNER_TEMP/full-audit"
cp Cargo.lock "$RUNNER_TEMP/full-audit/Cargo.lock"
cd "$RUNNER_TEMP/full-audit"
{
echo '### Full cargo audit (no ignores applied)'
echo '```'
cargo audit -f Cargo.lock || true
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
# Drift guard: the hickory ignores are only valid while hickory-proto is
# pinned at 0.25.x. If the lockfile has moved past that, the upstream fix
# is in and the ignores in .cargo/audit.toml must be removed. Fail loudly.
- name: Fail if hickory moved past 0.25.x while ignores remain
run: |
set -euo pipefail
version="$(grep -A1 'name = "hickory-proto"' Cargo.lock | grep '^version' | head -1 | cut -d'"' -f2)"
echo "hickory-proto in Cargo.lock: ${version:-not present}"
ignores_present=false
if grep -q 'RUSTSEC-2026-011' .cargo/audit.toml; then
ignores_present=true
fi
# Drift = the lockfile no longer matches what the ignores assume:
# hickory moved off 0.25.x (fix shipped) OR was removed entirely
# (dead ignore entries left behind). Both must fail.
if [ "$ignores_present" = true ] && { [ -z "${version}" ] || [[ "${version}" != 0.25.* ]]; }; then
echo "::error::hickory-proto is ${version:-absent from Cargo.lock} but the RUSTSEC-2026-0118/0119 ignores are still in .cargo/audit.toml. The upstream fix appears to be available or the dependency is gone; remove the ignores."
exit 1
fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version."
+137 -3
View File
@@ -9,18 +9,25 @@ on:
permissions:
contents: read
# Cancel superseded runs on the same PR/branch; never cancel runs on main.
concurrency:
group: pr-checks-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
jobs:
fmt-clippy-test:
name: fmt + clippy + test
fmt-clippy:
name: fmt + clippy
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
@@ -37,5 +44,132 @@ jobs:
- name: cargo clippy
run: cargo clippy --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
toolchain: [stable, beta]
# Beta is informational: an upstream beta regression should warn, not block merges.
continue-on-error: ${{ matrix.toolchain == 'beta' }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: ${{ matrix.toolchain }}
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: ${{ matrix.toolchain }}
- name: cargo test
run: cargo test --workspace
build-release:
name: build --release
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release
- name: cargo build --release
run: cargo build --release --workspace
audit:
name: cargo audit
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit
- name: Install cargo-audit
run: cargo install --locked cargo-audit
# Hard-fail gate. Suppressions live in .cargo/audit.toml (read automatically
# from the repo root), each with no available upstream fix. A green check
# here means "no NEW advisories", not "advisory-clean": surface the active
# ignore list so a reviewer sees exactly what is being accepted.
- name: Show active audit ignores
run: |
{
echo '### cargo audit: accepted advisories (.cargo/audit.toml)'
echo '```toml'
sed -n '/^\[advisories\]/,$p' .cargo/audit.toml
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
- name: cargo audit
run: cargo audit
msrv:
name: MSRV (Rust 1.91)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust 1.91
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # 1.91
with:
toolchain: "1.91"
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: msrv
- name: cargo check on MSRV
run: cargo check --workspace --all-targets
docker-build:
name: Docker build smoke test
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Build image
run: docker build -t gitlawb-node:ci-test .
- name: Smoke test --version
run: docker run --rm gitlawb-node:ci-test --version
+120
View File
@@ -0,0 +1,120 @@
name: PR Triage
on:
pull_request:
types: [opened, edited, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: write
concurrency:
group: pr-triage-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
triage:
name: Quality-signal triage
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Label and guide
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const pr = context.payload.pull_request;
const { owner, repo } = context.repo;
const prNumber = pr.number;
// Trusted authors bypass triage entirely.
const trusted = ["OWNER", "MEMBER", "COLLABORATOR"];
if (trusted.includes(pr.author_association)) {
core.info(`Author association ${pr.author_association} is trusted; skipping triage.`);
return;
}
// Labels this workflow owns. Anything here may be added or removed each run.
const MANAGED = ["needs-issue", "needs-description", "needs-tests", "workflow-change"];
const want = new Set();
const body = pr.body || "";
// Linked issue: look for "closes/fixes/resolves #N" or any bare #N reference.
const hasLinkedIssue =
/\b(close[sd]?|fix(e[sd])?|resolve[sd]?)\b[^\n]*#\d+/i.test(body) ||
/#\d+/.test(body);
if (!hasLinkedIssue) want.add("needs-issue");
// Description substance: strip HTML comments and markdown headings, then measure.
const stripped = body
.replace(/<!--[\s\S]*?-->/g, "")
.replace(/^#+.*$/gm, "")
.trim();
if (stripped.length < 50) want.add("needs-description");
// File-based signals.
const files = await github.paginate(github.rest.pulls.listFiles, {
owner, repo, pull_number: prNumber, per_page: 100,
});
const names = files.map(f => f.filename);
const isFork = pr.head.repo && pr.head.repo.full_name !== pr.base.repo.full_name;
const touchesWorkflows = names.some(n => n.startsWith(".github/workflows/"));
if (isFork && touchesWorkflows) want.add("workflow-change");
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or — the common Rust
// pattern — an inline #[test] / #[cfg(test)] block added inside the source file.
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
// Reconcile labels: add wanted managed labels, remove managed labels no longer wanted.
const current = (await github.rest.issues.listLabelsOnIssue({
owner, repo, issue_number: prNumber,
})).data.map(l => l.name);
const toAdd = [...want].filter(l => !current.includes(l));
const toRemove = MANAGED.filter(l => current.includes(l) && !want.has(l));
if (toAdd.length) {
await github.rest.issues.addLabels({ owner, repo, issue_number: prNumber, labels: toAdd });
}
for (const name of toRemove) {
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNumber, name })
.catch(() => {}); // ignore if already gone
}
// Guidance comment: only for advisory signals, and only once.
const advisory = ["needs-issue", "needs-description", "needs-tests"].filter(l => want.has(l));
if (!advisory.length) return;
const MARKER = "<!-- gitlawb-triage -->";
const comments = await github.paginate(github.rest.issues.listComments, {
owner, repo, issue_number: prNumber, per_page: 100,
});
if (comments.some(c => c.body && c.body.includes(MARKER))) {
core.info("Guidance comment already present; not reposting.");
return;
}
const reasons = {
"needs-issue": "Link the issue this addresses (`Closes #123`). For protocol changes, open an issue first.",
"needs-description": "Add a short summary of what changes and why — the template's Summary and Motivation sections.",
"needs-tests": "This changes Rust source but no tests changed. Tests are required for fixes and strongly encouraged for features.",
};
const lines = advisory.map(l => `- ${reasons[l]}`).join("\n");
const comment = [
MARKER,
"Thanks for the contribution. A couple of things will help us review this faster:",
"",
lines,
"",
"See [CONTRIBUTING.md](https://github.com/Gitlawb/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
].join("\n");
await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: comment });
+21
View File
@@ -146,6 +146,27 @@ jobs:
- name: Build
run: cargo build --release --target ${{ matrix.target }} -p gl -p git-remote-gitlawb -p gitlawb-node
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Skipped for cross-compiled targets that can't run
# on the host runner (aarch64 linux on an x86_64 runner).
- name: Smoke test binaries
if: ${{ matrix.target != 'aarch64-unknown-linux-musl' }}
shell: bash
run: |
set -euo pipefail
VERSION="${{ needs.release-please.outputs.version }}"
BIN_DIR="target/${{ matrix.target }}/release"
for bin in gl git-remote-gitlawb gitlawb-node; do
echo "== $bin --version =="
out="$("$BIN_DIR/$bin" --version)"
echo "$out"
# Each binary prints "<name> <version>"; assert the released version is present.
grep -qF "$VERSION" <<<"$out" || {
echo "::error::$bin --version did not report expected version $VERSION (got: $out)"
exit 1
}
done
- name: Package
id: pkg
shell: bash
+39
View File
@@ -0,0 +1,39 @@
name: Stale PRs
on:
schedule:
- cron: "30 1 * * *" # daily at 01:30 UTC
workflow_dispatch:
permissions:
contents: read
pull-requests: write
issues: write
jobs:
stale:
name: Flag and close ignored PRs
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9
with:
days-before-stale: 14
days-before-close: 7
# Act on PRs only; never on issues.
days-before-issue-stale: -1
days-before-issue-close: -1
# Only PRs the triage step flagged as needing an author response.
only-pr-labels: "needs-issue,needs-description"
stale-pr-label: "stale"
exempt-pr-labels: "workflow-change,security,pinned"
remove-stale-when-updated: true
stale-pr-message: >
This PR has been inactive for 14 days and is still missing a linked issue
or a description. It will be closed in 7 days if there's no update. Push a
change or leave a comment to keep it open — no hard feelings, you can reopen
anytime.
close-pr-message: >
Closing due to inactivity. Reopen whenever you're ready to update it — see
CONTRIBUTING.md for what helps us review quickly.
ascending: true
-3
View File
@@ -23,6 +23,3 @@ keys/
# Logs
*.log
.openclaude-profile.json
# Local planning / scratch docs (never commit)
docs/superpowers/
+18
View File
@@ -55,6 +55,24 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
## What gets merged, what gets closed
We welcome contributions from humans and agents alike. To keep review sustainable, PRs are
expected to clear a basic quality bar:
- **Link an issue.** Bug fixes and features should reference an issue (`Closes #123`). For
protocol-level changes (identity, signatures, UCAN, ref certs, wire formats), open the
issue *before* writing code.
- **One change per PR.** Unrelated churn slows review and gets sent back.
- **Tests and a green pipeline.** New behavior needs tests; `cargo fmt`, `cargo clippy`,
and the full CI suite must pass.
- **A real description.** Say what changes and why. "Update code" is not a description.
A triage bot labels PRs that are missing these and leaves a short note. Nothing is closed
automatically while you're engaging. A flagged PR that goes 14 days with no linked issue or
description gets a stale warning, and is closed 7 days later if still untouched. Closed PRs can
be reopened at any time once updated.
## Development environment
**Requirements:**
Generated
+129 -3
View File
@@ -8,6 +8,16 @@ version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe438c63458706e03479442743baae6c88256498e6431708f6dfc520a26515d3"
[[package]]
name = "aead"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d122413f284cf2d62fb1b7db97e02edb8cda96d769b16e443a4f6195e35662b0"
dependencies = [
"crypto-common",
"generic-array",
]
[[package]]
name = "ahash"
version = "0.8.12"
@@ -1975,6 +1985,30 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]]
name = "chrono"
version = "0.4.44"
@@ -2001,6 +2035,17 @@ dependencies = [
"unsigned-varint 0.8.0",
]
[[package]]
name = "cipher"
version = "0.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [
"crypto-common",
"inout",
"zeroize",
]
[[package]]
name = "clap"
version = "4.5.60"
@@ -2307,9 +2352,41 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"rand_core 0.6.4",
"typenum",
]
[[package]]
name = "crypto_box"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "16182b4f39a82ec8a6851155cc4c0cda3065bb1db33651726a29e1951de0f009"
dependencies = [
"aead",
"chacha20",
"crypto_secretbox",
"curve25519-dalek",
"salsa20",
"subtle",
"zeroize",
]
[[package]]
name = "crypto_secretbox"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d6cf87adf719ddf43a805e92c6870a531aedda35ff640442cbaf8674e141e1"
dependencies = [
"aead",
"chacha20",
"cipher",
"generic-array",
"poly1305",
"salsa20",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek"
version = "4.1.3"
@@ -2479,7 +2556,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de"
dependencies = [
"data-encoding",
"syn 2.0.117",
"syn 1.0.109",
]
[[package]]
@@ -3255,8 +3332,11 @@ version = "0.3.9"
dependencies = [
"anyhow",
"base64",
"chacha20poly1305",
"chrono",
"cid",
"crypto_box",
"curve25519-dalek",
"ed25519-dalek",
"hex",
"multibase",
@@ -3290,6 +3370,7 @@ dependencies = [
"cid",
"clap",
"dirs-next",
"ed25519-dalek",
"futures",
"gitlawb-core",
"hex",
@@ -3919,6 +4000,15 @@ dependencies = [
"serde_core",
]
[[package]]
name = "inout"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01"
dependencies = [
"generic-array",
]
[[package]]
name = "ipconfig"
version = "0.3.4"
@@ -4811,6 +4901,12 @@ version = "1.70.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe"
[[package]]
name = "opaque-debug"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]]
name = "openssl-probe"
version = "0.2.1"
@@ -5060,6 +5156,17 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]]
name = "portable-atomic"
version = "1.13.1"
@@ -5736,9 +5843,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.10"
version = "0.103.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e"
dependencies = [
"aws-lc-rs",
"ring",
@@ -5787,6 +5894,15 @@ version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6518fc26bced4d53678a22d6e423e9d8716377def84545fe328236e3af070e7f"
[[package]]
name = "salsa20"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97a22f5af31f73a954c10289c93e8a50cc23d971e80ee446f1f6f7137a088213"
dependencies = [
"cipher",
]
[[package]]
name = "schannel"
version = "0.1.29"
@@ -7035,6 +7151,16 @@ version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "universal-hash"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc1de2c688dc15305988b563c3854064043356019f97a4b46276fe734c4f07ea"
dependencies = [
"crypto-common",
"subtle",
]
[[package]]
name = "unsigned-varint"
version = "0.7.2"
+1
View File
@@ -11,6 +11,7 @@ members = [
[workspace.package]
version = "0.3.9"
edition = "2021"
rust-version = "1.91"
license = "MIT OR Apache-2.0"
authors = ["gitlawb contributors"]
repository = "https://github.com/gitlawb/node"
+1
View File
@@ -3,6 +3,7 @@ name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
+1
View File
@@ -3,6 +3,7 @@ name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
+4
View File
@@ -3,6 +3,7 @@ name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
@@ -23,6 +24,9 @@ chrono = { workspace = true }
uuid = { workspace = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
crypto_box = { version = "0.9", features = ["std", "chacha20"] }
chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
+296
View File
@@ -0,0 +1,296 @@
//! Envelope encryption for withheld blobs (Option B). A random content key
//! encrypts the blob (XChaCha20-Poly1305); the content key is wrapped to each
//! recipient via an X25519 box keyed from their Ed25519 `did:key`. The node
//! seals with public keys only; readers open with their own private key.
use crate::identity::Keypair;
use anyhow::{Context, Result};
use ed25519_dalek::VerifyingKey;
use zeroize::Zeroizing;
/// X25519 public key (Montgomery u) for an Ed25519 verifying key.
fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
use curve25519_dalek::edwards::CompressedEdwardsY;
let edwards = CompressedEdwardsY::from_slice(vk.as_bytes())
.ok()
.and_then(|c| c.decompress())
.context("verifying key is not a valid edwards point")?;
Ok(edwards.to_montgomery().to_bytes())
}
/// X25519 secret scalar for an Ed25519 seed (SHA-512 of seed, lower 32, clamped).
/// Returns the scalar wrapped in `Zeroizing`, and scrubs the intermediate
/// SHA-512 digest, so no copy of this secret material lingers in freed memory.
fn x25519_secret_from_seed(seed: &[u8; 32]) -> Zeroizing<[u8; 32]> {
use sha2::{Digest, Sha512};
use zeroize::Zeroize;
let mut h = Sha512::digest(seed);
let mut s = Zeroizing::new([0u8; 32]);
s.copy_from_slice(&h[..32]);
s[0] &= 248;
s[31] &= 127;
s[31] |= 64;
h.as_mut_slice().zeroize();
s
}
use base64::{engine::general_purpose::STANDARD as B64, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit},
XChaCha20Poly1305, XNonce,
};
use crypto_box::{
aead::{AeadCore, OsRng},
ChaChaBox, PublicKey as XPublic, SecretKey as XSecret,
};
use rand::RngCore;
use serde::{Deserialize, Serialize};
const MAGIC: &[u8] = b"GLENC";
const VERSION: u8 = 2;
#[derive(Serialize, Deserialize)]
struct Recipient {
eph: String, // base64 ephemeral x25519 pubkey (32B)
nonce: String, // base64 box nonce (24B)
wrap: String, // base64 wrapped content key
}
#[derive(Serialize, Deserialize)]
struct Header {
alg: String,
nonce: String, // base64 body nonce (24B)
recipients: Vec<Recipient>,
}
/// Encrypt `plaintext` so any of `recipients` (Ed25519 keys) can decrypt.
pub fn seal_blob(plaintext: &[u8], recipients: &[VerifyingKey]) -> Result<Vec<u8>> {
if recipients.is_empty() {
return Err(anyhow::anyhow!("seal_blob: no recipients"));
}
let mut content_key = [0u8; 32];
OsRng.fill_bytes(&mut content_key);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
.map_err(|e| anyhow::anyhow!("content key: {e}"))?;
let mut body_nonce = [0u8; 24];
OsRng.fill_bytes(&mut body_nonce);
let body = body_cipher
.encrypt(XNonce::from_slice(&body_nonce), plaintext)
.map_err(|e| anyhow::anyhow!("body encrypt: {e}"))?;
let mut wrapped = Vec::with_capacity(recipients.len());
for vk in recipients {
let recip_x = XPublic::from(x25519_public(vk)?);
let eph = XSecret::generate(&mut OsRng);
let abox = ChaChaBox::new(&recip_x, &eph);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let ct = abox
.encrypt(&nonce, &content_key[..])
.map_err(|e| anyhow::anyhow!("wrap: {e}"))?;
wrapped.push(Recipient {
eph: B64.encode(eph.public_key().as_bytes()),
nonce: B64.encode(nonce),
wrap: B64.encode(ct),
});
}
let header = Header {
alg: "xchacha20poly1305".into(),
nonce: B64.encode(body_nonce),
recipients: wrapped,
};
let header_json = serde_json::to_vec(&header).context("encode header")?;
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
out.extend_from_slice(&header_json);
out.extend_from_slice(&body);
Ok(out)
}
/// Decrypt an envelope with `keypair`. Errors if not a recipient or on auth fail.
pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
let mut p = 0;
if envelope.len() < MAGIC.len() + 1 + 4 || &envelope[..MAGIC.len()] != MAGIC {
return Err(anyhow::anyhow!("bad envelope magic"));
}
p += MAGIC.len();
if envelope[p] != VERSION {
return Err(anyhow::anyhow!("unsupported envelope version"));
}
p += 1;
let hlen = u32::from_le_bytes(envelope[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header: Header =
serde_json::from_slice(envelope.get(p..p + hlen).context("truncated header")?)
.context("decode header")?;
let body = &envelope[p + hlen..];
let my_x = XSecret::from(*x25519_secret_from_seed(&keypair.to_seed()));
// Identities are blinded: no entry says which recipient it belongs to, so
// try each one. The ChaChaBox AEAD tag authenticates, so exactly the
// reader's own entry unwraps; every other entry fails cleanly.
let mut content_key: Option<Vec<u8>> = None;
for entry in &header.recipients {
let eph = match B64
.decode(&entry.eph)
.ok()
.and_then(|b| <[u8; 32]>::try_from(b.as_slice()).ok())
{
Some(b) => XPublic::from(b),
None => continue,
};
// from_slice panics on a wrong length, and the envelope is attacker
// controlled, so validate the 24-byte box nonce before using it.
let nonce = match B64
.decode(&entry.nonce)
.ok()
.and_then(|n| <[u8; 24]>::try_from(n.as_slice()).ok())
{
Some(n) => n,
None => continue,
};
let wrap = match B64.decode(&entry.wrap) {
Ok(w) => w,
Err(_) => continue,
};
let abox = ChaChaBox::new(&eph, &my_x);
if let Ok(ck) = abox.decrypt(
crypto_box::aead::generic_array::GenericArray::from_slice(&nonce),
wrap.as_slice(),
) {
content_key = Some(ck);
break;
}
}
let content_key = content_key.context("not a recipient of this envelope")?;
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
.map_err(|e| anyhow::anyhow!("content key: {e}"))?;
let body_nonce = B64
.decode(&header.nonce)
.ok()
.and_then(|n| <[u8; 24]>::try_from(n.as_slice()).ok())
.context("invalid body nonce")?;
body_cipher
.decrypt(XNonce::from_slice(&body_nonce), body)
.map_err(|_| anyhow::anyhow!("body decrypt failed"))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::identity::Keypair;
#[test]
fn ed25519_to_x25519_keypair_agrees() {
// The X25519 public derived from the Ed25519 public must equal the
// X25519 public of the X25519 secret derived from the same seed.
let kp = Keypair::generate();
let seed = kp.to_seed();
let xpub_from_public = x25519_public(&kp.verifying_key()).unwrap();
let xsec = x25519_secret_from_seed(&seed);
let xpub_from_secret = crypto_box::SecretKey::from(*xsec).public_key().to_bytes();
assert_eq!(xpub_from_public, xpub_from_secret);
}
#[test]
fn seal_open_round_trip_for_recipients() {
let owner = Keypair::generate();
let reader_a = Keypair::generate();
let reader_b = Keypair::generate();
let msg = b"private blob contents";
let env = seal_blob(msg, &[owner.verifying_key(), reader_a.verifying_key()]).unwrap();
assert_eq!(open_blob(&env, &owner).unwrap(), msg);
assert_eq!(open_blob(&env, &reader_a).unwrap(), msg);
assert!(
open_blob(&env, &reader_b).is_err(),
"non-recipient must fail"
);
}
#[test]
fn tampered_envelope_fails() {
let owner = Keypair::generate();
let mut env = seal_blob(b"hi", &[owner.verifying_key()]).unwrap();
let last = env.len() - 1;
env[last] ^= 0x01;
assert!(open_blob(&env, &owner).is_err());
}
#[test]
fn v2_header_contains_no_recipient_pubkey() {
// The blinded envelope header must not carry any recipient's public key.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Slice out the header bytes using the envelope framing:
// MAGIC | version(1B) | header_len(4B LE) | header_json | body
let mut p = MAGIC.len() + 1; // skip MAGIC + version byte
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header = &env[p..p + hlen];
let header_str = String::from_utf8_lossy(header);
let pubkey_b64 = B64.encode(reader.verifying_key().as_bytes());
assert!(
!header_str.contains(&pubkey_b64),
"recipient public key must not appear in the blinded header"
);
}
#[test]
fn v1_envelope_is_rejected() {
let reader = Keypair::generate();
let mut env = seal_blob(b"hi", &[reader.verifying_key()]).unwrap();
// Flip the version byte (immediately after MAGIC) from 2 to 1.
env[MAGIC.len()] = 1;
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("unsupported envelope version"),
"expected version-rejection error, got: {err}"
);
}
#[test]
fn malformed_nonce_returns_err_not_panic() {
// from_slice panics on wrong-length input; a crafted envelope on the
// public recovery path must surface an error, never panic.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Split the envelope framing into header JSON and body.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
let reframe = |header: &serde_json::Value| -> Vec<u8> {
let hj = serde_json::to_vec(header).unwrap();
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(hj.len() as u32).to_le_bytes());
out.extend_from_slice(&hj);
out.extend_from_slice(body);
out
};
let bad_nonce = serde_json::Value::String(B64.encode([0u8; 12]));
// Corrupted per-recipient nonce: entry is skipped, no match.
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
header["recipients"][0]["nonce"] = bad_nonce.clone();
assert!(open_blob(&reframe(&header), &reader).is_err());
// Corrupted body nonce: unwrap succeeds, body nonce is rejected.
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
header["nonce"] = bad_nonce;
assert!(open_blob(&reframe(&header), &reader).is_err());
}
}
+17 -1
View File
@@ -52,7 +52,9 @@ impl Keypair {
URL_SAFE_NO_PAD.encode(sig.to_bytes())
}
/// Export the signing key as raw 32-byte seed (wrapped in Zeroizing).
/// The raw 32-byte Ed25519 seed, wrapped in `Zeroizing` so the copy is
/// scrubbed on drop. This is the only seed accessor; it is used to derive
/// the X25519 secret for envelope decryption (see `crate::encrypt`).
pub fn to_seed(&self) -> Zeroizing<[u8; 32]> {
Zeroizing::new(self.signing_key.to_bytes())
}
@@ -166,6 +168,20 @@ mod tests {
assert_eq!(kp.verifying_key(), kp2.verifying_key());
}
// `to_seed()` is the only seed accessor on `Keypair` (the raw, unzeroized
// `seed_bytes()` was removed in #41). Pin its return type so a bare-array
// accessor can't slip back in, and confirm it is deterministic. The
// from_seed -> verifying_key round-trip is already covered by
// `from_seed_round_trip`.
#[test]
fn to_seed_returns_zeroizing_and_is_deterministic() {
let kp = Keypair::generate();
// The type annotation is the regression guard: the seed must be handed
// out wrapped in `Zeroizing`, never as a bare `[u8; 32]`.
let seed: Zeroizing<[u8; 32]> = kp.to_seed();
assert_eq!(*kp.to_seed(), *seed); // stable across calls
}
#[test]
fn sign_b64_decodes_to_valid_signature() {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
+1
View File
@@ -1,6 +1,7 @@
pub mod cert;
pub mod cid;
pub mod did;
pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
+2
View File
@@ -3,6 +3,7 @@ name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
@@ -11,6 +12,7 @@ path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
ed25519-dalek = { workspace = true }
tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
+6 -6
View File
@@ -1,9 +1,10 @@
//! Changelog endpoint — unified timeline of commits, merged PRs, and closed issues.
use axum::extract::{Path, Query, State};
use axum::extract::{Extension, Path, Query, State};
use axum::Json;
use serde::Deserialize;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::git::store;
use crate::state::AppState;
@@ -27,12 +28,11 @@ pub async fn get_changelog(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(query): Query<ChangelogQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let limit = query.limit.min(100);
+123
View File
@@ -0,0 +1,123 @@
//! Authenticated discovery + fetch for encrypted withheld blobs (Option B1).
use axum::extract::{Extension, Path, State};
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs
/// Returns [{oid, cid}] for every encrypted blob in the repo, to any caller who
/// can read the repo. Not recipient-scoped: recipient identities are not stored,
/// so access control here is repo readability and decryption is gated by the
/// envelope crypto (only a real recipient can open an envelope).
pub async fn list_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
.map(|(oid, cid)| serde_json::json!({ "oid": oid, "cid": cid }))
.collect();
Ok(Json(serde_json::json!({ "blobs": blobs })))
}
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blob/{oid}
/// Returns raw envelope bytes to callers who can read the repo; the envelope
/// crypto still ensures only true recipients can decrypt.
pub async fn get_encrypted_blob(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo, oid)): Path<(String, String, String)>,
) -> Result<Vec<u8>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")));
}
let cid = state
.db
.encrypted_blob_cid(&record.id, &oid)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")))?;
let bytes = crate::ipfs_pin::cat(&state.config.ipfs_api, &cid)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
Ok(bytes)
}
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate
/// Returns [{oid, cid}] for every encrypted blob in the repo, for peer-mirror
/// replication (Option B2). Gated by repo readability, like discovery, so a
/// non-readable repo does not expose its blob index; for the intended case (a
/// public repo with withheld subtrees) the public root keeps this open to peers.
/// Recipient identities are deliberately withheld: the v2 envelopes no longer
/// carry recipient public keys, so peers must not learn the reader set either. A
/// mirror detects a re-seal by the CID changing (the OID is stable across
/// re-seals). Ciphertext metadata only, never plaintext.
pub async fn replicate_encrypted_blobs(
State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows
.into_iter()
.map(|(oid, cid)| replicate_blob_json(oid, cid))
.collect();
Ok(Json(serde_json::json!({ "blobs": blobs })))
}
/// Serialize one blob for the replication wire. Recipient identities are
/// intentionally absent so a mirror never learns the reader set.
fn replicate_blob_json(oid: String, cid: String) -> serde_json::Value {
serde_json::json!({ "oid": oid, "cid": cid })
}
#[cfg(test)]
mod tests {
use super::replicate_blob_json;
#[test]
fn replicate_blob_json_omits_recipients() {
let v = replicate_blob_json("oid1".into(), "cidA".into());
assert_eq!(v["oid"], "oid1");
assert_eq!(v["cid"], "cidA");
assert!(
v.get("recipients").is_none(),
"replication wire must not carry recipient identities"
);
}
}
+35
View File
@@ -1,8 +1,14 @@
use crate::db::{RepoRecord, VisibilityRule};
use crate::error::{AppError, Result};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
pub mod agents;
pub mod arweave;
pub mod bounties;
pub mod certs;
pub mod changelog;
pub mod encrypted;
pub mod events;
pub mod ipfs;
pub mod issues;
@@ -19,3 +25,32 @@ pub mod stars;
pub mod tasks;
pub mod visibility;
pub mod webhooks;
/// Resolve a repo for a read request and enforce path-scoped visibility.
///
/// Returns 404 (`RepoNotFound`) if the repo does not exist or the caller may not
/// read `path`, using the same opaque response the git serve path returns so
/// existence is not confirmed. Returns the record and its visibility rules so a
/// content handler can apply an extra per-path check without a second DB query.
///
/// Callers pass `"/"` for repo-level reads (listings); content endpoints pass the
/// specific path so a withheld subtree is denied even on an otherwise-public repo.
pub(crate) async fn authorize_repo_read(
state: &AppState,
owner: &str,
name: &str,
caller: Option<&str>,
path: &str,
) -> Result<(RepoRecord, Vec<VisibilityRule>)> {
let record = state
.db
.get_repo(owner, name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
if visibility_check(&rules, record.is_public, &record.owner_did, caller, path) == Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
Ok((record, rules))
}
+39 -25
View File
@@ -101,12 +101,11 @@ pub async fn create_pr(
pub async fn list_prs(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let prs = state.db.list_prs(&record.id).await?;
Ok(Json(
@@ -118,12 +117,11 @@ pub async fn list_prs(
pub async fn get_pr(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<PullRequest>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
@@ -138,12 +136,11 @@ pub async fn get_pr(
pub async fn get_pr_diff(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
@@ -156,6 +153,25 @@ pub async fn get_pr_diff(
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
// Withhold the entire diff if it touches a path the caller cannot read, so a
// PR diff cannot leak private-subtree content of an otherwise-public repo.
let touched = store::branch_diff_names(&disk_path, &pr.target_branch, &pr.source_branch)
.map_err(|e| AppError::Git(e.to_string()))?;
for p in &touched {
let gate = format!("/{}", p.trim_start_matches('/'));
if crate::visibility::visibility_check(
&rules,
record.is_public,
&record.owner_did,
caller,
&gate,
) == crate::visibility::Decision::Deny
{
return Err(AppError::NotFound(format!("PR #{number} not found")));
}
}
let diff = store::branch_diff(&disk_path, &pr.target_branch, &pr.source_branch)
.map_err(|e| AppError::Git(e.to_string()))?;
@@ -324,12 +340,11 @@ pub async fn create_review(
pub async fn list_reviews(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
@@ -383,12 +398,11 @@ pub async fn create_comment(
pub async fn list_comments(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
+97 -35
View File
@@ -186,12 +186,11 @@ pub async fn list_repos(
pub async fn get_repo(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<RepoResponse>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let count = state.db.count_stars(&record.id).await.unwrap_or(0);
Ok(Json(to_response(&record, &state, count)))
}
@@ -200,12 +199,11 @@ pub async fn get_repo(
pub async fn list_commits(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let disk_path = state
.repo_store
@@ -222,6 +220,7 @@ pub async fn list_commits(
pub async fn get_blob(
State(state): State<AppState>,
Path((owner, name, file_path)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Response> {
use axum::http::header;
use axum::response::IntoResponse;
@@ -238,11 +237,10 @@ pub async fn get_blob(
return Err(AppError::BadRequest("invalid file path".into()));
}
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let gate_path = format!("/{file_path}");
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, &gate_path).await?;
let disk_path = state
.repo_store
@@ -283,12 +281,11 @@ pub async fn get_blob(
pub async fn get_tree_root(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let disk_path = state
.repo_store
@@ -305,12 +302,11 @@ pub async fn get_tree_root(
pub async fn get_tree(
State(state): State<AppState>,
Path((owner, name, tree_path)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let disk_path = state
.repo_store
@@ -686,6 +682,15 @@ pub async fn git_receive_pack(
let ipfs_api = state.config.ipfs_api.clone();
let repo_path_clone = disk_path.clone();
let db_clone = state.db.clone();
let rules_for_enc = rules_opt.clone();
let repo_id = record.id.clone();
let owner_did = record.owner_did.clone();
let is_public = record.is_public;
let irys_url = state.config.irys_url.clone();
let http_client = std::sync::Arc::clone(&state.http_client);
let node_did_str = state.node_did.to_string();
let node_seed = state.node_keypair.to_seed();
let repo_name = record.name.clone();
tokio::spawn(async move {
let pinned = crate::ipfs_pin::pin_new_objects(
&ipfs_api,
@@ -700,6 +705,65 @@ pub async fn git_receive_pack(
tracing::info!(sha = %sha, %cid, "pinned");
}
}
// Option B1: encrypt-then-pin the withheld blobs so authorized
// readers can recover them when the origin cannot serve them.
if let Some(rules) = rules_for_enc.filter(|r| !r.is_empty()) {
let p = repo_path_clone.clone();
let owner = owner_did.clone();
let recip = tokio::task::spawn_blocking(move || {
crate::git::visibility_pack::withheld_blob_recipients(
&p, &rules, is_public, &owner,
)
})
.await;
if let Ok(Ok(recipients)) = recip {
let delta = crate::encrypted_pin::encrypt_and_pin(
&ipfs_api,
&repo_path_clone,
&db_clone,
&repo_id,
&node_seed,
&recipients,
)
.await;
// Option B3: anchor a per-push manifest of the blobs sealed
// this push to Arweave, so the oid->cid index survives total
// node loss. Best-effort; never fails the push.
if !delta.is_empty() && !irys_url.is_empty() {
let owner_short = owner_did.split(':').next_back().unwrap_or(&owner_did);
let repo_slug = format!("{owner_short}/{repo_name}");
let ts = chrono::Utc::now().to_rfc3339();
let manifest = crate::arweave::EncryptedManifest {
repo: &repo_slug,
owner_did: &owner_did,
node_did: &node_did_str,
timestamp: &ts,
blobs: &delta,
};
match crate::arweave::anchor_encrypted_manifest(
&http_client,
&irys_url,
&manifest,
)
.await
{
Ok(tx) if !tx.is_empty() => tracing::info!(
repo = %repo_slug,
tx_id = %tx,
"anchored encrypted manifest to Arweave"
),
Ok(_) => {}
Err(e) => tracing::warn!(
repo = %repo_slug,
err = %e,
"encrypted manifest anchor failed"
),
}
}
}
}
});
}
@@ -916,12 +980,11 @@ pub async fn git_receive_pack(
pub async fn list_refs(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let _record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (_record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let repo_slug = format!("{owner}/{repo}");
let refs = state.db.list_branch_cids(&repo_slug).await?;
@@ -1024,11 +1087,10 @@ pub async fn fork_repo(
Path((owner, name)): Path<(String, String)>,
Json(req): Json<ForkRepoRequest>,
) -> Result<(StatusCode, Json<RepoResponse>)> {
let source = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Enforce read visibility on the source before cloning: an unauthorized
// caller must not be able to fork (full mirror) a repo they cannot read.
let (source, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
let fork_name = req.name.unwrap_or_else(|| source.name.clone());
let forker_did = auth.0;
+175
View File
@@ -103,6 +103,108 @@ pub async fn anchor_ref_update(
Ok(tx_id)
}
/// A per-push manifest of the blobs encrypted this push (Option B3). The
/// `blobs` slice is `(oid, cid)` tuples. Anchored directly to Arweave as its JSON
/// body so the discovery index survives total node loss. Recipient identities are
/// never part of the manifest.
pub struct EncryptedManifest<'a> {
pub repo: &'a str,
pub owner_did: &'a str,
pub node_did: &'a str,
pub timestamp: &'a str,
pub blobs: &'a [(String, String)],
}
/// Anchor a per-push encrypted-blob manifest to Arweave via Irys. The manifest
/// JSON body is the payload (not a CID pointer to IPFS), so the index is
/// permanent and self-contained. Recipient identities are deliberately omitted:
/// the anchor is permanent and public, and the v2 envelopes no longer expose
/// recipients, so the reader set must not be written to Arweave either.
///
/// Returns the Irys/Arweave transaction ID, or `Ok("")` when `irys_url` is empty
/// (anchoring disabled) or there are no blobs to anchor.
pub async fn anchor_encrypted_manifest(
client: &reqwest::Client,
irys_url: &str,
manifest: &EncryptedManifest<'_>,
) -> Result<String> {
if irys_url.is_empty() || manifest.blobs.is_empty() {
return Ok(String::new());
}
let blobs_json: Vec<serde_json::Value> = manifest
.blobs
.iter()
.map(|(oid, cid)| manifest_blob_json(oid, cid))
.collect();
let payload = json!({
"schema": "gitlawb/encrypted-manifest/v1",
"repo": manifest.repo,
"owner_did": manifest.owner_did,
"node_did": manifest.node_did,
"timestamp": manifest.timestamp,
"blobs": blobs_json,
});
let body = serde_json::to_vec(&payload)?;
let url = format!("{}/upload", irys_url.trim_end_matches('/'));
let resp = client
.post(&url)
.header("Content-Type", "application/json")
.header("x-irys-tags", build_manifest_tags_header(manifest))
.body(body)
.send()
.await
.map_err(|e| anyhow::anyhow!("Irys upload failed: {e}"))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
return Err(anyhow::anyhow!("Irys returned {status}: {body}"));
}
let json: serde_json::Value = resp
.json()
.await
.map_err(|e| anyhow::anyhow!("failed to parse Irys response: {e}"))?;
let tx_id = json["id"]
.as_str()
.ok_or_else(|| anyhow::anyhow!("no 'id' in Irys response: {json}"))?
.to_string();
tracing::info!(
repo = %manifest.repo,
tx_id = %tx_id,
blobs = manifest.blobs.len(),
"anchored encrypted manifest to Arweave"
);
Ok(tx_id)
}
/// Serialize one blob for the Arweave manifest. Recipient identities are
/// intentionally absent so the permanent public anchor never records who can
/// read a blob.
fn manifest_blob_json(oid: &str, cid: &str) -> serde_json::Value {
json!({ "oid": oid, "cid": cid })
}
/// Build the Irys tag header for an encrypted-blob manifest. `Repo` and `Schema`
/// are the tags the `gl` recovery query filters on.
fn build_manifest_tags_header(manifest: &EncryptedManifest<'_>) -> String {
[
"App-Name:gitlawb".to_string(),
"Schema:gitlawb/encrypted-manifest/v1".to_string(),
format!("Repo:{}", sanitize_tag(manifest.repo)),
format!("Owner-DID:{}", sanitize_tag(manifest.owner_did)),
format!("Node-DID:{}", sanitize_tag(manifest.node_did)),
]
.join(",")
}
/// Arweave permanent URL for a given Irys transaction ID.
pub fn arweave_url(tx_id: &str) -> String {
format!("https://arweave.net/{tx_id}")
@@ -193,6 +295,79 @@ mod tests {
);
}
#[tokio::test]
async fn test_manifest_anchor_noop_when_url_empty() {
let client = reqwest::Client::new();
let blobs = vec![("oid1".to_string(), "cid1".to_string())];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
assert_eq!(
anchor_encrypted_manifest(&client, "", &m).await.unwrap(),
""
);
}
#[tokio::test]
async fn test_manifest_anchor_noop_when_no_blobs() {
let client = reqwest::Client::new();
let blobs: Vec<(String, String)> = vec![];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
// Non-empty URL, but no blobs: still a no-op.
assert_eq!(
anchor_encrypted_manifest(&client, "https://example.invalid", &m)
.await
.unwrap(),
""
);
}
#[tokio::test]
async fn test_manifest_anchor_success() {
let mut server = mockito::Server::new_async().await;
let _mock = server
.mock("POST", "/upload")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"id":"MANIFESTTX123","timestamp":1710000000000,"version":"1.0.0"}"#)
.create_async()
.await;
let client = reqwest::Client::new();
let blobs = vec![("oid1".to_string(), "cid1".to_string())];
let m = EncryptedManifest {
repo: "alice/r",
owner_did: "did:key:zO",
node_did: "did:key:zN",
timestamp: "2026-06-11T00:00:00Z",
blobs: &blobs,
};
let r = anchor_encrypted_manifest(&client, &server.url(), &m).await;
assert_eq!(r.unwrap(), "MANIFESTTX123");
_mock.assert_async().await;
}
#[test]
fn manifest_blob_json_omits_recipients() {
let v = manifest_blob_json("oid1", "cidA");
assert_eq!(v["oid"], "oid1");
assert_eq!(v["cid"], "cidA");
assert!(
v.get("recipients").is_none(),
"Arweave manifest must not anchor recipient identities"
);
}
#[test]
fn test_sanitize_tag() {
assert_eq!(sanitize_tag("alice/myrepo"), "alice/myrepo");
+96
View File
@@ -720,6 +720,32 @@ const MIGRATIONS: &[Migration] = &[
"CREATE INDEX IF NOT EXISTS idx_visibility_rules_repo ON visibility_rules(repo_id)",
],
},
Migration {
version: 4,
name: "encrypted_blobs",
stmts: &[
r#"CREATE TABLE IF NOT EXISTS encrypted_blobs (
repo_id TEXT NOT NULL,
oid TEXT NOT NULL,
cid TEXT NOT NULL,
recipients TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (repo_id, oid)
)"#,
"CREATE INDEX IF NOT EXISTS idx_encrypted_blobs_repo ON encrypted_blobs(repo_id)",
],
},
Migration {
version: 5,
name: "encrypted_blobs_blind_recipients",
stmts: &[
// Replace the cleartext recipient DID list with an opaque, node-keyed
// tag used only to detect a recipient-set change. Existing rows get an
// empty tag and are re-sealed on the next push.
"ALTER TABLE encrypted_blobs DROP COLUMN IF EXISTS recipients",
"ALTER TABLE encrypted_blobs ADD COLUMN IF NOT EXISTS recipients_tag TEXT NOT NULL DEFAULT ''",
],
},
];
// ── Repos ─────────────────────────────────────────────────────────────────────
@@ -1628,6 +1654,76 @@ impl Db {
Ok(())
}
pub async fn record_encrypted_blob(
&self,
repo_id: &str,
oid: &str,
cid: &str,
recipients_tag: &str,
) -> Result<()> {
sqlx::query(
"INSERT INTO encrypted_blobs (repo_id, oid, cid, recipients_tag, created_at)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (repo_id, oid) DO UPDATE SET cid = EXCLUDED.cid, recipients_tag = EXCLUDED.recipients_tag",
)
.bind(repo_id)
.bind(oid)
.bind(cid)
.bind(recipients_tag)
.bind(Utc::now().to_rfc3339())
.execute(&self.pool)
.await?;
Ok(())
}
/// (oid, cid) for every encrypted blob in the repo, unscoped by caller. Used
/// by both the B2 replication view and B1 discovery. Recipient identities are
/// not stored, so authorization is the caller's repo readability, not a per
/// recipient check. Ciphertext metadata only.
pub async fn list_all_encrypted_blobs(&self, repo_id: &str) -> Result<Vec<(String, String)>> {
let rows = sqlx::query("SELECT oid, cid FROM encrypted_blobs WHERE repo_id = $1")
.bind(repo_id)
.fetch_all(&self.pool)
.await?;
let mut out = Vec::new();
for row in rows {
let oid: String = row.get("oid");
let cid: String = row.get("cid");
out.push((oid, cid));
}
Ok(out)
}
/// The CID of one encrypted blob, or None if there is no such row. Recipient
/// authorization is not enforced here: the handler checks repo readability and
/// the envelope crypto gates decryption.
pub async fn encrypted_blob_cid(&self, repo_id: &str, oid: &str) -> Result<Option<String>> {
let row = sqlx::query("SELECT cid FROM encrypted_blobs WHERE repo_id = $1 AND oid = $2")
.bind(repo_id)
.bind(oid)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|r| r.get("cid")))
}
/// The opaque recipients tag stored for an encrypted blob, or None if there is
/// no row. Used only to decide whether a re-seal is needed (the recipient set
/// changed); the tag is a node-keyed fingerprint, not the DID list.
pub async fn encrypted_blob_recipients_tag(
&self,
repo_id: &str,
oid: &str,
) -> Result<Option<String>> {
let row = sqlx::query(
"SELECT recipients_tag FROM encrypted_blobs WHERE repo_id = $1 AND oid = $2",
)
.bind(repo_id)
.bind(oid)
.fetch_optional(&self.pool)
.await?;
Ok(row.map(|r| r.get("recipients_tag")))
}
pub async fn list_pinned_cids(&self) -> Result<Vec<PinnedCidRecord>> {
let rows = sqlx::query(
"SELECT sha256_hex, cid, pinned_at, pinata_cid FROM pinned_cids ORDER BY pinned_at DESC",
+136
View File
@@ -0,0 +1,136 @@
//! Encrypt-then-pin for withheld blobs (Option B1). Each withheld blob is sealed
//! to its recipient DIDs and the envelope pinned to IPFS, recorded in
//! `encrypted_blobs`. Best-effort per blob: a failure is logged and skipped,
//! never pinned in plaintext.
use std::collections::{BTreeSet, HashMap};
use std::path::Path;
use std::str::FromStr;
use ed25519_dalek::VerifyingKey;
use gitlawb_core::did::Did;
use gitlawb_core::encrypt::seal_blob;
use crate::db::Db;
use hmac::{Hmac, Mac};
use sha2::Sha256;
type HmacSha256 = Hmac<Sha256>;
/// Opaque, node-keyed fingerprint of a blob's recipient set. Stored in place of
/// the cleartext DID list so a DB compromise cannot reveal the reader set; used
/// only to detect a recipient-set change so an unchanged blob is not re-sealed.
/// Order-insensitive (the input `BTreeSet` is already sorted).
pub fn recipients_tag(node_seed: &[u8; 32], dids: &BTreeSet<String>) -> String {
let mut mac = HmacSha256::new_from_slice(node_seed).expect("HMAC accepts any key length");
mac.update(b"gitlawb/recipients-tag/v1");
for did in dids {
mac.update(b"\n");
mac.update(did.as_bytes());
}
hex::encode(mac.finalize().into_bytes())
}
/// Resolve a DID string to its Ed25519 verifying key, or None if it carries no
/// inline key (e.g. did:web / did:gitlawb).
fn did_to_key(did: &str) -> Option<VerifyingKey> {
Did::from_str(did).ok()?.to_verifying_key().ok()
}
/// Encrypt and pin every withheld blob. `recipients` maps blob oid -> DID set;
/// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob
/// actually sealed and recorded this call (the per-push delta), used by Option B3
/// to anchor a manifest. Recipient identities are never stored or returned.
pub async fn encrypt_and_pin(
ipfs_api: &str,
repo_path: &Path,
db: &Db,
repo_id: &str,
node_seed: &[u8; 32],
recipients: &HashMap<String, BTreeSet<String>>,
) -> Vec<(String, String)> {
let mut sealed = Vec::new();
for (oid, dids) in recipients {
// Skip only if an existing envelope already covers exactly these
// recipients. If the recipient set changed (e.g. a reader was added to
// the rule), re-seal so the new reader can recover the blob. Reader
// removal is not retroactive: the old envelope is already public. The
// comparison is on the opaque node-keyed tag, never the DID list.
let tag = recipients_tag(node_seed, dids);
match db.encrypted_blob_recipients_tag(repo_id, oid).await {
Ok(Some(stored_tag)) if stored_tag == tag => continue,
Ok(_) => {}
Err(e) => {
// A DB read failure is not a cache miss: re-sealing here would do
// an avoidable IPFS write during a partial outage. Skip and retry
// on the next push.
tracing::warn!(oid = %oid, err = %e, "recipients_tag lookup failed; skipping reseal");
continue;
}
}
let keys: Vec<VerifyingKey> = dids.iter().filter_map(|d| did_to_key(d)).collect();
if keys.is_empty() {
tracing::warn!(oid = %oid, "no resolvable recipient keys; skipping encrypted pin");
continue;
}
let data = match crate::git::store::read_object(repo_path, oid) {
Ok(Some((_t, bytes))) => bytes,
_ => continue,
};
let envelope = match seal_blob(&data, &keys) {
Ok(e) => e,
Err(e) => {
tracing::warn!(oid = %oid, err = %e, "seal_blob failed; skipping");
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
Ok(c) if !c.is_empty() => c,
_ => continue,
};
if let Err(e) = db.record_encrypted_blob(repo_id, oid, &cid, &tag).await {
tracing::warn!(oid = %oid, err = %e, "record_encrypted_blob failed");
continue;
}
sealed.push((oid.clone(), cid.clone()));
}
sealed
}
#[cfg(test)]
mod tests {
use super::recipients_tag;
use std::collections::BTreeSet;
fn set(dids: &[&str]) -> BTreeSet<String> {
dids.iter().map(|s| s.to_string()).collect()
}
#[test]
fn tag_is_order_insensitive() {
let seed = [7u8; 32];
let a = recipients_tag(&seed, &set(&["did:key:zA", "did:key:zB"]));
let b = recipients_tag(&seed, &set(&["did:key:zB", "did:key:zA"]));
assert_eq!(a, b);
}
#[test]
fn tag_differs_for_different_sets() {
let seed = [7u8; 32];
let a = recipients_tag(&seed, &set(&["did:key:zA"]));
let b = recipients_tag(&seed, &set(&["did:key:zA", "did:key:zB"]));
assert_ne!(a, b);
}
#[test]
fn tag_is_keyed_by_node_seed() {
let dids = set(&["did:key:zA", "did:key:zB"]);
let a = recipients_tag(&[1u8; 32], &dids);
let b = recipients_tag(&[2u8; 32], &dids);
assert_ne!(
a, b,
"tag must depend on the node seed, not be a plain hash"
);
}
}
+83
View File
@@ -314,6 +314,39 @@ pub fn branch_diff(repo_path: &Path, target_branch: &str, source_branch: &str) -
Ok(String::from_utf8_lossy(&output.stdout).to_string())
}
/// The repo-relative paths changed by `git diff target...source` (the same range
/// as `branch_diff`). Used to enforce per-path visibility on a PR diff: if the
/// caller cannot read one of these paths, the diff is withheld.
pub fn branch_diff_names(
repo_path: &Path,
target_branch: &str,
source_branch: &str,
) -> Result<Vec<String>> {
let output = Command::new("git")
.args([
"diff",
"--name-only",
"-z",
&format!("{target_branch}...{source_branch}"),
])
.current_dir(repo_path)
.output()
.context("failed to run git diff --name-only")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git diff --name-only failed: {stderr}");
}
// Split on NUL (`-z`) so paths containing newlines keep their exact bytes;
// `--name-only` without `-z` would quote/escape such paths and they would no
// longer match the visibility globs in get_pr_diff, leaking the diff.
Ok(output
.stdout
.split(|b| *b == b'\0')
.filter(|s| !s.is_empty())
.map(|s| String::from_utf8_lossy(s).into_owned())
.collect())
}
/// Merge source_branch into target_branch in a bare repo using a temporary worktree.
/// Returns the new merge commit hash.
pub fn merge_branch(
@@ -399,3 +432,53 @@ pub fn repo_disk_path(repos_dir: &Path, owner_did: &str, repo_name: &str) -> Pat
let owner_slug = owner_did.replace([':', '/'], "_");
repos_dir.join(owner_slug).join(format!("{repo_name}.git"))
}
#[cfg(test)]
mod tests {
use super::branch_diff_names;
use std::path::Path;
use std::process::Command;
#[test]
fn branch_diff_names_lists_changed_paths() {
let td = tempfile::TempDir::new().unwrap();
let work: &Path = td.path();
let g = |args: &[&str]| {
assert!(Command::new("git")
.args(args)
.current_dir(work)
.status()
.unwrap()
.success());
};
g(&["init", "-q"]);
g(&["config", "user.email", "t@t"]);
g(&["config", "user.name", "t"]);
std::fs::write(work.join("base.txt"), b"base\n").unwrap();
g(&["add", "."]);
g(&["commit", "-qm", "base"]);
let main = {
let o = Command::new("git")
.args(["symbolic-ref", "--short", "HEAD"])
.current_dir(work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
g(&["checkout", "-q", "-b", "feature"]);
std::fs::create_dir_all(work.join("secret")).unwrap();
std::fs::write(work.join("secret/x.txt"), b"secret\n").unwrap();
g(&["add", "."]);
g(&["commit", "-qm", "feat"]);
let names = branch_diff_names(work, &main, "feature").unwrap();
assert!(
names.iter().any(|p| p == "secret/x.txt"),
"expected secret/x.txt in changed paths, got {names:?}"
);
assert!(
!names.iter().any(|p| p == "base.txt"),
"unchanged file must not appear: {names:?}"
);
}
}
+66 -1
View File
@@ -7,7 +7,7 @@ use crate::db::VisibilityRule;
use crate::git::store;
use crate::visibility::{visibility_check, Decision};
use anyhow::{Context, Result};
use std::collections::HashSet;
use std::collections::{BTreeSet, HashMap, HashSet};
use std::path::Path;
/// List every (blob_oid, "/repo/relative/path") pair reachable from any branch
@@ -87,6 +87,42 @@ pub fn replicable_objects(all: Vec<String>, withheld: &HashSet<String>) -> Vec<S
.collect()
}
/// For every blob withheld from anonymous, the DIDs allowed to read it: the
/// owner plus any reader DID that `visibility_check` Allows at some path the
/// blob appears at. Least-privilege: a reader of one private subtree is not a
/// recipient of a blob that only lives in another.
pub fn withheld_blob_recipients(
repo_path: &Path,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
) -> Result<HashMap<String, BTreeSet<String>>> {
let withheld = withheld_blob_oids(repo_path, rules, is_public, owner_did, None)?;
if withheld.is_empty() {
return Ok(HashMap::new());
}
let mut candidates: BTreeSet<String> = BTreeSet::new();
for r in rules {
for d in &r.reader_dids {
candidates.insert(d.clone());
}
}
let mut out: HashMap<String, BTreeSet<String>> = HashMap::new();
for (oid, path) in blob_paths(repo_path)? {
if !withheld.contains(&oid) {
continue;
}
let entry = out.entry(oid).or_default();
entry.insert(owner_did.to_string());
for did in &candidates {
if visibility_check(rules, is_public, owner_did, Some(did), &path) == Decision::Allow {
entry.insert(did.clone());
}
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -230,4 +266,33 @@ mod tests {
let got = replicable_objects(all.clone(), &withheld);
assert_eq!(got, all);
}
#[test]
fn recipients_are_owner_plus_allowed_readers_only() {
let (_td, repo, secret_oid, public_oid) = fixture();
let reader = "did:key:zReader";
let rules = vec![rule("/secret/**", &[reader])];
let map = withheld_blob_recipients(&repo, &rules, true, OWNER).unwrap();
let recips = map.get(&secret_oid).expect("secret blob has recipients");
assert!(recips.contains(OWNER));
assert!(recips.contains(reader));
assert!(
!map.contains_key(&public_oid),
"public blob is not encrypted"
);
}
#[test]
fn node_seal_open_round_trip() {
use gitlawb_core::encrypt::{open_blob, seal_blob};
use gitlawb_core::identity::Keypair;
let (_td, repo, secret_oid, _public) = fixture();
let (_t, bytes) = crate::git::store::read_object(&repo, &secret_oid)
.unwrap()
.unwrap();
let reader = Keypair::generate();
let env = seal_blob(&bytes, &[reader.verifying_key()]).unwrap();
assert_eq!(open_blob(&env, &reader).unwrap(), bytes);
}
}
+13
View File
@@ -72,6 +72,19 @@ pub async fn pin_git_object(ipfs_api: &str, sha256_hex: &str, data: &[u8]) -> Re
Ok(cid)
}
/// Fetch raw bytes for a CID from the local Kubo node (`/api/v0/cat`).
pub async fn cat(ipfs_api: &str, cid: &str) -> Result<Vec<u8>> {
if ipfs_api.is_empty() {
return Err(anyhow::anyhow!("IPFS not configured"));
}
let url = format!("{}/api/v0/cat?arg={}", ipfs_api.trim_end_matches('/'), cid);
let resp = reqwest::Client::new().post(&url).send().await?;
if !resp.status().is_success() {
return Err(anyhow::anyhow!("ipfs cat {cid}: {}", resp.status()));
}
Ok(resp.bytes().await?.to_vec())
}
/// List all git objects in the given bare repo and pin any that are not yet
/// recorded in `pinned_cids`.
///
+1
View File
@@ -5,6 +5,7 @@ mod bootstrap;
mod cert;
mod config;
mod db;
mod encrypted_pin;
mod error;
mod git;
mod graphql;
+14 -1
View File
@@ -342,7 +342,8 @@ pub fn build_router(state: AppState) -> Router {
.route(
"/api/v1/repos/{owner}/{repo}/replicas",
get(replicas::list_replicas),
);
)
.layer(middleware::from_fn(auth::optional_signature));
// git-upload-pack (clone/fetch) — same raised body limit as receive-pack so
// large pack responses from the server don't get truncated on the client side.
@@ -356,6 +357,18 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/repos/{owner}/{repo}/withheld-paths",
axum::routing::get(visibility::withheld_paths),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blobs",
axum::routing::get(crate::api::encrypted::list_encrypted_blobs),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blob/{oid}",
axum::routing::get(crate::api::encrypted::get_encrypted_blob),
)
.route(
"/api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate",
axum::routing::get(crate::api::encrypted::replicate_encrypted_blobs),
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit))
.layer(middleware::from_fn(auth::optional_signature));
+180
View File
@@ -12,6 +12,7 @@
//! 5. On success, register ourselves as a replica with the origin node so
//! its `replica_count` reflects reality (best-effort, idempotent).
use std::collections::HashMap;
use std::path::Path;
use std::sync::Arc;
@@ -47,6 +48,44 @@ fn classify_mirror(withheld: Option<Vec<String>>) -> MirrorMode {
}
}
/// One encrypted blob as advertised by an origin's `encrypted-blobs/replicate`
/// endpoint (Option B2). Ciphertext metadata only; recipient identities are
/// withheld from peers, so a re-seal is detected by the CID changing.
#[derive(Debug, Clone, PartialEq, Eq, serde::Deserialize)]
struct ReplicaBlob {
oid: String,
cid: String,
}
/// The shape of the `encrypted-blobs/replicate` JSON response.
#[derive(Debug, serde::Deserialize)]
struct ReplicateResponse {
#[serde(default)]
blobs: Vec<ReplicaBlob>,
}
/// Decide which of the origin's encrypted blobs this mirror must (re)replicate.
///
/// `have` maps each already-stored blob's oid to the CID the mirror pinned. A
/// remote blob is returned when the mirror has no row for that oid, or when the
/// stored CID differs from the advertised one. A re-seal regenerates the
/// envelope (new content key, nonce, and per-recipient wraps), so the CID
/// changes while the OID stays stable; comparing CIDs detects a re-seal without
/// the mirror ever holding recipient identities.
fn blobs_needing_replication(
remote: &[ReplicaBlob],
have: &HashMap<String, String>,
) -> Vec<ReplicaBlob> {
remote
.iter()
.filter(|b| match have.get(&b.oid) {
None => true,
Some(stored_cid) => stored_cid != &b.cid,
})
.cloned()
.collect()
}
/// Start the background sync worker. Returns immediately; the worker runs
/// as a detached tokio task that exits cleanly when `shutdown_rx` flips
/// to `true`.
@@ -167,6 +206,20 @@ async fn process_batch(
machine_id,
)
.await;
// Option B2: carry the encrypted withheld-blob envelopes too, so an
// authorized reader can recover private content from this mirror if
// the origin dies. `item.repo` is the slug "{owner_short}/{name}",
// which is the id upsert_mirror_repo wrote (the local repo_id).
replicate_encrypted_blobs(
client,
&origin_url,
owner_short,
repo_name,
db,
&item.repo,
&config.ipfs_api,
)
.await;
let _ = db.mark_sync_done(&item.id).await;
crate::metrics::record_sync_processed("done");
@@ -277,6 +330,81 @@ async fn register_replica_with_origin(
}
}
/// Replicate the origin's encrypted withheld blobs onto this mirror (Option B2).
///
/// After the git objects are mirrored, fetch the origin's replication listing,
/// then for each blob the mirror does not already hold (or whose CID changed,
/// i.e. the origin re-sealed) pull the ciphertext envelope over IPFS, pin it
/// locally, and record the `encrypted_blobs` row keyed by this mirror's local
/// `repo_id`. The mirror stores no recipient identities.
///
/// Best-effort and idempotent: any per-blob failure is logged and skipped, to be
/// retried on the next sync. Confidentiality is never at risk; the mirror only
/// ever handles ciphertext and never decrypts. Cleanly a no-op when IPFS is
/// unconfigured, the origin reports no encrypted blobs, or the replicate endpoint
/// is absent (older peer) or unreachable.
async fn replicate_encrypted_blobs(
client: &reqwest::Client,
origin_url: &str,
owner: &str,
repo: &str,
db: &Db,
repo_id: &str,
ipfs_api: &str,
) {
if ipfs_api.is_empty() {
return;
}
let url = format!("{origin_url}/api/v1/repos/{owner}/{repo}/encrypted-blobs/replicate");
let resp = match client.get(&url).send().await {
Ok(r) if r.status().is_success() => r,
_ => return,
};
let parsed: ReplicateResponse = match resp.json().await {
Ok(p) => p,
Err(e) => {
warn!(repo = %repo, err = %e, "failed to parse encrypted-blobs/replicate response");
return;
}
};
if parsed.blobs.is_empty() {
return;
}
let have: HashMap<String, String> = match db.list_all_encrypted_blobs(repo_id).await {
Ok(rows) => rows.into_iter().collect(),
Err(e) => {
warn!(repo = %repo, err = %e, "failed to list local encrypted blobs for replication");
return;
}
};
for blob in blobs_needing_replication(&parsed.blobs, &have) {
let envelope = match crate::ipfs_pin::cat(ipfs_api, &blob.cid).await {
Ok(bytes) => bytes,
Err(e) => {
warn!(oid = %blob.oid, cid = %blob.cid, err = %e, "failed to fetch encrypted envelope over IPFS; will retry next sync");
continue;
}
};
match crate::ipfs_pin::pin_git_object(ipfs_api, &blob.oid, &envelope).await {
Ok(cid) if !cid.is_empty() => {
if cid != blob.cid {
warn!(oid = %blob.oid, expected = %blob.cid, got = %cid, "replicated envelope CID mismatch; skipping record");
continue;
}
if let Err(e) = db.record_encrypted_blob(repo_id, &blob.oid, &cid, "").await {
warn!(oid = %blob.oid, err = %e, "failed to record replicated encrypted blob");
}
}
_ => {
warn!(oid = %blob.oid, "failed to pin replicated encrypted envelope; will retry next sync");
}
}
}
}
/// Run a git subprocess, returning an error with stderr on non-zero exit.
async fn git_run(args: &[&str]) -> anyhow::Result<()> {
let out = tokio::process::Command::new("git")
@@ -423,6 +551,58 @@ mod tests {
assert!(matches!(mode, MirrorMode::Plain));
}
fn rb(oid: &str, cid: &str) -> ReplicaBlob {
ReplicaBlob {
oid: oid.to_string(),
cid: cid.to_string(),
}
}
#[test]
fn replicate_stores_new_blob() {
let remote = vec![rb("oid1", "cidA")];
let have = HashMap::new();
assert_eq!(blobs_needing_replication(&remote, &have), remote);
}
#[test]
fn replicate_skips_already_present_same_cid() {
let remote = vec![rb("oid1", "cidA")];
let mut have = HashMap::new();
have.insert("oid1".to_string(), "cidA".to_string());
assert!(blobs_needing_replication(&remote, &have).is_empty());
}
#[test]
fn replicate_restores_on_cid_change() {
// The origin re-sealed: same oid, new envelope, new cid.
let remote = vec![rb("oid1", "cidB")];
let mut have = HashMap::new();
have.insert("oid1".to_string(), "cidA".to_string());
assert_eq!(blobs_needing_replication(&remote, &have), remote);
}
#[test]
fn replicate_empty_remote_is_noop() {
assert!(blobs_needing_replication(&[], &HashMap::new()).is_empty());
}
#[test]
fn replicate_response_parses() {
// An older origin may still send a recipients field; it must be ignored.
let json = r#"{"blobs":[{"oid":"o1","cid":"c1","recipients":["did:key:zA"]}]}"#;
let parsed: ReplicateResponse = serde_json::from_str(json).unwrap();
assert_eq!(parsed.blobs.len(), 1);
assert_eq!(parsed.blobs[0].oid, "o1");
assert_eq!(parsed.blobs[0].cid, "c1");
}
#[test]
fn replicate_response_empty_blobs_parses() {
let parsed: ReplicateResponse = serde_json::from_str(r#"{"blobs":[]}"#).unwrap();
assert!(parsed.blobs.is_empty());
}
fn g(args: &[&str], dir: &Path) {
assert!(Command::new("git")
.args(args)
+1
View File
@@ -3,6 +3,7 @@ name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
+664
View File
@@ -29,6 +29,23 @@ pub struct CloneArgs {
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
pub node: String,
/// Arweave gateway for B3 manifest discovery/fetch when a node cannot supply
/// the encrypted-blob mapping.
#[arg(
long,
default_value = "https://arweave.net",
env = "GITLAWB_ARWEAVE_GATEWAY"
)]
pub arweave_gateway: String,
/// Public IPFS gateway for fetching encrypted envelopes during B3 recovery.
#[arg(
long,
default_value = "https://dweb.link",
env = "GITLAWB_IPFS_GATEWAY"
)]
pub ipfs_gateway: String,
}
/// Run a git command inside `dir`, erroring with stderr on failure.
@@ -217,6 +234,294 @@ struct WithheldPathsResponse {
reinclude: Vec<String>,
}
/// After the base clone, recover encrypted blobs the caller is authorized for
/// that are missing locally: fetch the envelope, decrypt with the caller's key,
/// install as a loose object. Returns the repo-relative paths recovered.
/// Best-effort; logs and continues on any per-blob failure.
async fn recover_encrypted_blobs(
node: &str,
owner: &str,
name: &str,
dest: &Path,
keypair: &gitlawb_core::identity::Keypair,
) -> Result<Vec<String>> {
use gitlawb_core::encrypt::open_blob;
use std::collections::HashMap;
use std::io::Write;
let dest_str = dest.to_str().context("dest path not utf-8")?;
let client = NodeClient::new(node, Some(keypair.clone()));
let resp = match client
.get_signed(&format!("/api/v1/repos/{owner}/{name}/encrypted-blobs"))
.await
{
Ok(r) if r.status().is_success() => r,
_ => return Ok(vec![]),
};
let body: serde_json::Value = resp.json().await.context("parsing encrypted-blobs")?;
let blobs = body
.get("blobs")
.and_then(|b| b.as_array())
.cloned()
.unwrap_or_default();
if blobs.is_empty() {
return Ok(vec![]);
}
// Map oid -> repo-relative path from the cloned tree.
let ls = Command::new("git")
.args(["-C", dest_str, "ls-tree", "-r", "HEAD"])
.output()?;
let mut oid_to_path: HashMap<String, String> = HashMap::new();
for line in String::from_utf8_lossy(&ls.stdout).lines() {
if let Some((meta, path)) = line.split_once('\t') {
if let Some(oid) = meta.split_whitespace().nth(2) {
oid_to_path.insert(oid.to_string(), path.to_string());
}
}
}
let mut recovered = Vec::new();
for entry in blobs {
let Some(oid) = entry.get("oid").and_then(|o| o.as_str()) else {
continue;
};
// Skip if already present locally.
let present = Command::new("git")
.args(["-C", dest_str, "cat-file", "-e", oid])
.status()
.map(|s| s.success())
.unwrap_or(false);
if present {
continue;
}
let env_resp = match client
.get_signed(&format!(
"/api/v1/repos/{owner}/{name}/encrypted-blob/{oid}"
))
.await
{
Ok(r) if r.status().is_success() => r,
_ => continue,
};
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
Err(e) => {
eprintln!("warning: could not decrypt {oid}: {e}");
continue;
}
};
// Install as a loose object; verify the OID matches.
let mut child = Command::new("git")
.args(["-C", dest_str, "hash-object", "-w", "-t", "blob", "--stdin"])
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.spawn()?;
child.stdin.take().unwrap().write_all(&plaintext)?;
let out = child.wait_with_output()?;
let written = String::from_utf8_lossy(&out.stdout).trim().to_string();
if written == oid {
if let Some(p) = oid_to_path.get(oid) {
recovered.push(p.clone());
}
} else {
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
}
}
Ok(recovered)
}
/// One blob entry in an Arweave-anchored encrypted manifest. The manifest also
/// carries a `recipients` field per blob, but `gl` does not need it: authorization
/// is enforced by whether `open_blob` can decrypt with the caller's key. Unknown
/// JSON fields are ignored by serde, so `recipients` is simply not declared here.
#[derive(Deserialize)]
struct ManifestBlob {
oid: String,
cid: String,
}
/// An Arweave-anchored per-push encrypted manifest (Option B3).
#[derive(Deserialize)]
struct Manifest {
#[serde(default)]
timestamp: String,
#[serde(default)]
blobs: Vec<ManifestBlob>,
}
/// Extract transaction ids from an Arweave GraphQL `transactions` response.
fn parse_tx_ids(v: &serde_json::Value) -> Vec<String> {
v.get("data")
.and_then(|d| d.get("transactions"))
.and_then(|t| t.get("edges"))
.and_then(|e| e.as_array())
.map(|edges| {
edges
.iter()
.filter_map(|edge| {
edge.get("node")
.and_then(|n| n.get("id"))
.and_then(|i| i.as_str())
.map(String::from)
})
.collect()
})
.unwrap_or_default()
}
/// Merge per-push manifests into a single `oid -> cid` map, latest-wins by the
/// manifest `timestamp` (RFC3339, compared lexicographically; a later push that
/// re-sealed a blob overrides the earlier entry).
fn merge_manifests(manifests: Vec<Manifest>) -> std::collections::HashMap<String, String> {
let mut best: std::collections::HashMap<String, (String, String)> =
std::collections::HashMap::new(); // oid -> (cid, timestamp)
for m in manifests {
for b in m.blobs {
match best.get(&b.oid) {
Some((_, ts)) if ts.as_str() >= m.timestamp.as_str() => {}
_ => {
best.insert(b.oid, (b.cid, m.timestamp.clone()));
}
}
}
}
best.into_iter().map(|(oid, (cid, _))| (oid, cid)).collect()
}
/// Option B3 fallback recovery, with no dependency on a gitlawb node API. Query
/// the Arweave gateway for this repo's encrypted manifests, merge them, and for
/// each blob still missing locally that the caller can decrypt, pull the envelope
/// from a public IPFS gateway, decrypt, and install it as a loose object. Returns
/// the repo-relative paths recovered. Best-effort; silent when gateways are
/// unreachable, leaving the clone exactly as node-based recovery left it.
async fn recover_from_arweave(
arweave_gateway: &str,
ipfs_gateway: &str,
owner: &str,
name: &str,
dest: &Path,
keypair: &gitlawb_core::identity::Keypair,
) -> Result<Vec<String>> {
use gitlawb_core::encrypt::open_blob;
use std::collections::HashMap;
use std::io::Write;
let dest_str = dest.to_str().context("dest path not utf-8")?;
let owner_short = owner.split(':').next_back().unwrap_or(owner);
let slug = format!("{owner_short}/{name}");
let ag = arweave_gateway.trim_end_matches('/');
let ig = ipfs_gateway.trim_end_matches('/');
// Bound every gateway request: this runs on every clone, so a slow or hung
// public gateway must not stall it. Best-effort recovery, so a timeout just
// skips the affected blob.
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.unwrap_or_else(|_| reqwest::Client::new());
// 1. Discover manifest transaction ids via Arweave GraphQL.
let query = r#"query($repo:String!){transactions(tags:[{name:"App-Name",values:["gitlawb"]},{name:"Schema",values:["gitlawb/encrypted-manifest/v1"]},{name:"Repo",values:[$repo]}],first:100){edges{node{id}}}}"#;
let gql_body = serde_json::json!({ "query": query, "variables": { "repo": slug } });
let resp = match client
.post(format!("{ag}/graphql"))
.json(&gql_body)
.send()
.await
{
Ok(r) if r.status().is_success() => r,
_ => return Ok(vec![]),
};
let gql: serde_json::Value = match resp.json().await {
Ok(v) => v,
Err(_) => return Ok(vec![]),
};
let tx_ids = parse_tx_ids(&gql);
if tx_ids.is_empty() {
return Ok(vec![]);
}
// 2. Fetch and parse each manifest body, then merge latest-wins per oid.
let mut manifests = Vec::new();
for tx in tx_ids {
let m = match client.get(format!("{ag}/{tx}")).send().await {
Ok(r) if r.status().is_success() => r,
_ => continue,
};
if let Ok(parsed) = m.json::<Manifest>().await {
manifests.push(parsed);
}
}
let oid_cid = merge_manifests(manifests);
if oid_cid.is_empty() {
return Ok(vec![]);
}
// Map oid -> repo-relative path from the cloned tree.
let ls = Command::new("git")
.args(["-C", dest_str, "ls-tree", "-r", "HEAD"])
.output()?;
let mut oid_to_path: HashMap<String, String> = HashMap::new();
for line in String::from_utf8_lossy(&ls.stdout).lines() {
if let Some((meta, path)) = line.split_once('\t') {
if let Some(oid) = meta.split_whitespace().nth(2) {
oid_to_path.insert(oid.to_string(), path.to_string());
}
}
}
// 3. Recover each missing blob the caller can decrypt.
let mut recovered = Vec::new();
for (oid, cid) in oid_cid {
// Local presence check. GIT_NO_LAZY_FETCH stops git from making a wasted
// promisor fetch attempt (we are recovering precisely because the promisor
// cannot supply the blob), and `.output()` captures git's "missing object"
// stderr so that expected case does not leak a confusing error to the user.
let present = Command::new("git")
.args(["-C", dest_str, "cat-file", "-e", &oid])
.env("GIT_NO_LAZY_FETCH", "1")
.output()
.map(|o| o.status.success())
.unwrap_or(false);
if present {
continue;
}
let env_resp = match client.get(format!("{ig}/ipfs/{cid}")).send().await {
Ok(r) if r.status().is_success() => r,
_ => continue,
};
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
// open_blob succeeds only if this caller is a recipient: this is the
// authorization gate (no node, no DID check needed).
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
Err(_) => continue,
};
let mut child = Command::new("git")
.args(["-C", dest_str, "hash-object", "-w", "-t", "blob", "--stdin"])
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.spawn()?;
child.stdin.take().unwrap().write_all(&plaintext)?;
let out = child.wait_with_output()?;
let written = String::from_utf8_lossy(&out.stdout).trim().to_string();
if written == oid {
if let Some(p) = oid_to_path.get(&oid) {
recovered.push(p.clone());
}
} else {
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
}
}
Ok(recovered)
}
pub async fn run(args: CloneArgs) -> Result<()> {
let (url, owner, name) = parse_repo(&args.repo)?;
let dest_name = args.dir.unwrap_or_else(|| name.clone());
@@ -236,6 +541,60 @@ pub async fn run(args: CloneArgs) -> Result<()> {
}
setup_partial_clone(&dest, &url, &withheld, &reinclude, args.branch.as_deref())?;
if let Ok(keypair) = load_keypair_from_dir(None) {
// Node-based recovery first (B1/B2), then the B3 Arweave/IPFS gateway
// fallback for any authorized blobs the node could not supply.
let mut paths = recover_encrypted_blobs(&args.node, &owner, &name, &dest, &keypair)
.await
.unwrap_or_default();
let from_arweave = recover_from_arweave(
&args.arweave_gateway,
&args.ipfs_gateway,
&owner,
&name,
&dest,
&keypair,
)
.await
.unwrap_or_default();
paths.extend(from_arweave);
if !paths.is_empty() {
// Re-include recovered paths if this was a sparse clone, then
// materialize them in the working tree.
let spec = dest.join(".git/info/sparse-checkout");
if spec.exists() {
match std::fs::read_to_string(&spec) {
Ok(mut s) => {
for p in &paths {
s.push_str(&format!("/{p}\n"));
}
if let Err(e) = std::fs::write(&spec, &s) {
eprintln!(
"warning: failed to update sparse-checkout, recovered files may not appear: {e}"
);
}
}
Err(e) => {
eprintln!(
"warning: failed to read sparse-checkout, recovered files may not appear: {e}"
);
}
}
}
if let Err(e) = git(&dest, &["checkout", "--", "."]) {
eprintln!(
"warning: checkout after recovery failed, recovered files may not appear: {e}"
);
}
println!(
"Recovered {} private file(s) you are authorized to read",
paths.len()
);
}
}
println!("Done. Cloned into {dest_name}");
Ok(())
}
@@ -428,6 +787,269 @@ mod tests {
.is_err());
}
#[test]
fn parse_tx_ids_extracts_node_ids() {
let v: serde_json::Value = serde_json::from_str(
r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}},{"node":{"id":"TX2"}}]}}}"#,
)
.unwrap();
assert_eq!(parse_tx_ids(&v), vec!["TX1".to_string(), "TX2".to_string()]);
}
#[test]
fn parse_tx_ids_empty_on_no_edges() {
let v: serde_json::Value =
serde_json::from_str(r#"{"data":{"transactions":{"edges":[]}}}"#).unwrap();
assert!(parse_tx_ids(&v).is_empty());
}
#[test]
fn manifest_parses_and_ignores_recipients() {
let m: Manifest = serde_json::from_str(
r#"{"timestamp":"2026-06-11T00:00:00Z","blobs":[{"oid":"o1","cid":"c1","recipients":["did:key:zA"]}]}"#,
)
.unwrap();
assert_eq!(m.timestamp, "2026-06-11T00:00:00Z");
assert_eq!(m.blobs.len(), 1);
assert_eq!(m.blobs[0].oid, "o1");
assert_eq!(m.blobs[0].cid, "c1");
}
#[test]
fn merge_manifests_latest_wins_per_oid() {
let older = Manifest {
timestamp: "2026-06-10T00:00:00Z".to_string(),
blobs: vec![ManifestBlob {
oid: "o1".to_string(),
cid: "cidOLD".to_string(),
}],
};
let newer = Manifest {
timestamp: "2026-06-11T00:00:00Z".to_string(),
blobs: vec![
ManifestBlob {
oid: "o1".to_string(),
cid: "cidNEW".to_string(),
},
ManifestBlob {
oid: "o2".to_string(),
cid: "cid2".to_string(),
},
],
};
let merged = merge_manifests(vec![older, newer]);
assert_eq!(merged.get("o1").map(String::as_str), Some("cidNEW"));
assert_eq!(merged.get("o2").map(String::as_str), Some("cid2"));
}
#[test]
fn merge_manifests_is_order_independent() {
let older = Manifest {
timestamp: "2026-06-10T00:00:00Z".to_string(),
blobs: vec![ManifestBlob {
oid: "o1".to_string(),
cid: "cidOLD".to_string(),
}],
};
let newer = Manifest {
timestamp: "2026-06-11T00:00:00Z".to_string(),
blobs: vec![ManifestBlob {
oid: "o1".to_string(),
cid: "cidNEW".to_string(),
}],
};
// Newer first, older second: newer must still win.
let merged = merge_manifests(vec![newer, older]);
assert_eq!(merged.get("o1").map(String::as_str), Some("cidNEW"));
}
/// Read-path end-to-end over a mocked Arweave + IPFS gateway: discover the
/// manifest via GraphQL, fetch it, fetch the envelope, decrypt with the
/// caller's key, and install the previously-withheld blob.
#[tokio::test]
async fn recover_from_arweave_installs_authorized_blob() {
use gitlawb_core::encrypt::seal_blob;
use gitlawb_core::identity::Keypair;
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n"), ("secret/b.txt", b"SECRET\n")]);
let dest = td.path().join("dest");
// Make the bare honor `--filter=blob:none` over file:// so the withheld
// blob is genuinely omitted from the local store, not just unchecked-out.
let bare = url.strip_prefix("file://").unwrap();
assert!(Command::new("git")
.args(["-C", bare, "config", "uploadpack.allowFilter", "true"])
.status()
.unwrap()
.success());
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
assert!(
!dest.join("secret/b.txt").exists(),
"secret starts withheld"
);
let oid = {
let out = Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
"HEAD:secret/b.txt",
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
// Simulate origin death: drop the promisor remote so `cat-file -e` cannot
// lazily fetch the withheld blob. This is exactly the B3 premise (the node
// can no longer serve it), and forces recovery to go through Arweave/IPFS.
std::fs::remove_dir_all(url.strip_prefix("file://").unwrap()).unwrap();
let reader = Keypair::generate();
let envelope = seal_blob(b"SECRET\n", &[reader.verifying_key()]).unwrap();
let cid = "testcid123";
let mut server = mockito::Server::new_async().await;
let _gql = server
.mock("POST", "/graphql")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}}]}}}"#)
.create_async()
.await;
let manifest_body = serde_json::json!({
"timestamp": "2026-06-11T00:00:00Z",
"blobs": [{ "oid": oid, "cid": cid, "recipients": [] }],
})
.to_string();
let _tx = server
.mock("GET", "/TX1")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(manifest_body)
.create_async()
.await;
let _blob = server
.mock("GET", format!("/ipfs/{cid}").as_str())
.with_status(200)
.with_body(envelope)
.create_async()
.await;
let paths = recover_from_arweave(
&server.url(),
&server.url(),
"alice",
"myrepo",
&dest,
&reader,
)
.await
.unwrap();
assert_eq!(paths, vec!["secret/b.txt".to_string()]);
let present = Command::new("git")
.args(["-C", dest.to_str().unwrap(), "cat-file", "-e", &oid])
.env("GIT_NO_LAZY_FETCH", "1")
.output()
.unwrap()
.status
.success();
assert!(
present,
"authorized reader's blob must be installed locally"
);
}
/// A caller who is not a recipient cannot decrypt the envelope, so nothing is
/// recovered even though the manifest and envelope are reachable.
#[tokio::test]
async fn recover_from_arweave_skips_unauthorized() {
use gitlawb_core::encrypt::seal_blob;
use gitlawb_core::identity::Keypair;
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n"), ("secret/b.txt", b"SECRET\n")]);
let dest = td.path().join("dest");
let bare = url.strip_prefix("file://").unwrap();
assert!(Command::new("git")
.args(["-C", bare, "config", "uploadpack.allowFilter", "true"])
.status()
.unwrap()
.success());
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
let oid = {
let out = Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
"HEAD:secret/b.txt",
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
// Simulate origin death (see the authorized test) so the withheld blob
// cannot be lazily fetched from the promisor remote.
std::fs::remove_dir_all(url.strip_prefix("file://").unwrap()).unwrap();
// Sealed to a different reader; the caller below is not a recipient.
let authorized = Keypair::generate();
let envelope = seal_blob(b"SECRET\n", &[authorized.verifying_key()]).unwrap();
let intruder = Keypair::generate();
let cid = "testcid123";
let mut server = mockito::Server::new_async().await;
let _gql = server
.mock("POST", "/graphql")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}}]}}}"#)
.create_async()
.await;
let manifest_body = serde_json::json!({
"timestamp": "2026-06-11T00:00:00Z",
"blobs": [{ "oid": oid, "cid": cid, "recipients": [] }],
})
.to_string();
let _tx = server
.mock("GET", "/TX1")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(manifest_body)
.create_async()
.await;
let _blob = server
.mock("GET", format!("/ipfs/{cid}").as_str())
.with_status(200)
.with_body(envelope)
.create_async()
.await;
let paths = recover_from_arweave(
&server.url(),
&server.url(),
"alice",
"myrepo",
&dest,
&intruder,
)
.await
.unwrap();
assert!(paths.is_empty(), "non-recipient must recover nothing");
let present = Command::new("git")
.args(["-C", dest.to_str().unwrap(), "cat-file", "-e", &oid])
.env("GIT_NO_LAZY_FETCH", "1")
.output()
.unwrap()
.status
.success();
assert!(!present, "non-recipient must not install the blob");
}
#[test]
fn parse_repo_accepts_url_and_bare() {
let (url, o, n) = parse_repo("gitlawb://did:key:zAbc/myrepo").unwrap();
@@ -447,4 +1069,46 @@ mod tests {
// An extra slash would otherwise smuggle a path segment into the name.
assert!(parse_repo("owner/name/extra").is_err());
}
#[test]
fn recovered_blob_installs_with_matching_oid() {
use gitlawb_core::encrypt::{open_blob, seal_blob};
use gitlawb_core::identity::Keypair;
let (td, url) = bare_remote(&[("public/a.txt", b"pub\n"), ("secret/b.txt", b"SECRET\n")]);
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
let oid = {
let out = std::process::Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
"HEAD:secret/b.txt",
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let reader = Keypair::generate();
let env = seal_blob(b"SECRET\n", &[reader.verifying_key()]).unwrap();
let plaintext = open_blob(&env, &reader).unwrap();
let mut child = std::process::Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"hash-object",
"-w",
"-t",
"blob",
"--stdin",
])
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.spawn()
.unwrap();
use std::io::Write;
child.stdin.take().unwrap().write_all(&plaintext).unwrap();
let out = child.wait_with_output().unwrap();
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), oid);
}
}