mirror of
https://github.com/Twigpine/node.git
synced 2026-11-09 18:08:02 -10:00
v0.5.0
86
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 github-actions[bot]
|
5f63f28215 |
chore(main): release 0.5.0 (#142)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>v0.5.0 |
||
|
Kevin CodexandClaude Opus 4.8
|
06388ec26a |
feat(gl): sanctioned iCaptcha client flow + secure git lifecycle (#138)
* feat(gl): sanctioned iCaptcha client flow + secure git lifecycle Builds the client side of the iCaptcha gate so a legitimate agent passes the challenge transparently, with no manual headers and no GITLAWB_ICAPTCHA_PROOF env var. Server enforcement is unchanged. New crate `icaptcha-client` (blocking, shared by gl + the git helper): - deterministic solvers for the computational challenge types (arithmetic, algebra, sequence) matching the iCaptcha generators; anagram/logic/LLM fall back to a solver hook / interactive prompt. - obtain_proof(): POST /v1/challenge (requesterId = caller DID), solve, POST /v1/answer, handle escalation, return the signed proof. Optional bearer auth via GITLAWB_ICAPTCHA_API_KEY. gl (crates/gl): - http.rs: unify post/put/delete through one signed sender that, on a 403 iCaptcha challenge (detected via x-icaptcha-url / x-icaptcha-level headers), solves it and retries the same signed request with the x-icaptcha-proof header (bounded retry absorbs the ~5 min proof TTL). Removes the env hack. Emits an actionable hint on 401 not_an_agent (old-CLI / unregistered). - repo.rs: `repo info` checks status and returns a real 404 instead of a stub card with `?` fields; `repo info`/`clone` resolve the owner from the local identity or an explicit owner/name — never the node's own DID. - doctor.rs: warn on gl-vs-node version drift; add an iCaptcha reachability check. git-remote-gitlawb: same shared client — push stays signed (RFC 9421); on a 403 iCaptcha (safety net; push is signed-only, not gated) it solves and retries. Clone/fetch unsigned; missing repo keeps its clear 404. gitlawb-node: the 403 icaptcha_proof_required response now advertises the service url + required level as JSON fields and x-icaptcha-url / x-icaptcha-level headers (mirroring the human_detected pattern) so clients discover them instead of scraping the message. The sub == authenticated-DID and expiry/level checks are unchanged — enforcement is not weakened. README: documents the full lifecycle (identity -> register -> repo create with auto-iCaptcha -> push -> clone), the requesterId == DID rule, and proof TTL. Tested: solver unit tests; full workspace green against Postgres (node still rejects missing/expired/wrong-subject proofs). * fix(icaptcha-client,gl): address PR #138 review feedback Rebased onto main (resolving the git-remote-gitlawb/main.rs conflict from #119) and addressed the review findings: - [beardthelion P2 / jatmn P2 / CodeRabbit] Removed the dead iCaptcha retry loop from the push path. git_receive_pack never calls verify_request (only create/fork/register are iCaptcha-gated), so a push 403 never carries x-icaptcha-* headers and the loop could never solve. Taking main's push path (via #119) deletes it, which also resolves the pack-clone (retries no longer copy the whole pack) and the interactive-prompt-over-git-stream threads, and restores the sanitized info/refs error path. git-remote-gitlawb no longer depends on icaptcha-client. - [beardthelion P1] Sanitize and length-bound iCaptcha error bodies before they reach the terminal. The origin is only as trusted as the node that advertised it, so its non-2xx bodies (and Failed{reason}) are attacker-influenceable; they went raw into bail! (CWE-150, the #137 class). Now bounded-read + C0/C1-stripped + capped, same shape as #137. - [beardthelion P2] Do not trust or hand credentials to a node-chosen iCaptcha URL. x-icaptcha-url was used as the base with no scheme/host check and the API key was sent to it on the first hop (SSRF + key exfiltration). resolve_solver_url now honors an advertised URL only when https + host-allowlisted (public default or the operator's GITLAWB_ICAPTCHA_URL), else falls back to the trusted origin; the API key is attached only to the operator's own configured origin. - [jatmn P2 / CodeRabbit] gl repo clone with a bare name now derives the short owner key via resolve_owner_did, matching the other commands' gitlawb://z.../name URL shape instead of the colon-bearing full DID. Skipped (with reason): jatmn P3 (doctor GITLAWB_ICAPTCHA_URL) is resolved by the P2 change — that env var is now read by the write path, so the doctor's probe is accurate. CodeRabbit User-Agent-version nit lived in the push code reverted to main. README push wording kept ("signed-only, no per-push challenge") — now accurate since the retry path is gone; the API-key bullet was updated for the new trust model. Tests: added sanitize + resolve_solver_url unit tests; full workspace green. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 <[email protected]> |
||
|
|
2df6ff9d30 |
fix(node): close two spam-vector root causes (trust upsert + ungated push) (#152)
* fix(node): close two spam-vector root causes (trust upsert + ungated push) The June 2026 spam wave kept re-materializing after cleanup because two paths let an unauthenticated-by-captcha actor keep operating: 1. `update_trust_score` was an upsert. Any authenticated push/issue/PR from a DID with no agent row silently INSERTed one with a fresh `registered_at`, bypassing the iCaptcha gate on /api/register — so deregistered spam DIDs came back the moment they pushed. Make it an UPDATE-only no-op for unregistered DIDs; registration is the sole way into the agents table. Regression test added. 2. `git-receive-pack` had no rate limit. The per-DID limiter on the creation routes can't brake a push flood from a DID farm (one throwaway identity per repo), so the node absorbed several pushes/sec per IP, each a Tigris round-trip + peer-notify fan-out. Add a per- client-IP limiter (Fly-Client-IP / X-Forwarded-For) on the push path, default 600/h, `GITLAWB_PUSH_RATE_LIMIT` override, 0 disables. Fails open without a proxy header so self-hosted nodes aren't broken. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix(node): address PR review — advertisement throttle, trusted-proxy IP key, bounded limiter Resolves the review findings on the push rate limiter: P1 — advertisement path bypassed the limiter. A push hits GET info/refs?service=git-receive-pack first, which calls acquire_fresh() (a Tigris download); it sat in git_read_routes under only optional_signature, so the flood was reachable via the cheap GET without ever sending the throttled POST. Now git_info_refs applies the same per-IP brake before the fresh acquire. P1 — client-supplied forwarded headers were trusted. The key came from Fly-Client-IP / the first X-Forwarded-For hop, both client-settable, so a flooder could rotate the header and never fill a bucket. Replaced with a TrustedProxy policy (GITLAWB_TRUSTED_PROXY = fly | x-forwarded-for | unset): trust only the operator's edge header, fall back to the real socket peer (ConnectInfo) otherwise. Fly config trusts Fly-Client-IP; the AWS/Caddy compose trusts the rightmost X-Forwarded-For hop. P2 — unbounded limiter key set / insert-before-reject. Added a max_keys cap (reject-before-insert, inline eviction when full) so a varied key cannot grow the map, and a rejected request never allocates. Applied to both the per-IP and per-DID limiters. P2 — malformed forwarded header disabled the brake. Empty leading XFF hop returned None and skipped the limiter; empty Fly-Client-IP returned Some(""). client_key now takes the rightmost XFF hop and always falls back to the peer, so a malformed header can never turn the limiter off. P2 — missing tests. Added: middleware 429 path, per-peer isolation, key-cap eviction/rejection, trusted-proxy header resolution, and an integration test asserting the receive-pack advertisement is throttled before the acquire. Also serve with into_make_service_with_connect_info so the peer address is available. New env: GITLAWB_PUSH_RATE_LIMIT, GITLAWB_TRUSTED_PROXY (documented in .env.example). Co-Authored-By: Claude Opus 4.8 <[email protected]> * Update infra/aws/compose.yaml.tftpl Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(node): amortize the rate limiter's capacity eviction scan `RateLimiter::check` ran a full O(max_keys) `retain` sweep on every new-key miss once the map was at capacity — which is exactly the distinct-key-flood state the cap defends against, so the limiter serialized all traffic behind a 200k-entry scan per request under the mutex. Gate the inline sweep to run at most once per `sweep_interval` (min of the window and 1s), tracked by `last_sweep`. Between sweeps a new key is rejected without scanning; the background `cleanup()` loop still reclaims independently. The fast path for existing keys is unchanged, the key cap invariant holds (map never exceeds max_keys, rejected keys never allocate), and post-flood self-healing stays bounded to one interval. Added a test asserting a burst of capacity misses does not trigger repeated eviction scans. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> |
||
|
Mizuki Hayashiandmizuki0x
|
2620e973e3 |
fix(repos): log the cause when repo create fails (#103)
create_repo was the one git handler that mapped its error without logging
it (info_refs, receive_pack, acquire all log with repo context). So a
failed init_bare returned an opaque 500 "initializing bare repo" with no
server-side trail to the real cause.
Log the full anyhow chain ({:#}, so the leaf git/filesystem error is kept)
with owner+repo context, matching the other handlers. Response unchanged.
Co-authored-by: mizuki0x <[email protected]>
|
||
|
beardthelionandt
|
e3a20317d5 |
ci: pin cargo-audit on the blocking gate, float the weekly canary (#151)
The PR-blocking audit job ran `cargo install --locked cargo-audit` with no version, so the scanner floated to the latest release on every run; a regressed release could red the PR gate repo-wide with no code change. Pin it to 0.22.2, the version the unpinned install currently resolves to and the one the audit-gate learnings were validated against. `cargo install --version X` is an exact pin (not a caret), so this freezes the binary while the advisory DB is still fetched fresh at scan time, which is what the gate actually keys on. Keep a forcing function so the pin cannot silently rot: leave the weekly Scheduled Audit deliberately unpinned as a drift canary. That job is cron-only (never pull_request/push-triggered), so it never gates a merge; floating there surfaces a regressed latest as a red weekly run or a changed report, which is the drift signal, without ever blocking a PR. Also assert the installed version after the pinned install so an accidental future unpin fails loud instead of silently running a different scanner, include the version in the rust-cache key so cache hits stay deterministic, and document the yank/bump path in the step comment. Co-authored-by: t <t@t> |
||
|
beardthelionandt
|
c296e23678 |
test(node,git): wire run_git_service teardown end-to-end (#62) (#150)
The #61 tests build KillGroupOnDrop by hand and never call run_git_service, so dropping process_group(0), the guard-arming, or the post-reap disarm left them green. Add two tests that drive the real run_git_service through an injected fake git (a minimal git_bin seam; production passes "git") and assert the production spawn path actually wires the teardown: - future-drop mid-flight SIGTERMs the whole group, reaping git and its grandchild (RED without process_group(0) or the guard-arming) - a completed request disarms the guard, leaving a live grandchild untouched (RED without the post-reap disarm) Each assertion was shown RED against its mutation and green when restored. Parallel-safe: the fake is reached via the seam, not a PATH mutation. Co-authored-by: t <t@t> |
||
|
beardthelionandt
|
20d6848846 |
fix(gl): sign the CLI's /ipfs/pins reads under the #134 auth gate (#146)
* fix(gl,ipfs): sign gl ipfs list so it works under the #134 pins auth gate /api/v1/ipfs/pins now 401s anonymous callers (#134). Load the caller's identity (--dir, default keystore) and use NodeClient::get_signed; when no identity exists, propagate load_keypair_from_dir's existing 'gl identity new' error instead of a raw 401. Tests: signed-headers present, empty, and no-identity-issues-no-request (mockito). * fix(gl,node): sign the gl node status pins panel, degrade gracefully The pins panel signs its /api/v1/ipfs/pins read (#134 gates it behind auth) via an injectable fetch_pins helper with four states: pins, empty, unavailable (signed read rejected/errored), and needs-identity (no keypair, no request issued). cmd_status loads the identity gracefully so a missing keystore never aborts status; peers/repos/p2p/events panels stay anonymous. Tests drive fetch_pins directly against a mock node. * fix(gl): address review — surface pins HTTP errors, correct empty-panel label - gl ipfs list: check the /ipfs/pins response status before parsing, so a rejected signed read (e.g. 401) surfaces as an error instead of silently printing 'No IPFS pins recorded'. Matches the sibling gl ipfs get. - gl node status: PinsPanel::Empty now renders 'Pinned CIDs: 0' (a reachable node with zero pins), reserving 'unavailable' for the failure state; the prior label reused 'IPFS not configured' and misreported the zero-pins case. - Carry the resolved count in PinsPanel::Pins, removing the duplicated count-fallback closure between fetch_pins and the render arm. * test(gl,node): cover fetch_pins error branches and pins-panel rendering Extract the pins-panel render into a testable pins_status_line helper and assert all four states' output (closes the untested cmd_status render path, including the empty->'Pinned CIDs: 0' label). Add fetch_pins tests for the two error branches that were code-traced but unexecuted: a malformed 200 body and a transport error both degrade to Unavailable without panicking. * style(gl): rustfmt the ipfs list pins-response parse * fix(gl,node): honor --dir identity selector in `gl node status` (#146) `gl node status` always loaded the default keystore via `load_keypair_from_dir(None)`, so a user who created or selected an identity with `--dir` saw the status pins panel render "sign in to view" even though `gl ipfs list --dir` authenticated fine. Add a `--dir` option to `NodeCmd::Status` and thread it into the signed pins fetch, so both CLI pins callers select the same identity directory. An explicit `--dir` that can't be loaded (missing, corrupt, or unreadable) no longer masquerades as "no identity, run `gl identity new`": the pins identity is resolved up front into Keyed / Anonymous / DirUnusable, and the unusable explicit-dir case renders a distinct "no usable identity in <dir>" panel line that names the path, while a missing default keystore (no `--dir`) still degrades quietly. A pins failure never aborts the dashboard. Tests cover both flag branches (Some/None) and the resolver both ways: a good `--dir` loads Keyed, an unusable one reports DirUnusable/IdentityError without issuing a request. The new symbols were absent before, so they compile-RED first. * test(gl,node): pin the pins-identity resolver to the --dir key Follow-up test hardening on the #146 --dir wiring after review: - resolve_pins_auth_loads_keyed_identity_from_explicit_dir now destructures Keyed and asserts loaded.did() equals the --dir key's DID. A bare Keyed(_) match passed even when the load fell through to the default keystore; the DID assertion catches that regression (confirmed red under a simulated fallthrough, green on the real code). - add resolve_pins_auth_none_is_never_dir_unusable: an absent --dir must degrade to Anonymous or Keyed, never DirUnusable. It asserts only the None to DirUnusable mapping, so it is stable whether or not the test runner has a default ~/.gitlawb identity. Test module only; no production behavior change. --------- Co-authored-by: t <t@t> |
||
|
beardthelionandt
|
4891db3889 |
fix(node): gate the ref-updates feeds on read visibility (#112, #114) (#143)
* feat(node,visibility): add pure fail-closed feed-visibility filter (#112,#114) Add ref_update_row_visible: given the deduped local repo set + their visibility rules, decide whether a received_ref_updates row (keyed by its peer-supplied slug) is visible to the caller. Matches slug->repo by name plus prefix-tolerant owner key, and drops any row matching a local repo the caller cannot read at root; rows matching no local repo (remote/ gossip-only) and malformed slugs pass through. Fail-closed by construction (no default-keep). Pure/I-O-free so visibility.rs stays exhaustively unit-tested. Shared gate for the GraphQL (#112) and REST (#114) ref-updates feeds; call sites land in the following commits. Includes a DB-layer test pinning that a private canonical row beats a public mirror row in list_all_repos_deduped (else the feed gate would over-serve). * fix(node,graphql): gate refUpdates on read visibility (#112) The GraphQL ref_updates resolver mapped every received_ref_updates row straight out with no visibility gate, so an anonymous caller could read a private repo's ref metadata (refName, SHAs, pusherDid, nodeDid) by name. Thread the caller DID and retain rows through the shared ref_update_row_visible filter (deduped local set + rules, same '/' decision the repos resolver uses); applies to both the repo:Some and repo:None branches. Remote/gossip-only rows still pass. Removes the now-live filter's dead_code allow. Tested via the GraphQL schema: anon leak reproduced before the gate (RED) and closed after (GREEN); owner still sees their private row; full-DID and truncated-key slug aliases fail closed; remote rows kept. * fix(node,events): gate global ref-updates feed on read visibility (#114) GET /api/v1/events/ref-updates returned every received_ref_updates row across all repos, including private ones, to any anonymous caller (the REST analog of #112). Thread the caller DID via optional_signature and retain rows through the shared ref_update_row_visible filter, deriving count from the filtered set. Keeps the crate::error::Result return type; the deduped/rules loads fail closed (500) on a DB error rather than serving unfiltered. Remote/gossip-only rows still pass. Tested via the crate's HTTP-API harness: anon leak reproduced before the gate (RED) and closed after (GREEN); owner still sees their private row; mixed feed yields only public rows with a filtered count; full-DID and truncated-key slug aliases fail closed; remote rows kept. * fix(node,graphql): gate the ref-update subscription broadcast on announce (#112,#114) Code review surfaced a third reader of the same ref metadata the query and REST feeds gate: the GraphQL subscription. /graphql/ws is mounted after the optional_signature layer (no caller to gate against), and the push handler broadcast every ref update to it unconditionally — outside the if-announce block that already gates the sibling gossip and Arweave sends. So an anonymous websocket subscriber received live ref metadata (repo, refs, SHAs, pusher/node DID) for private repos as they were pushed — the subscription analog of the #112/#114 leak. Move the ref_update_tx broadcast inside the existing if-announce guard so only publicly-readable pushes reach the unauthenticated subscription, matching gossip and Arweave. Pin the gate decision with a replication_withheld_set test (announce false for private, true for public). Also harden the feed filter's owner-key normalization: normalize the slug owner to the last ':'-segment (same rule as record_key) instead of stripping only did🔑, so a multi-segment DID (did:web) and its full-DID slug match and fail closed rather than over-serving. Regression test added (RED under the old normalization). * docs(node,graphql): pin the subscription write-side gate invariant; add did:web keep test Code review flagged that the unauthenticated /graphql/ws ref_updates subscription now depends on a single-point invariant: safety rests entirely on every sender to ref_update_tx being announce-gated, since the resolver has no caller to gate against. Document that on the resolver so a future sender can't silently reopen the leak. Add the symmetric keep-side test for the did:web owner-key normalization (public multi-segment-DID repo row is kept for anon), so a regression that over-drops legitimate did:web rows is caught alongside the existing drop-side test. * docs(node,visibility): correct ref-update slug normalization rationale (#112,#114) The comment claimed parity with did_matches/DEDUP_CTE, but the last-segment normalization deliberately diverges: the emitter (publish_ref_update) broadcasts the short, method-stripped slug, so a repo's own rows arrive keyed by the trailing segment and the gate must match on it to stay fail-closed. Reusing the did:key-aware keep-whole rule would fail open on a did:web repo's short slug. Add feed_private_didweb_short_slug_dropped_for_anon to pin that load-bearing canonical case; relabel the full-DID test as the non-canonical hand-forged form. * fix(node,events): gate ref-update feeds before the limit via paging (#114) The REST global feed and the GraphQL ref_updates resolver loaded `limit` rows and dropped private ones in memory, so an anonymous request could get fewer than `limit` events (or zero) when the newest rows named private local repos, even though older public ref updates existed. Add a shared collect_visible_ref_updates helper that applies the fail-closed gate before the limit: it over-fetches in bounded pages (128 rows, capped at max(limit, 2048) scanned) until `limit` visible rows are collected or the table is exhausted. Both feeds route through it so the one gate cannot drift. Back it with list_ref_updates_page (stable timestamp,id ordering for offset paging) and drop the now-unused list_ref_updates / list_ref_updates_filtered. The page size is a parameter on an inner fn so a regression can exercise multi-page paging. Regressions: newest-private, older-public, small limit still returns the latest visible rows on both surfaces (RED before, GREEN after); and a page=2 test pins offset paging across page boundaries (no skip or duplicate). * docs(node,visibility): correct slug-builder attribution in ref-update gate comment The comment credited publish_ref_update with building the wire slug, but that p2p method only broadcasts a pre-built RefUpdateEvent; git_receive_pack assembles the {last-segment}/{name} slug and hands it over. Fix the attribution. * fix(node,events): bound feed limit and withhold quarantined mirrors (#114) Resolve two CodeRabbit findings on the ref-update feeds: - Clamp the effective limit inside the shared collector (max 200). REST already capped, but the GraphQL resolver passed its caller-provided limit uncapped, letting a large request return unbounded rows and scan unbounded DB rows. Both surfaces are now bounded identically. - Fold quarantined mirrors into the gate's match universe as unreadable. list_all_repos_deduped excludes quarantined rows, so a ref-update naming a quarantined local mirror matched nothing, was misclassified as remote, and was served to anon, bypassing the withheld-from-every-listing-surface contract. Add list_quarantined_repos and drop matching rows fail-closed. Regressions: oversized-limit clamp (201 rows -> 200) and quarantined-mirror withheld-from-anon, both RED before, GREEN after. * fix(node,events): gate the repo-scoped events feed on read visibility (#112, #114) GET /api/v1/repos/{owner}/{repo}/events (list_repo_events) served a repo's ref certificates and received gossip ref-updates with no caller identity and no visibility check, so an anonymous caller who guessed a private repo's owner/name read its ref metadata. It was the last repo-scoped reader in the #112/#114 ref-metadata leak class. Gate it in two layers, using the caller DID the optional_signature layer already supplies: - The cert half is gated by a repo-root authorize_repo_read (keyed by the unique repo record id): a denied, quarantined, or not-hosted repo returns an opaque 404 (RepoNotFound), never a 403. - The gossip half is filtered per row through the shared collect_visible_ref_updates. received_ref_updates rows are keyed by the lossy, non-unique wire slug {last-segment}/{name}, so two owners (did:web:a:alice, did:web:b:alice) collide on `alice/name` and a plain slug query would serve a colliding private repo's rows to anyone allowed to read this one; the per-row gate drops them fail-closed. Both halves now propagate DB errors as 500 instead of swallowing them into an empty 200. Removes the now-unused list_repo_ref_updates. Deliberate behavior change: a repo this node does not host returns 404 (it holds no visibility record for it and fails closed; remote gossip is read via the global /api/v1/events/ref-updates feed). Drops list_repo_events from the egress-guard KNOWN_UNGATED allowlist; the source-scrape staleness assert enforces the removal. Tests cover anon-private, owner-private (both datasets), anon/authed-public, quarantined, released-from-quarantine, non-owner, not-hosted, did:web (owner and anon), the slug collision, oversized-limit clamping, and DB-error-fails-closed on both the gate and the cert fetch. * fix(node,events): clamp negative ref-update feed limits to zero (#114) A caller-supplied `?limit=-1` cleared the `.min(MAX_VISIBLE_REF_UPDATES)` upper cap unchanged and reached `all_events.truncate(limit as usize)` in list_repo_events, where `-1 as usize` is usize::MAX, so the truncate was a no-op and the handler returned the full unbounded local ref-cert set. Floor both feed handlers' limit parse with `.clamp(0, MAX_VISIBLE_REF_UPDATES)`. Only the repo handler's clamp is load-bearing: it guards the local `truncate`, which does not pass through the shared collector. The global feed's clamp is a consistency measure (its limit is re-clamped in the collector, whose want==0 short-circuits before any scan), kept in lockstep so the two per-handler caps the module comment calls out don't drift. Tests, both load-bearing by execution: - repo_events_negative_limit_clamped: seeds local certs (the vulnerable half), RED (3) before the clamp and green (0) after. - feed_negative_limit_returns_empty: seeds 5 visible rows; green (0) with the collector clamp, 5 without it, proving the collector is the global feed's guard. * fix(node,events): drop quarantined repos before the feed visibility gate (#114) The ref-update feeds folded quarantined mirrors into the deduped repo set with is_public=false and relied on the row visibility gate to withhold them. That held for anonymous callers but not for one matching the mirror's owner_did: visibility_check short-circuits to Allow for the owner before is_public is read, so an owner-matched quarantined row was served on both the REST global feed and the GraphQL ref_updates resolver. Drop any row that names a quarantined repo in the shared collector, before the visibility gate runs, so the drop bypasses the owner short-circuit. Quarantine is a status decided at admission and checked separately from the mirror's visibility fields, so it must deny every caller including the owner. Extract the slug match into one ref_update_row_names_repo predicate reused by both the gate and the quarantine drop, so the two cannot disagree about which rows a repo owns. Tests cover the owner case at the collector, REST handler, and GraphQL resolver in both bare-key and full-did:key owner forms, plus a must-not case proving the drop withholds only quarantined rows while still serving unrelated visible ones. * test(node,events): align did:web feed tests with the #141 canonical slug The rebase onto main adopts #141's normalize_owner_key on the repo-scoped ref-update read path, which keeps a non-did:key DID intact instead of taking the last URL segment. Align the two did:web fixtures that still assumed the old {last-segment}/{name} wire slug: - repo_events_did_web_owner_reads_own_gossip stored "alice/widget", but the emit side (api/repos) publishes "did:web:example.com:alice/widget", so the read query no longer matched and the owner saw only the cert half. Store the emit-side slug so the gossip KEEP branch is exercised again. - repo_events_gossip_slug_collision_withheld_from_anon is removed. Its premise was that two did:web owners collide on the last-segment slug; #141 gives them distinct full-DID slugs, so the repo-scoped query fetches zero rows and the per-row drop is never reached (verified by probe: fetched=0). Global-feed collision coverage stays in feed_full_did_slug_dropped_for_anon and feed_truncated_key_slug_dropped_for_anon. * docs(node): correct ref-update slug comments for the #141 canonical form The rebase onto #141 changed the emit-side wire slug from the last-segment form to normalize_owner_key (bare short key for a single-segment did:key, full DID otherwise). Four comments still described the old behavior: - visibility.rs ref_update_row_names_repo: the emitter builds the slug via normalize_owner_key, not {last-segment}/{name}. The next_back reduction on the matcher is what tolerates the untrusted/crafted short form; it does not mirror an already-stripped emit form. - events.rs list_repo_events gossip half: the residual collision the per-row filter still guards is a did:key canonical owner vs its bare short-key mirror (both normalize to the same slug), not two did:web owners (#141 gives those distinct slugs). - the two did:web feed-visibility tests had their canonical/crafted framing backwards: post-#141 the full-DID slug is the emit form and the short slug is the crafted one. Comment-only; no behavior change. * test(node,events): add load-bearing two-repo owner-key collision deny A public bare-key mirror (z6MkX) and a private did:key canonical repo (did:key:z6MkX) normalize to the same owner key post-#141, so their gossip rows share the z6MkX/... slug space on the untrusted wire. Assert an anon global-feed read sees the public mirror's row but NOT the private canonical sibling's: the shared gate keys on the full slug (owner + name), so a readable public repo under an owner key does not unlock that owner's other private repos' rows. The removed repo-scoped did:web collision test never covered this pair. Verified load-bearing: disabling the per-row visibility drop serves z6MkX/secret to anon (RED); the gate drops it (GREEN). * fix(node,events): keyset-paginate the ref-update feed collector collect_visible_ref_updates paged received_ref_updates with LIMIT/OFFSET, so a concurrent gossip/notify insert (which sorts to the front under timestamp DESC) shifted every later offset and let one feed response duplicate or skip a visible row. Switch the collector to a keyset cursor on (timestamp, id): each page reads rows strictly older than the last row fetched, so a newer insert lands above the window and cannot shift it. One collector feeds the REST global, REST per-repo, and GraphQL paths, so this covers all three; there is no client-facing cursor, so the internal cursor tracks the last fetched row. Adds tests that OFFSET duplicates a row under a concurrent insert and that keyset stays stable. --------- Co-authored-by: t <t@t> |
||
|
beardthelionandt
|
6f36fc07b8 |
fix(node,git-remote): gate receive-pack advertisement, sign client fetch/push (#119)
* fix(node,git-remote): gate receive-pack advertisement, sign client fetch/push
The info/refs ref advertisement was gated only for service=git-upload-pack
(the visibility check sat inside `if service == "git-upload-pack"`), so an
anonymous GET /{owner}/{repo}.git/info/refs?service=git-receive-pack returned a
private repo's branch names and commit tips, bypassing the read gate. Run the
visibility "/" check for both services.
Now that the advertisement is gated for both services, git-remote-gitlawb has to
authenticate its own requests, or a private repo's owner can neither fetch nor
push it. Sign the Phase-1 advertisement GET and the Phase-2 pack POST for both
services over path_and_query when an identity keypair is present; public repos
stay anonymous. This also repairs a pre-existing break where an owner could not
fetch their own private repo, because the upload-pack POST was sent unsigned.
Tests: node-side gate (anon denied, owner-signed cleared, both services); a
real-signature seam proving the client's own signature verifies under the node's
verification primitives; real served-content withholding against an on-disk repo
(the owner gets the ref bytes, a denied caller's 404 omits them); and mockito
coverage that the client attaches and omits signatures correctly.
* docs(node,git-remote): correct auth-contract comments on receive-pack gating (#119)
receive-pack POST is signature-required and separately owner-gated, not
visibility_check-gated; push implies read, not read-subset-of-push.
* test(node): assert unsigned receive-pack POST is rejected (#119)
No test covered the signature-required half of the receive-pack auth
contract. Wire the POST route behind require_signature as production does and
assert an unsigned POST returns 401 before reaching the handler; 401 (not the
handler's 404/500) proves require_signature rejected it pre-handler.
* test(node): RAII-clean the served-refs test's temp dirs on panic (#119)
advertisement_serves_real_refs_only_to_authorized_callers created /tmp/<slug>/<name>.git and /tmp/gl-seam-src-<short> and only removed them after the final assertion, leaking those dirs on the runner when an earlier assertion failed. Wrap each known path in a DirGuard whose Drop calls remove_dir_all, bound right after creation, so cleanup runs on success or panic. tempfile::TempDir does not fit here because repo_store::for_testing fixes the path layout.
* test(node): drop git_info_refs from KNOWN_UNGATED now that #119 gates it
The repo-scoped egress guard's allowlist (added in #122, merged to main
after this branch was opened) still listed git_info_refs as ungated. This
branch makes the info/refs visibility gate unconditional for both
git-upload-pack and git-receive-pack, so the handler is now gated and the
staleness assert requires removing it from the allowlist.
* style(node,git-remote): tidy #119 review nits (em dashes, dead binding, test header)
Post-rebase ce-code-review pass on #119. No behavior change:
- replace em dashes in the added comments with commas/colons/parentheses
- drop the dead `let _ = COVERED_COMPONENTS;` binding (and its import); the
`missing_components().is_empty()` assert already covers the intent
- add a section header above the new git-info-refs test block to match the
module's sectioning convention
* test(node): recurse nested api modules in the egress-gate guard
The completeness scan in every_repo_scoped_handler_is_gated used a flat
read_dir over src/api, so a future api/<module>/mod.rs could add a
repo-scoped handler the scrape never inspects. Walk the api tree recursively
via a new collect_rs_files helper (covered by its own unit test), skipping
only the top-level mod.rs so nested module files are checked too.
* fix(git-remote): keep public fetches anonymous, sign only on a 404 retry (#119)
git-remote-gitlawb signed every git-upload-pack request whenever an identity
existed, disclosing the caller's DID to any node it fetched a public repo from,
even though public repos need no auth. Fetch now sends the advertisement GET
anonymously and escalates to a single signed retry only when the node denies it
with 404 (how it withholds a private repo from an unauthenticated reader); if the
retry was needed, the upload-pack POST is signed too. Push (git-receive-pack) is
unchanged and signs from the first request, since a push can never be anonymous.
Verified against the node by execution: an unsigned private fetch returns 404,
not 401, and a signed retry clears the read gate for the owner and for a granted
non-owner reader, while a signed non-reader stays 404.
Tests drive handle_connect end to end (recording mock): public fetch stays
anonymous for GET and POST, private fetch retries signed once and signs the POST,
a keyless fetch does not retry, a still-denied signed retry surfaces 404 once, and
push signs from the first request.
* fix(node): scan nested api sources by relative path in the egress guard (#119)
The egress-gate completeness scan skipped already-covered files by basename
(listed.contains(fname)), so a nested api/<module>/repos.rs would be skipped
because the top-level repos.rs is listed, letting a future nested module add an
ungated repo-scoped handler the scrape never inspects. Extract the skip logic into
unlisted_source_files and match on the path relative to api_root, so only the exact
top-level files already covered are skipped and a nested file with a colliding
basename is still scanned. Completes CodeRabbit's outstanding request.
---------
Co-authored-by: t <t@t>
|
||
|
beardthelion
|
563c456803 |
fix(node,gossip): route gossip HTTP through the no-redirect client (#93) (#140)
gossip_task built its own reqwest::Client::new(), whose default redirect policy follows up to 10 redirects, and used it for the bootstrap announce and the periodic peer /health ping. A peer host (public at announce time, per is_public_http_url) could answer /health with 302 Location: http://169.254.169.254/... and the node would follow it into cloud metadata or internal addresses. Blind SSRF: only request status feeds mark_peer_ping. Reuse the shared state.http_client (Policy::none, already on AppState), matching sync.rs and the announce fan-out hardened in #78. Extract ping_peer_health so the redirect-not-followed behavior is covered by a mockito test (302 to an internal target registered expect(0), asserted not hit). |
||
|
GravireiandGravirei
|
6c95592d18 |
fix(node): prefer canonical repo row over mirror row in get_repo (#124) (#141)
* fix(node): prefer canonical repo row over mirror row in get_repo (#124) get_repo matched both mirror rows (bare short-DID, is_public=true, no visibility rules) and canonical rows (full did🔑 DID, correct rules), returning one nondeterministically. When the mirror row won, the visibility gate saw is_public=true + empty rules, allowing unauthenticated read of private/withheld content. Add ORDER BY to prefer canonical rows (UUID id, no slash) over mirror rows (slash-form id), matching the ranking logic already used by DEDUP_CTE and dedupe_canonical_repos. Test seeds mirror FIRST so an unordered fetch_optional returns the mirror row — proving the test locks in the fix. * fix(test): date canonical row after mirror to guard CASE term in get_repo fix (#124) * fix(node): tighten get_repo owner matching to did:key only, not any DID method (#124 P2) * fix(node): refactor owner DID normalization to a dedicated function in create_repo * fix(node): align clone URL and slug construction with tightened did:key matching - Make normalize_owner_key pub(crate) and reuse it in api/repos, api/events, api/peers, and visibility modules. - Add read-gate authorize_repo_read and to_response clone_url slug tests. * fix(node): hoist owner-key CASE into shared const + address #124 review feedback - Hoist 7 verbatim SQL CASE copies into OWNER_KEY_CASE_SQL const (dedup_cte(), get_repo, count_repos_deduped), keeping one literal copy in the v7 migration with a cross-reference comment. - Add 9 unit tests for normalize_owner_key over the full boundary set. - Add LIMIT 1 to get_repo query for single-row intent. - Add non_key_owner_bare_short_does_not_match is_owner regression test. - Update is_owner doc to reflect did:key-only rule. - Document non-key clone_url disk-path constraint in to_response. * test(node): add SQL round-trip test for normalize_owner_key vs OWNER_KEY_CASE_SQL Verifies every boundary value produces identical results from the Rust function and the SQL CASE expression, preventing silent drift. --------- Co-authored-by: Gravirei <[email protected]> |
||
|
GravireiandGravirei
|
466a550915 |
fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126) (#133)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)
The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.
* perf(ipfs): check object existence before allowed-blob walk
Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.
* refactor(ipfs): improve formatting and readability in get_by_cid function
✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.
✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.
• P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.
* refactor(ipfs): streamline object retrieval by separating type and content reading
* docs(ipfs): update get_by_cid comment to reflect split object retrieval
* Run cargo fmt on store.rs
---------
Co-authored-by: Gravirei <[email protected]>
|
||
|
Yang Yuandparanoa233
|
09a0cb23b9 |
fix(remote): include HTTP error response body (#137)
* fix(remote): include HTTP error response body * fix(remote): sanitize HTTP error response bodies --------- Co-authored-by: paranoa233 <[email protected]> |
||
|
Kevin Codex
|
cd4680ebeb |
chore(aws): codify iCaptcha enforce in the EC2 node Terraform (#139)
The AWS EC2 node (manila) was upgraded to v0.4.0 and enabled iCaptcha enforce by hand-editing its running /opt/gitlawb/compose.yaml — which would be lost on instance replacement or a user-data re-run. Codify it so rebuilds stay enforced: - variables.tf: add icaptcha_mode (default "enforce"), icaptcha_pubkey (default the pinned live icaptcha.gitlawb.com Ed25519 key), and icaptcha_required_level (default 3). - main.tf: pass the three vars into the compose templatefile render. - compose.yaml.tftpl: set ICAPTCHA_MODE / ICAPTCHA_PUBKEY / ICAPTCHA_REQUIRED_LEVEL on the node service environment. Operators can override per-deployment via terraform.tfvars. |
||
|
Kevin Codex
|
8d96cab4f7 |
chore(node): seed Gitlawb nodes in bootstrap-peers; ignore node1 fly config (#132)
- bootstrap-peers.json: add node2, node3, and manila (manila.gitlawb.com) to the canonical seed list alongside node.gitlawb.com, and normalize the Gitlawb operator label to "Gitlawb". DIDs left null — bootstrap merges peers on http_url and the parser treats did as optional. - .gitignore: ignore infra/fly/node1.fly.toml. node1's production Fly config is pulled from Fly (fly config save), not tracked in the repo. |
||
|
github-actions[bot]
|
0190fbbbf6 |
chore(main): release 0.4.0 (#131)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>v0.4.0 |
||
|
beardthelion
|
86341d195d |
test(node): guard that every repo-scoped handler is visibility-gated (#122)
* test(node): guard that every repo-scoped handler is visibility-gated Mirrors the existing every_in_scope_mutation_has_its_gate source-string CI test on the egress side: scrapes every repo-scoped handler (any pub/pub(crate) async fn taking Path<(String,String..)>) and asserts it carries an authz marker, either a read gate (authorize_repo_read / visibility_check) or a write/owner/self gate (require_repo_owner / require_owner / did_matches / &auth.0), or is listed in KNOWN_UNGATED with its tracking issue. Verb-agnostic on purpose: a repo-scoped read named outside list_/get_ (git_info_refs, replicate_encrypted_blobs, withheld_paths) is still checked, so a new ungated egress of any name fails CI instead of shipping as the next leak. A completeness scan over src/api/ asserts every module declaring a repo-scoped handler is wired into the guard, so a whole new file cannot add an ungated handler the scrape never looks at. KNOWN_UNGATED is the live gap list (certs/issues/labels/ bounties/stars per #120, plus events/webhooks/replicas/protected pending PR #113); the staleness assert forces each entry out the moment its gate lands. A count floor trips if the scrape silently breaks, and unit tests pin handler_names and is_repo_scoped. Verified by execution: removing an allowlist entry trips the missing-gate assert; listing a gated handler trips the staleness assert; dropping a source file trips the completeness scan; breaking the parser trips the count floor; and a non-list/get handler (git_info_refs) is confirmed scraped and gate-checked. * test(node): don't let a service-conditional gate satisfy the egress guard (#122) The guard counted a handler as gated on bare substring presence of a gate marker, so git_info_refs false-passed on visibility_check( even though that check only runs under `if service == "git-upload-pack"` and leaves the git-receive-pack advertisement ungated. Add gate_runs_unconditionally, which ignores markers that sit only inside an `if service ==` block, and track git_info_refs in KNOWN_UNGATED until #119 makes the gate unconditional. When that lands, the staleness assert forces the entry's removal. * fix(review): harden gate_runs_unconditionally against unclosed blocks (#122) Code review of the egress-guard hardening: - Clamp the span-advance index so an unclosed `if service ==` block can't slice past the body end and panic (latent: real source is balanced, but the scraper should not crash on a pathological future body). Keep span-to-EOF as the fail-safe direction. - Document that only `if service ==` is detected, not `match service`, so a future match-arm gate isn't silently treated as full. - Add unit cases: gate inside both service blocks (conditional), gate inside and outside (full), and the unclosed-block no-panic regression. |
||
|
Kevin Codex
|
8b9ceec25e |
feat(node): iCaptcha-aware repo propagation gate with quarantine (#125)
Phase 2 of the iCaptcha work: extend the proof-of-intelligence gate from
direct writes to federated propagation, so a node that enforces iCaptcha
can defend against spam repos that originate on a node which does not.
The proof now travels with the repo and is re-verified at mirror time:
- Capture & persist: create_repo/fork_repo store the verified proof
(raw token + claims) against the new repo. New repo_icaptcha_proofs table.
- Serve: GET /api/v1/repos/{owner}/{repo}/icaptcha-proof returns the stored
token, gated on whole-repo "/" read like the other replication endpoints.
- Admit at mirror time: the sync worker fetches the origin's proof and
icaptcha::admit_mirror re-verifies it offline — signature + level +
owner-DID binding, but NOT expiry — and consumes the jti in the local
ledger so one solved challenge admits at most one mirror per node.
- Quarantine: in enforce mode a mirror that fails the gate is kept on disk
but hidden from clone/serve/push (authorize_repo_read + the git serve and
receive-pack handlers) and from listings (dedup CTE + federated list), via
a new repos.quarantined column. Shadow logs only; off admits normally.
Fail-safe: off/shadow/no-key always admit.
Hardening from review:
- git-receive-pack also rejects quarantined mirrors (no writes while hidden).
- mirror admission keys "first-time" on DB-row absence, not on-disk presence,
so a clone that crashed before the DB insert is still gated on retry.
- mirror jti is spent against a fixed forward retention window, not the
proof's already-past exp, so the replay record isn't swept early.
Known limitation: an iCaptcha proof binds to the owner DID, not the repo, so
a malicious origin can get one spam repo per harvested public proof admitted
(per-node jti single-use bounds the amplification). Fully binding a proof to
a repo needs a repo-scoped iCaptcha challenge (protocol change, future work).
The operator release HTTP endpoint is deferred (no admin-auth surface yet);
DB methods are in place.
Tested: verify_token expiry-skip + signature/level/DID enforcement on the
propagation path; proof round-trip; quarantine set/release, preservation
across re-sync, exclusion from listings; mirror jti retention survives a
sweep at the proof's exp. Full workspace green against Postgres.
|
||
|
Kevin Codex
|
12bfb1b3c8 |
fix(release): use simple release-type + generic Cargo.toml updaters (#130)
release-type: rust at path . is incompatible with this virtual workspace root. release-please's Rust strategy unconditionally appends a CargoToml update for the root Cargo.toml, and that updater throws "is not a package manifest (might be a cargo workspace)" whenever the manifest has no [package] section — so every run died on the final step. Switch to release-type: simple (touches only a version file + CHANGELOG, never a manifest) and bump each crate version through the generic updater via extra-files plus an inline x-release-please-version annotation. Preserves the single vX.Y.Z tag, all action outputs, and the entire downstream release.yml unchanged. |
||
|
Kevin Codex
|
788a8686c1 |
fix(release): give crates explicit versions for release-please (#129)
release-please's Rust workspace strategy versions each member crate in its own Cargo.toml and cannot rewrite an inherited `version.workspace = true`, so it failed with 'value at path package.version is not tagged'. Give each of the five crates an explicit version = "0.3.9" (kept in lockstep by release-please) and drop the now-unused [workspace.package].version. Inter-crate deps are path-only, so no version requirements need syncing. |
||
|
beardthelion
|
174f25a206 |
fix(node): gate GET /ipfs/{cid} on per-caller path-scoped visibility (#110) (#128)
* fix(node): gate GET /ipfs/{cid} on per-caller path-scoped visibility (#110)
get_by_cid served any git object by raw SHA-256 with no auth and no
visibility check. For a public repo with a path-scoped deny rule, an
unauthenticated caller could recover a withheld blob's oid from the
served trees, derive its CID, and fetch the cleartext here, bypassing
the pack-path withholding.
Bind optional caller identity (optional_signature on a dedicated
/ipfs/{cid} sub-router; the pins route stays unsigned, tracked by #121)
and gate each iterated repo row against its OWN rules: visibility_check
at "/" denies the repo, and withheld_blob_oids skips a blob withheld
from the caller when the row carries path-scoped rules. The row is gated
directly rather than via authorize_repo_read, whose fuzzy get_repo
re-resolve could authorize a different physical row than the one read.
The per-repo withheld set is memoized on repo.id within the request.
Adds get_by_cid to the authz_guard drift table (visibility_check marker).
Scope: this closes the direct unauthenticated scan. A stale-public
mirror row still serves withheld content on both this path and the pack
path; that pre-existing leak is tracked separately in #124.
* fix(review): batch visibility-rule lookup and fail-closed walk panic in get_by_cid
Code-review follow-ups for #110:
- Replace the per-repo list_visibility_rules call with a single
list_visibility_rules_for_repos batch query (the existing #97 pattern),
removing the N+1 on an unauthenticated scan and the "one repo's DB error
500s the whole request" semantics.
- Fail closed on a spawn_blocking JoinError (walk task panic) by skipping the
repo, matching the walk-error arm, instead of 500-ing the whole scan.
- Add a regression test for the repo-level "/" deny branch (private repo,
anon denied / owner served) and note the access gate in the module doc.
* test(node): cover the fail-closed walk-error arm in get_by_cid
Induce a withheld_blob_oids walk error (a ref pointing at a non-tree-ish
blob, same technique as visibility_pack::fails_closed_when_a_ref_cannot_be_traversed)
and assert the handler skips the whole repo: the withheld blob 404s with no
leak, and the public blob in the same repo also 404s (proving fail-closed-skip
rather than serve-on-error). Verified load-bearing: mutating the Ok(Err) arm to
treat the error as no-withholding makes it RED (secret served 200).
|
||
|
Kevin Codex
|
3f08d9936b |
Release automation (#127)
* ci(release): add Windows target and cross-platform packaging
Add x86_64-pc-windows-msvc to the release matrix. Windows ships a .zip
(7z) while unix keeps .tar.gz; a portable sha256 helper handles
sha256sum/shasum/certutil. Smoke test and packaging now resolve the
.exe suffix on Windows. Release attach uses globs to cover both formats.
* feat(release): vendor npm wrapper and auto-publish to npm
Vendor the @gitlawb/gl npm wrapper (optionalDependencies platform-package
pattern) into npm/, replacing the standalone gl-npm repo. A new npm-publish
job downloads the four unix release tarballs from the GitHub release, lays in
the binaries, stamps every package.json with the release version, and runs
npm publish --provenance. Guarded on NPM_TOKEN so it no-ops until the secret
is configured.
* feat(release): auto-bump Homebrew tap on release
Add a homebrew-bump job that regenerates Formula/gl.rb in Gitlawb/homebrew-tap
from the release tarballs' sha256 checksums (macOS+Linux, arm64+x64) and pushes
the bump. Guarded on HOMEBREW_TAP_PAT so it no-ops until the secret and tap repo
exist. Enables 'brew install gitlawb/tap/gl'.
* feat(release): Windows installer and web auto-sync on release
Add install.ps1 (Windows PowerShell installer, source of truth in node).
Add a web-sync job that opens a PR to Gitlawb/web on each release syncing
install.sh + install.ps1 + public/version.json, plus idempotent drift fixes
(remove orphaned public/bin, fix stale skill.md releases link). Guarded on
WEB_SYNC_PAT. Vercel deploys on merge.
* docs(readme): document npm, Homebrew, and Windows install methods
* ci(release): build only CLI binaries on Windows
The gitlawb-node daemon targets Linux/macOS; building it on Windows risks
a compile failure that would block the whole release (npm/brew/web jobs all
depend on release-binaries). Windows now builds only gl + git-remote-gitlawb;
the smoke-test and package loops follow the per-target binary list.
* fix(release): reconcile release-please manifest with v0.3.9 and bump action to v5
The manifest still said 0.3.8 while the only tag is a manual v0.3.9 and
Cargo.toml is 0.3.9, so release-please had no valid baseline (no v0.3.8 tag)
and never cut a release. Set the manifest to 0.3.9 so release-please diffs
v0.3.9..main and opens a proper release PR for the next version. Also pin
release-please-action to v5.0.0 to match the openclaude pipeline.
* ci(release): set explicit minimal permissions on homebrew-bump and web-sync
CodeQL flagged both jobs for lacking a permissions block, leaving GITHUB_TOKEN
at broad default scopes. Both only need contents: read (cross-repo writes use
dedicated PATs).
* fix(release): harden installers and workflow per review
- install.ps1: fail closed if the .sha256 cannot be fetched; pure ASCII (no BOM)
- npm install.js: fail (exit 1 / throw) when the platform payload or a binary is
missing instead of reporting a successful but broken install
- workflow: top-level permissions {} + persist-credentials: false on read-only
checkouts
- docs: PowerShell-only Windows note; text language tag on npm/README fenced block
* fix(installer): require core CLI binaries on Windows
install.ps1 only failed when zero binaries were present, so a partial archive
missing gl.exe could still report success. Require gl.exe + git-remote-gitlawb.exe
(gitlawb-node.exe stays optional, matching the Windows build) and fail early if a
required binary is absent.
* ci(release): make Windows non-blocking and stop persisting cross-repo PATs
- release-binaries: continue-on-error for the Windows leg so a failure there
no longer fails the job or blocks npm/Homebrew (matches the best-effort intent)
- homebrew-bump/web-sync: persist-credentials: false on the PAT-backed checkouts;
supply the token only on the git push via an x-access-token URL
* fix(release): gate web-sync on binaries and detect native arch in installer
- web-sync now needs release-binaries so the site never advertises a release
whose artifacts failed to build
- install.ps1 reads PROCESSOR_ARCHITEW6432 before PROCESSOR_ARCHITECTURE so a
32-bit PowerShell host on x64 Windows is not wrongly rejected
* ci(release): make npm publish idempotent across reruns
Skip any @gitlawb package version already on the registry so rerunning the
job after a partial publish no longer errors on the first existing package
and leaves the wrapper/platform packages out of sync.
* fix(release): harden web-sync and tap commit per review
- web-sync: verify the Windows ZIP + .sha256 exist on the release before
syncing install.ps1/version.json (Windows is best-effort, so a failed leg
must not make the site advertise a missing installer)
- stage generated files before the no-op check so brand-new untracked files
(formula, version.json) are committed instead of skipped
- gh pr create: only swallow the 'PR already exists' case; surface auth/API errors
- npm README: scope npm/yarn/pnpm/bun install to macOS/Linux/WSL, point native
Windows at the PowerShell installer
* ci(release): guard npm-publish before doing any work
Check NPM_TOKEN in an up-front guard step (like homebrew-bump/web-sync) and
skip checkout, release downloads, extraction, version rewrites, and publish
when it is absent, instead of doing all that work and bailing in Publish.
* fix(release): robust web PR error handling and Windows installer hardening
- web-sync: capture gh pr list output so a real API/auth failure surfaces
instead of being misread as 'PR already exists'
- install.ps1: enforce TLS 1.2 and use -UseBasicParsing so downloads work on
stock Windows PowerShell 5.1; remove binaries no longer shipped on upgrade so
an in-place update can't leave a mixed-version install; handle an empty user
PATH without a leading separator
|
||
|
Kevin CodexandClaude Opus 4.8
|
adc20f9eff |
feat(node): iCaptcha proof-of-intelligence gate on create_repo + register (#108)
* feat(node): iCaptcha proof-of-intelligence gate on create_repo + register Add an optional gate that requires callers to present an iCaptcha proof (an Ed25519-signed token from icaptcha.gitlawb.com, earned by solving an escalating challenge) on the two spam-prone endpoints. Proofs are verified OFFLINE with ed25519-dalek against the published public key, and bound to the authenticated agent DID so they cannot be shared. Backward-compatible by default: ICAPTCHA_MODE is `off` unless set, so deploying changes nothing. `shadow` verifies and logs would-be rejections but always allows; `enforce` rejects requests lacking a valid, strong-enough proof. If the public key cannot be loaded at startup the gate stays inert (fail safe) so an iCaptcha outage can never break repo creation or registration. - New src/icaptcha.rs: env-driven verifier (OnceLock), /v1/pubkey fetch at startup or ICAPTCHA_PUBKEY override, proof parse + signature + expiry + level + DID-binding checks; mode-aware decide() split out for testing. - Wired into create_repo and register (X-ICaptcha-Proof header). git push (receive-pack) is intentionally NOT gated. - 11 unit tests incl. verification of a real captured proof (JS-signed, Rust-verified), tamper/expiry/wrong-DID/level/missing-header rejections, and the off/shadow/enforce/fail-safe mode matrix. Author: Kevin Codex <[email protected]> * fix(node): address review on icaptcha gate (validate config, bound key fetch) Addresses CodeRabbit and @jatmn review feedback on #108: - parse_mode now returns Option and rejects unrecognized ICAPTCHA_MODE values; init() warns and falls back to off instead of silently disabling the gate on a typo like "enforced". [P2] - ICAPTCHA_REQUIRED_LEVEL: warn on a non-empty unparseable value instead of silently lowering the threshold to the default. [P2] - fetch_key uses a reqwest client with a 5s timeout so a hung /v1/pubkey can never block node startup; still fails inert on timeout/error. [P2] - .env.example: reorder ICAPTCHA_* keys (MODE, PUBKEY, REQUIRED_LEVEL, URL) to satisfy dotenv-linter. [P3] - add parse_mode unit test (rejects junk, accepts documented values). cargo test -p gitlawb-node icaptcha: 12 passed. clippy + fmt clean. Author: Kevin Codex <[email protected]> * fix(node): consume iCaptcha proof jti so a proof can't be replayed Addresses @jatmn P2 on #108: the verifier checked exp/level/sub but never consumed the proof's jti, so one solved challenge was a reusable bearer proof for that DID until expiry, covering every gated create_repo/register in the window. Now, in enforce mode, each verified proof's jti is recorded once and replays are rejected. - db: migration v7 adds icaptcha_consumed_proofs(jti PK, expires_at); new consume_proof_jti (INSERT ON CONFLICT DO NOTHING -> bool) and sweep_expired_proofs. DB-backed so it holds across restarts and instances. - icaptcha: ProofClaims carries jti; verify returns the claims; decide() stays pure and returns Allow | Reject | Consume{jti,exp}; check() is now async and spends the jti via the DB, rejecting replays (enforce only; shadow observes without consuming). - handlers: create_repo/register await check(&state.db, ...). - main: periodic sweep of expired proof rows alongside rate-limit cleanup. - tests: 12 pass incl. decide() yields Consume with the proof's jti. Author: Kevin Codex <[email protected]> * fix(node): consume iCaptcha proof only after request passes validation Addresses CodeRabbit (Major) on #108: check() spent the jti as soon as the proof verified, but create_repo/register ran it before name/DID/existence checks, so a rejected request (bad name, already-exists, wrong DID) permanently burned a valid proof without the action succeeding. Split verification from consumption: - icaptcha: replace check() with verify_request() -> ProofGuard (verifies and rejects invalid/missing proofs early; off/shadow/inert yield a no-op guard) plus ProofGuard::consume(db) which spends the jti right before the write and rejects replays. Guard is #[must_use]. - create_repo/register: verify_request() up front, then consume() immediately before the first write, after cheap validation passes. Replay protection unchanged (consume-before-write). 12 tests pass; clippy clean. Author: Kevin Codex <[email protected]> * fix(node): give iCaptcha rejections a dedicated error code Addresses CodeRabbit on #108: iCaptcha failure paths returned AppError::Unauthorized, which maps to the "not_an_agent" API code and conflates a valid-but-unproven agent with a failed signature. Add a dedicated AppError::IcaptchaProofRequired variant mapped to the "icaptcha_proof_required" code (HTTP 401), and use it for both the missing/invalid-proof and replay paths so clients can distinguish "solve a challenge" from "your signature is invalid". 12 tests pass; clippy + fmt clean. Author: Kevin Codex <[email protected]> * fix(node): gate fork_repo with iCaptcha and cover the proof ledger Address remaining PR #108 review comments: - fork_repo was the third repo-creation entrypoint and bypassed the iCaptcha gate. Verify the proof up front (fail fast on missing/invalid) and consume it only just before the git clone, so cheap rejections (bad name, conflict, withheld subtree) never burn a valid proof — matching create_repo's verify/consume split. - Add #[sqlx::test] coverage for the single-use ledger that the gate's anti-replay value depends on: consume_proof_jti (first-use vs replay vs distinct jti) and sweep_expired_proofs (only expired rows removed, unexpired spent proofs still block replays). - Return 403 Forbidden (was 401) for IcaptchaProofRequired: the caller is authenticated but forbidden without a valid proof. The distinct icaptcha_proof_required code is unchanged. - Recommend pinning ICAPTCHA_PUBKEY in production in .env.example to avoid the startup fetch dependency. The two shadow-mode review comments were already resolved by the earlier verify/consume refactor (decide/verify are IO-free; check_proof_jti gone). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
beardthelion
|
828dd279a2 |
fix(node): gate repo-listing and stats surfaces on visibility (#97, #99, #101, #104) (#111)
* fix(node): NFC-normalize visibility matcher to close NFC/NFD under-withholding (#101) A deny rule authored in NFC byte-compared unequal to a path committed in NFD (or vice versa), so blob_paths/withheld_blob_oids failed to classify the blob and it shipped in cleartext on the replication/pack path. Normalize both the glob prefix and the candidate path to NFC at the single matcher seam (glob_matches + specificity); NFC not NFKC so compatibility forms are not folded into distinct paths. Live serve is unaffected (git resolves byte-exact; the gate only becomes more restrictive). Adds a matcher unit test and a replication-path integration test with the variant byte inside the rule-covered directory name. * fix(node): fail closed on the full-scan pin path to stop dangling-blob leak (#99) The withheld set is computed over reachable objects (rev-list --all + HEAD), but the full-scan pin fallback (cat-file --batch-all-objects) includes dangling objects the reachable walk never classified. Subtracting the reachable-only withheld set therefore could not catch a private blob orphaned by a force-push/amend, which pinned to IPFS/Pinata in cleartext. Invert the default for blobs on the full-scan path: a candidate replicates only if it is a non-blob (commits/trees are structural) or a reachable, visibility-allowed blob; anything else (a dangling blob, or a withheld one) is dropped. The delta path keeps the cheaper reachable-only subtraction since delta candidates are always reachable, so no extra walk on the hot path. Adds list_all_objects_with_type/all_blob_oids (typed enumeration), replicable_blob_set + replicable_objects_fail_closed, and threads a full_scan flag through resolve_candidates_for_push. Pin functions now take a pre-filtered object list. Regression tests: dangling blob excluded end-to-end, fail-closed unit semantics, and the all-blob universe includes dangling + excludes non-blobs. * fix(node): gate repo-listing surfaces on visibility to stop private-repo enumeration (#97) list_repos, list_federated_repos, and the GraphQL repos query returned every repo row with no visibility check, so an unauthenticated caller could enumerate private repos' name, owner, description, and clone URL — and X-Total-Count leaked the private-repo count even when rows were filtered. Each surface now keeps a row only when visibility_check at "/" allows the caller, the same decision the per-repo content endpoints make — not a bare is_public filter, which would diverge for is_public=false-with-root-reader and is_public=true-with-root-deny. The "/" decision depends on owner short/full-DID matching and JSON reader-DID membership, so it is applied in Rust over an over-fetched set rather than pushed into SQL; the paged path derives X-Total-Count from the filtered set (dropping the SQL window count) and paginates after filtering, so neither the page nor the count leaks. Adds a batch root-rule query (list_visibility_rules_for_repos) to avoid N+1, and converts list_all_repos_paged into list_all_repos_deduped (mirror dedup preserved, pagination moved to Rust). Tests: anonymous hides private repo + count, owner sees own private repo, authorized root-reader sees an is_public=false repo (proving the visibility_check path, not is_public), and federated listing hides private from anonymous. * fix(review): share the listing visibility seam, extract fail-closed pin helper, add listing tests Code-review follow-up on the VIS-subtree fixes (no behavior change to the three shipped fixes; all confirmed sound by review): - DRY the '/' listing gate into one shared visibility::listable_at_root used by list_repos, list_federated_repos, and the GraphQL repos query, so the three surfaces enforce one rule instead of three drifting copies (enforce-shared-rule-on-every-reader-path). - Extract the full-scan fail-closed pin block into fail_closed_full_scan_objects, mirroring replication_withheld_set's degraded-path shape; drop the duplicated comment and the per-site no_rules sentinel (use &[]). - Add the three listing tests the review flagged as missing: anonymous GraphQL repos hides a private repo, the paged X-Total-Count path excludes the private count (the KTD2 exploit shape), and an is_public=true repo under a root deny is not listed (proves the gate is visibility_check, not a bare is_public filter). * fix(review): route the last inline root-visibility check through listable_at_root Round-2 review follow-up. replication_withheld_set still made the announce decision with an inline visibility_check(.., "/") == Allow — the fourth copy of the listing gate the seam refactor was meant to collapse. Route it through crate::visibility::listable_at_root (caller is always None, the anonymous replication audience), so every reader of the '/' decision now shares one seam. Also drop an em dash from the fail_closed_full_scan_objects doc comment. * fix(node): gate the /api/v1/stats repos count through visibility to close the count oracle (#104) The stats endpoint returned count_repos_deduped(): an unfiltered, unauthenticated count of every logical repo including is_public=false and mode-A hide-existence repos. A polling observer could detect that a hidden repo was created (N to N+1), the exact existence signal mode-A suppresses everywhere else. Count only the repos an anonymous caller could list: over-fetch the deduped set, batch-load visibility rules, and keep rows that pass listable_at_root(.., None), mirroring the listing seam in api/repos.rs. The caller is always None (meta_routes carries no auth layer). Fail closed: any DB error collapses the count to 0, since an under-count cannot leak existence. count_repos_deduped is retained as the tested reference for the DEDUP_CTE dedup count (its tests pin the CASE the live list paths share); it has no production caller after this change and is allowed dead outside tests. Tests: bare-private (no-rule branch), hide-existence mode-A with both repos is_public=true (the Some(rule) branch), public-under-root-deny, list-parity, sibling-fields-preserved, and empty-DB. * test(review): add listing visibility regression guards (#97) Three gaps the review flagged as uncovered: - owner-filtered listing (?owner=) still hides a private repo and its count from anonymous (a distinct SQL bind branch from the unfiltered path). - offset past the visible set returns an empty page while X-Total-Count stays the full visible total (guards against deriving the total from the cut page). - a logical repo present as both a canonical row (with a root-deny rule) and a gossip mirror row stays hidden: pins that the DEDUP_CTE picks the canonical survivor so the batch rule lookup finds the deny. If dedup ever picked the mirror (slash-form id, no rule) the gate would fall back to is_public and leak. * fix(review): address CodeRabbit feedback on #111 - visibility_pack NFC/NFD test: accept a SHA-1 or SHA-256 object id (40 | 64) instead of hardcoding 40, matching the fixture guard later in the file, so the test is not SHA-1-only. - federated listing test: also assert the response `count` reflects only the visible repos, guarding against a regression to the pre-filter total. - add an authenticated-caller GraphQL test: an owner sees their own private repo via `repos`, covering the resolver's caller-DID path that the anonymous-only test did not exercise. - stats: count via the no-stars `list_all_repos_deduped()` (same DEDUP_CTE) instead of the stars-joined variant, since the count never needs stars. * fix(review): match owner filter to did_matches for bare-owner mirror rows (#111) Addresses jatmn's P2. The shared listing CTE filtered owners with `owner_did = $1 OR owner_did LIKE '%:' || $1`, so a full `did:key:z...` query did not match a mirror-only row stored with the bare owner `z...`. #97 routed the no-limit owner-filtered path (used by `gl repo list`) through this SQL predicate, regressing the full-DID-against-bare-mirror direction that `crate::api::did_matches` handled in the old Rust path. Normalize the bound owner to its did:key short form and compare it against the row's owner key using the same CASE the dedup grouping already applies, so full and short forms match symmetrically and a non-key DID still matches only exactly. The no-arg `list_all_repos_deduped()` binds NULL and is unaffected. Regression test: a bare-owner mirror-only row is returned when queried by both the full `did🔑` form and the bare short form. Confirmed RED (test fails) before the fix. |
||
|
beardthelion
|
6ae316cc88 |
fix(node): gate fork_repo on per-caller path-scoped visibility (#98) (#109)
* fix(node): gate fork_repo on per-caller path-scoped visibility (#98) fork_repo authorized the caller only at the repo root "/" and then git clone --mirror'd the full source, discarding the visibility rules. A non-owner with root read but denied a path-scoped subtree could fork the full mirror and obtain blobs the filtered git_upload_pack path withholds. Refuse the fork when the caller has any withheld glob (per-caller withheld_globs non-empty), the same decision the read path makes; owners and authorized subtree readers are unaffected. Fail closed with a not-found response, before repo_store.acquire, so no withheld object is materialized. Add unit coverage for the policy and a direct-mount integration test pinning the gate wiring. * fix(review): note fork-gate glob-grammar dependency, import withheld_globs Address code-review findings on the #98 fork gate: - Document that fork_withheld_blocks's prefix-probe equivalence with the serve path depends on validate_path_glob keeping "/" the only whole-repo scope, mirroring has_path_scoped_rule's caveat (latent-coupling guard). - Import withheld_globs and call it unqualified, consistent with the sibling visibility_check import in the same module. |
||
|
beardthelion
|
6809201daa |
fix(node): anchor the real old_sha and issue a per-ref certificate (#72)
* fix(node): anchor the real old_sha and issue a per-ref certificate (#26) The push handler discarded the real old->new transition before anchoring and only certified the first ref of a multi-ref push, so the permanent Arweave record was a weaker, incomplete copy of facts the node had already parsed. - Thread the parsed old_sha through the clone moved into the anchor task instead of hardcoding a 64-zero placeholder. This also fixes the SHA width mismatch: object ids are 40-hex SHA-1, not 64-hex. - Issue a signed RefCertificate for every ref the push advanced, each carrying that ref's real old/new, rather than one cert for the first ref (which, on an empty push, signed a timestamp string as new_sha). Lock the anchor-body contract with a test asserting the upload carries the real old_sha and new_sha, never a placeholder. Scope is the integrity subset only. Carrying the signed certificate on Turbo, capturing the pusher signature, chaining certificates, and gl-side verification remain open under #26. Refs #26 * fix(node): thread real old_sha through every ref-update fan-out path PR #72 made the Arweave/DB anchor path per-ref with the real old_sha but left three sibling fan-out paths in git_receive_pack flattening multi-ref pushes and/or emitting a placeholder old_sha. Bring them in line: - gossip publish_ref_update now forwards the real old_sha (was ""), matching the RefUpdateEvent contract that documents all-zeros for a created ref - HTTP /sync/notify sends one signed request per ref with its real old_sha; the receiver is single-ref, so a multi-ref push no longer drops refs 2..N - GraphQL RefUpdateBroadcast emits one broadcast per ref with the real old_sha instead of only the first ref + a zero placeholder No receiver, struct, or wire-schema change; the all-zeros-for-created-ref convention is preserved on the wire. Addresses jatmn's CHANGES_REQUESTED and the two CodeRabbit findings. * test(node): regression-guard the per-ref sync-notify fan-out Extract the HTTP /sync/notify send into notify_peer_of_ref (one ref) and notify_peer_of_refs (one peer, looping refs), collapsing the deeply-nested inline block in git_receive_pack and giving the fan-out a unit-testable seam. The ref_name is now also logged on every warn arm. Add mockito regression tests: a two-ref push must emit one request per ref each carrying its real old_sha (guards against re-flattening to .first() with a zero placeholder, the bug PR #72 fixed), and a created ref must forward the all-zeros hash verbatim. Behavior is unchanged; this only adds the seam, the tests, and ref_name in the logs. * style(node): match sibling test conventions in repos tests Prefix the two new sync-notify tests with test_ and use the _mock binding convention (matching arweave.rs/pinata.rs). No behavior change. * style(node): fix rustfmt import order and dedupe the zero-sha literal - Reorder the repos test-module imports so `use crate::...` precedes the external `gitlawb_core` import (unblocks cargo fmt --check / CI). - Webhook `created` check reuses the existing ZERO_SHA constant instead of repeating the 40-hex literal. - Arweave test fixtures use the canonical 40-hex zero old_sha, matching the SHA-1 width the per-ref anchoring contract now carries. * perf(node): run peer notification after local broadcast and anchoring The post-push background task awaited the HTTP peer-notify fan-out before the GraphQL subscription broadcast and Arweave anchoring. With per-ref notification a slow or unreachable peer could delay both local subscribers and permanent anchoring by the sum of the peer round-trips. Move the peer-notify loop to run last; peers already learned of the push via the gossip publish earlier in the task, so this only deprioritizes the best-effort HTTP fallback. * style(node): correct notify-ordering comment and reuse ZERO_SHA in test Review follow-ups on the two prior commits: - The peer-notify ordering comment claimed peers 'already learned via the gossip publish', but that publish is gated on a present p2p handle; with no handle (the very case the HTTP fallback exists for) it does not run. State the actual rationale: peer-notify is the lowest-priority best-effort step, ordered last so it cannot delay local broadcast or anchoring. - The all-zeros notify test still inlined the 40-hex literal; point it at the ZERO_SHA constant like the production path. |
||
|
beardthelion
|
3e1e9045e4 |
fix(node): close under-withholding via full ref scope and full-history classification (#42) (#84)
* fix(node): close under-withholding via full ref scope and reachable-only pin set
blob_paths walked only refs/heads/* and refs/tags/* and skipped silently on a
failed git ls-tree, so a blob reachable only through another namespace, or a ref
that failed to traverse, could fall out of the withheld set and ship in cleartext.
Walk every ref and fail closed on traversal error.
The pin enumerators (ipfs_pin, pinata) used git cat-file --batch-all-objects,
which includes unreachable/dangling objects that have no path and cannot be
classified for withholding. Switch them to git rev-list --objects --all so the
pin set matches the reachable graph blob_paths evaluates.
* fix(node): include HEAD in the withheld-blob walk to close pin/serve asymmetry
blob_paths classified objects from for-each-ref only, but the pin set
(git rev-list --objects --all) and git upload-pack both also expose
objects reachable from HEAD. A detached HEAD (or any HEAD pointing
outside the ref-covered set) therefore put a blob into the pin/serve
path that the withholding walk never classified — a latent cleartext
leak of a withheld blob. Walk HEAD too (guarded for unborn HEAD on an
empty repo) so the withheld set is a superset of every exposure path.
Adds a regression test: a blob reachable only via a detached HEAD must
still be withheld (fails without this change).
* fix(node): classify blobs across full history, not just ref tips
blob_paths walked only ref-tip trees (git ls-tree -r <ref>), so a blob
that existed in an older commit and was deleted or rotated at the tip
was never classified, while the pin set (git rev-list --objects --all)
and upload-pack still expose it from history. That under-withheld the
blob into the IPFS/Pinata replication paths in cleartext.
Enumerate all reachable commits (git rev-list --all + HEAD for the
detached case) and walk each commit tree, deduping (oid, path) pairs.
ls-tree per commit is kept over rev-list --objects so every path a blob
lives at is still represented for the per-path visibility check. A ref
that resolves to a non-commit now fails closed explicitly, since
rev-list --all silently skips it where the old per-ref ls-tree errored.
Regression test: a secret blob deleted at the tip but reachable in
history is withheld.
* fix(node): share one history walk and fully peel ref types in withheld classifier
Addresses two review findings on the under-withholding classifier:
- withheld_blob_recipients walked blob_paths twice (once via
withheld_blob_oids, once directly). Extract withheld_from_pairs so a
single history walk feeds both the withheld set and the recipient map.
- The ref-type fail-closed guard used for-each-ref %(*objecttype), which
peels only one tag level and so misclassified a nested annotated tag of
a commit as a non-commit, refusing the whole walk (over-withhold). List
refnames and peel each fully with <ref>^{} through one cat-file
--batch-check; reap the child even if the stdin write fails. Fail closed
on a short read or any non-commit/missing target.
Adds tests: annotated + nested-tag of a commit does not fail closed,
annotated tag of a blob still fails closed, and a blob at both an allowed
and a denied path is not withheld.
* fix(node): drain cat-file stdout concurrently to close the peel deadlock
The ref-type guard peeled every <ref>^{} through git cat-file --batch-check,
writing all queries to stdin before draining stdout. On a repo with enough
output the child's stdout pipe fills, it stops reading stdin, and write_all
blocks forever — hanging every serve/pin that classifies the repo. Batching
the queries does NOT bound this: cat-file echoes the full query on a
'<query> missing' line, so output scales with refname length, and a few
hundred long-named dangling refs already exceed the pipe buffer.
Run the stdin write on a separate thread so the main thread drains stdout
concurrently via wait_with_output, which reaps the child unconditionally
before any error is surfaced; the writer is joined after the drain so it
cannot deadlock, and a vanished stdin pipe becomes a broken-pipe write error.
One pass, no batch bound, deadlock-free regardless of ref count or refname
length.
Extracted as assert_all_refs_are_commits with corrected doc attribution.
Detect the missing-object line by its trailing status word. Adds a
timeout-bounded regression test with 500 long-named dangling refs (>64 KiB of
cat-file output) that fails rather than hangs if the deadlock returns.
* fix(node): parse ls-tree -z raw paths and fail closed on non-UTF-8 to stop under-withholding
Plain `git ls-tree -r` C-quotes any path with non-ASCII or special bytes
(café.txt becomes "secret/caf\303\251.txt"), so the stored path no longer
matched a visibility rule like /secret/** and the blob was served/pinned in
cleartext. Switch blob_paths to `git ls-tree -rz` and split the NUL-delimited
records, which emit paths raw; the TAB field separator survives -z so the
per-record parse is otherwise unchanged.
Parse strictly with str::from_utf8 and fail closed on invalid UTF-8 rather than
from_utf8_lossy: a lossy decode would rewrite an invalid byte in a denied
directory name to U+FFFD, and the mangled path would again miss its deny rule
(the same under-withholding class one layer down).
Tests: a non-ASCII path stays withheld (and the public blob still replicates),
and a non-UTF-8 path fails closed.
|
||
|
beardthelion
|
1960f6b28f |
ci(triage): label fork PRs and stop auto-closing PRs that only lack an issue link (#107)
* ci(triage): run PR triage on pull_request_target so fork PRs can be labeled Fork PRs run the triage job with a read-only GITHUB_TOKEN under the pull_request event regardless of the workflow's declared write permissions, so addLabels/createComment fail with 403 'Resource not accessible by integration' for every external contributor (e.g. #103). Switch the trigger to pull_request_target, which runs in the base-repo context with the declared write token. The job never checks out or runs PR head code (it only reads context.payload and the pulls/files API and manages labels), so it carries none of the untrusted-code-execution risk that makes pull_request_target dangerous. * ci(stale): drop needs-issue from auto-close trigger, keep it advisory needs-issue flags PRs that link no issue, but linking one is a SHOULD in CONTRIBUTING (required only for protocol-level changes), and many legitimate small PRs (docs, obvious fixes) have none. Coupling it to the 14/21-day auto-close was overreach: a substantive, well-described PR could be closed purely for missing an issue reference. Keep needs-issue as an advisory label and guidance nudge; leave only needs-description (an empty/thin body, a far stronger low-effort signal) as the auto-close trigger. * ci(stale): align stale message with description-only close condition The auto-close trigger narrowed to needs-description, but the stale message still told contributors a PR could close for a missing linked issue. Drop that clause so the warning matches what actually closes a PR. * style(ci): replace em dashes in triage/stale contributor messages * docs(contributing): match stale close policy to description-only trigger Auto-close is driven by needs-description only; a missing issue link stays advisory. Align the contributor docs with the workflow so the OR wording no longer implies a missing issue link can close a PR. |
||
|
beardthelion
|
abc9ad03ac |
fix(node): fail closed when a recipient DID can't be resolved (#47) (#67)
* fix(node): fail closed when a recipient DID can't be resolved (#47) encrypt_and_pin resolved recipient DIDs with filter_map, silently dropping any that failed to resolve and sealing the envelope to the remaining subset while still recording the full intended recipient set as covered. A reader whose DID could not be resolved at seal time (any did:web/did:gitlawb, or a malformed did:key) was then permanently locked out: the dedup compares the recorded full set against the desired set, matches, and never re-seals. Resolve all recipient DIDs up front via resolve_all_recipients; if any fail, log and skip the blob rather than seal to a partial set. Nothing partial is recorded, so the dedup stays honest and the blob re-seals on a later push once resolution is available. Add a bounded per-blob warn plus one aggregate coverage warn so a fully-migrated did:gitlawb org (zero encrypted-pin coverage) is one greppable line, not a scrape, and give the previously silent read_object and pin_git_object failure arms warn logs. * test(node): extract plan_seal so the #47 fail-closed gate is unit-tested The fail-closed invariant lived only in resolve_all_recipients, tested in isolation; nothing proved encrypt_and_pin acts on it (a refactor falling through to a partial seal would pass every test). encrypt_and_pin needs a live Postgres Db, which CI does not provide, so it can't be tested directly. Extract the per-blob seal/skip decision into a pure plan_seal() -> SealPlan (no DB, no IO), behaviour-preserving, and unit-test its branches: seals only when all recipients resolve, fails closed on any unresolvable DID, skips the empty set, skips an unchanged tag, re-seals a changed set. * style(node): rustfmt encrypted_pin (Seal variant + assert_eq wrap) |
||
|
beardthelion
|
338ff83584 |
fix(node): skip withheld-walk when no path-scoped rule can withhold (#60)
* feat(node): add has_path_scoped_rule predicate for withheld-walk short-circuit Shared predicate (rules.iter().any(path_glob != "/")) both serve and replication paths will use to skip the per-blob withheld walk when no rule can withhold. Doc records the "/"-gate precondition and the validate_path_glob dependency. Adds unit tests plus a gate-aware safety invariant test. * fix(node): skip withheld-walk when no path-scoped rule can withhold (#50) The serve path (git_upload_pack) ran the full per-ref withheld_blob_oids walk on every clone/fetch before falling back to a plain upload_pack, even for repos with no path-scoped rules where the walk can never withhold anything. Cost grew with #42 (per-ref git ls-tree -r). Guard both call sites on has_path_scoped_rule: the serve path skips the spawn_blocking walk entirely and serves upload_pack directly; the replication path generalizes its is_empty() short-circuit to also cover root-only rule sets. Safe because the whole-repo "/" gate runs before both sites, so a denying root rule 404s/withholds upstream and never reaches the predicate. No change to served-pack contents on the success path. * fix(node): gate encrypt-then-pin walk on has_path_scoped_rule (#50) The post-push encrypt-then-pin block in git_receive_pack gated the withheld_blob_recipients call on `!r.is_empty()`, which is true for a root-only `/` rule set. withheld_blob_recipients then runs the same per-ref `git ls-tree` walk (withheld_blob_oids) the short-circuit is meant to avoid, and returns an empty recipient map anyway. Gate it on has_path_scoped_rule to match the serve and replication sites; the skip is behavior-preserving (empty map -> encrypt_and_pin is a no-op). Also align the two remaining call sites to the visibility_pack import alias for consistency. * test(node): cover the serve decision across both has_path_scoped_rule branches Drives the git_upload_pack serve decision over a real bare repo for the INV-2 caller (a reader allowed at whole-repo "/" but denied a path-scoped subtree), exercising both branches the predicate selects: root-only rules skip the walk and serve the full pack (asserting the walk would withhold nothing, so the skip is sound), and a /secret/** rule runs the walk and excludes the secret blob from the served set while keeping the public one. |
||
|
beardthelion
|
636d3abe17 |
chore(deps): bump tar to 0.4.46 and rand to 0.9.3 to clear advisories (#105)
tar 0.4.45 had a PAX header desynchronization issue (GHSA-3pv8-6f4r-ffg2, fixed in 0.4.46); rand 0.9.2 was unsound with a custom logger using rand::rng() (GHSA-cq8v-f236-94qc, fixed in 0.9.3). Both resolve as lockfile-only bumps with no manifest change. Build and cargo audit pass. lru 0.12.5 (GHSA-rhfx-m35p-ff5j) stays pinned by aws-sdk-s3 (lru ^0.12.2) and the hickory advisories remain accepted risk per .cargo/audit.toml (#76). |
||
|
beardthelion
|
2f6611a30f |
fix(core): zeroize the derived X25519 secret (#65) (#91)
x25519_secret_from_seed derived the X25519 private scalar from the Ed25519 seed and returned it as a bare [u8; 32]. Because [u8; 32] is Copy with no Drop (and sha2's digest output likewise has no zeroize), the scalar, the SHA-512 digest, and the returned temporary were released without being scrubbed, leaving secret-derived material in freed memory. This is the same Copy-no-Drop class fixed for the raw seed in #41, on the derived secret. Return Zeroizing<[u8; 32]>, build the scalar directly into a zeroizing buffer so no bare secret local persists, and explicitly wipe the SHA-512 digest before it drops. Callers deref the result into crypto_box::SecretKey, which already scrubs its own copy. No behavior change: the derived scalar and all decryption output are byte-identical, covered by the existing ed25519_to_x25519_keypair_agrees and seal_open_round_trip_for_recipients tests. |
||
|
beardthelion
|
3e8e333aa0 |
fix(node): dedupe mirror and canonical repo rows on list surfaces (#6) (#73)
* fix(node): dedupe mirror and canonical repo rows on list surfaces (#6) The non-paged GET /api/v1/repos legacy path and list_federated_repos returned both the short-owner peer mirror row and the canonical did:key row for the same logical repo, so profiles rendered the repo twice. Only the paged path collapsed them, in SQL. Add a dedupe_canonical_repos helper that groups by (normalized owner, name), keeps the canonical non-mirror row (tie broken by earliest created_at), and carries the group's latest updated_at onto the survivor, matching the paged SQL dedup. Apply it at both non-paged surfaces and cover it with unit tests. * fix(node): dedupe GraphQL + stats repo surfaces and key mirror detection on the repo id Addresses jatmn's review on #73 (dedup not applied on every reader path; mirror detection keyed on a user-settable string). - GraphQL repos and /api/v1/stats now collapse mirror+canonical rows, via new Db::list_all_repos_deduped and count_repos_deduped that share the DISTINCT ON CTE with list_all_repos_paged so the dedup rule cannot drift. - Mirror detection keys on the structural slash-form id (written only by upsert_mirror_repo) instead of the description == 'mirrored from peer' string, in both the SQL paths and dedupe_canonical_repos. - Deterministic survivor on a full created_at tie (id ASC) in both implementations. - Legacy REST list and federated keep their method-scoped did_matches owner filter in Rust; it does not compose with the method-blind SQL group key, so those paths intentionally stay on the Rust helper. - Adds sqlx and unit tests for the new surfaces, the structural marker, and the tiebreak. * test(node): cover mirror-only, empty, and count/list-consistency dedup cases Follow-ups from code review on the dedup change: - list_all_repos_deduped/count_repos_deduped: mirror-only group survives, empty table returns empty/0, and count_repos_deduped equals the deduped list length (guards the two independent SQL queries against grouping-key drift). - Document list_all_repos as the raw, non-deduped enumeration path (object lookup only), so it is not mistaken for a listing-surface method. * style(node): apply cargo fmt to dedup test code * fix(node): make the repo dedup key did:key-aware so distinct DID methods don't collapse The dedup grouping key took the last ':' segment of owner_did (split_part / rsplit), so two repos owned by did:key:X and did:gitlawb:X with the same name collapsed into one logical repo on the list, paged, count, stats, and GraphQL surfaces. That is the exact cross-method collision the codebase already guards against in did_matches. Replace it with a did:key-aware key that strips a did🔑 prefix only when the remainder is a bare id (no ':'), otherwise keeps the full DID, reproducing did_matches/key_id as an equivalence relation: did:key:X and a bare mirror X still collapse, while distinct methods never merge. Applied byte-identically across DEDUP_CTE (DISTINCT ON / PARTITION BY / ORDER BY), count_repos_deduped, the empty-page count fallback, and the in-memory dedupe_canonical_repos, so the SQL and Rust paths agree. The backing index lived in the already-released migration v1, so v1 is left untouched and a new migration v7 drops idx_repos_owner_short_name and builds idx_repos_owner_key_name on the matching expression; the CASE must stay byte-identical to the queries or Postgres stops using it. Tests cover both the in-memory and SQL paths: distinct methods stay separate, bare-id and did:key forms collapse, and the residual-colon guard keeps a malformed did:key:did:gitlawb:X distinct from the bare method DID. 216 pass. |
||
|
beardthelion
|
f28fa02236 |
fix(security): gate webhook creation through the public-host validator (#81) (#92)
Webhook creation only scheme-checked the URL, so an authenticated repo owner could register a hook pointed at loopback, private, or metadata endpoints and make the node POST to them on repo events. Route the URL through is_public_http_url, the same validator the peer announce path already uses. Wiring it up surfaced a bypass in that shared validator: it folded IPv4-mapped and compatible IPv6 but not 6to4 or NAT64, so [2002:7f00:1::] (127.0.0.1) slipped through. Fold 6to4 and the NAT64 well-known prefix, reject the NAT64 local-use prefix and the rest of 64:ff9b::/32 outright (the embedded v4 offset varies by prefix length, so a decode would be error-prone), and reject 0.0.0.0/8. The change only ever rejects more, so the peer announce handler, the DB-boundary writer, and the boot-time prune inherit the same hardening. Closes #81. |
||
|
beardthelion
|
1af4fdf485 |
feat(node): pin the per-push object delta instead of re-enumerating the whole repo (#90)
* feat(node): add push-delta object enumeration module
Compute the per-push object delta (git rev-list --objects <new> --not <old>)
for the pin path instead of enumerating the whole repo, with a fully-peeled
ref-type guard (cat-file -t '<sha>^{}') that fails closed to a full scan on a
non-commit tip, any git error, or the GITLAWB_PIN_FORCE_FULLSCAN kill-switch.
The whole-repo lister moves here for the fallback and the reconciliation sweep.
Pin-enumeration scope only (N16); the withheld filter is unchanged.
* feat(node): pin the per-push object delta instead of the whole repo
Wire the push-delta enumeration into the IPFS and Pinata pin paths so pin work
scales with push size, not repo size (N16).
- U2: both pin_new_objects twins take a candidate OID set instead of scanning
the whole repo internally; repo_path is retained for reading object bytes.
- U4: extract the anonymous replication withheld gate (caller=None, no caller
param) into a shared helper returning (announce, withheld), so the push path
and the future reconciliation sweep cannot drift. The per-caller read-serve
withheld call in git_upload_pack is deliberately left separate.
- U3: the post-receive handler resolves the push delta once (off-thread),
falls back to a full scan on a non-commit tip / git error / kill-switch, and
passes the candidate set to both pin paths. Logs delta size and fallbacks.
The reconciliation sweep (U5) is deferred: it depends on the history-deep
blob_paths fix (#84) landing on main first.
* fix(review): log degraded pin-candidate paths, extract wiring to push_delta
Code review found the push candidate computation collapsed both a failed
full-scan fallback and a panicked spawn_blocking task to a silent empty set,
diverging from the sibling withheld path that logs and fails closed.
Extract resolve_candidates_for_push into git::push_delta (it owns the
PinCandidates dispatch and full-scan fallback), logging every degraded path
(fallback taken, fallback failed, task panicked) instead of failing silently.
Pinning nothing on a hard failure is still safe (the withheld filter always
runs; the reconciliation sweep backstops durability) but is now observable.
Moving the dispatch out of the HTTP handler also makes it unit-testable.
Add tests: tag-of-tag-of-commit peels to a Delta, and resolve_candidates_for_push
delta + non-commit-tip full-scan-fallback paths.
* fix(node): fail announce closed when withheld walk fails (#90)
A failed or panicked withheld_blob_oids walk left announce=true while
withheld=None, so a push we could not vet would still gossip ref updates,
notify peers, and write a permanent Arweave anchor. Force announce false
in that arm so the gate fails closed on both axes.
|
||
|
beardthelion
|
3d880cd56f |
fix(node): reject malformed path globs with non-trailing or empty-segment wildcards (#74) (#75)
validate_path_glob checked `ends_with("/**")` without first stripping it, so any
glob that still ended in `/**` slipped past the wildcard guard even when it held a
`*` elsewhere in the prefix (e.g. `/a*b/**`, `/a/**/**`). The prefix is fed to
glob_prefix/glob_matches as a literal, so such a rule matched no real path and
silently granted nobody, the exact silent-misconfiguration this validator exists
to prevent.
Strip the one permitted trailing `/**` first, then reject any remaining `*`.
While here, close a related hole in the same validator: `glob_prefix` greedily
trims all trailing `**` and `/`, so `//**` collapsed to `/` (whole repo) and
bypassed the explicit `/**` guard, and `//secret/**` became a dead rule whose
prefix never matches a real single-slash path. Reject empty path segments so the
validator and glob_prefix agree on the canonical form.
Adds `/a*b/**`, `/a/**/**`, `/**/**`, `//**`, `//secret/**` to the rejection test.
|
||
|
|
0a15e763d2 |
feat(node): owner-only push enforcement behind GITLAWB_ENFORCE_OWNER_PUSH (#31) (#68)
* feat(node): owner-only push enforcement behind GITLAWB_ENFORCE_OWNER_PUSH (#31) A valid did:key signature on git-receive-pack is authentication, not authorization: any party can generate a key, derive its DID, sign, and push to an unprotected branch. Owner checks only ran on protected branches, so a non-owner could write to unprotected branches of an arbitrary repo. Add an opt-in flag GITLAWB_ENFORCE_OWNER_PUSH (default false, mirroring GITLAWB_REQUIRE_SIGNED_PEER_WRITES) that requires the authenticated pusher to be the repo owner on every branch. Default-off keeps live nodes unaffected. Design follows the maintainer's review on #31: - Authorize on the canonical AuthenticatedDid injected by require_signature, via Extension<AuthenticatedDid> on the handler — not a re-parse of the headers. The legacy branch-protection check is left as-is. - Fail closed: an absent identity or a non-owner caller is rejected. The policy lives in a pure fn (owner_push_rejection) so it is unit-tested without a DB. - The gate runs before branch protection, so one rejection yields one error body. - Factor the duplicated owner-match into auth::is_repo_owner, reused by receive-pack, protect.rs, and visibility.rs (each keeps its own message). events.rs / peers.rs are left out — they aren't owner checks. - Name the push gate by intent (caller_authorized_to_push) so the planned Phase 2 UCAN git/push capability is a pure addition, not a rewrite. Tests: owner (full + short DID) allowed, non-owner rejected, missing-DID rejected, flag-off legacy. Docs: .env.example + docs/RUN-A-NODE.md. Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix(node): address review on #68 — 403 status, finish owner-match consolidation Per @beardthelion's review: - Return 403 Forbidden (not 400) for authorization denials: the owner-push gate, the branch-protection denial, and the owner checks in protect.rs / visibility.rs. 400 reads as a malformed request and some git/CI clients retry it. - Finish the is_repo_owner consolidation: route the branch-protection check and the repos list filter through the helper. The branch-protection check now uses the verified AuthenticatedDid (auth.0) instead of re-parsing the headers. - Drop the second identity source on the push path entirely: the post-push event recording also uses auth.0, so extract_did_from_auth (and the handler's HeaderMap param + import) are removed. - Tests assert the Forbidden variant (so a future status regression fails loudly) and add direct coverage for is_repo_owner / caller_authorized_to_push. - Docs: warn that enabling the flag blocks delegated and CI agents until the Phase 2 UCAN git/push capability is honored. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: FrozenRaspberry <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> |
||
|
beardthelion
|
2153b0b7a6 |
fix(gl): paginate gl-clone Arweave recovery and make /encrypted-blobs parsing schema-strict (#49) (#70)
* fix(gl): make /encrypted-blobs recovery parsing schema-strict Deserialize the /encrypted-blobs 200 body into a typed EncryptedBlobsResponse instead of poking at a serde_json::Value and defaulting blobs to [] via unwrap_or_default(). A server schema regression now fails closed (a hard parse error) rather than silently yielding nothing to recover, mirroring the existing WithheldPathsResponse convention. The sole caller still does not abort the clone but now surfaces the error instead of swallowing it into an empty result. Part of #49. * fix(gl): paginate Arweave manifest discovery and rank ties by block height Discovery hard-coded a single first:100 GraphQL query with no pagination, so a repo with more than 100 encrypted-manifest anchors (one per push) silently lost recovery coverage for any blob whose most recent seal fell outside that window. Follow cursor pagination (after + pageInfo) until coverage is complete, deduping transaction ids and bounding the loop (1000 pages / 10000 ids, degenerate-cursor guard) so a hostile or buggy gateway cannot grow the downstream per-id fetch loop without limit. A full page returned without pageInfo, a cap hit, or a failure after discovery began now warns; a page-1 failure (unconfigured/unreachable gateway) stays silent as before. Pagination concatenates pages in gateway order, which made merge_manifests' equal-timestamp tie-break (previously insertion-order, i.e. accidentally newest on single-page HEIGHT_DESC) gateway-dependent. Rank candidates by the composite key (timestamp, Arweave block height, cid): timestamp primary keeps the existing distinct-timestamp latest-wins; block height breaks equal-timestamp ties by anchor recency (a not-yet-mined anchor counts as newest), preserving the pre-pagination behavior; cid is a final deterministic tiebreak for same-block anchors. Height is parsed defensively so a present-but-unreadable height (e.g. a gateway stringifying large ints) cannot masquerade as a pending/newest anchor. Part of #49. * fix(review): bound edges per Arweave page and cover degenerate-cursor/dedup Code-review follow-ups on the #49 recovery change: - Cap edges parsed per GraphQL page at ARWEAVE_PAGE_SIZE so a hostile gateway returning one oversized page cannot defeat the MAX_TX_IDS budget or flood the downstream per-id manifest fetch loop (adversarial review). - Collapse the two identical gateway-failure arms in the discovery loop into a single warn site, and hoist PAGE_SIZE to a named const tied to the query's first:100 (maintainability). - Add tests: per-page edge bound, non-advancing-cursor termination (infinite-loop guard), and cross-page tx-id dedup fetched exactly once (testing). Part of #49. * fix(review): make the degenerate-cursor test assert, and pin the page-size invariant Round-2 review follow-ups: - The non-advancing-cursor test bound its mock to a _-prefixed handle with expect_at_most(3) and never asserted it; mockito defaults assert_on_drop=false, so the test passed even if the guard were removed. Bind it, tighten to expect(2) (the guard breaks on the second page, so discovery makes exactly two POSTs), and assert_async. - Correct the parse_tx_page docstring: the per-page take() bounds the TxRef vec and fetch loop, not the response-body deserialization (an unbounded-body concern shared by every gateway read here). - Hoist the discovery query to a const and add query_first_matches_page_size so the query first:N literal and ARWEAVE_PAGE_SIZE are kept equal by a test rather than a comment. Part of #49. |
||
|
beardthelion
|
96fcfdb1ca |
fix(node): preserve promisor mirror mode on unknown withheld-paths lookup (#48) (#69)
Peer sync decided mirror mode from the origin's withheld-paths answer via classify_mirror, whose _ => Plain arm collapsed both Some(empty) (genuinely public) and None (lookup 404'd / 5xx / network / parse error) into Plain. For an existing promisor mirror that meant fetch_repo(.., Plain) unset remote.origin.promisor + partialclonefilter and ran git fetch --refetch, so a transient withheld-paths outage downgraded a still-withheld mirror to a full clone and broke syncs until the endpoint recovered. Replace classify_mirror with resolve_mirror_mode(withheld, exists, promisor): only Some(empty) downgrades an existing mirror; None preserves the on-disk promisor mode. A genuine public transition always returns Some(empty), never None, so preserving on None cannot mask one; it only suppresses the transient-error false positive. Add a three-valued PromisorProbe read from git config --get exit codes so a transient probe failure (Unknown) is not mistaken for a definitive NotPromisor and cannot itself trigger the downgrade. One warn marks the preserve path, derived from the resolved mode so it cannot drift. Tests cover every resolve_mirror_mode arm (incl. the regression and the indeterminate-probe case) and the probe's Promisor / NotPromisor / Unknown outcomes against real git repos. |
||
|
beardthelion
|
c9f43b0105 |
fix(core): route seed access through the zeroizing wrapper (#41) (#64)
* fix(core): route seed access through the zeroizing wrapper (#41) Remove Keypair::seed_bytes(), which returned a bare [u8; 32] copy of the Ed25519 private seed. Because [u8; 32] is Copy with no Drop, each caller's copy was released without being scrubbed, leaving secret key material in freed memory. The zeroizing accessor to_seed() -> Zeroizing<[u8; 32]> already existed beside it; this makes it the sole seed accessor and migrates the two in-crate callers (envelope open path and a crypto test) to it via deref coercion. No behavior change to envelope encryption: the derived X25519 secret is byte-identical, covered by the existing round-trip tests. Adds an identity test pinning to_seed() as the only seed accessor. * test(core): pin to_seed's Zeroizing return type, drop duplicate round-trip The to_seed_is_sole_seed_accessor test was named for a compile-time property its body couldn't assert, and its from_seed -> DID check duplicated from_seed_round_trip. Replace it with a type-annotated binding that fails to compile if to_seed ever returns a bare [u8; 32] again, plus the determinism check, which is the only behavior it uniquely covered. * fix(node): migrate node_seed pin caller to to_seed() main grew a third seed_bytes() caller (repos.rs, from #40) after this branch was cut. Route it through to_seed() too: node_seed is moved into a long-lived tokio::spawn pin task, so the Zeroizing wrapper scrubs the copy when that task ends. encrypt_and_pin takes &[u8; 32]; the wrapper deref-coerces, so the call site is unchanged. |
||
|
Kevin CodexandClaude Opus 4.8
|
97d37fd541 |
Perf/alloc and copy cleanups (#80)
* perf: trim allocations and redundant copies on hot paths - Cargo.toml: enable thin LTO + symbol stripping for release builds - git-remote-gitlawb: attach pack body after signing to move (not clone) potentially large pack bytes, halving peak push memory - gitlawb-attest: deserialize payload by reference instead of cloning it - core/http_sig: compare components by &str to avoid per-item String alloc - node/tigris: drop redundant to_vec() copy of the response body - node/rate_limit: only allocate the key String on first insert, not per request - node/sync: resolve the peer table once per batch instead of per item Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * perf(sync): skip peer lookup on empty sync batch Return early from process_batch when dequeue_pending_syncs yields nothing — the common case on the 30s idle poll — so an empty batch no longer fires an unnecessary list_peers() DB query (CodeRabbit, PR #80). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test(sync): cover per-batch peer origin-URL resolution Extract the peer -> origin-URL lookup from process_batch into a pure resolve_origin_url() helper and unit-test it (match + trailing-slash trim, unknown DID, empty peer list). This is the per-batch resolution PR #80 introduced; pulling it out of the DB-bound process_batch lets it be tested without Postgres. No behavior change. Addresses the needs-tests label. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(tigris): use a unique temp dir per archive extraction The fixed `.{file_name}.tmp-extract` path let two concurrent extractions of the same repo share one temp dir, so one's remove_dir_all/unpack could clobber the other's in-progress work. Suffix the temp dir with a fresh UUID so each extraction is isolated; this also removes the need to pre-clear a leftover dir (a unique name can't collide with an interrupted run). The on-failure cleanup still removes our own dir, and success consumes it via rename (CodeRabbit). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(tigris): serialize repo publish per path to close the swap race Unique temp dirs let extractions unpack in parallel, but the final remove_dir_all + rename swap could still interleave for the same local_path, causing a nondeterministic overwrite or a rename-onto-non-empty failure. Guard just the swap with a process-wide mutex keyed by local_path so only one extraction publishes per repo at a time; the expensive unpack stays parallel (CodeRabbit). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * docs(tigris): restore decompress_repo doc comment ordering The earlier publish_lock insertion landed between decompress_repo's doc comment and its fn, gluing the two doc blocks onto publish_lock and leaving decompress_repo undocumented. Reorder so each fn keeps its own doc. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * docs(tigris): note publish_lock map is never evicted Document the known unbounded-growth limitation of the per-path LOCKS map: bounded by the repo count so negligible for normal use, but high-volume deployments may want LRU/weak-ref eviction. No behavior change (CodeRabbit). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
|
a8cc33a185 |
fix(security): reject non-public peer URLs + prune poisoned peers (#78)
* fix(security): reject non-public peer URLs + prune poisoned peers The unauthenticated POST /api/v1/peers/announce only validated the http(s) scheme, so anyone could register loopback/private 'peers' (localhost:5432, localhost:22, internal admin URLs). On every push the node fans out signed sync-notify POSTs to all peers — turning this into an SSRF probe and burying the real peers under junk so node-origin repos stopped replicating. Observed on node.gitlawb.com: 126/194 peer rows were injected junk (fake DIDs z6MkScan…/z6MkAdmin…, localhost targets). - is_public_http_url(): reject non-http(s), loopback, unspecified, private (RFC1918 + ULA), link-local, and localhost/.local/.internal hosts. Applied at announce time. - db.prune_non_public_peers(): boot-time sweep removing already-injected rows, run from main() alongside prune_self_peers — self-heals poisoned tables on deploy. - tests for the validator (accept public, reject loopback/private/internal, incl. IPv6 [::1]). Co-Authored-By: OpenClaude <[email protected]> * fix(security): address PR #78 review — build, redirect SSRF, validation gaps Resolves the P1/P2 review feedback on the peer-announce SSRF fix: - main.rs: revert the unrelated, incomplete `storage::build()` refactor back to the tigris init from main (the `storage` module never existed, breaking the build). Keeps only the prune_non_public_peers boot call. - Redirect bypass: set redirect::Policy::none() on all peer-facing reqwest clients (shared http_client, sync worker, ping_peer) so a peer cannot answer `302 -> http://127.0.0.1/` and bypass the public-URL check. ping_peer now reuses the shared no-redirect client instead of bare reqwest::get. - Move the public-URL guard into db::upsert_peer so the bootstrap announce-back path (main.rs) can no longer re-poison the table after the prune. - is_public_http_url: strip trailing-dot FQDNs (`localhost.`), fold IPv4-mapped/compatible IPv6 (`::ffff:127.0.0.1`, `::127.0.0.1`) to IPv4 before range checks, and reject CGNAT (100.64.0.0/10). Added tests for each. - prune_non_public_peers: collect bad DIDs then DELETE ... WHERE did = ANY($1) in one statement, so a transient error no longer abandons remaining rows. Follow-up (separate issue): DNS-resolution rebinding guard rejecting peer hostnames that resolve to private IPs. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * fix(security): satisfy rustfmt + don't drop redirect guard on builder failure - peers.rs: collapse the CGNAT/private check onto one line (cargo fmt --check). - sync.rs: replace the unwrap_or_else(reqwest::Client::new()) fallback with .expect(). The default client follows redirects, so the fallback silently reintroduced the SSRF vector Policy::none() exists to close (CodeRabbit). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: OpenClaude <[email protected]> Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> |
||
|
beardthelion
|
2202b0097f |
feat(node): enforce per-route authorization across the REST and GraphQL surface (#87)
* test(node): add HTTP-API integration harness with CI Postgres
Adds a #[cfg(test)] test-support module: a migrated AppState over a real
#[sqlx::test] pool (test_state), a DB-free variant (test_state_lazy), the
assembled router (app), and signed_request_as for injecting an authenticated
DID without RFC-9421 signing. A test-callable run_migrations reuses the
production migrate() path, and sqlx gains the macros/migrate features so
#[sqlx::test] is available. CI's test job gets a postgres service behind a
pg_isready health gate so the DB-backed tests can run.
The proving test exercises the owner gate on PUT /visibility end to end:
non-owner is rejected with 400, owner succeeds. signed_request_as sets a JSON
content-type so the Json extractor does not 415 before the handler runs.
* feat(node): enforce per-route authorization on REST mutations
Closes the missing-authorization cluster on the signed REST surface, where
'authenticated' meant 'any participant' because identities are permissionless.
Adds two shared helpers in api/mod.rs: require_repo_owner (403) and did_matches,
a method-safe DID comparison that collapses the full did:key vs bare short form
without matching across methods (did:web/did:gitlawb share the base58 space).
Per-route gates:
- merge_pr is owner-only (subsumes branch protection); close_pr and close_issue
allow the repo owner or the PR/issue author (the issue author is read from the
git-JSON blob, with a None author falling back to owner-only).
- create_review, create_comment, create_issue_comment, and create_bounty are
read-gated (participants need read access, not ownership; this also closes the
private-repo bounty leak).
- the task handlers bind the acting DID to the authenticated signer.
- dispute_bounty, previously ungated, now requires the creator or claimant; the
other bounty and replica identity checks switch from raw equality to did_matches.
A source-level guard test asserts every in-scope mutation handler still carries
its expected gate marker, so a removed gate or an unclassified new route fails
CI. DB-backed tests (via the integration harness) cover merge owner-only and the
task signer-binding; did_matches has unit coverage for the cross-method case.
* feat(node): authenticate GraphQL mutations
GraphQL mutations were a fully unauthenticated parallel write path to the task
system (N2): the /graphql route had no auth layer and the resolvers took the
acting DID as plain arguments. Apply optional_signature to the /graphql POST so
a verified DID is attached when a signature is present (queries stay open), and
thread it into request-scoped data. Each mutation now reads the verified signer
via require_signer and binds the acting DID to it (rejecting an unsigned request
or one whose claimed delegator/assignee/by_did differs from the signer), reusing
did_matches for the comparison. Read-only subscriptions (/graphql/ws) are left
open. A resolver-level test covers the unsigned, mismatched, and matching cases.
* fix(node): path-scope the get_tree visibility gate
get_tree gated on the repo root ("/") regardless of the requested subtree, so
a caller denied a withheld subtree could still enumerate its filenames and blob
SHAs (N3). Gate on the requested path instead, mirroring get_blob, and reject
traversal segments. A cross-caller test shows the rejection is path-scoped: a
non-reader is denied the withheld subtree but passes the gate on a non-withheld
path.
* fix(node): authorize task completion and read-gate PR/issue creation
Two authorization gaps surfaced by an adversarial review of the auth stack,
both necessary-but-insufficient siblings of fixes already in it.
complete_task/fail_task bound the acting DID to the signer but never checked
the caller against the task's assignee, and finish_task updated by id alone, so
any authenticated did:key could finish or fail any task in any state. Load the
task and require the caller to be its assignee (REST and GraphQL), and add an
'AND status=claimed' predicate so only a claimed task transitions. The
now-unused by_did request fields are removed.
create_pr/create_issue resolved the repo with get_repo but skipped
authorize_repo_read, so a non-reader could open a PR (firing the owner's
webhooks) or file an issue against a private repo they cannot read. Gate both
on read access like their create_review/create_comment/create_bounty siblings.
Adds DB-backed tests covering non-assignee rejection (including the empty-body
bypass), the claimed-state predicate, and private-repo PR/issue denial.
* test(node): harden the authz drift-guard against false passes
The guard asserted a marker STRING appeared in each handler body, which a bare
identifier in a comment or log line could satisfy even after the real gate was
deleted, and its body slice over-ran on any handler not declared exactly
'pub async fn'.
- Markers are now gate-shaped: a call (require_repo_owner(, did_matches(,
authorize_repo_read() or a binding/comparison expression
(caller != &record.owner_did, let owner_did = auth.0), never a field name.
- Full-line comments are stripped before matching, so a gate that survives only
as a comment no longer counts as enforced.
- The body slice bounds at the next top-level fn item across pub async,
pub(crate) async, async, pub, and bare fn forms.
- Adds rows for the now-read-gated create_pr/create_issue and for fork_repo,
create_repo, set_profile, claim_bounty; register is excluded with a note (it
trusts the body did, tracked as P3 D3-1).
- Adds a self-test proving a comment-only marker does not satisfy a row.
* fix(node): P3 consistency batch from the adversarial review
Five low-severity items from the stack review. The sixth (a did_matches
false-negative on non-canonical multibase encodings of the same key) is left
as-is: it is a self-inflicted owner lockout unreachable with the conforming
client, never an impersonation vector, so decoding keys in the auth path is not
warranted.
- register bound the registered DID to the request body, not the signer, so a
signed caller could create or refresh a trust row under a victim DID. Bind it
to auth.0 (403 on mismatch) and add a drift-guard row now that it is gated.
- Bounty submit/approve/cancel returned 400 for an identity-mismatch denial
where dispute returned 403. Return Forbidden, matching the rest of the surface.
- The visibility/protect owner gates (require_owner and the protect.rs inline
checks) returned 400 for a non-owner; return 403 like require_repo_owner, and
assert the exact code in the harness.
- get_tree rejected ./.. segments but not empty interior segments, so a path
like secret//x could reach git while the gate saw a different string. Reject
empty interior segments too (the empty path remains the root listing).
- Tighten the get_tree path-scope test to assert an exact 200 on the
non-withheld branch so a future upstream 4xx/5xx cannot masquerade as
gate-pass.
Adds a register-binding regression test.
* fix(node): unify protect/visibility owner checks on did_matches
CodeRabbit flagged that protect_branch/unprotect_branch and visibility's
require_owner used a trailing-segment owner compare (split(':').next_back())
distinct from the hardened did_matches, so the two owner-match idioms could
drift. It is not exploitable (the authenticated caller is always the full did:
form while the short form is a bare suffix, so no wrong caller passes), but it
is a real inconsistency, and the trailing-segment form is a false-negative for a
bare-stored owner. Replace both with did_matches (collapses did:key full vs bare
on both sides, never across methods), and extend the drift guard to assert
require_owner itself uses did_matches, not just that it is called.
|
||
|
beardthelion
|
ff492b4521 |
feat(node): signature-gated agent self-deregister (#29) (#63)
* feat(node): signature-gated agent self-deregister (#29) Agents whose keys are rotated or compromised could never be removed from the registry, so an orphaned DID persisted and could win ?capability= routing. Give the DID's key holder a way to retire their own agent. A v5 migration adds an agent lifecycle status (default 'active') plus a deactivated_at timestamp. list_agents drops non-active agents via a pure filter_discoverable helper, so a revoked DID no longer appears in discovery or capability routing; get_agent stays unfiltered so a revoked DID still resolves and surfaces its status. DELETE /api/v1/agents/{did} lets an rfc9421-authenticated caller mark its own agent revoked. Authorization is on the verified AuthenticatedDid via a pure caller_matches_did gate (a DID can only retire itself), unit-tested including the cross-DID rejection. Revocation is terminal: register_agent's ON CONFLICT leaves status untouched, so re-registering a leaked DID cannot resurrect it. Authority is team-owned (the key holder); no external contract, chain, or operator power. Lost-key retirement and any successor pointer are out of scope. * fix(node): preserve original deactivated_at on repeat revoke A retried DELETE /api/v1/agents/{did} re-ran the revoke UPDATE and overwrote deactivated_at with a fresh timestamp, losing the true first-retired time. Use COALESCE(deactivated_at, $2) so the original timestamp is kept; the row still matches so the idempotent 204 is unchanged. |
||
|
beardthelion
|
2d4105e479 |
chore(deps): bump quinn-proto 0.11.14 -> 0.11.15 (RUSTSEC-2026-0185) (#83)
quinn-proto 0.11.14 (pulled transitively via libp2p's QUIC transport) is flagged by RUSTSEC-2026-0185. 0.11.15 is a semver-compatible patch that clears it; cargo audit passes clean after the bump. Lockfile-only, no manifest or code change, so it carries no ignore entry. Repo-wide: unblocks cargo audit on main and every open branch. |
||
|
beardthelion
|
803d83efdb |
fix(node): reap leaked git child processes (#53) (#61)
* fix(docker): reap orphan processes via tini + init, cap pids The node runs as PID 1 in the container and never reaps reparented orphans, so git grandchildren (e.g. pack-objects left behind when a served clone disconnects) accumulate as zombies until fork() fails with EAGAIN. Run under tini and set compose init: true so reparented orphans are reaped, and add pids_limit as a blast-radius cap. Refs #53 * fix(node): reap leaked git child processes (#53) Serving git over smart-HTTP could leak zombie processes that, over weeks, exhaust the PID table until fork() fails with EAGAIN and all git operations break until restart. The serve path (run_git_service) now runs git in its own process group and SIGKILLs that group when the request future is dropped (client disconnect) or returns early, so git and its pack-objects child are torn down together instead of orphaning the grandchild to PID 1 as a zombie. A drop guard, disarmed once the request completes cleanly, owns this teardown. create_issue and build_filtered_pack spawned a std Child and dropped it unwaited if the stdin write failed (and build_filtered_pack could panic on a missing stdin handle). Both now always wait the child before surfacing any error, so the failure path cannot leak. Adds libc for the group-kill syscall and three regression tests covering direct-child kill, grandchild reaping via the group, and the disarmed no-kill path. Refs #53 * fix(review): reap-then-disarm on serve-path errors; tini/comment nits Address code-review findings on the #53 fix: - run_git_service no longer early-returns on a stdin-write error. It captures the write result, always reaps the child, then disarms the group guard before surfacing the error. Previously a write/wait error path could drop the still-armed guard after the child was already reaped, so the group SIGKILL could in theory land on a reused pgid. (correctness, reliability, adversarial all flagged this region.) - Drop tini's -g flag: the node manages its own children and graceful shutdown, so group-wide signalling would only disturb in-flight git helpers on shutdown. - Correct the SAFETY comment (kill(2) is not relevant to async-signal-safety here) and the alive() test-helper comment (EPERM does not read as alive). * fix(node): SIGTERM (not SIGKILL) the served-git group on disconnect SIGKILL gave git no chance to run its signal cleanup, so an aborted receive-pack could leave a stale .git/*.lock that blocks the next push. SIGTERM lets git remove its lock files before exiting. Signalling the whole process group still reaches pack-objects directly, so the grandchild is terminated rather than orphaned, and a single immediate signal avoids any delayed-kill pid-reuse window. Refs #53 * docs(node): correct stale SIGKILL comments to SIGTERM in served-git teardown |
||
|
beardthelion
|
7533b74f79 |
ci: harden PR pipeline (audit, MSRV, beta matrix, release + docker smoke) (#57)
* ci: harden PR pipeline (audit, MSRV, release build, beta matrix, docker smoke) Restructure the single stable-only fmt+clippy+test job into separate jobs: fmt+clippy (stable), a test matrix on stable+beta (beta non-blocking), build --release, cargo audit, an MSRV check pinned to Rust 1.91, and a Docker build smoke test mirroring the release workflow. Add concurrency cancellation for superseded PR runs and per-job timeouts. Declare rust-version = "1.91" in [workspace.package] and inherit it in every member crate so the stated MSRV is a real, CI-enforced contract. * ci: disable credential persistence on checkout steps Set persist-credentials: false on all six actions/checkout steps. These jobs run untrusted PR code via cargo and docker build, and checkout's default leaves the GITHUB_TOKEN in the local git config where that code could read it. None of the jobs push or otherwise need git credentials. * fix(deps): bump rustls-webpki to 0.103.13 Clears three advisories flagged by the new cargo audit job: a reachable panic in CRL parsing and two name-constraint bypasses (wildcard and URI). Semver-compatible patch, lockfile only, no manifest or code change. cargo also corrects a stale lockfile reference while rewriting Cargo.lock: data-encoding-macro-internal 0.1.17 (a build-time macro crate) requires syn 1.x, and its dependency entry is re-pinned from syn 2.0.117 to the 1.0.109 it actually resolves to. Unrelated to the bump, unavoidable when cargo touches the lock, and the more-correct state. * ci(audit): suppress no-fix advisories with justification and a re-check The cargo audit job stays hard-fail. The remaining three advisories have no available upstream fix, so they are suppressed in .cargo/audit.toml (the location cargo-audit actually reads; a repo-root audit.toml is not read), each with an inline rationale and removal condition: - RUSTSEC-2026-0118/0119 (hickory-proto 0.25.2, via libp2p-dns 0.44): reachable, since the node wraps QUIC in the system DNS resolver. No fix until libp2p-dns ships hickory 0.26. Accepted, reachable risk. - RUSTSEC-2023-0071 (rsa Marvin): in the lockfile via sqlx's unconditional sqlx-mysql, but not linked (sqlx is postgres-only), so no RSA oracle exists; no upstream fix regardless. Two controls keep the suppression honest. The PR audit job echoes the active ignore list to the job summary, so a green check is not read as advisory-clean. A weekly scheduled job re-runs cargo audit without the ignores for visibility and hard-fails if hickory-proto moves off 0.25.x while the ignores remain, so the suppression cannot silently outlive the upstream fix. * ci(audit): reference tracking issue #76 in the hickory ignores * ci(audit): treat hickory-proto removal as drift too The scheduled drift guard only failed when hickory-proto was present and off 0.25.x, so removing it from Cargo.lock entirely left the RUSTSEC-2026-0118 /0119 ignores in place with the guard still passing. Treat an absent version as drift as well, so dead ignore entries are caught whether the dependency is bumped past 0.25.x or removed. |
||
|
beardthelion
|
abdc775770 |
feat(node): blind recipient identities at rest and gate B1 by repo readability (#40)
* feat(node): blind recipient identities at rest and gate B1 by repo readability
The origin no longer stores recipient DIDs. Migration v5 replaces the
encrypted_blobs.recipients column with an opaque, node-keyed recipients_tag
used only to detect a recipient-set change for re-seal. B1 discovery and fetch
are now gated by the same repo-readability check the git read path uses, not by
per-recipient matching; decryption is gated by the envelope crypto, so a
non-recipient who can read the repo sees a blob's {oid, cid} but cannot open it.
encrypt_and_pin keys the tag from the node seed and returns {oid, cid}; the
Arweave manifest tuple drops the now-unused recipient vec.
A DB compromise no longer reveals the reader set; recovering it would require
brute-forcing candidate DID sets against the keyed tag with the node key.
* chore: remove redundant .gitignore entry (covered by local exclude)
* fix(node): gate /replicate by repo readability; harden reseal on DB error
Address review on the at-rest blinding change:
- The encrypted-blobs/replicate listing returned {oid, cid} with no visibility
check, so a non-readable repo's blob index was reachable by an unauthenticated
caller who guessed {owner}/{repo}. Gate it by the same repo-readability check
discovery and fetch use. For the intended case (a public repo with withheld
subtrees) the public root keeps this open to peers; only fully non-readable
repos are withheld, which is the desired behavior.
- encrypt_and_pin treated a recipients_tag DB read error as a cache miss and
resealed, causing avoidable IPFS writes during a partial outage; skip and retry
on the next push instead.
- Correct the get_encrypted_blob doc comment to describe repo-readability access.
|
||
|
beardthelion
|
e37ea7fec6 |
fix(node): enforce path-scoped visibility on the REST read API (#52)
* fix(node): enforce path-scoped visibility on the REST read API The repo-scoped REST read endpoints (blob, tree, commits, refs, changelog, pulls, pr diff, repo metadata) read straight from disk after a bare repo lookup, with no visibility check, so private repositories and private subtrees of public repositories were served to unauthenticated callers, bypassing the withholding the git and replication paths enforce. Add a shared authorize_repo_read gate (resolve repo, load rules, visibility_check, 404 on deny) and call it from every repo-scoped read handler. get_blob gates at the requested file path; get_pr_diff withholds the whole diff when it touches a path the caller cannot read. Attach optional_signature to the read routes so authorized readers authenticate. Listings stay gated at the repo root, consistent with mode B exposing tree and commit SHAs over git. Closes #51 * fix(node): gate PR reviews/comments and fork on repo read visibility The REST visibility gate added in this PR missed three read surfaces: list_reviews and list_comments served a private repo's PR reviews and comments to unauthenticated callers, and fork_repo cloned a full mirror of the source without checking the caller could read it. Route all three through authorize_repo_read at the repo root, matching the git-protocol gate. Also make branch_diff_names fail closed: it ignored git's exit status and returned an empty path list on error, which would let get_pr_diff serve a diff it should have withheld. Fix changelog.rs import order to satisfy cargo fmt. * fix(node): NUL-delimit branch_diff_names to keep path-gate enforcement exact git diff --name-only quotes and escapes paths containing newlines, so they no longer match the visibility globs in get_pr_diff. On a public repo where / is allowed but a subtree is denied, a touched restricted path could be missed by the deny check and the full PR diff disclosed. Use -z and split on NUL to preserve exact path bytes. |
||
|
beardthelion
|
197c4e1cf1 |
Add contributor templates, issue forms, and PR triage automation (#58)
* docs: add pull request template * docs: add structured issue forms and template chooser config * chore: add CODEOWNERS for CI, release, and security paths * ci: add quality-signal PR triage workflow * ci: add label-gated auto-stale workflow for ignored PRs * docs: document PR quality bar and triage/stale policy in CONTRIBUTING * ci: recognize inline #[cfg(test)] tests and fix triage comment grammar |