2 Commits

Include renames

Author	SHA1	Message	Date
JATMNandClaude Opus 5.5	a23c81d75a	fix(deps): bump follow-redirects and override ws in scripts/ (#525) ... Resolve Dependabot alerts for the ANS-104 golden-vector tooling in scripts/: - follow-redirects 1.16.0 -> 1.16.2 (lockfile only, within axios's ^1.16.0 range): GHSA-mrwp-f3q8-3g9p, GHSA-q3jh-wrc6-jgq8, GHSA-8r9p-f939-6h3c. - ws 8.18.0 -> 8.22.1 via an npm override. @ethersproject/providers 5.8.0 (latest v5) pins ws to exactly 8.18.0, so a lockfile bump alone cannot move it: GHSA-96hv-2xvq-fx4p, GHSA-58qx-3vcg-4xpx. elliptic (GHSA-848j-6mx2-7j84, low) has no patched release and is left as is. Co-authored-by: Claude Opus 5.5 <[email protected]>	2026-11-06 01:00:41 +00:00
beardthelionandGravirei	2c341f71ac	fix(node): ANS-104 codec and recovery policy types (#26 split 2/4) (#524) ... * fix(node): ANS-104 transport, three-outcome probe, verify endpoint (#26 split 2/4) Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split into four narrow PRs. This is Split PR 2 (Arweave transport and verification). The P1 finding the reviewer assigned to this split: the legacy `anchor_item_present` mapped `BAD_REQUEST` and `GONE` to `Ok(false)`, and the recovery code interpreted false as permission to reclaim the row and pay for another upload. Neither 400 nor 410 proves a previously paid item was never accepted: 410 can describe an artifact that existed but is no longer served, and 400 can be produced by gateway, proxy, or routing failure. The concrete failure sequence the reviewer named: bundler accepts the item, the response or terminal DB write is lost, recovery probes the persisted item_id, the gateway returns 400 or 410, and the node pays for a second immutable item for the same transition. Fix: model the probe result as three outcomes, and authorize a re-upload only on a trustworthy, protocol-defined absence. NEW MODULE crates/gitlawb-node/src/ans104.rs - DataItem: the on-wire ANS-104 shape (signature, owner, target, anchor, tags, data — all base64url-encoded except target/anchor). - deep_hash: the canonical signing input per the spec. The signature is over a SHA-256 mix of the dataitem/list/map discriminators, the signature type, owner, target, anchor, the recursively-hashed tag list, and the data payload. A hand-rolled sha256 of the JSON body would produce a hash no Arweave gateway would recognize; the deep-hash is the only correct form. - sign_data_item: Ed25519 over deep_hash. - verify_data_item: parses the owner, checks the signature against an expected pubkey, and reports a specific failure reason on each branch. - 7 unit tests covering round-trip, flipped signature, wrong expected key, mutated data, wire-shape round-trip, deep-hash stability, and the empty-tags edge case. NEW MODULE crates/gitlawb-node/src/arweave_v2.rs - ProbeOutcome: Present (2xx, body parses as ANS-104, sig verifies), DefinitivelyAbsent (404 with a known JSON body shape), Indeterminate (everything else). The reviewer's three-outcome model. - ProbeOutcome::permits_reupload: the recovery policy. ONLY DefinitivelyAbsent authorizes a paid re-upload. Indeterminate keeps the outbox non-terminal; Present skips re-payment. - probe_anchor_item: the gateway probe. 2xx bound to a different item id is Indeterminate, not Present. 2xx with a body that does not parse as ANS-104 is Indeterminate. Oversized bodies are Indeterminate (defense against body-stuffing). - read_capped_body: enforces PROBE_MAX_BODY_BYTES (1 MiB) on the full body, with a Content-Length fast-path that rejects before reading. - verify_anchor: the full path the public verify endpoint takes. Fetches the data item, parses it, verifies the Ed25519 signature against the persisted node_did, decodes the data payload as JSON, and reports the result. - 15 unit tests pinning each classification boundary: 400/410/5xx/transport/2xx-non-json/2xx-bad-sig/2xx-different- owner are all Indeterminate; 404 with empty or known-JSON body is DefinitivelyAbsent; 2xx with a valid signed item is Present; the re-upload policy is exhaustive. NEW DB METHOD crates/gitlawb-node/src/db/mod.rs - get_arweave_anchor_by_item_id: looks up an existing arweave_anchors row by its id (which is the Irys tx id, the same value the Arweave gateway uses to serve the data item). Used by the public verify endpoint to fetch the persisted node_did so the envelope signature can be verified against it. NEW CONFIG crates/gitlawb-node/src/config.rs - GITLAWB_ARWEAVE_GATEWAY_URL: the Arweave gateway the probe reads from. Defaults to https://arweave.net; operators can point it at a private mirror. NEW HTTP ENDPOINT crates/gitlawb-node/src/api/arweave.rs - GET /api/v1/arweave/anchors/verify/{item_id}: the public verify endpoint. Returns {item_id, status, verified, owner_did, data_payload, error} where status is one of "verified" / "definitively_absent" / "indeterminate". - 3 endpoint-level tests covering the verified, 404, and 400 shapes via the live router, asserting the public surface surfaces the three outcomes. WIRE-UP crates/gitlawb-node/src/server.rs - The verify route is registered alongside the existing /api/v1/arweave/anchors list route. NOT IN THIS SLICE (the bundler upload, the next commit): - The handler that builds a DataItem from a ref-cert and POSTs it to the bundler is not in this PR. The ANS-104 module is ready to be called by it; the cert/push-event flow in PR 1 persists the row that the bundler upload would consume. - PR 1's recovery drain produces an `anchor_jobs` row keyed on the (repo, ref, old, new) tuple; the next slice reads that row, builds the ANS-104 data item, POSTs to the bundler, and writes the resulting tx id to arweave_anchors. Compiles clean, 1113 tests pass with 0 regressions, clippy clean under -D warnings, fmt clean. Cross-PR overlap (declared in the PR description): - #134 (anchors auth): composes. The verify endpoint reads from arweave_anchors; #134's auth layer applies unchanged. The probe model is server-side and unauthenticated by design. - #285 (advisory-lock session affinity): independent. The probe is read-only. - #306 (Content-Digest on signed requests): independent. The probe does not sign requests. - #314 (small-order Ed25519): composes. The verify path calls Did::to_verifying_key which already enforces the small-order check from #314. - #324 (libp2p keypair persistence): independent. - #325 (gossip ref-update auth): independent. The verify path is for the public Arweave gateway, not gossip. - #382 (replication withheld-subtree trees): independent. * fix(node): address reviewer findings on #26 split 2/4 (transport) - P1 (DB lookup): get_arweave_anchor_by_item_id now filters on irys_tx_id (the externally-routable item id) rather than the internal UUID primary key. The production writer stores a fresh UUID in `id` and the Irys response / ANS-104 derived id in `irys_tx_id`; the old filter would always 404. New integration tests exercise the production writer path with distinct values for the two columns. - P1 (verify-gate): verify_anchor now gates on repo read via authorize_repo_read. The route is split in server.rs: the public list_anchors stays ungated (#134 tracks that surface), the verify route gets optional_signature per the team memory `axum-layer-vs-merge-pitfall.md`. Denials on row-lookup, gate, and missing repo all collapse to the same opaque 404 so the public endpoint does not leak anchor-row existence. - P1 (ANS-104 deep-hash): rewritten to the verified SHA-384 recursive list/blob algorithm (matches `arbundles` JS reference). deep_hash returns [u8; 48]; signature is over the raw 48-byte digest. Three reference vectors from `Irys-xyz/arbundles/src/__tests__/deepHash.spec.ts` are bit-exact-asserted as the interop canary per the team memory `self-roundtrip-tests-do-not-prove-interop.md`. - P1 (artifact-identity): v2 verify now derives the protocol id via DataItem::id() (base64url(SHA256(signature))) and requires equality with the requested URL item_id. A node key signs many data items, so a valid signature only proves who signed the response — not that the served item is the one the caller asked to verify. The team memory `verify-against-artifact-id-not-signer.md` is the policy. - P1 (verifier can't read this head's writes): verify_anchor now accepts BOTH the v2 ANS-104 format and the v1 raw-JSON shape the live path on this branch actually writes. The v1 path does a field-equality check on repo, ref_name, old_sha, new_sha, node_did against the persisted row (no signature in v1; Irys storage plus the JSON parse are the integrity guarantee). - P2 (ProbeOutcome in result): AnchorVerifyResult now carries outcome: ProbeOutcome. The HTTP handler maps the structured field to the status string instead of parsing the human- readable error message — any future wording change in arweave_v2.rs can no longer silently reclassify a DefinitivelyAbsent as Indeterminate. - P2 (empty 404 body): classify_404 returns Indeterminate for bodyless 404s. The team memory `distinguish-unknown-from-empty.md` is the policy: a bodyless 404 from a proxy is not a proof of absence, and the recovery policy (permits_reupload -> true on DefinitivelyAbsent) authorizes a paid, irreversible re-upload. - P2 (streaming cap): read_capped_body no longer uses resp.bytes() (which buffers the full response before checking size). It now reads chunk-by-chunk and aborts the moment the cumulative byte count crosses the cap, so a chunked response without Content-Length cannot force unbounded allocation. * fix(node): address three reviewer findings on #26 split 2/4 (round 2) * fix(node): address five reviewer findings on #26 split 2/4 (round 3) * fix(node): address reviewers round-4 on #26 split 2/4 (interop) * fix(node): address coderabbit on #26 split 2/4 (raw tags, vint overflow, golden key) * fix(node): enforce artifact-id in probe_item on #26 split 2/4 * fix(node): require expected owner for probe Present on #26 split 2/4 * fix(node): narrow #26 split 2/4 to codec and policy types * fix(node): silence dead-code for deferred slice API on #26 split 2/4 * fix(node): doc and bounds-check cleanup in ans104 on #26 split 2/4 * docs(node): point tests doc at Ed25519 golden script on #26 split 2/4 * fix(node): strict owner lengths, sign-before-hash on #26 split 2/4 * fix(node): keep ans104 tags and field widths consistent on hash and encode DataItem keeps both the decoded `tags` and the original Avro `raw_tags`, both public. After a parse, changing `tags` left deep_hash and verify_data_item using the stale raw bytes, and to_binary wrote a header tag count that no longer matched the payload, so the frame failed to parse back. Take the tag block for hashing and encoding from one helper that decodes `raw_tags` and requires it to equal `tags`, and fail closed otherwise. The header count comes from the same validated pairs. Parsed multi-block and size-prefixed items still re-encode byte-exact and keep their signatures. deep_hash now also rejects a non-empty target or anchor that is not 32 bytes, as to_binary already does, so sign_data_item fails instead of signing an item that cannot be encoded. The legacy non-base64url anchor fallback is kept. The module and signature-type docs now say binary parsing accepts sigtypes 1 to 4 while signing and verifying are Ed25519 only on this slice. * build(scripts): declare the golden script's direct npm dependencies ans104_golden_ed25519.mjs imports bs58, @noble/ed25519 and base64url, and ans104_golden.mjs imports base64url, but package.json listed only arbundles, so they resolved only through hoisting. Declare them at the versions already locked, update the lockfile root entry (no resolved version or hash changes), and note the regeneration command in the golden output file. --------- Co-authored-by: Gravirei <[email protected]>	2026-11-05 21:12:46 +00:00