Compare commits

...
Author SHA1 Message Date
beardthelion 4185fb1b51 build(scripts): declare the golden script's direct npm dependencies
ans104_golden_ed25519.mjs imports bs58, @noble/ed25519 and base64url, and
ans104_golden.mjs imports base64url, but package.json listed only arbundles, so
they resolved only through hoisting. Declare them at the versions already
locked, update the lockfile root entry (no resolved version or hash changes),
and note the regeneration command in the golden output file.
2026-11-05 14:50:05 -06:00
beardthelion 52df22416d fix(node): keep ans104 tags and field widths consistent on hash and encode
DataItem keeps both the decoded `tags` and the original Avro `raw_tags`, both
public. After a parse, changing `tags` left deep_hash and verify_data_item
using the stale raw bytes, and to_binary wrote a header tag count that no
longer matched the payload, so the frame failed to parse back.

Take the tag block for hashing and encoding from one helper that decodes
`raw_tags` and requires it to equal `tags`, and fail closed otherwise. The
header count comes from the same validated pairs. Parsed multi-block and
size-prefixed items still re-encode byte-exact and keep their signatures.

deep_hash now also rejects a non-empty target or anchor that is not 32 bytes,
as to_binary already does, so sign_data_item fails instead of signing an item
that cannot be encoded. The legacy non-base64url anchor fallback is kept.

The module and signature-type docs now say binary parsing accepts sigtypes 1
to 4 while signing and verifying are Ed25519 only on this slice.
2026-11-05 14:50:05 -06:00
beardthelion c97c6447c3 Merge remote-tracking branch 'origin/main' into tmp-385-rehome 2026-11-05 13:25:39 -06:00
Gravirei 950698a788 fix(node): strict owner lengths, sign-before-hash on #26 split 2/4 2026-09-07 13:13:59 +06:00
Gravirei a3d7903cf3 docs(node): point tests doc at Ed25519 golden script on #26 split 2/4 2026-09-07 01:00:34 +06:00
Gravirei c9292d8dd7 fix(node): doc and bounds-check cleanup in ans104 on #26 split 2/4 2026-09-06 21:04:50 +06:00
Gravirei 0271f49b0b fix(node): silence dead-code for deferred slice API on #26 split 2/4 2026-09-06 09:17:03 +06:00
Gravirei 8ffe22e134 fix(node): narrow #26 split 2/4 to codec and policy types 2026-09-06 09:07:45 +06:00
Gravirei 91a8ac6e10 fix(node): require expected owner for probe Present on #26 split 2/4 2026-09-05 21:58:35 +06:00
Gravirei 1e0a0efce0 fix(node): enforce artifact-id in probe_item on #26 split 2/4 2026-09-05 18:02:35 +06:00
Gravirei a6726fc355 fix(node): address coderabbit on #26 split 2/4 (raw tags, vint overflow, golden key) 2026-09-05 09:27:16 +06:00
Gravirei c4e7c2088f fix(node): address reviewers round-4 on #26 split 2/4 (interop) 2026-09-05 09:08:39 +06:00
Gravirei f4c23401d0 fix(node): address five reviewer findings on #26 split 2/4 (round 3) 2026-08-31 00:18:51 +06:00
Gravirei 5691bf2854 fix(node): address three reviewer findings on #26 split 2/4 (round 2) 2026-08-30 20:47:57 +06:00
Gravirei f2658b4701 fix(node): address reviewer findings on #26 split 2/4 (transport)
- P1 (DB lookup): get_arweave_anchor_by_item_id now filters on
  irys_tx_id (the externally-routable item id) rather than the
  internal UUID primary key. The production writer stores a fresh
  UUID in `id` and the Irys response / ANS-104 derived id in
  `irys_tx_id`; the old filter would always 404. New
  integration tests exercise the production writer path with
  distinct values for the two columns.
- P1 (verify-gate): verify_anchor now gates on repo read via
  authorize_repo_read. The route is split in server.rs: the
  public list_anchors stays ungated (#134 tracks that surface),
  the verify route gets optional_signature per the team memory
  `axum-layer-vs-merge-pitfall.md`. Denials on row-lookup, gate,
  and missing repo all collapse to the same opaque 404 so the
  public endpoint does not leak anchor-row existence.
- P1 (ANS-104 deep-hash): rewritten to the verified SHA-384
  recursive list/blob algorithm (matches `arbundles` JS
  reference). deep_hash returns [u8; 48]; signature is over the
  raw 48-byte digest. Three reference vectors from
  `Irys-xyz/arbundles/src/__tests__/deepHash.spec.ts` are
  bit-exact-asserted as the interop canary per the team memory
  `self-roundtrip-tests-do-not-prove-interop.md`.
- P1 (artifact-identity): v2 verify now derives the protocol id
  via DataItem::id() (base64url(SHA256(signature))) and requires
  equality with the requested URL item_id. A node key signs
  many data items, so a valid signature only proves who signed
  the response — not that the served item is the one the caller
  asked to verify. The team memory
  `verify-against-artifact-id-not-signer.md` is the policy.
- P1 (verifier can't read this head's writes): verify_anchor
  now accepts BOTH the v2 ANS-104 format and the v1 raw-JSON
  shape the live path on this branch actually writes. The v1
  path does a field-equality check on repo, ref_name, old_sha,
  new_sha, node_did against the persisted row (no signature in
  v1; Irys storage plus the JSON parse are the integrity
  guarantee).
- P2 (ProbeOutcome in result): AnchorVerifyResult now carries
  outcome: ProbeOutcome. The HTTP handler maps the structured
  field to the status string instead of parsing the human-
  readable error message — any future wording change in
  arweave_v2.rs can no longer silently reclassify a
  DefinitivelyAbsent as Indeterminate.
- P2 (empty 404 body): classify_404 returns Indeterminate for
  bodyless 404s. The team memory
  `distinguish-unknown-from-empty.md` is the policy: a
  bodyless 404 from a proxy is not a proof of absence, and the
  recovery policy (permits_reupload -> true on
  DefinitivelyAbsent) authorizes a paid, irreversible re-upload.
- P2 (streaming cap): read_capped_body no longer uses
  resp.bytes() (which buffers the full response before checking
  size). It now reads chunk-by-chunk and aborts the moment the
  cumulative byte count crosses the cap, so a chunked response
  without Content-Length cannot force unbounded allocation.
2026-08-30 02:08:34 +06:00
Gravirei a53a63a5f9 fix(node): ANS-104 transport, three-outcome probe, verify endpoint (#26 split 2/4)
Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split
into four narrow PRs. This is Split PR 2 (Arweave transport and
verification).

The P1 finding the reviewer assigned to this split: the legacy
`anchor_item_present` mapped `BAD_REQUEST` and `GONE` to `Ok(false)`,
and the recovery code interpreted false as permission to reclaim the
row and pay for another upload. Neither 400 nor 410 proves a
previously paid item was never accepted: 410 can describe an
artifact that existed but is no longer served, and 400 can be
produced by gateway, proxy, or routing failure. The concrete
failure sequence the reviewer named: bundler accepts the item,
the response or terminal DB write is lost, recovery probes the
persisted item_id, the gateway returns 400 or 410, and the node
pays for a second immutable item for the same transition.

Fix: model the probe result as three outcomes, and authorize a
re-upload only on a trustworthy, protocol-defined absence.

NEW MODULE crates/gitlawb-node/src/ans104.rs
  - DataItem: the on-wire ANS-104 shape (signature, owner, target,
    anchor, tags, data — all base64url-encoded except target/anchor).
  - deep_hash: the canonical signing input per the spec. The
    signature is over a SHA-256 mix of the dataitem/list/map
    discriminators, the signature type, owner, target, anchor, the
    recursively-hashed tag list, and the data payload. A
    hand-rolled sha256 of the JSON body would produce a hash no
    Arweave gateway would recognize; the deep-hash is the only
    correct form.
  - sign_data_item: Ed25519 over deep_hash.
  - verify_data_item: parses the owner, checks the signature against
    an expected pubkey, and reports a specific failure reason on
    each branch.
  - 7 unit tests covering round-trip, flipped signature, wrong
    expected key, mutated data, wire-shape round-trip, deep-hash
    stability, and the empty-tags edge case.

NEW MODULE crates/gitlawb-node/src/arweave_v2.rs
  - ProbeOutcome: Present (2xx, body parses as ANS-104, sig
    verifies), DefinitivelyAbsent (404 with a known JSON body
    shape), Indeterminate (everything else). The reviewer's
    three-outcome model.
  - ProbeOutcome::permits_reupload: the recovery policy. ONLY
    DefinitivelyAbsent authorizes a paid re-upload. Indeterminate
    keeps the outbox non-terminal; Present skips re-payment.
  - probe_anchor_item: the gateway probe. 2xx bound to a different
    item id is Indeterminate, not Present. 2xx with a body that
    does not parse as ANS-104 is Indeterminate. Oversized bodies
    are Indeterminate (defense against body-stuffing).
  - read_capped_body: enforces PROBE_MAX_BODY_BYTES (1 MiB) on the
    full body, with a Content-Length fast-path that rejects
    before reading.
  - verify_anchor: the full path the public verify endpoint takes.
    Fetches the data item, parses it, verifies the Ed25519
    signature against the persisted node_did, decodes the data
    payload as JSON, and reports the result.
  - 15 unit tests pinning each classification boundary:
    400/410/5xx/transport/2xx-non-json/2xx-bad-sig/2xx-different-
    owner are all Indeterminate; 404 with empty or known-JSON
    body is DefinitivelyAbsent; 2xx with a valid signed item is
    Present; the re-upload policy is exhaustive.

NEW DB METHOD crates/gitlawb-node/src/db/mod.rs
  - get_arweave_anchor_by_item_id: looks up an existing
    arweave_anchors row by its id (which is the Irys tx id, the
    same value the Arweave gateway uses to serve the data item).
    Used by the public verify endpoint to fetch the persisted
    node_did so the envelope signature can be verified against it.

NEW CONFIG crates/gitlawb-node/src/config.rs
  - GITLAWB_ARWEAVE_GATEWAY_URL: the Arweave gateway the probe
    reads from. Defaults to https://arweave.net; operators can
    point it at a private mirror.

NEW HTTP ENDPOINT crates/gitlawb-node/src/api/arweave.rs
  - GET /api/v1/arweave/anchors/verify/{item_id}: the public
    verify endpoint. Returns {item_id, status, verified,
    owner_did, data_payload, error} where status is one of
    "verified" / "definitively_absent" / "indeterminate".
  - 3 endpoint-level tests covering the verified, 404, and 400
    shapes via the live router, asserting the public surface
    surfaces the three outcomes.

WIRE-UP crates/gitlawb-node/src/server.rs
  - The verify route is registered alongside the existing
    /api/v1/arweave/anchors list route.

NOT IN THIS SLICE (the bundler upload, the next commit):
  - The handler that builds a DataItem from a ref-cert and POSTs
    it to the bundler is not in this PR. The ANS-104 module is
    ready to be called by it; the cert/push-event flow in PR 1
    persists the row that the bundler upload would consume.
  - PR 1's recovery drain produces an `anchor_jobs` row keyed on
    the (repo, ref, old, new) tuple; the next slice reads that row,
    builds the ANS-104 data item, POSTs to the bundler, and writes
    the resulting tx id to arweave_anchors.

Compiles clean, 1113 tests pass with 0 regressions, clippy clean
under -D warnings, fmt clean.

Cross-PR overlap (declared in the PR description):

  - #134 (anchors auth): composes. The verify endpoint reads from
    arweave_anchors; #134's auth layer applies unchanged. The
    probe model is server-side and unauthenticated by design.
  - #285 (advisory-lock session affinity): independent. The probe
    is read-only.
  - #306 (Content-Digest on signed requests): independent. The
    probe does not sign requests.
  - #314 (small-order Ed25519): composes. The verify path calls
    Did::to_verifying_key which already enforces the small-order
    check from #314.
  - #324 (libp2p keypair persistence): independent.
  - #325 (gossip ref-update auth): independent. The verify path
    is for the public Arweave gateway, not gossip.
  - #382 (replication withheld-subtree trees): independent.
2026-08-29 01:45:46 +06:00
9 changed files with 3470 additions and 0 deletions

No files matched your search

File diff suppressed because it is too large. Load diff
+62
View File
@@ -0,0 +1,62 @@
//! #26 Split PR 2 — three-outcome recovery policy types.
//!
//! This module owns the recovery classification policy the reviewer
//! demanded: only a trustworthy, protocol-defined absence authorizes
//! a paid re-upload. Everything else stays non-terminal.
//!
//! Deliberately narrow: the ANS-104 codec lives in `ans104`, and the
//! gateway probe, the `verify_anchor` path, and the public verify
//! endpoint are deferred until the uploader, retrieval proof, and
//! recovery consumer can be proven as one vertical slice (reviewer 2,
//! round 5). A probe against a real provider contract — same-network
//! upload/read pair, redirect-safe client, envelope-supplying read
//! API, provider-documented absence evidence — belongs in that slice,
//! not here. What ships here is the policy type plus the rule that
//! only `DefinitivelyAbsent` permits spending another upload.
/// Outcome of a gateway probe for a persisted `item_id`.
// Policy surface for the recovery slice: no construction on this
// head (the probe arrives with the provider contract); the unit
// test pins the rule meanwhile.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ProbeOutcome {
/// The persisted item was served back with a verifiable envelope
/// bound to the requested id. No re-upload allowed.
Present,
/// Trustworthy, protocol-defined proof the item was never
/// accepted. Authorizes re-upload. The exact evidence that
/// establishes this is defined by the provider contract in the
/// vertical slice that introduces the probe — never by a
/// mock-only body shape.
DefinitivelyAbsent,
/// Anything else: transport failure, oversized body, bad
/// signature, id mismatch, ambiguous gateway response. The
/// outbox stays non-terminal.
Indeterminate,
}
impl ProbeOutcome {
/// True iff the recovery code is allowed to spend another paid
/// upload request. Only `DefinitivelyAbsent` qualifies.
#[allow(dead_code)] // the recovery consumer lives in a later slice
pub fn permits_reupload(self) -> bool {
matches!(self, ProbeOutcome::DefinitivelyAbsent)
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Only `DefinitivelyAbsent` authorizes a paid re-upload. This is
/// the policy the reviewer named: collapsing any ambiguous outcome
/// to absent spends a second immutable upload for the same
/// transition.
#[test]
fn only_definitively_absent_authorizes_reupload() {
assert!(!ProbeOutcome::Present.permits_reupload());
assert!(ProbeOutcome::DefinitivelyAbsent.permits_reupload());
assert!(!ProbeOutcome::Indeterminate.permits_reupload());
}
}
+2
View File
@@ -1,5 +1,7 @@
mod ans104;
mod api;
mod arweave;
mod arweave_v2;
mod auth;
mod bootstrap;
mod cert;
+4
View File
@@ -261,6 +261,10 @@ pub fn build_router(state: AppState) -> Router {
.merge(Router::new().route("/api/v1/ipfs/pins", get(ipfs::list_pins)));
// ── Arweave permanent anchors ──────────────────────────────────────────
// List endpoint only (public; issue #134 tracks surfacing
// visibility rules on list). The verify endpoint and its probe
// are deferred to the vertical slice with the uploader and a
// real provider contract (reviewer 2, #26 split 2/4 round 5).
let arweave_routes = Router::new().route("/api/v1/arweave/anchors", get(arweave::list_anchors));
// ── Bounty routes (write — require HTTP Signature) ─────────────────
+33
View File
@@ -0,0 +1,33 @@
import { createData, EthereumSigner } from "arbundles";
import { createHash } from "node:crypto";
import base64url from "base64url";
const data = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()_+-=[]{};':\",./<>?`~";
const tags = [
{ name: "tag1", value: "value1" },
{ name: "tag2", value: "value2" },
];
const anchor = "thisSentenceIs32BytesLongTrustMe";
const target = "OXcT1sVRSA5eGwt2k6Yuz8-3e3g9WJi5uSE99CWqsBs";
const signer = new EthereumSigner("8da4ef21b864d2cc526dbdb2a120bd2874c36c9d0a1fb7f8c63d7f7a8b41de8f");
const item = createData(data, signer, { anchor, target, tags });
// Sign so the captured signature is real: `createData` alone leaves a
// zeroed placeholder, whose id is just sha256(zeros).
await item.sign(signer);
const raw = item.getRaw();
const id = item.id;
// `item.signature` is base64url(rawSignature). Decode and sha256.
const sigBytes = base64url.toBuffer(item.signature);
const idCheck = base64url.encode(createHash("sha256").update(sigBytes).digest());
if (idCheck !== id) {
console.error(`MISMATCH: id=${id} sha256(sig)=${idCheck}`);
process.exit(1);
}
console.log("id =", id);
console.log("signature_b64=", item.signature);
console.log("signature_len=", sigBytes.length);
console.log("binary_len =", raw.length);
console.log("binary_hex =", raw.toString("hex"));
+40
View File
@@ -0,0 +1,40 @@
import { createData, SolanaSigner } from "arbundles";
import bs58 from "bs58";
import { getPublicKey } from "@noble/ed25519";
import { createHash } from "node:crypto";
import base64url from "base64url";
// Deterministic Ed25519 keypair from seed 0x01 * 32.
const seed = Buffer.alloc(32, 0x01);
const pub = await getPublicKey(seed);
const secret64 = Buffer.concat([seed, Buffer.from(pub)]);
const signer = new SolanaSigner(bs58.encode(secret64));
const data = "hello gitlawb ed25519 golden";
const tags = [
{ name: "App-Name", value: "gitlawb" },
{ name: "Schema", value: "gitlawb/ref-update/v1" },
];
const item = createData(data, signer, { tags });
await item.sign(signer);
const raw = Buffer.from(item.getRaw());
const id = item.id;
const sigBytes = base64url.toBuffer(item.signature);
const idCheck = base64url.encode(createHash("sha256").update(sigBytes).digest());
if (idCheck !== id) {
console.error(`MISMATCH: id=${id} sha256(sig)=${idCheck}`);
process.exit(1);
}
const sigData = await item.getSignatureData();
console.log("id =", id);
console.log("signature_b64=", item.signature);
console.log("signature_len=", sigBytes.length);
console.log("owner_b64 =", item.owner);
console.log("owner_len =", base64url.toBuffer(item.owner).length);
console.log("sigtype =", item.signatureType);
console.log("binary_len =", raw.length);
console.log("binary_hex =", raw.toString("hex"));
console.log("deephash_hex =", Buffer.from(sigData).toString("hex"));
console.log("data_b64 =", base64url.encode(Buffer.from(data)));
console.log("isValid =", await item.isValid());
+39
View File
@@ -0,0 +1,39 @@
ANS-104 golden vectors, captured from arbundles 0.10.x.
To regenerate: run `npm ci` in scripts/, then `node ans104_golden_ed25519.mjs`.
Ed25519 vector (the interop pin used by
`ans104::tests::dataitem_matches_arbundles_golden_vector`):
captured by = scripts/ans104_golden_ed25519.mjs (deterministic seed
0x01 * 32 via SolanaSigner, sigtype 2, `await item.sign`)
data = "hello gitlawb ed25519 golden"
tags = [{name:"App-Name",value:"gitlawb"},
{name:"Schema",value:"gitlawb/ref-update/v1"}]
target = absent
anchor = absent
Outputs:
signature_type = 2 (Ed25519)
signature_len = 64 bytes
owner_len = 32 bytes
binary_len = 192 bytes
id = SGrcBs-ITTyzvd7eIB5kk2GdBWoxB_iTi9iJ6KOe_RE (base64url of sha256(signature))
deephash_hex = f15c82431767f14ac9e66ab8e995a8cd08e094be3773245163b53c12feb50aefc55d9f8c1098fabcfbf11a462706d347
(arbundles `getSignatureData` 8-element fold)
binary_hex:
0200da41825fd44ca3b2705af18fce86ed6d04d0204331965d9af5d5cb1a740fcc6587ee81501b1d7928c54c0f174fde8893560d785db4d988a3161113b10a2028038a88e3dd7409f195fd52db2d3cba5d72ca6709bf1d94121bf3748801b40f6f5c00000200000000000000300000000000000004104170702d4e616d650e6769746c6177620c536368656d612a6769746c6177622f7265662d7570646174652f76310068656c6c6f206769746c617762206564323535313920676f6c64656e
The Rust side will:
1. Parse binary via DataItem::from_binary (signature_type, signature,
owner, target, anchor, tags, data).
2. Compute deep_hash via DataItem::deep_hash (the 8-element arbundles
fold) and compare to deephash_hex above.
3. Verify the Ed25519 signature via DataItem::verify_data_item WITHOUT
re-signing.
4. Assert sha256(signature_bytes) base64url-encodes to the captured id.
5. Assert to_binary round-trips byte-exact (signature slot preserved).
Legacy Ethereum script (scripts/ans104_golden.mjs) now signs before
capture as well; its output is illustrative only — the node verifies
Ed25519 (sigtype 2) on the verify path.
+1624
View File
File diff suppressed because it is too large. Load diff
+12
View File
@@ -0,0 +1,12 @@
{
"name": "ans104-golden",
"version": "0.0.0",
"private": true,
"type": "module",
"dependencies": {
"@noble/ed25519": "^1.7.5",
"arbundles": "^0.10.0",
"base64url": "^3.0.1",
"bs58": "^4.0.1"
}
}