Repositions and packages the one validated, unique win so people can see it and
reuse it (per the positioning review).
- README: new lede leads with "tell what actually changed, not what just moved" +
the live `reagraph demo` contrast (59 of 56 nodes "changed" vs 1 real change) +
the zero-setup `uv tool install reagraph && reagraph demo` + a GIF placeholder
(record with examples/demo.tape). The feature list stays, below the pitch.
- .github/actions/reagraph-diff/ (composite action): diffs two build graphs with
the byte-agnostic diff, writes a Markdown summary, and posts/updates a PR comment
(stable marker, no dupes); outputs the real changed-count. Inputs base/head/by/
comment/reagraph-version.
- .github/workflows/release-diff.yml: example consumer (copy into your repo) that
diffs two builds on release; the in-repo default runs green against the committed
real-REA fixtures, with the live "pull from REA in CI" pattern shown commented.
- docs/GITHUB_ACTION.md: usage, both modes (pre-exported graphs vs live pull),
inputs/outputs, PR-comment behavior.
- README "## Diff your releases in CI" section links it: "a bot comment on every
release — the real change, not the thousands of phantom ones."
Validated: the exact `reagraph diff` the action runs exits 0 on the fixtures
(+4 nodes added / 2 rewired on the added-module build); all workflow YAML parses.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Ran the diff benchmark on two real 14k-node builds (webpack + deps) differing by
one injected exfil endpoint: raw id-diff = ~13,500 changes, signature diff ~30x
tighter with zero spurious, and the injected endpoint is found. A same-bytes
determinism control shows REA's own analysis floor (~0.2% of nodes: 14,321 vs
14,342 node counts, signature diff +26/-5) -- REA node_ids are stable for identical
bytes, but it emits a slightly different node set per run. The signature diff adds
no floor of its own; it reports REA's faithfully.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Testing on a real large app surfaced a hard limit: REA's MCP stdio response has a
size budget, and a large application's full graph overflows it -- the analysis
completes but the tool returns `resource_constraint ... exceeds the stdio response
budget` and the pull records nothing. REA's CLI emits the complete graph as JSON,
so this adds a CLI-JSON transport that bypasses MCP entirely.
- reagraph/rea_client.py: ReaClient(cli_cmd=..., cli_timeout=...). When cli_cmd is
set, findings() runs `<cli_cmd> analyze <target> --format json --full-output`,
parses the {ok,data,meta} envelope (tolerant of stray progress lines), and maps
data through the same adapter as the MCP path. Honors the cache (transport=cli
scope) and surfaces a REA error envelope without folding or caching it.
- reagraph/cli.py: `reagraph pull --rea-cli-cmd <base>` selects the CLI transport.
- tests/test_rea_client_cli.py (+5): fake-CLI coverage -- full-graph mapping,
progress-noise tolerance, error-not-folded/not-cached, cache reuse (broken CLI
still served from cache), exact command construction.
- docs/LIVE_PULL.md + README: the large-app route and the stdio-budget finding.
Verified end to end on a real 14,344-node / 14,593-edge app (webpack + deps,
~28 MB ASAR, ~200 MB analysis JSON): `reagraph pull --rea-cli-cmd` ingests it in
~108 s, `reagraph summary` renders a bounded overview (god nodes: artifact root,
webpack/types, @webassemblyjs/ast, ajv...), and the signature diff recovers an
injected change from full node-id churn at 14k-node scale in ~60 ms.
190 tests pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Verified the live integration against the REAL rea-agents MCP server (run on
Linux in Docker, where its analyzer does not need the macOS Swift/xcrun helper):
`reagraph pull` produced a real 56-node / 68-edge graph of a small Electron app.
Two adapter fixes, grounded on the real output shape:
- _find_jag: real analyze_javascript_application returns an Evidence envelope
whose JAG is nested at normalized_result.graph (alongside input_path / format /
inventory ids), not directly at normalized_result. The adapter now probes each
base AND its `.graph` child, so the real graph is found instead of falling open
to a single evidence node (the live "1 node, 0 edges" symptom).
- _node_source_file: real locations are a "source-range" with the file under
location.value.source (e.g. "src/store.js"), not `.path`. Accept either.
Back-compatible: the prior normalized_result-direct and value.path shapes still
map. New regression test asserts the nested shape maps to a real fragment (not a
fail-open node) with confidence and source_file.
Tooling + docs:
- examples/rea-docker/ (Dockerfile + README): reproducible REA-on-Linux image
(node >=24.11 + procps, which REA needs to spawn its analyzer worker) and the
exact pull steps.
- docs/LIVE_PULL.md: the Docker how-to, plus an honest caveat that REA node
identities are artifact-version scoped (ids embed artifact_sha256 and some keys
embed line ranges), so `reagraph diff` overstates change between two raw REA
builds; byte-agnostic diff holds for stable-identity graphs (graphify source,
same build), and REA's own compare_application_versions is the cross-build tool.
159 tests pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Builds the server side of reagraph.remote_cache: the shared analysis-cache tier
("the real moat") whose client already shipped. A repeat analysis of the same
artifact -- already run by someone else, on another machine -- is now served
from a shared store without re-running the tool.
reagraph/cache_server.py: a pure-stdlib HTTP service (ThreadingHTTPServer) that
is a thin layer over the SAME reagraph.cache CacheBackend the local cache uses,
speaking the exact wire contract documented on remote_cache.UrllibTransport:
GET /{key} -> bytes|404, PUT /{key} -> 201, DELETE /{key} -> 204,
GET /keys -> {"keys":[...]}, GET /health -> ok.
Safety posture: keys must be sha256 hex (^[0-9a-f]{64}$) -- this is also the
path-traversal defense (a request segment is never turned into a filesystem
path; only a validated key reaches the backend, which shards it). Public read /
token-gated write, tokens from the environment ONLY (never argv -- it leaks via
ps), constant-time compared. Upload size cap (413) and cache-entry-shape
validation (422). Every verb is wrapped: an unexpected error is a 500, never a
crashed serving thread.
make_server/serve/main public API; `python -m reagraph.cache_server` and a new
`reagraph serve-cache` CLI subcommand (host/port/dir/max-value-mb/
no-validate-payload; tokens via env). Exported from reagraph.__init__.
Tests (20 new, 158 total): unit tests over a real loopback socket (round-trip,
404 miss, bad-key 400 before any backend call, /keys, delete, /health, size-cap
413, payload 422, read/write auth, 500-stays-alive) and a conformance suite
driving the REAL RemoteCacheBackend / TieredBackend / Cache against the real
server (cross-machine round-trip, read-only push is a no-op, tiered local-warm
on a remote hit, fail-open on a dead port). Verified live via the CLI end to end.
docs/CACHE_SERVER.md (operator + client guide, trust/privacy model) and deploy/
(reference Dockerfile, .dockerignore, compose.yaml, fly.toml, README) for a
hosted instance. README/roadmap updated: the shared cache is in the tree.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
LIVE_PULL.md and README now document what the live pull actually does against
the real REA MCP server: spawn with --mcp, discover tools, call
analyze_javascript_application with input_path, and report-and-skip (no fold, no
cache) on REA's structured error body. Records REA's own runtime requirements
(Node ^22.19 || ^24.11 || >=26; macOS Swift 5.7+ for its startup helper), which
are the real gate on a non-empty graph from the live analyzer.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- remote_cache.py RemoteCacheBackend: client for a SHARED cache tier. Read-only
by default (benefit from others' public analyses, upload
nothing); opt-in push gated by allow_push (fails closed) so
private artifacts never leak; fail-open; TieredBackend
(local-first, warms local on remote hit). Hosted server is the
documented next step, not built here.
- align.py seed-propagation pass: grows matches outward from confident
anchors via mutual-unique-best neighbor overlap, recovering
structure on STRIPPED binaries at a lower confidence tier;
abstains on ties (no guessing). Existing behavior preserved.
- doctor.py + CLI 'reagraph doctor' preflights the live pull (mcp / node+npx /
resolved REA command / graphify) with fixes; pure inspection,
no spawn. Honest: ready != a real REA run verified.
- rea_client DEFAULT_TOOLS fixed to REA's real tool (analyze_javascript_
application) so a live pull actually calls something.
- docs: CACHE.md (shared client shipped, server next), ATTEST.md (propagation),
new LIVE_PULL.md (prereqs + doctor + honest not-verified-vs-real-REA).
CLI now: add/diff/stats/summary/coverage/query/pull/cache/attest/doctor/serve.
115 tests pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The flagship. Given a graphify SOURCE graph and a REAgraph SHIPPED graph of the
same program, align them structurally and surface what the source cannot explain.
- fingerprint.py Weisfeiler-Lehman color refinement + degree + name-keys +
god-nodes (isomorphic graphs share a WL-label multiset).
- align.py source<->binary node matching: name where symbols survive
(0.95), WL structure where they do not (0.70-0.94), 'both'
(<=0.99); one-to-one; ABSTAINS on ambiguity instead of guessing.
- attest.py residual = shipped-but-unexplained nodes (the money bucket) +
unexplained edges (new flows between known nodes); source_only
(tree-shaken); confidence-weighted integrity score in [0,1].
format_report never over-claims malice.
- cli: 'reagraph attest --source ... --shipped ...'. Full CLI now:
add/diff/stats/summary/coverage/query/pull/cache/attest/serve.
- docs/ATTEST.md flipped to shipped, with honest limitations.
End-to-end test: a clean build attests ~1.0 with no residual; an injected node +
new flow drops integrity and surfaces the residual. 94 tests pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
- artifact.py stable content id for a file (streaming sha256) or a directory
tree (deterministic, order-independent, symlink-safe).
- cache.py content-addressed store keyed by hash(artifact_id + tool + args +
extractor_version); LocalCacheBackend (sharded, atomic, fail-open)
under $REAGRAPH_CACHE / ~/.cache/reagraph; pluggable CacheBackend
so a shared/remote tier drops in later. stats()/clear().
- cached.py cache_analysis() wrapper: on a hit, return the fragment with NO
analysis call; on a miss, run and store. Fail-open.
- rea_client cache-aware: a (artifact, tool, args) hit skips the REA call;
misses populate the cache. Opt-in via ReaClient(cache=, refresh=).
- cli 'pull' is cache-aware by default (--no-cache / --refresh); new
'cache stats|clear'. Full CLI: add/diff/stats/summary/coverage/
query/pull/cache/serve.
- docs/CACHE.md (local ships; shared tier is roadmap, with an explicit opt-in
privacy constraint so private artifacts never leak).
Integration test proves a cache hit makes zero REA calls. 68 tests pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
REAgraph now connects to REA at runtime (MCP client, no vendored REA code):
- rea_client.py ReaClient implements bridge.ReaSource; spawns/connects to REA
via $REAGRAPH_REA_CMD (default: npx -y rea-agents@latest), lists
tools, calls analyses, yields fragments. mcp lazy-imported;
connect-vs-logic split so it tests offline; fail-open per tool.
- rea_adapter.py REA JavaScript Application Graph -> node-link fragments, grounded
against morluto/rea's real JAG schema (node_id/observations/edges/
relation/epistemic 'state'); fail-open to an evidence node on any
unknown tool or garbage shape; never raises.
- cli: 'reagraph pull <target>' connects to REA and streams findings into the
graph via bridge (crash-safe). Full CLI: add/diff/stats/summary/coverage/
query/pull/serve.
- examples/fake_rea_mcp_server.py + rea_live_demo.sh exercise the whole loop
offline (no REA/node); degrades to 'reagraph add' when mcp is absent.
- docs/REA_INTEGRATION.md; DESIGN note on optional future acceleration.
42 tests pass. Independent project; not affiliated with REA.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Build-out mapping documented REA pain points to real modules:
- serve.py MCP server (remember/recall/diff/stats); mcp lazy-imported so the
rest of REAgraph works without it. -> persistent RE memory.
- bridge.py incremental live ingest; saves after each finding (crash-safe) and
skips a bad finding instead of aborting. -> fixes 'one ZodError
discards all findings'.
- summarize.py bounded god-node + cluster map for graphs of any size (100k nodes
in ~0.16s). -> fixes 'gigabyte output is undigestible'.
- coverage.py honest observed/inferred/unverified report + exploration frontier;
'absence is not proof of absence'. -> fixes 'incomplete findings
are misleading'.
- CI (3.9-3.12) + runnable examples/demo.sh (v1->v2: 7 unchanged, 1 added exfil
edge despite every symbol relocating).
- docs: ARCHITECTURE.md (composition, not vendored code) + PAINPOINTS.md.
CLI now: add / diff / stats / summary / coverage / query / serve. 29 tests pass.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
REAgraph is the memory + structure layer for reverse engineering: it folds
one-shot findings into a durable, queryable graph (dedup + confidence upgrade on
re-observation) and diffs builds by structural identity instead of byte offset.
Working core (stdlib-only, tested without REA/Ghidra installed):
- reagraph.memory: node-link store; merge() dedups by node id and
(source,target,relation), upgrades EXTRACTED>INFERRED on re-observation.
- reagraph.diff: byte-agnostic structural diff (fixes the REA #4520 class of bug
where a moved-but-unchanged module reads as 'changed').
- reagraph.ingest: REA JAG epistemic state -> graphify confidence; drops
'unavailable'; preserves the original relation as context 'rea:<relation>'.
- reagraph CLI: add / diff / stats / query (query delegates to graphify).
Graphs are written in graphify's node-link JSON so graphify query/path/explain
works directly. Independent project; not affiliated with REA (see NOTICE).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
