Add MCP server, live bridge, bounded summary, honest coverage, CI + demo

Build-out mapping documented REA pain points to real modules:
- serve.py      MCP server (remember/recall/diff/stats); mcp lazy-imported so the
                rest of REAgraph works without it. -> persistent RE memory.
- bridge.py     incremental live ingest; saves after each finding (crash-safe) and
                skips a bad finding instead of aborting. -> fixes 'one ZodError
                discards all findings'.
- summarize.py  bounded god-node + cluster map for graphs of any size (100k nodes
                in ~0.16s). -> fixes 'gigabyte output is undigestible'.
- coverage.py   honest observed/inferred/unverified report + exploration frontier;
                'absence is not proof of absence'. -> fixes 'incomplete findings
                are misleading'.
- CI (3.9-3.12) + runnable examples/demo.sh (v1->v2: 7 unchanged, 1 added exfil
                edge despite every symbol relocating).
- docs: ARCHITECTURE.md (composition, not vendored code) + PAINPOINTS.md.

CLI now: add / diff / stats / summary / coverage / query / serve. 29 tests pass.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

.github/workflows/ci.yml

@@ -0,0 +1,18 @@
+name: CI
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: pip install -e . pytest
+      - run: pytest -q

README.md

@@ -40,7 +40,19 @@ findings accumulate.
 - `reagraph diff A B` — a **structural** diff of two graphs by node/edge
   identity, immune to byte relocation. This is the fix for the #4520 class of
   bug: a module that only moved in the file reads as *unchanged*.
-- `reagraph stats` — coverage of the accumulated graph.
+- `reagraph summary` — a **bounded** map of a graph of any size (god nodes +
+  clusters), so a gigabyte-scale analysis becomes a short overview an agent can
+  actually read.
+- `reagraph coverage` — an **honest** verification report: what is observed vs
+  only inferred vs unverified, with the frontier of "probe here next" and the
+  explicit reminder that absence of a finding is not proof of absence.
+- `reagraph stats` — totals and confidence breakdown of the accumulated graph.
+- `reagraph serve` — run the REAgraph **MCP server** (`remember` / `recall` /
+  `diff` / `stats`) so any agent uses REAgraph as its reverse-engineering memory
+  (needs the `mcp` package).
+- `reagraph.bridge` — fold a live stream of findings in incrementally, saving
+  after each one (crash-safe) and skipping a bad finding instead of aborting the
+  run (so one malformed symbol never discards the whole analysis).
 
 ## Quickstart
 

docs/ARCHITECTURE.md

@@ -0,0 +1,110 @@
+# REAgraph architecture
+
+REAgraph is a thin memory and structure layer. It vendors no code from REA and no
+code from graphify. It composes with both at the protocol and format level: it
+consumes REA's output format (JAG), it writes graphify's node-link JSON, and it
+delegates query, path, and explain to graphify installed as a dependency. REAgraph
+itself does not decompile or reverse-engineer anything; it depends on a sensor
+(REA) for that and adds memory, structural identity, and diff on top.
+
+## Data flow
+
+```
+  agent (Claude Code / Codex / opencode)
+      |
+      |  1. asks REA to analyze a region
+      v
+    REA  (sensor: decompile / reverse-engineer)
+      |
+      |  2. finding as JAG (evidence + epistemic state)
+      v
+  reagraph  (memory / diff / coverage)
+      |      - ingest: JAG epistemic -> graphify confidence
+      |      - memory: merge into reagraph.json, dedup + confidence upgrade
+      |      - diff:   structural, byte-agnostic version diff
+      |
+      |  3. reagraph.json written in graphify node-link JSON
+      v
+  graphify  (query / path / explain over the accumulated graph)
+      |
+      |  4. answer
+      v
+    agent
+```
+
+The agent probes with REA, folds each finding into REAgraph, and queries the
+accumulated graph before paying for another probe. Cost stops evaporating and
+starts compounding, because the graph is a plain file on disk.
+
+## Why composition, not copying
+
+- Licensing. REA is MIT, graphify is Apache-2.0, and REAgraph is MIT. Keeping the
+  three as separate projects that talk over formats and protocols avoids mixing
+  code under different licenses and keeps REAgraph's own license clean. REAgraph is
+  an independent project and is not affiliated with or endorsed by REA.
+- Maintenance. By consuming REA's output and delegating to graphify rather than
+  forking either, REAgraph rides both upstreams. Improvements to REA's analysis or
+  graphify's query engine arrive for free; REAgraph only maintains the thin layer
+  in between.
+- Protocol-level composition. Both REA and graphify are MCP servers, so REAgraph
+  composes at the protocol level rather than by embedding either codebase. The
+  live-ingest and server pieces (see roadmap) sit in that MCP loop.
+
+## Data model
+
+The graph is graphify's node-link JSON:
+
+```json
+{"directed": true, "multigraph": false,
+ "nodes": [{"id": "...", "label": "...", "confidence": "EXTRACTED"}],
+ "links": [{"source": "...", "target": "...", "relation": "calls",
+            "confidence": "INFERRED", "context": "rea:invokes"}]}
+```
+
+- Confidence. graphify's vocabulary is EXTRACTED (observed) above INFERRED
+  (inferred). REA's JAG epistemic states map in: `observed` becomes EXTRACTED,
+  `inferred` becomes INFERRED, and `unavailable` edges are dropped because there is
+  nothing to assert. The original relation is preserved in `context` as
+  `rea:<relation>` so provenance is never lost.
+- Identity. A node's identity is its `id`. An edge's identity is the tuple
+  `(source, target, relation)`. Identity is never a byte offset or file location.
+  This is the rule that makes a memory out of a stream of findings and the rule
+  that makes the diff byte-agnostic: a thing that merely moved in the file is still
+  the same node and the same edge.
+- Memory rule. When the same node or edge is seen again, it is never duplicated.
+  REAgraph keeps the union of attributes and upgrades confidence to the stronger of
+  the two, so a later observed fact promotes an earlier inferred one. Re-observing a
+  thing makes the graph more certain, not larger.
+
+## Module map
+
+Existing modules (present and tested in this repo):
+
+- `memory` — the node-link graph store. `merge()` dedups nodes by id and edges by
+  `(source, target, relation)` and upgrades confidence on re-observation. `stats()`
+  reports node, edge, and confidence counts.
+- `diff` — structural, byte-agnostic diff over the same identities. A moved but
+  otherwise unchanged symbol reads as unchanged; only added, removed, or rewired
+  structure shows up.
+- `ingest` — maps a REA JAG document to a node-link fragment, auto-detecting JAG
+  versus plain node-link input. Pure and tolerant of missing fields.
+- `summarize` — a bounded god-node + cluster map of a graph of any size, so a
+  gigabyte-scale analysis stays digestible instead of being dumped in full.
+- `coverage` — an honest verification report (observed vs inferred vs unverified)
+  plus the exploration frontier; absence is never reported as proof of absence.
+- `bridge` — incremental live ingest: merge each finding as it is produced, save
+  after every one (crash-safe), and skip a bad finding instead of aborting.
+- `serve` — the REAgraph MCP server exposing `remember` / `recall` / `diff` /
+  `stats`; the `mcp` package is lazy-imported so the rest of REAgraph works
+  without it.
+- `cli` — the `reagraph` command: `add`, `diff`, `stats`, `summary`, `coverage`,
+  `query` (delegates to the installed graphify binary), and `serve`.
+
+Planned (described in the roadmap, not yet in this repo): demand-scoped
+re-analysis (only re-reverse the region a query needs; roadmap item 3).
+
+Two further roadmap items are not modules but cross-cutting work: a shared analysis
+cache keyed by artifact hash (roadmap item 4), and structural alignment of a
+graphify source graph to a REA binary graph of the same program (roadmap item 5),
+which is a known-hard research problem and is carried with confidence so the tool
+ranks and abstains rather than asserting.

docs/DESIGN.md

@@ -22,7 +22,16 @@ graph that makes a sensor's output *compound*.
 - `reagraph.ingest` — REA JAG (`epistemic`: observed/inferred/unavailable) ->
   graphify confidence; drops `unavailable`; preserves the original relation in
   `context` as `rea:<relation>`.
-- `reagraph` CLI — `add` / `diff` / `stats` / `query` (query delegates to graphify).
+- `reagraph.summarize` — bounded god-node + cluster overview for graphs of any
+  size (the fix for gigabyte-scale, undigestible output).
+- `reagraph.coverage` — honest observed/inferred/unverified report plus the
+  exploration frontier ("probe here next"); absence is never reported as proof.
+- `reagraph.bridge` — incremental live ingest: merge each finding as it lands,
+  save after every one (crash-safe), skip a bad finding instead of aborting.
+- `reagraph.serve` — MCP server (`remember` / `recall` / `diff` / `stats`); the
+  `mcp` package is lazy-imported so the rest of REAgraph works without it.
+- `reagraph` CLI — `add` / `diff` / `stats` / `summary` / `coverage` / `query`
+  (query delegates to graphify) / `serve`.
 
 The graph is written in graphify's node-link JSON, so `graphify query/path/explain
 --graph reagraph.json` works out of the box.

docs/PAINPOINTS.md

@@ -0,0 +1,32 @@
+# Reverse-engineering pain points and how REAgraph relates to them
+
+This table maps documented reverse-engineering pain points to what REAgraph does
+about each. The pain points are reported, not independently verified here; where a
+specific REA issue is cited it is linked. For each row the "what REAgraph does"
+column is tied to a real module or a stated roadmap item, and the verdict is honest
+about whether REAgraph fixes, mitigates, or does not address the problem. REAgraph
+is an independent project and is not affiliated with or endorsed by REA.
+
+| # | Reported pain point | What REAgraph does about it | Module / roadmap | Verdict |
+|---|---------------------|-----------------------------|------------------|---------|
+| 1 | Weak reliability on large software; partial results and timeouts | Does not make the sensor faster. Each finding that does come back is persisted, so repeated runs compound instead of restarting from zero; the planned shared cache keyed by artifact hash means work done once is not re-paid. | `memory` (today); shared-cache (roadmap item 4) | Mitigates |
+| 2 | Incomplete findings are misleading; missing a finding does not prove absence | The planned coverage module tracks what the graph actually covers versus what is only inferred or unseen, so gaps are explicit rather than silently read as absence. Confidence (EXTRACTED vs INFERRED) is carried on every node and edge today so unverified parts are visible. | `coverage` (roadmap item 3); confidence in `memory`/`ingest` (today) | Mitigates |
+| 3 | Setup depends on Ghidra / Hopper / IDA and varies by OS | REA's own toolchain setup is upstream and out of scope for REAgraph. A hosted or cached path (the shared cache and the lazy re-analysis direction) can mitigate it by serving prior results without a local sensor install for already-analyzed artifacts. | out of scope (upstream); shared-cache / lazy-RE (roadmap) | Mitigates (partly); otherwise upstream |
+| 4 | Gigabyte-scale output is hard to digest | The planned summarize module produces a bounded, navigable map of the accumulated graph instead of a full dump; graphify query/path/explain already let an agent ask targeted questions rather than reading everything. | `summarize` (planned); graphify query (today) | Mitigates |
+| 5 | Privacy and legal: results go to the model provider and need authorization | Out of scope. REAgraph is a local memory and structure layer and does not change who runs the sensor or where its data flows. Keeping the graph as a local file does avoid adding a new remote sink, but authorization and provider data handling remain the user's and the sensor's responsibility. | out of scope (upstream) | Out of scope |
+| 6 | Findings evaporate per session; cost does not compound | Core purpose. `reagraph add` folds each finding into a durable `reagraph.json` with dedup and confidence upgrade, so findings persist across sessions and accumulate. The agent can query memory before paying for another probe. | `memory` (today) | Fixes (for persistence/compounding) |
+| 7 | Byte-offset version diff reports everything changed when code moved ([REA #4520](https://github.com/morluto/rea/issues/4520)) | `reagraph diff` identifies nodes by id and edges by `(source, target, relation)`, never by byte offset, so a module that only moved reads as unchanged and only real added / removed / rewired structure shows up. | `diff` (today) | Fixes (this diff class) |
+| 8 | Provider-selection friction blocks tasks that should not need deep analysis ([REA #4555](https://github.com/morluto/rea/issues/4555)) | Provider selection is REA's own concern and is upstream. REAgraph can reduce how often deep analysis is needed at all, by answering from accumulated memory first, but it does not remove the selection step itself. | out of scope (upstream); `memory` reduces demand | Out of scope (demand reduced) |
+| 9 | Crash on bad metadata discards partial findings (REA Swift ZodError [#4542](https://github.com/morluto/rea/issues/4542)) | The planned live-ingest bridge is designed to fail open: a bad or partial finding is merged for what it does carry rather than taking down the run, so partial work is kept instead of discarded. The ingest layer today is already tolerant of missing fields and never raises on them. | `bridge` fail-open (roadmap item 1); tolerant `ingest` (today) | Mitigates |
+
+## Notes
+
+- Pain points 5, and the setup and provider-selection parts of 3 and 8, are
+  upstream of REAgraph or outside its scope. They are listed for honesty, not
+  because REAgraph claims to solve them.
+- "Fixes" is used only for problems REAgraph directly removes within its layer:
+  persistence and compounding (6) and the byte-offset diff class (7). Everything
+  else is "mitigates" or "out of scope".
+- Rows that point to planned modules (coverage, summarize, bridge) describe roadmap
+  work, not shipped behavior. See [ARCHITECTURE.md](ARCHITECTURE.md) and
+  [DESIGN.md](DESIGN.md) for what exists today versus what is planned.

examples/README.md

@@ -0,0 +1,29 @@
+# REAgraph examples
+
+A self-contained demo of the two things REAgraph does today: **persistent
+memory** and a **byte-agnostic structural diff**.
+
+## Run it
+
+```bash
+bash examples/demo.sh
+```
+
+The script uses `./.venv/bin/reagraph` if it exists, otherwise falls back to
+`python -m reagraph.cli`. It builds a fresh graph at `/tmp/reagraph_demo.json`
+(wiped at the start, so it is safely re-runnable).
+
+## What it shows
+
+- **`add` → persistent memory.** A reverse-engineering finding
+  (`sample_findings/login.jag.json`, a small REA JAG doc of a login/auth path)
+  is folded into a durable graph. The finding stops living in a chat session and
+  starts compounding.
+- **`stats` → coverage.** Node/edge counts and the observed (`EXTRACTED`) vs
+  inferred (`INFERRED`) confidence split of the accumulated graph.
+- **`diff` → byte-agnostic version diff.** `login_v2.jag.json` is the *same*
+  program with every symbol relocated (addresses and file layout all shifted),
+  plus **one** added exfil edge (`session_create → analytics.evil.example`). A
+  byte-offset diff would report "everything changed". REAgraph diffs by
+  node/edge identity, so it reports the truth: mostly unchanged, one node added,
+  one edge added, `session_create` rewired — the real signal isn't buried.

examples/demo.sh

@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+#
+# REAgraph demo: persistent memory + byte-agnostic structural diff.
+#
+# Runs three steps against the sample findings in ./sample_findings:
+#   1. add   — fold a REA finding into a fresh persistent graph (/tmp)
+#   2. stats — show what the accumulated memory now knows
+#   3. diff  — structurally compare v1 vs v2 of the same program
+#
+# Re-runnable: it wipes its own /tmp graph at the start.
+
+set -euo pipefail
+
+# Resolve paths relative to this script, so it runs from anywhere.
+HERE="$(cd "$(dirname "$0")" && pwd)"
+REPO="$(cd "$HERE/.." && pwd)"
+FINDINGS="$HERE/sample_findings"
+GRAPH="/tmp/reagraph_demo.json"
+
+# Prefer the project venv's reagraph; fall back to the module.
+if [ -x "$REPO/.venv/bin/reagraph" ]; then
+  REAGRAPH="$REPO/.venv/bin/reagraph"
+else
+  REAGRAPH="python -m reagraph.cli"
+fi
+
+echo "== using: $REAGRAPH"
+echo "== graph: $GRAPH"
+echo
+
+# Start from a clean slate so the demo is deterministic and re-runnable.
+rm -f "$GRAPH"
+
+echo "---------------------------------------------------------------"
+echo "STEP 1  reagraph add  ->  build persistent memory from a finding"
+echo "  A reverse-engineering finding (login.jag.json) is folded into a"
+echo "  durable graph. Findings stop evaporating; memory compounds."
+echo "---------------------------------------------------------------"
+$REAGRAPH add "$FINDINGS/login.jag.json" --graph "$GRAPH"
+echo
+
+echo "---------------------------------------------------------------"
+echo "STEP 2  reagraph stats  ->  what the accumulated graph knows"
+echo "  Coverage of the persisted graph: node/edge counts and the"
+echo "  observed(EXTRACTED) vs inferred(INFERRED) confidence split."
+echo "---------------------------------------------------------------"
+$REAGRAPH stats --graph "$GRAPH"
+echo
+
+echo "---------------------------------------------------------------"
+echo "STEP 3  reagraph diff v1 v2  ->  structural, byte-agnostic diff"
+echo "  v2 is the SAME program with every symbol relocated, plus ONE"
+echo "  added exfil edge (session_create -> analytics.evil.example)."
+echo "  A byte-offset diff would cry 'everything changed'. REAgraph"
+echo "  reads it correctly: mostly unchanged, one node added, one edge"
+echo "  added (session_create rewired)."
+echo "---------------------------------------------------------------"
+$REAGRAPH diff "$FINDINGS/login.jag.json" "$FINDINGS/login_v2.jag.json"
+echo
+
+echo "== done. persistent graph left at: $GRAPH"

reagraph/__init__.py

@@ -5,7 +5,7 @@ The two pieces that work today:
   - `diff`:   structural, byte-agnostic diff of two graphs.
 """
 
-from . import memory, diff, ingest
+from . import memory, diff, ingest, summarize, coverage, bridge, serve
 
-__all__ = ["memory", "diff", "ingest"]
+__all__ = ["memory", "diff", "ingest", "summarize", "coverage", "bridge", "serve"]
 __version__ = "0.0.1"

reagraph/bridge.py

@@ -0,0 +1,130 @@
+"""Live-ingest bridge: fold REA findings into the persistent graph AS they arrive.
+
+This is the "help the agent *during* RE" loop. Instead of running a full
+reverse-engineering pass and only *then* building a graph, a source streams
+findings one at a time and each one is merged into the on-disk graph
+immediately — and the graph is SAVED after every merge. If the run crashes on
+finding N, findings 1..N-1 are already durably on disk. That crash-safety is the
+core selling point over REA, whose Swift-symbol ZodError on one bad node discards
+*all* partial findings.
+
+The source is kept abstract (`ReaSource`) so we are not coupled to REA today: a
+real REA-MCP client just needs to yield JAG / node-link dicts from `findings()`.
+
+Fail-open is the other half of the pitch: a single malformed finding is logged to
+stderr and skipped, never aborting the stream.
+"""
+
+from __future__ import annotations
+
+import sys
+from typing import Callable, Iterable, Protocol, runtime_checkable
+
+from . import ingest, memory
+
+
+@runtime_checkable
+class ReaSource(Protocol):
+    """Anything that can yield reverse-engineering findings as dicts.
+
+    A finding is a JAG doc (nodes/edges with `epistemic`) or an already
+    node-link doc (nodes + links/edges). A real REA-MCP client implements this
+    later; keeping it a Protocol means we never import or depend on REA here.
+    """
+
+    def findings(self) -> Iterable[dict]:
+        ...
+
+
+def _normalize(doc: dict) -> dict:
+    """ingest.load_any, but for a dict already in hand (no file on disk).
+
+    Mirrors ingest.load_any's auto-detection: node-link passes through, JAG docs
+    are mapped via ingest_jag, plain {nodes, edges} is rewritten to node-link.
+    Raises on anything that is not a usable finding so the caller can skip it.
+    """
+    if not isinstance(doc, dict):
+        raise ValueError(f"finding is not a dict: {type(doc).__name__}")
+    if "links" in doc:
+        doc.setdefault("nodes", [])
+        return doc
+    looks_like_jag = any(
+        "epistemic" in e for e in doc.get("edges", []) if isinstance(e, dict)
+    ) or any("epistemic" in n for n in doc.get("nodes", []) if isinstance(n, dict))
+    if looks_like_jag:
+        return ingest.ingest_jag(doc)
+    if "nodes" in doc or "edges" in doc:
+        return {
+            "directed": True,
+            "multigraph": False,
+            "nodes": doc.get("nodes", []),
+            "links": doc.get("edges", []),
+        }
+    raise ValueError("finding has no nodes/links/edges")
+
+
+def _is_empty(fragment: dict) -> bool:
+    return not fragment.get("nodes") and not fragment.get("links")
+
+
+def ingest_stream(
+    source: "ReaSource | Iterable[dict]",
+    graph_path="reagraph.json",
+    on_merge: Callable[[dict], None] | None = None,
+) -> dict:
+    """Stream findings from `source` into the graph at `graph_path`.
+
+    For each finding: normalize it, merge it into the on-disk graph, and SAVE —
+    so a crash never loses already-merged progress. `on_merge(stats)` is called
+    after each successful save. A malformed or empty finding is logged to stderr
+    and skipped, never aborting the stream. Returns the final stats dict.
+
+    `source` may be a ReaSource (its `findings()` is iterated) or any iterable of
+    dicts directly.
+    """
+    items = source.findings() if isinstance(source, ReaSource) else source
+
+    graph = memory.load(graph_path)
+    # Make sure the file exists from the start so crash-safety holds even if the
+    # very first finding is the one that crashes the process.
+    memory.save(graph, graph_path)
+
+    for i, doc in enumerate(items):
+        try:
+            fragment = _normalize(doc)
+            if _is_empty(fragment):
+                raise ValueError("finding normalized to an empty fragment")
+        except Exception as exc:  # fail-open: one bad finding must not abort the run
+            print(f"reagraph.bridge: skipped finding {i}: {exc}", file=sys.stderr)
+            continue
+        memory.merge(graph, fragment)
+        memory.save(graph, graph_path)
+        if on_merge is not None:
+            on_merge(memory.stats(graph))
+
+    return memory.stats(graph)
+
+
+def ingest_many(docs: Iterable[dict], graph_path="reagraph.json", on_merge=None) -> dict:
+    """Convenience: stream a plain list/iterable of finding dicts into the graph."""
+    return ingest_stream(docs, graph_path=graph_path, on_merge=on_merge)
+
+
+class FileReaSource:
+    """A ReaSource that reads JAG / node-link findings from files on disk.
+
+    Useful for testing and batch use: point it at a list of paths and it yields
+    one finding dict per file via ingest.load_any. A file that fails to load is
+    skipped (logged to stderr) so one bad file does not abort the batch.
+    """
+
+    def __init__(self, paths: Iterable):
+        self.paths = list(paths)
+
+    def findings(self) -> Iterable[dict]:
+        for path in self.paths:
+            try:
+                yield ingest.load_any(path)
+            except Exception as exc:
+                print(f"reagraph.bridge: skipped file {path}: {exc}", file=sys.stderr)
+                continue

reagraph/cli.py

@@ -3,18 +3,24 @@
     reagraph add <finding.jag.json> [--graph reagraph.json]
     reagraph diff <a.json> <b.json>
     reagraph stats [--graph reagraph.json]
-    reagraph query "<question>" [--graph reagraph.json]   # delegates to graphify
+    reagraph summary [--graph reagraph.json] [--top N]       # bounded map of a big graph
+    reagraph coverage [--graph reagraph.json]                # what's verified vs unverified
+    reagraph query "<question>" [--graph reagraph.json]      # delegates to graphify
+    reagraph serve [--graph reagraph.json]                   # run the MCP server
 """
 
 from __future__ import annotations
 
 import argparse
+import os
 import shutil
 import subprocess
 import sys
 
+from . import coverage as coveragemod
 from . import diff as diffmod
 from . import memory
+from . import summarize as summarizemod
 from .ingest import load_any
 
 DEFAULT_GRAPH = "reagraph.json"
@@ -48,6 +54,24 @@ def _cmd_stats(args) -> int:
     return 0
 
 
+def _cmd_summary(args) -> int:
+    graph = memory.load(args.graph)
+    print(summarizemod.format_summary(summarizemod.summarize(graph, top=args.top)))
+    return 0
+
+
+def _cmd_coverage(args) -> int:
+    print(coveragemod.gaps_report(memory.load(args.graph)))
+    return 0
+
+
+def _cmd_serve(args) -> int:
+    # The MCP server resolves its graph via REAGRAPH_GRAPH; forward --graph.
+    os.environ["REAGRAPH_GRAPH"] = args.graph
+    from . import serve  # lazy: pulls in mcp only when actually serving
+    return serve.main()
+
+
 def _cmd_query(args) -> int:
     if not shutil.which("graphify"):
         print("graphify is not installed; install it to query the graph:\n"
@@ -75,11 +99,24 @@ def build_parser() -> argparse.ArgumentParser:
     s.add_argument("--graph", default=DEFAULT_GRAPH)
     s.set_defaults(func=_cmd_stats)
 
+    sm = sub.add_parser("summary", help="bounded map (god nodes + clusters) of a big graph")
+    sm.add_argument("--graph", default=DEFAULT_GRAPH)
+    sm.add_argument("--top", type=int, default=20)
+    sm.set_defaults(func=_cmd_summary)
+
+    cv = sub.add_parser("coverage", help="what's observed vs inferred vs unverified (honest gaps)")
+    cv.add_argument("--graph", default=DEFAULT_GRAPH)
+    cv.set_defaults(func=_cmd_coverage)
+
     q = sub.add_parser("query", help="query the graph (delegates to graphify)")
     q.add_argument("question")
     q.add_argument("--graph", default=DEFAULT_GRAPH)
     q.set_defaults(func=_cmd_query)
 
+    sv = sub.add_parser("serve", help="run the REAgraph MCP server (needs the 'mcp' package)")
+    sv.add_argument("--graph", default=DEFAULT_GRAPH)
+    sv.set_defaults(func=_cmd_serve)
+
     return p
 
 

reagraph/coverage.py

@@ -0,0 +1,176 @@
+"""Honest coverage: make verification status and gaps first-class.
+
+Two documented REA pain points this answers:
+  - "incomplete findings can be misleading -- missing a finding does not prove
+    absence." A graph that is silent about a thing is not a graph that cleared it.
+  - users "not knowing what the agent actually VERIFIED." Observed facts and
+    guesses look identical once they are nodes on a screen.
+
+So here we separate, out loud, what was OBSERVED (EXTRACTED) from what was only
+INFERRED, and we surface the things we point at but never actually analyzed
+(edges whose endpoint has no node entry of its own -- "ghost"/unresolved
+targets). The `frontier` is the concrete "we have NOT verified this yet; probe
+here next" list. The `gaps_report` says all of this in plain words and refuses
+to let absence read as proof of absence.
+
+graphify's confidence vocab (see memory.py): EXTRACTED = observed,
+INFERRED = inferred/guessed. Pure stdlib, deterministic.
+"""
+
+from __future__ import annotations
+
+# The exact phrase tests and readers can rely on. Framing, not decoration:
+# a silent graph has not cleared anything.
+ABSENCE_DISCLAIMER = (
+    "Absence of a finding is NOT proof of absence: anything not listed as "
+    "observed below is simply unverified, not confirmed safe or confirmed absent."
+)
+
+
+def _node_index(graph: dict) -> dict:
+    """Map node id -> node dict, keeping the first entry for a given id."""
+    index = {}
+    for n in graph.get("nodes", []):
+        nid = n.get("id")
+        if nid is not None and nid not in index:
+            index[nid] = n
+    return index
+
+
+def _edges(graph: dict) -> list:
+    return graph.get("links", []) or graph.get("edges", []) or []
+
+
+def _conf(obj: dict) -> str:
+    return obj.get("confidence") or ""
+
+
+def _tally(objs, conf_of) -> dict:
+    tally = {"EXTRACTED": 0, "INFERRED": 0, "other": 0}
+    for o in objs:
+        c = conf_of(o)
+        tally[c if c in ("EXTRACTED", "INFERRED") else "other"] += 1
+    return tally
+
+
+def coverage(graph: dict) -> dict:
+    """Observed-vs-inferred counts for nodes AND edges, the unresolved endpoints
+    we never analyzed, and an overall verified fraction of the edges.
+
+    verified_fraction = EXTRACTED edges / total edges (0.0 when there are none).
+    `ghost_targets` are ids that appear as an edge source or target but have no
+    node entry of their own -- things we point at but never actually analyzed.
+    """
+    nodes = graph.get("nodes", [])
+    edges = _edges(graph)
+    index = _node_index(graph)
+
+    # Endpoints referenced by edges that have no node entry of their own.
+    ghosts = set()
+    for e in edges:
+        for endpoint in (e.get("source"), e.get("target")):
+            if endpoint is not None and endpoint not in index:
+                ghosts.add(endpoint)
+
+    edge_conf = _tally(edges, _conf)
+    total_edges = len(edges)
+    verified_fraction = (edge_conf["EXTRACTED"] / total_edges) if total_edges else 0.0
+
+    return {
+        "nodes": {
+            "total": len(nodes),
+            "confidence": _tally(nodes, _conf),
+        },
+        "edges": {
+            "total": total_edges,
+            "confidence": edge_conf,
+        },
+        "ghost_targets": sorted(ghosts, key=str),
+        "verified_fraction": verified_fraction,
+    }
+
+
+def frontier(graph: dict) -> list:
+    """The exploration frontier: INFERRED edges whose target is a ghost (no node
+    entry) or an INFERRED-only node. These are the concrete, not-yet-verified
+    hops -- "probe here next".
+
+    Returns [{source, target, relation, reason}], deterministic (sorted).
+    Fully-observed edges and edges into confirmed (EXTRACTED) nodes are excluded.
+    """
+    index = _node_index(graph)
+    items = []
+    for e in _edges(graph):
+        if _conf(e) != "INFERRED":
+            continue
+        target = e.get("target")
+        tnode = index.get(target)
+        if tnode is None:
+            reason = "target unresolved (ghost) -- pointed at but never analyzed"
+        elif _conf(tnode) == "INFERRED":
+            reason = "target only inferred, never observed"
+        else:
+            # Target is observed (or otherwise ranked); edge is not frontier.
+            continue
+        items.append({
+            "source": e.get("source"),
+            "target": target,
+            "relation": e.get("relation", ""),
+            "reason": reason,
+        })
+    items.sort(key=lambda d: (str(d["source"]), str(d["target"]), str(d["relation"])))
+    return items
+
+
+def gaps_report(graph: dict) -> str:
+    """An honest, plain-words report of what is observed, what is only inferred,
+    and what is unverified/unknown -- the anti-false-confidence feature."""
+    cov = coverage(graph)
+    front = frontier(graph)
+
+    nconf = cov["nodes"]["confidence"]
+    econf = cov["edges"]["confidence"]
+    pct = round(cov["verified_fraction"] * 100, 1)
+
+    lines = [
+        "REAgraph coverage / honesty report",
+        "=" * 36,
+        "",
+        ABSENCE_DISCLAIMER,
+        "",
+        "OBSERVED (verified, confidence=EXTRACTED):",
+        f"  nodes observed : {nconf['EXTRACTED']}",
+        f"  edges observed : {econf['EXTRACTED']}",
+        f"  verified fraction of edges : {pct}% "
+        f"({econf['EXTRACTED']}/{cov['edges']['total']})",
+        "",
+        "ONLY INFERRED (guessed, confidence=INFERRED, NOT verified):",
+        f"  nodes inferred : {nconf['INFERRED']}",
+        f"  edges inferred : {econf['INFERRED']}",
+        "",
+        "UNVERIFIED / UNKNOWN:",
+        f"  unresolved targets we point at but never analyzed (ghosts) : "
+        f"{len(cov['ghost_targets'])}",
+    ]
+    for gid in cov["ghost_targets"]:
+        lines.append(f"    ? {gid}  (no node entry -- never analyzed)")
+
+    lines.append("")
+    lines.append(
+        f"NOT-YET-VERIFIED frontier (probe here next) : {len(front)} item(s)"
+    )
+    for item in front:
+        lines.append(
+            f"    -> {item['source']} --{item['relation']}--> {item['target']}"
+            f"  [{item['reason']}]"
+        )
+
+    if nconf["other"] or econf["other"]:
+        lines.append("")
+        lines.append(
+            f"UNLABELLED confidence: {nconf['other']} node(s), "
+            f"{econf['other']} edge(s) carry no confidence and cannot be "
+            "treated as verified."
+        )
+
+    return "\n".join(lines)

reagraph/serve.py

@@ -0,0 +1,188 @@
+"""MCP server: expose reagraph as reverse-engineering memory to any agent.
+
+Runs as an MCP stdio server (Claude Code / opencode / Codex) exposing four
+tools over the same persistent node-link graph the CLI uses:
+
+    remember(finding) -> fold a JAG/node-link dict OR a file path into memory
+    recall(query)     -> a compact relevant slice (delegates to graphify, or
+                         falls back to top-degree nodes when graphify is absent)
+    diff(a, b)        -> structural, byte-agnostic diff report of two graphs
+    stats()           -> accumulated-graph coverage
+
+The tool bodies live as plain `_remember/_recall/_diff/_stats` functions so they
+are unit-testable without `mcp` or a live client. `mcp` is imported lazily inside
+`build_server()`, so this module imports fine when `mcp` is not installed.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import subprocess
+from pathlib import Path
+
+from . import diff as diffmod
+from . import memory
+from .ingest import load_any
+
+DEFAULT_GRAPH = "reagraph.json"
+
+
+def _graph_path(graph: str | None = None) -> str:
+    """Resolve the persistent graph path: explicit arg > env > default."""
+    return graph or os.environ.get("REAGRAPH_GRAPH") or DEFAULT_GRAPH
+
+
+def _as_fragment(finding) -> dict:
+    """Accept a node-link/JAG dict, a JSON string, or a file path -> fragment."""
+    if isinstance(finding, dict):
+        return load_any_dict(finding)
+    if isinstance(finding, str):
+        p = Path(finding)
+        if p.exists():
+            return load_any(p)
+        # Not a path: try parsing it as inline JSON.
+        try:
+            return load_any_dict(json.loads(finding))
+        except (ValueError, TypeError):
+            raise ValueError(f"finding is neither an existing file nor valid JSON: {finding!r}")
+    raise TypeError(f"unsupported finding type: {type(finding).__name__}")
+
+
+def load_any_dict(doc: dict) -> dict:
+    """In-memory twin of ingest.load_any: node-link vs JAG auto-detect on a dict."""
+    # Write-free reuse of ingest's detection by round-tripping through ingest_jag
+    # when the doc looks like a JAG, else treat as node-link.
+    from .ingest import ingest_jag
+
+    if "links" in doc:
+        out = dict(doc)
+        out.setdefault("nodes", [])
+        return out
+    looks_like_jag = any(
+        "epistemic" in e for e in doc.get("edges", []) if isinstance(e, dict)
+    ) or any("epistemic" in n for n in doc.get("nodes", []) if isinstance(n, dict))
+    if looks_like_jag:
+        return ingest_jag(doc)
+    return {
+        "directed": True,
+        "multigraph": False,
+        "nodes": doc.get("nodes", []),
+        "links": doc.get("edges", []),
+    }
+
+
+def _remember(graph_path: str, finding) -> dict:
+    """Fold a finding into the persistent graph; return the stats delta."""
+    graph = memory.load(graph_path)
+    before = memory.stats(graph)
+    memory.merge(graph, _as_fragment(finding))
+    memory.save(graph, graph_path)
+    after = memory.stats(graph)
+    return {
+        "graph": graph_path,
+        "before": before,
+        "after": after,
+        "delta": {
+            "nodes": after["nodes"] - before["nodes"],
+            "edges": after["edges"] - before["edges"],
+        },
+    }
+
+
+def _top_degree_nodes(graph: dict, limit: int = 10) -> list[dict]:
+    """Fallback relevance: nodes ranked by incident-edge count."""
+    deg: dict = {}
+    for e in graph.get("links", []) or graph.get("edges", []):
+        for endpoint in (e.get("source"), e.get("target")):
+            if endpoint is not None:
+                deg[endpoint] = deg.get(endpoint, 0) + 1
+    labels = {n.get("id"): n.get("label", n.get("id")) for n in graph.get("nodes", [])}
+    ranked = sorted(deg.items(), key=lambda kv: (-kv[1], str(kv[0])))
+    return [{"id": nid, "label": labels.get(nid, nid), "degree": d} for nid, d in ranked[:limit]]
+
+
+def _recall(graph_path: str, query: str) -> str:
+    """Return a compact relevant slice. Uses graphify if on PATH, else a
+    top-degree fallback. Never crashes if graphify is absent or errors."""
+    if shutil.which("graphify"):
+        try:
+            proc = subprocess.run(
+                ["graphify", "query", query, "--graph", graph_path],
+                capture_output=True,
+                text=True,
+            )
+            out = (proc.stdout or "").strip()
+            if proc.returncode == 0 and out:
+                return out
+            # Fall through to the local fallback on non-zero/empty output.
+        except OSError:
+            pass  # graphify vanished between the which() and the run(); fall back.
+    graph = memory.load(graph_path)
+    top = _top_degree_nodes(graph)
+    if not top:
+        return f"(graphify unavailable; empty graph at {graph_path})"
+    lines = [f"(graphify unavailable) top {len(top)} nodes by degree for {query!r}:"]
+    lines += [f"  {n['degree']:>3}  {n['id']}  ({n['label']})" for n in top]
+    return "\n".join(lines)
+
+
+def _diff(a, b) -> str:
+    """Structural, byte-agnostic diff report of two graph files/dicts."""
+    ga = a if isinstance(a, dict) else load_any(a)
+    gb = b if isinstance(b, dict) else load_any(b)
+    return diffmod.format_report(diffmod.diff(ga, gb))
+
+
+def _stats(graph_path: str) -> dict:
+    """Accumulated-graph coverage for the persistent graph."""
+    return memory.stats(memory.load(graph_path))
+
+
+def build_server():
+    """Create the MCP server with the four tools registered.
+
+    Lazily imports `mcp`; raises a clear RuntimeError if it is not installed."""
+    try:
+        from mcp.server.fastmcp import FastMCP
+    except ImportError as exc:  # pragma: no cover - exercised only without mcp
+        raise RuntimeError(
+            "the 'mcp' package is required to run the reagraph MCP server; "
+            "install it with: pip install mcp"
+        ) from exc
+
+    server = FastMCP("reagraph")
+
+    @server.tool()
+    def remember(finding, graph: str | None = None) -> dict:
+        """Fold a JAG/node-link finding (dict, JSON string, or file path) into
+        persistent memory. Returns the stats delta."""
+        return _remember(_graph_path(graph), finding)
+
+    @server.tool()
+    def recall(query: str, graph: str | None = None) -> str:
+        """Return a compact relevant slice of the graph for a query."""
+        return _recall(_graph_path(graph), query)
+
+    @server.tool()
+    def diff(a: str, b: str) -> str:
+        """Structural, byte-agnostic diff report of two graph files."""
+        return _diff(a, b)
+
+    @server.tool()
+    def stats(graph: str | None = None) -> dict:
+        """Show accumulated-graph coverage."""
+        return _stats(_graph_path(graph))
+
+    return server
+
+
+def main(argv=None) -> int:
+    """Run the MCP stdio server."""
+    build_server().run()
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

reagraph/summarize.py

@@ -0,0 +1,168 @@
+"""Turn a huge graph into a COMPACT, bounded map an agent can actually read.
+
+The pain point (REA at scale): analysis output reaches gigabyte scale, far too
+big for an agent to digest. A million-node call graph does not fit in a context
+window and re-reading it is hopeless. REAgraph's answer is to never hand the
+agent the whole graph -- hand it a *bounded* overview whose size does not grow
+with the graph:
+
+  - `god_nodes`:  the highest-degree hubs (the few functions everything touches).
+  - `components`: weakly-connected clusters, a light stand-in for community
+                  detection (graphify can run Leiden later for real communities).
+  - `summarize`:  a single small dict -- totals, confidence breakdown, the top
+                  god-nodes and largest clusters -- capped so a 1M-node graph
+                  still yields a summary a few lines long.
+
+Pure stdlib, deterministic, and linear-ish: plain dict degree counting and a
+union-find for components, never networkx.
+"""
+
+from __future__ import annotations
+
+from . import memory
+
+
+def _node_labels(graph: dict) -> dict:
+    """Map node id -> label (falling back to the id when no label is set)."""
+    labels = {}
+    for n in graph.get("nodes", []):
+        nid = n.get("id")
+        if nid is None:
+            continue
+        labels[nid] = n.get("label") or nid
+    return labels
+
+
+def _degrees(graph: dict) -> dict:
+    """Total degree (in + out) per node id, counting every incident edge."""
+    deg: dict = {}
+    for n in graph.get("nodes", []):
+        nid = n.get("id")
+        if nid is not None:
+            deg.setdefault(nid, 0)
+    for e in graph.get("links", []) or graph.get("edges", []):
+        s, t = e.get("source"), e.get("target")
+        if s is not None:
+            deg[s] = deg.get(s, 0) + 1
+        if t is not None:
+            deg[t] = deg.get(t, 0) + 1
+    return deg
+
+
+def god_nodes(graph: dict, top: int = 20) -> list:
+    """The `top` highest-degree nodes as [{id, label, degree}].
+
+    Ranked by degree (in + out) descending; ties broken deterministically by id
+    ascending. Returns at most `top` entries."""
+    deg = _degrees(graph)
+    labels = _node_labels(graph)
+    ranked = sorted(deg.items(), key=lambda kv: (-kv[1], str(kv[0])))
+    out = []
+    for nid, d in ranked[: max(0, top)]:
+        out.append({"id": nid, "label": labels.get(nid, nid), "degree": d})
+    return out
+
+
+def components(graph: dict) -> list:
+    """Weakly-connected components (edges treated as undirected for grouping).
+
+    Each component is returned as a sorted list of node ids. Components are
+    ordered by size descending, then by their minimum id -- a light stand-in for
+    community detection (graphify can run Leiden later)."""
+    parent: dict = {}
+
+    def find(x):
+        root = x
+        while parent[root] != root:
+            root = parent[root]
+        # Path compression.
+        while parent[x] != root:
+            parent[x], x = root, parent[x]
+        return root
+
+    def add(x):
+        if x not in parent:
+            parent[x] = x
+
+    def union(a, b):
+        ra, rb = find(a), find(b)
+        if ra != rb:
+            parent[ra] = rb
+
+    for n in graph.get("nodes", []):
+        nid = n.get("id")
+        if nid is not None:
+            add(nid)
+    for e in graph.get("links", []) or graph.get("edges", []):
+        s, t = e.get("source"), e.get("target")
+        if s is None or t is None:
+            continue
+        add(s)
+        add(t)
+        union(s, t)
+
+    groups: dict = {}
+    for nid in parent:
+        groups.setdefault(find(nid), []).append(nid)
+
+    clusters = [sorted(members, key=str) for members in groups.values()]
+    clusters.sort(key=lambda c: (-len(c), str(c[0])))
+    return clusters
+
+
+def summarize(graph: dict, top: int = 20, max_clusters: int = 10) -> dict:
+    """A bounded overview of the graph, regardless of how big the graph is.
+
+    Caps every list so a million-node graph still yields a small summary:
+      - totals + edge confidence breakdown (reuses `memory.stats`),
+      - the top god-nodes (<= `top`),
+      - the largest clusters (<= `max_clusters`), each with a representative
+        label (the highest-degree member)."""
+    base = memory.stats(graph)
+    deg = _degrees(graph)
+    labels = _node_labels(graph)
+
+    clusters = components(graph)
+    cluster_summaries = []
+    for members in clusters[: max(0, max_clusters)]:
+        # Representative = highest-degree member, tie-broken by id.
+        rep = min(members, key=lambda nid: (-deg.get(nid, 0), str(nid)))
+        cluster_summaries.append({
+            "size": len(members),
+            "representative": rep,
+            "representative_label": labels.get(rep, rep),
+        })
+
+    return {
+        "nodes": base["nodes"],
+        "edges": base["edges"],
+        "edge_confidence": base["edge_confidence"],
+        "num_components": len(clusters),
+        "god_nodes": god_nodes(graph, top=top),
+        "largest_clusters": cluster_summaries,
+    }
+
+
+def format_summary(summary: dict) -> str:
+    """A short, human/agent-readable text block for a summary dict."""
+    conf = summary.get("edge_confidence", {})
+    lines = [
+        "Graph overview (bounded):",
+        f"  nodes      : {summary.get('nodes', 0)}",
+        f"  edges      : {summary.get('edges', 0)}"
+        f"  (EXTRACTED {conf.get('EXTRACTED', 0)} /"
+        f" INFERRED {conf.get('INFERRED', 0)} /"
+        f" other {conf.get('other', 0)})",
+        f"  components : {summary.get('num_components', 0)}",
+        "  god nodes:",
+    ]
+    for g in summary.get("god_nodes", []):
+        lines.append(f"    {g['label']}  (degree {g['degree']})")
+    lines.append("  largest clusters:")
+    for c in summary.get("largest_clusters", []):
+        lines.append(
+            f"    {c['representative_label']}  (+{c['size'] - 1} more)"
+            if c["size"] > 1
+            else f"    {c['representative_label']}  (singleton)"
+        )
+    return "\n".join(lines)

tests/test_bridge.py

@@ -0,0 +1,109 @@
+"""The live-ingest bridge guarantees, tested with a fake in-memory source.
+
+No REA/Ghidra needed: a fake source yields three findings (one malformed) and we
+assert the two selling points vs REA — crash-safety (graph saved after every
+merge) and fail-open (one bad finding is skipped, not fatal) — plus confidence
+upgrade across a stream (observed later promotes an earlier inferred fact).
+"""
+
+from pathlib import Path
+
+from reagraph import bridge, memory
+
+
+class FakeReaSource:
+    """Yields JAG findings in order; the middle one is malformed/empty."""
+
+    def findings(self):
+        # 1. INFERRED: we merely guess check() calls net().
+        yield {
+            "nodes": [{"id": "check", "epistemic": "inferred"},
+                      {"id": "net", "epistemic": "inferred"}],
+            "edges": [{"source": "check", "target": "net",
+                       "relation": "calls", "epistemic": "inferred"}],
+        }
+        # 2. Malformed: no nodes/links/edges. Must be skipped, not fatal.
+        yield {"garbage": "no graph here"}
+        # 3. OBSERVED: we actually see the same node and edge.
+        yield {
+            "nodes": [{"id": "check", "epistemic": "observed"},
+                      {"id": "net", "epistemic": "observed"}],
+            "edges": [{"source": "check", "target": "net",
+                       "relation": "calls", "epistemic": "observed"}],
+        }
+
+
+def test_stream_merges_good_skips_bad_and_is_crash_safe(tmp_path):
+    graph_path = tmp_path / "reagraph.json"
+
+    # Crash-safety probe: after every merge the on-disk file must exist and be
+    # loadable. We also record how many times on_merge fired (once per GOOD
+    # finding — the bad one never reaches a merge).
+    merges = []
+
+    def on_merge(stats):
+        assert graph_path.exists(), "graph must be saved after each merge"
+        memory.load(graph_path)  # must be valid JSON on disk, not half-written
+        merges.append(stats)
+
+    final = bridge.ingest_stream(FakeReaSource(), graph_path=str(graph_path), on_merge=on_merge)
+
+    # The bad finding was skipped (not fatal): only the 2 good findings merged.
+    assert len(merges) == 2
+    # Both good findings describe the same node/edge -> deduped, not duplicated.
+    assert final["nodes"] == 2
+    assert final["edges"] == 1
+    # Confidence UPGRADED across the stream: observed (finding 3) promoted the
+    # inferred fact from finding 1.
+    assert final["edge_confidence"]["EXTRACTED"] == 1
+    assert final["edge_confidence"]["INFERRED"] == 0
+
+    # The upgrade order is observable in the saved graph too.
+    g = memory.load(graph_path)
+    check = next(n for n in g["nodes"] if n["id"] == "check")
+    assert check["confidence"] == "EXTRACTED"
+
+
+def test_confidence_only_upgrades_never_downgrades(tmp_path):
+    graph_path = tmp_path / "g.json"
+    # Observed first, then a later inferred re-observation must NOT downgrade it.
+    docs = [
+        {"nodes": [{"id": "a", "epistemic": "observed"}, {"id": "b", "epistemic": "observed"}],
+         "edges": [{"source": "a", "target": "b", "relation": "calls", "epistemic": "observed"}]},
+        {"nodes": [{"id": "a", "epistemic": "inferred"}, {"id": "b", "epistemic": "inferred"}],
+         "edges": [{"source": "a", "target": "b", "relation": "calls", "epistemic": "inferred"}]},
+    ]
+    final = bridge.ingest_many(docs, graph_path=str(graph_path))
+    assert final["edges"] == 1
+    assert final["edge_confidence"]["EXTRACTED"] == 1
+
+
+def test_file_source_reads_jag_from_disk(tmp_path, capsys):
+    import json
+
+    good = tmp_path / "good.json"
+    good.write_text(json.dumps({
+        "nodes": [{"id": "x", "epistemic": "observed"}, {"id": "y", "epistemic": "observed"}],
+        "edges": [{"source": "x", "target": "y", "relation": "calls", "epistemic": "observed"}],
+    }), encoding="utf-8")
+    bad = tmp_path / "bad.json"
+    bad.write_text("{ not valid json", encoding="utf-8")
+
+    graph_path = tmp_path / "out.json"
+    # The unreadable file is skipped, the good one is ingested.
+    final = bridge.ingest_stream(
+        bridge.FileReaSource([str(good), str(bad)]),
+        graph_path=str(graph_path),
+    )
+    assert graph_path.exists()
+    # x and y nodes from the good file; the bad file contributed nothing.
+    assert final["nodes"] == 2
+    assert "skipped file" in capsys.readouterr().err
+
+
+def test_empty_stream_still_creates_graph_file(tmp_path):
+    graph_path = tmp_path / "empty.json"
+    final = bridge.ingest_stream([], graph_path=str(graph_path))
+    assert graph_path.exists()  # crash-safe from the very first step
+    assert final == {"nodes": 0, "edges": 0,
+                     "edge_confidence": {"EXTRACTED": 0, "INFERRED": 0, "other": 0}}

tests/test_coverage.py

@@ -0,0 +1,102 @@
+"""Honesty guarantees: observed vs inferred vs unverified, tested without REA."""
+
+from reagraph import coverage
+
+
+def _sample_graph():
+    """A small graph mixing EXTRACTED and INFERRED, with one edge pointing at a
+    target that has no node entry of its own ("ghost")."""
+    return {
+        "directed": True,
+        "multigraph": False,
+        "nodes": [
+            {"id": "main", "label": "main()", "confidence": "EXTRACTED"},
+            {"id": "auth", "label": "auth()", "confidence": "EXTRACTED"},
+            {"id": "maybe_crypto", "label": "crypto?", "confidence": "INFERRED"},
+        ],
+        "links": [
+            # Fully observed: both ends are nodes, edge is EXTRACTED.
+            {"source": "main", "target": "auth", "relation": "calls",
+             "confidence": "EXTRACTED"},
+            # Inferred edge into an INFERRED-only node -> frontier.
+            {"source": "auth", "target": "maybe_crypto", "relation": "calls",
+             "confidence": "INFERRED"},
+            # Inferred edge into a ghost target (no node entry) -> frontier.
+            {"source": "main", "target": "ghost_fn", "relation": "calls",
+             "confidence": "INFERRED"},
+        ],
+    }
+
+
+def test_coverage_counts_and_verified_fraction():
+    cov = coverage.coverage(_sample_graph())
+
+    assert cov["nodes"]["total"] == 3
+    assert cov["nodes"]["confidence"]["EXTRACTED"] == 2
+    assert cov["nodes"]["confidence"]["INFERRED"] == 1
+
+    assert cov["edges"]["total"] == 3
+    assert cov["edges"]["confidence"]["EXTRACTED"] == 1
+    assert cov["edges"]["confidence"]["INFERRED"] == 2
+
+    # Only "ghost_fn" is referenced by an edge but has no node entry.
+    assert cov["ghost_targets"] == ["ghost_fn"]
+
+    # 1 of 3 edges is observed.
+    assert cov["verified_fraction"] == 1 / 3
+
+
+def test_frontier_is_exactly_the_unverified_hops():
+    front = coverage.frontier(_sample_graph())
+
+    pairs = {(f["source"], f["target"], f["relation"]) for f in front}
+    # The two inferred, not-yet-verified hops...
+    assert ("auth", "maybe_crypto", "calls") in pairs
+    assert ("main", "ghost_fn", "calls") in pairs
+    # ...and NOT the fully-observed edge.
+    assert ("main", "auth", "calls") not in pairs
+    assert len(front) == 2
+
+    # Deterministic ordering.
+    assert front == coverage.frontier(_sample_graph())
+    assert [f["source"] for f in front] == ["auth", "main"]
+
+    # Every frontier item carries a human reason.
+    assert all(f["reason"] for f in front)
+
+
+def test_frontier_excludes_inferred_edge_into_observed_node():
+    # An INFERRED edge whose target IS observed is not a verification gap.
+    g = {
+        "nodes": [
+            {"id": "a", "confidence": "EXTRACTED"},
+            {"id": "b", "confidence": "EXTRACTED"},
+        ],
+        "links": [
+            {"source": "a", "target": "b", "relation": "calls",
+             "confidence": "INFERRED"},
+        ],
+    }
+    assert coverage.frontier(g) == []
+
+
+def test_gaps_report_is_honest_about_absence():
+    report = coverage.gaps_report(_sample_graph())
+
+    # The anti-false-confidence framing is present (stable substring).
+    assert "Absence of a finding is NOT proof of absence" in report
+
+    # It names the unverified items plainly.
+    assert "ghost_fn" in report
+    assert "maybe_crypto" in report
+    # It distinguishes observed from inferred.
+    assert "OBSERVED" in report
+    assert "ONLY INFERRED" in report
+    assert "UNVERIFIED / UNKNOWN" in report
+
+
+def test_empty_graph_has_zero_verified_fraction():
+    cov = coverage.coverage({"nodes": [], "links": []})
+    assert cov["verified_fraction"] == 0.0
+    assert cov["ghost_targets"] == []
+    assert coverage.frontier({"nodes": [], "links": []}) == []

tests/test_serve.py

@@ -0,0 +1,116 @@
+"""MCP-server tool logic, tested without `mcp` or a live client.
+
+These call the plain `_remember/_recall/_diff/_stats` functions directly, so the
+graph memory wired behind the MCP tools is verified without importing `mcp`.
+"""
+
+import json
+
+from reagraph import serve
+
+# A first-pass JAG finding: one inferred call edge.
+JAG_PASS1 = {
+    "nodes": [{"id": "check", "label": "check()", "epistemic": "inferred"}],
+    "edges": [{"source": "check", "target": "net", "relation": "calls", "epistemic": "inferred"}],
+}
+# A second pass that OBSERVED the same edge plus a new node.
+JAG_PASS2 = {
+    "nodes": [
+        {"id": "check", "label": "check()", "epistemic": "observed"},
+        {"id": "net", "label": "net()", "epistemic": "observed"},
+    ],
+    "edges": [{"source": "check", "target": "net", "relation": "calls", "epistemic": "observed"}],
+}
+
+
+def test_module_imports_without_mcp():
+    # The module must import even though `mcp` is not installed here.
+    assert hasattr(serve, "build_server")
+    assert hasattr(serve, "main")
+
+
+def test_remember_then_stats_dict_fragment(tmp_path):
+    gp = str(tmp_path / "reagraph.json")
+    d1 = serve._remember(gp, JAG_PASS1)
+    assert d1["delta"] == {"nodes": 1, "edges": 1}
+
+    d2 = serve._remember(gp, JAG_PASS2)
+    # Re-observing the same edge adds only the new `net` node, no dup edge.
+    assert d2["delta"] == {"nodes": 1, "edges": 0}
+
+    s = serve._stats(gp)
+    assert s["nodes"] == 2 and s["edges"] == 1
+    # Confidence was upgraded on re-observation, not duplicated.
+    assert s["edge_confidence"]["EXTRACTED"] == 1
+    assert s["edge_confidence"]["INFERRED"] == 0
+
+
+def test_remember_accepts_file_path(tmp_path):
+    finding = tmp_path / "finding.json"
+    finding.write_text(json.dumps(JAG_PASS2), encoding="utf-8")
+    gp = str(tmp_path / "reagraph.json")
+    d = serve._remember(gp, str(finding))
+    assert d["delta"]["nodes"] == 2 and d["delta"]["edges"] == 1
+
+
+def test_remember_accepts_json_string(tmp_path):
+    gp = str(tmp_path / "reagraph.json")
+    d = serve._remember(gp, json.dumps(JAG_PASS1))
+    assert d["delta"] == {"nodes": 1, "edges": 1}
+
+
+def test_remember_accepts_node_link_dict(tmp_path):
+    gp = str(tmp_path / "reagraph.json")
+    frag = {
+        "nodes": [{"id": "a", "confidence": "EXTRACTED"}, {"id": "b", "confidence": "EXTRACTED"}],
+        "links": [{"source": "a", "target": "b", "relation": "calls", "confidence": "EXTRACTED"}],
+    }
+    d = serve._remember(gp, frag)
+    assert d["delta"] == {"nodes": 2, "edges": 1}
+
+
+def test_diff_is_byte_agnostic(tmp_path):
+    a = tmp_path / "a.json"
+    b = tmp_path / "b.json"
+    g = {
+        "nodes": [{"id": "x"}, {"id": "y"}],
+        "links": [{"source": "x", "target": "y", "relation": "calls"}],
+    }
+    a.write_text(json.dumps(g), encoding="utf-8")
+    b.write_text(json.dumps(g), encoding="utf-8")
+    report = serve._diff(str(a), str(b))
+    assert "Structural diff (byte-agnostic):" in report
+    assert "unchanged nodes : 2" in report
+
+
+def test_diff_accepts_dicts():
+    a = {"nodes": [{"id": "x"}], "links": []}
+    b = {"nodes": [{"id": "x"}, {"id": "z"}], "links": []}
+    report = serve._diff(a, b)
+    assert "added nodes     : 1" in report
+    assert "+ z" in report
+
+
+def test_recall_fallback_when_graphify_absent(tmp_path, monkeypatch):
+    gp = str(tmp_path / "reagraph.json")
+    serve._remember(gp, JAG_PASS2)
+    # Pretend graphify is not on PATH.
+    monkeypatch.setattr(serve.shutil, "which", lambda _name: None)
+    out = serve._recall(gp, "what calls net")
+    assert "graphify unavailable" in out
+    # The most-connected nodes surface in the fallback slice.
+    assert "check" in out and "net" in out
+
+
+def test_recall_fallback_empty_graph(tmp_path, monkeypatch):
+    gp = str(tmp_path / "reagraph.json")
+    monkeypatch.setattr(serve.shutil, "which", lambda _name: None)
+    out = serve._recall(gp, "anything")
+    assert "empty graph" in out
+
+
+def test_graph_path_env_override(tmp_path, monkeypatch):
+    monkeypatch.setenv("REAGRAPH_GRAPH", str(tmp_path / "env.json"))
+    assert serve._graph_path() == str(tmp_path / "env.json")
+    # Explicit arg wins over env.
+    assert serve._graph_path("explicit.json") == "explicit.json"

tests/test_summarize.py

@@ -0,0 +1,98 @@
+"""The compaction guarantee: a huge graph collapses to a bounded overview."""
+
+from reagraph import summarize
+
+
+def _hub_and_two_clusters():
+    """A few hundred nodes: one big star around `hubA`, a smaller star around
+    `hubB`, and a third disconnected pair -- three weakly-connected clusters."""
+    nodes = [{"id": "hubA", "label": "hubA()"}, {"id": "hubB", "label": "hubB()"}]
+    links = []
+    # hubA is the biggest hub: 200 leaves call it.
+    for i in range(200):
+        nid = f"a{i}"
+        nodes.append({"id": nid, "label": f"{nid}()"})
+        links.append({"source": nid, "target": "hubA", "relation": "calls",
+                      "confidence": "EXTRACTED"})
+    # hubB is a smaller hub: 50 leaves call it.
+    for i in range(50):
+        nid = f"b{i}"
+        nodes.append({"id": nid, "label": f"{nid}()"})
+        links.append({"source": nid, "target": "hubB", "relation": "calls",
+                      "confidence": "INFERRED"})
+    # A third, disconnected cluster: x -> y.
+    nodes.append({"id": "x", "label": "x()"})
+    nodes.append({"id": "y", "label": "y()"})
+    links.append({"source": "x", "target": "y", "relation": "calls",
+                  "confidence": "EXTRACTED"})
+    return {"directed": True, "multigraph": False, "nodes": nodes, "links": links}
+
+
+def test_god_nodes_rank_hubs_first_and_are_capped():
+    g = _hub_and_two_clusters()
+    gods = summarize.god_nodes(g, top=5)
+    assert len(gods) == 5
+    # The two hubs come first, biggest first.
+    assert gods[0]["id"] == "hubA" and gods[0]["degree"] == 200
+    assert gods[1]["id"] == "hubB" and gods[1]["degree"] == 50
+    assert gods[0]["label"] == "hubA()"
+
+
+def test_god_nodes_deterministic_tie_break_by_id():
+    g = _hub_and_two_clusters()
+    # All the leaves have degree 1; ties must break by id ascending.
+    gods = summarize.god_nodes(g, top=50)
+    tail = [x for x in gods if x["degree"] == 1]
+    assert tail == sorted(tail, key=lambda d: str(d["id"]))
+
+
+def test_components_find_the_three_clusters():
+    g = _hub_and_two_clusters()
+    comps = summarize.components(g)
+    assert len(comps) == 3
+    # Ordered by size descending: hubA star (201), hubB star (51), then x/y (2).
+    assert [len(c) for c in comps] == [201, 51, 2]
+    assert "hubA" in comps[0]
+    assert sorted(comps[2]) == ["x", "y"]
+
+
+def test_summarize_is_bounded_even_for_a_big_graph():
+    # 5000 leaves all pointing at one hub -> a graph far too big to read raw.
+    nodes = [{"id": "hub", "label": "hub()"}]
+    links = []
+    for i in range(5000):
+        nid = f"n{i}"
+        nodes.append({"id": nid})
+        links.append({"source": nid, "target": "hub", "relation": "calls"})
+    g = {"nodes": nodes, "links": links}
+
+    s = summarize.summarize(g, top=20, max_clusters=10)
+    assert s["nodes"] == 5001
+    assert len(s["god_nodes"]) <= 20
+    assert len(s["largest_clusters"]) <= 10
+    # The totals still reflect the real graph size even though lists are capped.
+    assert s["god_nodes"][0]["id"] == "hub"
+    assert s["num_components"] == 1
+
+
+def test_format_summary_is_a_short_string():
+    g = _hub_and_two_clusters()
+    text = summarize.format_summary(summarize.summarize(g))
+    assert isinstance(text, str)
+    assert "Graph overview" in text
+    # Short: an overview, not a dump of the whole graph.
+    assert len(text.splitlines()) < 60
+
+
+def test_scale_10k_chain_summarizes_without_error():
+    # A 10k-node chain: n0 -> n1 -> ... -> n9999 (one component, no hubs).
+    nodes = [{"id": f"n{i}"} for i in range(10000)]
+    links = [{"source": f"n{i}", "target": f"n{i + 1}", "relation": "calls"}
+             for i in range(9999)]
+    g = {"nodes": nodes, "links": links}
+
+    s = summarize.summarize(g, top=20, max_clusters=10)
+    assert s["nodes"] == 10000
+    assert s["num_components"] == 1
+    assert len(s["god_nodes"]) <= 20
+    assert isinstance(summarize.format_summary(s), str)