safishamsiandClaude Opus 4.8
77391fde49
Builds the server side of reagraph.remote_cache: the shared analysis-cache tier
("the real moat") whose client already shipped. A repeat analysis of the same
artifact -- already run by someone else, on another machine -- is now served
from a shared store without re-running the tool.
reagraph/cache_server.py: a pure-stdlib HTTP service (ThreadingHTTPServer) that
is a thin layer over the SAME reagraph.cache CacheBackend the local cache uses,
speaking the exact wire contract documented on remote_cache.UrllibTransport:
GET /{key} -> bytes|404, PUT /{key} -> 201, DELETE /{key} -> 204,
GET /keys -> {"keys":[...]}, GET /health -> ok.
Safety posture: keys must be sha256 hex (^[0-9a-f]{64}$) -- this is also the
path-traversal defense (a request segment is never turned into a filesystem
path; only a validated key reaches the backend, which shards it). Public read /
token-gated write, tokens from the environment ONLY (never argv -- it leaks via
ps), constant-time compared. Upload size cap (413) and cache-entry-shape
validation (422). Every verb is wrapped: an unexpected error is a 500, never a
crashed serving thread.
make_server/serve/main public API; `python -m reagraph.cache_server` and a new
`reagraph serve-cache` CLI subcommand (host/port/dir/max-value-mb/
no-validate-payload; tokens via env). Exported from reagraph.__init__.
Tests (20 new, 158 total): unit tests over a real loopback socket (round-trip,
404 miss, bad-key 400 before any backend call, /keys, delete, /health, size-cap
413, payload 422, read/write auth, 500-stays-alive) and a conformance suite
driving the REAL RemoteCacheBackend / TieredBackend / Cache against the real
server (cross-machine round-trip, read-only push is a no-op, tiered local-warm
on a remote hit, fail-open on a dead port). Verified live via the CLI end to end.
docs/CACHE_SERVER.md (operator + client guide, trust/privacy model) and deploy/
(reference Dockerfile, .dockerignore, compose.yaml, fly.toml, README) for a
hosted instance. README/roadmap updated: the shared cache is in the tree.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Deploying the REAgraph shared-cache server
This is a reference deployment of the REAgraph shared-cache server. It is not an officially hosted instance. Run your own with the files in this directory.
The server is a pure-stdlib Python HTTP service started by
python -m reagraph.cache_server. It serves a content-addressed cache of
reverse-engineering analysis fragments so repeat analyses are served without
re-running REA. See ../docs/CACHE_SERVER.md for the
protocol and ../docs/CACHE.md for how the client uses it.
The server ships inside the reagraph package, which is stdlib-only with no
runtime dependencies, so the image is tiny.
The server, in short
- CLI:
python -m reagraph.cache_server --host 0.0.0.0 --port 8787 --dir /dataplus--max-value-mband--no-validate-payload. - Auth tokens come from the environment only, never from CLI args:
REAGRAPH_CACHE_READ_TOKEN(optional; gates reads if set)REAGRAPH_CACHE_WRITE_TOKEN(gates PUT/DELETE if set)
- Storage dir is
--dir, orREAGRAPH_CACHE_SERVER_DIR. - It speaks plain HTTP. Put TLS in front with a proxy or platform.
Before you expose it
- Always set a write token for any internet-facing instance. Open writes
make the cache a public dumping ground. Set
REAGRAPH_CACHE_WRITE_TOKEN. - Set
REAGRAPH_CACHE_READ_TOKENtoo if the cache should not be world-readable. - Terminate TLS in front of the server. Fly does this for you. On a bare VPS use a reverse proxy (Caddy, nginx, Traefik) in front of port 8787.
- The
--diris a cache. Persist it on a volume so it survives restarts, but every entry is reproducible, so back it up lazily or not at all. Losing it only means the next analyses re-run and refill it.
Docker
Build from the repo root (the Dockerfile lives in deploy/ but the build
context is the repo root):
docker build -f deploy/Dockerfile -t reagraph-cache .
Run it with a write token and a persistent volume:
docker run -d \
-e REAGRAPH_CACHE_WRITE_TOKEN=your-write-token \
-v reagraph-cache:/data \
-p 8787:8787 \
reagraph-cache
Add -e REAGRAPH_CACHE_READ_TOKEN=your-read-token to gate reads as well.
The image runs as a non-root user, exposes 8787, and has a HEALTHCHECK that hits
/health with stdlib urllib (there is no curl in the slim base image).
Docker Compose (plain VPS)
compose.yaml builds from the repo-root context, maps a named
volume to /data, maps port 8787, and reads the tokens from the host
environment. Run it from the repo root:
REAGRAPH_CACHE_WRITE_TOKEN=your-write-token \
docker compose -f deploy/compose.yaml up -d
Export REAGRAPH_CACHE_READ_TOKEN in the same way if you want to gate reads.
Keep a reverse proxy in front for TLS.
Fly.io
fly.toml is a minimal, current-schema config using the placeholder
app name reagraph-cache. It builds from deploy/Dockerfile, mounts a volume at
/data, forces HTTPS, and health-checks /health. Deploy from the repo root so
the build context is the repo root:
# Create or register the app without deploying yet.
fly launch --no-deploy
# Set the write token as a secret (never a CLI arg or a committed file).
fly secrets set REAGRAPH_CACHE_WRITE_TOKEN=your-write-token
# Optional: gate reads too.
fly secrets set REAGRAPH_CACHE_READ_TOKEN=your-read-token
# Create the volume the [mounts] block expects.
fly volumes create reagraph_cache_data --size 1 --region iad
# Deploy.
fly deploy -c deploy/fly.toml
Fly terminates TLS for you, so clients reach it over https://.
Pointing a client at it
Cache(backend=RemoteCacheBackend("https://your-host", token=READ_TOKEN))
Use the read token (or the write token, which also reads) depending on whether you gated reads. See ../docs/CACHE.md for the full client story, including read-only-by-default and opt-in push.