Compare commits

..
Author SHA1 Message Date
Brent Vatne 3ac5186893 Roadmap: cross-run state reuse, a reviewer-tuning feedback loop, dropping OpenCode
Three next items, all extensions or simplifications of substrate already here.

Cross-run reuse: a re-review is stateless three times over, and the cheapest gap
is the one nobody has tried — no prior-run state reaches the prompt at all, so
the model is never told it has seen this PR before. Prior findings already
round-trip through the comment's embedded state; feeding them forward as trusted
host-built context needs no persisted engine state and works on any harness.

Engine session resume is mechanically confirmed to work (--session-id
pre-assigns, --resume restores full history across a different cwd, and
CLAUDE_CONFIG_DIR relocates transcripts), but records four things to settle
first: the economics invert once the prompt cache goes cold, a session is per
agent×chunk and chunks do not survive a push, a resumed transcript gives prompt
injection persistence across pushes, and the transcript format is version-coupled
to a CLI we pin and bump.

Feedback loop: LLP 0011 already produces the signal — matched replies,
adjudication verdicts, repeat-offender mining — and nothing consumes it. Notes
the degenerate outcome to design against: author friction is not correctness.

Drop OpenCode, but extract the harness interface before deleting anything. The
abstraction currently lives inside claude-code.ts and OpenCode is the silent
fallback for every non-Anthropic model, so the goal is that adding a harness
later is a new file plus a registry entry rather than a refactor.

Also records repo-configurable research sources as considered and deferred: it
reverses a stated LLP 0013 invariant, and research is an outbound disclosure
channel, so a repo-owned allowlist widens who can receive repository-derived
terms.
2026-08-08 11:39:27 -07:00
Brent Vatne a87b5085eb Remove the offline documentation index and its crawler (#71) 2026-08-08 11:00:07 -07:00
26 changed files with 368 additions and 972 deletions

No files matched your search

-1
View File
@@ -9,4 +9,3 @@ build/
# Claude Code local worktrees
.claude/worktrees/
research/data/
+5 -8
View File
@@ -325,14 +325,11 @@ downloads the fixed official `lysine.dev` static search index, validates and ind
it in memory once per MCP process, and rejects any entry outside the existing OkHttp
allowlist. Brave remains a fallback if that official index is unavailable.
An absolute `research.indexPath` remains available as an optional local fallback.
The MCP and its trusted updater ship with ECR: build that fallback from this
repository with `bun run research:update`, or from an installed package with
`review-research-mcp update`. This is operator/scheduled offline tooling, not a
step to run before each review. The built-in seed catalog lives in
`research/sources.json`; `seedUrls` are deterministic starting pages for the
bounded crawler, and extraction/indexing use no LLM. Installations requiring a fully
offline review can supply a separately built, verified index and omit the Brave key.
There is no offline index. Every passage a review sees is fetched live from the
provider allowlist during that review, so evidence is never served from a local
artifact whose contents ECR cannot vouch for. The crawler, its seed catalog, and
`research.indexPath` were removed once live discovery replaced them; a config still
naming `indexPath` fails to parse rather than silently ignoring it.
Installation-specific provider configuration is intentionally
deferred: when added, it should follow the trusted root-config model used for agents
without permitting PR-controlled URLs, commands, or executable parsers. Expo skills
+209
View File
@@ -55,6 +55,215 @@ PRs** (do these first, in this order):
isn't on `main` yet (nothing to build from base until this merges). See §3.9.
5. **Inline PR comments** (§1).
## Next up (2026-08-08)
The biggest items on deck. Each has substrate already in the repo, so the work is
mostly extending or simplifying something that exists — not greenfield.
### A. Cross-run state reuse — pick up where the last review left off
Today a re-review is stateless three times over. `review-cache.ts` (LLP
0005#review-result-cache) caches only a **whole** review result, keyed by a hash
of the entire input, so one changed byte anywhere discards everything. Every
Claude pass runs with `--no-session-persistence` (`claude-code.ts:343`), so
there is no session to resume even in principle. And — the cheapest gap to close
— **no prior-run state reaches the prompt at all**: prior findings live in the
comment's embedded state and are used only afterward, for suppression and
fingerprint matching. The model is never told it has seen this PR before.
(The research half of this is already fixed: removing the offline index let
`ci.ts` drop `!config.research.enabled` from `cacheAllowed`, so researched
reviews are cacheable again.)
Two independent levers, worth not conflating:
1. **Finer-grained result reuse.** Move below whole-review granularity so a
push that touches one file doesn't re-run every pass. §4 item 4 already
documents why this is hard and should be read first: chunk boundaries shift
under small edits (one modified file reshuffles every downstream chunk →
every hash misses), and findings depend on more than their own chunk, so
reusing an "unchanged" chunk's verdict can miss a break introduced by a
sibling change. The safer variant named there — reuse only when the *rest*
of the PR is unchanged (rebase / comment-only push) — reuses much less but
doesn't trade recall, and is probably the place to start.
2. **Prior-review context in the prompt (do this first).** Give the reviewer a
host-built block naming what it reported last time and what changed since —
built from the embedded comment state we already carry, so it is trusted,
structured, small, and bounded like every other prompt input.
The data is already there and needs no new storage: `ReviewState`
(`render.ts:518`) round-trips the whole `CoordinatorOutput` — every prior
finding with file, line, severity, category and rationale — plus `dismissed`,
`feedback`, `pins`, and the `inputHash`, through the PR comment on every run.
It is decoded on each run and used only afterward. Note this is *richer* than
a resumed transcript would be: it knows which findings a maintainer dismissed
and which ones the author answered. Note also the precedent — cross-run state
in this tool already travels in the comment, not in CI storage. This is what
actually delivers "this is a re-review, here is what moved"; it needs no
persisted engine state, survives a CLI upgrade, and works on any harness
(see C). Findings must still be re-derived from the current code — prior
findings are context, never a verdict to restate.
3. **Engine session reuse.** Mechanically confirmed to work (2026-08-08, claude
2.1.226): `--session-id <uuid>` *pre-assigns* an id, so ids can be derived
deterministically rather than scraped from stream-json; `--resume <id>` works
under `-p` and restores full history **across a different cwd**, so the
per-run `mkdtemp` head worktree is not a blocker; `CLAUDE_CONFIG_DIR` (already
in `CHILD_ENV_ALLOWLIST`) relocates transcripts somewhere cacheable. A warm
resume is much cheaper than a fresh pass — in a toy 4-turn measurement,
37k cached vs 18.5k re-created tokens.
Four things to settle before building it:
- **The economics invert when the cache is cold.** Prompt-cache TTL is 1 hour
on a subscription, 5 minutes on an API key. Warm, resume is a large win.
Cold, resume reprocesses the *entire* transcript (~100k+ tokens for a real
pass, per §4 item 1) where a fresh pass pays ~10–18k of new input. That is
roughly a 5× penalty on exactly the re-review that arrives the next day.
A resume that falls back to a fresh session past the TTL is strictly better
than either alone — and needs only a stored timestamp.
- **A session is per agent×chunk, and chunks are not stable across pushes**
(`review.ts:546`, and §4 item 4). Only single-chunk reviews — a diff under
`chunk.maxChangedLines` — have a session unit that survives a push at all.
For chunked PRs, "resume the session for chunk 3" is meaningless once
chunk 3 covers different files.
- **Injection gains persistence.** Today an injection can only affect the run
whose diff carries it. A resumed transcript lets text pushed in commit 1
shape the review of commit 5, after that text is gone. This is the real
security cost, and it argues for capping resume depth/age, re-running the
verifier statelessly, and never resuming across a base change.
- **The transcript format is internal and version-coupled.** Upstream states
it "changes between versions", and resuming after a CLI upgrade reprocesses
everything with no cache hits — and we pin and bump that CLI. The cache key
must include the CLI version, which means every bump discards every session.
If caching transcripts in the GitHub Actions cache: scope is favorable — a
PR's cache is written to `refs/pull/N/merge` and restorable **only by re-runs
of that same PR**, and a PR cannot write the default-branch scope, so there is
no cross-PR path. Cache **only** the `projects/<dir>/<id>.jsonl` files, never
`$CLAUDE_CONFIG_DIR` itself — that directory also holds credentials.
Decide the staleness budget before implementing. The tool's value proposition is
recall, and every reuse strategy above buys latency and quota by accepting some
probability of a stale conclusion.
### B. A feedback loop that tunes the reviewer
Author feedback shipped 2026-08-01 (LLP 0011) and already produces the signal:
replies matched deterministically to findings, enum-only records embedded in the
comment, adjudication verdicts, and `ecr feedback` mining a repo's whole PR
history into totals, reply rates, category/severity/agent breakdowns, and
"repeat offenders" — findings whose title recurs across PRs and draws a reply
every time. Adjacent signals already exist too: verifier keep/drop verdicts,
suppression, and the research usefulness counters.
What's missing is the loop. None of it feeds back into the prompts or config —
`ecr feedback` prints a report a human reads and acts on. The item is to close
that: aggregate the signal into a durable, reviewable tuning artifact and let a
run consume it.
Constraints to design against, all of which fall out of the existing trust
model rather than being new:
- **A PR author must not be able to tune the reviewer.** Feedback is authored by
untrusted parties on untrusted PRs. Anything derived from it must land in the
trusted base and go through review like any other config change — never be
applied live from the PR that produced it.
- **Prefer a reviewable artifact over auto-edited prompts.** A checked-in tuning
file a human approves keeps the change auditable and revertible; a model
silently rewriting `agents/*.md` does not.
- **Guard against the obvious degenerate outcome:** "authors disliked this
finding" is a signal about friction, not about correctness. Optimizing it
directly teaches the reviewer to stop reporting true positives people find
annoying. The hard floors from LLP 0011 (`critical`/`secrets`/`security` can
never be cleared) are the precedent for what must stay out of reach.
- Start read-only: report what a tuning change *would* do before letting it
change anything.
### C. Drop OpenCode, and make the harness seam real while doing it
Claude is the only harness we intend to support for now. Removing OpenCode is
mostly deletion, but the *point* of the item is the part that isn't: today the
engine abstraction is owned by one of its implementations, so "add a harness"
means editing the Claude module. The goal is that adding Codex, Gemini CLI, or a
direct-API harness later is a new file plus a registry entry — not a refactor.
What the seam looks like now:
- `Engine`, `engineForModel`, and `buildEngineMap` all live in
`src/core/claude-code.ts`. The interface is hosted inside an implementation.
- `engineForModel` hardcodes the whole routing rule as
`provider === "anthropic" ? claude-code : opencode`, so **OpenCode is the
silent fallback for every non-Anthropic model**. `engineOf[agent] ?? "opencode"`
appears at more than one dispatch site.
- Engine-specific knowledge leaks outward: `auth.ts` writes an OpenCode
`auth.json` under XDG, `setup-auth.ts` knows that path, `throttle.ts` parses
OpenCode's own log file for rate-limit evidence, and `doctor.ts` seeds its
engine set with `["opencode"]`.
So the work is:
1. **Extract the interface before deleting anything.** A `ReviewEngine` a harness
implements — start/stop, run one pass with (agent, model, system, prompt,
tools, schema), report usage/model-that-answered, surface throttle evidence —
living in its own module, not in a harness. `claude-code.ts` becomes one
implementation of it.
2. **Make model→harness a registry lookup, not a conditional.** A harness
declares the model prefixes it serves. An unmatched model must fail fast and
name the fix, never fall through to a default harness.
3. **Give harness-specific concerns a home in the harness.** Credential
materialization, CLI resolution, throttle/log parsing, and doctor probes are
per-harness behavior behind the same interface, not `if` branches in shared
code.
4. **Then delete OpenCode**: `opencode.ts`, the `opencode-ai`/`@opencode-ai/sdk`
dependencies, its auth.json path, its log parsing, and the mixed-engine
machinery in `review.ts` that exists only because one run could drive two
harnesses at once.
Two things to preserve rather than lose in the simplification. Mixed-engine
support is what makes step 2 honest — the abstraction should still permit two
harnesses in one run even while only one exists, because that is the case that
would otherwise be designed out and need rebuilding. And the per-agent model
override stays: harness choice follows from the model, so it stays a property of
an agent, never a global run mode.
The immediate payoff beyond deletion is that the research MCP wiring, the tool
allow/deny lists, and the session work in (A) each currently need a per-harness
implementation, and today that means editing both harness modules in step.
### Considered and deferred: repo-configurable research sources
Moving the documentation allowlist into `config.jsonc` so each repo points
research at its own stack. **Deferred 2026-08-08** — not the cost of building
it, but what it trades away.
`src/research-mcp/providers.ts` holds the runtime host/path allowlist and is the
outbound security boundary. Making it repo-owned reverses a stated invariant:
"Root config supplies only enablement, bounds, and an optional absolute fallback
index path—not an arbitrary command, endpoint, host, or argument vector" (LLP
0013#one-package-two-binaries). A PR head still could not add a host — research
config loads from the trusted base — but a repo admin could, and the 2026-08-07
audit established that research is an outbound disclosure channel where the
reviewing model chooses the query terms and URL path. So the feature widens who
can receive repository-derived terms. Two things also make it more than a config
change: retrieval is bespoke per provider (Apple DocC JSON, YouTrack JSON,
raw-markdown rewrites), so a repo-declared source realistically gets the generic
HTML path only; and the MCP tool schema is generated from the fixed list
(`z.enum(PROVIDERS)`), so per-repo sources make the tool schema, prompt provider
map, citation grounding, and audit provider field all vary by repo.
Revisit if adopters actually ask for internal documentation hosts. If so, the
shape most likely to survive is repo sources *added to* the package catalog
rather than replacing it, with exact hostname plus path prefix only — no pattern
language, and no pluggable adapters.
Note for whoever picks this up: `research/sources.json` is **not** this
allowlist. It is the seed catalog for the operator-only `update` crawler
(`cli.ts` lazily imports `crawler.js` there and nowhere else); no review-time
code path reads it. Editing it changes what an offline index build crawls, not
what a review is allowed to fetch.
## Recently shipped
- **Author feedback on findings (shipped 2026-08-01)** — a PR author's reply is
+16 -16
View File
@@ -119,24 +119,24 @@ has the same response-size/schema/URL constraints, is cached only within the MCP
process, and can never admit a URL outside the OkHttp provider allowlist. Brave is a
fallback when that first-party index is unavailable.
`review-research-mcp update` is a separate, operator-invoked command for trusted
scheduled jobs. The updater uses exact HTTPS host/path allowlists, manually
revalidates redirects, checks response content types, and bounds response size,
crawl depth, page count, timeout, and delay. The Expo crawl supplies a local fallback;
precise Expo search uses the live query rather than pretending an empty-query result
page is a complete Algolia index.
There is no offline index, and no crawler to build one. Live discovery replaced the
deterministic prefetch it was designed for, and the fallback it provided was never
free: an index is an artifact whose bytes sit outside the trust boundary the rest of
the system establishes. `research.indexPath` had to be an absolute path, so it
resolved against the runner's filesystem rather than the materialized trusted base —
the trusted-base mechanism protected the config that NAMED the index, never the
contents at that path, and nothing carried a digest. Index passages were also the one
evidence source returned unbounded, while every live source is length-capped.
The crawler is not part of the ordinary review workflow. An adopter requiring an
offline fallback may run `update` in a separate scheduled secretless job from a
pinned package and trusted checkout. That job must execute no PR code, receive no
model credential, and publish a verified read-only index artifact.
Removing it deletes that whole class of question rather than answering it. Every
passage a review sees is now fetched during that review from an allowlisted host,
bounded on entry. A config still naming `research.indexPath` fails to parse, so a
stale setting surfaces as an error instead of being silently ignored.
The generated index is deliberately not published in the npm package. Documentation
changes independently of ECR, and coupling a snapshot to every CLI install would make
releases and invalidation expensive. If supplied, an absolute trusted-base
`research.indexPath` is fallback evidence when remote discovery fails or misses.
Research-enabled reviews bypass result-cache reuse because remote search,
documentation, and optional index content can change without a config change.
This also removes the reason researched reviews could not reuse a cached result.
Reuse was disabled whenever research was enabled because evidence depended on mounted
index contents that the cache key could not represent; with no mounted artifact, the
ordinary input hash covers the whole review again.
## Query and Prompt Boundary
+1 -3
View File
@@ -16,8 +16,7 @@
},
"files": [
"build",
"templates",
"research/sources.json"
"templates"
],
"engines": {
"node": ">=20"
@@ -38,7 +37,6 @@
"llp:check": "./ref-check",
"dev": "bun run src/cli.ts",
"test:unit": "bun test",
"research:update": "bun run src/research-mcp/cli.ts update",
"research:evaluate:corpora": "bun run build && node scripts/research/evaluate-corpora.mjs",
"research:evaluate:expo": "bun run build && node scripts/research/evaluate-expo-prs.mjs",
"release": "bash scripts/release.sh",
-295
View File
@@ -1,295 +0,0 @@
{
"output": "data/docs-index.json",
"crawl": {
"maxPagesPerProvider": 250,
"maxDepth": 4,
"delayMs": 200,
"timeoutMs": 15000,
"maxResponseBytes": 5000000
},
"sources": [
{
"provider": "apple",
"sourceKind": "official-api",
"maxPages": 170,
"seedUrls": [
"https://developer.apple.com/documentation/swift",
"https://developer.apple.com/documentation/swiftui",
"https://developer.apple.com/documentation/uikit",
"https://developer.apple.com/documentation/foundation",
"https://developer.apple.com/documentation/avfoundation",
"https://developer.apple.com/documentation/coregraphics",
"https://developer.apple.com/documentation/photos",
"https://developer.apple.com/documentation/usernotifications",
"https://developer.apple.com/documentation/webkit",
"https://developer.apple.com/documentation/storekit",
"https://developer.apple.com/documentation/activitykit",
"https://developer.apple.com/documentation/widgetkit",
"https://developer.apple.com/documentation/appintents",
"https://developer.apple.com/documentation/network",
"https://developer.apple.com/documentation/photos/phasset",
"https://developer.apple.com/documentation/photos/phasset/location",
"https://developer.apple.com/documentation/network/nwpathmonitor",
"https://developer.apple.com/documentation/network/nwpathmonitor/pathupdatehandler",
"https://developer.apple.com/documentation/appintents/liveactivityintent",
"https://developer.apple.com/documentation/activitykit/activity",
"https://developer.apple.com/documentation/activitykit/activity/id",
"https://developer.apple.com/documentation/activitykit/activity/activities",
"https://developer.apple.com/documentation/activitykit/activity/activitystate",
"https://developer.apple.com/documentation/activitykit/activity/pushtokenupdates-swift.property",
"https://developer.apple.com/documentation/avfaudio/avaudiosession",
"https://developer.apple.com/documentation/avfaudio/avaudiosession/setactive(_:options:)",
"https://developer.apple.com/documentation/avfaudio/avaudiosession/setactiveoptions/notifyothersondeactivation",
"https://developer.apple.com/documentation/swiftui/view/widgeturl(_:)",
"https://developer.apple.com/design/human-interface-guidelines/"
]
},
{
"provider": "apple-releases",
"sourceKind": "release-notes",
"maxPages": 40,
"maxDepth": 2,
"seedUrls": [
"https://developer.apple.com/documentation/xcode-release-notes",
"https://developer.apple.com/documentation/ios-ipados-release-notes"
]
},
{
"provider": "swift-evolution",
"sourceKind": "official-guide",
"maxPages": 35,
"maxDepth": 1,
"seedUrls": [
"https://www.swift.org/swift-evolution/",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0302-concurrent-value-and-concurrent-closures.md",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0304-structured-concurrency.md",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0306-actors.md",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0316-global-actors.md",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0338-clarify-execution-non-actor-async.md",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0420-inheritance-of-actor-isolation.md",
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0430-transferring-parameters-and-results.md"
]
},
{
"provider": "sdwebimage",
"sourceKind": "official-api",
"maxPages": 30,
"maxDepth": 2,
"seedUrls": [
"https://sdwebimage.github.io/documentation/sdwebimage/",
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimageoptions/",
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagecontextoption/"
]
},
{
"provider": "android",
"sourceKind": "official-api",
"maxPages": 150,
"seedUrls": [
"https://developer.android.com/develop",
"https://developer.android.com/reference",
"https://developer.android.com/develop/ui/views/text-and-emoji/fonts-in-xml",
"https://developer.android.com/reference/androidx/biometric/BiometricPrompt.PromptInfo.Builder",
"https://developer.android.com/reference/android/media/AudioFocusRequest",
"https://developer.android.com/reference/android/media/AudioManager",
"https://developer.android.com/reference/android/net/ConnectivityManager",
"https://developer.android.com/reference/android/net/ConnectivityManager.NetworkCallback",
"https://developer.android.com/reference/android/content/Intent",
"https://developer.android.com/reference/android/app/NotificationManager",
"https://developer.android.com/reference/android/os/Build.VERSION_CODES",
"https://developer.android.com/reference/androidx/core/os/BuildCompat",
"https://developer.android.com/reference/androidx/work/Constraints.Builder",
"https://developer.android.com/reference/androidx/work/NetworkType",
"https://developer.android.com/develop/ui/compose/graphics/images/loading",
"https://developer.android.com/develop/ui/compose/graphics/images/customize",
"https://developer.android.com/reference/kotlin/androidx/compose/foundation/Image.composable",
"https://developers.google.com/android/reference/com/google/android/gms/location/FusedLocationProviderClient",
"https://developers.google.com/android/reference/com/google/android/gms/location/LocationCallback"
]
},
{
"provider": "android-releases",
"sourceKind": "release-notes",
"maxPages": 50,
"maxDepth": 2,
"seedUrls": [
"https://developer.android.com/about/versions/16",
"https://developer.android.com/about/versions/16/features",
"https://developer.android.com/about/versions/16/behavior-changes-all",
"https://developer.android.com/about/versions/16/behavior-changes-16",
"https://developer.android.com/about/versions/15/behavior-changes-all",
"https://developer.android.com/about/versions/15/behavior-changes-15",
"https://developer.android.com/about/versions/14/behavior-changes-all",
"https://developer.android.com/about/versions/14/behavior-changes-14"
]
},
{
"provider": "media3",
"sourceKind": "official-guide",
"maxPages": 40,
"maxDepth": 2,
"seedUrls": [
"https://developer.android.com/media/media3",
"https://developer.android.com/media/media3/session/background-playback",
"https://developer.android.com/jetpack/androidx/releases/media3",
"https://developer.android.com/reference/androidx/media3/common/Player",
"https://developer.android.com/reference/androidx/media3/session/MediaSessionService"
]
},
{
"provider": "glide",
"sourceKind": "official-guide",
"maxPages": 30,
"maxDepth": 2,
"seedUrls": [
"https://bumptech.github.io/glide/doc/caching.html",
"https://bumptech.github.io/glide/doc/configuration.html",
"https://bumptech.github.io/glide/javadocs/450/com/bumptech/glide/load/engine/cache/DiskCache.html",
"https://bumptech.github.io/glide/javadocs/450/com/bumptech/glide/load/engine/cache/InternalCacheDiskCacheFactory.html"
]
},
{
"provider": "okhttp",
"sourceKind": "official-guide",
"maxPages": 30,
"maxDepth": 2,
"seedUrls": [
"https://lysine.dev/okhttp/",
"https://lysine.dev/okhttp/recipes/",
"https://lysine.dev/okhttp/features/caching/",
"https://lysine.dev/okhttp/features/calls/",
"https://lysine.dev/okhttp/features/connections/",
"https://lysine.dev/okhttp/features/https/",
"https://lysine.dev/okhttp/features/interceptors/",
"https://lysine.dev/okhttp/changelogs/changelog/",
"https://lysine.dev/okhttp/5.x/okhttp/okhttp3/-ok-http-client/"
]
},
{
"provider": "kotlin-coroutines",
"sourceKind": "official-guide",
"maxPages": 40,
"maxDepth": 1,
"seedUrls": [
"https://kotlinlang.org/docs/coroutines-guide.html",
"https://kotlinlang.org/docs/coroutines-basics.html",
"https://kotlinlang.org/docs/cancellation-and-timeouts.html",
"https://kotlinlang.org/docs/exception-handling.html",
"https://kotlinlang.org/docs/shared-mutable-state-and-concurrency.html",
"https://kotlinlang.org/docs/flow.html",
"https://kotlinlang.org/api/kotlinx.coroutines/"
]
},
{
"provider": "gradle",
"sourceKind": "official-guide",
"maxPages": 40,
"maxDepth": 1,
"seedUrls": [
"https://docs.gradle.org/current/userguide/tooling_api.html",
"https://docs.gradle.org/current/userguide/composite_builds.html",
"https://docs.gradle.org/current/userguide/public_apis.html",
"https://docs.gradle.org/current/userguide/upgrading_version_9.html",
"https://docs.gradle.org/current/release-notes.html"
]
},
{
"provider": "agp",
"sourceKind": "release-notes",
"maxPages": 40,
"maxDepth": 1,
"seedUrls": [
"https://developer.android.com/build/releases/about-agp",
"https://developer.android.com/build/releases/gradle-plugin-roadmap",
"https://developer.android.com/build/releases/agp-9-0-0-release-notes",
"https://developer.android.com/reference/tools/gradle-api"
]
},
{
"provider": "jetbrains-issues",
"sourceKind": "issue-tracker",
"maxPages": 1,
"maxDepth": 0,
"seedUrls": [
"https://youtrack.jetbrains.com/issue/IDEA-329756/Importing-symlinked-Gradle-included-build-fails"
]
},
{
"provider": "expo",
"sourceKind": "official-api",
"maxPages": 100,
"maxDepth": 3,
"seedUrls": [
"https://docs.expo.dev/versions/latest/",
"https://docs.expo.dev/versions/latest/sdk/expo/",
"https://docs.expo.dev/versions/latest/sdk/camera/",
"https://docs.expo.dev/versions/latest/sdk/notifications/",
"https://docs.expo.dev/router/introduction/",
"https://docs.expo.dev/modules/overview/",
"https://docs.expo.dev/guides/new-architecture/"
]
},
{
"provider": "react-native",
"sourceKind": "official-api",
"maxPages": 80,
"maxDepth": 3,
"seedUrls": [
"https://reactnative.dev/docs/getting-started",
"https://reactnative.dev/docs/components-and-apis",
"https://reactnative.dev/docs/flatlist",
"https://reactnative.dev/docs/view",
"https://reactnative.dev/docs/platform",
"https://reactnative.dev/architecture/landing-page"
]
},
{
"provider": "react-native-reanimated",
"sourceKind": "official-guide",
"maxPages": 60,
"maxDepth": 3,
"seedUrls": [
"https://docs.swmansion.com/react-native-reanimated/docs/fundamentals/getting-started/",
"https://docs.swmansion.com/react-native-reanimated/docs/guides/testing/",
"https://docs.swmansion.com/react-native-reanimated/docs/guides/compatibility/",
"https://docs.swmansion.com/react-native-reanimated/docs/category/core/",
"https://docs.swmansion.com/react-native-reanimated/docs/category/advanced-apis/"
]
},
{
"provider": "react-native-gesture-handler",
"sourceKind": "official-guide",
"maxPages": 60,
"maxDepth": 3,
"seedUrls": [
"https://docs.swmansion.com/react-native-gesture-handler/docs/fundamentals/getting-started/",
"https://docs.swmansion.com/react-native-gesture-handler/docs/gestures/pan-gesture/",
"https://docs.swmansion.com/react-native-gesture-handler/docs/gestures/gesture-detector/"
]
},
{
"provider": "react-native-screens",
"sourceKind": "official-guide",
"maxPages": 50,
"maxDepth": 3,
"seedUrls": [
"https://docs.swmansion.com/react-native-screens/",
"https://github.com/software-mansion/react-native-screens/blob/main/README.md"
]
},
{
"provider": "react-native-worklets",
"sourceKind": "official-guide",
"maxPages": 60,
"maxDepth": 3,
"seedUrls": [
"https://docs.swmansion.com/react-native-worklets/docs/fundamentals/getting-started/",
"https://docs.swmansion.com/react-native-worklets/docs/guides/testing/",
"https://docs.swmansion.com/react-native-worklets/docs/category/fundamentals/",
"https://docs.swmansion.com/react-native-worklets/docs/fundamentals/sharing-memory/",
"https://docs.swmansion.com/react-native-worklets/docs/guides/feature-flags/"
]
}
]
}
-5
View File
@@ -3,8 +3,6 @@
// Default mode replays recorded fixtures (no network, no key) and validates
// the full MCP pipeline deterministically. Set RESEARCH_EVAL_LIVE=1 (plus
// BRAVE_SEARCH_API_KEY) to run the same cases against live search instead.
import { existsSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
@@ -12,8 +10,6 @@ import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
import { createDocumentationServer } from "../../build/research-mcp/server.js";
import { createRecordedFetch } from "./recorded-fetch.mjs";
const INDEX_PATH = fileURLToPath(new URL("../../research/data/docs-index.json", import.meta.url));
async function callSearch(serverOptions, name, args) {
// One server per case keeps every case inside the per-server call budget.
const server = await createDocumentationServer(serverOptions);
@@ -44,7 +40,6 @@ export async function runEvalCases(cases, { clientName, buildArguments, assertCa
...(process.env.BRAVE_SEARCH_API_KEY
? { braveApiKey: process.env.BRAVE_SEARCH_API_KEY }
: {}),
...(existsSync(INDEX_PATH) ? { indexPath: INDEX_PATH } : {}),
}
: { fetchImplementation: createRecordedFetch(cases), braveApiKey: "recorded-fixtures" };
@@ -1,68 +0,0 @@
import { expect, test } from "bun:test";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { resolveIndexOutputPath, updateDocumentationIndex } from "../../research-mcp/crawler.js";
test("default index output is relative to the source config directory", () => {
const configPath = path.join("repo", "research", "sources.json");
expect(resolveIndexOutputPath(configPath, "data/docs-index.json")).toBe(
path.resolve("repo", "research", "data", "docs-index.json"),
);
});
test("an explicit output path remains relative to the caller's working directory", () => {
expect(
resolveIndexOutputPath(
path.join("repo", "research", "sources.json"),
"data/docs-index.json",
"custom/index.json",
),
).toBe(path.resolve("custom", "index.json"));
});
test("an invalid seed is reported without aborting other seeds", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "review-research-crawler-"));
const configPath = path.join(directory, "sources.json");
const originalFetch = globalThis.fetch;
globalThis.fetch = async () =>
new Response(
"<html><head><title>Android Widget</title></head><body><main><h1>Android Widget</h1><p>A documented Android API with enough visible text to produce a searchable local index passage.</p></main></body></html>",
{ headers: { "content-type": "text/html" } },
);
try {
await writeFile(
configPath,
JSON.stringify({
output: "data/index.json",
crawl: {
maxPagesPerProvider: 2,
maxDepth: 0,
delayMs: 0,
timeoutMs: 1000,
maxResponseBytes: 4096,
},
sources: [
{
provider: "android",
sourceKind: "official-api",
seedUrls: [
"https://untrusted.example/reference/widget",
"https://developer.android.com/reference/widget",
],
},
],
}),
);
const result = await updateDocumentationIndex({ configPath });
expect(result.documentCount).toBe(1);
expect(result.providers[0]?.errors[0]).toContain("untrusted.example");
expect(JSON.parse(await readFile(result.outputPath, "utf8")).documentCount).toBe(1);
} finally {
globalThis.fetch = originalFetch;
await rm(directory, { recursive: true, force: true });
}
});
+5 -61
View File
@@ -7,45 +7,13 @@ import { fileURLToPath } from "node:url";
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { buildSearchIndex, writeSearchIndex } from "../../research-mcp/search-index.js";
test("stdio MCP lists and calls the read-only documentation search tool", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "review-research-mcp-"));
const indexPath = path.join(directory, "index.json");
const built = buildSearchIndex(
[
{
id: "apple:mainactor",
platform: "apple",
provider: "apple",
sourceKind: "official-api",
title: "MainActor",
url: "https://developer.apple.com/documentation/swift/mainactor",
passage: "A singleton actor whose executor is equivalent to the main dispatch queue.",
language: "swift",
indexedAt: "2026-08-06T00:00:00.000Z",
},
{
id: "apple:poisoned-mainactor",
platform: "apple",
provider: "apple",
sourceKind: "official-api",
title: "MainActor forged mirror",
url: "https://attacker.example/documentation/swift/mainactor",
passage: "MainActor dispatch queue documentation copied to an untrusted host.",
language: "swift",
indexedAt: "2026-08-06T00:00:00.000Z",
},
],
1,
"2026-08-06T00:00:00.000Z",
);
await writeSearchIndex(indexPath, built.serialized);
// The only test that drives the real stdio binary, so it owns the tool-metadata
// contract: names, annotations, and the schemas a model actually reads.
test("stdio MCP advertises the read-only documentation tools and their enforced bounds", async () => {
const cliPath = fileURLToPath(new URL("../../research-mcp/cli.ts", import.meta.url));
const transport = new StdioClientTransport({
command: process.execPath,
args: [cliPath, "serve", "--index", indexPath],
args: [cliPath, "serve"],
stderr: "pipe",
});
const client = new Client({ name: "test-client", version: "1.0.0" });
@@ -101,34 +69,12 @@ test("stdio MCP lists and calls the read-only documentation search tool", async
| undefined;
expect(searchInputSchema?.properties?.limit?.maximum).toBe(3);
expect(searchInputSchema?.properties?.limit?.default).toBe(3);
const response = await client.callTool({
name: "search_platform_docs",
arguments: {
platform: "apple",
providers: ["apple"],
sourceKinds: ["official-api"],
query: "MainActor dispatch queue",
limit: 3,
},
});
const content = response.content as Array<{ type: string; text?: string }>;
const textBlock = content.find((block) => block.type === "text");
expect(textBlock?.type).toBe("text");
const payload = JSON.parse(textBlock.text ?? "") as {
results: Array<{ title: string; provider: string; sourceKind: string }>;
};
expect(payload.results[0]?.title).toBe("MainActor");
expect(payload.results[0]?.provider).toBe("apple");
expect(payload.results[0]?.sourceKind).toBe("official-api");
expect(payload.results).toHaveLength(1);
} finally {
await client.close();
await rm(directory, { recursive: true, force: true });
}
});
test("stdio MCP starts without an index and reports unavailable remote discovery honestly", async () => {
test("stdio MCP reports unavailable remote discovery honestly", async () => {
const cliPath = fileURLToPath(new URL("../../research-mcp/cli.ts", import.meta.url));
const transport = new StdioClientTransport({
command: process.execPath,
@@ -152,7 +98,6 @@ test("stdio MCP starts without an index and reports unavailable remote discovery
const content = response.content as Array<{ type: string; text?: string }>;
const payload = JSON.parse(content.find((block) => block.type === "text")?.text ?? "") as {
retrieval: {
localIndex: unknown;
scopedWebSearch: boolean;
network: Record<string, number>;
};
@@ -162,7 +107,6 @@ test("stdio MCP starts without an index and reports unavailable remote discovery
expect(payload.retrieval).toMatchObject({
scopedWebSearch: false,
expoSearch: false,
localIndex: null,
});
// No key and no provider reachable, so the call must not have touched the network.
expect(payload.retrieval.network).toMatchObject({
+81 -112
View File
@@ -1,14 +1,6 @@
import { expect, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import {
buildSearchIndex,
loadSearchIndex,
searchDocumentation,
writeSearchIndex,
} from "../../research-mcp/search-index.js";
import { buildSearchIndex, searchDocumentation } from "../../research-mcp/search-index.js";
import type { IndexedChunk } from "../../research-mcp/types.js";
const chunks: IndexedChunk[] = [
@@ -33,36 +25,28 @@ const chunks: IndexedChunk[] = [
},
];
test("a serialized index reloads and filters search results by platform", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "review-research-index-"));
try {
const filePath = path.join(directory, "index.json");
const built = buildSearchIndex(chunks, 2, "2026-08-06T00:00:00.000Z");
await writeSearchIndex(filePath, built.serialized);
const loaded = await loadSearchIndex(filePath);
test("search filters results by platform and language", () => {
const index = buildSearchIndex(chunks);
const appleResults = searchDocumentation(loaded, "main actor dispatch queue", {
platform: "apple",
language: "swift",
limit: 5,
});
expect(appleResults).toHaveLength(1);
expect(appleResults[0]?.title).toBe("MainActor");
expect(appleResults[0]?.nextPassageId).toBe("apple:2");
const appleResults = searchDocumentation(index, "main actor dispatch queue", {
platform: "apple",
language: "swift",
limit: 5,
});
expect(appleResults).toHaveLength(1);
expect(appleResults[0]?.title).toBe("MainActor");
expect(appleResults[0]?.nextPassageId).toBe("apple:2");
const androidResults = searchDocumentation(loaded, "lifecycle callbacks", {
platform: "android",
limit: 5,
});
expect(androidResults).toHaveLength(1);
expect(androidResults[0]?.platform).toBe("android");
} finally {
await rm(directory, { recursive: true, force: true });
}
const androidResults = searchDocumentation(index, "lifecycle callbacks", {
platform: "android",
limit: 5,
});
expect(androidResults).toHaveLength(1);
expect(androidResults[0]?.platform).toBe("android");
});
test("search rejects empty and oversized queries", () => {
const index = buildSearchIndex(chunks, 2);
const index = buildSearchIndex(chunks);
expect(() => searchDocumentation(index, "\u0000\n", { platform: "all", limit: 5 })).toThrow(
/between 1 and 300/,
);
@@ -72,19 +56,16 @@ test("search rejects empty and oversized queries", () => {
});
test("symbol queries do not fall back to unrelated partial matches", () => {
const index = buildSearchIndex(
[
{
id: "android:unrelated",
platform: "android",
title: "Transient UI",
url: "https://developer.android.com/example",
passage: "A transient user interface action.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
],
1,
);
const index = buildSearchIndex([
{
id: "android:unrelated",
platform: "android",
title: "Transient UI",
url: "https://developer.android.com/example",
passage: "A transient user interface action.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
]);
expect(
searchDocumentation(index, "AudioManager AUDIOFOCUS_GAIN_TRANSIENT", {
@@ -95,19 +76,16 @@ test("symbol queries do not fall back to unrelated partial matches", () => {
});
test("symbol fallback keeps results anchored by canonical identity", () => {
const index = buildSearchIndex(
[
{
id: "apple:phasset",
platform: "apple",
title: "PHAsset",
url: "https://developer.apple.com/documentation/photos/phasset",
passage: "An image, video, or Live Photo in the Photos library.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
],
1,
);
const index = buildSearchIndex([
{
id: "apple:phasset",
platform: "apple",
title: "PHAsset",
url: "https://developer.apple.com/documentation/photos/phasset",
passage: "An image, video, or Live Photo in the Photos library.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
]);
const results = searchDocumentation(index, "PHAsset location", {
platform: "apple",
@@ -117,19 +95,16 @@ test("symbol fallback keeps results anchored by canonical identity", () => {
});
test("symbol anchors do not match identifier suffixes or generic acronyms", () => {
const index = buildSearchIndex(
[
{
id: "android:audio-focus-builder",
platform: "android",
title: "AudioFocusRequest.Builder API",
url: "https://developer.android.com/reference/android/media/AudioFocusRequest.Builder",
passage: "Builds an audio focus request with a custom listener.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
],
1,
);
const index = buildSearchIndex([
{
id: "android:audio-focus-builder",
platform: "android",
title: "AudioFocusRequest.Builder API",
url: "https://developer.android.com/reference/android/media/AudioFocusRequest.Builder",
passage: "Builds an audio focus request with a custom listener.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
]);
expect(
searchDocumentation(index, "OkHttp Request.Builder custom header API", {
@@ -140,31 +115,28 @@ test("symbol anchors do not match identifier suffixes or generic acronyms", () =
});
test("search filters named corpora and provenance classes", () => {
const index = buildSearchIndex(
[
{
id: "glide:cache",
platform: "android",
provider: "glide",
sourceKind: "official-guide",
title: "Glide caching",
url: "https://bumptech.github.io/glide/doc/caching.html",
passage: "Configure a custom DiskCache implementation.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
{
id: "jetbrains:cache",
platform: "android",
provider: "jetbrains-issues",
sourceKind: "issue-tracker",
title: "Cache issue",
url: "https://youtrack.jetbrains.com/issue/IDEA-1",
passage: "A user reported a DiskCache problem.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
],
2,
);
const index = buildSearchIndex([
{
id: "glide:cache",
platform: "android",
provider: "glide",
sourceKind: "official-guide",
title: "Glide caching",
url: "https://bumptech.github.io/glide/doc/caching.html",
passage: "Configure a custom DiskCache implementation.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
{
id: "jetbrains:cache",
platform: "android",
provider: "jetbrains-issues",
sourceKind: "issue-tracker",
title: "Cache issue",
url: "https://youtrack.jetbrains.com/issue/IDEA-1",
passage: "A user reported a DiskCache problem.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
]);
const results = searchDocumentation(index, "DiskCache", {
platform: "android",
@@ -178,20 +150,17 @@ test("search filters named corpora and provenance classes", () => {
});
test("a language filter excludes otherwise matching untagged documents", () => {
const index = buildSearchIndex(
[
...chunks,
{
id: "apple:untagged",
platform: "apple",
title: "MainActor overview",
url: "https://developer.apple.com/documentation/swift/mainactor-overview",
passage: "Main actor dispatch queue behavior without language metadata.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
],
3,
);
const index = buildSearchIndex([
...chunks,
{
id: "apple:untagged",
platform: "apple",
title: "MainActor overview",
url: "https://developer.apple.com/documentation/swift/mainactor-overview",
passage: "Main actor dispatch queue behavior without language metadata.",
indexedAt: "2026-08-06T00:00:00.000Z",
},
]);
const results = searchDocumentation(index, "MainActor dispatch queue", {
platform: "apple",
+6 -8
View File
@@ -106,16 +106,14 @@ test("research child environment forwards only locale, proxy variables, and the
expect(researchChildEnvironment({ PATH: "/private/bin" })).not.toHaveProperty("PATH");
});
test("research index fallback is absolute and research remains root-only", () => {
expect(
ReviewConfigSchema.safeParse({ research: { enabled: true, indexPath: "index.json" } }).success,
).toBe(false);
test("research is root-only and rejects an unknown key", () => {
expect(ReviewConfigSchema.safeParse({ research: { enabled: true } }).success).toBe(true);
// The offline index is gone; a stale config naming it must fail loudly rather
// than be silently ignored.
expect(
ReviewConfigSchema.safeParse({
research: { enabled: true, indexPath: path.join(tmpdir(), "index.json") },
}).success,
).toBe(true);
ReviewConfigSchema.safeParse({ research: { enabled: true, indexPath: "/tmp/index.json" } })
.success,
).toBe(false);
expect(ScopeReviewConfigSchema.safeParse({ research: { enabled: false } }).success).toBe(false);
});
+4 -5
View File
@@ -578,11 +578,10 @@ async function runLegacyCi(
// Dynamic stack context and model-backed reply adjudication have inputs outside
// the scoped diff. Keep those paths fresh until their inputs join the cache key.
// A maintainer's explicit /review is also always a real rerun.
// Research output depends on the mounted index contents, not merely its configured
// path. Until a signed index digest joins the cache key, a researched review must
// run fresh rather than reuse evidence from an older artifact at the same path.
const cacheAllowed =
!bypassTriggerGate && !stack && !feedback && !config.research.enabled && metadata !== undefined;
// Research no longer forces a fresh run: with the local index gone, every result
// is fetched live from an allowlisted host during the run, so there is no mounted
// artifact whose contents could drift out from under the cache key.
const cacheAllowed = !bypassTriggerGate && !stack && !feedback && metadata !== undefined;
let inputHash: string | undefined;
try {
+7 -4
View File
@@ -65,10 +65,14 @@ export const ReviewConfigSchema = z.object({
research: z
.object({
enabled: z.boolean().default(false),
// Removed with the offline index. An unknown key would be stripped silently,
// so name it explicitly: a config carrying it is stale, and quietly ignoring
// the setting is worse than refusing to start.
indexPath: z
.string()
.min(1)
.refine((value) => path.isAbsolute(value), "research.indexPath must be an absolute path")
.never({
error:
"research.indexPath was removed — documentation is now always fetched live from the provider allowlist. Delete this key.",
})
.optional(),
maxQueries: z.number().int().min(1).max(20).default(8),
resultsPerQuery: z.number().int().min(1).max(3).default(2),
@@ -433,7 +437,6 @@ export interface LoadedConfig {
/** Root-only configuration for the bundled, bounded documentation MCP. */
research: {
enabled: boolean;
indexPath?: string;
maxQueries: number;
resultsPerQuery: number;
timeoutMs: number;
+1 -5
View File
@@ -146,11 +146,7 @@ export async function createResearchMcpRuntime(
const auditPath = path.join(directory, "audit.jsonl");
const claudeConfigPath = path.join(directory, "mcp.json");
const server = bundledResearchServer();
const args = [
...server.args,
"serve",
...(config.indexPath ? ["--index", config.indexPath] : []),
];
const args = [...server.args, "serve"];
const child = researchChildEnvironment();
const environment = Object.fromEntries(
Object.entries({
-1
View File
@@ -37,7 +37,6 @@ export const RESEARCH_SEARCH_API_KEY = "BRAVE_SEARCH_API_KEY";
*/
export const RESEARCH_RUNTIME_ENV_KEYS = [
"REVIEW_RESEARCH_AUDIT_PATH",
"REVIEW_RESEARCH_INDEX_PATH",
"REVIEW_RESEARCH_MAX_CALLS",
"REVIEW_RESEARCH_MAX_RESULTS",
"REVIEW_RESEARCH_TIMEOUT_MS",
+10 -61
View File
@@ -1,28 +1,20 @@
#!/usr/bin/env node
// @ref LLP 0013#one-package-two-binaries [implements] — the package's second binary owns serve/update dispatch
// @ref LLP 0013#search-fetch-and-optional-index-boundary [implements] — review-facing serve and operator-only update stay separate
// @ref LLP 0013#one-package-two-binaries [implements] — the package's second binary serves the bounded MCP
// @ref LLP 0013#search-fetch-and-optional-index-boundary [implements] — live discovery is the only evidence path
import { parseArgs } from "node:util";
import { defaultConfigPath } from "./paths.js";
import { runStdioServer } from "./server.js";
import { PLATFORMS, type Platform } from "./types.js";
function printHelp() {
process.stdout.write(`review-research-mcp
Usage:
review-research-mcp [serve]
review-research-mcp serve [--index PATH]
review-research-mcp update [--config PATH] [--output PATH]
[--platform apple|android|react-native] [--max-pages NUMBER]
The serve command uses BRAVE_SEARCH_API_KEY for scoped web discovery, fetches only
allowlisted official pages, and optionally falls back to a local index. Expo-provider
searches use Expo's public documentation index. Its fetch_platform_doc tool can fetch
one exact allowlisted documentation URL without a search key and return focused,
section, or bounded-document extracted context. The update command is
an optional offline crawler for operator-managed fallback indexes.
The serve command uses BRAVE_SEARCH_API_KEY for scoped web discovery and fetches only
allowlisted official pages. Expo-provider searches use Expo's public documentation
index. Its fetch_platform_doc tool can fetch one exact allowlisted documentation URL
without a search key and return focused, section, or bounded-document extracted
context.
`);
}
@@ -48,16 +40,10 @@ async function main() {
printHelp();
return;
}
const { values } = parseArgs({
args: rest,
options: {
index: { type: "string" },
},
strict: true,
});
const indexPath = values.index ?? process.env.REVIEW_RESEARCH_INDEX_PATH;
if (rest.length > 0) {
throw new Error(`Unexpected argument: ${rest[0]}`);
}
await runStdioServer({
...(indexPath ? { indexPath } : {}),
...(process.env.REVIEW_RESEARCH_AUDIT_PATH
? { auditPath: process.env.REVIEW_RESEARCH_AUDIT_PATH }
: {}),
@@ -71,43 +57,6 @@ async function main() {
return;
}
if (command === "update") {
if (rest.includes("--help") || rest.includes("-h")) {
printHelp();
return;
}
const { values } = parseArgs({
args: rest,
options: {
config: { type: "string" },
output: { type: "string" },
platform: { type: "string", multiple: true },
"max-pages": { type: "string" },
},
strict: true,
});
const invalidPlatform = values.platform?.find(
(platform) => !PLATFORMS.includes(platform as Platform),
);
if (invalidPlatform) {
throw new Error(`Unknown platform: ${invalidPlatform}`);
}
const maxPages = values["max-pages"] ? Number(values["max-pages"]) : undefined;
if (maxPages !== undefined && (!Number.isInteger(maxPages) || maxPages < 1)) {
throw new Error("--max-pages must be a positive integer");
}
const { updateDocumentationIndex } = await import("./crawler.js");
const result = await updateDocumentationIndex({
configPath: values.config ?? defaultConfigPath,
...(values.output ? { outputPath: values.output } : {}),
...(values.platform ? { platforms: values.platform as Platform[] } : {}),
...(maxPages ? { maxPagesPerProvider: maxPages } : {}),
});
process.stderr.write(`${JSON.stringify(result, null, 2)}\n`);
return;
}
throw new Error(`Unknown command: ${command}`);
}
-214
View File
@@ -1,214 +0,0 @@
import { readFile } from "node:fs/promises";
import path from "node:path";
import { z } from "zod";
import { extractAppleDocCPage } from "./apple-docc.js";
import { fetchAllowedContent } from "./fetch-document.js";
import { chunkDocument, extractDocumentationPage } from "./html.js";
import { extractMarkdownDocumentationPage } from "./markdown.js";
import { getProvider, resolveAllowedUrl, type DocumentationProvider } from "./providers.js";
import { buildSearchIndex, writeSearchIndex } from "./search-index.js";
import { extractYouTrackIssue } from "./youtrack.js";
import {
PLATFORMS,
PROVIDERS,
SOURCE_KINDS,
type DiscoveredDocument,
type IndexedChunk,
type Platform,
type ProviderId,
type SourceDefinition,
type SourcesConfig,
} from "./types.js";
const sourcesConfigSchema = z.object({
output: z.string().min(1),
crawl: z.object({
maxPagesPerProvider: z.number().int().min(1).max(100_000),
maxDepth: z.number().int().min(0).max(20),
delayMs: z.number().int().min(0).max(60_000),
timeoutMs: z.number().int().min(100).max(120_000),
maxResponseBytes: z.number().int().min(1_024).max(20_000_000),
}),
sources: z
.array(
z.object({
provider: z.enum(PROVIDERS),
sourceKind: z.enum(SOURCE_KINDS),
seedUrls: z.array(z.string().url()).min(1),
maxPages: z.number().int().min(1).max(100_000).optional(),
maxDepth: z.number().int().min(0).max(20).optional(),
}),
)
.min(1),
});
interface CrawlItem {
url: URL;
depth: number;
}
interface CrawlResult {
provider: ProviderId;
platform: Platform;
documents: DiscoveredDocument[];
errors: string[];
}
export interface UpdateOptions {
configPath: string;
outputPath?: string;
platforms?: Platform[];
maxPagesPerProvider?: number;
}
function sleep(milliseconds: number) {
return new Promise((resolve) => setTimeout(resolve, milliseconds));
}
async function crawlProvider(
provider: DocumentationProvider,
source: SourceDefinition,
limits: SourcesConfig["crawl"],
): Promise<CrawlResult> {
const queue: CrawlItem[] = [];
const errors: string[] = [];
for (const seedUrl of source.seedUrls) {
try {
queue.push({ url: resolveAllowedUrl(provider, seedUrl), depth: 0 });
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
errors.push(`${seedUrl}: ${message}`);
}
}
const queued = new Set(queue.map((item) => item.url.href));
const visited = new Set<string>();
const documents: DiscoveredDocument[] = [];
while (queue.length > 0 && visited.size < limits.maxPagesPerProvider) {
const item = queue.shift();
if (!item || visited.has(item.url.href)) continue;
visited.add(item.url.href);
try {
const content = await fetchAllowedContent(provider, item.url, limits);
const sourceMetadata = {
provider: provider.id,
sourceKind: source.sourceKind,
};
const format = provider.responseFormat(item.url);
const page =
format === "docc-json"
? extractAppleDocCPage(content, item.url.href, sourceMetadata)
: format === "markdown"
? extractMarkdownDocumentationPage(
content,
item.url.href,
provider.platform,
sourceMetadata,
)
: format === "youtrack-json"
? extractYouTrackIssue(content, item.url.href, sourceMetadata)
: extractDocumentationPage(content, item.url.href, provider.platform, sourceMetadata);
if (page) {
documents.push(page.document);
if (item.depth < limits.maxDepth) {
for (const href of page.links) {
try {
const nextUrl = resolveAllowedUrl(provider, href, item.url.href);
if (!queued.has(nextUrl.href) && !visited.has(nextUrl.href)) {
queued.add(nextUrl.href);
queue.push({ url: nextUrl, depth: item.depth + 1 });
}
} catch {
// Off-allowlist and malformed links are intentionally ignored.
}
}
}
}
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
errors.push(`${item.url.href}: ${message}`);
}
if (limits.delayMs > 0 && queue.length > 0) {
await sleep(limits.delayMs);
}
}
return { provider: provider.id, platform: provider.platform, documents, errors };
}
export async function readSourcesConfig(configPath: string): Promise<SourcesConfig> {
return sourcesConfigSchema.parse(JSON.parse(await readFile(configPath, "utf8")));
}
export function resolveIndexOutputPath(
configPath: string,
configuredOutput: string,
outputPath?: string,
): string {
if (outputPath) return path.resolve(outputPath);
return path.resolve(path.dirname(path.resolve(configPath)), configuredOutput);
}
export async function updateDocumentationIndex(options: UpdateOptions) {
const absoluteConfigPath = path.resolve(options.configPath);
const config = await readSourcesConfig(absoluteConfigPath);
const selectedPlatforms = new Set(options.platforms ?? PLATFORMS);
const limits = {
...config.crawl,
...(options.maxPagesPerProvider ? { maxPagesPerProvider: options.maxPagesPerProvider } : {}),
};
const selectedSources = config.sources.filter((source) =>
selectedPlatforms.has(getProvider(source.provider).platform),
);
if (selectedSources.length === 0) {
throw new Error("No configured sources matched the selected platforms");
}
const crawlResults = await Promise.all(
selectedSources.map((source) => {
const sourceLimits = {
...limits,
maxPagesPerProvider: Math.min(
limits.maxPagesPerProvider,
source.maxPages ?? limits.maxPagesPerProvider,
),
maxDepth: Math.min(limits.maxDepth, source.maxDepth ?? limits.maxDepth),
};
return crawlProvider(getProvider(source.provider), source, sourceLimits);
}),
);
const indexedAt = new Date().toISOString();
const documents = crawlResults.flatMap((result) => result.documents);
const chunks: IndexedChunk[] = documents.flatMap((document) =>
chunkDocument(document, indexedAt),
);
if (chunks.length === 0) {
const details = crawlResults
.flatMap((result) => result.errors)
.slice(0, 10)
.join("\n");
throw new Error(`Index update produced no searchable content${details ? `:\n${details}` : ""}`);
}
const outputPath = resolveIndexOutputPath(absoluteConfigPath, config.output, options.outputPath);
const index = buildSearchIndex(chunks, documents.length, indexedAt);
await writeSearchIndex(outputPath, index.serialized);
return {
outputPath,
documentCount: documents.length,
chunkCount: chunks.length,
providers: crawlResults.map((result) => ({
provider: result.provider,
platform: result.platform,
documentCount: result.documents.length,
errors: result.errors,
})),
};
}
+1 -1
View File
@@ -100,7 +100,7 @@ function rankedAnchor(
query?: string,
): SearchResult | undefined {
if (!query?.trim()) return chunks[0] ? withScore(chunks[0]) : undefined;
const index = buildSearchIndex(chunks, 1);
const index = buildSearchIndex(chunks);
return (
searchDocumentation(index, query, {
platform: document.platform,
+9 -6
View File
@@ -7,14 +7,17 @@ import {
type DocumentationProvider,
} from "./providers.js";
import { readBodyWithLimit } from "./response.js";
import type { DiscoveredDocument, ProviderId, SourceKind, SourcesConfig } from "./types.js";
import type { DiscoveredDocument, ProviderId, SourceKind } from "./types.js";
import { extractYouTrackIssue } from "./youtrack.js";
import type { FetchImplementation } from "./brave-search.js";
export const onDemandFetchLimits: SourcesConfig["crawl"] = {
maxPagesPerProvider: 10,
maxDepth: 0,
delayMs: 0,
/** Per-attempt bounds for one documentation request. */
export interface DocumentFetchLimits {
timeoutMs: number;
maxResponseBytes: number;
}
export const onDemandFetchLimits: DocumentFetchLimits = {
timeoutMs: 10_000,
maxResponseBytes: 5_000_000,
};
@@ -22,7 +25,7 @@ export const onDemandFetchLimits: SourcesConfig["crawl"] = {
export async function fetchAllowedContent(
provider: DocumentationProvider,
documentUrl: URL,
limits: Pick<SourcesConfig["crawl"], "timeoutMs" | "maxResponseBytes">,
limits: DocumentFetchLimits,
fetchImplementation: FetchImplementation = fetch,
): Promise<string> {
let currentUrl = resolveAllowedRequestUrl(provider, provider.requestUrl(documentUrl).href);
+1 -1
View File
@@ -73,7 +73,7 @@ async function loadOkHttpSearchIndex(
}
});
const chunks = documents.flatMap((document) => chunkDocument(document, indexedAt));
return buildSearchIndex(chunks, documents.length, indexedAt);
return buildSearchIndex(chunks);
}
function cachedOkHttpSearchIndex(
-9
View File
@@ -1,9 +0,0 @@
import { fileURLToPath } from "node:url";
export const packageRoot = fileURLToPath(new URL("../..", import.meta.url));
export const defaultConfigPath = fileURLToPath(
new URL("../../research/sources.json", import.meta.url),
);
export const defaultIndexPath = fileURLToPath(
new URL("../../research/data/docs-index.json", import.meta.url),
);
+1 -1
View File
@@ -139,7 +139,7 @@ function bestPassage(
if (chunks.length === 0) {
return { passage: document.body.slice(0, 1_400), relevance: 0 };
}
const index = buildSearchIndex(chunks, 1, indexedAt);
const index = buildSearchIndex(chunks);
const result = searchDocumentation(index, query, {
platform: document.platform,
providers: document.provider ? [document.provider] : undefined,
+7 -44
View File
@@ -1,6 +1,3 @@
import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
import path from "node:path";
import MiniSearch, { type SearchOptions } from "minisearch";
import type {
@@ -10,7 +7,6 @@ import type {
PlatformFilter,
ProviderId,
SearchResult,
SerializedSearchIndex,
SourceKind,
} from "./types.js";
@@ -35,51 +31,18 @@ const miniSearchOptions = {
};
export interface BuiltSearchIndex {
serialized: SerializedSearchIndex;
miniSearch: MiniSearch<IndexedChunk>;
}
export function buildSearchIndex(
chunks: IndexedChunk[],
documentCount: number,
generatedAt = new Date().toISOString(),
): BuiltSearchIndex {
/**
* Build a throwaway in-memory index over one fetch's chunks, purely to rank them.
* Nothing is serialized or persisted: since the offline index was removed, every
* caller builds this per request and discards it.
*/
export function buildSearchIndex(chunks: IndexedChunk[]): BuiltSearchIndex {
const miniSearch = new MiniSearch<IndexedChunk>(miniSearchOptions);
miniSearch.addAll(chunks);
const providers = [
...new Set(chunks.map((chunk) => chunk.provider ?? chunk.platform)),
].sort() as ProviderId[];
return {
miniSearch,
serialized: {
schemaVersion: 1,
generatedAt,
documentCount,
chunkCount: chunks.length,
providers,
searchIndex: miniSearch.toJSON(),
},
};
}
export async function writeSearchIndex(filePath: string, index: SerializedSearchIndex) {
await mkdir(path.dirname(filePath), { recursive: true });
const temporaryPath = `${filePath}.tmp-${process.pid}`;
await writeFile(temporaryPath, `${JSON.stringify(index)}\n`, { mode: 0o644 });
await rename(temporaryPath, filePath);
}
export async function loadSearchIndex(filePath: string): Promise<BuiltSearchIndex> {
const serialized = JSON.parse(await readFile(filePath, "utf8")) as SerializedSearchIndex;
if (serialized.schemaVersion !== 1 || typeof serialized.searchIndex !== "object") {
throw new Error(`Unsupported or invalid search index at ${filePath}`);
}
const miniSearch = MiniSearch.loadJSON<IndexedChunk>(
JSON.stringify(serialized.searchIndex),
miniSearchOptions,
);
return { serialized, miniSearch };
return { miniSearch };
}
function searchOptions(combineWith: "AND" | "OR", exact = false): SearchOptions {
+2 -20
View File
@@ -15,7 +15,6 @@ import { searchExpoAlgolia } from "./expo-algolia.js";
import { searchOkHttpDocumentation } from "./okhttp-search.js";
import { getProvider, resolveAllowedUrl } from "./providers.js";
import { searchRemoteDocumentation } from "./remote-search.js";
import { loadSearchIndex, searchDocumentation } from "./search-index.js";
import { LANGUAGES, PROVIDERS, SOURCE_KINDS } from "./types.js";
import type { PlatformFilter, ProviderId, SearchResult } from "./types.js";
@@ -29,7 +28,6 @@ const providerGuidance =
"Provider map: apple=Apple SDK APIs and Human Interface Guidelines; apple-releases=Xcode and Apple platform release notes; swift-evolution=Swift Evolution proposals; sdwebimage=SDWebImage APIs and caching/loading behavior; android=Android, Jetpack, Compose, and Google Play services APIs; android-releases=Android platform releases and behavior changes; media3=Jetpack Media3; glide=Glide; okhttp=OkHttp; kotlin-coroutines=Kotlin coroutines; gradle=Gradle; agp=Android Gradle Plugin; jetbrains-issues=JetBrains YouTrack context; expo=Expo documentation; react-native=React Native core; react-native-reanimated=Reanimated; react-native-gesture-handler=Gesture Handler; react-native-screens=Screens; react-native-worklets=Worklets. Native source retains platform context: use apple/android for OS contracts and add the dependency provider for dependency-owned behavior; an Expo package path does not make a native API an Expo-docs query. Issue-tracker results are context, not API contracts.";
export interface DocumentationServerOptions {
indexPath?: string;
braveApiKey?: string;
fetchImplementation?: FetchImplementation;
auditPath?: string;
@@ -47,7 +45,6 @@ function defaultProviders(platform: PlatformFilter): ProviderId[] {
}
export async function createDocumentationServer(options: DocumentationServerOptions = {}) {
const index = options.indexPath ? await loadSearchIndex(options.indexPath) : undefined;
const maxCalls = Math.min(20, Math.max(1, options.maxCalls ?? 8));
const maxResultsPerCall = Math.min(3, Math.max(1, options.maxResultsPerCall ?? 3));
const timeoutMs = Math.min(60_000, Math.max(1_000, options.timeoutMs ?? 30_000));
@@ -62,7 +59,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
"search_platform_docs",
{
title: "Search official platform documentation",
description: `Search official platform, dependency, build-tool, and release documentation. Discovery uses scoped web search (or Expo's public documentation search), then fetches only allowlisted official pages; an optional local index is fallback evidence. Returns short passages with canonical source URLs. ${providerGuidance} ${queryGuidance}`,
description: `Search official platform, dependency, build-tool, and release documentation. Discovery uses scoped web search (or Expo's public documentation search), then fetches only allowlisted official pages. Returns short passages with canonical source URLs. ${providerGuidance} ${queryGuidance}`,
inputSchema: {
platform: z
.enum(["apple", "android", "react-native", "all"])
@@ -123,15 +120,6 @@ export async function createDocumentationServer(options: DocumentationServerOpti
// One deadline and one request ledger for everything this call issues.
const network = createResearchNetwork(baseFetch, timeoutMs);
try {
const localResults = index
? searchDocumentation(index, sanitizedQuery, {
platform,
limit: boundedLimit,
providers: selectedProviders,
...(sourceKinds ? { sourceKinds } : {}),
...(language ? { language } : {}),
})
: [];
const warnings: string[] = [];
const remoteResults: SearchResult[] = [];
const perProviderLimit = Math.max(1, Math.ceil(boundedLimit / selectedProviders.length));
@@ -226,7 +214,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
}
const seen = new Set<string>();
const results = [...remoteResults, ...localResults]
const results = remoteResults
.filter((result) => {
if (!result.provider) return false;
try {
@@ -249,12 +237,6 @@ export async function createDocumentationServer(options: DocumentationServerOpti
// What this single budget unit actually cost.
network: network_,
expoSearch: selectedProviders.includes("expo"),
localIndex: index
? {
generatedAt: index.serialized.generatedAt,
providers: index.serialized.providers,
}
: null,
},
...(uniqueWarnings.length > 0 ? { warnings: uniqueWarnings } : {}),
results,
-22
View File
@@ -82,25 +82,3 @@ export interface SerializedSearchIndex {
providers: ProviderId[];
searchIndex: unknown;
}
export interface CrawlLimits {
maxPagesPerProvider: number;
maxDepth: number;
delayMs: number;
timeoutMs: number;
maxResponseBytes: number;
}
export interface SourceDefinition {
provider: ProviderId;
sourceKind: SourceKind;
seedUrls: string[];
maxPages?: number;
maxDepth?: number;
}
export interface SourcesConfig {
output: string;
crawl: CrawlLimits;
sources: SourceDefinition[];
}
+2 -1
View File
@@ -39,7 +39,8 @@
//
// maxQueries bounds MCP CALLS, not requests — one search can issue a discovery
// request per provider plus a page fetch per candidate. timeoutMs is the MCP's own
// end-to-end deadline per call. indexPath remains an optional offline fallback.
// end-to-end deadline per call. Every passage is fetched live from the allowlist;
// there is no offline index.
// "research": {
// "enabled": true,
// "maxQueries": 8,