mirror of
https://github.com/expo/code-review-cli.git
synced 2026-11-14 17:49:25 -10:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3ac5186893 | ||
|
|
a87b5085eb | ||
|
|
8300f2b1fb |
No files matched your search
@@ -20,14 +20,15 @@
|
||||
// *.min.js, *.map, __snapshots__/*.snap, @generated markers).
|
||||
"noise": { "additionalIgnores": [] },
|
||||
|
||||
// Trusted host-side documentation research. Only bounded API identifiers leave
|
||||
// the runner; scoped web results are independently checked against fixed provider
|
||||
// allowlists before ECR fetches and fences official documentation passages.
|
||||
// Reviewer-invoked documentation research. The reviewing model chooses the
|
||||
// outbound query terms and URLs; they are shape-checked and bounded, not
|
||||
// confidentiality-checked. Results are independently re-checked against fixed
|
||||
// provider allowlists before ECR fetches and fences official passages.
|
||||
"research": {
|
||||
"enabled": true,
|
||||
"maxQueries": 8,
|
||||
"resultsPerQuery": 2,
|
||||
"timeoutMs": 15000
|
||||
"timeoutMs": 30000
|
||||
},
|
||||
|
||||
// Large diffs are split into focused chunks by changed-line count, plus a
|
||||
|
||||
@@ -9,4 +9,3 @@ build/
|
||||
|
||||
# Claude Code local worktrees
|
||||
.claude/worktrees/
|
||||
research/data/
|
||||
@@ -190,7 +190,7 @@ Enable it only in the root config, which CI loads from the PR's trusted base:
|
||||
"enabled": true,
|
||||
"maxQueries": 8,
|
||||
"resultsPerQuery": 2,
|
||||
"timeoutMs": 15000
|
||||
"timeoutMs": 30000
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -205,13 +205,26 @@ the current absolute Node executable, so a PR-owned `PATH` entry cannot replace
|
||||
either component. Each review gets an owner-only temporary MCP config and append-only
|
||||
audit. Claude receives that explicit config under `--strict-mcp-config`, with project
|
||||
settings and slash commands disabled; OpenCode receives the same fixed local command.
|
||||
The Brave credential is passed to the MCP child, not the model process.
|
||||
|
||||
The MCP is the outbound security boundary. Search queries are normalized before
|
||||
That command is a wrapper, not the server. Both engines merge a configured MCP `env`
|
||||
onto their own environment instead of replacing it, so the declared block alone cannot
|
||||
bound the child — the engine's model credential reaches it, and OpenCode additionally
|
||||
passes down the runner's whole ambient environment. The wrapper therefore rebuilds the
|
||||
environment from an explicit allowlist before starting the server, and loads no parser
|
||||
and opens no socket of its own. The server process sees the search key, the call
|
||||
bounds, and locale/proxy settings; it never sees a model credential. The Brave
|
||||
credential travels the same way and is never added to the model process env.
|
||||
|
||||
The MCP bounds the shape, host, and volume of outbound requests. It is not a
|
||||
confidentiality boundary: the reviewing model chooses the query terms and URLs, and a
|
||||
low-entropy identifier can carry repository-derived data past every check below.
|
||||
Enable research only where repository-derived terms may be shared with Brave and the
|
||||
documentation providers. Search queries are normalized before
|
||||
logging or networking: quoted literals, URLs, email addresses, paths, prose stop
|
||||
words, overlong/high-entropy tokens, and unsupported punctuation are removed;
|
||||
credential-shaped or secret-labeled input fails closed. The remaining query must be
|
||||
at most eight short tokens and contain an API-like symbol. Direct URLs must use plain
|
||||
credential-shaped or secret-labeled input fails closed. The remaining query must be at
|
||||
most eight short tokens and either contain an API-like symbol or be a short multi-word
|
||||
lowercase concept phrase. Direct URLs must use plain
|
||||
HTTPS with no credentials, port, query string, or fragment; suspicious/high-entropy
|
||||
path segments fail closed. The fixed provider host/path allowlist and redirect,
|
||||
response-size, content-type, and timeout checks still apply after that first gate.
|
||||
@@ -219,6 +232,17 @@ These deterministic checks greatly reduce accidental exfiltration; they are not
|
||||
proof that every low-entropy string is harmless, so reviewer prompts also forbid
|
||||
sending repository text and the review-wide MCP budget defaults to eight calls.
|
||||
|
||||
`maxQueries` bounds MCP calls, not network requests. One search selects up to four
|
||||
providers, and each issues its own discovery request plus a page fetch per candidate,
|
||||
so eight calls can mean roughly thirty discovery requests and over a hundred page
|
||||
downloads. Every call therefore reports its own ledger — discovery requests, page
|
||||
fetches, redirect hops, total HTTP requests, and elapsed time — and the review log and
|
||||
Actions summary report the totals. `timeoutMs` is the MCP's own end-to-end deadline for
|
||||
one call, enforced by the server across discovery, redirects, retrieval, and
|
||||
extraction; a call that hits it returns what it already has rather than failing. It has
|
||||
to live there because OpenCode's `timeout` bounds only tool discovery and Claude
|
||||
provides no per-call timeout at all.
|
||||
|
||||
For non-Expo providers, discovery sends a fixed, provider-owned `site:` scope plus
|
||||
the bounded query to Brave's fixed Web Search endpoint. Search snippets and titles
|
||||
are never treated as evidence. ECR independently rejects off-allowlist result URLs,
|
||||
@@ -276,8 +300,11 @@ same audit trail in the step summary, while `.runs/reviews.jsonl` keeps the quer
|
||||
plus bounded returned passages for short-lived operational inspection. Reviewers
|
||||
are instructed to attach `sources` only when documentation materially supports a
|
||||
finding. ECR accepts only exact URLs returned during that review, restores canonical
|
||||
titles, carries citations through coordination, and renders them below the finding;
|
||||
invented or unrelated citations are dropped.
|
||||
titles, carries citations through coordination, and renders them below the finding.
|
||||
A citation to a URL this review never retrieved is dropped outright. Relatedness is a
|
||||
separate, weaker guarantee: a cited finding is escalated to the verifier with the
|
||||
audited passage inline, which judges whether that passage actually supports the claim
|
||||
and strips the citation when it does not.
|
||||
|
||||
Reviewers also emit a bounded `researchDecisions` record only when documentation
|
||||
materially confirms a finding candidate or proves one safe. ECR grounds those records
|
||||
@@ -298,14 +325,11 @@ downloads the fixed official `lysine.dev` static search index, validates and ind
|
||||
it in memory once per MCP process, and rejects any entry outside the existing OkHttp
|
||||
allowlist. Brave remains a fallback if that official index is unavailable.
|
||||
|
||||
An absolute `research.indexPath` remains available as an optional local fallback.
|
||||
The MCP and its trusted updater ship with ECR: build that fallback from this
|
||||
repository with `bun run research:update`, or from an installed package with
|
||||
`review-research-mcp update`. This is operator/scheduled offline tooling, not a
|
||||
step to run before each review. The built-in seed catalog lives in
|
||||
`research/sources.json`; `seedUrls` are deterministic starting pages for the
|
||||
bounded crawler, and extraction/indexing use no LLM. Installations requiring a fully
|
||||
offline review can supply a separately built, verified index and omit the Brave key.
|
||||
There is no offline index. Every passage a review sees is fetched live from the
|
||||
provider allowlist during that review, so evidence is never served from a local
|
||||
artifact whose contents ECR cannot vouch for. The crawler, its seed catalog, and
|
||||
`research.indexPath` were removed once live discovery replaced them; a config still
|
||||
naming `indexPath` fails to parse rather than silently ignoring it.
|
||||
Installation-specific provider configuration is intentionally
|
||||
deferred: when added, it should follow the trusted root-config model used for agents
|
||||
without permitting PR-controlled URLs, commands, or executable parsers. Expo skills
|
||||
|
||||
+209
@@ -55,6 +55,215 @@ PRs** (do these first, in this order):
|
||||
isn't on `main` yet (nothing to build from base until this merges). See §3.9.
|
||||
5. **Inline PR comments** (§1).
|
||||
|
||||
## Next up (2026-08-08)
|
||||
|
||||
The biggest items on deck. Each has substrate already in the repo, so the work is
|
||||
mostly extending or simplifying something that exists — not greenfield.
|
||||
|
||||
### A. Cross-run state reuse — pick up where the last review left off
|
||||
|
||||
Today a re-review is stateless three times over. `review-cache.ts` (LLP
|
||||
0005#review-result-cache) caches only a **whole** review result, keyed by a hash
|
||||
of the entire input, so one changed byte anywhere discards everything. Every
|
||||
Claude pass runs with `--no-session-persistence` (`claude-code.ts:343`), so
|
||||
there is no session to resume even in principle. And — the cheapest gap to close
|
||||
— **no prior-run state reaches the prompt at all**: prior findings live in the
|
||||
comment's embedded state and are used only afterward, for suppression and
|
||||
fingerprint matching. The model is never told it has seen this PR before.
|
||||
|
||||
(The research half of this is already fixed: removing the offline index let
|
||||
`ci.ts` drop `!config.research.enabled` from `cacheAllowed`, so researched
|
||||
reviews are cacheable again.)
|
||||
|
||||
Two independent levers, worth not conflating:
|
||||
|
||||
1. **Finer-grained result reuse.** Move below whole-review granularity so a
|
||||
push that touches one file doesn't re-run every pass. §4 item 4 already
|
||||
documents why this is hard and should be read first: chunk boundaries shift
|
||||
under small edits (one modified file reshuffles every downstream chunk →
|
||||
every hash misses), and findings depend on more than their own chunk, so
|
||||
reusing an "unchanged" chunk's verdict can miss a break introduced by a
|
||||
sibling change. The safer variant named there — reuse only when the *rest*
|
||||
of the PR is unchanged (rebase / comment-only push) — reuses much less but
|
||||
doesn't trade recall, and is probably the place to start.
|
||||
2. **Prior-review context in the prompt (do this first).** Give the reviewer a
|
||||
host-built block naming what it reported last time and what changed since —
|
||||
built from the embedded comment state we already carry, so it is trusted,
|
||||
structured, small, and bounded like every other prompt input.
|
||||
|
||||
The data is already there and needs no new storage: `ReviewState`
|
||||
(`render.ts:518`) round-trips the whole `CoordinatorOutput` — every prior
|
||||
finding with file, line, severity, category and rationale — plus `dismissed`,
|
||||
`feedback`, `pins`, and the `inputHash`, through the PR comment on every run.
|
||||
It is decoded on each run and used only afterward. Note this is *richer* than
|
||||
a resumed transcript would be: it knows which findings a maintainer dismissed
|
||||
and which ones the author answered. Note also the precedent — cross-run state
|
||||
in this tool already travels in the comment, not in CI storage. This is what
|
||||
actually delivers "this is a re-review, here is what moved"; it needs no
|
||||
persisted engine state, survives a CLI upgrade, and works on any harness
|
||||
(see C). Findings must still be re-derived from the current code — prior
|
||||
findings are context, never a verdict to restate.
|
||||
|
||||
3. **Engine session reuse.** Mechanically confirmed to work (2026-08-08, claude
|
||||
2.1.226): `--session-id <uuid>` *pre-assigns* an id, so ids can be derived
|
||||
deterministically rather than scraped from stream-json; `--resume <id>` works
|
||||
under `-p` and restores full history **across a different cwd**, so the
|
||||
per-run `mkdtemp` head worktree is not a blocker; `CLAUDE_CONFIG_DIR` (already
|
||||
in `CHILD_ENV_ALLOWLIST`) relocates transcripts somewhere cacheable. A warm
|
||||
resume is much cheaper than a fresh pass — in a toy 4-turn measurement,
|
||||
37k cached vs 18.5k re-created tokens.
|
||||
|
||||
Four things to settle before building it:
|
||||
|
||||
- **The economics invert when the cache is cold.** Prompt-cache TTL is 1 hour
|
||||
on a subscription, 5 minutes on an API key. Warm, resume is a large win.
|
||||
Cold, resume reprocesses the *entire* transcript (~100k+ tokens for a real
|
||||
pass, per §4 item 1) where a fresh pass pays ~10–18k of new input. That is
|
||||
roughly a 5× penalty on exactly the re-review that arrives the next day.
|
||||
A resume that falls back to a fresh session past the TTL is strictly better
|
||||
than either alone — and needs only a stored timestamp.
|
||||
- **A session is per agent×chunk, and chunks are not stable across pushes**
|
||||
(`review.ts:546`, and §4 item 4). Only single-chunk reviews — a diff under
|
||||
`chunk.maxChangedLines` — have a session unit that survives a push at all.
|
||||
For chunked PRs, "resume the session for chunk 3" is meaningless once
|
||||
chunk 3 covers different files.
|
||||
- **Injection gains persistence.** Today an injection can only affect the run
|
||||
whose diff carries it. A resumed transcript lets text pushed in commit 1
|
||||
shape the review of commit 5, after that text is gone. This is the real
|
||||
security cost, and it argues for capping resume depth/age, re-running the
|
||||
verifier statelessly, and never resuming across a base change.
|
||||
- **The transcript format is internal and version-coupled.** Upstream states
|
||||
it "changes between versions", and resuming after a CLI upgrade reprocesses
|
||||
everything with no cache hits — and we pin and bump that CLI. The cache key
|
||||
must include the CLI version, which means every bump discards every session.
|
||||
|
||||
If caching transcripts in the GitHub Actions cache: scope is favorable — a
|
||||
PR's cache is written to `refs/pull/N/merge` and restorable **only by re-runs
|
||||
of that same PR**, and a PR cannot write the default-branch scope, so there is
|
||||
no cross-PR path. Cache **only** the `projects/<dir>/<id>.jsonl` files, never
|
||||
`$CLAUDE_CONFIG_DIR` itself — that directory also holds credentials.
|
||||
|
||||
Decide the staleness budget before implementing. The tool's value proposition is
|
||||
recall, and every reuse strategy above buys latency and quota by accepting some
|
||||
probability of a stale conclusion.
|
||||
|
||||
### B. A feedback loop that tunes the reviewer
|
||||
|
||||
Author feedback shipped 2026-08-01 (LLP 0011) and already produces the signal:
|
||||
replies matched deterministically to findings, enum-only records embedded in the
|
||||
comment, adjudication verdicts, and `ecr feedback` mining a repo's whole PR
|
||||
history into totals, reply rates, category/severity/agent breakdowns, and
|
||||
"repeat offenders" — findings whose title recurs across PRs and draws a reply
|
||||
every time. Adjacent signals already exist too: verifier keep/drop verdicts,
|
||||
suppression, and the research usefulness counters.
|
||||
|
||||
What's missing is the loop. None of it feeds back into the prompts or config —
|
||||
`ecr feedback` prints a report a human reads and acts on. The item is to close
|
||||
that: aggregate the signal into a durable, reviewable tuning artifact and let a
|
||||
run consume it.
|
||||
|
||||
Constraints to design against, all of which fall out of the existing trust
|
||||
model rather than being new:
|
||||
|
||||
- **A PR author must not be able to tune the reviewer.** Feedback is authored by
|
||||
untrusted parties on untrusted PRs. Anything derived from it must land in the
|
||||
trusted base and go through review like any other config change — never be
|
||||
applied live from the PR that produced it.
|
||||
- **Prefer a reviewable artifact over auto-edited prompts.** A checked-in tuning
|
||||
file a human approves keeps the change auditable and revertible; a model
|
||||
silently rewriting `agents/*.md` does not.
|
||||
- **Guard against the obvious degenerate outcome:** "authors disliked this
|
||||
finding" is a signal about friction, not about correctness. Optimizing it
|
||||
directly teaches the reviewer to stop reporting true positives people find
|
||||
annoying. The hard floors from LLP 0011 (`critical`/`secrets`/`security` can
|
||||
never be cleared) are the precedent for what must stay out of reach.
|
||||
- Start read-only: report what a tuning change *would* do before letting it
|
||||
change anything.
|
||||
|
||||
### C. Drop OpenCode, and make the harness seam real while doing it
|
||||
|
||||
Claude is the only harness we intend to support for now. Removing OpenCode is
|
||||
mostly deletion, but the *point* of the item is the part that isn't: today the
|
||||
engine abstraction is owned by one of its implementations, so "add a harness"
|
||||
means editing the Claude module. The goal is that adding Codex, Gemini CLI, or a
|
||||
direct-API harness later is a new file plus a registry entry — not a refactor.
|
||||
|
||||
What the seam looks like now:
|
||||
|
||||
- `Engine`, `engineForModel`, and `buildEngineMap` all live in
|
||||
`src/core/claude-code.ts`. The interface is hosted inside an implementation.
|
||||
- `engineForModel` hardcodes the whole routing rule as
|
||||
`provider === "anthropic" ? claude-code : opencode`, so **OpenCode is the
|
||||
silent fallback for every non-Anthropic model**. `engineOf[agent] ?? "opencode"`
|
||||
appears at more than one dispatch site.
|
||||
- Engine-specific knowledge leaks outward: `auth.ts` writes an OpenCode
|
||||
`auth.json` under XDG, `setup-auth.ts` knows that path, `throttle.ts` parses
|
||||
OpenCode's own log file for rate-limit evidence, and `doctor.ts` seeds its
|
||||
engine set with `["opencode"]`.
|
||||
|
||||
So the work is:
|
||||
|
||||
1. **Extract the interface before deleting anything.** A `ReviewEngine` a harness
|
||||
implements — start/stop, run one pass with (agent, model, system, prompt,
|
||||
tools, schema), report usage/model-that-answered, surface throttle evidence —
|
||||
living in its own module, not in a harness. `claude-code.ts` becomes one
|
||||
implementation of it.
|
||||
2. **Make model→harness a registry lookup, not a conditional.** A harness
|
||||
declares the model prefixes it serves. An unmatched model must fail fast and
|
||||
name the fix, never fall through to a default harness.
|
||||
3. **Give harness-specific concerns a home in the harness.** Credential
|
||||
materialization, CLI resolution, throttle/log parsing, and doctor probes are
|
||||
per-harness behavior behind the same interface, not `if` branches in shared
|
||||
code.
|
||||
4. **Then delete OpenCode**: `opencode.ts`, the `opencode-ai`/`@opencode-ai/sdk`
|
||||
dependencies, its auth.json path, its log parsing, and the mixed-engine
|
||||
machinery in `review.ts` that exists only because one run could drive two
|
||||
harnesses at once.
|
||||
|
||||
Two things to preserve rather than lose in the simplification. Mixed-engine
|
||||
support is what makes step 2 honest — the abstraction should still permit two
|
||||
harnesses in one run even while only one exists, because that is the case that
|
||||
would otherwise be designed out and need rebuilding. And the per-agent model
|
||||
override stays: harness choice follows from the model, so it stays a property of
|
||||
an agent, never a global run mode.
|
||||
|
||||
The immediate payoff beyond deletion is that the research MCP wiring, the tool
|
||||
allow/deny lists, and the session work in (A) each currently need a per-harness
|
||||
implementation, and today that means editing both harness modules in step.
|
||||
|
||||
### Considered and deferred: repo-configurable research sources
|
||||
|
||||
Moving the documentation allowlist into `config.jsonc` so each repo points
|
||||
research at its own stack. **Deferred 2026-08-08** — not the cost of building
|
||||
it, but what it trades away.
|
||||
|
||||
`src/research-mcp/providers.ts` holds the runtime host/path allowlist and is the
|
||||
outbound security boundary. Making it repo-owned reverses a stated invariant:
|
||||
"Root config supplies only enablement, bounds, and an optional absolute fallback
|
||||
index path—not an arbitrary command, endpoint, host, or argument vector" (LLP
|
||||
0013#one-package-two-binaries). A PR head still could not add a host — research
|
||||
config loads from the trusted base — but a repo admin could, and the 2026-08-07
|
||||
audit established that research is an outbound disclosure channel where the
|
||||
reviewing model chooses the query terms and URL path. So the feature widens who
|
||||
can receive repository-derived terms. Two things also make it more than a config
|
||||
change: retrieval is bespoke per provider (Apple DocC JSON, YouTrack JSON,
|
||||
raw-markdown rewrites), so a repo-declared source realistically gets the generic
|
||||
HTML path only; and the MCP tool schema is generated from the fixed list
|
||||
(`z.enum(PROVIDERS)`), so per-repo sources make the tool schema, prompt provider
|
||||
map, citation grounding, and audit provider field all vary by repo.
|
||||
|
||||
Revisit if adopters actually ask for internal documentation hosts. If so, the
|
||||
shape most likely to survive is repo sources *added to* the package catalog
|
||||
rather than replacing it, with exact hostname plus path prefix only — no pattern
|
||||
language, and no pluggable adapters.
|
||||
|
||||
Note for whoever picks this up: `research/sources.json` is **not** this
|
||||
allowlist. It is the seed catalog for the operator-only `update` crawler
|
||||
(`cli.ts` lazily imports `crawler.js` there and nowhere else); no review-time
|
||||
code path reads it. Editing it changes what an offline index build crawls, not
|
||||
what a review is allowed to fetch.
|
||||
|
||||
|
||||
## Recently shipped
|
||||
|
||||
- **Author feedback on findings (shipped 2026-08-01)** — a PR author's reply is
|
||||
|
||||
@@ -24,10 +24,20 @@ supplies only enablement, bounds, and an optional absolute fallback index path
|
||||
arbitrary command, endpoint, host, or argument vector.
|
||||
|
||||
The MCP remains a process boundary rather than an in-process import. Its stdout is
|
||||
the only result channel, its environment is an allowlist, and ECR caps calls and
|
||||
results. Each run uses an owner-only temporary MCP configuration and append-only audit.
|
||||
A tool failure remains local to research and never skips or weakens an ordinary review
|
||||
pass.
|
||||
the only result channel, and ECR caps calls, results, and per-call duration. Each run
|
||||
uses an owner-only temporary MCP configuration and append-only audit. A tool failure
|
||||
remains local to research and never skips or weakens an ordinary review pass.
|
||||
|
||||
The environment ECR declares in that configuration is a request, not a boundary. Both
|
||||
Claude Code and OpenCode merge a server's `env` onto the environment they already hold
|
||||
rather than replacing it, so the declared allowlist alone leaves the engine's model
|
||||
credential in the child — and under OpenCode the runner's entire ambient environment,
|
||||
because the SDK spawns its server with an unfiltered `process.env`. ECR therefore does
|
||||
not rely on the engines for this. The command in the configuration names a wrapper that
|
||||
rebuilds the environment from an explicit allowlist and then starts the real server.
|
||||
The wrapper is deliberately inert: it loads no parser and opens no socket, so the one
|
||||
process that still holds the engine's credentials is not the one that parses untrusted
|
||||
remote documents. Anything the engine merged in stops there.
|
||||
|
||||
Starting a review does not derive searches from the patch, prefetch documentation, or
|
||||
start a crawl. ECR only prepares the bounded MCP configuration and audit path. Reviewer
|
||||
@@ -41,9 +51,21 @@ identifier searches.
|
||||
`review-research-mcp serve` uses Brave Web Search as discovery for non-Expo
|
||||
providers. Each request combines the already-sanitized API/concept query with a fixed
|
||||
provider-owned `site:` scope. The endpoint is fixed, redirects are rejected, response
|
||||
size/time/hit count are capped, and only the fixed `BRAVE_SEARCH_API_KEY` credential
|
||||
is forwarded into the MCP child. Search titles, snippets, and ranking are untrusted
|
||||
discovery hints, never evidence.
|
||||
size/time/hit count are capped, and the fixed `BRAVE_SEARCH_API_KEY` is the only
|
||||
credential the wrapper admits into the server's environment. Search titles, snippets,
|
||||
and ranking are untrusted discovery hints, never evidence.
|
||||
|
||||
One reserved call is not one request. A search selects up to four providers, each of
|
||||
which issues its own discovery request and then downloads candidate pages until it has
|
||||
enough results, and every page may cost up to six round trips through the redirect
|
||||
chain. The call budget alone therefore understates outbound traffic by more than an
|
||||
order of magnitude, so each call carries a ledger — discovery requests, page fetches,
|
||||
redirect hops, total requests, elapsed time — recorded in the audit and summed for the
|
||||
review. The same seam carries one end-to-end deadline per call, because no engine can
|
||||
supply it: OpenCode's `mcp.timeout` bounds tool discovery rather than execution, and
|
||||
Claude passes no per-call timeout at all, leaving only a per-hop limit that multiplies
|
||||
instead of bounding. A call that reaches its deadline returns the evidence it already
|
||||
has rather than failing.
|
||||
|
||||
Every discovered URL must independently pass the provider's exact HTTPS host/path
|
||||
allowlist before a fetch begins. Redirects are handled manually and must remain on
|
||||
@@ -97,24 +119,24 @@ has the same response-size/schema/URL constraints, is cached only within the MCP
|
||||
process, and can never admit a URL outside the OkHttp provider allowlist. Brave is a
|
||||
fallback when that first-party index is unavailable.
|
||||
|
||||
`review-research-mcp update` is a separate, operator-invoked command for trusted
|
||||
scheduled jobs. The updater uses exact HTTPS host/path allowlists, manually
|
||||
revalidates redirects, checks response content types, and bounds response size,
|
||||
crawl depth, page count, timeout, and delay. The Expo crawl supplies a local fallback;
|
||||
precise Expo search uses the live query rather than pretending an empty-query result
|
||||
page is a complete Algolia index.
|
||||
There is no offline index, and no crawler to build one. Live discovery replaced the
|
||||
deterministic prefetch it was designed for, and the fallback it provided was never
|
||||
free: an index is an artifact whose bytes sit outside the trust boundary the rest of
|
||||
the system establishes. `research.indexPath` had to be an absolute path, so it
|
||||
resolved against the runner's filesystem rather than the materialized trusted base —
|
||||
the trusted-base mechanism protected the config that NAMED the index, never the
|
||||
contents at that path, and nothing carried a digest. Index passages were also the one
|
||||
evidence source returned unbounded, while every live source is length-capped.
|
||||
|
||||
The crawler is not part of the ordinary review workflow. An adopter requiring an
|
||||
offline fallback may run `update` in a separate scheduled secretless job from a
|
||||
pinned package and trusted checkout. That job must execute no PR code, receive no
|
||||
model credential, and publish a verified read-only index artifact.
|
||||
Removing it deletes that whole class of question rather than answering it. Every
|
||||
passage a review sees is now fetched during that review from an allowlisted host,
|
||||
bounded on entry. A config still naming `research.indexPath` fails to parse, so a
|
||||
stale setting surfaces as an error instead of being silently ignored.
|
||||
|
||||
The generated index is deliberately not published in the npm package. Documentation
|
||||
changes independently of ECR, and coupling a snapshot to every CLI install would make
|
||||
releases and invalidation expensive. If supplied, an absolute trusted-base
|
||||
`research.indexPath` is fallback evidence when remote discovery fails or misses.
|
||||
Research-enabled reviews bypass result-cache reuse because remote search,
|
||||
documentation, and optional index content can change without a config change.
|
||||
This also removes the reason researched reviews could not reuse a cached result.
|
||||
Reuse was disabled whenever research was enabled because evidence depended on mounted
|
||||
index contents that the cache key could not represent; with no mounted artifact, the
|
||||
ordinary input hash covers the whole review again.
|
||||
|
||||
## Query and Prompt Boundary
|
||||
|
||||
@@ -139,6 +161,15 @@ Media3, Glide, or OkHttp—when library behavior also matters. Repository owners
|
||||
not documentation ownership, so a `packages/expo-*` path does not replace Apple or
|
||||
Android with Expo documentation.
|
||||
|
||||
These checks bound the SHAPE of an outbound request, not its information content. The
|
||||
reviewing model reads repository data and then chooses the query terms and URL path,
|
||||
and a token that satisfies every rule above — short, low-entropy, API-shaped — can
|
||||
still encode repository-derived bytes. The provider allowlist fixes who receives a
|
||||
request; it does not constrain what the request says. Research is therefore an
|
||||
outbound disclosure channel to Brave and the documentation providers, appropriate only
|
||||
where repository-derived terms may be shared with them, and the reviewer prompt's
|
||||
instruction not to send repository text is a mitigation rather than an enforcement.
|
||||
|
||||
MCP output is schema-validated, limited to HTTPS URLs from the requested provider,
|
||||
truncated, and sanitized. Returned passages are labeled untrusted reference data in
|
||||
the reviewer prompt. The Brave credential is scoped to the MCP child and is never
|
||||
|
||||
+1
-3
@@ -16,8 +16,7 @@
|
||||
},
|
||||
"files": [
|
||||
"build",
|
||||
"templates",
|
||||
"research/sources.json"
|
||||
"templates"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
@@ -38,7 +37,6 @@
|
||||
"llp:check": "./ref-check",
|
||||
"dev": "bun run src/cli.ts",
|
||||
"test:unit": "bun test",
|
||||
"research:update": "bun run src/research-mcp/cli.ts update",
|
||||
"research:evaluate:corpora": "bun run build && node scripts/research/evaluate-corpora.mjs",
|
||||
"research:evaluate:expo": "bun run build && node scripts/research/evaluate-expo-prs.mjs",
|
||||
"release": "bash scripts/release.sh",
|
||||
|
||||
@@ -1,295 +0,0 @@
|
||||
{
|
||||
"output": "data/docs-index.json",
|
||||
"crawl": {
|
||||
"maxPagesPerProvider": 250,
|
||||
"maxDepth": 4,
|
||||
"delayMs": 200,
|
||||
"timeoutMs": 15000,
|
||||
"maxResponseBytes": 5000000
|
||||
},
|
||||
"sources": [
|
||||
{
|
||||
"provider": "apple",
|
||||
"sourceKind": "official-api",
|
||||
"maxPages": 170,
|
||||
"seedUrls": [
|
||||
"https://developer.apple.com/documentation/swift",
|
||||
"https://developer.apple.com/documentation/swiftui",
|
||||
"https://developer.apple.com/documentation/uikit",
|
||||
"https://developer.apple.com/documentation/foundation",
|
||||
"https://developer.apple.com/documentation/avfoundation",
|
||||
"https://developer.apple.com/documentation/coregraphics",
|
||||
"https://developer.apple.com/documentation/photos",
|
||||
"https://developer.apple.com/documentation/usernotifications",
|
||||
"https://developer.apple.com/documentation/webkit",
|
||||
"https://developer.apple.com/documentation/storekit",
|
||||
"https://developer.apple.com/documentation/activitykit",
|
||||
"https://developer.apple.com/documentation/widgetkit",
|
||||
"https://developer.apple.com/documentation/appintents",
|
||||
"https://developer.apple.com/documentation/network",
|
||||
"https://developer.apple.com/documentation/photos/phasset",
|
||||
"https://developer.apple.com/documentation/photos/phasset/location",
|
||||
"https://developer.apple.com/documentation/network/nwpathmonitor",
|
||||
"https://developer.apple.com/documentation/network/nwpathmonitor/pathupdatehandler",
|
||||
"https://developer.apple.com/documentation/appintents/liveactivityintent",
|
||||
"https://developer.apple.com/documentation/activitykit/activity",
|
||||
"https://developer.apple.com/documentation/activitykit/activity/id",
|
||||
"https://developer.apple.com/documentation/activitykit/activity/activities",
|
||||
"https://developer.apple.com/documentation/activitykit/activity/activitystate",
|
||||
"https://developer.apple.com/documentation/activitykit/activity/pushtokenupdates-swift.property",
|
||||
"https://developer.apple.com/documentation/avfaudio/avaudiosession",
|
||||
"https://developer.apple.com/documentation/avfaudio/avaudiosession/setactive(_:options:)",
|
||||
"https://developer.apple.com/documentation/avfaudio/avaudiosession/setactiveoptions/notifyothersondeactivation",
|
||||
"https://developer.apple.com/documentation/swiftui/view/widgeturl(_:)",
|
||||
"https://developer.apple.com/design/human-interface-guidelines/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "apple-releases",
|
||||
"sourceKind": "release-notes",
|
||||
"maxPages": 40,
|
||||
"maxDepth": 2,
|
||||
"seedUrls": [
|
||||
"https://developer.apple.com/documentation/xcode-release-notes",
|
||||
"https://developer.apple.com/documentation/ios-ipados-release-notes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "swift-evolution",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 35,
|
||||
"maxDepth": 1,
|
||||
"seedUrls": [
|
||||
"https://www.swift.org/swift-evolution/",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0302-concurrent-value-and-concurrent-closures.md",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0304-structured-concurrency.md",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0306-actors.md",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0316-global-actors.md",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0338-clarify-execution-non-actor-async.md",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0420-inheritance-of-actor-isolation.md",
|
||||
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0430-transferring-parameters-and-results.md"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "sdwebimage",
|
||||
"sourceKind": "official-api",
|
||||
"maxPages": 30,
|
||||
"maxDepth": 2,
|
||||
"seedUrls": [
|
||||
"https://sdwebimage.github.io/documentation/sdwebimage/",
|
||||
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
|
||||
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimageoptions/",
|
||||
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagecontextoption/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "android",
|
||||
"sourceKind": "official-api",
|
||||
"maxPages": 150,
|
||||
"seedUrls": [
|
||||
"https://developer.android.com/develop",
|
||||
"https://developer.android.com/reference",
|
||||
"https://developer.android.com/develop/ui/views/text-and-emoji/fonts-in-xml",
|
||||
"https://developer.android.com/reference/androidx/biometric/BiometricPrompt.PromptInfo.Builder",
|
||||
"https://developer.android.com/reference/android/media/AudioFocusRequest",
|
||||
"https://developer.android.com/reference/android/media/AudioManager",
|
||||
"https://developer.android.com/reference/android/net/ConnectivityManager",
|
||||
"https://developer.android.com/reference/android/net/ConnectivityManager.NetworkCallback",
|
||||
"https://developer.android.com/reference/android/content/Intent",
|
||||
"https://developer.android.com/reference/android/app/NotificationManager",
|
||||
"https://developer.android.com/reference/android/os/Build.VERSION_CODES",
|
||||
"https://developer.android.com/reference/androidx/core/os/BuildCompat",
|
||||
"https://developer.android.com/reference/androidx/work/Constraints.Builder",
|
||||
"https://developer.android.com/reference/androidx/work/NetworkType",
|
||||
"https://developer.android.com/develop/ui/compose/graphics/images/loading",
|
||||
"https://developer.android.com/develop/ui/compose/graphics/images/customize",
|
||||
"https://developer.android.com/reference/kotlin/androidx/compose/foundation/Image.composable",
|
||||
"https://developers.google.com/android/reference/com/google/android/gms/location/FusedLocationProviderClient",
|
||||
"https://developers.google.com/android/reference/com/google/android/gms/location/LocationCallback"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "android-releases",
|
||||
"sourceKind": "release-notes",
|
||||
"maxPages": 50,
|
||||
"maxDepth": 2,
|
||||
"seedUrls": [
|
||||
"https://developer.android.com/about/versions/16",
|
||||
"https://developer.android.com/about/versions/16/features",
|
||||
"https://developer.android.com/about/versions/16/behavior-changes-all",
|
||||
"https://developer.android.com/about/versions/16/behavior-changes-16",
|
||||
"https://developer.android.com/about/versions/15/behavior-changes-all",
|
||||
"https://developer.android.com/about/versions/15/behavior-changes-15",
|
||||
"https://developer.android.com/about/versions/14/behavior-changes-all",
|
||||
"https://developer.android.com/about/versions/14/behavior-changes-14"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "media3",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 40,
|
||||
"maxDepth": 2,
|
||||
"seedUrls": [
|
||||
"https://developer.android.com/media/media3",
|
||||
"https://developer.android.com/media/media3/session/background-playback",
|
||||
"https://developer.android.com/jetpack/androidx/releases/media3",
|
||||
"https://developer.android.com/reference/androidx/media3/common/Player",
|
||||
"https://developer.android.com/reference/androidx/media3/session/MediaSessionService"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "glide",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 30,
|
||||
"maxDepth": 2,
|
||||
"seedUrls": [
|
||||
"https://bumptech.github.io/glide/doc/caching.html",
|
||||
"https://bumptech.github.io/glide/doc/configuration.html",
|
||||
"https://bumptech.github.io/glide/javadocs/450/com/bumptech/glide/load/engine/cache/DiskCache.html",
|
||||
"https://bumptech.github.io/glide/javadocs/450/com/bumptech/glide/load/engine/cache/InternalCacheDiskCacheFactory.html"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "okhttp",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 30,
|
||||
"maxDepth": 2,
|
||||
"seedUrls": [
|
||||
"https://lysine.dev/okhttp/",
|
||||
"https://lysine.dev/okhttp/recipes/",
|
||||
"https://lysine.dev/okhttp/features/caching/",
|
||||
"https://lysine.dev/okhttp/features/calls/",
|
||||
"https://lysine.dev/okhttp/features/connections/",
|
||||
"https://lysine.dev/okhttp/features/https/",
|
||||
"https://lysine.dev/okhttp/features/interceptors/",
|
||||
"https://lysine.dev/okhttp/changelogs/changelog/",
|
||||
"https://lysine.dev/okhttp/5.x/okhttp/okhttp3/-ok-http-client/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "kotlin-coroutines",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 40,
|
||||
"maxDepth": 1,
|
||||
"seedUrls": [
|
||||
"https://kotlinlang.org/docs/coroutines-guide.html",
|
||||
"https://kotlinlang.org/docs/coroutines-basics.html",
|
||||
"https://kotlinlang.org/docs/cancellation-and-timeouts.html",
|
||||
"https://kotlinlang.org/docs/exception-handling.html",
|
||||
"https://kotlinlang.org/docs/shared-mutable-state-and-concurrency.html",
|
||||
"https://kotlinlang.org/docs/flow.html",
|
||||
"https://kotlinlang.org/api/kotlinx.coroutines/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "gradle",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 40,
|
||||
"maxDepth": 1,
|
||||
"seedUrls": [
|
||||
"https://docs.gradle.org/current/userguide/tooling_api.html",
|
||||
"https://docs.gradle.org/current/userguide/composite_builds.html",
|
||||
"https://docs.gradle.org/current/userguide/public_apis.html",
|
||||
"https://docs.gradle.org/current/userguide/upgrading_version_9.html",
|
||||
"https://docs.gradle.org/current/release-notes.html"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "agp",
|
||||
"sourceKind": "release-notes",
|
||||
"maxPages": 40,
|
||||
"maxDepth": 1,
|
||||
"seedUrls": [
|
||||
"https://developer.android.com/build/releases/about-agp",
|
||||
"https://developer.android.com/build/releases/gradle-plugin-roadmap",
|
||||
"https://developer.android.com/build/releases/agp-9-0-0-release-notes",
|
||||
"https://developer.android.com/reference/tools/gradle-api"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "jetbrains-issues",
|
||||
"sourceKind": "issue-tracker",
|
||||
"maxPages": 1,
|
||||
"maxDepth": 0,
|
||||
"seedUrls": [
|
||||
"https://youtrack.jetbrains.com/issue/IDEA-329756/Importing-symlinked-Gradle-included-build-fails"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "expo",
|
||||
"sourceKind": "official-api",
|
||||
"maxPages": 100,
|
||||
"maxDepth": 3,
|
||||
"seedUrls": [
|
||||
"https://docs.expo.dev/versions/latest/",
|
||||
"https://docs.expo.dev/versions/latest/sdk/expo/",
|
||||
"https://docs.expo.dev/versions/latest/sdk/camera/",
|
||||
"https://docs.expo.dev/versions/latest/sdk/notifications/",
|
||||
"https://docs.expo.dev/router/introduction/",
|
||||
"https://docs.expo.dev/modules/overview/",
|
||||
"https://docs.expo.dev/guides/new-architecture/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "react-native",
|
||||
"sourceKind": "official-api",
|
||||
"maxPages": 80,
|
||||
"maxDepth": 3,
|
||||
"seedUrls": [
|
||||
"https://reactnative.dev/docs/getting-started",
|
||||
"https://reactnative.dev/docs/components-and-apis",
|
||||
"https://reactnative.dev/docs/flatlist",
|
||||
"https://reactnative.dev/docs/view",
|
||||
"https://reactnative.dev/docs/platform",
|
||||
"https://reactnative.dev/architecture/landing-page"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "react-native-reanimated",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 60,
|
||||
"maxDepth": 3,
|
||||
"seedUrls": [
|
||||
"https://docs.swmansion.com/react-native-reanimated/docs/fundamentals/getting-started/",
|
||||
"https://docs.swmansion.com/react-native-reanimated/docs/guides/testing/",
|
||||
"https://docs.swmansion.com/react-native-reanimated/docs/guides/compatibility/",
|
||||
"https://docs.swmansion.com/react-native-reanimated/docs/category/core/",
|
||||
"https://docs.swmansion.com/react-native-reanimated/docs/category/advanced-apis/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "react-native-gesture-handler",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 60,
|
||||
"maxDepth": 3,
|
||||
"seedUrls": [
|
||||
"https://docs.swmansion.com/react-native-gesture-handler/docs/fundamentals/getting-started/",
|
||||
"https://docs.swmansion.com/react-native-gesture-handler/docs/gestures/pan-gesture/",
|
||||
"https://docs.swmansion.com/react-native-gesture-handler/docs/gestures/gesture-detector/"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "react-native-screens",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 50,
|
||||
"maxDepth": 3,
|
||||
"seedUrls": [
|
||||
"https://docs.swmansion.com/react-native-screens/",
|
||||
"https://github.com/software-mansion/react-native-screens/blob/main/README.md"
|
||||
]
|
||||
},
|
||||
{
|
||||
"provider": "react-native-worklets",
|
||||
"sourceKind": "official-guide",
|
||||
"maxPages": 60,
|
||||
"maxDepth": 3,
|
||||
"seedUrls": [
|
||||
"https://docs.swmansion.com/react-native-worklets/docs/fundamentals/getting-started/",
|
||||
"https://docs.swmansion.com/react-native-worklets/docs/guides/testing/",
|
||||
"https://docs.swmansion.com/react-native-worklets/docs/category/fundamentals/",
|
||||
"https://docs.swmansion.com/react-native-worklets/docs/fundamentals/sharing-memory/",
|
||||
"https://docs.swmansion.com/react-native-worklets/docs/guides/feature-flags/"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -3,8 +3,6 @@
|
||||
// Default mode replays recorded fixtures (no network, no key) and validates
|
||||
// the full MCP pipeline deterministically. Set RESEARCH_EVAL_LIVE=1 (plus
|
||||
// BRAVE_SEARCH_API_KEY) to run the same cases against live search instead.
|
||||
import { existsSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
|
||||
@@ -12,8 +10,6 @@ import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
|
||||
import { createDocumentationServer } from "../../build/research-mcp/server.js";
|
||||
import { createRecordedFetch } from "./recorded-fetch.mjs";
|
||||
|
||||
const INDEX_PATH = fileURLToPath(new URL("../../research/data/docs-index.json", import.meta.url));
|
||||
|
||||
async function callSearch(serverOptions, name, args) {
|
||||
// One server per case keeps every case inside the per-server call budget.
|
||||
const server = await createDocumentationServer(serverOptions);
|
||||
@@ -44,7 +40,6 @@ export async function runEvalCases(cases, { clientName, buildArguments, assertCa
|
||||
...(process.env.BRAVE_SEARCH_API_KEY
|
||||
? { braveApiKey: process.env.BRAVE_SEARCH_API_KEY }
|
||||
: {}),
|
||||
...(existsSync(INDEX_PATH) ? { indexPath: INDEX_PATH } : {}),
|
||||
}
|
||||
: { fetchImplementation: createRecordedFetch(cases), braveApiKey: "recorded-fixtures" };
|
||||
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
import { expect, test } from "bun:test";
|
||||
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import { resolveIndexOutputPath, updateDocumentationIndex } from "../../research-mcp/crawler.js";
|
||||
|
||||
test("default index output is relative to the source config directory", () => {
|
||||
const configPath = path.join("repo", "research", "sources.json");
|
||||
|
||||
expect(resolveIndexOutputPath(configPath, "data/docs-index.json")).toBe(
|
||||
path.resolve("repo", "research", "data", "docs-index.json"),
|
||||
);
|
||||
});
|
||||
|
||||
test("an explicit output path remains relative to the caller's working directory", () => {
|
||||
expect(
|
||||
resolveIndexOutputPath(
|
||||
path.join("repo", "research", "sources.json"),
|
||||
"data/docs-index.json",
|
||||
"custom/index.json",
|
||||
),
|
||||
).toBe(path.resolve("custom", "index.json"));
|
||||
});
|
||||
|
||||
test("an invalid seed is reported without aborting other seeds", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "review-research-crawler-"));
|
||||
const configPath = path.join(directory, "sources.json");
|
||||
const originalFetch = globalThis.fetch;
|
||||
globalThis.fetch = async () =>
|
||||
new Response(
|
||||
"<html><head><title>Android Widget</title></head><body><main><h1>Android Widget</h1><p>A documented Android API with enough visible text to produce a searchable local index passage.</p></main></body></html>",
|
||||
{ headers: { "content-type": "text/html" } },
|
||||
);
|
||||
try {
|
||||
await writeFile(
|
||||
configPath,
|
||||
JSON.stringify({
|
||||
output: "data/index.json",
|
||||
crawl: {
|
||||
maxPagesPerProvider: 2,
|
||||
maxDepth: 0,
|
||||
delayMs: 0,
|
||||
timeoutMs: 1000,
|
||||
maxResponseBytes: 4096,
|
||||
},
|
||||
sources: [
|
||||
{
|
||||
provider: "android",
|
||||
sourceKind: "official-api",
|
||||
seedUrls: [
|
||||
"https://untrusted.example/reference/widget",
|
||||
"https://developer.android.com/reference/widget",
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
|
||||
const result = await updateDocumentationIndex({ configPath });
|
||||
expect(result.documentCount).toBe(1);
|
||||
expect(result.providers[0]?.errors[0]).toContain("untrusted.example");
|
||||
expect(JSON.parse(await readFile(result.outputPath, "utf8")).documentCount).toBe(1);
|
||||
} finally {
|
||||
globalThis.fetch = originalFetch;
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
@@ -7,45 +7,13 @@ import { fileURLToPath } from "node:url";
|
||||
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
|
||||
|
||||
import { buildSearchIndex, writeSearchIndex } from "../../research-mcp/search-index.js";
|
||||
|
||||
test("stdio MCP lists and calls the read-only documentation search tool", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "review-research-mcp-"));
|
||||
const indexPath = path.join(directory, "index.json");
|
||||
const built = buildSearchIndex(
|
||||
[
|
||||
{
|
||||
id: "apple:mainactor",
|
||||
platform: "apple",
|
||||
provider: "apple",
|
||||
sourceKind: "official-api",
|
||||
title: "MainActor",
|
||||
url: "https://developer.apple.com/documentation/swift/mainactor",
|
||||
passage: "A singleton actor whose executor is equivalent to the main dispatch queue.",
|
||||
language: "swift",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
{
|
||||
id: "apple:poisoned-mainactor",
|
||||
platform: "apple",
|
||||
provider: "apple",
|
||||
sourceKind: "official-api",
|
||||
title: "MainActor forged mirror",
|
||||
url: "https://attacker.example/documentation/swift/mainactor",
|
||||
passage: "MainActor dispatch queue documentation copied to an untrusted host.",
|
||||
language: "swift",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
],
|
||||
1,
|
||||
"2026-08-06T00:00:00.000Z",
|
||||
);
|
||||
await writeSearchIndex(indexPath, built.serialized);
|
||||
|
||||
// The only test that drives the real stdio binary, so it owns the tool-metadata
|
||||
// contract: names, annotations, and the schemas a model actually reads.
|
||||
test("stdio MCP advertises the read-only documentation tools and their enforced bounds", async () => {
|
||||
const cliPath = fileURLToPath(new URL("../../research-mcp/cli.ts", import.meta.url));
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [cliPath, "serve", "--index", indexPath],
|
||||
args: [cliPath, "serve"],
|
||||
stderr: "pipe",
|
||||
});
|
||||
const client = new Client({ name: "test-client", version: "1.0.0" });
|
||||
@@ -101,34 +69,12 @@ test("stdio MCP lists and calls the read-only documentation search tool", async
|
||||
| undefined;
|
||||
expect(searchInputSchema?.properties?.limit?.maximum).toBe(3);
|
||||
expect(searchInputSchema?.properties?.limit?.default).toBe(3);
|
||||
|
||||
const response = await client.callTool({
|
||||
name: "search_platform_docs",
|
||||
arguments: {
|
||||
platform: "apple",
|
||||
providers: ["apple"],
|
||||
sourceKinds: ["official-api"],
|
||||
query: "MainActor dispatch queue",
|
||||
limit: 3,
|
||||
},
|
||||
});
|
||||
const content = response.content as Array<{ type: string; text?: string }>;
|
||||
const textBlock = content.find((block) => block.type === "text");
|
||||
expect(textBlock?.type).toBe("text");
|
||||
const payload = JSON.parse(textBlock.text ?? "") as {
|
||||
results: Array<{ title: string; provider: string; sourceKind: string }>;
|
||||
};
|
||||
expect(payload.results[0]?.title).toBe("MainActor");
|
||||
expect(payload.results[0]?.provider).toBe("apple");
|
||||
expect(payload.results[0]?.sourceKind).toBe("official-api");
|
||||
expect(payload.results).toHaveLength(1);
|
||||
} finally {
|
||||
await client.close();
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("stdio MCP starts without an index and reports unavailable remote discovery honestly", async () => {
|
||||
test("stdio MCP reports unavailable remote discovery honestly", async () => {
|
||||
const cliPath = fileURLToPath(new URL("../../research-mcp/cli.ts", import.meta.url));
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
@@ -151,14 +97,23 @@ test("stdio MCP starts without an index and reports unavailable remote discovery
|
||||
});
|
||||
const content = response.content as Array<{ type: string; text?: string }>;
|
||||
const payload = JSON.parse(content.find((block) => block.type === "text")?.text ?? "") as {
|
||||
retrieval: { localIndex: unknown; scopedWebSearch: boolean };
|
||||
retrieval: {
|
||||
scopedWebSearch: boolean;
|
||||
network: Record<string, number>;
|
||||
};
|
||||
warnings: string[];
|
||||
results: unknown[];
|
||||
};
|
||||
expect(payload.retrieval).toEqual({
|
||||
expect(payload.retrieval).toMatchObject({
|
||||
scopedWebSearch: false,
|
||||
expoSearch: false,
|
||||
localIndex: null,
|
||||
});
|
||||
// No key and no provider reachable, so the call must not have touched the network.
|
||||
expect(payload.retrieval.network).toMatchObject({
|
||||
searchRequests: 0,
|
||||
documentRequests: 0,
|
||||
redirects: 0,
|
||||
totalRequests: 0,
|
||||
});
|
||||
expect(payload.warnings[0]).toMatch(/BRAVE_SEARCH_API_KEY is not set/);
|
||||
expect(payload.results).toEqual([]);
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
import { expect, test } from "bun:test";
|
||||
|
||||
import {
|
||||
createResearchNetwork,
|
||||
ResearchDeadlineError,
|
||||
totalResearchNetwork,
|
||||
} from "../../research-mcp/network.js";
|
||||
import { fetchDocumentationUrl } from "../../research-mcp/direct-fetch.js";
|
||||
|
||||
const html =
|
||||
"<html><head><title>Doc</title></head><body><main><h1>Doc</h1>" +
|
||||
"<p>CameraView barcodeScannerSettings documentation body text.</p></main></body></html>";
|
||||
|
||||
function page(): Response {
|
||||
return new Response(html, { status: 200, headers: { "content-type": "text/html" } });
|
||||
}
|
||||
|
||||
test("the call deadline bounds a redirect chain that the per-hop timeout cannot", async () => {
|
||||
// Every hop answers quickly enough to beat the hardcoded 10s per-attempt limit,
|
||||
// which is exactly how a chain reaches 60s with no end-to-end bound.
|
||||
const network = createResearchNetwork(async (input) => {
|
||||
const url = new URL(typeof input === "string" ? input : ((input as URL).href ?? String(input)));
|
||||
await new Promise((resolve) => setTimeout(resolve, 60));
|
||||
const next = new URL(url.href);
|
||||
next.pathname += "/hop";
|
||||
return new Response(null, { status: 302, headers: { location: next.href } });
|
||||
}, 200);
|
||||
|
||||
await expect(
|
||||
fetchDocumentationUrl("https://docs.expo.dev/versions/latest/sdk/camera", {
|
||||
fetchImplementation: network.fetch,
|
||||
}),
|
||||
).rejects.toThrow();
|
||||
expect(network.expired()).toBe(true);
|
||||
// Without the deadline this walks all six hops; the deadline cuts it short.
|
||||
expect(network.counts().totalRequests).toBeLessThan(6);
|
||||
});
|
||||
|
||||
test("an expired deadline refuses the next request instead of starting it", async () => {
|
||||
let issued = 0;
|
||||
const network = createResearchNetwork(async () => {
|
||||
issued++;
|
||||
return page();
|
||||
}, 1);
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
|
||||
await expect(network.fetch("https://docs.expo.dev/x")).rejects.toBeInstanceOf(
|
||||
ResearchDeadlineError,
|
||||
);
|
||||
expect(issued).toBe(0);
|
||||
});
|
||||
|
||||
test("the ledger separates paid discovery from page fetches and redirect hops", async () => {
|
||||
const network = createResearchNetwork(async (input) => {
|
||||
const href = typeof input === "string" ? input : ((input as URL).href ?? String(input));
|
||||
if (href.startsWith("https://api.search.brave.com/")) {
|
||||
return new Response("{}", { status: 200, headers: { "content-type": "application/json" } });
|
||||
}
|
||||
if (href.endsWith("/redirect-me")) {
|
||||
return new Response(null, {
|
||||
status: 302,
|
||||
headers: { location: "https://docs.expo.dev/final" },
|
||||
});
|
||||
}
|
||||
return page();
|
||||
}, 10_000);
|
||||
|
||||
await network.fetch("https://api.search.brave.com/res/v1/web/search?q=x");
|
||||
await network.fetch("https://docs.expo.dev/redirect-me");
|
||||
await network.fetch("https://docs.expo.dev/final");
|
||||
|
||||
expect(network.counts()).toMatchObject({
|
||||
searchRequests: 1,
|
||||
// The redirect hop is a round trip, not a distinct page.
|
||||
documentRequests: 1,
|
||||
redirects: 1,
|
||||
totalRequests: 3,
|
||||
});
|
||||
});
|
||||
|
||||
test("a redirect from a discovery endpoint never drives a counter negative", async () => {
|
||||
// The search backends also use `redirect: "manual"` and treat a 3xx as an
|
||||
// error, so a redirect must undo the counter it actually incremented.
|
||||
const network = createResearchNetwork(
|
||||
async () =>
|
||||
new Response(null, {
|
||||
status: 302,
|
||||
headers: { location: "https://api.search.brave.com/res/v2/web/search" },
|
||||
}),
|
||||
10_000,
|
||||
);
|
||||
await network.fetch("https://api.search.brave.com/res/v1/web/search?q=x");
|
||||
|
||||
const counts = network.counts();
|
||||
expect(counts).toMatchObject({ searchRequests: 0, documentRequests: 0, redirects: 1 });
|
||||
expect(counts.documentRequests).toBeGreaterThanOrEqual(0);
|
||||
expect(counts.searchRequests).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
test("per-call ledgers sum into one review-wide total", () => {
|
||||
expect(
|
||||
totalResearchNetwork([
|
||||
{
|
||||
searchRequests: 4,
|
||||
documentRequests: 16,
|
||||
redirects: 2,
|
||||
totalRequests: 22,
|
||||
elapsedMs: 1_200,
|
||||
},
|
||||
{ searchRequests: 1, documentRequests: 3, redirects: 0, totalRequests: 4, elapsedMs: 300 },
|
||||
]),
|
||||
).toEqual({
|
||||
searchRequests: 5,
|
||||
documentRequests: 19,
|
||||
redirects: 2,
|
||||
totalRequests: 26,
|
||||
elapsedMs: 1_500,
|
||||
});
|
||||
});
|
||||
@@ -1,14 +1,6 @@
|
||||
import { expect, test } from "bun:test";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import {
|
||||
buildSearchIndex,
|
||||
loadSearchIndex,
|
||||
searchDocumentation,
|
||||
writeSearchIndex,
|
||||
} from "../../research-mcp/search-index.js";
|
||||
import { buildSearchIndex, searchDocumentation } from "../../research-mcp/search-index.js";
|
||||
import type { IndexedChunk } from "../../research-mcp/types.js";
|
||||
|
||||
const chunks: IndexedChunk[] = [
|
||||
@@ -33,36 +25,28 @@ const chunks: IndexedChunk[] = [
|
||||
},
|
||||
];
|
||||
|
||||
test("a serialized index reloads and filters search results by platform", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "review-research-index-"));
|
||||
try {
|
||||
const filePath = path.join(directory, "index.json");
|
||||
const built = buildSearchIndex(chunks, 2, "2026-08-06T00:00:00.000Z");
|
||||
await writeSearchIndex(filePath, built.serialized);
|
||||
const loaded = await loadSearchIndex(filePath);
|
||||
test("search filters results by platform and language", () => {
|
||||
const index = buildSearchIndex(chunks);
|
||||
|
||||
const appleResults = searchDocumentation(loaded, "main actor dispatch queue", {
|
||||
platform: "apple",
|
||||
language: "swift",
|
||||
limit: 5,
|
||||
});
|
||||
expect(appleResults).toHaveLength(1);
|
||||
expect(appleResults[0]?.title).toBe("MainActor");
|
||||
expect(appleResults[0]?.nextPassageId).toBe("apple:2");
|
||||
const appleResults = searchDocumentation(index, "main actor dispatch queue", {
|
||||
platform: "apple",
|
||||
language: "swift",
|
||||
limit: 5,
|
||||
});
|
||||
expect(appleResults).toHaveLength(1);
|
||||
expect(appleResults[0]?.title).toBe("MainActor");
|
||||
expect(appleResults[0]?.nextPassageId).toBe("apple:2");
|
||||
|
||||
const androidResults = searchDocumentation(loaded, "lifecycle callbacks", {
|
||||
platform: "android",
|
||||
limit: 5,
|
||||
});
|
||||
expect(androidResults).toHaveLength(1);
|
||||
expect(androidResults[0]?.platform).toBe("android");
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
const androidResults = searchDocumentation(index, "lifecycle callbacks", {
|
||||
platform: "android",
|
||||
limit: 5,
|
||||
});
|
||||
expect(androidResults).toHaveLength(1);
|
||||
expect(androidResults[0]?.platform).toBe("android");
|
||||
});
|
||||
|
||||
test("search rejects empty and oversized queries", () => {
|
||||
const index = buildSearchIndex(chunks, 2);
|
||||
const index = buildSearchIndex(chunks);
|
||||
expect(() => searchDocumentation(index, "\u0000\n", { platform: "all", limit: 5 })).toThrow(
|
||||
/between 1 and 300/,
|
||||
);
|
||||
@@ -72,19 +56,16 @@ test("search rejects empty and oversized queries", () => {
|
||||
});
|
||||
|
||||
test("symbol queries do not fall back to unrelated partial matches", () => {
|
||||
const index = buildSearchIndex(
|
||||
[
|
||||
{
|
||||
id: "android:unrelated",
|
||||
platform: "android",
|
||||
title: "Transient UI",
|
||||
url: "https://developer.android.com/example",
|
||||
passage: "A transient user interface action.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
],
|
||||
1,
|
||||
);
|
||||
const index = buildSearchIndex([
|
||||
{
|
||||
id: "android:unrelated",
|
||||
platform: "android",
|
||||
title: "Transient UI",
|
||||
url: "https://developer.android.com/example",
|
||||
passage: "A transient user interface action.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
]);
|
||||
|
||||
expect(
|
||||
searchDocumentation(index, "AudioManager AUDIOFOCUS_GAIN_TRANSIENT", {
|
||||
@@ -95,19 +76,16 @@ test("symbol queries do not fall back to unrelated partial matches", () => {
|
||||
});
|
||||
|
||||
test("symbol fallback keeps results anchored by canonical identity", () => {
|
||||
const index = buildSearchIndex(
|
||||
[
|
||||
{
|
||||
id: "apple:phasset",
|
||||
platform: "apple",
|
||||
title: "PHAsset",
|
||||
url: "https://developer.apple.com/documentation/photos/phasset",
|
||||
passage: "An image, video, or Live Photo in the Photos library.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
],
|
||||
1,
|
||||
);
|
||||
const index = buildSearchIndex([
|
||||
{
|
||||
id: "apple:phasset",
|
||||
platform: "apple",
|
||||
title: "PHAsset",
|
||||
url: "https://developer.apple.com/documentation/photos/phasset",
|
||||
passage: "An image, video, or Live Photo in the Photos library.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
]);
|
||||
|
||||
const results = searchDocumentation(index, "PHAsset location", {
|
||||
platform: "apple",
|
||||
@@ -117,19 +95,16 @@ test("symbol fallback keeps results anchored by canonical identity", () => {
|
||||
});
|
||||
|
||||
test("symbol anchors do not match identifier suffixes or generic acronyms", () => {
|
||||
const index = buildSearchIndex(
|
||||
[
|
||||
{
|
||||
id: "android:audio-focus-builder",
|
||||
platform: "android",
|
||||
title: "AudioFocusRequest.Builder API",
|
||||
url: "https://developer.android.com/reference/android/media/AudioFocusRequest.Builder",
|
||||
passage: "Builds an audio focus request with a custom listener.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
],
|
||||
1,
|
||||
);
|
||||
const index = buildSearchIndex([
|
||||
{
|
||||
id: "android:audio-focus-builder",
|
||||
platform: "android",
|
||||
title: "AudioFocusRequest.Builder API",
|
||||
url: "https://developer.android.com/reference/android/media/AudioFocusRequest.Builder",
|
||||
passage: "Builds an audio focus request with a custom listener.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
]);
|
||||
|
||||
expect(
|
||||
searchDocumentation(index, "OkHttp Request.Builder custom header API", {
|
||||
@@ -140,31 +115,28 @@ test("symbol anchors do not match identifier suffixes or generic acronyms", () =
|
||||
});
|
||||
|
||||
test("search filters named corpora and provenance classes", () => {
|
||||
const index = buildSearchIndex(
|
||||
[
|
||||
{
|
||||
id: "glide:cache",
|
||||
platform: "android",
|
||||
provider: "glide",
|
||||
sourceKind: "official-guide",
|
||||
title: "Glide caching",
|
||||
url: "https://bumptech.github.io/glide/doc/caching.html",
|
||||
passage: "Configure a custom DiskCache implementation.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
{
|
||||
id: "jetbrains:cache",
|
||||
platform: "android",
|
||||
provider: "jetbrains-issues",
|
||||
sourceKind: "issue-tracker",
|
||||
title: "Cache issue",
|
||||
url: "https://youtrack.jetbrains.com/issue/IDEA-1",
|
||||
passage: "A user reported a DiskCache problem.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
],
|
||||
2,
|
||||
);
|
||||
const index = buildSearchIndex([
|
||||
{
|
||||
id: "glide:cache",
|
||||
platform: "android",
|
||||
provider: "glide",
|
||||
sourceKind: "official-guide",
|
||||
title: "Glide caching",
|
||||
url: "https://bumptech.github.io/glide/doc/caching.html",
|
||||
passage: "Configure a custom DiskCache implementation.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
{
|
||||
id: "jetbrains:cache",
|
||||
platform: "android",
|
||||
provider: "jetbrains-issues",
|
||||
sourceKind: "issue-tracker",
|
||||
title: "Cache issue",
|
||||
url: "https://youtrack.jetbrains.com/issue/IDEA-1",
|
||||
passage: "A user reported a DiskCache problem.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
]);
|
||||
|
||||
const results = searchDocumentation(index, "DiskCache", {
|
||||
platform: "android",
|
||||
@@ -178,20 +150,17 @@ test("search filters named corpora and provenance classes", () => {
|
||||
});
|
||||
|
||||
test("a language filter excludes otherwise matching untagged documents", () => {
|
||||
const index = buildSearchIndex(
|
||||
[
|
||||
...chunks,
|
||||
{
|
||||
id: "apple:untagged",
|
||||
platform: "apple",
|
||||
title: "MainActor overview",
|
||||
url: "https://developer.apple.com/documentation/swift/mainactor-overview",
|
||||
passage: "Main actor dispatch queue behavior without language metadata.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
],
|
||||
3,
|
||||
);
|
||||
const index = buildSearchIndex([
|
||||
...chunks,
|
||||
{
|
||||
id: "apple:untagged",
|
||||
platform: "apple",
|
||||
title: "MainActor overview",
|
||||
url: "https://developer.apple.com/documentation/swift/mainactor-overview",
|
||||
passage: "Main actor dispatch queue behavior without language metadata.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
]);
|
||||
|
||||
const results = searchDocumentation(index, "MainActor dispatch queue", {
|
||||
platform: "apple",
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
import { expect, test } from "bun:test";
|
||||
import { spawn } from "node:child_process";
|
||||
import { copyFile, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { ReviewConfigSchema, ScopeReviewConfigSchema } from "../config/schema.js";
|
||||
import {
|
||||
@@ -19,6 +22,70 @@ import {
|
||||
toResearchProvenance,
|
||||
} from "../core/research.js";
|
||||
|
||||
/**
|
||||
* The declared `env` block is NOT the boundary. Both Claude Code and OpenCode merge
|
||||
* it onto the environment they already hold, so this asserts on the environment the
|
||||
* wrapper actually hands the server — the only place the guarantee is real.
|
||||
*/
|
||||
test("the wrapper hands the MCP server a constructed environment, not an inherited one", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "ecr-wrapper-test-"));
|
||||
try {
|
||||
const recordPath = path.join(directory, "record.json");
|
||||
// Stands in for the real server: the wrapper resolves ./cli.{js,ts} beside itself.
|
||||
await writeFile(
|
||||
path.join(directory, "cli.ts"),
|
||||
`import { writeFileSync } from "node:fs";\n` +
|
||||
`writeFileSync(${JSON.stringify(recordPath)}, JSON.stringify(Object.keys(process.env).sort()));\n`,
|
||||
"utf8",
|
||||
);
|
||||
for (const name of ["wrapper.ts", "child-env.ts"]) {
|
||||
await copyFile(
|
||||
fileURLToPath(new URL(`../research-mcp/${name}`, import.meta.url)),
|
||||
path.join(directory, name),
|
||||
);
|
||||
}
|
||||
|
||||
const exitCode = await new Promise<number>((resolve) => {
|
||||
const child = spawn(process.execPath, [path.join(directory, "wrapper.ts"), "serve"], {
|
||||
env: {
|
||||
// Everything an engine was measured to merge in, plus the legitimate block.
|
||||
PATH: process.env.PATH ?? "",
|
||||
HOME: "/root",
|
||||
ANTHROPIC_API_KEY: "must-not-reach-the-server",
|
||||
CLAUDE_CODE_OAUTH_TOKEN: "must-not-reach-the-server",
|
||||
AWS_SECRET_ACCESS_KEY: "must-not-reach-the-server",
|
||||
GITHUB_TOKEN: "must-not-reach-the-server",
|
||||
OPENCODE_CONFIG_CONTENT: "{}",
|
||||
NODE_OPTIONS: "--require /evil.js",
|
||||
LANG: "C.UTF-8",
|
||||
LC_ALL: "C.UTF-8",
|
||||
BRAVE_SEARCH_API_KEY: "search-only",
|
||||
REVIEW_RESEARCH_AUDIT_PATH: path.join(directory, "audit.jsonl"),
|
||||
REVIEW_RESEARCH_MAX_CALLS: "8",
|
||||
REVIEW_RESEARCH_MAX_RESULTS: "2",
|
||||
REVIEW_RESEARCH_TIMEOUT_MS: "30000",
|
||||
},
|
||||
stdio: "ignore",
|
||||
});
|
||||
child.on("exit", (code) => resolve(code ?? 1));
|
||||
});
|
||||
expect(exitCode).toBe(0);
|
||||
|
||||
const names = JSON.parse(await readFile(recordPath, "utf8")) as string[];
|
||||
expect(names.sort()).toEqual([
|
||||
"BRAVE_SEARCH_API_KEY",
|
||||
"LANG",
|
||||
"LC_ALL",
|
||||
"REVIEW_RESEARCH_AUDIT_PATH",
|
||||
"REVIEW_RESEARCH_MAX_CALLS",
|
||||
"REVIEW_RESEARCH_MAX_RESULTS",
|
||||
"REVIEW_RESEARCH_TIMEOUT_MS",
|
||||
]);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test("research child environment forwards only locale, proxy variables, and the fixed search key", () => {
|
||||
expect(
|
||||
researchChildEnvironment({
|
||||
@@ -39,16 +106,14 @@ test("research child environment forwards only locale, proxy variables, and the
|
||||
expect(researchChildEnvironment({ PATH: "/private/bin" })).not.toHaveProperty("PATH");
|
||||
});
|
||||
|
||||
test("research index fallback is absolute and research remains root-only", () => {
|
||||
expect(
|
||||
ReviewConfigSchema.safeParse({ research: { enabled: true, indexPath: "index.json" } }).success,
|
||||
).toBe(false);
|
||||
test("research is root-only and rejects an unknown key", () => {
|
||||
expect(ReviewConfigSchema.safeParse({ research: { enabled: true } }).success).toBe(true);
|
||||
// The offline index is gone; a stale config naming it must fail loudly rather
|
||||
// than be silently ignored.
|
||||
expect(
|
||||
ReviewConfigSchema.safeParse({
|
||||
research: { enabled: true, indexPath: path.join(tmpdir(), "index.json") },
|
||||
}).success,
|
||||
).toBe(true);
|
||||
ReviewConfigSchema.safeParse({ research: { enabled: true, indexPath: "/tmp/index.json" } })
|
||||
.success,
|
||||
).toBe(false);
|
||||
expect(ScopeReviewConfigSchema.safeParse({ research: { enabled: false } }).success).toBe(false);
|
||||
});
|
||||
|
||||
|
||||
+4
-5
@@ -578,11 +578,10 @@ async function runLegacyCi(
|
||||
// Dynamic stack context and model-backed reply adjudication have inputs outside
|
||||
// the scoped diff. Keep those paths fresh until their inputs join the cache key.
|
||||
// A maintainer's explicit /review is also always a real rerun.
|
||||
// Research output depends on the mounted index contents, not merely its configured
|
||||
// path. Until a signed index digest joins the cache key, a researched review must
|
||||
// run fresh rather than reuse evidence from an older artifact at the same path.
|
||||
const cacheAllowed =
|
||||
!bypassTriggerGate && !stack && !feedback && !config.research.enabled && metadata !== undefined;
|
||||
// Research no longer forces a fresh run: with the local index gone, every result
|
||||
// is fetched live from an allowlisted host during the run, so there is no mounted
|
||||
// artifact whose contents could drift out from under the cache key.
|
||||
const cacheAllowed = !bypassTriggerGate && !stack && !feedback && metadata !== undefined;
|
||||
let inputHash: string | undefined;
|
||||
|
||||
try {
|
||||
|
||||
+1
-1
@@ -44,7 +44,7 @@ const RESEARCH_CONFIG_DEFAULTS: LoadedConfig["research"] = {
|
||||
enabled: false,
|
||||
maxQueries: 8,
|
||||
resultsPerQuery: 2,
|
||||
timeoutMs: 15_000,
|
||||
timeoutMs: 30_000,
|
||||
};
|
||||
|
||||
/** Default OpenCode tool toggles for a reviewer: read the repo, never mutate it. */
|
||||
|
||||
@@ -65,10 +65,14 @@ export const ReviewConfigSchema = z.object({
|
||||
research: z
|
||||
.object({
|
||||
enabled: z.boolean().default(false),
|
||||
// Removed with the offline index. An unknown key would be stripped silently,
|
||||
// so name it explicitly: a config carrying it is stale, and quietly ignoring
|
||||
// the setting is worse than refusing to start.
|
||||
indexPath: z
|
||||
.string()
|
||||
.min(1)
|
||||
.refine((value) => path.isAbsolute(value), "research.indexPath must be an absolute path")
|
||||
.never({
|
||||
error:
|
||||
"research.indexPath was removed — documentation is now always fetched live from the provider allowlist. Delete this key.",
|
||||
})
|
||||
.optional(),
|
||||
maxQueries: z.number().int().min(1).max(20).default(8),
|
||||
resultsPerQuery: z.number().int().min(1).max(3).default(2),
|
||||
@@ -433,7 +437,6 @@ export interface LoadedConfig {
|
||||
/** Root-only configuration for the bundled, bounded documentation MCP. */
|
||||
research: {
|
||||
enabled: boolean;
|
||||
indexPath?: string;
|
||||
maxQueries: number;
|
||||
resultsPerQuery: number;
|
||||
timeoutMs: number;
|
||||
|
||||
+48
-35
@@ -9,6 +9,8 @@ import { fileURLToPath } from "node:url";
|
||||
|
||||
import type { LoadedConfig } from "../config/schema.js";
|
||||
import { readResearchAudit } from "../research-mcp/audit.js";
|
||||
import { researchChildEnvironment } from "../research-mcp/child-env.js";
|
||||
import { totalResearchNetwork, type ResearchNetworkCounts } from "../research-mcp/network.js";
|
||||
import type { Finding, FindingSource, ResearchDecision } from "./schema.js";
|
||||
export { OPENCODE_RESEARCH_TOOLS } from "./tools.js";
|
||||
|
||||
@@ -57,6 +59,12 @@ export interface ResearchProvenance {
|
||||
* url-rejected, budget-exhausted). Rejected input text is never recorded.
|
||||
*/
|
||||
rejections?: Array<{ tool: string; reason: string; count: number }>;
|
||||
/**
|
||||
* Review-wide outbound totals. Reported separately from the call budget because
|
||||
* `maxQueries` bounds MCP calls, and one call fans out into several discovery
|
||||
* requests plus a page download per candidate.
|
||||
*/
|
||||
network?: ResearchNetworkCounts;
|
||||
/** Grounded reviewer declarations that documentation changed a concrete decision. */
|
||||
decisions?: Array<ResearchDecision & { agent: string }>;
|
||||
usefulness?: ResearchUsefulness;
|
||||
@@ -103,36 +111,18 @@ export interface ResearchMcpRuntime {
|
||||
cleanup(): Promise<void>;
|
||||
}
|
||||
|
||||
const RESEARCH_PROXY_ENV_KEYS = [
|
||||
"HTTP_PROXY",
|
||||
"HTTPS_PROXY",
|
||||
"NO_PROXY",
|
||||
"http_proxy",
|
||||
"https_proxy",
|
||||
"no_proxy",
|
||||
] as const;
|
||||
const RESEARCH_SEARCH_API_KEY = "BRAVE_SEARCH_API_KEY";
|
||||
|
||||
export function researchChildEnvironment(
|
||||
source: NodeJS.ProcessEnv = process.env,
|
||||
): NodeJS.ProcessEnv {
|
||||
const environment: NodeJS.ProcessEnv = {
|
||||
LANG: "C.UTF-8",
|
||||
LC_ALL: "C.UTF-8",
|
||||
...(process.platform === "win32" && source.SystemRoot ? { SystemRoot: source.SystemRoot } : {}),
|
||||
};
|
||||
for (const key of RESEARCH_PROXY_ENV_KEYS) {
|
||||
if (source[key]) environment[key] = source[key];
|
||||
}
|
||||
if (source[RESEARCH_SEARCH_API_KEY]) {
|
||||
environment[RESEARCH_SEARCH_API_KEY] = source[RESEARCH_SEARCH_API_KEY];
|
||||
}
|
||||
return environment;
|
||||
}
|
||||
export { researchChildEnvironment };
|
||||
|
||||
/**
|
||||
* The engine spawns the WRAPPER, not the server. Both Claude Code and OpenCode
|
||||
* merge the config's `env` block onto their own environment instead of replacing
|
||||
* it, so the block below cannot bound the child on its own; the wrapper rebuilds
|
||||
* the environment from an explicit allowlist before the real server starts. See
|
||||
* research-mcp/wrapper.ts.
|
||||
*/
|
||||
export function bundledResearchServer(): { command: string; args: string[] } {
|
||||
const builtEntry = fileURLToPath(new URL("../research-mcp/cli.js", import.meta.url));
|
||||
const sourceEntry = fileURLToPath(new URL("../research-mcp/cli.ts", import.meta.url));
|
||||
const builtEntry = fileURLToPath(new URL("../research-mcp/wrapper.js", import.meta.url));
|
||||
const sourceEntry = fileURLToPath(new URL("../research-mcp/wrapper.ts", import.meta.url));
|
||||
return {
|
||||
command: process.execPath,
|
||||
args: [existsSync(builtEntry) ? builtEntry : sourceEntry],
|
||||
@@ -141,8 +131,12 @@ export function bundledResearchServer(): { command: string; args: string[] } {
|
||||
|
||||
/**
|
||||
* Create one owner-only MCP configuration and append-only audit for a review run.
|
||||
* The model process receives only the config path; the Brave credential is passed
|
||||
* directly to the bounded MCP child and never added to the model process env.
|
||||
* The model process receives only the config path.
|
||||
*
|
||||
* The `env` block below is a request the engine merges rather than applies — see
|
||||
* bundledResearchServer. The wrapper it names is what actually bounds the server's
|
||||
* environment, so the Brave credential and these limits are the only things the
|
||||
* server can see, whatever the engine passed down.
|
||||
*/
|
||||
export async function createResearchMcpRuntime(
|
||||
config: LoadedConfig["research"],
|
||||
@@ -152,11 +146,7 @@ export async function createResearchMcpRuntime(
|
||||
const auditPath = path.join(directory, "audit.jsonl");
|
||||
const claudeConfigPath = path.join(directory, "mcp.json");
|
||||
const server = bundledResearchServer();
|
||||
const args = [
|
||||
...server.args,
|
||||
"serve",
|
||||
...(config.indexPath ? ["--index", config.indexPath] : []),
|
||||
];
|
||||
const args = [...server.args, "serve"];
|
||||
const child = researchChildEnvironment();
|
||||
const environment = Object.fromEntries(
|
||||
Object.entries({
|
||||
@@ -164,6 +154,10 @@ export async function createResearchMcpRuntime(
|
||||
REVIEW_RESEARCH_AUDIT_PATH: auditPath,
|
||||
REVIEW_RESEARCH_MAX_CALLS: String(config.maxQueries),
|
||||
REVIEW_RESEARCH_MAX_RESULTS: String(config.resultsPerQuery),
|
||||
// The MCP enforces this itself as a per-call deadline. OpenCode's `timeout`
|
||||
// only bounds tool DISCOVERY and Claude has no equivalent, so neither engine
|
||||
// can bound how long a call actually runs — the server has to.
|
||||
REVIEW_RESEARCH_TIMEOUT_MS: String(config.timeoutMs),
|
||||
}).flatMap(([key, value]) => (value === undefined ? [] : [[key, value]])),
|
||||
);
|
||||
await writeFile(
|
||||
@@ -233,6 +227,8 @@ export async function researchProvenanceFromAudit(
|
||||
warnings: [...new Set(warnings)].slice(0, 10),
|
||||
};
|
||||
const provenance = toResearchProvenance(run);
|
||||
const ledgers = records.flatMap((record) => (record.network ? [record.network] : []));
|
||||
if (ledgers.length > 0) provenance.network = totalResearchNetwork(ledgers);
|
||||
if (rejections.length > 0) {
|
||||
const counts = new Map<string, number>();
|
||||
for (const rejection of rejections) {
|
||||
@@ -313,6 +309,13 @@ export function formatResearchProgress(provenance: ResearchProvenance): string[]
|
||||
);
|
||||
}
|
||||
}
|
||||
if (provenance.network) {
|
||||
const { searchRequests, documentRequests, redirects, totalRequests } = provenance.network;
|
||||
lines.push(
|
||||
` research network: ${searchRequests} search request(s), ${documentRequests} page fetch(es), ` +
|
||||
`${redirects} redirect(s) — ${totalRequests} HTTP request(s) total`,
|
||||
);
|
||||
}
|
||||
for (const rejection of provenance.rejections ?? []) {
|
||||
lines.push(
|
||||
` research: ${rejection.count} ${rejection.tool} call(s) rejected before execution (${rejection.reason})`,
|
||||
@@ -364,6 +367,16 @@ export function renderResearchMarkdown(provenance: ResearchProvenance): string {
|
||||
);
|
||||
}
|
||||
}
|
||||
if (provenance.network) {
|
||||
const { searchRequests, documentRequests, redirects, totalRequests, elapsedMs } =
|
||||
provenance.network;
|
||||
lines.push(
|
||||
"",
|
||||
`Outbound: **${searchRequests}** search request(s), **${documentRequests}** page fetch(es), ` +
|
||||
`**${redirects}** redirect(s) — **${totalRequests}** HTTP request(s) across ` +
|
||||
`${(elapsedMs / 1000).toFixed(1)}s of call time.`,
|
||||
);
|
||||
}
|
||||
for (const rejection of provenance.rejections ?? []) {
|
||||
lines.push(
|
||||
`- ⚠️ ${rejection.count} \`${escapeMarkdownLabel(rejection.tool)}\` call(s) rejected before execution (${escapeMarkdownLabel(rejection.reason)}).`,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { appendFile, mkdir, readFile, rmdir } from "node:fs/promises";
|
||||
import { randomUUID } from "node:crypto";
|
||||
|
||||
import type { ResearchNetworkCounts } from "./network.js";
|
||||
import type { SearchResult } from "./types.js";
|
||||
|
||||
export type ResearchToolName = "search_platform_docs" | "fetch_platform_doc";
|
||||
@@ -52,6 +53,12 @@ export interface ResearchAuditResult {
|
||||
>
|
||||
>;
|
||||
warnings: string[];
|
||||
/**
|
||||
* What this one reserved call actually spent on the network. Recorded because
|
||||
* the reservation count alone is a poor proxy: a single search can issue four
|
||||
* paid discovery requests and sixteen page downloads.
|
||||
*/
|
||||
network?: ResearchNetworkCounts;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
@@ -196,6 +203,7 @@ export class ResearchAudit {
|
||||
input: ResearchAuditInput,
|
||||
results: SearchResult[],
|
||||
warnings: string[] = [],
|
||||
network?: ResearchNetworkCounts,
|
||||
): Promise<void> {
|
||||
await this.append({
|
||||
type: "completed",
|
||||
@@ -204,6 +212,7 @@ export class ResearchAudit {
|
||||
input,
|
||||
results: results.map(boundedResult),
|
||||
warnings: warnings.slice(0, 10).map((warning) => warning.slice(0, 500)),
|
||||
...(network ? { network } : {}),
|
||||
timestamp: new Date().toISOString(),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
// @ref LLP 0013#one-package-two-binaries [implements] — the bounded MCP's environment is constructed, never inherited
|
||||
/**
|
||||
* The single definition of what the bounded documentation MCP is allowed to see
|
||||
* in its environment.
|
||||
*
|
||||
* Two callers share it, and they need it for different reasons:
|
||||
*
|
||||
* - `createResearchMcpRuntime` writes `researchChildEnvironment()` into the
|
||||
* engine's MCP configuration. That block is a REQUEST, not a guarantee: both
|
||||
* Claude Code and OpenCode merge it onto the environment the engine already
|
||||
* has rather than replacing it, so the declared allowlist alone never bounds
|
||||
* the child.
|
||||
* - `wrapper.ts` applies `researchWrapperEnvironment()` when it spawns the real
|
||||
* server. That IS the guarantee — the server process is handed a constructed
|
||||
* environment, so whatever the engine merged in stops at the wrapper.
|
||||
*
|
||||
* Nothing here imports a parser, a network client, or the config schema: the
|
||||
* wrapper must stay loadable without pulling untrusted-content machinery into
|
||||
* the one process that still holds the engine's credentials.
|
||||
*/
|
||||
|
||||
const PROXY_ENV_KEYS = [
|
||||
"HTTP_PROXY",
|
||||
"HTTPS_PROXY",
|
||||
"NO_PROXY",
|
||||
"http_proxy",
|
||||
"https_proxy",
|
||||
"no_proxy",
|
||||
] as const;
|
||||
|
||||
export const RESEARCH_SEARCH_API_KEY = "BRAVE_SEARCH_API_KEY";
|
||||
|
||||
/**
|
||||
* Bounds ECR sets on the child through the config env block. These are inputs to
|
||||
* the server, so the wrapper must forward them; every one is a number, a path ECR
|
||||
* itself chose, or both, and none is a credential.
|
||||
*/
|
||||
export const RESEARCH_RUNTIME_ENV_KEYS = [
|
||||
"REVIEW_RESEARCH_AUDIT_PATH",
|
||||
"REVIEW_RESEARCH_MAX_CALLS",
|
||||
"REVIEW_RESEARCH_MAX_RESULTS",
|
||||
"REVIEW_RESEARCH_TIMEOUT_MS",
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* Locale, proxy configuration, and the search-only credential — the environment
|
||||
* the MCP actually needs to do its job.
|
||||
*
|
||||
* NODE_OPTIONS is deliberately absent: it is arbitrary code injection into the
|
||||
* process that parses untrusted remote documents. So are PATH and HOME — the
|
||||
* wrapper spawns the server by absolute path and the server writes only to the
|
||||
* audit path it is given.
|
||||
*/
|
||||
export function researchChildEnvironment(
|
||||
source: NodeJS.ProcessEnv = process.env,
|
||||
): NodeJS.ProcessEnv {
|
||||
const environment: NodeJS.ProcessEnv = {
|
||||
LANG: "C.UTF-8",
|
||||
LC_ALL: "C.UTF-8",
|
||||
...(process.platform === "win32" && source.SystemRoot ? { SystemRoot: source.SystemRoot } : {}),
|
||||
};
|
||||
for (const key of PROXY_ENV_KEYS) {
|
||||
if (source[key]) environment[key] = source[key];
|
||||
}
|
||||
if (source[RESEARCH_SEARCH_API_KEY]) {
|
||||
environment[RESEARCH_SEARCH_API_KEY] = source[RESEARCH_SEARCH_API_KEY];
|
||||
}
|
||||
return environment;
|
||||
}
|
||||
|
||||
/** The child environment plus the bounds ECR passes through the config block. */
|
||||
export function researchWrapperEnvironment(
|
||||
source: NodeJS.ProcessEnv = process.env,
|
||||
): NodeJS.ProcessEnv {
|
||||
const environment = researchChildEnvironment(source);
|
||||
for (const key of RESEARCH_RUNTIME_ENV_KEYS) {
|
||||
if (source[key] !== undefined) environment[key] = source[key];
|
||||
}
|
||||
return environment;
|
||||
}
|
||||
+11
-61
@@ -1,28 +1,20 @@
|
||||
#!/usr/bin/env node
|
||||
// @ref LLP 0013#one-package-two-binaries [implements] — the package's second binary owns serve/update dispatch
|
||||
// @ref LLP 0013#search-fetch-and-optional-index-boundary [implements] — review-facing serve and operator-only update stay separate
|
||||
// @ref LLP 0013#one-package-two-binaries [implements] — the package's second binary serves the bounded MCP
|
||||
// @ref LLP 0013#search-fetch-and-optional-index-boundary [implements] — live discovery is the only evidence path
|
||||
|
||||
import { parseArgs } from "node:util";
|
||||
|
||||
import { defaultConfigPath } from "./paths.js";
|
||||
import { runStdioServer } from "./server.js";
|
||||
import { PLATFORMS, type Platform } from "./types.js";
|
||||
|
||||
function printHelp() {
|
||||
process.stdout.write(`review-research-mcp
|
||||
|
||||
Usage:
|
||||
review-research-mcp [serve]
|
||||
review-research-mcp serve [--index PATH]
|
||||
review-research-mcp update [--config PATH] [--output PATH]
|
||||
[--platform apple|android|react-native] [--max-pages NUMBER]
|
||||
|
||||
The serve command uses BRAVE_SEARCH_API_KEY for scoped web discovery, fetches only
|
||||
allowlisted official pages, and optionally falls back to a local index. Expo-provider
|
||||
searches use Expo's public documentation index. Its fetch_platform_doc tool can fetch
|
||||
one exact allowlisted documentation URL without a search key and return focused,
|
||||
section, or bounded-document extracted context. The update command is
|
||||
an optional offline crawler for operator-managed fallback indexes.
|
||||
The serve command uses BRAVE_SEARCH_API_KEY for scoped web discovery and fetches only
|
||||
allowlisted official pages. Expo-provider searches use Expo's public documentation
|
||||
index. Its fetch_platform_doc tool can fetch one exact allowlisted documentation URL
|
||||
without a search key and return focused, section, or bounded-document extracted
|
||||
context.
|
||||
`);
|
||||
}
|
||||
|
||||
@@ -48,21 +40,16 @@ async function main() {
|
||||
printHelp();
|
||||
return;
|
||||
}
|
||||
const { values } = parseArgs({
|
||||
args: rest,
|
||||
options: {
|
||||
index: { type: "string" },
|
||||
},
|
||||
strict: true,
|
||||
});
|
||||
const indexPath = values.index ?? process.env.REVIEW_RESEARCH_INDEX_PATH;
|
||||
if (rest.length > 0) {
|
||||
throw new Error(`Unexpected argument: ${rest[0]}`);
|
||||
}
|
||||
await runStdioServer({
|
||||
...(indexPath ? { indexPath } : {}),
|
||||
...(process.env.REVIEW_RESEARCH_AUDIT_PATH
|
||||
? { auditPath: process.env.REVIEW_RESEARCH_AUDIT_PATH }
|
||||
: {}),
|
||||
maxCalls: boundedInteger("REVIEW_RESEARCH_MAX_CALLS", 8, 1, 20),
|
||||
maxResultsPerCall: boundedInteger("REVIEW_RESEARCH_MAX_RESULTS", 3, 1, 3),
|
||||
timeoutMs: boundedInteger("REVIEW_RESEARCH_TIMEOUT_MS", 30_000, 1_000, 60_000),
|
||||
...(process.env.BRAVE_SEARCH_API_KEY
|
||||
? { braveApiKey: process.env.BRAVE_SEARCH_API_KEY }
|
||||
: {}),
|
||||
@@ -70,43 +57,6 @@ async function main() {
|
||||
return;
|
||||
}
|
||||
|
||||
if (command === "update") {
|
||||
if (rest.includes("--help") || rest.includes("-h")) {
|
||||
printHelp();
|
||||
return;
|
||||
}
|
||||
const { values } = parseArgs({
|
||||
args: rest,
|
||||
options: {
|
||||
config: { type: "string" },
|
||||
output: { type: "string" },
|
||||
platform: { type: "string", multiple: true },
|
||||
"max-pages": { type: "string" },
|
||||
},
|
||||
strict: true,
|
||||
});
|
||||
const invalidPlatform = values.platform?.find(
|
||||
(platform) => !PLATFORMS.includes(platform as Platform),
|
||||
);
|
||||
if (invalidPlatform) {
|
||||
throw new Error(`Unknown platform: ${invalidPlatform}`);
|
||||
}
|
||||
const maxPages = values["max-pages"] ? Number(values["max-pages"]) : undefined;
|
||||
if (maxPages !== undefined && (!Number.isInteger(maxPages) || maxPages < 1)) {
|
||||
throw new Error("--max-pages must be a positive integer");
|
||||
}
|
||||
|
||||
const { updateDocumentationIndex } = await import("./crawler.js");
|
||||
const result = await updateDocumentationIndex({
|
||||
configPath: values.config ?? defaultConfigPath,
|
||||
...(values.output ? { outputPath: values.output } : {}),
|
||||
...(values.platform ? { platforms: values.platform as Platform[] } : {}),
|
||||
...(maxPages ? { maxPagesPerProvider: maxPages } : {}),
|
||||
});
|
||||
process.stderr.write(`${JSON.stringify(result, null, 2)}\n`);
|
||||
return;
|
||||
}
|
||||
|
||||
throw new Error(`Unknown command: ${command}`);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,214 +0,0 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
import { z } from "zod";
|
||||
|
||||
import { extractAppleDocCPage } from "./apple-docc.js";
|
||||
import { fetchAllowedContent } from "./fetch-document.js";
|
||||
import { chunkDocument, extractDocumentationPage } from "./html.js";
|
||||
import { extractMarkdownDocumentationPage } from "./markdown.js";
|
||||
import { getProvider, resolveAllowedUrl, type DocumentationProvider } from "./providers.js";
|
||||
import { buildSearchIndex, writeSearchIndex } from "./search-index.js";
|
||||
import { extractYouTrackIssue } from "./youtrack.js";
|
||||
import {
|
||||
PLATFORMS,
|
||||
PROVIDERS,
|
||||
SOURCE_KINDS,
|
||||
type DiscoveredDocument,
|
||||
type IndexedChunk,
|
||||
type Platform,
|
||||
type ProviderId,
|
||||
type SourceDefinition,
|
||||
type SourcesConfig,
|
||||
} from "./types.js";
|
||||
|
||||
const sourcesConfigSchema = z.object({
|
||||
output: z.string().min(1),
|
||||
crawl: z.object({
|
||||
maxPagesPerProvider: z.number().int().min(1).max(100_000),
|
||||
maxDepth: z.number().int().min(0).max(20),
|
||||
delayMs: z.number().int().min(0).max(60_000),
|
||||
timeoutMs: z.number().int().min(100).max(120_000),
|
||||
maxResponseBytes: z.number().int().min(1_024).max(20_000_000),
|
||||
}),
|
||||
sources: z
|
||||
.array(
|
||||
z.object({
|
||||
provider: z.enum(PROVIDERS),
|
||||
sourceKind: z.enum(SOURCE_KINDS),
|
||||
seedUrls: z.array(z.string().url()).min(1),
|
||||
maxPages: z.number().int().min(1).max(100_000).optional(),
|
||||
maxDepth: z.number().int().min(0).max(20).optional(),
|
||||
}),
|
||||
)
|
||||
.min(1),
|
||||
});
|
||||
|
||||
interface CrawlItem {
|
||||
url: URL;
|
||||
depth: number;
|
||||
}
|
||||
|
||||
interface CrawlResult {
|
||||
provider: ProviderId;
|
||||
platform: Platform;
|
||||
documents: DiscoveredDocument[];
|
||||
errors: string[];
|
||||
}
|
||||
|
||||
export interface UpdateOptions {
|
||||
configPath: string;
|
||||
outputPath?: string;
|
||||
platforms?: Platform[];
|
||||
maxPagesPerProvider?: number;
|
||||
}
|
||||
|
||||
function sleep(milliseconds: number) {
|
||||
return new Promise((resolve) => setTimeout(resolve, milliseconds));
|
||||
}
|
||||
|
||||
async function crawlProvider(
|
||||
provider: DocumentationProvider,
|
||||
source: SourceDefinition,
|
||||
limits: SourcesConfig["crawl"],
|
||||
): Promise<CrawlResult> {
|
||||
const queue: CrawlItem[] = [];
|
||||
const errors: string[] = [];
|
||||
for (const seedUrl of source.seedUrls) {
|
||||
try {
|
||||
queue.push({ url: resolveAllowedUrl(provider, seedUrl), depth: 0 });
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
errors.push(`${seedUrl}: ${message}`);
|
||||
}
|
||||
}
|
||||
const queued = new Set(queue.map((item) => item.url.href));
|
||||
const visited = new Set<string>();
|
||||
const documents: DiscoveredDocument[] = [];
|
||||
|
||||
while (queue.length > 0 && visited.size < limits.maxPagesPerProvider) {
|
||||
const item = queue.shift();
|
||||
if (!item || visited.has(item.url.href)) continue;
|
||||
visited.add(item.url.href);
|
||||
|
||||
try {
|
||||
const content = await fetchAllowedContent(provider, item.url, limits);
|
||||
const sourceMetadata = {
|
||||
provider: provider.id,
|
||||
sourceKind: source.sourceKind,
|
||||
};
|
||||
const format = provider.responseFormat(item.url);
|
||||
const page =
|
||||
format === "docc-json"
|
||||
? extractAppleDocCPage(content, item.url.href, sourceMetadata)
|
||||
: format === "markdown"
|
||||
? extractMarkdownDocumentationPage(
|
||||
content,
|
||||
item.url.href,
|
||||
provider.platform,
|
||||
sourceMetadata,
|
||||
)
|
||||
: format === "youtrack-json"
|
||||
? extractYouTrackIssue(content, item.url.href, sourceMetadata)
|
||||
: extractDocumentationPage(content, item.url.href, provider.platform, sourceMetadata);
|
||||
if (page) {
|
||||
documents.push(page.document);
|
||||
if (item.depth < limits.maxDepth) {
|
||||
for (const href of page.links) {
|
||||
try {
|
||||
const nextUrl = resolveAllowedUrl(provider, href, item.url.href);
|
||||
if (!queued.has(nextUrl.href) && !visited.has(nextUrl.href)) {
|
||||
queued.add(nextUrl.href);
|
||||
queue.push({ url: nextUrl, depth: item.depth + 1 });
|
||||
}
|
||||
} catch {
|
||||
// Off-allowlist and malformed links are intentionally ignored.
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
errors.push(`${item.url.href}: ${message}`);
|
||||
}
|
||||
|
||||
if (limits.delayMs > 0 && queue.length > 0) {
|
||||
await sleep(limits.delayMs);
|
||||
}
|
||||
}
|
||||
|
||||
return { provider: provider.id, platform: provider.platform, documents, errors };
|
||||
}
|
||||
|
||||
export async function readSourcesConfig(configPath: string): Promise<SourcesConfig> {
|
||||
return sourcesConfigSchema.parse(JSON.parse(await readFile(configPath, "utf8")));
|
||||
}
|
||||
|
||||
export function resolveIndexOutputPath(
|
||||
configPath: string,
|
||||
configuredOutput: string,
|
||||
outputPath?: string,
|
||||
): string {
|
||||
if (outputPath) return path.resolve(outputPath);
|
||||
return path.resolve(path.dirname(path.resolve(configPath)), configuredOutput);
|
||||
}
|
||||
|
||||
export async function updateDocumentationIndex(options: UpdateOptions) {
|
||||
const absoluteConfigPath = path.resolve(options.configPath);
|
||||
const config = await readSourcesConfig(absoluteConfigPath);
|
||||
const selectedPlatforms = new Set(options.platforms ?? PLATFORMS);
|
||||
const limits = {
|
||||
...config.crawl,
|
||||
...(options.maxPagesPerProvider ? { maxPagesPerProvider: options.maxPagesPerProvider } : {}),
|
||||
};
|
||||
|
||||
const selectedSources = config.sources.filter((source) =>
|
||||
selectedPlatforms.has(getProvider(source.provider).platform),
|
||||
);
|
||||
if (selectedSources.length === 0) {
|
||||
throw new Error("No configured sources matched the selected platforms");
|
||||
}
|
||||
|
||||
const crawlResults = await Promise.all(
|
||||
selectedSources.map((source) => {
|
||||
const sourceLimits = {
|
||||
...limits,
|
||||
maxPagesPerProvider: Math.min(
|
||||
limits.maxPagesPerProvider,
|
||||
source.maxPages ?? limits.maxPagesPerProvider,
|
||||
),
|
||||
maxDepth: Math.min(limits.maxDepth, source.maxDepth ?? limits.maxDepth),
|
||||
};
|
||||
return crawlProvider(getProvider(source.provider), source, sourceLimits);
|
||||
}),
|
||||
);
|
||||
|
||||
const indexedAt = new Date().toISOString();
|
||||
const documents = crawlResults.flatMap((result) => result.documents);
|
||||
const chunks: IndexedChunk[] = documents.flatMap((document) =>
|
||||
chunkDocument(document, indexedAt),
|
||||
);
|
||||
if (chunks.length === 0) {
|
||||
const details = crawlResults
|
||||
.flatMap((result) => result.errors)
|
||||
.slice(0, 10)
|
||||
.join("\n");
|
||||
throw new Error(`Index update produced no searchable content${details ? `:\n${details}` : ""}`);
|
||||
}
|
||||
|
||||
const outputPath = resolveIndexOutputPath(absoluteConfigPath, config.output, options.outputPath);
|
||||
const index = buildSearchIndex(chunks, documents.length, indexedAt);
|
||||
await writeSearchIndex(outputPath, index.serialized);
|
||||
|
||||
return {
|
||||
outputPath,
|
||||
documentCount: documents.length,
|
||||
chunkCount: chunks.length,
|
||||
providers: crawlResults.map((result) => ({
|
||||
provider: result.provider,
|
||||
platform: result.platform,
|
||||
documentCount: result.documents.length,
|
||||
errors: result.errors,
|
||||
})),
|
||||
};
|
||||
}
|
||||
@@ -100,7 +100,7 @@ function rankedAnchor(
|
||||
query?: string,
|
||||
): SearchResult | undefined {
|
||||
if (!query?.trim()) return chunks[0] ? withScore(chunks[0]) : undefined;
|
||||
const index = buildSearchIndex(chunks, 1);
|
||||
const index = buildSearchIndex(chunks);
|
||||
return (
|
||||
searchDocumentation(index, query, {
|
||||
platform: document.platform,
|
||||
|
||||
@@ -7,14 +7,17 @@ import {
|
||||
type DocumentationProvider,
|
||||
} from "./providers.js";
|
||||
import { readBodyWithLimit } from "./response.js";
|
||||
import type { DiscoveredDocument, ProviderId, SourceKind, SourcesConfig } from "./types.js";
|
||||
import type { DiscoveredDocument, ProviderId, SourceKind } from "./types.js";
|
||||
import { extractYouTrackIssue } from "./youtrack.js";
|
||||
import type { FetchImplementation } from "./brave-search.js";
|
||||
|
||||
export const onDemandFetchLimits: SourcesConfig["crawl"] = {
|
||||
maxPagesPerProvider: 10,
|
||||
maxDepth: 0,
|
||||
delayMs: 0,
|
||||
/** Per-attempt bounds for one documentation request. */
|
||||
export interface DocumentFetchLimits {
|
||||
timeoutMs: number;
|
||||
maxResponseBytes: number;
|
||||
}
|
||||
|
||||
export const onDemandFetchLimits: DocumentFetchLimits = {
|
||||
timeoutMs: 10_000,
|
||||
maxResponseBytes: 5_000_000,
|
||||
};
|
||||
@@ -22,7 +25,7 @@ export const onDemandFetchLimits: SourcesConfig["crawl"] = {
|
||||
export async function fetchAllowedContent(
|
||||
provider: DocumentationProvider,
|
||||
documentUrl: URL,
|
||||
limits: Pick<SourcesConfig["crawl"], "timeoutMs" | "maxResponseBytes">,
|
||||
limits: DocumentFetchLimits,
|
||||
fetchImplementation: FetchImplementation = fetch,
|
||||
): Promise<string> {
|
||||
let currentUrl = resolveAllowedRequestUrl(provider, provider.requestUrl(documentUrl).href);
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
// @ref LLP 0013#search-fetch-and-optional-index-boundary [implements] — one per-call deadline and one request ledger for every outbound request
|
||||
/**
|
||||
* The outbound seam for a single MCP tool call: one deadline and one ledger,
|
||||
* wrapped around the fetch implementation every search and fetch path already
|
||||
* receives.
|
||||
*
|
||||
* Two problems made this necessary, and both are structural rather than
|
||||
* incidental:
|
||||
*
|
||||
* - No engine can bound a call's duration. OpenCode's `mcp.timeout` bounds tool
|
||||
* DISCOVERY, not execution, and Claude passes no per-call timeout at all. The
|
||||
* only hardcoded bound was 10s PER REDIRECT HOP, which multiplies: a page that
|
||||
* redirects five times costs 60s, and a four-provider search walking its
|
||||
* candidates costs minutes. The deadline has to live here.
|
||||
* - The audit reserved one budget unit per tool call and reported nothing about
|
||||
* what that unit spent. One call can issue four paid searches and sixteen page
|
||||
* downloads, so "8 calls" understated real traffic by more than an order of
|
||||
* magnitude.
|
||||
*
|
||||
* Wrapping fetch (rather than threading a signal through every signature) means
|
||||
* every existing caller is covered automatically, including paths added later.
|
||||
*/
|
||||
import type { FetchImplementation } from "./brave-search.js";
|
||||
|
||||
/** Discovery endpoints. Everything else is a documentation page or asset. */
|
||||
const SEARCH_ENDPOINTS = [
|
||||
"https://api.search.brave.com/",
|
||||
"https://qex7pb7d46-dsn.algolia.net/",
|
||||
"https://lysine.dev/okhttp/search/search_index.json",
|
||||
];
|
||||
|
||||
export interface ResearchNetworkCounts {
|
||||
/** Discovery requests, i.e. the ones that cost search-provider quota. */
|
||||
searchRequests: number;
|
||||
/** Documentation page and asset requests, excluding redirect hops. */
|
||||
documentRequests: number;
|
||||
/** Redirect responses followed. Each one is an extra round trip. */
|
||||
redirects: number;
|
||||
/** Every HTTP request actually issued, redirect hops included. */
|
||||
totalRequests: number;
|
||||
elapsedMs: number;
|
||||
}
|
||||
|
||||
export class ResearchDeadlineError extends Error {
|
||||
constructor(timeoutMs: number) {
|
||||
super(`Documentation research call exceeded its ${timeoutMs}ms deadline`);
|
||||
this.name = "ResearchDeadlineError";
|
||||
}
|
||||
}
|
||||
|
||||
export interface ResearchNetwork {
|
||||
/** Drop-in replacement for the caller's fetch: deadline-bound and counted. */
|
||||
fetch: FetchImplementation;
|
||||
/** The call deadline, for loops that should stop rather than fail per-item. */
|
||||
signal: AbortSignal;
|
||||
expired(): boolean;
|
||||
counts(): ResearchNetworkCounts;
|
||||
}
|
||||
|
||||
export function createResearchNetwork(
|
||||
base: FetchImplementation,
|
||||
timeoutMs: number,
|
||||
): ResearchNetwork {
|
||||
const startedAt = Date.now();
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(new ResearchDeadlineError(timeoutMs)), timeoutMs);
|
||||
// The MCP is a short-lived stdio process; never hold the loop open for this.
|
||||
timer.unref?.();
|
||||
|
||||
let searchRequests = 0;
|
||||
let documentRequests = 0;
|
||||
let redirects = 0;
|
||||
let totalRequests = 0;
|
||||
|
||||
const wrapped: FetchImplementation = async (input, init) => {
|
||||
if (controller.signal.aborted) throw new ResearchDeadlineError(timeoutMs);
|
||||
|
||||
const href = typeof input === "string" ? input : input instanceof URL ? input.href : input.url;
|
||||
const isSearch = SEARCH_ENDPOINTS.some((endpoint) => href.startsWith(endpoint));
|
||||
totalRequests++;
|
||||
if (isSearch) searchRequests++;
|
||||
else documentRequests++;
|
||||
|
||||
// Keep the caller's own per-attempt timeout AND add the call deadline, so a
|
||||
// single slow hop still fails fast while the whole call stays bounded.
|
||||
const signal = init?.signal
|
||||
? AbortSignal.any([init.signal, controller.signal])
|
||||
: controller.signal;
|
||||
|
||||
const response = await base(input, { ...init, signal });
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
redirects++;
|
||||
// A redirect hop is a round trip, not a distinct resource, so undo the
|
||||
// classification count. It has to be the SAME counter that was
|
||||
// incremented: the search backends use `redirect: "manual"` too and treat
|
||||
// a 3xx as an error, so unconditionally decrementing documentRequests
|
||||
// drives it negative the first time a discovery endpoint redirects.
|
||||
if (isSearch) searchRequests--;
|
||||
else documentRequests--;
|
||||
}
|
||||
return response;
|
||||
};
|
||||
|
||||
return {
|
||||
fetch: wrapped,
|
||||
signal: controller.signal,
|
||||
expired: () => controller.signal.aborted,
|
||||
counts: () => ({
|
||||
searchRequests,
|
||||
documentRequests,
|
||||
redirects,
|
||||
totalRequests,
|
||||
elapsedMs: Date.now() - startedAt,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
/** Sum per-call ledgers into one review-wide total. */
|
||||
export function totalResearchNetwork(
|
||||
counts: readonly ResearchNetworkCounts[],
|
||||
): ResearchNetworkCounts {
|
||||
return counts.reduce<ResearchNetworkCounts>(
|
||||
(total, entry) => ({
|
||||
searchRequests: total.searchRequests + entry.searchRequests,
|
||||
documentRequests: total.documentRequests + entry.documentRequests,
|
||||
redirects: total.redirects + entry.redirects,
|
||||
totalRequests: total.totalRequests + entry.totalRequests,
|
||||
elapsedMs: total.elapsedMs + entry.elapsedMs,
|
||||
}),
|
||||
{ searchRequests: 0, documentRequests: 0, redirects: 0, totalRequests: 0, elapsedMs: 0 },
|
||||
);
|
||||
}
|
||||
@@ -73,7 +73,7 @@ async function loadOkHttpSearchIndex(
|
||||
}
|
||||
});
|
||||
const chunks = documents.flatMap((document) => chunkDocument(document, indexedAt));
|
||||
return buildSearchIndex(chunks, documents.length, indexedAt);
|
||||
return buildSearchIndex(chunks);
|
||||
}
|
||||
|
||||
function cachedOkHttpSearchIndex(
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
export const packageRoot = fileURLToPath(new URL("../..", import.meta.url));
|
||||
export const defaultConfigPath = fileURLToPath(
|
||||
new URL("../../research/sources.json", import.meta.url),
|
||||
);
|
||||
export const defaultIndexPath = fileURLToPath(
|
||||
new URL("../../research/data/docs-index.json", import.meta.url),
|
||||
);
|
||||
@@ -98,6 +98,12 @@ export interface RemoteSearchOptions {
|
||||
fetchImplementation?: FetchImplementation;
|
||||
language?: Language;
|
||||
sourceKinds?: SourceKind[];
|
||||
/**
|
||||
* The call deadline. The wrapped fetch already fails once it passes, but the
|
||||
* candidate walk below should stop and return what it has rather than march
|
||||
* through every remaining candidate collecting identical timeout warnings.
|
||||
*/
|
||||
deadline?: AbortSignal;
|
||||
}
|
||||
|
||||
export interface RemoteSearchResponse {
|
||||
@@ -133,7 +139,7 @@ function bestPassage(
|
||||
if (chunks.length === 0) {
|
||||
return { passage: document.body.slice(0, 1_400), relevance: 0 };
|
||||
}
|
||||
const index = buildSearchIndex(chunks, 1, indexedAt);
|
||||
const index = buildSearchIndex(chunks);
|
||||
const result = searchDocumentation(index, query, {
|
||||
platform: document.platform,
|
||||
providers: document.provider ? [document.provider] : undefined,
|
||||
@@ -229,7 +235,11 @@ export async function searchRemoteDocumentation(
|
||||
// downloading every discovery candidate.
|
||||
const fetched: SearchResult[] = [];
|
||||
let candidateIndex = 0;
|
||||
while (fetched.length < limit && candidateIndex < candidates.length) {
|
||||
while (
|
||||
fetched.length < limit &&
|
||||
candidateIndex < candidates.length &&
|
||||
!options.deadline?.aborted
|
||||
) {
|
||||
const outstanding = limit - fetched.length;
|
||||
const batch = candidates.slice(candidateIndex, candidateIndex + outstanding);
|
||||
candidateIndex += batch.length;
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
import MiniSearch, { type SearchOptions } from "minisearch";
|
||||
|
||||
import type {
|
||||
@@ -10,7 +7,6 @@ import type {
|
||||
PlatformFilter,
|
||||
ProviderId,
|
||||
SearchResult,
|
||||
SerializedSearchIndex,
|
||||
SourceKind,
|
||||
} from "./types.js";
|
||||
|
||||
@@ -35,51 +31,18 @@ const miniSearchOptions = {
|
||||
};
|
||||
|
||||
export interface BuiltSearchIndex {
|
||||
serialized: SerializedSearchIndex;
|
||||
miniSearch: MiniSearch<IndexedChunk>;
|
||||
}
|
||||
|
||||
export function buildSearchIndex(
|
||||
chunks: IndexedChunk[],
|
||||
documentCount: number,
|
||||
generatedAt = new Date().toISOString(),
|
||||
): BuiltSearchIndex {
|
||||
/**
|
||||
* Build a throwaway in-memory index over one fetch's chunks, purely to rank them.
|
||||
* Nothing is serialized or persisted: since the offline index was removed, every
|
||||
* caller builds this per request and discards it.
|
||||
*/
|
||||
export function buildSearchIndex(chunks: IndexedChunk[]): BuiltSearchIndex {
|
||||
const miniSearch = new MiniSearch<IndexedChunk>(miniSearchOptions);
|
||||
miniSearch.addAll(chunks);
|
||||
|
||||
const providers = [
|
||||
...new Set(chunks.map((chunk) => chunk.provider ?? chunk.platform)),
|
||||
].sort() as ProviderId[];
|
||||
return {
|
||||
miniSearch,
|
||||
serialized: {
|
||||
schemaVersion: 1,
|
||||
generatedAt,
|
||||
documentCount,
|
||||
chunkCount: chunks.length,
|
||||
providers,
|
||||
searchIndex: miniSearch.toJSON(),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export async function writeSearchIndex(filePath: string, index: SerializedSearchIndex) {
|
||||
await mkdir(path.dirname(filePath), { recursive: true });
|
||||
const temporaryPath = `${filePath}.tmp-${process.pid}`;
|
||||
await writeFile(temporaryPath, `${JSON.stringify(index)}\n`, { mode: 0o644 });
|
||||
await rename(temporaryPath, filePath);
|
||||
}
|
||||
|
||||
export async function loadSearchIndex(filePath: string): Promise<BuiltSearchIndex> {
|
||||
const serialized = JSON.parse(await readFile(filePath, "utf8")) as SerializedSearchIndex;
|
||||
if (serialized.schemaVersion !== 1 || typeof serialized.searchIndex !== "object") {
|
||||
throw new Error(`Unsupported or invalid search index at ${filePath}`);
|
||||
}
|
||||
const miniSearch = MiniSearch.loadJSON<IndexedChunk>(
|
||||
JSON.stringify(serialized.searchIndex),
|
||||
miniSearchOptions,
|
||||
);
|
||||
return { serialized, miniSearch };
|
||||
return { miniSearch };
|
||||
}
|
||||
|
||||
function searchOptions(combineWith: "AND" | "OR", exact = false): SearchOptions {
|
||||
|
||||
+25
-28
@@ -4,6 +4,7 @@ import { z } from "zod";
|
||||
|
||||
import { ResearchAudit } from "./audit.js";
|
||||
import type { FetchImplementation } from "./brave-search.js";
|
||||
import { createResearchNetwork } from "./network.js";
|
||||
import { sanitizeDocumentationQuery } from "./query-sanitizer.js";
|
||||
import {
|
||||
DIRECT_DOCUMENT_CONTEXT_MODES,
|
||||
@@ -14,7 +15,6 @@ import { searchExpoAlgolia } from "./expo-algolia.js";
|
||||
import { searchOkHttpDocumentation } from "./okhttp-search.js";
|
||||
import { getProvider, resolveAllowedUrl } from "./providers.js";
|
||||
import { searchRemoteDocumentation } from "./remote-search.js";
|
||||
import { loadSearchIndex, searchDocumentation } from "./search-index.js";
|
||||
import { LANGUAGES, PROVIDERS, SOURCE_KINDS } from "./types.js";
|
||||
import type { PlatformFilter, ProviderId, SearchResult } from "./types.js";
|
||||
|
||||
@@ -28,12 +28,13 @@ const providerGuidance =
|
||||
"Provider map: apple=Apple SDK APIs and Human Interface Guidelines; apple-releases=Xcode and Apple platform release notes; swift-evolution=Swift Evolution proposals; sdwebimage=SDWebImage APIs and caching/loading behavior; android=Android, Jetpack, Compose, and Google Play services APIs; android-releases=Android platform releases and behavior changes; media3=Jetpack Media3; glide=Glide; okhttp=OkHttp; kotlin-coroutines=Kotlin coroutines; gradle=Gradle; agp=Android Gradle Plugin; jetbrains-issues=JetBrains YouTrack context; expo=Expo documentation; react-native=React Native core; react-native-reanimated=Reanimated; react-native-gesture-handler=Gesture Handler; react-native-screens=Screens; react-native-worklets=Worklets. Native source retains platform context: use apple/android for OS contracts and add the dependency provider for dependency-owned behavior; an Expo package path does not make a native API an Expo-docs query. Issue-tracker results are context, not API contracts.";
|
||||
|
||||
export interface DocumentationServerOptions {
|
||||
indexPath?: string;
|
||||
braveApiKey?: string;
|
||||
fetchImplementation?: FetchImplementation;
|
||||
auditPath?: string;
|
||||
maxCalls?: number;
|
||||
maxResultsPerCall?: number;
|
||||
/** End-to-end deadline for one tool call. Neither engine can enforce this. */
|
||||
timeoutMs?: number;
|
||||
}
|
||||
|
||||
function defaultProviders(platform: PlatformFilter): ProviderId[] {
|
||||
@@ -44,10 +45,11 @@ function defaultProviders(platform: PlatformFilter): ProviderId[] {
|
||||
}
|
||||
|
||||
export async function createDocumentationServer(options: DocumentationServerOptions = {}) {
|
||||
const index = options.indexPath ? await loadSearchIndex(options.indexPath) : undefined;
|
||||
const maxCalls = Math.min(20, Math.max(1, options.maxCalls ?? 8));
|
||||
const maxResultsPerCall = Math.min(3, Math.max(1, options.maxResultsPerCall ?? 3));
|
||||
const timeoutMs = Math.min(60_000, Math.max(1_000, options.timeoutMs ?? 30_000));
|
||||
const audit = new ResearchAudit(options.auditPath, maxCalls);
|
||||
const baseFetch = options.fetchImplementation ?? fetch;
|
||||
const server = new McpServer({
|
||||
name: "review-research-mcp",
|
||||
version: "0.2.0",
|
||||
@@ -57,7 +59,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
"search_platform_docs",
|
||||
{
|
||||
title: "Search official platform documentation",
|
||||
description: `Search official platform, dependency, build-tool, and release documentation. Discovery uses scoped web search (or Expo's public documentation search), then fetches only allowlisted official pages; an optional local index is fallback evidence. Returns short passages with canonical source URLs. ${providerGuidance} ${queryGuidance}`,
|
||||
description: `Search official platform, dependency, build-tool, and release documentation. Discovery uses scoped web search (or Expo's public documentation search), then fetches only allowlisted official pages. Returns short passages with canonical source URLs. ${providerGuidance} ${queryGuidance}`,
|
||||
inputSchema: {
|
||||
platform: z
|
||||
.enum(["apple", "android", "react-native", "all"])
|
||||
@@ -115,16 +117,9 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
query: sanitizedQuery,
|
||||
};
|
||||
const requestId = await audit.reserve("search_platform_docs", auditInput);
|
||||
// One deadline and one request ledger for everything this call issues.
|
||||
const network = createResearchNetwork(baseFetch, timeoutMs);
|
||||
try {
|
||||
const localResults = index
|
||||
? searchDocumentation(index, sanitizedQuery, {
|
||||
platform,
|
||||
limit: boundedLimit,
|
||||
providers: selectedProviders,
|
||||
...(sourceKinds ? { sourceKinds } : {}),
|
||||
...(language ? { language } : {}),
|
||||
})
|
||||
: [];
|
||||
const warnings: string[] = [];
|
||||
const remoteResults: SearchResult[] = [];
|
||||
const perProviderLimit = Math.max(1, Math.ceil(boundedLimit / selectedProviders.length));
|
||||
@@ -145,7 +140,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
const documents = await searchExpoAlgolia(
|
||||
sanitizedQuery,
|
||||
perProviderLimit,
|
||||
options.fetchImplementation ?? fetch,
|
||||
network.fetch,
|
||||
);
|
||||
return {
|
||||
results: documents.map((document, position) => ({
|
||||
@@ -178,7 +173,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
const results = await searchOkHttpDocumentation(
|
||||
sanitizedQuery,
|
||||
perProviderLimit,
|
||||
options.fetchImplementation ?? fetch,
|
||||
network.fetch,
|
||||
);
|
||||
if (results.length > 0) {
|
||||
return { results, warnings: [] as string[] };
|
||||
@@ -199,9 +194,8 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
try {
|
||||
return await searchRemoteDocumentation(provider, sanitizedQuery, perProviderLimit, {
|
||||
apiKey: options.braveApiKey,
|
||||
...(options.fetchImplementation
|
||||
? { fetchImplementation: options.fetchImplementation }
|
||||
: {}),
|
||||
fetchImplementation: network.fetch,
|
||||
deadline: network.signal,
|
||||
...(language ? { language } : {}),
|
||||
...(sourceKinds ? { sourceKinds } : {}),
|
||||
});
|
||||
@@ -220,7 +214,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
}
|
||||
|
||||
const seen = new Set<string>();
|
||||
const results = [...remoteResults, ...localResults]
|
||||
const results = remoteResults
|
||||
.filter((result) => {
|
||||
if (!result.provider) return false;
|
||||
try {
|
||||
@@ -235,17 +229,14 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
})
|
||||
.slice(0, boundedLimit);
|
||||
const uniqueWarnings = [...new Set(warnings)].slice(0, 10);
|
||||
const network_ = network.counts();
|
||||
const payload = {
|
||||
notice: untrustedMaterialNotice,
|
||||
retrieval: {
|
||||
scopedWebSearch: Boolean(options.braveApiKey),
|
||||
// What this single budget unit actually cost.
|
||||
network: network_,
|
||||
expoSearch: selectedProviders.includes("expo"),
|
||||
localIndex: index
|
||||
? {
|
||||
generatedAt: index.serialized.generatedAt,
|
||||
providers: index.serialized.providers,
|
||||
}
|
||||
: null,
|
||||
},
|
||||
...(uniqueWarnings.length > 0 ? { warnings: uniqueWarnings } : {}),
|
||||
results,
|
||||
@@ -256,6 +247,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
auditInput,
|
||||
results,
|
||||
uniqueWarnings,
|
||||
network_,
|
||||
);
|
||||
return {
|
||||
content: [{ type: "text", text: JSON.stringify(payload, null, 2) }],
|
||||
@@ -339,16 +331,18 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
context,
|
||||
};
|
||||
const requestId = await audit.reserve("fetch_platform_doc", auditInput);
|
||||
// A single URL still costs up to six round trips through the redirect
|
||||
// chain, so this call needs the same deadline and ledger as a search.
|
||||
const network = createResearchNetwork(baseFetch, timeoutMs);
|
||||
try {
|
||||
const fetched = await fetchDocumentationUrl(target.url.href, {
|
||||
provider: target.provider,
|
||||
...(sanitizedQuery ? { query: sanitizedQuery } : {}),
|
||||
context,
|
||||
limit,
|
||||
...(options.fetchImplementation
|
||||
? { fetchImplementation: options.fetchImplementation }
|
||||
: {}),
|
||||
fetchImplementation: network.fetch,
|
||||
});
|
||||
const network_ = network.counts();
|
||||
const payload = {
|
||||
notice: untrustedMaterialNotice,
|
||||
retrieval: {
|
||||
@@ -357,6 +351,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
sourceKind: fetched.sourceKind,
|
||||
canonicalUrl: fetched.canonicalUrl,
|
||||
context: fetched.context,
|
||||
network: network_,
|
||||
},
|
||||
results: fetched.results,
|
||||
};
|
||||
@@ -370,6 +365,8 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
url: fetched.canonicalUrl,
|
||||
},
|
||||
fetched.results,
|
||||
[],
|
||||
network_,
|
||||
);
|
||||
return {
|
||||
content: [{ type: "text", text: JSON.stringify(payload, null, 2) }],
|
||||
|
||||
@@ -82,25 +82,3 @@ export interface SerializedSearchIndex {
|
||||
providers: ProviderId[];
|
||||
searchIndex: unknown;
|
||||
}
|
||||
|
||||
export interface CrawlLimits {
|
||||
maxPagesPerProvider: number;
|
||||
maxDepth: number;
|
||||
delayMs: number;
|
||||
timeoutMs: number;
|
||||
maxResponseBytes: number;
|
||||
}
|
||||
|
||||
export interface SourceDefinition {
|
||||
provider: ProviderId;
|
||||
sourceKind: SourceKind;
|
||||
seedUrls: string[];
|
||||
maxPages?: number;
|
||||
maxDepth?: number;
|
||||
}
|
||||
|
||||
export interface SourcesConfig {
|
||||
output: string;
|
||||
crawl: CrawlLimits;
|
||||
sources: SourceDefinition[];
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env node
|
||||
// @ref LLP 0013#one-package-two-binaries [implements] — the environment boundary the engine's MCP config cannot provide
|
||||
/**
|
||||
* Environment boundary for the bounded documentation MCP.
|
||||
*
|
||||
* ECR declares a minimal `env` in the engine's MCP configuration, but neither
|
||||
* engine treats that as a replacement: Claude Code and OpenCode both MERGE it
|
||||
* onto the environment the engine already holds. Verified by spawning each
|
||||
* engine against a probe MCP that records variable names — the child saw the
|
||||
* engine's model credential on both, and OpenCode additionally passed through
|
||||
* the runner's whole ambient environment.
|
||||
*
|
||||
* So the boundary has to be ours. The engine spawns this wrapper; the wrapper
|
||||
* spawns the real server with an environment it CONSTRUCTS. Whatever the engine
|
||||
* merged in reaches this process and stops here.
|
||||
*
|
||||
* This file deliberately does almost nothing. It loads no HTML/JSON parser and
|
||||
* opens no socket, because it is the one process in the chain that still holds
|
||||
* the engine's credentials. Everything that touches untrusted remote content
|
||||
* runs in the child, which never receives them.
|
||||
*/
|
||||
import { spawn } from "node:child_process";
|
||||
import { existsSync } from "node:fs";
|
||||
import { constants } from "node:os";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { researchWrapperEnvironment } from "./child-env.js";
|
||||
|
||||
const builtEntry = fileURLToPath(new URL("./cli.js", import.meta.url));
|
||||
const sourceEntry = fileURLToPath(new URL("./cli.ts", import.meta.url));
|
||||
|
||||
const child = spawn(
|
||||
// The current interpreter by absolute path, never a PATH lookup: during a
|
||||
// review the cwd is the untrusted PR-head tree.
|
||||
process.execPath,
|
||||
[existsSync(builtEntry) ? builtEntry : sourceEntry, ...process.argv.slice(2)],
|
||||
{
|
||||
env: researchWrapperEnvironment(process.env),
|
||||
// The child owns the engine's stdio directly, so the wrapper never sits in
|
||||
// the MCP byte stream and cannot truncate, buffer, or reorder a message.
|
||||
stdio: "inherit",
|
||||
},
|
||||
);
|
||||
|
||||
const FORWARDED_SIGNALS = ["SIGINT", "SIGTERM", "SIGHUP"] as const;
|
||||
for (const signal of FORWARDED_SIGNALS) {
|
||||
// The engine signals the process it spawned — us. Pass it on, or the server
|
||||
// outlives the review and keeps its audit lock.
|
||||
process.on(signal, () => {
|
||||
child.kill(signal);
|
||||
});
|
||||
}
|
||||
|
||||
child.on("error", (error: Error) => {
|
||||
process.stderr.write(`review-research-mcp: failed to start bounded server: ${error.message}\n`);
|
||||
process.exitCode = 1;
|
||||
});
|
||||
|
||||
child.on("exit", (code, signal) => {
|
||||
// Report a signal death as the conventional 128+n rather than a silent 0, so a
|
||||
// killed server is distinguishable from a clean shutdown.
|
||||
process.exitCode = signal ? 128 + (constants.signals[signal] ?? 0) : (code ?? 1);
|
||||
});
|
||||
+15
-4
@@ -26,10 +26,21 @@
|
||||
|
||||
// Optional bounded platform research (ROOT-ONLY; off by default). Reviewer and
|
||||
// cross-file passes can call ECR's bundled MCP for exact API-symbol searches and
|
||||
// supported documentation URLs. The MCP sanitizes queries, uses fixed provider
|
||||
// allowlists, audits results, and never receives model credentials. BRAVE_SEARCH_API_KEY
|
||||
// enables fixed site-scoped discovery; Expo uses its public documentation search.
|
||||
// indexPath remains an optional offline fallback only.
|
||||
// supported documentation URLs. The MCP uses fixed provider allowlists and audits
|
||||
// every call. It runs behind a wrapper that rebuilds its environment from an
|
||||
// explicit allowlist, because both engines MERGE the configured env onto their own
|
||||
// rather than replacing it — so the server sees the search key and these limits,
|
||||
// and never the model credential. BRAVE_SEARCH_API_KEY enables fixed site-scoped
|
||||
// discovery; Expo uses its public documentation search.
|
||||
//
|
||||
// Queries are shape-checked, not confidentiality-checked: the reviewing model
|
||||
// chooses the outbound terms, so enable this only where repository-derived terms
|
||||
// may be shared with Brave and the documentation providers.
|
||||
//
|
||||
// maxQueries bounds MCP CALLS, not requests — one search can issue a discovery
|
||||
// request per provider plus a page fetch per candidate. timeoutMs is the MCP's own
|
||||
// end-to-end deadline per call. Every passage is fetched live from the allowlist;
|
||||
// there is no offline index.
|
||||
// "research": {
|
||||
// "enabled": true,
|
||||
// "maxQueries": 8,
|
||||
|
||||
Reference in new issue
Block a user