Compare commits

...
1 Commits
Author SHA1 Message Date
Brent Vatne 0648bac4b7 Improve documentation research grounding (#64) 2026-08-07 16:27:13 -07:00
30 changed files with 1066 additions and 111 deletions

No files matched your search

+32 -3
View File
@@ -232,20 +232,41 @@ configs cannot alter its network behavior or limits. Result-cache reuse remains
disabled while research is enabled because web results and documentation can change
without a config change.
The fixed provider catalog covers Apple/Android APIs plus Media3, Glide, OkHttp,
The fixed provider catalog covers Apple/Android APIs plus SDWebImage, Media3, Glide, OkHttp,
Kotlin coroutines, Gradle/AGP, Swift concurrency/evolution, platform release/API
availability, Expo, React Native, Reanimated, Gesture Handler, Screens, and Worklets.
Queries are short exact symbols plus at most one useful member or behavior term. For
example, `CameraView barcodeScannerSettings` is useful; a source snippet, import path,
or natural-language question is not. The MCP publishes the same guidance in its tool
metadata. An empty result stays empty; it is not replaced with a loose semantic guess.
The tool metadata also includes an explicit provider map, so the reviewing model can
distinguish core platform APIs from release notes, dependency-owned documentation,
build-tool references, and issue-tracker context before choosing a corpus.
Reviewer instructions require grounding whenever a judgment depends on an externally
owned API contract, whether the evidence confirms a finding or dismisses a candidate
as safe; model memory alone is not treated as sufficient for those decisions.
Native source keeps its platform context: Apple or Android documents the OS contract,
while an explicit dependency provider documents library-owned behavior. Providers are
additive when both contracts matter. A path under `packages/expo-*` does not by itself
route Swift or Kotlin code to Expo's JavaScript documentation.
Direct clients can also call `fetch_platform_doc` with an exact documentation URL.
The tool infers the narrowest matching provider (or accepts an explicit provider
hint), then applies the same fixed HTTPS host/path allowlist, manual redirect checks,
10-second timeout, 5 MB response limit, content-type validation, extraction, and
passage bounds as search-discovered pages. An optional `query` ranks passages only
within that one page; it never broadens discovery. For example,
passage bounds as search-discovered pages. It returns normalized extracted text, never
raw HTML or DocC JSON. An optional `query` selects context only within that one page;
it never broadens discovery. Context expands progressively:
- `focused` returns the best passage plus adjacent passages.
- `section` (the default) returns a contiguous window of at most 12,000 characters
around the best passage.
- `document` returns at most 20,000 characters of extracted page text and should be
used only when the contract is spread across the page.
The response reports returned and original character counts, whether it was truncated,
the anchor passage id, and bounded available passage ids. Search results also carry
neighboring passage ids so an agent can recognize when more local context exists. For example,
`https://developer.apple.com/documentation/swiftui/view/menustyle(_:)` is resolved to
Apple's DocC JSON and returned with the canonical page URL and API availability.
@@ -258,6 +279,14 @@ finding. ECR accepts only exact URLs returned during that review, restores canon
titles, carries citations through coordination, and renders them below the finding;
invented or unrelated citations are dropped.
Reviewers also emit a bounded `researchDecisions` record only when documentation
materially confirms a finding candidate or proves one safe. ECR grounds those records
against the exact MCP audit and discards ungrounded claims. After verification and
suppression, the log and Actions summary report final findings with citations,
supported and dismissed candidates, and unique audited results materially used versus
unused. Counts use canonical URLs rather than passage count, so repeated hits do not
inflate usefulness.
For a query routed to the `expo` provider, `serve` POSTs the already-sanitized query
directly to Expo's public Algolia search endpoint and returns canonical
`docs.expo.dev` hits. The endpoint, application id, and browser-visible search-only
+55 -25
View File
@@ -8,9 +8,11 @@
**Related:** [LLP 0001](0001-trust-model.principles.md), [LLP 0002](0002-review-engine-pipeline.explainer.md), [LLP 0003](0003-model-runtimes-and-credentials.explainer.md), [LLP 0006](0006-config-schema-loading-routing.explainer.md), [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md)
`@expo/code-review-cli` ships a documentation-search MCP as a second binary in
the same npm package. ECR invokes it before model startup and passes the resulting
passages to reviewer prompts as bounded, explicitly untrusted evidence. The model
never receives the MCP as an agent tool.
the same npm package. ECR exposes that bounded MCP directly to reviewer and cross-file
passes. Those agents decide when an external contract needs evidence, search for a
precise symbol, or fetch an exact supported documentation URL already present in the
review context. Coordinator, verifier, stack-verifier, and no-tools fallback passes do
not receive the MCP.
## One Package, Two Binaries
@@ -22,9 +24,10 @@ supplies only enablement, bounds, and an optional absolute fallback index path
arbitrary command, endpoint, host, or argument vector.
The MCP remains a process boundary rather than an in-process import. Its stdout is
the only result channel, its environment is an allowlist, it runs from the OS temp
directory, and ECR caps its runtime and output. A failure is logged and review
continues without research; it never skips or weakens an ordinary review pass.
the only result channel, its environment is an allowlist, and ECR caps calls and
results. Each run uses an owner-only temporary MCP configuration and append-only audit.
A tool failure remains local to research and never skips or weakens an ordinary review
pass.
## Search, Fetch, and Optional Index Boundary
@@ -52,8 +55,17 @@ or accepts a fixed provider id as a hint when corpora overlap. The supplied URL
pass that provider's canonical HTTPS host/path allowlist before network access. The
request adapter, every redirect, content type, timeout, and response size then pass
the same checks as a search-discovered document. The tool fetches one page only and
returns at most five bounded extracted passages. An optional query ranks passages
within the fetched page; it cannot turn the operation into discovery.
returns normalized extracted text—never raw HTML or DocC JSON. An optional query
selects context within the page; it cannot turn the operation into discovery.
Direct fetch uses progressive disclosure. `focused` returns a small contiguous group
around the best passage. `section`, the default, returns at most 12,000 contiguous
characters around that match. `document` returns at most 20,000 extracted characters
and is reserved for contracts spread across a page. Every response reports the
extracted document length, returned length, truncation, anchor passage id, and a
bounded passage-id inventory. Search passages carry adjacent ids so a reviewer can
recognize missing local context and expand the canonical URL instead of broadening
discovery.
This path is useful for documentation links already present in review context. For
Apple symbol links it converts the canonical `/documentation/...` URL into Apple's
@@ -93,40 +105,58 @@ documentation, and optional index content can change without a config change.
## Query and Prompt Boundary
ECR derives queries only from added native-code identifiers. It removes comments,
string literals, deleted lines, file paths, and raw snippets; caps identifiers and
query count; and selects named providers from known code signals. The MCP tool
metadata tells direct clients to use exact symbols plus one member, behavior, or
constraint term, and to retry broad results with a narrower symbol. It explicitly
rejects prose, code, package/import names, paths, secrets, and sensitive context as
query material. MCP output is
schema-validated, limited to HTTPS URLs from the requested provider, truncated,
sanitized, and wrapped in an untrusted platform-research fence. The Brave credential
is never written to MCP output, prompts, logs, or run artifacts. Issue tracker text
keeps distinct provenance and is never presented as an API contract.
Reviewer prompts and MCP tool metadata tell direct clients to use exact symbols plus
one member, behavior, or constraint term, and to retry a broad result with at most one
narrower query. The MCP then deterministically removes quoted literals, URLs, email
addresses, paths, prose stop words, overlong or high-entropy tokens, and unsupported
punctuation. Credential-shaped or secret-labeled input fails closed. The remaining
query must contain an API-like symbol, stay under eight short tokens, and fit the
review-wide call budget.
Provider selection preserves native platform ownership. Swift or Objective-C source
uses Apple for OS contracts; Kotlin, Java, and Gradle source uses Android for platform
contracts. Explicit library signals add the owning provider—such as SDWebImage,
Media3, Glide, or OkHttp—when library behavior also matters. Repository ownership is
not documentation ownership, so a `packages/expo-*` path does not replace Apple or
Android with Expo documentation.
MCP output is schema-validated, limited to HTTPS URLs from the requested provider,
truncated, and sanitized. Returned passages are labeled untrusted reference data in
the reviewer prompt. The Brave credential is scoped to the MCP child and is never
written to MCP output, prompts, logs, or run artifacts. Issue tracker text keeps
distinct provenance and is never presented as an API contract.
## Research Provenance and Citations
Research is observable without exposing the Brave key or full responses. Each review
Research is observable without exposing the Brave key or unbounded responses. Each review
prints the exact bounded queries and every accepted result's title, provider,
provenance class, and canonical URL. GitHub Actions receives that same data in its
step summary. The JSONL run log additionally stores at most the already-bounded
passage returned to the reviewer so operators can determine whether research was
useful after the fact; it still excludes PR title/body and model transcripts.
The prompt may select a source for a finding only by copying its exact title and URL
from injected research evidence. Model output crosses an engine-side grounding seam:
URLs absent from the trusted prepass are removed and accepted titles are restored
from the canonical evidence. Grounded sources are unioned across duplicate reviewer
The reviewer may select a source for a finding only by copying its exact title and URL
from an MCP result. Model output crosses an engine-side grounding seam: URLs absent
from the run's append-only MCP audit are removed and accepted titles are restored from
canonical evidence. Grounded sources are unioned across duplicate reviewer
findings, preserved through coordinator rewrites by the finding fingerprint, stored
in hidden comment state, and rendered as visible links. Sources do not participate
in fingerprints or severity/decision logic, and findings that did not materially use
research omit them.
A reviewer may also emit a bounded, conclusion-only `researchDecisions` record when
documentation materially confirms a finding candidate or proves one safe. These
records never enter policy or decision logic. ECR discards any record without an exact
audited source. After verification and suppression produce the final findings, ECR
counts final cited findings, supported and dismissed candidates, and unique result
URLs materially used versus unused. The Actions summary displays those metrics and
grounded candidate conclusions; the JSONL run record preserves the same structured
data with the queries and bounded results.
## Installation-Specific Research Is Deferred
The built-in source catalog covers Apple, Android, platform releases, Swift
Evolution, Media3, Glide, OkHttp, Kotlin coroutines, Gradle/AGP, selected JetBrains
Evolution, SDWebImage, Media3, Glide, OkHttp, Kotlin coroutines, Gradle/AGP, selected JetBrains
issues, Expo, React Native, Reanimated, Gesture Handler, Screens, and Worklets.
The canonical OkHttp host is backed by both the
[maintainer's transfer announcement](https://jakewharton.com/the-lysine-contingency/)
+12
View File
@@ -70,6 +70,18 @@
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0430-transferring-parameters-and-results.md"
]
},
{
"provider": "sdwebimage",
"sourceKind": "official-api",
"maxPages": 30,
"maxDepth": 2,
"seedUrls": [
"https://sdwebimage.github.io/documentation/sdwebimage/",
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimageoptions/",
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagecontextoption/"
]
},
{
"provider": "android",
"sourceKind": "official-api",
+13
View File
@@ -55,11 +55,24 @@ test("research prompts require exact, material citations and preserve them throu
test("agent-visible research instructions teach precise, sanitized MCP use", () => {
const section = platformResearchToolsSection(true).join("\n");
const prose = section.replace(/\s+/g, " ");
expect(section).toContain("`fetch_platform_doc`");
expect(section).toContain("`search_platform_docs`");
expect(section).toContain("Never send source text, prose, literals, paths, URLs, credentials");
expect(section).toContain("UNTRUSTED reference data");
expect(section).toContain("exact returned title");
expect(prose).toContain("do not rely on model memory");
expect(prose).toContain("reporting");
expect(prose).toContain("dismissing the candidate as safe");
expect(prose).toContain("availability/version gates, lifecycle, threading, permissions");
expect(section).toContain("`researchDecisions`");
expect(section).toContain("`supported-finding`");
expect(section).toContain("`dismissed-candidate`");
expect(section).toContain("Omit generic context and unused results");
expect(prose).toContain("Native source retains platform context");
expect(section).toContain("`packages/expo-*`");
expect(section).toContain("`section` context");
expect(section).toContain("`document` only when");
expect(platformResearchToolsSection(false)).toEqual([]);
});
+2 -1
View File
@@ -28,7 +28,7 @@ test("research audit shares one call budget and records bounded completed result
sourceKind: "official-api",
title: "MainActor",
url: "https://developer.apple.com/documentation/swift/mainactor",
passage: "Main actor documentation.",
passage: `Main actor documentation.${"x".repeat(25_000)}`,
indexedAt: "2026-08-07T00:00:00.000Z",
},
]);
@@ -39,6 +39,7 @@ test("research audit shares one call budget and records bounded completed result
const records = await readResearchAudit(auditPath);
expect(records).toHaveLength(2);
expect(records[0]?.results[0]?.title).toBe("MainActor");
expect(records[0]?.results[0]?.passage).toHaveLength(20_000);
expect(records[1]?.error).toBe("no page");
} finally {
await rm(directory, { recursive: true, force: true });
@@ -68,6 +68,118 @@ test("direct fetch accepts an allowlisted SwiftUI symbol URL and uses DocC JSON"
expect(result.results[0]?.passage).toContain("menu style");
});
test("direct fetch resolves SDWebImage's JavaScript route through its static DocC JSON", async () => {
const requested: string[] = [];
const result = await fetchDocumentationUrl(
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
{
limit: 1,
fetchImplementation: async (input) => {
const url = new URL(input instanceof Request ? input.url : input.toString());
requested.push(url.href);
return new Response(
JSON.stringify({
metadata: {
title: "SDWebImageManager",
role: "symbol",
modules: [{ name: "SDWebImage" }],
},
identifier: { interfaceLanguage: "swift" },
abstract: [
{
type: "text",
text: "Coordinates asynchronous image downloading with the image cache.",
},
],
primaryContentSections: [],
}),
{ headers: { "content-type": "application/json" } },
);
},
},
);
expect(requested).toEqual([
"https://sdwebimage.github.io/data/documentation/sdwebimage/sdwebimagemanager.json",
]);
expect(result).toMatchObject({
provider: "sdwebimage",
sourceKind: "official-api",
canonicalUrl: "https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
});
expect(result.results[0]).toMatchObject({
title: "SDWebImageManager",
framework: "SDWebImage",
language: "swift",
});
});
test("direct fetch progressively expands focused, section, and document context", async () => {
const paragraphs = [
...Array.from(
{ length: 18 },
(_, index) =>
`<p>Prelude ${index}: lifecycle background information ${"before ".repeat(80)}</p>`,
),
`<p>UniqueTargetMember behavior: the callback runs only after the owner becomes active. ${"contract ".repeat(80)}</p>`,
...Array.from(
{ length: 18 },
(_, index) =>
`<p>Follow-up ${index}: compatibility background information ${"after ".repeat(80)}</p>`,
),
].join("\n");
const fetchImplementation = async () =>
new Response(
`<!doctype html><html><head><title>TargetSymbol | Android Developers</title></head><body><main><h1>TargetSymbol</h1>${paragraphs}</main></body></html>`,
{ headers: { "content-type": "text/html" } },
);
const url = "https://developer.android.com/reference/android/app/Activity";
const focused = await fetchDocumentationUrl(url, {
query: "UniqueTargetMember behavior",
context: "focused",
limit: 3,
fetchImplementation,
});
expect(focused.results).toHaveLength(3);
expect(focused.results.some((result) => result.passage.includes("UniqueTargetMember"))).toBe(
true,
);
expect(focused.results.some((result) => result.previousPassageId || result.nextPassageId)).toBe(
true,
);
expect(focused.context).toMatchObject({
mode: "focused",
truncated: true,
});
expect(focused.context.anchorPassageId).toMatch(/#\d+$/);
const section = await fetchDocumentationUrl(url, {
query: "UniqueTargetMember behavior",
context: "section",
fetchImplementation,
});
expect(section.results).toHaveLength(1);
expect(section.results[0]?.passage).toContain("UniqueTargetMember");
expect(section.context.returnedCharacters).toBeLessThanOrEqual(12_000);
expect(section.context.returnedCharacters).toBeGreaterThan(
focused.results.reduce((total, result) => total + result.passage.length, 0),
);
const document = await fetchDocumentationUrl(url, {
context: "document",
fetchImplementation,
});
expect(document.results).toHaveLength(1);
expect(document.context).toMatchObject({
mode: "document",
returnedCharacters: 20_000,
truncated: true,
});
expect(document.context.documentCharacters).toBeGreaterThan(20_000);
expect(document.context.availablePassageCount).toBeGreaterThan(3);
});
test("direct fetch rejects foreign hosts before network access", async () => {
let requests = 0;
await expect(
@@ -94,6 +206,11 @@ test("direct fetch revalidates redirects against the selected provider", async (
});
test("direct URL inference prefers narrow release and dependency providers", () => {
expect(
resolveDirectDocumentationTarget(
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
).provider,
).toBe("sdwebimage");
expect(
resolveDirectDocumentationTarget(
"https://developer.apple.com/documentation/xcode-release-notes/xcode-26-release-notes",
+2
View File
@@ -40,6 +40,8 @@ test("chunking produces bounded overlapping passages with stable identifiers", (
const chunks = chunkDocument(document, "2026-08-06T00:00:00.000Z", 500, 50);
expect(chunks.length).toBeGreaterThan(1);
expect(chunks[0]?.id ?? "").toMatch(/^android:[a-f0-9]{16}#0$/);
expect(chunks[0]?.nextPassageId).toBe(chunks[1]?.id);
expect(chunks[1]?.previousPassageId).toBe(chunks[0]?.id);
expect(
chunkDocument(document, "2026-08-06T00:00:00.000Z", 500, 50).map((chunk) => chunk.id),
).toEqual(chunks.map((chunk) => chunk.id));
+26 -1
View File
@@ -62,14 +62,39 @@ test("stdio MCP lists and calls the read-only documentation search tool", async
expect(searchTool?.annotations?.readOnlyHint).toBe(true);
expect(searchTool?.annotations?.openWorldHint).toBe(true);
expect(searchTool?.description ?? "").toMatch(/exact API symbols plus one behavior/);
expect(searchTool?.description ?? "").toContain(
"apple-releases=Xcode and Apple platform release notes",
);
expect(searchTool?.description ?? "").toContain("media3=Jetpack Media3");
expect(searchTool?.description ?? "").toContain(
"sdwebimage=SDWebImage APIs and caching/loading behavior",
);
expect(searchTool?.description ?? "").toContain("Native source retains platform context");
expect(searchTool?.description ?? "").toContain("react-native-worklets=Worklets");
expect(fetchTool?.annotations?.readOnlyHint).toBe(true);
expect(fetchTool?.description ?? "").toMatch(/every redirect/);
const inputSchema = searchTool?.inputSchema as {
properties?: { query?: { description?: string } };
properties?: {
query?: { description?: string };
providers?: { description?: string };
};
};
const fetchInputSchema = fetchTool?.inputSchema as {
properties?: { context?: { default?: string; description?: string } };
};
expect(inputSchema.properties?.query?.description ?? "").toMatch(
/CameraView barcodeScannerSettings/,
);
expect(inputSchema.properties?.providers?.description ?? "").toContain(
"agp=Android Gradle Plugin",
);
expect(inputSchema.properties?.providers?.description ?? "").toContain(
"jetbrains-issues=JetBrains YouTrack context",
);
expect(fetchInputSchema.properties?.context?.default).toBe("section");
expect(fetchInputSchema.properties?.context?.description ?? "").toContain(
"document=bounded extracted page text",
);
const response = await client.callTool({
name: "search_platform_docs",
@@ -15,6 +15,7 @@ import {
reactNativeWorkletsProvider,
resolveAllowedUrl,
resolveAllowedRequestUrl,
sdWebImageProvider,
swiftEvolutionProvider,
} from "../../research-mcp/providers.js";
@@ -72,6 +73,22 @@ test("dependency and issue providers stay on their exact owners and paths", () =
"https://lysine.dev/okhttp/features/interceptors",
);
expect(okHttpProvider.responseFormat(okHttpDoc)).toBe("html");
const sdWebImageDoc = resolveAllowedUrl(
sdWebImageProvider,
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
);
expect(sdWebImageDoc.hostname).toBe("sdwebimage.github.io");
expect(sdWebImageProvider.requestUrl(sdWebImageDoc).href).toBe(
"https://sdwebimage.github.io/data/documentation/sdwebimage/sdwebimagemanager.json",
);
expect(
sdWebImageProvider.acceptsRequest(
new URL("https://sdwebimage.github.io/data/documentation/sdwebimage/sdwebimagemanager.json"),
),
).toBe(true);
expect(sdWebImageDoc.href).toBe(
"https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
);
const proposal = resolveAllowedUrl(
swiftEvolutionProvider,
"https://github.com/swiftlang/swift-evolution/blob/main/proposals/0306-actors.md",
@@ -96,6 +113,8 @@ test("dependency and issue providers stay on their exact owners and paths", () =
[okHttpProvider, "https://github.com/square/okhttp/blob/main/docs/recipes.md"],
[okHttpProvider, "https://github.com/attacker/okhttp/blob/main/docs/recipes.md"],
[okHttpProvider, "https://lysine.dev/retrofit/"],
[sdWebImageProvider, "https://sdwebimage.github.io/documentation/other-framework/"],
[sdWebImageProvider, "https://github.com/SDWebImage/SDWebImage"],
[
swiftEvolutionProvider,
"https://github.com/attacker/swift-evolution/blob/main/proposals/fake.md",
@@ -110,6 +110,61 @@ test("Apple discovery searches Brave once, rejects foreign results, and fetches
expect(response.results[0]?.passage).toContain("setActive");
});
test("SDWebImage discovery stays on its official scope and fetches static DocC JSON", async () => {
const fetchImplementation = async (input: string | URL | Request): Promise<Response> => {
const url = new URL(input instanceof Request ? input.url : input.toString());
if (url.hostname === "api.search.brave.com") {
expect(url.searchParams.get("q") ?? "").toContain(
"site:sdwebimage.github.io/documentation/sdwebimage",
);
return jsonResponse({
web: {
results: [
{
title: "SDWebImageManager",
url: "https://sdwebimage.github.io/documentation/sdwebimage/sdwebimagemanager/",
},
],
},
});
}
expect(url.href).toBe(
"https://sdwebimage.github.io/data/documentation/sdwebimage/sdwebimagemanager.json",
);
return jsonResponse({
metadata: {
title: "SDWebImageManager",
role: "symbol",
modules: [{ name: "SDWebImage" }],
},
identifier: { interfaceLanguage: "swift" },
abstract: [
{
type: "text",
text: "Coordinates asynchronous image downloading with the image cache.",
},
],
primaryContentSections: [],
});
};
const response = await searchRemoteDocumentation(
"sdwebimage",
"SDWebImageManager shared cache",
1,
{ apiKey: "test-key", fetchImplementation },
);
expect(response.warnings).toEqual([]);
expect(response.results[0]).toMatchObject({
provider: "sdwebimage",
sourceKind: "official-api",
title: "SDWebImageManager",
framework: "SDWebImage",
language: "swift",
});
});
test("Android discovery fetches only allowlisted official HTML and returns a bounded passage", async () => {
let searchRequests = 0;
const fetchImplementation = async (input: string | URL | Request): Promise<Response> => {
@@ -19,6 +19,7 @@ const chunks: IndexedChunk[] = [
url: "https://developer.apple.com/documentation/swift/mainactor",
passage: "A singleton actor whose executor is equivalent to the main dispatch queue.",
language: "swift",
nextPassageId: "apple:2",
indexedAt: "2026-08-06T00:00:00.000Z",
},
{
@@ -47,6 +48,7 @@ test("a serialized index reloads and filters search results by platform", async
});
expect(appleResults).toHaveLength(1);
expect(appleResults[0]?.title).toBe("MainActor");
expect(appleResults[0]?.nextPassageId).toBe("apple:2");
const androidResults = searchDocumentation(loaded, "lifecycle callbacks", {
platform: "android",
+159
View File
@@ -6,13 +6,20 @@ import path from "node:path";
import { ReviewConfigSchema, ScopeReviewConfigSchema } from "../config/schema.js";
import { platformResearchSection } from "../core/prompts.js";
import {
boundResearchDecisions,
collectPlatformResearch,
deriveResearchQueries,
formatResearchEvidence,
formatResearchProgress,
formatResearchUsefulness,
groundResearchDecisions,
groundResearchSources,
renderResearchMarkdown,
renderResearchUsefulnessMarkdown,
researchChildEnvironment,
RESEARCH_DECISION_BYTES_LIMIT,
RESEARCH_DECISION_COUNT_LIMIT,
summarizeResearchUsefulness,
toResearchProvenance,
} from "../core/research.js";
import { buildSearchIndex, writeSearchIndex } from "../research-mcp/search-index.js";
@@ -58,6 +65,11 @@ test("routes Media3 and Swift concurrency identifiers to named corpora", () => {
},
]);
expect(queries).toContainEqual({
platform: "android",
providers: ["android"],
query: "MediaSessionService",
});
expect(queries).toContainEqual({
platform: "android",
providers: ["media3"],
@@ -66,6 +78,44 @@ test("routes Media3 and Swift concurrency identifiers to named corpora", () => {
expect(queries.some((query) => query.providers[0] === "swift-evolution")).toBe(true);
});
test("native Expo packages search the platform first and add dependency owners", () => {
const queries = deriveResearchQueries(
[
{
path: "packages/expo-image/ios/SVGVariablesImageLoader.swift",
patch: "+let manager = SDWebImageManager.shared",
},
{
path: "packages/expo-app-metrics/android/src/NetworkRequestInterceptor.kt",
patch: "+val client = OkHttpClient.Builder().build()",
},
],
20,
);
expect(queries).toContainEqual({
platform: "apple",
providers: ["apple"],
query: "SDWebImageManager.shared",
});
expect(queries).toContainEqual({
platform: "apple",
providers: ["sdwebimage"],
query: "SDWebImageManager.shared",
});
expect(queries).toContainEqual({
platform: "android",
providers: ["android"],
query: "OkHttpClient.Builder",
});
expect(queries).toContainEqual({
platform: "android",
providers: ["okhttp"],
query: "OkHttpClient.Builder",
});
expect(queries.every((query) => query.providers[0] !== "expo")).toBe(true);
});
test("routes Expo and React Native ecosystem imports without forwarding module strings", () => {
const queries = deriveResearchQueries(
[
@@ -312,6 +362,115 @@ test("finding citations are exact selections from research evidence", () => {
]);
});
test("research usefulness counts grounded candidate decisions and unique used results", () => {
const query = { platform: "apple" as const, providers: ["apple"], query: "Widget behavior" };
const evidence = [
{
query,
provider: "apple",
sourceKind: "official-api",
title: "Widget API",
url: "https://developer.apple.com/documentation/widgetkit/widget",
passage: "A widget contract.",
},
{
query,
provider: "apple",
sourceKind: "official-guide",
title: "Widget lifecycle",
url: "https://developer.apple.com/documentation/widgetkit/keeping-a-widget-up-to-date",
passage: "A lifecycle contract.",
},
{
query,
provider: "apple",
sourceKind: "official-guide",
title: "Unused result",
url: "https://developer.apple.com/documentation/widgetkit/creating-a-widget-extension",
passage: "Background only.",
},
];
const decisions = groundResearchDecisions(
[
{
outcome: "supported-finding",
summary: "The documented lifecycle confirms the stale update path.",
sources: [{ title: "forged", url: evidence[0]!.url }],
},
{
outcome: "dismissed-candidate",
summary: "The documented default makes the suspected fallback safe.",
sources: [{ title: "forged", url: evidence[1]!.url }],
},
{
outcome: "supported-finding",
summary: "This declaration is not grounded.",
sources: [{ title: "attacker", url: "https://attacker.example/fake" }],
},
],
evidence,
"correctness",
);
expect(decisions).toHaveLength(2);
expect(decisions[0]?.sources[0]?.title).toBe("Widget API");
const provenance = {
...toResearchProvenance({ queries: [query], evidence, warnings: [], promptText: "" }),
decisions,
};
const usefulness = summarizeResearchUsefulness(provenance, [
{
severity: "warning",
category: "correctness",
file: "Widget.swift",
line: 12,
title: "Widget update stays stale",
rationale: "The update does not reload.",
sources: [{ title: "Widget API", url: evidence[0]!.url }],
},
]);
expect(usefulness).toEqual({
finalFindingsWithSources: 1,
citedResultCount: 1,
supportedFindingCandidates: 1,
dismissedCandidates: 1,
decisionResultCount: 2,
utilizedResultCount: 2,
unusedResultCount: 1,
});
expect(formatResearchUsefulness(usefulness)).toContain("2 result(s) materially used, 1 unused");
const markdown = renderResearchUsefulnessMarkdown({ ...provenance, usefulness });
expect(markdown).toContain("**2/3**");
expect(markdown).toContain("**Supported finding** (correctness)");
expect(markdown).toContain(
"[Widget API](<https://developer.apple.com/documentation/widgetkit/widget>)",
);
});
test("cross-agent research decisions are deterministic and bounded by count and bytes", () => {
const decisions = Array.from({ length: 24 }, (_, index) => ({
outcome: index % 2 === 0 ? ("supported-finding" as const) : ("dismissed-candidate" as const),
summary: `${String(24 - index).padStart(2, "0")}-${"s".repeat(220)}`,
sources: [
{
title: "T".repeat(220),
url: `https://developer.apple.com/documentation/example/${String(index).padStart(2, "0")}/${"u".repeat(1_200)}`,
},
],
agent: `agent-${String(24 - index).padStart(2, "0")}`,
}));
const bounded = boundResearchDecisions(decisions);
expect(bounded.decisions.length).toBeLessThanOrEqual(RESEARCH_DECISION_COUNT_LIMIT);
expect(Buffer.byteLength(JSON.stringify(bounded.decisions), "utf8")).toBeLessThanOrEqual(
RESEARCH_DECISION_BYTES_LIMIT,
);
expect(bounded.omitted).toBe(decisions.length - bounded.decisions.length);
expect(bounded.decisions.map((decision) => decision.agent)).toEqual(
bounded.decisions.map((decision) => decision.agent).sort(),
);
});
test("one-shot stdio MCP results are validated, bounded, and fenced as untrusted", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "ecr-research-test-"));
temporaryDirectories.push(directory);
+36
View File
@@ -87,6 +87,7 @@ test("structured parsers expose draft-07 contracts for provider-side validation"
properties: {
findings: { items: { properties: Record<string, unknown>; required: string[] } };
trace: { properties: Record<string, unknown> };
researchDecisions: { items: { properties: Record<string, unknown> } };
};
};
expect(reviewer.$schema).toBe("http://json-schema.org/draft-07/schema#");
@@ -94,6 +95,7 @@ test("structured parsers expose draft-07 contracts for provider-side validation"
expect(reviewer.properties.findings.items.properties.agent).toBeUndefined();
expect(reviewer.properties.trace.properties).toHaveProperty("checked");
expect(reviewer.properties.trace.properties).toHaveProperty("uncertainties");
expect(reviewer.properties.researchDecisions.items.properties).toHaveProperty("outcome");
const coordinator = parseCoordinatorOutput.jsonSchema as {
properties: Record<string, unknown>;
@@ -142,6 +144,40 @@ test("reviewer trace parses bounded diagnostics and fails soft when malformed",
expect(malformed.findings.map((finding) => finding.title)).toEqual(["Keep the real finding"]);
});
test("reviewer research decisions parse bounded conclusions and fail soft when malformed", () => {
const source = {
title: "NWPathMonitor",
url: "https://developer.apple.com/documentation/network/nwpathmonitor",
};
const output = parseReviewerOutput(
JSON.stringify({
findings: [],
researchDecisions: [
{
outcome: "dismissed-candidate",
summary: "The documented callback behavior makes this path safe.",
sources: [source],
},
],
}),
);
expect(output.researchDecisions).toEqual([
{
outcome: "dismissed-candidate",
summary: "The documented callback behavior makes this path safe.",
sources: [source],
},
]);
const malformed = parseReviewerOutput(
JSON.stringify({
findings: [],
researchDecisions: [{ outcome: "read-background", summary: "No", sources: [] }],
}),
);
expect(malformed.researchDecisions).toBeUndefined();
});
test("a malformed coordinator-authored review trace is ignored, not a consolidation failure", () => {
const output = parseCoordinatorOutput(
JSON.stringify({
+1 -1
View File
@@ -427,7 +427,7 @@ export interface LoadedConfig {
additionalIgnores: string[];
additionalMarkers: string[];
};
/** Root-only trusted host-side documentation research. */
/** Root-only configuration for the bundled, bounded documentation MCP. */
research: {
enabled: boolean;
indexPath?: string;
+1 -1
View File
@@ -19,7 +19,7 @@ export interface RunLogRecord {
metadata: Pick<ReviewMetadata, "baseRef" | "headRef">;
reviewedFiles: string[];
filteredFiles: FilteredFile[];
/** Bounded outbound queries plus the exact allowlisted documentation results returned. */
/** Bounded queries/results plus grounded research decisions and final usefulness metrics. */
research?: ResearchProvenance;
agentCosts: Record<string, number>;
totalCost: number;
+19 -1
View File
@@ -125,9 +125,23 @@ export function platformResearchToolsSection(enabled: boolean): string[] {
"",
"Official documentation research tools are available for this pass:",
"- Use `fetch_platform_doc` when the PR or surrounding source already contains an",
" exact supported documentation URL.",
" exact supported documentation URL. Its default `section` context returns a",
" bounded contiguous window around the best match. Use `focused` for a small",
" matched-plus-adjacent view, and `document` only when qualifications are spread",
" across the page and the broader extracted context is materially necessary.",
"- Use `search_platform_docs` only when an external API contract, availability,",
" lifecycle rule, or dependency behavior materially affects a possible finding.",
"- Native source retains platform context. Use `apple` or `android` for OS contracts",
" and add the dependency provider for dependency-owned behavior. Do not choose",
" `expo` merely because native code lives under a `packages/expo-*` path.",
"- When your judgment depends on an API owned outside this repository, do not rely",
" on model memory. Ground the contract with these tools before either reporting",
" the finding or dismissing the candidate as safe. Prioritize newly introduced or",
" changed API use, availability/version gates, lifecycle, threading, permissions,",
" persistence, callbacks, and documented default behavior.",
"- Search returns focused passages with canonical URLs and neighboring passage IDs.",
" If a result lacks enough context, fetch that exact returned URL with a short query",
" and expand progressively. Do not issue several broader searches for the same page.",
"- Form short searches from an exact API symbol/member plus at most one behavior",
" term. Never send source text, prose, literals, paths, URLs, credentials, or",
" other repository data as a search query. The tool sanitizes and may reject it.",
@@ -137,6 +151,10 @@ export function platformResearchToolsSection(enabled: boolean): string[] {
" enough. Documentation does not force a finding; omit weak or irrelevant results.",
"- When a finding materially relies on documentation, copy the exact returned title",
" and canonical URL into that finding's `sources` array. Never invent or edit a URL.",
"- When documentation materially changes a candidate decision, add one top-level",
" `researchDecisions` item. Use `supported-finding` when it confirms a finding, or",
" `dismissed-candidate` when it proves a suspected issue is safe. Give a short",
" conclusion and exact returned sources. Omit generic context and unused results.",
];
}
+208 -39
View File
@@ -12,7 +12,7 @@ import { z } from "zod";
import type { LoadedConfig } from "../config/schema.js";
import { readResearchAudit } from "../research-mcp/audit.js";
import { run } from "./exec.js";
import type { Finding, FindingSource } from "./schema.js";
import type { Finding, FindingSource, ResearchDecision } from "./schema.js";
export { OPENCODE_RESEARCH_TOOLS } from "./tools.js";
export type ResearchPlatform = "apple" | "android" | "react-native";
@@ -61,9 +61,32 @@ export interface ResearchProvenance {
queries: ResearchQuery[];
results: ResearchResultRecord[];
warnings: string[];
/** Grounded reviewer declarations that documentation changed a concrete decision. */
decisions?: Array<ResearchDecision & { agent: string }>;
usefulness?: ResearchUsefulness;
error?: string;
}
export interface ResearchUsefulness {
finalFindingsWithSources: number;
citedResultCount: number;
supportedFindingCandidates: number;
dismissedCandidates: number;
decisionResultCount: number;
utilizedResultCount: number;
unusedResultCount: number;
}
export const RESEARCH_DECISION_COUNT_LIMIT = 16;
export const RESEARCH_DECISION_BYTES_LIMIT = 20_000;
type GroundedResearchDecision = ResearchDecision & { agent: string };
export interface BoundedResearchDecisions {
decisions: GroundedResearchDecision[];
omitted: number;
}
export const RESEARCH_MCP_SERVER_NAME = "platform_docs";
export const CLAUDE_RESEARCH_TOOLS = [
`mcp__${RESEARCH_MCP_SERVER_NAME}__search_platform_docs`,
@@ -337,48 +360,65 @@ const REACT_NATIVE_PROVIDERS = new Set([
function providersFor(file: ResearchInputFile, code: string, signals: string): string[] {
const path = file.path.toLowerCase();
const text = code.toLowerCase();
const platform = platformFor(file);
const providers: string[] = [];
const add = (provider: string, matches: boolean) => {
if (matches && !providers.includes(provider)) providers.push(provider);
};
add(
"react-native-reanimated",
/react-native-reanimated/.test(signals) || path.includes("react-native-reanimated"),
);
add(
"react-native-gesture-handler",
/react-native-gesture-handler/.test(signals) || path.includes("react-native-gesture-handler"),
);
add(
"react-native-screens",
/react-native-screens/.test(signals) || path.includes("react-native-screens"),
);
add(
"react-native-worklets",
/react-native-worklets/.test(signals) || path.includes("react-native-worklets"),
);
add(
"expo",
/(?:^|\n)(?:expo|expo-[a-z0-9-]+|@expo\/[a-z0-9-]+)(?:\n|$)/.test(signals) ||
/(?:^|\/)packages\/expo(?:-[^/]+)?(?:\/|$)/.test(path),
);
add("react-native", /(?:^|\n)react-native(?:\/[^\n]+)?(?:\n|$)/.test(signals));
if (providers.length > 0) return providers;
if (/androidx\.media3|mediasessionservice|\bexoplayer\b/.test(text)) return ["media3"];
if (/com\.bumptech\.glide|\bglide\b/.test(text)) return ["glide"];
if (/okhttp3|\bokhttpclient\b|\brequest\.builder\b/.test(text)) return ["okhttp"];
if (/kotlinx\.coroutines|\bcoroutinescope\b|\bmutable(?:state|shared)flow\b/.test(text)) {
return ["kotlin-coroutines"];
// Native source is owned by the native platform first. An Expo package path is
// repository ownership, not documentation ownership: packages/expo-image/ios
// must search Apple and SDWebImage contracts rather than Expo's JavaScript docs.
if (platform === "apple") add("apple", true);
if (platform === "android") add("android", true);
if (platform === "react-native") {
add(
"react-native-reanimated",
/react-native-reanimated/.test(signals) || path.includes("react-native-reanimated"),
);
add(
"react-native-gesture-handler",
/react-native-gesture-handler/.test(signals) || path.includes("react-native-gesture-handler"),
);
add(
"react-native-screens",
/react-native-screens/.test(signals) || path.includes("react-native-screens"),
);
add(
"react-native-worklets",
/react-native-worklets/.test(signals) || path.includes("react-native-worklets"),
);
add(
"expo",
/(?:^|\n)(?:expo|expo-[a-z0-9-]+|@expo\/[a-z0-9-]+)(?:\n|$)/.test(signals) ||
/(?:^|\/)packages\/expo(?:-[^/]+)?(?:\/|$)/.test(path),
);
add("react-native", /(?:^|\n)react-native(?:\/[^\n]+)?(?:\n|$)/.test(signals));
}
if (/\.gradle(?:\.kts)?$/.test(path) || /(?:^|\/)build\.gradle/.test(path)) {
return /com\.android|android\s*\{|compilesdk|targetsdk/.test(text)
? ["agp", "gradle"]
: ["gradle"];
// Explicit framework/dependency signals are additive. Keeping the platform
// provider alongside the dependency lets one pass check both the OS contract and
// the wrapper/library behavior without a package-path heuristic hiding either.
if (platform === "apple") {
add("sdwebimage", /\bsdwebimage(?:manager|options|context|cache|loader)?\b/.test(text));
add("expo", /\bexpomodulescore\b/.test(text));
}
const platform = platformFor(file);
if (platform === "apple") return ["apple"];
if (platform === "android") return ["android"];
return ["react-native"];
if (platform === "android") {
add("media3", /androidx\.media3|mediasessionservice|\bexoplayer\b/.test(text));
add("glide", /com\.bumptech\.glide|\bglide\b/.test(text));
add("okhttp", /okhttp3|\bokhttpclient\b|\brequest\.builder\b/.test(text));
add(
"kotlin-coroutines",
/kotlinx\.coroutines|\bcoroutinescope\b|\bmutable(?:state|shared)flow\b/.test(text),
);
const isGradle = /\.gradle(?:\.kts)?$/.test(path) || /(?:^|\/)build\.gradle/.test(path);
add("agp", isGradle && /com\.android|android\s*\{|compilesdk|targetsdk/.test(text));
add("gradle", isGradle);
add("expo", /\bexpo\.modules\.kotlin\b/.test(text));
}
return providers.length > 0 ? providers : ["react-native"];
}
function lineQuery(line: string): string | null {
@@ -668,7 +708,7 @@ export function toResearchProvenance(run: ResearchRun): ResearchProvenance {
sourceKind: cleanEvidenceText(item.sourceKind, 80),
title: cleanEvidenceText(item.title, 240),
url: item.url,
passage: cleanEvidenceText(item.passage, 1_200),
passage: cleanEvidenceText(item.passage, 20_000),
...(item.availability?.length
? { availability: item.availability.map((value) => cleanEvidenceText(value, 240)) }
: {}),
@@ -772,7 +812,7 @@ export function mergeResearchSources(
.slice(0, 5);
}
/** Keep only exact URLs returned by the trusted research prepass and restore their canonical titles. */
/** Keep only exact URLs returned by this review's MCP calls and restore canonical titles. */
export function groundResearchSources(
findings: Finding[],
evidence: ResearchEvidence[],
@@ -795,6 +835,135 @@ export function groundResearchSources(
});
}
/**
* Keep only reviewer decisions backed by an exact URL from this run's MCP audit.
* An ungrounded declaration is discarded so model output cannot inflate usefulness.
*/
export function groundResearchDecisions(
decisions: ResearchDecision[],
evidence: ResearchEvidence[],
agent: string,
): Array<ResearchDecision & { agent: string }> {
const allowed = new Map(
evidence.map((item) => [
item.url,
{ title: cleanEvidenceText(item.title, 240), url: item.url },
]),
);
return decisions.flatMap((decision) => {
const sources = mergeResearchSources(
decision.sources.flatMap((source) => {
const canonical = allowed.get(source.url);
return canonical ? [canonical] : [];
}),
);
if (sources.length === 0) return [];
return [
{
outcome: decision.outcome,
summary: cleanEvidenceText(decision.summary, 240),
sources,
agent: cleanEvidenceText(agent, 120),
},
];
});
}
/**
* Bound the cross-agent decision channel after grounding. Reviewer tasks finish
* concurrently, so sort before applying limits to keep the retained set stable.
*/
export function boundResearchDecisions(
decisions: GroundedResearchDecision[],
): BoundedResearchDecisions {
const sorted = [...decisions].sort((left, right) => {
const leftKey = `${left.agent}\0${left.outcome}\0${left.summary}\0${left.sources[0]?.url ?? ""}`;
const rightKey = `${right.agent}\0${right.outcome}\0${right.summary}\0${right.sources[0]?.url ?? ""}`;
return leftKey.localeCompare(rightKey);
});
const kept = sorted.slice(0, RESEARCH_DECISION_COUNT_LIMIT);
let omitted = sorted.length - kept.length;
while (
kept.length > 0 &&
Buffer.byteLength(JSON.stringify(kept), "utf8") > RESEARCH_DECISION_BYTES_LIMIT
) {
kept.pop();
omitted++;
}
return { decisions: kept, omitted };
}
/** Count unique audited results that materially affected the final review. */
export function summarizeResearchUsefulness(
provenance: ResearchProvenance,
findings: Finding[],
): ResearchUsefulness {
const resultUrls = new Set(provenance.results.map((result) => result.url));
const citedUrls = new Set(
findings.flatMap((finding) =>
(finding.sources ?? []).flatMap((source) => (resultUrls.has(source.url) ? [source.url] : [])),
),
);
const decisions = provenance.decisions ?? [];
const decisionUrls = new Set(
decisions.flatMap((decision) =>
decision.sources.flatMap((source) => (resultUrls.has(source.url) ? [source.url] : [])),
),
);
const utilizedUrls = new Set([...citedUrls, ...decisionUrls]);
return {
finalFindingsWithSources: findings.filter((finding) =>
(finding.sources ?? []).some((source) => resultUrls.has(source.url)),
).length,
citedResultCount: citedUrls.size,
supportedFindingCandidates: decisions.filter(
(decision) => decision.outcome === "supported-finding",
).length,
dismissedCandidates: decisions.filter((decision) => decision.outcome === "dismissed-candidate")
.length,
decisionResultCount: decisionUrls.size,
utilizedResultCount: utilizedUrls.size,
unusedResultCount: Math.max(0, resultUrls.size - utilizedUrls.size),
};
}
export function formatResearchUsefulness(usefulness: ResearchUsefulness): string {
return (
` research usefulness: ${usefulness.finalFindingsWithSources} final finding(s) cited ` +
`${usefulness.citedResultCount} unique result(s); ` +
`${usefulness.supportedFindingCandidates} supported and ` +
`${usefulness.dismissedCandidates} dismissed candidate(s); ` +
`${usefulness.utilizedResultCount} result(s) materially used, ` +
`${usefulness.unusedResultCount} unused`
);
}
export function renderResearchUsefulnessMarkdown(provenance: ResearchProvenance): string {
const usefulness = provenance.usefulness;
if (!usefulness) return "";
const totalUniqueResults = usefulness.utilizedResultCount + usefulness.unusedResultCount;
const lines = [
"### 📚 Documentation research usefulness",
"",
`- Final findings with grounded citations: **${usefulness.finalFindingsWithSources}**`,
`- Unique results cited by final findings: **${usefulness.citedResultCount}**`,
`- Candidate decisions: **${usefulness.supportedFindingCandidates} supported**, **${usefulness.dismissedCandidates} dismissed**`,
`- Unique results materially used: **${usefulness.utilizedResultCount}/${totalUniqueResults}**`,
];
if (provenance.decisions?.length) {
lines.push("", "Grounded candidate decisions:");
for (const decision of provenance.decisions) {
const sources = decision.sources
.map((source) => `[${escapeMarkdownLabel(source.title)}](<${source.url}>)`)
.join(", ");
lines.push(
`- **${decision.outcome === "supported-finding" ? "Supported finding" : "Dismissed candidate"}** (${escapeMarkdownLabel(decision.agent)}): ${escapeMarkdownLabel(decision.summary)} — ${sources}`,
);
}
}
return lines.join("\n");
}
export async function collectPlatformResearch(
files: ResearchInputFile[],
config: LoadedConfig["research"],
+37
View File
@@ -55,6 +55,7 @@ import type {
FeedbackRecord,
Finding,
FindingSource,
ResearchDecision,
ReviewerTraceNotes,
ReviewTrace,
} from "./schema.js";
@@ -69,12 +70,17 @@ import { reviewSetupRefNotes } from "./config-refs.js";
import { verifyFindings } from "./verify.js";
import { applyInlineIgnores } from "./suppress.js";
import {
boundResearchDecisions,
createResearchMcpRuntime,
formatResearchProgress,
formatResearchUsefulness,
groundResearchDecisions,
groundResearchSources,
mergeResearchSources,
researchProvenanceFromAudit,
renderResearchMarkdown,
renderResearchUsefulnessMarkdown,
summarizeResearchUsefulness,
} from "./research.js";
import type { ResearchEvidence, ResearchMcpRuntime, ResearchProvenance } from "./research.js";
@@ -472,6 +478,9 @@ export async function runReview(
// reviewers produced before the failure — partial findings are exactly what's
// needed to debug a run that died mid-way.
const agentFindings: Record<string, Finding[]> = {};
// Reviewer-declared, conclusion-only records of cases where documentation changed
// a concrete candidate decision. They remain inert until exact-source grounding.
const agentResearchDecisions: Record<string, ResearchDecision[]> = {};
// Bounded, conclusion-only diagnostics for machine consumers of the hidden
// comment state. These are deliberately separate from findings: they never reach
// the coordinator, verification, policy, or decision paths.
@@ -769,6 +778,9 @@ export async function runReview(
trackTokens(task.bucket, tokens);
trackModel(task.bucket, taskModel(task), model);
(agentFindings[task.bucket] ??= []).push(...value.findings);
if (value.researchDecisions) {
(agentResearchDecisions[task.bucket] ??= []).push(...value.researchDecisions);
}
if (value.trace) {
mergeTraceNotes(agentTrace, task.bucket, value.trace);
}
@@ -916,6 +928,21 @@ export async function runReview(
sourcesByFp.set(fp, mergeResearchSources(sourcesByFp.get(fp), finding.sources));
}
}
if (researchRecord) {
const groundedDecisions = Object.entries(agentResearchDecisions).flatMap(([agent, records]) =>
groundResearchDecisions(records, researchEvidence, agent),
);
const { decisions, omitted } = boundResearchDecisions(groundedDecisions);
if (decisions.length > 0) researchRecord = { ...researchRecord, decisions };
if (omitted > 0) {
const warning = `${omitted} grounded research decision(s) omitted by output bounds`;
researchRecord = {
...researchRecord,
warnings: [...researchRecord.warnings, warning],
};
progress(` research: ${warning}`);
}
}
// A substituted model means the review did not run on the model this repo
// configured — the findings may be from a weaker (or free-tier) model entirely.
@@ -1256,6 +1283,16 @@ export async function runReview(
}
}
// Measure utility only after verification, suppression, citation carry-through,
// and feedback adjudication have produced the final finding set. Counts use
// unique audited URLs, not passages, so duplicate search hits cannot inflate them.
if (researchRecord) {
const usefulness = summarizeResearchUsefulness(researchRecord, output.findings);
researchRecord = { ...researchRecord, usefulness };
progress(formatResearchUsefulness(usefulness));
await appendStepSummary(renderResearchUsefulnessMarkdown(researchRecord));
}
// Surface provider throttling as a fact about the run: passes already waited or
// backed off, but the operator should still SEE that it happened (a run that
// was rate-limited is slower and may carry partial passes — that's the cause).
+22 -1
View File
@@ -29,6 +29,20 @@ export const FindingSourceSchema = z.object({
});
export type FindingSource = z.infer<typeof FindingSourceSchema>;
export const RESEARCH_DECISION_OUTCOMES = ["supported-finding", "dismissed-candidate"] as const;
/**
* A reviewer-declared decision that documentation materially changed. Sources are
* later grounded against the MCP audit exactly like finding citations; an ungrounded
* record is discarded and can never inflate usefulness metrics.
*/
export const ResearchDecisionSchema = z.object({
outcome: z.enum(RESEARCH_DECISION_OUTCOMES),
summary: z.string().min(1).max(240),
sources: z.array(FindingSourceSchema).min(1).max(5),
});
export type ResearchDecision = z.infer<typeof ResearchDecisionSchema>;
/** A single unit of changed code, produced by a ReviewSource. */
export interface DiffEntry {
/** Path relative to the repo root, in the new tree. */
@@ -72,7 +86,7 @@ export const FindingSchema = z.object({
.max(5)
.optional()
.describe(
"Exact documentation sources used to support this finding; copy title and URL from the injected research evidence and omit when unused",
"Exact documentation sources used to support this finding; copy the returned title and canonical URL from this review's MCP results and omit when unused",
),
/**
* Verbatim snippet of the flagged code, copied from the file. Used to
@@ -181,6 +195,7 @@ export type ReviewerTraceNotes = z.infer<typeof ReviewerTraceNotesSchema>;
const ReviewerModelOutputSchema = z.object({
findings: z.array(ModelFindingSchema).default([]),
trace: ReviewerTraceNotesSchema.optional(),
researchDecisions: z.array(ResearchDecisionSchema).max(8).optional(),
});
/**
@@ -192,12 +207,18 @@ export const ReviewerOutputSchema = z
.object({
findings: z.array(ModelFindingSchema).default([]),
trace: z.unknown().optional(),
researchDecisions: z.unknown().optional(),
})
.transform((output) => {
const trace = ReviewerTraceNotesSchema.safeParse(output.trace);
const researchDecisions = z
.array(ResearchDecisionSchema)
.max(8)
.safeParse(output.researchDecisions);
return {
findings: output.findings,
...(trace.success ? { trace: trace.data } : {}),
...(researchDecisions.success ? { researchDecisions: researchDecisions.data } : {}),
};
});
export type ReviewerOutput = z.infer<typeof ReviewerOutputSchema>;
+20 -2
View File
@@ -10,6 +10,7 @@ export interface ResearchAuditInput {
providers?: string[];
query?: string;
url?: string;
context?: string;
}
export interface ResearchAuditResult {
@@ -21,7 +22,17 @@ export interface ResearchAuditResult {
SearchResult,
"id" | "platform" | "provider" | "sourceKind" | "title" | "url" | "passage"
> &
Partial<Pick<SearchResult, "availability" | "framework" | "language" | "symbol">>
Partial<
Pick<
SearchResult,
| "availability"
| "framework"
| "language"
| "symbol"
| "previousPassageId"
| "nextPassageId"
>
>
>;
warnings: string[];
error?: string;
@@ -47,6 +58,7 @@ type AuditEvent =
const LOCK_RETRIES = 200;
const LOCK_DELAY_MS = 10;
const MAX_AUDITED_PASSAGE_CHARACTERS = 20_000;
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
@@ -66,13 +78,19 @@ function boundedResult(result: SearchResult): ResearchAuditResult["results"][num
sourceKind: result.sourceKind,
title: result.title.slice(0, 240),
url: result.url.slice(0, 2_000),
passage: result.passage.slice(0, 1_400),
// This equals the direct-fetch document ceiling, so the artifact preserves the
// exact bounded text shown to the reviewer without ever storing the raw page.
passage: result.passage.slice(0, MAX_AUDITED_PASSAGE_CHARACTERS),
...(result.availability?.length
? { availability: result.availability.slice(0, 20).map((value) => value.slice(0, 240)) }
: {}),
...(result.framework ? { framework: result.framework.slice(0, 240) } : {}),
...(result.language ? { language: result.language } : {}),
...(result.symbol ? { symbol: result.symbol.slice(0, 240) } : {}),
...(result.previousPassageId
? { previousPassageId: result.previousPassageId.slice(0, 240) }
: {}),
...(result.nextPassageId ? { nextPassageId: result.nextPassageId.slice(0, 240) } : {}),
};
}
+2 -1
View File
@@ -20,7 +20,8 @@ Usage:
The serve command uses BRAVE_SEARCH_API_KEY for scoped web discovery, fetches only
allowlisted official pages, and optionally falls back to a local index. Expo-provider
searches use Expo's public documentation index. Its fetch_platform_doc tool can fetch
one exact allowlisted documentation URL without a search key. The update command is
one exact allowlisted documentation URL without a search key and return focused,
section, or bounded-document extracted context. The update command is
an optional offline crawler for operator-managed fallback indexes.
`);
}
+115 -14
View File
@@ -5,11 +5,18 @@ import { chunkDocument } from "./html.js";
import { getProvider, resolveAllowedUrl } from "./providers.js";
import { assertSafeDocumentationUrlShape } from "./query-sanitizer.js";
import { buildSearchIndex, searchDocumentation } from "./search-index.js";
import type { ProviderId, SearchResult, SourceKind } from "./types.js";
import type {
DiscoveredDocument,
IndexedChunk,
ProviderId,
SearchResult,
SourceKind,
} from "./types.js";
/** Specific corpora precede their broader host/path parents during URL inference. */
const DIRECT_PROVIDER_ORDER: readonly ProviderId[] = [
"apple-releases",
"sdwebimage",
"apple",
"swift-evolution",
"android-releases",
@@ -33,6 +40,7 @@ const DIRECT_SOURCE_KIND: Record<ProviderId, SourceKind> = {
apple: "official-api",
"apple-releases": "release-notes",
"swift-evolution": "official-guide",
sdwebimage: "official-api",
android: "official-api",
"android-releases": "release-notes",
media3: "official-guide",
@@ -60,9 +68,79 @@ export interface DirectDocumentationFetch {
provider: ProviderId;
sourceKind: SourceKind;
canonicalUrl: string;
context: DirectDocumentationContext;
results: SearchResult[];
}
export const DIRECT_DOCUMENT_CONTEXT_MODES = ["focused", "section", "document"] as const;
export type DirectDocumentContextMode = (typeof DIRECT_DOCUMENT_CONTEXT_MODES)[number];
export interface DirectDocumentationContext {
mode: DirectDocumentContextMode;
returnedCharacters: number;
documentCharacters: number;
truncated: boolean;
anchorPassageId?: string;
availablePassageCount: number;
availablePassageIds: string[];
passageIdsTruncated?: boolean;
}
const SECTION_CONTEXT_CHARACTERS = 12_000;
const DOCUMENT_CONTEXT_CHARACTERS = 20_000;
function withScore(chunk: IndexedChunk, score = 0): SearchResult {
return { ...chunk, score };
}
function rankedAnchor(
chunks: IndexedChunk[],
document: DiscoveredDocument,
provider: ProviderId,
query?: string,
): SearchResult | undefined {
if (!query?.trim()) return chunks[0] ? withScore(chunks[0]) : undefined;
const index = buildSearchIndex(chunks, 1);
return (
searchDocumentation(index, query, {
platform: document.platform,
providers: [provider],
limit: 1,
})[0] ?? (chunks[0] ? withScore(chunks[0]) : undefined)
);
}
function contiguousWindow(body: string, anchor: string, maxCharacters: number): string {
if (body.length <= maxCharacters) return body;
const exactIndex = body.indexOf(anchor);
const prefixIndex = exactIndex >= 0 ? exactIndex : body.indexOf(anchor.slice(0, 160));
const anchorIndex = prefixIndex >= 0 ? prefixIndex : 0;
const centeredStart = Math.max(0, anchorIndex - Math.floor(maxCharacters / 3));
let start = centeredStart;
const previousBoundary = body.lastIndexOf("\n\n", centeredStart);
if (previousBoundary >= Math.max(0, centeredStart - 300)) start = previousBoundary + 2;
let end = Math.min(body.length, start + maxCharacters);
const nextBoundary = body.indexOf("\n\n", end - 300);
if (nextBoundary >= 0 && nextBoundary <= start + maxCharacters) end = nextBoundary;
return body.slice(start, end).trim();
}
function focusedResults(
chunks: IndexedChunk[],
anchor: SearchResult | undefined,
limit: number,
): SearchResult[] {
if (!anchor) return [];
const anchorIndex = Math.max(
0,
chunks.findIndex((chunk) => chunk.id === anchor.id),
);
const start = Math.max(0, Math.min(anchorIndex - Math.floor(limit / 2), chunks.length - limit));
return chunks
.slice(start, start + limit)
.map((chunk) => withScore(chunk, chunk.id === anchor.id ? anchor.score : 0));
}
/** Resolve a caller-supplied URL against the fixed provider allowlist. */
export function resolveDirectDocumentationTarget(
rawUrl: string,
@@ -96,6 +174,7 @@ export async function fetchDocumentationUrl(
provider?: ProviderId;
query?: string;
limit?: number;
context?: DirectDocumentContextMode;
fetchImplementation?: FetchImplementation;
} = {},
): Promise<DirectDocumentationFetch> {
@@ -114,25 +193,47 @@ export async function fetchDocumentationUrl(
const indexedAt = new Date().toISOString();
const chunks = chunkDocument(document, indexedAt);
const limit = Math.min(5, Math.max(1, options.limit ?? 3));
let results: SearchResult[];
if (options.query?.trim()) {
const index = buildSearchIndex(chunks, 1, indexedAt);
results = searchDocumentation(index, options.query, {
platform: document.platform,
providers: [target.provider],
limit,
});
if (results.length === 0) {
results = chunks.slice(0, limit).map((chunk) => ({ ...chunk, score: 0 }));
}
} else {
results = chunks.slice(0, limit).map((chunk) => ({ ...chunk, score: 0 }));
const mode = options.context ?? "section";
const anchor = rankedAnchor(chunks, document, target.provider, options.query);
let results: SearchResult[] = [];
if (mode === "focused") {
results = focusedResults(chunks, anchor, limit);
} else if (anchor) {
const maxCharacters =
mode === "document" ? DOCUMENT_CONTEXT_CHARACTERS : SECTION_CONTEXT_CHARACTERS;
const passage =
mode === "document"
? document.body.slice(0, maxCharacters).trim()
: contiguousWindow(document.body, anchor.passage, maxCharacters);
const {
previousPassageId: _previous,
nextPassageId: _next,
...anchorWithoutNeighbors
} = anchor;
results = [
{
...anchorWithoutNeighbors,
id: `${anchor.id.replace(/#\d+$/, "")}#${mode}`,
passage,
},
];
}
const returnedCharacters = results.reduce((total, result) => total + result.passage.length, 0);
return {
provider: target.provider,
sourceKind: target.sourceKind,
canonicalUrl: target.url.href,
context: {
mode,
returnedCharacters,
documentCharacters: document.body.length,
truncated: returnedCharacters < document.body.length,
...(anchor ? { anchorPassageId: anchor.id } : {}),
availablePassageCount: chunks.length,
availablePassageIds: chunks.slice(0, 100).map((chunk) => chunk.id),
...(chunks.length > 100 ? { passageIdsTruncated: true } : {}),
},
results,
};
}
+5 -1
View File
@@ -164,11 +164,15 @@ export function chunkDocument(
.digest("hex")
.slice(0, 16);
const passageId = (chunkIndex: number) =>
`${document.provider ?? document.platform}:${documentId}#${chunkIndex}`;
return passages.map((passage, chunkIndex) => ({
...document,
body: undefined,
id: `${document.provider ?? document.platform}:${documentId}#${chunkIndex}`,
id: passageId(chunkIndex),
passage,
...(chunkIndex > 0 ? { previousPassageId: passageId(chunkIndex - 1) } : {}),
...(chunkIndex + 1 < passages.length ? { nextPassageId: passageId(chunkIndex + 1) } : {}),
indexedAt,
}));
}
+42
View File
@@ -188,6 +188,47 @@ export const swiftEvolutionProvider: DocumentationProvider = {
},
};
const sdWebImageDocumentPrefix = "/documentation/sdwebimage";
/** SDWebImage publishes a static DocC archive whose visible routes are JS shells. */
export const sdWebImageProvider: DocumentationProvider = {
id: "sdwebimage",
platform: "apple",
displayName: "SDWebImage documentation",
accepts(url) {
return (
isSecurePublicUrl(url, "sdwebimage.github.io") &&
hasAllowedPath(url, [sdWebImageDocumentPrefix])
);
},
acceptsRequest(url) {
if (this.accepts(url)) return true;
if (
!isSecurePublicUrl(url, "sdwebimage.github.io") ||
!url.pathname.startsWith(`/data${sdWebImageDocumentPrefix}`) ||
!url.pathname.endsWith(".json")
) {
return false;
}
const correspondingDocumentUrl = new URL(url.href);
correspondingDocumentUrl.pathname = url.pathname.slice("/data".length).replace(/\.json$/, "");
return hasAllowedPath(correspondingDocumentUrl, [sdWebImageDocumentPrefix]);
},
canonicalize(url) {
const hadTrailingSlash = url.pathname.endsWith("/");
const canonical = canonicalizeDocumentationUrl(url);
if (hadTrailingSlash && !canonical.pathname.endsWith("/")) canonical.pathname += "/";
return canonical;
},
requestUrl(documentUrl) {
const documentPath = documentUrl.pathname.replace(/\/$/, "").toLowerCase();
return new URL(`/data${documentPath}.json`, documentUrl.origin);
},
responseFormat() {
return "docc-json";
},
};
const androidPrefixes = [
"/build",
"/develop",
@@ -394,6 +435,7 @@ const providers: Record<ProviderId, DocumentationProvider> = {
apple: appleProvider,
"apple-releases": appleReleasesProvider,
"swift-evolution": swiftEvolutionProvider,
sdwebimage: sdWebImageProvider,
android: androidProvider,
"android-releases": androidReleasesProvider,
media3: media3Provider,
+4
View File
@@ -31,6 +31,10 @@ const providerSearchDefinitions: Record<Exclude<ProviderId, "expo">, ProviderSea
scopes: ["github.com/swiftlang/swift-evolution/blob/main/proposals"],
sourceKind: "official-guide",
},
sdwebimage: {
scopes: ["sdwebimage.github.io/documentation/sdwebimage"],
sourceKind: "official-api",
},
android: {
scopes: ["developer.android.com/reference"],
sourceKind: "official-api",
+4
View File
@@ -28,6 +28,8 @@ const miniSearchOptions = {
"symbol",
"language",
"availability",
"previousPassageId",
"nextPassageId",
"indexedAt",
],
};
@@ -172,6 +174,8 @@ export function searchDocumentation(
...(match.symbol ? { symbol: String(match.symbol) } : {}),
...(match.language ? { language: match.language as Language } : {}),
...(Array.isArray(match.availability) ? { availability: match.availability.map(String) } : {}),
...(match.previousPassageId ? { previousPassageId: String(match.previousPassageId) } : {}),
...(match.nextPassageId ? { nextPassageId: String(match.nextPassageId) } : {}),
indexedAt: String(match.indexedAt),
score: match.score,
}));
+28 -8
View File
@@ -5,7 +5,11 @@ import { z } from "zod";
import { ResearchAudit } from "./audit.js";
import type { FetchImplementation } from "./brave-search.js";
import { sanitizeDocumentationQuery } from "./query-sanitizer.js";
import { fetchDocumentationUrl, resolveDirectDocumentationTarget } from "./direct-fetch.js";
import {
DIRECT_DOCUMENT_CONTEXT_MODES,
fetchDocumentationUrl,
resolveDirectDocumentationTarget,
} from "./direct-fetch.js";
import { searchExpoAlgolia } from "./expo-algolia.js";
import { searchOkHttpDocumentation } from "./okhttp-search.js";
import { getProvider, resolveAllowedUrl } from "./providers.js";
@@ -20,6 +24,9 @@ const untrustedMaterialNotice =
const queryGuidance =
"Formulate short documentation queries from exact API symbols plus one behavior or constraint term. Good: `CameraView barcodeScannerSettings`, `NWPathMonitor pathUpdateHandler`, `GestureDetector simultaneous gestures`. Avoid questions, prose, package/import names, code snippets, literals, paths, credentials, and other sensitive context. If the first result is broad, retry with a narrower symbol or member name.";
const providerGuidance =
"Provider map: apple=Apple SDK APIs and Human Interface Guidelines; apple-releases=Xcode and Apple platform release notes; swift-evolution=Swift Evolution proposals; sdwebimage=SDWebImage APIs and caching/loading behavior; android=Android, Jetpack, Compose, and Google Play services APIs; android-releases=Android platform releases and behavior changes; media3=Jetpack Media3; glide=Glide; okhttp=OkHttp; kotlin-coroutines=Kotlin coroutines; gradle=Gradle; agp=Android Gradle Plugin; jetbrains-issues=JetBrains YouTrack context; expo=Expo documentation; react-native=React Native core; react-native-reanimated=Reanimated; react-native-gesture-handler=Gesture Handler; react-native-screens=Screens; react-native-worklets=Worklets. Native source retains platform context: use apple/android for OS contracts and add the dependency provider for dependency-owned behavior; an Expo package path does not make a native API an Expo-docs query. Issue-tracker results are context, not API contracts.";
export interface DocumentationServerOptions {
indexPath?: string;
braveApiKey?: string;
@@ -50,7 +57,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
"search_platform_docs",
{
title: "Search official platform documentation",
description: `Search official platform, dependency, build-tool, and release documentation. Discovery uses scoped web search (or Expo's public documentation search), then fetches only allowlisted official pages; an optional local index is fallback evidence. Selected issue-tracker passages are context, not API contracts. Returns short passages with canonical source URLs. ${queryGuidance}`,
description: `Search official platform, dependency, build-tool, and release documentation. Discovery uses scoped web search (or Expo's public documentation search), then fetches only allowlisted official pages; an optional local index is fallback evidence. Returns short passages with canonical source URLs. ${providerGuidance} ${queryGuidance}`,
inputSchema: {
platform: z
.enum(["apple", "android", "react-native", "all"])
@@ -67,9 +74,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
.min(1)
.max(4)
.optional()
.describe(
"Optional named corpora to search. Select the dependency that owns the API; use expo for Expo APIs and react-native for React Native core.",
),
.describe(`Optional named corpora to search. ${providerGuidance}`),
sourceKinds: z
.array(z.enum(SOURCE_KINDS))
.min(1)
@@ -247,7 +252,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
{
title: "Fetch an official documentation URL",
description:
"Fetch one caller-supplied documentation URL from the fixed Apple, Android, Expo, React Native, dependency, build-tool, release-note, or issue-source allowlist. The URL and every redirect are revalidated before download. Returns bounded extracted passages and the canonical source URL; use query only to rank passages within that page.",
"Fetch one caller-supplied documentation URL from the fixed Apple, Android, Expo, React Native, dependency, build-tool, release-note, or issue-source allowlist. The URL and every redirect are revalidated before download. Returns normalized extracted text, never raw HTML or DocC JSON. `focused` returns the best passage with adjacent passages, `section` (default) returns a bounded contiguous window around the best passage, and `document` returns extracted page text up to a hard ceiling. Use query only to select context within that page.",
inputSchema: {
url: z
.string()
@@ -264,7 +269,19 @@ export async function createDocumentationServer(options: DocumentationServerOpti
.max(300)
.optional()
.describe("Optional short phrase used only to select the most relevant page passages"),
limit: z.number().int().min(1).max(5).default(3),
context: z
.enum(DIRECT_DOCUMENT_CONTEXT_MODES)
.default("section")
.describe(
"Context breadth: focused=matched and adjacent passages, section=bounded contiguous window around the match, document=bounded extracted page text",
),
limit: z
.number()
.int()
.min(1)
.max(5)
.default(3)
.describe("Passage count for focused context; ignored by section/document context"),
},
annotations: {
readOnlyHint: true,
@@ -273,7 +290,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
openWorldHint: true,
},
},
async ({ url, provider, query, limit }) => {
async ({ url, provider, query, context, limit }) => {
const sanitizedQuery = query ? sanitizeDocumentationQuery(query) : undefined;
// Resolve and validate before recording anything. This prevents credentials,
// query strings, or covert high-entropy path data from reaching either the
@@ -283,12 +300,14 @@ export async function createDocumentationServer(options: DocumentationServerOpti
providers: [target.provider],
...(sanitizedQuery ? { query: sanitizedQuery } : {}),
url: target.url.href,
context,
};
const requestId = await audit.reserve("fetch_platform_doc", auditInput);
try {
const fetched = await fetchDocumentationUrl(target.url.href, {
provider: target.provider,
...(sanitizedQuery ? { query: sanitizedQuery } : {}),
context,
limit: Math.min(limit, maxResultsPerCall),
...(options.fetchImplementation
? { fetchImplementation: options.fetchImplementation }
@@ -301,6 +320,7 @@ export async function createDocumentationServer(options: DocumentationServerOpti
provider: fetched.provider,
sourceKind: fetched.sourceKind,
canonicalUrl: fetched.canonicalUrl,
context: fetched.context,
},
results: fetched.results,
};
+3
View File
@@ -7,6 +7,7 @@ export const PROVIDERS = [
"apple",
"apple-releases",
"swift-evolution",
"sdwebimage",
"android",
"android-releases",
"media3",
@@ -64,6 +65,8 @@ export interface IndexedChunk {
symbol?: string;
language?: Language;
availability?: string[];
previousPassageId?: string;
nextPassageId?: string;
indexedAt: string;
}
+6 -6
View File
@@ -24,12 +24,12 @@
// *.min.js, *.map, __snapshots__/*.snap, @generated markers).
"noise": { "additionalIgnores": [] },
// Optional trusted host-side platform research (ROOT-ONLY; off by default).
// ECR derives short API identifiers from native diffs and sends them to its
// bundled MCP before model startup. BRAVE_SEARCH_API_KEY enables fixed site-scoped
// discovery; returned URLs are independently allowlisted before ECR fetches and
// fences official passages. Expo uses its public documentation search. The model
// never receives an MCP tool. indexPath is an optional offline fallback only.
// Optional bounded platform research (ROOT-ONLY; off by default). Reviewer and
// cross-file passes can call ECR's bundled MCP for exact API-symbol searches and
// supported documentation URLs. The MCP sanitizes queries, uses fixed provider
// allowlists, audits results, and never receives model credentials. BRAVE_SEARCH_API_KEY
// enables fixed site-scoped discovery; Expo uses its public documentation search.
// indexPath remains an optional offline fallback only.
// "research": {
// "enabled": true,
// "maxQueries": 8,
+19 -6
View File
@@ -216,7 +216,14 @@ Return **only** a single fenced ```json code block, an object of this shape:
"rationale": "**Confidence:** High — why certainty is high.<br>**Impact if shipped:** Medium — concrete expected consequence.\\n\\n<details>\\n<summary>Evidence and reasoning</summary>\\n\\nFull failure/exploit path.\\n\\n</details>",
"evidence": "one contiguous line of the flagged code, copied VERBATIM",
"suggestion": "optional concrete fix, or omit",
"sources": [{ "title": "exact injected documentation title", "url": "exact injected URL" }]
"sources": [{ "title": "exact returned documentation title", "url": "exact returned URL" }]
}
],
"researchDecisions": [
{
"outcome": "supported-finding | dismissed-candidate",
"summary": "short conclusion that the documentation materially established",
"sources": [{ "title": "exact returned documentation title", "url": "exact returned URL" }]
}
],
"trace": {
@@ -226,10 +233,16 @@ Return **only** a single fenced ```json code block, an object of this shape:
}
```
`sources` is optional. Include it only when injected platform research materially
supports the finding. Copy the exact title and URL from that research; the engine
rejects sources outside the trusted result set. Omit it for findings that did not
use documentation research.
`sources` is optional. Include it only when documentation returned by the research
MCP materially supports the finding. Copy the exact returned title and canonical URL;
the engine rejects sources outside this review's audited MCP results. Omit it for
findings that did not use documentation research.
`researchDecisions` is optional. Include an item only when documentation materially
changes a concrete candidate decision. Use `supported-finding` when it confirms a
finding. Use `dismissed-candidate` when it proves a suspected issue is safe. Copy exact
returned sources. Do not list generic background reading or unused results. The engine
discards records whose URLs do not appear in this review's audited MCP results.
`line` is the start line in the new version of the file, or `null` if not
line-specific. `evidence` is used to help verify the finding, so make it easy to
@@ -237,4 +250,4 @@ locate: copy **one contiguous line** of the flagged code **verbatim** (not spann
multiple lines, no `…` elisions, no paraphrasing). For a structural/"missing" issue,
quote the single most relevant real line (e.g. the early `return` that skips the
handling). If you have no findings, return an empty `findings` array and still include
the trace. Emit no prose outside the JSON block.
the trace plus any applicable `researchDecisions`. Emit no prose outside the JSON block.