mirror of
https://github.com/expo/code-review-cli.git
synced 2026-11-14 09:58:11 -10:00
Compare commits
9
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
da80f6d3ca | ||
|
|
62c013a321 | ||
|
|
65ffc81014 | ||
|
|
92257f9bc9 | ||
|
|
047d360b1b | ||
|
|
813859b6cf | ||
|
|
26acefef90 | ||
|
|
36dd50772a | ||
|
|
d993621e62 |
No files matched your search
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -31,6 +31,15 @@
|
||||
"timeoutMs": 30000
|
||||
},
|
||||
|
||||
// Active Jev verification cascade. Confident local judgments act immediately;
|
||||
// ambiguous and protected findings defer to the reasoning verifier.
|
||||
"jev": {
|
||||
"model": "jev-1.13.0",
|
||||
"minConfidence": 0.9,
|
||||
"timeoutMs": 10000,
|
||||
"maxContextChars": 30000
|
||||
},
|
||||
|
||||
// Large diffs are split into focused chunks by changed-line count, plus a
|
||||
// cross-cutting pass for multi-file issues. Diffs under maxChangedLines are one
|
||||
// full-context pass. Defaults shown; raise/lower per your model + PR sizes.
|
||||
File renamed without changes.
File renamed without changes.
@@ -90,5 +90,7 @@ jobs:
|
||||
CLAUDE_CODE_REVIEW_SHARED_API_TOKEN: ${{ secrets.CLAUDE_CODE_REVIEW_SHARED_API_TOKEN }}
|
||||
# Search-only credential used for fixed, site-scoped documentation discovery.
|
||||
BRAVE_SEARCH_API_KEY: ${{ secrets.BRAVE_SEARCH_API_KEY }}
|
||||
# Shadow-only Jev evaluator; unused while config.jev.enabled is false.
|
||||
TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
|
||||
# Optional: override the model for every agent.
|
||||
REVIEWER_MODEL: ${{ vars.REVIEWER_MODEL }}
|
||||
@@ -9,3 +9,4 @@ build/
|
||||
|
||||
# Claude Code local worktrees
|
||||
.claude/worktrees/
|
||||
.expo-agents/code-review/.runs/
|
||||
@@ -0,0 +1,230 @@
|
||||
# Plan: inline PR comments for in-diff findings (rev 2, post adversarial review)
|
||||
|
||||
## Goal
|
||||
|
||||
Findings anchored to a line in the PR diff land as inline review comments on
|
||||
that line. The main issue comment keeps every finding but renders an inlined
|
||||
one SHORT: title + one-line rationale + link to the inline thread. The main
|
||||
comment stays the single durable state store (embedded state, dismissals,
|
||||
feedback records, pins, `/dismiss` target). Inline comments are a derived
|
||||
presentation layer, converged on every run.
|
||||
|
||||
## Non-goals (v1)
|
||||
|
||||
- No formal `POST /pulls/{n}/reviews` review event: a batched review is atomic
|
||||
(one bad anchor 422s the whole post) and cannot be upserted across runs.
|
||||
Individual review comments isolate failures and PATCH in place.
|
||||
- No multi-line comments; single `line` + `side: "RIGHT"` only.
|
||||
- No GraphQL thread resolution, and no detection of author-resolved/outdated
|
||||
threads. Accepted risk: the PR author can collapse inline threads; the main
|
||||
comment therefore keeps a one-line rationale per finding (see Short form).
|
||||
- `ecr feedback` retroactive crawl keeps reading issue comments only. The
|
||||
review-comments fetch in `replyComments` is gated on the reporter having an
|
||||
ENABLED `inline` option, so the crawl (which passes none) gains no fetches.
|
||||
- Scopes removed from the routing manifest strand their comments — same
|
||||
pre-existing behavior as their main comments.
|
||||
- Break-glass (`postSkipNote`) leaves inline threads untouched even though it
|
||||
overwrites the embedded state. Accepted, documented.
|
||||
|
||||
## Config
|
||||
|
||||
Root-only key in `ReviewConfigSchema`, locked out of scope configs with
|
||||
`z.never` like `feedback`:
|
||||
|
||||
```jsonc
|
||||
inline: {
|
||||
enabled: false, // ship dark
|
||||
maxComments: 20 // cap per posted comment's reporter (per PR in single/
|
||||
// legacy mode; per scope in per-scope mode — documented)
|
||||
}
|
||||
```
|
||||
|
||||
Wired into `LoadedConfig`, `load.ts` defaults, all five reporter construction
|
||||
sites, and `reviewPostingConfigFingerprint` (posting behavior changes with it,
|
||||
so a saved deferred review must not post under a different inline policy).
|
||||
Turning it OFF later leaves existing threads up (documented; `clear()` still
|
||||
sweeps them on mode switches, and a manual cleanup is possible).
|
||||
|
||||
## Inline comment identity
|
||||
|
||||
Marker, at byte 0 of the body, parsed with an ANCHORED regex and a hex-only
|
||||
charset:
|
||||
|
||||
```
|
||||
<!-- <tag>:inline:fp=<[a-f0-9]{6,64}> -->
|
||||
```
|
||||
|
||||
- Identity = author (`user.login === ownLogin`) + anchored marker; marker alone
|
||||
is never identity (selectOwnComments model). ownLogin unresolvable ⇒ nothing
|
||||
is ours AND no creates either (fail closed, no duplicate storms).
|
||||
- Byte-0 anchoring also narrows the shared `github-actions[bot]` confused-deputy
|
||||
risk: an echo-bot that prefixes any content fails the anchor.
|
||||
- fp is the id the owning main comment renders (plain, or scope-namespaced for
|
||||
the aggregate). Substring safety vs `<!-- tag -->` / `<!-- tag:scope -->`
|
||||
holds in both directions (closing ` -->` differs), incl. a scope named
|
||||
`inline`.
|
||||
|
||||
## Inline comment body
|
||||
|
||||
Marker line, then the finding rendered in full (severity, title, category,
|
||||
`id:<fp>` token, rationale, sources, suggestion — every model-written field
|
||||
through `stripStateMarkers`), then a footer: "Reply here to respond; write
|
||||
`id:<fp>` in your reply (unquoted) to formally answer it."
|
||||
|
||||
**Self-reply backstop (was blocker):** the body carries an unquoted `id:` token,
|
||||
and today's `OWN_COMMENT_RE` (`:state=|:fingerprints=`) would NOT exclude it if
|
||||
identity/tag filters ever miss (bot-login fallback, commentTag rename, second
|
||||
posting identity). Fixes, all three:
|
||||
- `responses.ts` gains a tag-independent backstop: a body whose UNQUOTED lines
|
||||
contain `:inline:fp=` is never treated as a reply (quoted lines excluded so a
|
||||
GitHub "Quote reply" of our inline comment — which copies the marker behind
|
||||
`> ` — still counts as a genuine reply).
|
||||
- `replyComments` applies the same unquoted-line rule for the tag-specific
|
||||
marker.
|
||||
- Test: an inline body is never matched as a reply even with `ownLogin: null`
|
||||
and a foreign tag; a quote-reply containing the quoted marker IS matched.
|
||||
|
||||
## Reporter mechanics
|
||||
|
||||
New option `inline?: LoadedConfig["inline"]`.
|
||||
|
||||
### Fetch layer
|
||||
|
||||
`fetchAllReviewComments()`: paginated GET `repos/{repo}/pulls/{n}/comments`
|
||||
(fields: id, body, path, line, user.login, html_url, in_reply_to_id,
|
||||
author_association). Cached in a MODULE-LEVEL map keyed
|
||||
`prNumber\nrepo\ncwd` (the prAuthorByPr pattern) with the same 30s TTL and
|
||||
invalidate-on-mutation — routed single-mode runs call `clear()` on a fresh
|
||||
reporter per scope, and per-instance caching would multiply listings. Note:
|
||||
this endpoint DOES honor sort/direction (unlike issue comments) — do not copy
|
||||
that rationale comment; a full scan is still needed.
|
||||
|
||||
### Sync
|
||||
|
||||
`syncInline(targets, link, opts: { teardown: boolean })` runs inside
|
||||
`report()` / `reportAggregate()` before rendering; returns `Map<fp, url>`.
|
||||
|
||||
Targets = ACTIVE findings only (not dismissed, not reply-cleared, not
|
||||
requalified) with `line != null` and `diffLines.get(file).has(line)`.
|
||||
|
||||
**Cap (sticky, was churn bug):** order candidates by severity rank then fp
|
||||
(deterministic); fill slots preferring fps that ALREADY have a live inline
|
||||
comment, then admit new ones. A live thread is never evicted to create a new
|
||||
one.
|
||||
|
||||
**Teardown gating (was blocker):** stubs/deletes run ONLY when
|
||||
`opts.teardown` is true, which callers set as:
|
||||
- `report()`: `!review.couldNotComplete`.
|
||||
- `reportAggregate()`: caller-passed flag — ci.ts passes
|
||||
`!scopesFilter && results.every(r => !r.review.couldNotComplete)`. A partial
|
||||
`--scopes` run or any failed scope makes the run additive-only (creates +
|
||||
patches, no stubs/deletes), because carried/truncated/failed state cannot
|
||||
prove a finding is gone. (`scopedFingerprint` is opaque hex — a per-scope
|
||||
stale set cannot be derived from fps, so the gate is all-or-nothing.)
|
||||
- `applyDismissal`: never computes a stale set. Additive-only: stub threads of
|
||||
the explicitly dismissed fps; on `/undismiss`, re-patch the full body if the
|
||||
finding is present in state (skip if truncated away).
|
||||
- Prereq fix: both failure literals (`ci.ts` legacy catch and
|
||||
`failureReview()`) gain `couldNotComplete: true` — today they post
|
||||
empty-findings reviews WITHOUT the flag and would tear everything down. Side
|
||||
effects checked: the decision label becomes the accurate "No review — every
|
||||
pass failed", and caching is unaffected (failures already never carry
|
||||
inputHash).
|
||||
|
||||
**Stub text (was blocker):** neutral — "This finding is no longer tracked
|
||||
inline — see the main review comment for current status." Never
|
||||
"resolved/dismissed": a capped-out or drifted finding is still active.
|
||||
|
||||
**Mutations** (sequential, never parallel):
|
||||
- create: POST with body/commit_id=headSha/path/line/side. Creates require
|
||||
headSha AND a live-head check: one `gh pr view --json headRefOid` per sync
|
||||
(only when creates exist); mismatch ⇒ skip creates, log (a mid-review push
|
||||
means our diff no longer matches the head; a stale commit_id can silently
|
||||
anchor to wrong code, and a force-push 422s every create).
|
||||
- patch: only when the normalized body differs (`\r\n` → `\n`) — no PATCH-per-
|
||||
finding-per-run forever.
|
||||
- stale (teardown only): thread has replies ⇒ PATCH to the neutral stub
|
||||
(marker kept, so a returning finding revives the thread); no replies ⇒
|
||||
DELETE. Reply detection: any listed comment with `in_reply_to_id` == our id
|
||||
(same snapshot; the reply-between-list-and-delete race is accepted, noted in
|
||||
tests).
|
||||
- duplicates (crash window): keep newest per fp, older are stale.
|
||||
- 422 on create ⇒ log once + skip that finding (renders full-form). 403/429
|
||||
(secondary rate limit) ⇒ log + abort ALL remaining mutations this run.
|
||||
- Audit (was silent): stderr one-liner + step-summary line:
|
||||
`inline sync: N created, N patched, N stubbed, N deleted, N failed`.
|
||||
|
||||
Every mutation individually try/caught; the inline layer never blocks the main
|
||||
comment. Listing failure ⇒ empty map, no mutations.
|
||||
|
||||
### clear()
|
||||
|
||||
Extends to inline comments under this reporter's tag, gated on marker+author
|
||||
only (NOT on `enabled`, so mode switches clean up after a disable): replied
|
||||
threads are STUBBED, bare ones deleted — never delete a thread humans wrote in
|
||||
(ci.ts calls clear() on mode switches and scope reconciliation). This also
|
||||
covers the fp-namespace flip on per-scope↔single switches, symmetric with how
|
||||
main comments are cleared today.
|
||||
|
||||
### Main-comment short form
|
||||
|
||||
- `LinkContext.inlineUrls?: Map<string, string>`. NEVER mutate the shared
|
||||
routed-flow `link` object — render with a copy `{ ...link, inlineUrls }`.
|
||||
- `renderFindingLines` with an inline URL: `- **title** —
|
||||
[file:line](thread-url) _(category)_ · id:… · 💬 [inline](url)` + reply
|
||||
annotation + ONE-LINE truncated rationale (~160 chars, word boundary) — the
|
||||
audit trail survives the author collapsing/resolving inline threads.
|
||||
Sources/suggestion/full rationale live inline only. Missing URL ⇒ full form
|
||||
(tested).
|
||||
- Embedded state unchanged.
|
||||
|
||||
### Inline replies → feedback
|
||||
|
||||
- `replyComments()` maps review comments too — ONLY when
|
||||
`options.inline?.enabled` (keeps `ecr feedback` crawl and non-inline repos at
|
||||
zero extra fetches). Excludes own login; excludes bodies whose unquoted lines
|
||||
carry our markers.
|
||||
- **Id namespace (was collision risk):** review-comment-sourced ReplyComments
|
||||
get `id = -reviewCommentId`. Negative ids flow into FeedbackRecord.commentId
|
||||
(schema unchanged — number stays a number) and are consistent across runs, so
|
||||
mergeFeedback's `commentId` equality and buildAdjudicationItems' body lookup
|
||||
stay collision-free vs the positive issue-comment space.
|
||||
- `applyPins` gains a cross-stream guard: a pin is lifted only by a maintainer
|
||||
record whose commentId is newer AND in the SAME stream (same sign). A
|
||||
different-stream comparison is undecidable ordering ⇒ keep the pin (fail
|
||||
closed).
|
||||
- `ReplyComment.threadFp?: string`: set when the reply's `in_reply_to_id`
|
||||
resolves to OUR (author+anchored-marker) top-level inline comment's fp.
|
||||
`matchReplies` treats a known threadFp as a match in EVERY `match` mode
|
||||
(structural signal, not text matching; documented + tested). `citedId` is
|
||||
unchanged: clearing still requires the replier's own unquoted `id:` token.
|
||||
- Newest-wins across streams: negative ids lose to positive ones by raw
|
||||
comparison; acceptable because only the newest record per fp matters and a
|
||||
same-fp cross-stream tie is rare; the applyPins guard removes the dangerous
|
||||
consequence.
|
||||
|
||||
## Files touched
|
||||
|
||||
- `src/config/schema.ts`, `src/config/load.ts` — `inline` key + lock + defaults.
|
||||
- `src/core/deferred-review.ts` — fingerprint includes `inline`.
|
||||
- `src/reporters/github.ts` — review-comments fetch (shared cache), syncInline
|
||||
+ pure exported planner, replyComments extension, clear(), applyDismissal
|
||||
additive sync.
|
||||
- `src/core/render.ts` — inline marker helpers (anchored parse), inline body
|
||||
renderer, `LinkContext.inlineUrls`, short form + rationale truncation.
|
||||
- `src/core/responses.ts` — threadFp, unquoted-marker backstop.
|
||||
- `src/core/schema.ts` — applyPins same-stream guard.
|
||||
- `src/commands/ci.ts` — wire config, `couldNotComplete: true` on both failure
|
||||
literals, aggregate teardown flag. `review.ts`, `post-review.ts`,
|
||||
`dismiss.ts` — wire config.
|
||||
- Tests: planner (create/patch/stub/delete, sticky cap, dedupe, additive mode),
|
||||
marker anchor/forgery, short form + missing-URL fallback, backstop vs
|
||||
quote-reply, threadFp in all match modes, applyPins stream guard,
|
||||
couldNotComplete literals.
|
||||
- `templates/config.jsonc`, `README.md`.
|
||||
|
||||
## Rollout
|
||||
|
||||
Ship dark. Note in README: first enable on a large PR fires one notification
|
||||
per created inline comment (cap bounds it). Enable on this repo first, then
|
||||
expo/expo after one observed cycle.
|
||||
@@ -4,7 +4,10 @@ A config-driven, multi-agent AI code reviewer. Specialist agents review a diff i
|
||||
parallel; a coordinator consolidates their findings into one structured review.
|
||||
The same engine runs locally (advisory) and in CI (posts one PR comment). The CLI
|
||||
is the **engine** — each repo supplies its own agents and settings under
|
||||
`.expo-code-review/`, so behavior is configured per-repo, not baked in.
|
||||
`.expo-agents/code-review/`, so behavior is configured per-repo, not baked in.
|
||||
(Repos set up before 0.15 keep their `.expo-code-review/` directory: every command
|
||||
looks for the new name first and falls back to the old one, and `ecr init` reuses
|
||||
an existing legacy directory rather than starting a second one.)
|
||||
|
||||
> **Status: experimental.** Comment-only and non-blocking — it never blocks a merge
|
||||
> and never auto-approves. See [`ROADMAP.md`](./ROADMAP.md).
|
||||
@@ -94,7 +97,7 @@ here ([LLP 0001](./llp/0001-trust-model.principles.md)) is built around that:
|
||||
## Usage
|
||||
|
||||
Run via `npx @expo/code-review-cli <command>` (or the `ecr` / `expo-code-review`
|
||||
binary once installed). On a repo that already has `.expo-code-review/` set up,
|
||||
binary once installed). On a repo that already has `.expo-agents/code-review/` set up,
|
||||
getting model credentials for local runs is one command:
|
||||
|
||||
```bash
|
||||
@@ -108,7 +111,7 @@ package.
|
||||
### First-time setup
|
||||
|
||||
```bash
|
||||
# 1. Scaffold .expo-code-review/ + a CI workflow (--no-workflow to skip)
|
||||
# 1. Scaffold .expo-agents/code-review/ + a CI workflow (--no-workflow to skip)
|
||||
npx @expo/code-review-cli init
|
||||
# 2. Get model credentials — guided; prints the export lines for your shell config
|
||||
npx @expo/code-review-cli setup-auth
|
||||
@@ -142,7 +145,7 @@ ecr review --pr 123
|
||||
ecr review --pr 123 --post
|
||||
# Preview once, save the exact result, and post it later without another model run
|
||||
ecr review --repo owner/repo --pr 123 --save-review --json
|
||||
ecr post-review --artifact .expo-code-review/.runs/deferred/<artifact>.json --repo owner/repo --pr 123
|
||||
ecr post-review --artifact .expo-agents/code-review/.runs/deferred/<artifact>.json --repo owner/repo --pr 123
|
||||
```
|
||||
|
||||
Options (most to least common):
|
||||
@@ -178,7 +181,7 @@ is a ready example to adapt.
|
||||
|
||||
| Command | What it does |
|
||||
| --- | --- |
|
||||
| `ecr init [--no-workflow] [--force]` | Scaffold `.expo-code-review/` (config, agents, prompts) + a CI workflow. |
|
||||
| `ecr init [--no-workflow] [--force]` | Scaffold `.expo-agents/code-review/` (config, agents, prompts) + a CI workflow. |
|
||||
| `ecr init --monorepo` | …and add a `routing.jsonc` routing manifest (one default scope). |
|
||||
| `ecr init --scope <dir>` | Scaffold a per-team scope under `<dir>` and register it in the manifest. |
|
||||
| `ecr setup-auth [--yes]` | Walk through getting model credentials for local runs (ChatGPT/Claude sign-in and/or API keys), printing the `export` lines for your shell config. |
|
||||
@@ -215,7 +218,7 @@ Markdown, `//` in JSONC):
|
||||
|
||||
A target is a file, a `dir/`, `glob:<pattern>`, `file#symbol`, or `doc.md#heading` —
|
||||
never a line number (line numbers rot without any signal). The check is strict on
|
||||
purpose: any backticked token in `.expo-code-review/` that names something that
|
||||
purpose: any backticked token in `.expo-agents/code-review/` that names something that
|
||||
exists in the repo **must** be a ref, because stale citations are exactly the ones
|
||||
nobody annotated. Mark the rare false positive once with
|
||||
`<!-- @ref-ignore knex.raw() -->`. Refs are always repo-root-relative, including in
|
||||
@@ -274,12 +277,41 @@ The boundary, in brief:
|
||||
many discovery and page fetches, so each call reports its own request ledger and
|
||||
the run reports totals.
|
||||
- **Root-only in routed monorepos** (it starts a host process); scope configs
|
||||
cannot alter it. Result-cache reuse is disabled while research is enabled,
|
||||
because web content can change without a config change.
|
||||
cannot alter it. Research results are fetched and audited per run; cached review
|
||||
results remain keyed by the ordinary trusted review inputs and make no network call.
|
||||
|
||||
Full detail — providers, query grammar, `fetch_platform_doc` modes, provenance and
|
||||
citation grounding: [LLP 0013](./llp/0013-platform-research.explainer.md).
|
||||
|
||||
## Jev selective verification
|
||||
|
||||
ECR can use [TypeSafe AI's Jev](https://docs.typesafe.ai/) as the fast first stage of
|
||||
finding verification. Generative agents still discover and explain problems. Jev
|
||||
answers one narrow typed question against bounded local source, then either keeps a
|
||||
clearly supported finding, drops a clearly contradicted ordinary finding, or defers
|
||||
to the existing reasoning verifier.
|
||||
|
||||
There is no shadow mode or separate `enabled` flag. Add the root-only block and provide
|
||||
`TYPESAFE_API_KEY`; removing the block restores the original verification path:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"jev": {
|
||||
"model": "jev-1.13.0",
|
||||
"minConfidence": 0.9,
|
||||
"timeoutMs": 10000,
|
||||
"maxContextChars": 30000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Each request contains one finding and bounded source from its file. Critical, security,
|
||||
secrets, and cited findings are never dropped by Jev alone. Low confidence, missing
|
||||
context, credentials, timeouts, and provider errors fall back to the reasoning or
|
||||
original verification path. Confidence is distribution concentration, not proof, so
|
||||
the model and threshold are explicit. See [LLP 0014](./llp/0014-jev-selective-verification.explainer.md)
|
||||
for the research, decision table, and threat model.
|
||||
|
||||
## Monorepos (routing manifest)
|
||||
|
||||
A monorepo can route different subtrees to different reviewer rosters from a single
|
||||
@@ -291,18 +323,18 @@ and no locking.
|
||||
|
||||
```
|
||||
your-monorepo/
|
||||
.expo-code-review/
|
||||
.expo-agents/code-review/
|
||||
routing.jsonc # the manifest — infra-owned, ordered scope list + locked defaults
|
||||
config.jsonc # the default/root scope; the ONLY place auth/tokenEnv lives
|
||||
shared.md coordinator.md agents/
|
||||
apps/
|
||||
api/.expo-code-review/{config.jsonc(NO auth),coordinator.md,agents/} # api team
|
||||
web/.expo-code-review/{config.jsonc(NO auth),coordinator.md,agents/} # web team
|
||||
api/.expo-agents/code-review/{config.jsonc(NO auth),coordinator.md,agents/} # api team
|
||||
web/.expo-agents/code-review/{config.jsonc(NO auth),coordinator.md,agents/} # web team
|
||||
.github/workflows/expo-code-review.yml # unchanged shape: one workflow, one `ecr ci`
|
||||
```
|
||||
|
||||
```jsonc
|
||||
// .expo-code-review/routing.jsonc
|
||||
// .expo-agents/code-review/routing.jsonc
|
||||
{
|
||||
// Central guardrails every scope inherits and CANNOT override.
|
||||
"defaults": {
|
||||
@@ -322,7 +354,7 @@ your-monorepo/
|
||||
```
|
||||
|
||||
```jsonc
|
||||
// apps/api/.expo-code-review/config.jsonc (the api team owns this)
|
||||
// apps/api/.expo-agents/code-review/config.jsonc (the api team owns this)
|
||||
{
|
||||
// NO "auth" block — locked centrally; a tokenEnv here is rejected by loader + CI guard.
|
||||
"model": "anthropic/claude-sonnet-5",
|
||||
@@ -352,9 +384,9 @@ your-monorepo/
|
||||
|
||||
Enforced in code and by an independent CI guard step, not by convention:
|
||||
|
||||
- **auth and research are locked to the root.** `tokenEnv` is honored only in the
|
||||
- **auth, research, and Jev are locked to the root.** `tokenEnv` is honored only in the
|
||||
root `config.jsonc` / `routing.jsonc` `defaults.auth`; a scope config declaring
|
||||
`auth`/`breakGlass`/`research` fails to parse, and the CI guard sweeps every
|
||||
`auth`/`breakGlass`/`research`/`jev` fails to parse, and the CI guard sweeps every
|
||||
config file repo-wide and refuses to run unless `tokenEnv` appears exactly once,
|
||||
root-owned, equal to `ECR_EXPECTED_TOKEN_ENV`. Routing globs choose *which
|
||||
roster* reviews a file, never *which secret* is sent.
|
||||
@@ -378,8 +410,8 @@ Enforced in code and by an independent CI guard step, not by convention:
|
||||
itself, so even a custom workflow that checks out the head still gets
|
||||
base-commit configuration.
|
||||
|
||||
Enforce ownership with CODEOWNERS: `/.expo-code-review/routing.jsonc @your-infra`,
|
||||
`/apps/api/.expo-code-review/ @your-api-team`. Full detail:
|
||||
Enforce ownership with CODEOWNERS: `/.expo-agents/code-review/routing.jsonc @your-infra`,
|
||||
`/apps/api/.expo-agents/code-review/ @your-api-team`. Full detail:
|
||||
[LLP 0006](./llp/0006-config-schema-loading-routing.explainer.md) (routing) and
|
||||
[LLP 0001](./llp/0001-trust-model.principles.md) (trust model).
|
||||
|
||||
@@ -396,7 +428,7 @@ Enforce ownership with CODEOWNERS: `/.expo-code-review/routing.jsonc @your-infra
|
||||
- **Chunking** — small PRs run in a single pass; large PRs are split into chunks
|
||||
bounded by changed lines, plus one combined **cross-cutting pass** that looks
|
||||
for issues spanning multiple changed files across every agent's concern.
|
||||
- **Agents** — every `.md` file in `.expo-code-review/agents/` is an agent. They
|
||||
- **Agents** — every `.md` file in `.expo-agents/code-review/agents/` is an agent. They
|
||||
run in parallel with read-only repo tools (`read`/`grep`/`glob`/`list`).
|
||||
- **Coordinator** — a single pass that dedupes, re-judges severity, and produces
|
||||
the final `{ decision, findings, summary }`.
|
||||
@@ -426,7 +458,7 @@ Every run reports what it spent and how much was served from the prompt cache, i
|
||||
three places: one `Token usage — …` line in the job log, a per-pass table + cache
|
||||
hit rate in the GitHub Actions step summary (which also preserves each run's posted
|
||||
comment, since the PR comment is updated in place), and one JSON line per run in
|
||||
`.expo-code-review/.runs/reviews.jsonl` (uploaded as a CI artifact) with per-pass
|
||||
`.expo-agents/code-review/.runs/reviews.jsonl` (uploaded as a CI artifact) with per-pass
|
||||
tokens, raw per-agent findings, bounded reviewer traces, and coverage notes.
|
||||
|
||||
Each reviewer can also return a compact trace (up to three concrete checks and two
|
||||
@@ -452,10 +484,10 @@ cacheable size (~1–4K tokens) show `cache read 0` — expected, not a bug.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Configuration — <code>.expo-code-review/</code></b></summary>
|
||||
<summary><b>Configuration — <code>.expo-agents/code-review/</code></b></summary>
|
||||
|
||||
```
|
||||
.expo-code-review/
|
||||
.expo-agents/code-review/
|
||||
config.jsonc # model, policy, noise, auth, break-glass, comment tag
|
||||
shared.md # instructions prepended to every agent (optional)
|
||||
coordinator.md # the consolidation prompt (required)
|
||||
@@ -480,7 +512,7 @@ temperature: 0.1
|
||||
```
|
||||
|
||||
For a real-world example, see eas-cli's
|
||||
[`.expo-code-review/`](https://github.com/expo/eas-cli/tree/main/.expo-code-review)
|
||||
[`.expo-agents/code-review/`](https://github.com/expo/eas-cli/tree/main/.expo-code-review)
|
||||
— correctness/security/consistency agents, a stronger model for security, and
|
||||
per-repo `noise.additionalIgnores`.
|
||||
|
||||
@@ -651,6 +683,44 @@ the defaults are asymmetric: [LLP 0011](./llp/0011-author-feedback.explainer.md)
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Inline PR comments (findings on the diff line)</b></summary>
|
||||
|
||||
Off by default. When enabled, each finding anchored to a line that is actually in
|
||||
the PR's diff is posted as an inline review comment on that line, carrying the
|
||||
full rationale, sources, and suggestion. The main comment still lists every
|
||||
finding, but an inlined one renders short — title, links, category, id, and a
|
||||
one-line rationale — plus a `💬 inline comment` link to the thread. The main
|
||||
comment remains the single durable state store: `/dismiss`, embedded state, and
|
||||
reply adjudication are unchanged.
|
||||
|
||||
```jsonc
|
||||
"inline": {
|
||||
"enabled": false,
|
||||
"maxComments": 20 // per PR (single comment mode) / per scope (per-scope mode)
|
||||
}
|
||||
```
|
||||
|
||||
Behavior notes:
|
||||
|
||||
- **Threads converge, never churn.** Each inline comment carries a per-finding
|
||||
marker and is patched in place across re-reviews. The cap is sticky: a live
|
||||
thread is never evicted to create a new one. A finding that leaves the set is
|
||||
stubbed if humans replied in its thread (never deleted), deleted only when bare.
|
||||
- **Fail-safe teardown.** A failed run, a `--scopes` partial run, or a truncated
|
||||
aggregate never tears threads down — teardown only runs when the target set is
|
||||
provably complete. Creates are skipped (and logged) when the PR head moved
|
||||
during the review.
|
||||
- **Replies on inline threads count as author feedback.** Replying in a thread
|
||||
matches the reply to that finding structurally; clearing still requires the
|
||||
`id:…` token in the replier's own words, same as main-comment replies.
|
||||
- **Notification fan-out.** The first enabled run on a PR sends one notification
|
||||
per created inline comment; `maxComments` bounds it.
|
||||
- **Disabling later** leaves existing threads up (the sync no longer runs); a
|
||||
comment-mode switch — or the reporter's clear path — stubs/removes them.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>External context (--context-file)</b></summary>
|
||||
|
||||
@@ -680,7 +750,7 @@ variables: `ATLANTIS_BOT_LOGIN` (the Atlantis bot's comment login, e.g.
|
||||
<details>
|
||||
<summary><b>Run logs</b></summary>
|
||||
|
||||
Each run appends a JSON line to `.expo-code-review/.runs/reviews.jsonl` with the
|
||||
Each run appends a JSON line to `.expo-agents/code-review/.runs/reviews.jsonl` with the
|
||||
inputs, decision, finding count, duration, per-agent cost, and aggregate token
|
||||
usage (incl. prompt-cache read/write counts) — for auditing and measuring
|
||||
cost/latency/cache reuse over time. It also records the same bounded `reviewTrace`
|
||||
@@ -689,7 +759,7 @@ one-line summary to the terminal / CI job log at the end of each run, so cache r
|
||||
is visible even in CI (where the run log is ephemeral).
|
||||
|
||||
`ecr review --save-review` additionally writes a versioned artifact under
|
||||
`.expo-code-review/.runs/deferred/` with owner-only permissions. It contains the
|
||||
`.expo-agents/code-review/.runs/deferred/` with owner-only permissions. It contains the
|
||||
verified final review and bounded feedback metadata, but no credential. `ecr
|
||||
post-review` schema-validates it and refuses to post if its explicit repo/PR, live
|
||||
head commit, or local comment-policy fingerprint no longer matches.
|
||||
@@ -722,20 +792,25 @@ fallback key; `ecr doctor` diagnoses setup.
|
||||
} }
|
||||
```
|
||||
|
||||
- **Meta / Muse Spark 1.2 (public Model API).** Set the default model and any
|
||||
model-pinned agent frontmatter to `meta/muse-spark-1.2`. ECR supplies the fixed
|
||||
- **Meta / Muse Spark 1.3 (public Model API).** Set the default model and any
|
||||
model-pinned agent frontmatter to `meta/muse-spark-1.3`. ECR supplies the fixed
|
||||
public Responses endpoint and the `@ai-sdk/openai` adapter; the generic
|
||||
Chat-Completions compatibility adapter is intentionally not used because it
|
||||
loses Muse's reasoning continuity across tool turns. Standard and Contributor
|
||||
model ids are supported.
|
||||
|
||||
```jsonc
|
||||
"model": "meta/muse-spark-1.2",
|
||||
"model": "meta/muse-spark-1.3",
|
||||
"auth": { "providers": {
|
||||
"meta": { "mode": "api-key", "tokenEnv": "META_API_KEY" }
|
||||
} }
|
||||
```
|
||||
|
||||
Standard and Contributor 1.2/1.3 model ids have known capability metadata. Any
|
||||
future model explicitly named as `meta/<model>` is still routed to the fixed Meta
|
||||
endpoint with neutral metadata, so a renamed family such as `meta/muse-foo` does
|
||||
not require an ECR release; Meta validates an unknown id on its first request.
|
||||
|
||||
Create the key in the [Meta AI developer portal](https://developer.meta.com/ai/).
|
||||
Locally, export it as `META_API_KEY`. In each review-running workflow, replace
|
||||
the Anthropic credential line with
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "1.30.0",
|
||||
"@opencode-ai/sdk": "1.18.4",
|
||||
"@typesafe-ai/sdk": "0.6.0",
|
||||
"cheerio": "1.1.2",
|
||||
"minisearch": "7.2.0",
|
||||
"opencode-ai": "1.18.4",
|
||||
@@ -109,6 +110,8 @@
|
||||
|
||||
"@types/node": ["@types/[email protected]", "", { "dependencies": { "undici-types": "~5.26.4" } }, "sha512-DO+2/jZinXfROG7j7WKFn/3C6nFwxy2lLpgLjEXJz+0XKphZlTLJ14mo8Vfg8X5BWN6XjyESXq+LcYdT7tR3bA=="],
|
||||
|
||||
"@typesafe-ai/sdk": ["@typesafe-ai/[email protected]", "", {}, "sha512-IddX+Q0XM+VagOUZFeP7wZjaO4SHMdvnh2zEBdrZZnXedWI3BNK1lKhMx3ayrkFWvVLbVcUHJy6AVZlY+e6Jaw=="],
|
||||
|
||||
"accepts": ["[email protected]", "", { "dependencies": { "mime-types": "^3.0.0", "negotiator": "^1.0.0" } }, "sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng=="],
|
||||
|
||||
"ajv": ["[email protected]", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA=="],
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
**Author:** Philippe Loulidi / Claude
|
||||
**Date:** 2026-07-30
|
||||
**Role:** Root
|
||||
**Related:** [LLP 0001](0001-trust-model.principles.md), [LLP 0002](0002-review-engine-pipeline.explainer.md), [LLP 0003](0003-model-runtimes-and-credentials.explainer.md), [LLP 0004](0004-diff-noise-and-prompts.explainer.md), [LLP 0005](0005-verification-fingerprints-rendering.explainer.md), [LLP 0006](0006-config-schema-loading-routing.explainer.md), [LLP 0007](0007-cli-commands-and-ci.explainer.md), [LLP 0008](0008-sources-and-reporters.explainer.md), [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md), [LLP 0013](0013-platform-research.explainer.md)
|
||||
**Related:** [LLP 0001](0001-trust-model.principles.md), [LLP 0002](0002-review-engine-pipeline.explainer.md), [LLP 0003](0003-model-runtimes-and-credentials.explainer.md), [LLP 0004](0004-diff-noise-and-prompts.explainer.md), [LLP 0005](0005-verification-fingerprints-rendering.explainer.md), [LLP 0006](0006-config-schema-loading-routing.explainer.md), [LLP 0007](0007-cli-commands-and-ci.explainer.md), [LLP 0008](0008-sources-and-reporters.explainer.md), [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md), [LLP 0013](0013-platform-research.explainer.md), [LLP 0014](0014-jev-selective-verification.explainer.md)
|
||||
|
||||
This is the root map for `@expo/code-review-cli` (`ecr`). It records what the system does, how the subsystems divide, the invariants that hold across all of them, and where each concern's rationale lives. It is intentionally thin: every subsystem's detail belongs in the doc that owns it. Read this first, then follow the link for the part you care about.
|
||||
|
||||
@@ -29,6 +29,7 @@ The code divides by concern. Each directory maps to the doc that owns its ration
|
||||
- `src/sources/` (`local-git`, `github-pr`) + `src/reporters/` (`terminal`, `github`) — where the diff comes from and where findings go [observed: AGENTS.md:14-15; src/sources/, src/reporters/]. Owned by [LLP 0008](0008-sources-and-reporters.explainer.md).
|
||||
- `templates/` + the scaffolded CI workflow — the files `ecr init` writes into an adopting repo; they define adopter-facing behavior and must stay in sync with the code [observed: AGENTS.md:19-20]. Owned by [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md).
|
||||
- `src/research-mcp/` + `research/` — the bundled read-only documentation MCP, scoped remote discovery and allowlisted fetch adapters, optional index builder, source catalog, and host-side evidence prepass. Owned by [LLP 0013](0013-platform-research.explainer.md).
|
||||
- `src/core/jev.ts` — optional typed first stage of finding verification against bounded local source. Owned by [LLP 0014](0014-jev-selective-verification.explainer.md).
|
||||
- The trust model spans all of the above and is stated as principles in [LLP 0001](0001-trust-model.principles.md).
|
||||
|
||||
## Cross-Cutting Invariants
|
||||
@@ -70,5 +71,6 @@ Start here, then follow the owning doc.
|
||||
| What do the CLI commands do, and how does CI orchestrate them? | [LLP 0007](0007-cli-commands-and-ci.explainer.md) | `src/cli.ts`, `src/commands/*` |
|
||||
| Where does the diff come from and where do findings go? | [LLP 0008](0008-sources-and-reporters.explainer.md) | `src/sources/*`, `src/reporters/*` |
|
||||
| What does `ecr init` scaffold into an adopting repo? | [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md) | `templates/`, scaffolded CI workflow |
|
||||
| How does Jev make narrow decisions and defer uncertain findings? | [LLP 0014](0014-jev-selective-verification.explainer.md) | `src/core/jev.ts`, `src/core/verify.ts` |
|
||||
|
||||
Non-doc sources of truth: `AGENTS.md` holds the working conventions (import style, error handling, the `opencode-ai`/`@opencode-ai/sdk` pinning rule, security invariants); `ROADMAP.md` lists open and deferred items — incremental review, inline PR comments, result-level caching, and a Codex token rotator among them are still open and must not be assumed implemented [observed: ROADMAP.md]; and `templates/` is the adopter-facing contract, so treat it as behavior, not sample text.
|
||||
@@ -11,7 +11,7 @@
|
||||
|
||||
## Pipeline Stages
|
||||
|
||||
`runReview()` runs a fixed sequence: filter noise → resolve the read root (the trust boundary) → start the engines → preflight that every model id resolves → chunk the diff → run agent×chunk passes plus one cross-cutting pass through a growable bounded queue → coordinate → verify → suppress → reconcile the summary → log. The whole run's resource lifecycle — engine handles, auth material, cwd, and the read-root worktree — is released in a single `finally` (`review.ts:947-951`), so no partial-startup path leaks a server or a worktree.
|
||||
`runReview()` runs a fixed sequence: filter noise → resolve the read root (the trust boundary) → start the engines → preflight that every model id resolves → chunk the diff → run agent×chunk passes plus one cross-cutting pass through a growable bounded queue → coordinate → verify → suppress → reconcile the summary → log. Optional Jev classification is the first stage inside verification: clear local judgments act and uncertainty defers to the reasoning verifier. [LLP 0014](0014-jev-selective-verification.explainer.md) owns that boundary. The whole run's resource lifecycle — engine handles, auth material, cwd, and the read-root worktree — is released in a single `finally` (`review.ts:947-951`), so no partial-startup path leaks a server or a worktree.
|
||||
|
||||
Two ordering decisions inside that lifecycle are load-bearing and easy to break by "cleaning up":
|
||||
|
||||
@@ -77,7 +77,7 @@ Rate-limit evidence is tracked per engine separately and surfaced with the cause
|
||||
|
||||
`writeRunLog()` (`src/core/log.ts`) appends one JSON line per run — cost, tokens (run-wide and per bucket), findings, coverage notes, decision — so runs are auditable and cost/latency measurable later. The record deliberately excludes PR title and body text, keeping only `baseRef`/`headRef`, to avoid persisting secrets that might appear in author-controlled text [observed] (`log.ts:13-18`). Findings may quote changed source lines, but only content the review already publishes verbatim in the PR comment.
|
||||
|
||||
Each successful reviewer pass may also return a bounded conclusion-only trace: up to three concrete checks and two unresolved questions, with each note capped at 240 characters. Trace parsing is deliberately fail-soft: malformed optional diagnostics are dropped while valid findings from the same response survive, so observability can never create a coverage gap. `runReview()` merges and de-duplicates valid notes per reviewer, sorts reviewer ids, caps both the stored roster and the complete decoded trace (6 KB), labels the result `unverified-model-diagnostics`, and attaches it to `CoordinatorOutput.reviewTrace` only after the decision pipeline finishes. `attachReviewTrace` strips any coordinator-authored value before attaching the engine-built trace; even a malformed injected coordinator value parses as absent rather than failing consolidation. The trace therefore never reaches coordination, verification, suppression, or decision logic. Because the GitHub comment already embeds the full coordinator output as durable state, downstream agents can recover the trace without adding visible review prose or storing raw transcripts and chain-of-thought. The same trace rides the run log for short-lived operational inspection [observed: `schema.ts` `ReviewerTraceNotesSchema`/`ReviewTraceSchema`/`ReviewerOutputSchema`; `review.ts` `mergeTraceNotes`/`buildReviewTrace`/`attachReviewTrace`; `render.ts` embeds the full review].
|
||||
Each successful reviewer pass may also return a bounded conclusion-only trace: up to three concrete checks and two unresolved questions, with each note capped at 240 characters. Trace parsing is deliberately fail-soft: malformed optional diagnostics are dropped while valid findings from the same response survive, so observability can never create a coverage gap. `runReview()` merges and de-duplicates valid notes per reviewer, sorts reviewer ids, caps both the stored roster and the complete decoded trace (6 KB), labels the result `unverified-model-diagnostics`, and attaches it to `CoordinatorOutput.reviewTrace` only after the decision pipeline finishes. `attachReviewTrace` strips any coordinator-authored value before attaching the engine-built trace; even a malformed injected coordinator value parses as absent rather than failing consolidation. The trace therefore never reaches coordination, verification, suppression, or decision logic. Because the GitHub comment already embeds the full coordinator output as durable state, downstream agents can recover the trace without adding visible review prose or storing raw transcripts and chain-of-thought. The same trace rides the run log for short-lived operational inspection [observed: `schema.ts` `ReviewerTraceNotesSchema`/`ReviewTraceSchema`/`ReviewerOutputSchema`; `review.ts` `mergeTraceNotes`/`buildReviewTrace`/`attachReviewTrace`; `render.ts` embeds the full review]. Optional Jev aggregate usage is a second log sink; individual suppressions use the ordinary `verifierDropped` audit trail, and neither contains extra source text.
|
||||
|
||||
Observability must never break a review, so both output sinks swallow their own errors: `safeLog` wraps `writeRunLog` and discards any write error [observed] (`review.ts:1269-1275`), and `appendStepSummary` is a no-op outside Actions and swallows append failures [observed] (`step-summary.ts:9-19`). A consequence worth stating: a missing or corrupt run-log entry is therefore **not** an error signal — the log's presence or absence proves nothing about whether the review succeeded.
|
||||
|
||||
|
||||
@@ -109,12 +109,15 @@ the Responses API and preserve encrypted reasoning across tool turns. The generi
|
||||
Completions path loses that continuity [observed:
|
||||
`opencode.ts` `META_MODEL_API_BASE_URL` / `META_MUSE_MODELS` / `buildOpencodeConfig`].
|
||||
|
||||
The declared model catalog is an allowlist (`muse-spark-1.2` and
|
||||
`muse-spark-1.2-contributor`), not a reflection of arbitrary config strings. This keeps
|
||||
the existing preflight honest: a misspelled or not-yet-supported Meta model remains
|
||||
absent from the server's provider model list and fails once before any pass. Both models
|
||||
declare their public context/output limits, multimodal inputs, high reasoning, and
|
||||
encrypted reasoning inclusion [observed: `opencode.ts` `META_MUSE_MODELS`].
|
||||
Provider selection is forward-compatible at the provider boundary, not coupled to the
|
||||
current Muse family name: every referenced non-empty `meta/<model>` id is declared on
|
||||
the synthesized Meta provider. Known Muse Spark 1.2 and 1.3 standard/Contributor ids
|
||||
receive their published limits, multimodal inputs, high reasoning, and encrypted
|
||||
reasoning inclusion from `META_MUSE_MODELS`; a future id such as `meta/muse-foo` is
|
||||
declared with neutral metadata rather than guessed capabilities. This deliberately
|
||||
moves validation of an unknown-but-well-formed Meta id from OpenCode's local preflight
|
||||
to its first Meta request, in exchange for new Meta names working without an ECR
|
||||
release [observed: `opencode.ts` `META_MUSE_MODELS` / `buildOpencodeConfig`].
|
||||
|
||||
## Claude Code CLI Containment
|
||||
|
||||
|
||||
@@ -19,7 +19,8 @@ caller, [LLP 0007](0007-cli-commands-and-ci.explainer.md) and [LLP 0001](0001-tr
|
||||
## Root vs Scope Config
|
||||
|
||||
A scope config is the root schema minus the centrally locked keys. `ScopeReviewConfigSchema`
|
||||
takes `ReviewConfigSchema`, omits `auth`, `breakGlass`, and `commentTag`, and re-adds each as
|
||||
takes `ReviewConfigSchema`, omits centrally owned keys including `auth`, `breakGlass`,
|
||||
`commentTag`, `research`, and `jev`, and re-adds each as
|
||||
`z.never().optional()` so declaring any of them is a hard parse error, not a later runtime
|
||||
check [observed: `schema.ts:232-247`]. The point of failing at the Zod level is that an IDE or
|
||||
`doctor` catches it before CI does [observed: `schema.ts:224-225` comment]. The remaining
|
||||
@@ -84,6 +85,13 @@ commit `43b31a4` follow-up]. The actual secret values never enter this subsystem
|
||||
owns the wiring and the `FORBIDDEN_TOKEN_ENVS` list, and schema/load only ever touch the env-var
|
||||
NAME [observed: `AGENTS.md` security invariants; `src/core/auth.ts:28,45,206`].
|
||||
|
||||
`jev` is root-only for the same trust reason as research: adding it sends bounded
|
||||
repository-derived text to another provider. A scope may neither activate it nor alter
|
||||
its model, threshold, or limits. Nested scopes inherit the root's resolved settings, so
|
||||
one PR has one centrally owned disclosure policy [observed: `schema.ts`
|
||||
`ScopeReviewConfigSchema`; `load.ts` `loadScopeConfig`]. The data boundary and active
|
||||
selective cascade live in [LLP 0014](0014-jev-selective-verification.explainer.md).
|
||||
|
||||
## Model Resolution
|
||||
|
||||
The `REVIEWER_MODEL` env override is applied on top of frontmatter/default model resolution using
|
||||
@@ -184,8 +192,7 @@ hardcoded to a read-only tool set (`read`/`grep`/`glob`/`list`) with no schema k
|
||||
write/bash [observed: `load.ts:126-155`, `load.ts:22-23`].
|
||||
|
||||
`resolveConfigDir` is the escape hatch (graft 1): precedence is explicit param → `ECR_CONFIG_DIR` env
|
||||
→ the repo's default `.expo-code-review/`, and with neither override the result is byte-identical to
|
||||
the default path [observed: `load.ts:29-41`]. `hasConfig` resolves the config dir the same way, so
|
||||
→ the repo's default setup dir [observed: `load.ts`]. **The default setup dir has two spellings** (0.15.0): `.expo-agents/code-review/` is the home — one `.expo-agents/` directory holds every Expo agent tool's per-repo files (this tool's and `@expo/verify`'s) instead of one root dot-directory per tool — and the pre-0.15 `.expo-code-review/` stays fully supported. `configDirFor(base)` returns whichever exists, new name first, and the new name when neither does; it is used for the root and for every scope directory, `ecr init` writes into an existing legacy directory rather than doubling it, the config-file sweep (`isConfigDirPath`) and the reference index accept both, and the CI templates upload `.runs/reviews.jsonl` from either location. `.agents/` was rejected as the shared name because Codex, Cursor, Gemini CLI and Copilot already scan `.agents/skills/`. `hasConfig` resolves the config dir the same way, so
|
||||
`doctor`'s "no config" diagnostic can never disagree with what the loader actually reads
|
||||
[observed: `load.ts:55-60`]. The override designates an alternate ROOT config dir, so `config.jsonc`
|
||||
and `routing.jsonc` always travel together: `loadRoutingManifest` reads through the same
|
||||
|
||||
@@ -56,7 +56,7 @@ The three workflows share a posture built entirely around one fact: a reviewer j
|
||||
**Concurrency defaults by auth mode.** The commented `chunk.concurrency` default is 6 for an API key and 3 for a subscription (OAuth) credential, "one account handles many parallel streams poorly, and several PRs may review on the same credential at once" [observed: `templates/config.jsonc:28-30`].
|
||||
|
||||
**Muse Spark is opt-in without weakening the default secret surface.** The root config
|
||||
documents `meta/muse-spark-1.2` plus a `META_API_KEY` auth entry, and each
|
||||
documents `meta/muse-spark-1.3` plus a `META_API_KEY` auth entry, and each
|
||||
review-running workflow shows the corresponding secret line commented out. It is not
|
||||
forwarded by the default scaffold: adopters replace the Anthropic line and set
|
||||
`ECR_EXPECTED_TOKEN_ENV=META_API_KEY`, so the model process receives only the
|
||||
@@ -64,6 +64,15 @@ credential actually selected by trusted config. The default remains Anthropic an
|
||||
`dismiss.yml` still receives no model credential [observed: `templates/config.jsonc`;
|
||||
`templates/workflow.yml`, `command.yml`, and `atlantis.yml`].
|
||||
|
||||
**Jev is an independent optional secret.** Each review-running workflow maps
|
||||
`TYPESAFE_API_KEY`; adding `jev` to trusted root config activates the selective cascade.
|
||||
GitHub supplies an empty value when the repository secret is absent, so the original
|
||||
verification path remains available. ECR sends bounded finding/file source directly
|
||||
to TypeSafe and logs aggregate usage; the key never enters `auth.tokenEnv` or reviewer
|
||||
engines. `dismiss.yml` still receives no model or Jev credential.
|
||||
[LLP 0014](0014-jev-selective-verification.explainer.md) owns the disclosure and
|
||||
fallback rules.
|
||||
|
||||
**Keep the `**/*` catch-all first in `routing.jsonc`.** Scopes match last-match-wins, so a `**/*` catch-all must come first and more specific scopes come after to override it [observed: `templates/routing.jsonc:27-31`; `AGENTS.md:17-18`]. Nothing validates the order — reversing it silently changes which config applies to already-routed files [inferred: no code checks that the `**/*` catch-all comes first, so reversing it silently changes routing].
|
||||
|
||||
**`scope-config.jsonc` deliberately omits `auth` and `commentTag`.** It is a *different* template from the root `config.jsonc`, not a copy. A scope config carries no `auth` block — credentials are root-only, and the loader plus the CI guard reject a scope config that declares one [observed: `templates/scope-config.jsonc:1-3`; `AGENTS.md:51`]. It also omits `commentTag`, because a scope's PR-comment marker is always derived as `<rootTag>:<scope-name>` so `ecr ci` and `ecr review --scope --post` target the same comment; declaring one is rejected by the scope schema [observed: `templates/scope-config.jsonc:23-25`]. Reusing the root `config.jsonc` template for a scope — a tempting "simplification" — would break the trust boundary (LLP 0006).
|
||||
|
||||
@@ -228,7 +228,16 @@ the queries and bounded results.
|
||||
|
||||
The built-in source catalog covers Apple, Android, platform releases, Swift
|
||||
Evolution, SDWebImage, Media3, Glide, OkHttp, Kotlin coroutines, Gradle/AGP, selected JetBrains
|
||||
issues, Expo, React Native, Reanimated, Gesture Handler, Screens, and Worklets.
|
||||
issues, Expo, React, React Native, WHATWG and W3C web specifications, Chrome DevTools
|
||||
Protocol, Flow, TypeScript, the Android NDK, CMake, CocoaPods, Metro, Reanimated,
|
||||
Gesture Handler, Screens, and Worklets. The React Native
|
||||
additions cover the contracts its implementation must preserve: React semantics, its
|
||||
web-compatible APIs, the repository's type systems, and its native build and dependency
|
||||
managers. Hermes documentation is intentionally not fetched from the engine's `main`
|
||||
branch because it can disagree with React Native's vendored revision. Repository PR and
|
||||
issue history is intentionally not a
|
||||
live evidence provider: it belongs in a versioned offline evaluation corpus used to
|
||||
improve prompts, while the runtime MCP remains an API-contract channel.
|
||||
The canonical OkHttp host is backed by both the
|
||||
[maintainer's transfer announcement](https://jakewharton.com/the-lysine-contingency/)
|
||||
and [Commonhaus's project-transfer record](https://www.commonhaus.org/activity/315.html).
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
# LLP 0014: Jev Selective Verification
|
||||
|
||||
**Type:** Explainer
|
||||
**Status:** Active
|
||||
**Systems:** Engine, Config, Security, Observability, Templates
|
||||
**Author:** Expo
|
||||
**Date:** 2026-09-21
|
||||
**Related:** [LLP 0001](0001-trust-model.principles.md), [LLP 0002](0002-review-engine-pipeline.explainer.md), [LLP 0005](0005-verification-fingerprints-rendering.explainer.md), [LLP 0006](0006-config-schema-loading-routing.explainer.md), [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md)
|
||||
|
||||
Jev is a discriminative model from TypeSafe AI. It answers typed questions over
|
||||
provided state and returns a probability distribution; it does not browse a repo,
|
||||
discover open-ended defects, explain causality, or write remediation. ECR therefore
|
||||
uses Jev inside verification, after generative reviewers have produced findings.
|
||||
|
||||
## Decision
|
||||
|
||||
ECR uses an active `generate → classify → defer` cascade. Jev is neither a shadow
|
||||
observer nor another general reviewer agent:
|
||||
|
||||
1. generative reviewers discover and explain candidate defects;
|
||||
2. deterministic code confines paths and grades quoted evidence;
|
||||
3. Jev makes one atomic local support judgment per finding;
|
||||
4. high-confidence support keeps the finding, while high-confidence contradiction
|
||||
drops only ordinary correctness/quality findings;
|
||||
5. uncertainty and protected findings defer to the existing reasoning verifier.
|
||||
|
||||
Adding the root `jev` config activates the cascade. There is no separate `enabled`
|
||||
flag and no shadow phase. If the config, credential, or a usable response is absent,
|
||||
the finding follows the pre-Jev verification rules.
|
||||
|
||||
## Evidence Survey
|
||||
|
||||
The primary implementation sources are TypeSafe's [Jev announcement](https://typesafe.ai/blog/introducing-system-one-models-and-jev),
|
||||
[System One guide](https://docs.typesafe.ai/concepts/how-to-build-with-system-one),
|
||||
[confidence guidance](https://docs.typesafe.ai/confidence),
|
||||
[confidence-routing pattern](https://docs.typesafe.ai/patterns/confidence-routing),
|
||||
[citation-check cookbook](https://docs.typesafe.ai/cookbooks/citation_check),
|
||||
[software-development cascade](https://docs.typesafe.ai/cookbooks/sde_cascade),
|
||||
[Jev 1.13 model notes](https://docs.typesafe.ai/model-jaggedness/jev-1.13), and the
|
||||
open-source [JavaScript SDK](https://github.com/typesafe-ai/typesafe-sdk-js).
|
||||
|
||||
The consistent vendor guidance is to keep deterministic control flow in code, ask
|
||||
narrow independent questions over relevant state, act only above a risk-appropriate
|
||||
confidence threshold, and route uncertain results to a stronger reasoner. Confidence
|
||||
measures concentration of the answer distribution, not factual correctness. Jev is
|
||||
also documented as weak on arithmetic, counting, dates, complex indirection, and
|
||||
irrelevant context. These limits rule out using it as the only verifier.
|
||||
|
||||
The architecture also follows older work on selective prediction and abstention:
|
||||
[SelectiveNet](https://proceedings.mlr.press/v97/geifman19a),
|
||||
[Learning to Defer](https://proceedings.mlr.press/v119/mozannar20b.html),
|
||||
[Calibrated Learning to Defer](https://proceedings.mlr.press/v162/verma22c.html), and
|
||||
[Language Model Cascades](https://arxiv.org/abs/2207.10342). Their shared idea is
|
||||
that a useful classifier should act on its competence region and reject or defer the
|
||||
rest. [Generative Verifiers](https://openreview.net/forum?id=Ccwp4tFEtE) further
|
||||
supports retaining a generative verifier for multi-step reasoning rather than treating
|
||||
a discriminative score as universal proof.
|
||||
|
||||
Code-review-specific evidence argues for conservative authority. Research on
|
||||
[LLM code-review overcorrection](https://arxiv.org/abs/2603.00539),
|
||||
[security code review](https://arxiv.org/abs/2401.16310), and
|
||||
[judge calibration and bias](https://aclanthology.org/2025.acl-long.808/) shows that
|
||||
binary judgments can be confidently wrong and that security claims need stronger
|
||||
grounding. Vendor benchmarks measure agreement with other model outputs rather than
|
||||
human-adjudicated ECR findings, so they establish speed and structured behavior, not
|
||||
an acceptable false-negative rate for this repository.
|
||||
|
||||
The useful philosophical analogy is dual-process reasoning: Jev supplies a fast,
|
||||
bounded System 1 judgment; the tool-using verifier supplies slower System 2 analysis.
|
||||
The relevant engineering principle is a reject option, not majority voting. Jev's
|
||||
different model family adds diversity, but agreement between two models is not ground
|
||||
truth and disagreement is a reason to inspect, not to average blindly.
|
||||
|
||||
## Active Selective Cascade
|
||||
|
||||
The policy is intentionally asymmetric:
|
||||
|
||||
| Jev result | Ordinary correctness/quality | Critical, security, secrets, or cited |
|
||||
| --- | --- | --- |
|
||||
| supported at or above threshold | keep | cited findings defer; other protected findings keep |
|
||||
| contradicted at or above threshold | drop with an audit reason | defer to reasoning verifier |
|
||||
| `needs_reasoning` or low confidence | defer to reasoning verifier | defer to reasoning verifier |
|
||||
| missing response or provider failure | use original verification path | use original verification path |
|
||||
|
||||
“Protected” controls the dangerous direction: Jev may help retain such a finding, but
|
||||
cannot suppress it alone. Cited findings always defer because Jev receives repository
|
||||
source, not the audited external passage whose support must be checked.
|
||||
|
||||
The default `minConfidence` is `0.9`. It is a conservative operating point, not a
|
||||
claim of 90% correctness. Repositories may change it explicitly, but model version,
|
||||
question text, state shape, threshold, and truncation policy form one classifier and
|
||||
must be reviewed together. A dated model is preferred over a moving alias.
|
||||
|
||||
## State and Security Boundaries
|
||||
|
||||
Each request includes finding category, severity, file, line, title, rationale, and
|
||||
quoted evidence plus bounded source from that file. ECR sends no PR title/body,
|
||||
unrelated file, agent transcript, author reply, or credential. `maxContextChars`
|
||||
bounds disclosure and context rot; truncation centers on the quoted evidence where
|
||||
possible. Four requests run concurrently, with a bounded timeout and one retry.
|
||||
|
||||
The API origin is explicit. Ambient TypeSafe URL/model variables cannot redirect the
|
||||
request. SDK logging is off because bodies contain source. `TYPESAFE_API_KEY` is
|
||||
captured before reviewer startup, withheld from OpenCode, and forbidden as a reviewer
|
||||
provider credential. Paths remain confined to the materialized review tree before
|
||||
any source is read.
|
||||
|
||||
Jev is not a prompt-injection detector or security boundary. The code and finding
|
||||
text are attacker-controlled inputs, and model output is schema-validated before the
|
||||
policy sees it. Deterministic code continues to own credential isolation, path
|
||||
confinement, evidence matching, category floors, and final decision derivation.
|
||||
|
||||
## Failure and Observability
|
||||
|
||||
The cascade fails back, not closed. Missing credentials disable only Jev for that run.
|
||||
An individual timeout, malformed response, or provider error sends that finding
|
||||
through the original verification path. Error text is not persisted because providers
|
||||
may echo source. The run log stores aggregate model, duration, token, cost, evaluated,
|
||||
and failure counts; any finding dropped by Jev appears in the ordinary
|
||||
`verifierDropped` audit trail with its confidence.
|
||||
|
||||
Published pricing for the pinned integration is $0.042 per million input tokens with
|
||||
no output-token charge. Cost is recorded in the `jev-verifier` bucket. Provider limits
|
||||
are not treated as local capacity guarantees.
|
||||
|
||||
## What Jev Replaces
|
||||
|
||||
Jev replaces only straightforward finding verification calls. It does not replace:
|
||||
|
||||
- reviewer agents that discover bugs and supply explanations;
|
||||
- the coordinator that merges and prioritizes findings;
|
||||
- deterministic evidence, trust, and security checks;
|
||||
- the reasoning verifier for ambiguity, multi-file behavior, runtime semantics,
|
||||
citations, or protected negative judgments.
|
||||
|
||||
This division gives Jev real authority and cost benefit without pretending that a
|
||||
fast typed classifier is a complete code reviewer.
|
||||
|
||||
## Rejected Alternatives
|
||||
|
||||
**A standalone `jev.md` reviewer agent.** Jev cannot use ECR's browsing tools or emit
|
||||
the finding schema's explanation and remediation. Presenting it as a peer reviewer
|
||||
would hide a different and narrower capability behind the same abstraction.
|
||||
|
||||
**Shadow-only operation.** It adds latency and cost without improving the review. The
|
||||
safe boundary comes from abstention, protected categories, and fallback behavior, not
|
||||
from making every result inert.
|
||||
|
||||
**Replace all verification.** Local classification cannot resolve missing repository
|
||||
context, multi-step execution, arithmetic, temporal behavior, or external citations.
|
||||
|
||||
**Use confidence as proof.** A concentrated distribution can still be wrong. The
|
||||
threshold is a routing policy and protected contradictions still require reasoning.
|
||||
|
||||
**Send the entire diff or repository.** Unrelated context increases disclosure and
|
||||
context rot. One bounded file is the smallest state that supports the local question.
|
||||
@@ -45,6 +45,7 @@
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "1.30.0",
|
||||
"@opencode-ai/sdk": "1.18.4",
|
||||
"@typesafe-ai/sdk": "0.6.0",
|
||||
"cheerio": "1.1.2",
|
||||
"minisearch": "7.2.0",
|
||||
"opencode-ai": "1.18.4",
|
||||
|
||||
@@ -141,6 +141,94 @@ const cases = [
|
||||
title: "FlatList",
|
||||
body: "getItemLayout is an optional optimization for FlatList. It lets the list skip the measurement of dynamic content when the row height is known ahead of time.",
|
||||
},
|
||||
{
|
||||
label: "react",
|
||||
provider: "react",
|
||||
platform: "react-native",
|
||||
providers: ["react"],
|
||||
query: "useEffect dependency array",
|
||||
expected: "react.dev/reference/react/useEffect",
|
||||
url: "https://react.dev/reference/react/useEffect",
|
||||
title: "useEffect",
|
||||
body: "useEffect synchronizes a component with an external system. Every reactive value read by the effect belongs in its dependency array.",
|
||||
},
|
||||
{
|
||||
label: "flow",
|
||||
provider: "flow",
|
||||
platform: "react-native",
|
||||
providers: ["flow"],
|
||||
query: "exact object types",
|
||||
expected: "flow.org/en/docs/types/objects",
|
||||
url: "https://flow.org/en/docs/types/objects/",
|
||||
title: "Object Types",
|
||||
body: "An exact object type rejects extra properties. Flow object types are exact by default unless they are explicitly declared inexact.",
|
||||
},
|
||||
{
|
||||
label: "typescript",
|
||||
provider: "typescript",
|
||||
platform: "react-native",
|
||||
providers: ["typescript"],
|
||||
query: "declaration file module exports",
|
||||
expected: "typescriptlang.org/docs/handbook/declaration-files",
|
||||
url: "https://www.typescriptlang.org/docs/handbook/declaration-files/templates/module-d-ts.html",
|
||||
title: "Modules .d.ts",
|
||||
body: "A module declaration file describes the runtime exports of a JavaScript module. Its export shape must match the values consumers can import.",
|
||||
},
|
||||
{
|
||||
label: "web-standards",
|
||||
provider: "web-standards",
|
||||
platform: "react-native",
|
||||
providers: ["web-standards"],
|
||||
sourceKind: "standard",
|
||||
query: "AbortSignal fetch cancellation",
|
||||
expected: "fetch.spec.whatwg.org",
|
||||
url: "https://fetch.spec.whatwg.org/",
|
||||
title: "Fetch Standard",
|
||||
body: "An AbortSignal can abort a fetch operation. The fetch promise rejects when the request is terminated by the signal.",
|
||||
},
|
||||
{
|
||||
label: "chrome-devtools-protocol",
|
||||
provider: "chrome-devtools-protocol",
|
||||
platform: "react-native",
|
||||
providers: ["chrome-devtools-protocol"],
|
||||
sourceKind: "standard",
|
||||
query: "Runtime evaluate awaitPromise",
|
||||
expected: "chromedevtools.github.io/devtools-protocol/tot/Runtime",
|
||||
url: "https://chromedevtools.github.io/devtools-protocol/tot/Runtime/",
|
||||
title: "Runtime domain",
|
||||
body: "Runtime.evaluate evaluates an expression on the inspected page. The awaitPromise option waits for a returned promise and reports its resolved result.",
|
||||
},
|
||||
{
|
||||
label: "android-ndk",
|
||||
provider: "android-ndk",
|
||||
providers: ["android-ndk"],
|
||||
query: "JNIEnv local references",
|
||||
expected: "developer.android.com/ndk/guides/jni-tips",
|
||||
url: "https://developer.android.com/ndk/guides/jni-tips",
|
||||
title: "JNI tips",
|
||||
body: "JNI local references are valid only for the duration of the current native method call and on the current thread. Long-lived references must be promoted to global references.",
|
||||
},
|
||||
{
|
||||
label: "cmake",
|
||||
provider: "cmake",
|
||||
providers: ["cmake"],
|
||||
query: "target_link_libraries transitive",
|
||||
expected: "cmake.org/cmake/help/latest/command/target_link_libraries",
|
||||
url: "https://cmake.org/cmake/help/latest/command/target_link_libraries.html",
|
||||
title: "target_link_libraries",
|
||||
body: "target_link_libraries links a target to libraries and can publish transitive link dependencies through the target's interface.",
|
||||
},
|
||||
{
|
||||
label: "cocoapods",
|
||||
provider: "cocoapods",
|
||||
platform: "apple",
|
||||
providers: ["cocoapods"],
|
||||
query: "Podfile use_frameworks linkage",
|
||||
expected: "guides.cocoapods.org/syntax/podfile",
|
||||
url: "https://guides.cocoapods.org/syntax/podfile.html",
|
||||
title: "Podfile Syntax Reference",
|
||||
body: "The use_frameworks directive configures pods to build as frameworks and accepts a linkage option for static or dynamic frameworks.",
|
||||
},
|
||||
{
|
||||
label: "react-native-reanimated",
|
||||
provider: "react-native-reanimated",
|
||||
|
||||
@@ -285,6 +285,52 @@ test("parseClaudeResult: success maps text/cost/tokens/model", () => {
|
||||
expect(r.structuredOutputFailure).toBe(false);
|
||||
});
|
||||
|
||||
test("parseClaudeResult: records thinking tokens as reasoning, without inflating output", () => {
|
||||
const stdout = JSON.stringify({
|
||||
type: "result",
|
||||
subtype: "success",
|
||||
is_error: false,
|
||||
result: "the findings",
|
||||
total_cost_usd: 0.5,
|
||||
usage: {
|
||||
input_tokens: 10,
|
||||
output_tokens: 200,
|
||||
output_tokens_details: { thinking_tokens: 170 },
|
||||
cache_creation_input_tokens: 0,
|
||||
cache_read_input_tokens: 0,
|
||||
},
|
||||
});
|
||||
const r = parseClaudeResult(stdout);
|
||||
expect(r.tokens.reasoning).toBe(170);
|
||||
expect(r.tokens.output).toBe(200);
|
||||
});
|
||||
|
||||
test("parseClaudeResult: omits reasoning when the CLI reports no breakdown", () => {
|
||||
const stdout = JSON.stringify({
|
||||
type: "result",
|
||||
subtype: "success",
|
||||
is_error: false,
|
||||
result: "the findings",
|
||||
total_cost_usd: 0.5,
|
||||
usage: { input_tokens: 10, output_tokens: 200 },
|
||||
});
|
||||
expect(parseClaudeResult(stdout).tokens.reasoning).toBeUndefined();
|
||||
});
|
||||
|
||||
test("parseClaudeResult: survives a null/!object output_tokens_details", () => {
|
||||
for (const details of [null, "nope", 7]) {
|
||||
const stdout = JSON.stringify({
|
||||
type: "result",
|
||||
subtype: "success",
|
||||
is_error: false,
|
||||
result: "x",
|
||||
total_cost_usd: 0,
|
||||
usage: { input_tokens: 1, output_tokens: 2, output_tokens_details: details },
|
||||
});
|
||||
expect(parseClaudeResult(stdout).tokens.reasoning).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
test("parseClaudeResult: reads the final result from stream-json JSONL", () => {
|
||||
const stdout = [
|
||||
JSON.stringify({ type: "system", subtype: "init", model: "claude-opus-5" }),
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { mkdir, mkdtemp, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
CONFIG_DIRNAME,
|
||||
LEGACY_CONFIG_DIRNAME,
|
||||
configDirFor,
|
||||
isConfigDirPath,
|
||||
resolveConfigDir,
|
||||
} from "../config/load.js";
|
||||
|
||||
describe("setup dir: .expo-agents/code-review with the legacy .expo-code-review fallback", () => {
|
||||
test("a fresh repo resolves to the new name", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "ecr-dir-"));
|
||||
expect(configDirFor(root)).toBe(path.join(root, CONFIG_DIRNAME));
|
||||
});
|
||||
|
||||
test("a pre-0.15 repo keeps using .expo-code-review unchanged", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "ecr-dir-"));
|
||||
await mkdir(path.join(root, LEGACY_CONFIG_DIRNAME), { recursive: true });
|
||||
expect(configDirFor(root)).toBe(path.join(root, LEGACY_CONFIG_DIRNAME));
|
||||
expect(resolveConfigDir(root)).toBe(path.join(root, LEGACY_CONFIG_DIRNAME));
|
||||
});
|
||||
|
||||
test("when both exist the new name wins", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "ecr-dir-"));
|
||||
await mkdir(path.join(root, LEGACY_CONFIG_DIRNAME), { recursive: true });
|
||||
await mkdir(path.join(root, CONFIG_DIRNAME), { recursive: true });
|
||||
await writeFile(path.join(root, CONFIG_DIRNAME, "config.jsonc"), "{}");
|
||||
expect(configDirFor(root)).toBe(path.join(root, CONFIG_DIRNAME));
|
||||
});
|
||||
|
||||
test("an explicit override still beats both", async () => {
|
||||
const root = await mkdtemp(path.join(tmpdir(), "ecr-dir-"));
|
||||
await mkdir(path.join(root, LEGACY_CONFIG_DIRNAME), { recursive: true });
|
||||
expect(resolveConfigDir(root, "custom")).toBe(path.join(root, "custom"));
|
||||
});
|
||||
|
||||
test("isConfigDirPath recognizes either spelling, nowhere else", () => {
|
||||
expect(isConfigDirPath("/r/.expo-code-review")).toBe(true);
|
||||
expect(isConfigDirPath("/r/apps/api/.expo-agents/code-review")).toBe(true);
|
||||
expect(isConfigDirPath("/r/.expo-agents")).toBe(false);
|
||||
expect(isConfigDirPath("/r/code-review")).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -503,7 +503,9 @@ test("setup problems are labelled by setup-dir position when the config lives ou
|
||||
setupDirs: [path.join(trustedBase, ".expo-code-review")],
|
||||
});
|
||||
expect(report.problems).toHaveLength(1);
|
||||
expect(report.problems[0]!.file).toBe(path.join(".expo-code-review", "agents", "security.md"));
|
||||
expect(report.problems[0]!.file).toBe(
|
||||
path.join(".expo-agents", "code-review", "agents", "security.md"),
|
||||
);
|
||||
});
|
||||
|
||||
test("citedPathsTouchedBy reports cited paths a PR changed, dirs included", async () => {
|
||||
|
||||
@@ -50,6 +50,36 @@ test("run with input: timeout kills the child and resolves timedOut (no throw)",
|
||||
expect(Date.now() - started).toBeLessThan(10_000);
|
||||
});
|
||||
|
||||
/**
|
||||
* Wait until `pid` no longer names a LIVE process. `kill(pid, 0)` alone is racy
|
||||
* here: it still succeeds while the killed grandchild is a zombie awaiting init's
|
||||
* reap (the direct `sh` dies first, the orphaned `sleep` reparents, and reaping is
|
||||
* asynchronous) — which is exactly what flaked on loaded CI runners. A zombie is
|
||||
* dead for this test's purpose, so a `ps` state of `Z` (or no row) counts as gone.
|
||||
* A process that genuinely survived the group kill (`sleep 30`) outlives the whole
|
||||
* poll window and still fails the assertion.
|
||||
*/
|
||||
async function processGone(pid: number, timeoutMs = 5000): Promise<boolean> {
|
||||
const started = Date.now();
|
||||
for (;;) {
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
} catch {
|
||||
return true; // ESRCH: fully gone
|
||||
}
|
||||
const state = (
|
||||
await run("ps", ["-o", "state=", "-p", String(pid)], { check: false })
|
||||
).stdout.trim();
|
||||
if (state === "" || state.startsWith("Z")) {
|
||||
return true; // reaped between the two checks, or dead-but-unreaped
|
||||
}
|
||||
if (Date.now() - started >= timeoutMs) {
|
||||
return false;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
}
|
||||
|
||||
test("run with input: timeout kills the whole process group, not just the direct child", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "ecr-exec-test-"));
|
||||
const pidFile = join(dir, "grandchild.pid");
|
||||
@@ -61,13 +91,14 @@ test("run with input: timeout kills the whole process group, not just the direct
|
||||
});
|
||||
expect(result.timedOut).toBe(true);
|
||||
const grandchildPid = Number((await readFile(pidFile, "utf8")).trim());
|
||||
// If only the direct `sh` had been signaled, this detached grandchild `sleep`
|
||||
// would still be alive; process.kill throws ESRCH once the whole group is gone.
|
||||
expect(() => process.kill(grandchildPid, 0)).toThrow();
|
||||
// If only the direct `sh` had been signaled, this grandchild `sleep 30` would
|
||||
// still be alive well past the poll window; the group kill leaves it dead (or
|
||||
// momentarily zombie) within milliseconds.
|
||||
expect(await processGone(grandchildPid)).toBe(true);
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
}, 10_000);
|
||||
|
||||
test("run with input: escalates to SIGKILL after the grace period when the child traps SIGTERM", async () => {
|
||||
const started = Date.now();
|
||||
|
||||
@@ -0,0 +1,369 @@
|
||||
import { test, expect } from "bun:test";
|
||||
|
||||
import {
|
||||
activeFindings,
|
||||
hasUnquotedMarker,
|
||||
inlineBodiesEqual,
|
||||
planInlineSync,
|
||||
} from "../reporters/github.js";
|
||||
import type { ExistingInlineComment } from "../reporters/github.js";
|
||||
import {
|
||||
inlineCommentMarker,
|
||||
inlineStubBody,
|
||||
oneLineRationale,
|
||||
parseInlineMarkerFp,
|
||||
renderInlineCommentBody,
|
||||
renderMarkdown,
|
||||
} from "../core/render.js";
|
||||
import type { LinkContext } from "../core/render.js";
|
||||
import { hasUnquotedInlineMarker, matchReplies, replyIsNewer } from "../core/responses.js";
|
||||
import type { ReplyComment } from "../core/responses.js";
|
||||
import { applyPins, fingerprintFinding } from "../core/schema.js";
|
||||
import type { CoordinatorOutput, FeedbackRecord, Finding } from "../core/schema.js";
|
||||
|
||||
const finding = (over: Partial<Finding> = {}): Finding => ({
|
||||
severity: "warning",
|
||||
category: "quality",
|
||||
file: "a.ts",
|
||||
line: 5,
|
||||
title: "Observer restarts for cached instances",
|
||||
rationale: "getInstances() restarts the push token observer for cached instances.",
|
||||
...over,
|
||||
});
|
||||
|
||||
const existing = (over: Partial<ExistingInlineComment> = {}): ExistingInlineComment => ({
|
||||
id: 1,
|
||||
fp: "aaaa11",
|
||||
body: "old",
|
||||
hasReplies: false,
|
||||
...over,
|
||||
});
|
||||
|
||||
const TAG = "expo-ai-code-reviewer";
|
||||
|
||||
// ---- marker format + anchored parse ----
|
||||
|
||||
test("inline marker roundtrips through the anchored parser", () => {
|
||||
const body = renderInlineCommentBody(finding(), TAG, "deadbeef11");
|
||||
expect(body.startsWith(inlineCommentMarker(TAG, "deadbeef11"))).toBe(true);
|
||||
expect(parseInlineMarkerFp(body, TAG)).toBe("deadbeef11");
|
||||
expect(parseInlineMarkerFp(inlineStubBody(TAG, "deadbeef11"), TAG)).toBe("deadbeef11");
|
||||
});
|
||||
|
||||
test("parseInlineMarkerFp: a marker anywhere but byte 0 parses to nothing", () => {
|
||||
const forged = `prose first\n${inlineCommentMarker(TAG, "deadbeef11")}`;
|
||||
expect(parseInlineMarkerFp(forged, TAG)).toBeNull();
|
||||
// A quote-reply carrying the marker behind `> ` is not identity either.
|
||||
expect(parseInlineMarkerFp(`> ${inlineCommentMarker(TAG, "deadbeef11")}`, TAG)).toBeNull();
|
||||
});
|
||||
|
||||
test("parseInlineMarkerFp: constrains the fp to the fingerprint alphabet and the exact tag", () => {
|
||||
expect(parseInlineMarkerFp(`<!-- ${TAG}:inline:fp=NOTHEX -->`, TAG)).toBeNull();
|
||||
expect(parseInlineMarkerFp(`<!-- other-tag:inline:fp=deadbeef11 -->`, TAG)).toBeNull();
|
||||
// A scoped tag never cross-parses under the root tag.
|
||||
expect(parseInlineMarkerFp(`<!-- ${TAG}:scope:inline:fp=deadbeef11 -->`, TAG)).toBeNull();
|
||||
});
|
||||
|
||||
// ---- inline body ----
|
||||
|
||||
test("renderInlineCommentBody: id token, severity, rationale, suggestion, and no state markers", () => {
|
||||
const body = renderInlineCommentBody(
|
||||
finding({ suggestion: "cache it", rationale: "watch <!-- evil --> out" }),
|
||||
TAG,
|
||||
"deadbeef11",
|
||||
);
|
||||
expect(body).toContain("`id:deadbeef11`");
|
||||
expect(body).toContain("🟡 Warning");
|
||||
expect(body).toContain("**Suggestion:** cache it");
|
||||
// Untrusted text can never smuggle a raw HTML comment opener into our body.
|
||||
expect(body).toContain("<!-- evil -->");
|
||||
expect(body).not.toMatch(/:(?:state|fingerprints)=/);
|
||||
});
|
||||
|
||||
test("inlineStubBody: neutral — never claims resolved or dismissed", () => {
|
||||
const stub = inlineStubBody(TAG, "deadbeef11");
|
||||
expect(stub.toLowerCase()).not.toContain("resolved");
|
||||
expect(stub.toLowerCase()).not.toContain("dismissed");
|
||||
expect(stub).toContain("main review comment");
|
||||
});
|
||||
|
||||
// ---- oneLineRationale ----
|
||||
|
||||
test("oneLineRationale: collapses whitespace, escapes <, truncates at a word boundary", () => {
|
||||
expect(oneLineRationale("a\n\nb c")).toBe("a b c");
|
||||
expect(oneLineRationale("<details>x</details>")).toBe("<details>x</details>");
|
||||
const long = Array.from({ length: 60 }, (_, i) => `word${i}`).join(" ");
|
||||
const short = oneLineRationale(long);
|
||||
expect(short.length).toBeLessThanOrEqual(161);
|
||||
expect(short.endsWith("…")).toBe(true);
|
||||
// Cut lands on a word boundary: the kept prefix is followed by a space in the input.
|
||||
const kept = short.slice(0, -1);
|
||||
expect(long.startsWith(kept)).toBe(true);
|
||||
expect(long[kept.length]).toBe(" ");
|
||||
});
|
||||
|
||||
// ---- planner ----
|
||||
|
||||
test("planInlineSync: creates new, patches live, stubs replied stale, deletes bare stale", () => {
|
||||
const target = { fp: "aaaa11", finding: finding() };
|
||||
const plan = planInlineSync({
|
||||
targets: [target, { fp: "bbbb22", finding: finding({ file: "b.ts" }) }],
|
||||
existing: [
|
||||
existing({ id: 10, fp: "aaaa11" }),
|
||||
existing({ id: 11, fp: "gone01", hasReplies: true }),
|
||||
existing({ id: 12, fp: "gone02", hasReplies: false }),
|
||||
],
|
||||
maxComments: 20,
|
||||
teardown: true,
|
||||
});
|
||||
expect(plan.patch.map((p) => p.comment.fp)).toEqual(["aaaa11"]);
|
||||
expect(plan.create.map((c) => c.fp)).toEqual(["bbbb22"]);
|
||||
expect(plan.stub.map((s) => s.fp)).toEqual(["gone01"]);
|
||||
expect(plan.remove.map((r) => r.fp)).toEqual(["gone02"]);
|
||||
});
|
||||
|
||||
test("planInlineSync: teardown=false is additive-only, but still removes duplicates of a kept fp", () => {
|
||||
const plan = planInlineSync({
|
||||
targets: [{ fp: "aaaa11", finding: finding() }],
|
||||
existing: [
|
||||
existing({ id: 10, fp: "aaaa11" }),
|
||||
existing({ id: 9, fp: "aaaa11" }), // older duplicate (crash window)
|
||||
existing({ id: 11, fp: "gone01" }),
|
||||
existing({ id: 12, fp: "gone02", hasReplies: true }),
|
||||
],
|
||||
maxComments: 20,
|
||||
teardown: false,
|
||||
});
|
||||
expect(plan.stub).toEqual([]);
|
||||
expect(plan.remove.map((r) => r.id)).toEqual([9]); // duplicate only, never live stale threads
|
||||
expect(plan.patch.map((p) => p.comment.id)).toEqual([10]); // newest duplicate is the live one
|
||||
});
|
||||
|
||||
test("planInlineSync: the cap is sticky — a live thread is never evicted for a new finding", () => {
|
||||
const critical = { fp: "cccc33", finding: finding({ severity: "critical", file: "c.ts" }) };
|
||||
const existingWarning = { fp: "aaaa11", finding: finding() };
|
||||
const plan = planInlineSync({
|
||||
targets: [critical, existingWarning],
|
||||
existing: [existing({ id: 10, fp: "aaaa11", hasReplies: true })],
|
||||
maxComments: 1,
|
||||
teardown: true,
|
||||
});
|
||||
// The warning already has a live thread; the (higher-severity) newcomer waits.
|
||||
expect(plan.patch.map((p) => p.comment.fp)).toEqual(["aaaa11"]);
|
||||
expect(plan.create).toEqual([]);
|
||||
expect(plan.stub).toEqual([]);
|
||||
});
|
||||
|
||||
test("planInlineSync: severity then fp orders admissions deterministically", () => {
|
||||
const plan = planInlineSync({
|
||||
targets: [
|
||||
{ fp: "bbbb22", finding: finding({ file: "b.ts" }) },
|
||||
{ fp: "aaaa11", finding: finding() },
|
||||
{ fp: "cccc33", finding: finding({ severity: "critical", file: "c.ts" }) },
|
||||
],
|
||||
existing: [],
|
||||
maxComments: 2,
|
||||
teardown: true,
|
||||
});
|
||||
expect(plan.create.map((c) => c.fp)).toEqual(["cccc33", "aaaa11"]);
|
||||
});
|
||||
|
||||
// ---- body compare + marker scan ----
|
||||
|
||||
test("inlineBodiesEqual is CRLF-insensitive (the API returns CRLF bodies)", () => {
|
||||
expect(inlineBodiesEqual("a\r\nb", "a\nb")).toBe(true);
|
||||
expect(inlineBodiesEqual("a\nb", "a\nc")).toBe(false);
|
||||
});
|
||||
|
||||
test("hasUnquotedMarker: quoted copies do not count", () => {
|
||||
const marker = inlineCommentMarker(TAG, "deadbeef11");
|
||||
expect(hasUnquotedMarker(`${marker}\nbody`, marker)).toBe(true);
|
||||
expect(hasUnquotedMarker(`> ${marker}\nmy actual reply`, marker)).toBe(false);
|
||||
});
|
||||
|
||||
// ---- reply-matcher backstop ----
|
||||
|
||||
test("hasUnquotedInlineMarker: tag-independent, unquoted lines only", () => {
|
||||
expect(hasUnquotedInlineMarker(renderInlineCommentBody(finding(), "any-tag", "abcdef"))).toBe(
|
||||
true,
|
||||
);
|
||||
expect(hasUnquotedInlineMarker(`> <!-- any-tag:inline:fp=abcdef -->\nreal reply`)).toBe(false);
|
||||
});
|
||||
|
||||
test("matchReplies: our inline body is never matched as a reply, even with a maintainer login", () => {
|
||||
const f = finding();
|
||||
const fp = fingerprintFinding(f);
|
||||
const body = renderInlineCommentBody(f, "some-other-tag", fp); // carries an unquoted id: token
|
||||
const records = matchReplies(
|
||||
[{ id: 5, body, login: "bot", maintainer: true }],
|
||||
[{ finding: f, fp }],
|
||||
{
|
||||
match: "both",
|
||||
},
|
||||
);
|
||||
expect(records).toEqual([]);
|
||||
});
|
||||
|
||||
test("matchReplies: a quote-reply of an inline comment still matches", () => {
|
||||
const f = finding();
|
||||
const fp = fingerprintFinding(f);
|
||||
const quoted = renderInlineCommentBody(f, TAG, fp)
|
||||
.split("\n")
|
||||
.map((line) => `> ${line}`)
|
||||
.join("\n");
|
||||
const records = matchReplies(
|
||||
[{ id: 5, body: `${quoted}\n\nThis is intentional.`, login: "author", maintainer: false }],
|
||||
[{ finding: f, fp }],
|
||||
{ match: "both" },
|
||||
);
|
||||
expect(records.map((r) => r.fp)).toEqual([fp]);
|
||||
expect(records[0]!.citedId).toBe(false); // a quoted id never counts as citing
|
||||
});
|
||||
|
||||
// ---- threadFp ----
|
||||
|
||||
test("matchReplies: threadFp matches in every match mode and never sets citedId", () => {
|
||||
const f = finding();
|
||||
const fp = fingerprintFinding(f);
|
||||
for (const match of ["quote", "id", "both"] as const) {
|
||||
const records = matchReplies(
|
||||
[
|
||||
{
|
||||
id: -50,
|
||||
body: "we cache these on purpose",
|
||||
login: "author",
|
||||
maintainer: false,
|
||||
threadFp: fp,
|
||||
},
|
||||
],
|
||||
[{ finding: f, fp }],
|
||||
{ match },
|
||||
);
|
||||
expect(records.map((r) => r.fp)).toEqual([fp]);
|
||||
expect(records[0]!.citedId).toBe(false);
|
||||
expect(records[0]!.commentId).toBe(-50);
|
||||
}
|
||||
});
|
||||
|
||||
test("matchReplies: an unknown threadFp matches nothing", () => {
|
||||
const f = finding();
|
||||
const records = matchReplies(
|
||||
[{ id: -50, body: "x", login: "author", maintainer: false, threadFp: "ffffff" }],
|
||||
[{ finding: f, fp: fingerprintFinding(f) }],
|
||||
{ match: "both" },
|
||||
);
|
||||
expect(records).toEqual([]);
|
||||
});
|
||||
|
||||
// ---- cross-stream ids ----
|
||||
|
||||
test("replyIsNewer: magnitude within a stream, issue comments win across streams", () => {
|
||||
expect(replyIsNewer(200, 100)).toBe(true); // issue: newer wins
|
||||
expect(replyIsNewer(-200, -100)).toBe(true); // inline: raw 200 is newer than raw 100
|
||||
expect(replyIsNewer(-100, -200)).toBe(false);
|
||||
expect(replyIsNewer(100, -999)).toBe(true); // cross-stream: issue comment wins
|
||||
expect(replyIsNewer(-999, 100)).toBe(false);
|
||||
});
|
||||
|
||||
test("matchReplies: the newer inline reply wins within the inline stream", () => {
|
||||
const f = finding();
|
||||
const fp = fingerprintFinding(f);
|
||||
const mk = (id: number, login: string): ReplyComment => ({
|
||||
id,
|
||||
body: "",
|
||||
login,
|
||||
maintainer: false,
|
||||
threadFp: fp,
|
||||
});
|
||||
const records = matchReplies([mk(-100, "older"), mk(-200, "newer")], [{ finding: f, fp }], {
|
||||
match: "both",
|
||||
});
|
||||
expect(records[0]!.by).toBe("newer");
|
||||
});
|
||||
|
||||
test("applyPins: a cross-stream 'newer' maintainer reply never lifts a pin; same-stream magnitude does", () => {
|
||||
const record = (commentId: number): FeedbackRecord => ({
|
||||
fp: "aaaa11",
|
||||
by: "m",
|
||||
commentId,
|
||||
maintainer: true,
|
||||
citedId: true,
|
||||
applied: false,
|
||||
});
|
||||
// Pin taken against an inline reply (negative id): an issue reply cannot lift it.
|
||||
expect(applyPins([record(999)], [{ fp: "aaaa11", commentId: -100 }]).pins).toHaveLength(1);
|
||||
// A NEWER inline maintainer reply (raw 200 > raw 100) lifts it.
|
||||
expect(applyPins([record(-200)], [{ fp: "aaaa11", commentId: -100 }]).pins).toHaveLength(0);
|
||||
// An OLDER inline reply does not.
|
||||
expect(applyPins([record(-50)], [{ fp: "aaaa11", commentId: -100 }]).pins).toHaveLength(1);
|
||||
});
|
||||
|
||||
// ---- activeFindings ----
|
||||
|
||||
test("activeFindings: drops dismissed, reply-cleared, and requalified findings", () => {
|
||||
const keep = { finding: finding(), fp: "keep01" };
|
||||
const dismissed = { finding: finding({ file: "b.ts" }), fp: "gone01" };
|
||||
const cleared = { finding: finding({ file: "c.ts" }), fp: "gone02" };
|
||||
const requalified = {
|
||||
finding: finding({ file: "d.ts", requalifiedBy: { prNumber: 2, file: "x", reason: "r" } }),
|
||||
fp: "gone03",
|
||||
};
|
||||
const record: FeedbackRecord = {
|
||||
fp: "gone02",
|
||||
by: "author",
|
||||
commentId: 5,
|
||||
maintainer: false,
|
||||
applied: true,
|
||||
};
|
||||
const active = activeFindings(
|
||||
[keep, dismissed, cleared, requalified],
|
||||
[{ fp: "gone01" }],
|
||||
[record],
|
||||
);
|
||||
expect(active.map((entry) => entry.fp)).toEqual(["keep01"]);
|
||||
});
|
||||
|
||||
// ---- short form ----
|
||||
|
||||
test("renderMarkdown: an inlined finding renders short with the thread link; others render full", () => {
|
||||
const inlined = finding({ suggestion: "cache the observer" });
|
||||
const other = finding({ file: "b.ts", title: "Other finding", suggestion: "do the thing" });
|
||||
const review: CoordinatorOutput = {
|
||||
decision: "approve_with_comments",
|
||||
findings: [inlined, other],
|
||||
summary: "s",
|
||||
incomplete: [],
|
||||
};
|
||||
const link: LinkContext = {
|
||||
repo: "o/r",
|
||||
prNumber: 1,
|
||||
inlineUrls: new Map([
|
||||
[fingerprintFinding(inlined), "https://github.com/o/r/pull/1#discussion_r123"],
|
||||
]),
|
||||
};
|
||||
const body = renderMarkdown(review, TAG, [], link);
|
||||
expect(body).toContain("[inline comment](https://github.com/o/r/pull/1#discussion_r123)");
|
||||
// Short form: the one-line rationale survives, the suggestion moves inline-only.
|
||||
expect(body).toContain("getInstances() restarts the push token observer");
|
||||
expect(body).not.toContain("cache the observer");
|
||||
// The non-inlined finding keeps its full form.
|
||||
expect(body).toContain("**Suggestion:** do the thing");
|
||||
});
|
||||
|
||||
test("renderMarkdown: a non-github inline URL is ignored (full form)", () => {
|
||||
const f = finding({ suggestion: "s1" });
|
||||
const review: CoordinatorOutput = {
|
||||
decision: "approve_with_comments",
|
||||
findings: [f],
|
||||
summary: "s",
|
||||
incomplete: [],
|
||||
};
|
||||
const link: LinkContext = {
|
||||
repo: "o/r",
|
||||
prNumber: 1,
|
||||
inlineUrls: new Map([[fingerprintFinding(f), "https://evil.example/x"]]),
|
||||
};
|
||||
const body = renderMarkdown(review, TAG, [], link);
|
||||
expect(body).not.toContain("evil.example");
|
||||
expect(body).toContain("**Suggestion:** s1");
|
||||
});
|
||||
@@ -0,0 +1,170 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { TypeSafeClient } from "@typesafe-ai/sdk";
|
||||
|
||||
import { ReviewConfigSchema, ScopeReviewConfigSchema } from "../config/schema.js";
|
||||
import { FORBIDDEN_TOKEN_ENVS } from "../core/auth.js";
|
||||
import {
|
||||
buildJevState,
|
||||
evaluateFindingsWithJev,
|
||||
jevDisposition,
|
||||
withoutJevCredential,
|
||||
} from "../core/jev.js";
|
||||
import type { Finding } from "../core/schema.js";
|
||||
|
||||
const finding: Finding = {
|
||||
severity: "warning",
|
||||
category: "correctness",
|
||||
file: "src/example.ts",
|
||||
line: 2,
|
||||
title: "The fallback returns the wrong value",
|
||||
rationale: "The changed branch returns false when the caller requires true.",
|
||||
evidence: "return false",
|
||||
};
|
||||
|
||||
const config = {
|
||||
model: "jev-1.13.0",
|
||||
minConfidence: 0.9,
|
||||
timeoutMs: 10_000,
|
||||
maxContextChars: 30_000,
|
||||
};
|
||||
|
||||
describe("Jev config", () => {
|
||||
test("is activated by presence and absent by default", () => {
|
||||
expect(ReviewConfigSchema.parse({}).jev).toBeUndefined();
|
||||
expect(ReviewConfigSchema.parse({ jev: {} }).jev).toEqual(config);
|
||||
});
|
||||
|
||||
test("is root-only", () => {
|
||||
expect(ScopeReviewConfigSchema.safeParse({ jev: {} }).success).toBe(false);
|
||||
});
|
||||
|
||||
test("cannot be repurposed as a reviewer provider credential", () => {
|
||||
expect(FORBIDDEN_TOKEN_ENVS.has("TYPESAFE_API_KEY")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("evaluateFindingsWithJev", () => {
|
||||
test("falls back cleanly when the key is missing", async () => {
|
||||
const result = await evaluateFindingsWithJev({
|
||||
config,
|
||||
candidates: [{ finding, sourceContext: "return false" }],
|
||||
env: {},
|
||||
});
|
||||
expect(result.summary.unavailable).toBe("TYPESAFE_API_KEY is not set");
|
||||
expect(result.evaluations.size).toBe(0);
|
||||
});
|
||||
|
||||
test("returns a typed atomic judgment without persisting source", async () => {
|
||||
const requests: unknown[] = [];
|
||||
const client = new TypeSafeClient({
|
||||
apiKey: "test-key",
|
||||
defaultModel: config.model,
|
||||
retry: { maxRetries: 0 },
|
||||
fetch: async (_input, init) => {
|
||||
requests.push(JSON.parse(String(init?.body)));
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
model: "jev-1.13.0",
|
||||
answers: {
|
||||
support: {
|
||||
type: "choice",
|
||||
choice: "supported",
|
||||
confidence: 0.91,
|
||||
probabilities: { supported: 0.91, needs_reasoning: 0.07, contradicted: 0.02 },
|
||||
},
|
||||
},
|
||||
usage: { input_tokens: 123, output_tokens: 0 },
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } },
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
const result = await evaluateFindingsWithJev({
|
||||
config,
|
||||
candidates: [{ finding, sourceContext: "export function fallback() { return false }" }],
|
||||
client,
|
||||
env: {},
|
||||
});
|
||||
|
||||
expect(requests).toHaveLength(1);
|
||||
expect(result.summary.evaluated).toBe(1);
|
||||
expect(result.summary.inputTokens).toBe(123);
|
||||
expect([...result.evaluations.values()][0]?.support).toBe("supported");
|
||||
expect(JSON.stringify(result.summary)).not.toContain("return false");
|
||||
});
|
||||
|
||||
test("fails open per finding on provider errors", async () => {
|
||||
const client = new TypeSafeClient({
|
||||
apiKey: "test-key",
|
||||
retry: { maxRetries: 0 },
|
||||
fetch: async () => {
|
||||
throw new Error("request contained sensitive source");
|
||||
},
|
||||
});
|
||||
const result = await evaluateFindingsWithJev({
|
||||
config,
|
||||
candidates: [{ finding, sourceContext: "secret source" }],
|
||||
client,
|
||||
env: {},
|
||||
});
|
||||
|
||||
expect(result.summary.failed).toBe(1);
|
||||
expect(result.summary.unavailable).toBeUndefined();
|
||||
expect(JSON.stringify(result.summary)).not.toContain("sensitive source");
|
||||
});
|
||||
});
|
||||
|
||||
describe("jevDisposition", () => {
|
||||
const evaluation = (
|
||||
support: "supported" | "needs_reasoning" | "contradicted",
|
||||
confidence = 0.95,
|
||||
) => ({
|
||||
fingerprint: "fp",
|
||||
model: config.model,
|
||||
support,
|
||||
confidence,
|
||||
probabilities: { supported: 0.02, needs_reasoning: 0.03, contradicted: 0.95 },
|
||||
});
|
||||
|
||||
test("acts on clear ordinary findings and defers uncertainty", () => {
|
||||
expect(jevDisposition(finding, evaluation("supported"), 0.9)).toBe("keep");
|
||||
expect(jevDisposition(finding, evaluation("contradicted"), 0.9)).toBe("drop");
|
||||
expect(jevDisposition(finding, evaluation("needs_reasoning"), 0.9)).toBe("defer");
|
||||
expect(jevDisposition(finding, evaluation("contradicted", 0.89), 0.9)).toBe("defer");
|
||||
});
|
||||
|
||||
test("never drops protected or cited findings by itself", () => {
|
||||
expect(
|
||||
jevDisposition({ ...finding, category: "security" }, evaluation("contradicted"), 0.9),
|
||||
).toBe("defer");
|
||||
expect(
|
||||
jevDisposition({ ...finding, severity: "critical" }, evaluation("contradicted"), 0.9),
|
||||
).toBe("defer");
|
||||
expect(
|
||||
jevDisposition(
|
||||
{ ...finding, sources: [{ title: "Docs", url: "https://example.com/docs" }] },
|
||||
evaluation("supported"),
|
||||
0.9,
|
||||
),
|
||||
).toBe("defer");
|
||||
});
|
||||
});
|
||||
|
||||
test("buildJevState bounds large source around the cited evidence", () => {
|
||||
const state = buildJevState(
|
||||
finding,
|
||||
`before-${"x".repeat(2_000)}return false${"y".repeat(2_000)}`,
|
||||
1_000,
|
||||
);
|
||||
expect(state.sourceContext.length).toBeLessThanOrEqual(1_000);
|
||||
expect(state.sourceContext).toContain("return false");
|
||||
expect(state.sourceContext).toContain("source omitted by ecr");
|
||||
});
|
||||
|
||||
test("withoutJevCredential restores the key after hiding it from a child spawn", async () => {
|
||||
const env = { TYPESAFE_API_KEY: "secret", KEEP: "visible" };
|
||||
const seen = await withoutJevCredential(async () => ({ ...env }), env);
|
||||
expect(seen).toEqual({ KEEP: "visible" });
|
||||
expect(env.TYPESAFE_API_KEY).toBe("secret");
|
||||
});
|
||||
@@ -342,10 +342,10 @@ test("an upstream-alias auth entry synthesizes a provider block with exactly the
|
||||
expect(opencode.provider?.openai).toBeUndefined();
|
||||
});
|
||||
|
||||
test("Meta auth synthesizes the public Muse Responses provider without embedding its key", () => {
|
||||
test("Meta auth synthesizes the public Muse 1.3 Responses provider without embedding its key", () => {
|
||||
const config = configWith({
|
||||
agents: [{ id: "correctness", model: "meta/muse-spark-1.2" }],
|
||||
coordinatorModel: "meta/muse-spark-1.2",
|
||||
agents: [{ id: "correctness", model: "meta/muse-spark-1.3" }],
|
||||
coordinatorModel: "meta/muse-spark-1.3-contributor",
|
||||
auth: [{ mode: "api-key", provider: "meta", tokenEnv: "META_API_KEY" }],
|
||||
});
|
||||
const opencode = buildOpencodeConfig(config) as {
|
||||
@@ -354,7 +354,14 @@ test("Meta auth synthesizes the public Muse Responses provider without embedding
|
||||
{
|
||||
npm: string;
|
||||
options: { baseURL: string; apiKey: string };
|
||||
models: Record<string, { reasoning?: boolean; options?: Record<string, unknown> }>;
|
||||
models: Record<
|
||||
string,
|
||||
{
|
||||
reasoning?: boolean;
|
||||
limit?: { context: number; output: number };
|
||||
options?: Record<string, unknown>;
|
||||
}
|
||||
>;
|
||||
}
|
||||
>;
|
||||
};
|
||||
@@ -365,12 +372,17 @@ test("Meta auth synthesizes the public Muse Responses provider without embedding
|
||||
baseURL: META_MODEL_API_BASE_URL,
|
||||
apiKey: "{env:META_API_KEY}",
|
||||
});
|
||||
expect(Object.keys(meta!.models)).toEqual(["muse-spark-1.2"]);
|
||||
expect(meta!.models["muse-spark-1.2"]?.reasoning).toBe(true);
|
||||
expect(meta!.models["muse-spark-1.2"]?.options).toMatchObject({
|
||||
expect(Object.keys(meta!.models)).toEqual(["muse-spark-1.3", "muse-spark-1.3-contributor"]);
|
||||
expect(meta!.models["muse-spark-1.3"]?.reasoning).toBe(true);
|
||||
expect(meta!.models["muse-spark-1.3"]?.limit).toEqual({
|
||||
context: 1_048_576,
|
||||
output: 1_048_576,
|
||||
});
|
||||
expect(meta!.models["muse-spark-1.3"]?.options).toMatchObject({
|
||||
reasoningEffort: "high",
|
||||
include: ["reasoning.encrypted_content"],
|
||||
});
|
||||
expect(meta!.models["muse-spark-1.3-contributor"]?.reasoning).toBe(true);
|
||||
});
|
||||
|
||||
test("a Meta-named upstream alias does not redirect the upstream credential to Meta", () => {
|
||||
@@ -400,25 +412,22 @@ test("a Meta-named upstream alias does not redirect the upstream credential to M
|
||||
expect(Object.keys(alias!.models)).toEqual(["muse-spark-1.2"]);
|
||||
});
|
||||
|
||||
test("Meta provider does not register an unknown Muse id, so preflight can reject it", () => {
|
||||
test("Meta provider registers future model families neutrally and only for the meta prefix", () => {
|
||||
const config = configWith({
|
||||
agents: [{ id: "correctness", model: "meta/muse-spark-typo" }],
|
||||
coordinatorModel: "meta/muse-spark-typo",
|
||||
agents: [
|
||||
{ id: "correctness", model: "meta/muse-foo" },
|
||||
{ id: "security", model: "openai/muse-foo" },
|
||||
],
|
||||
coordinatorModel: "meta/a-future-family-2",
|
||||
auth: [{ mode: "api-key", provider: "meta", tokenEnv: "META_API_KEY" }],
|
||||
});
|
||||
const opencode = buildOpencodeConfig(config) as {
|
||||
provider?: Record<string, { models: Record<string, unknown> }>;
|
||||
};
|
||||
expect(opencode.provider?.meta?.models).toEqual({});
|
||||
expect(
|
||||
findUnknownModels(["meta/muse-spark-typo"], { meta: ["muse-spark-1.2"] }, ["meta"]),
|
||||
).toEqual([
|
||||
{
|
||||
model: "meta/muse-spark-typo",
|
||||
reason: "model",
|
||||
suggestions: ["muse-spark-1.2"],
|
||||
},
|
||||
]);
|
||||
expect(opencode.provider?.meta?.models).toEqual({
|
||||
"muse-foo": {},
|
||||
"a-future-family-2": {},
|
||||
});
|
||||
});
|
||||
|
||||
test("buildOpencodeConfig registers the no-tools stack verifier (empty tool list)", () => {
|
||||
|
||||
@@ -221,4 +221,16 @@ test("direct URL inference prefers narrow release and dependency providers", ()
|
||||
"https://developer.android.com/reference/androidx/media3/common/Player",
|
||||
).provider,
|
||||
).toBe("media3");
|
||||
expect(
|
||||
resolveDirectDocumentationTarget("https://developer.android.com/ndk/guides/cpp-support"),
|
||||
).toMatchObject({ provider: "android-ndk", sourceKind: "official-guide" });
|
||||
expect(resolveDirectDocumentationTarget("https://fetch.spec.whatwg.org/")).toMatchObject({
|
||||
provider: "web-standards",
|
||||
sourceKind: "standard",
|
||||
});
|
||||
expect(
|
||||
resolveDirectDocumentationTarget(
|
||||
"https://chromedevtools.github.io/devtools-protocol/tot/Runtime/",
|
||||
),
|
||||
).toMatchObject({ provider: "chrome-devtools-protocol", sourceKind: "standard" });
|
||||
});
|
||||
@@ -39,12 +39,18 @@ test("stdio MCP advertises the read-only documentation tools and their enforced
|
||||
);
|
||||
expect(searchTool?.description ?? "").toContain("Native source retains platform context");
|
||||
expect(searchTool?.description ?? "").toContain("react-native-worklets=Worklets");
|
||||
expect(searchTool?.description ?? "").toContain("metro=Metro bundler");
|
||||
expect(searchTool?.description ?? "").toContain(
|
||||
"web-standards=WHATWG and W3C web platform standards",
|
||||
);
|
||||
expect(searchTool?.description ?? "").toContain("android-ndk=Android NDK");
|
||||
expect(fetchTool?.annotations?.readOnlyHint).toBe(true);
|
||||
expect(fetchTool?.description ?? "").toMatch(/every redirect/);
|
||||
const inputSchema = searchTool?.inputSchema as {
|
||||
properties?: {
|
||||
query?: { description?: string };
|
||||
providers?: { description?: string };
|
||||
providers?: { description?: string; items?: { enum?: string[] } };
|
||||
sourceKinds?: { description?: string; items?: { enum?: string[] } };
|
||||
};
|
||||
};
|
||||
const fetchInputSchema = fetchTool?.inputSchema as {
|
||||
@@ -59,6 +65,12 @@ test("stdio MCP advertises the read-only documentation tools and their enforced
|
||||
expect(inputSchema.properties?.providers?.description ?? "").toContain(
|
||||
"jetbrains-issues=JetBrains YouTrack context",
|
||||
);
|
||||
expect(inputSchema.properties?.providers?.items?.enum).toContain("typescript");
|
||||
expect(inputSchema.properties?.providers?.items?.enum).toContain("chrome-devtools-protocol");
|
||||
expect(inputSchema.properties?.sourceKinds?.items?.enum).toContain("standard");
|
||||
expect(inputSchema.properties?.sourceKinds?.description ?? "").toContain(
|
||||
"external compatibility specification",
|
||||
);
|
||||
expect(fetchInputSchema.properties?.context?.default).toBe("section");
|
||||
expect(fetchInputSchema.properties?.context?.description ?? "").toContain(
|
||||
"document=bounded extracted page text",
|
||||
|
||||
@@ -1,22 +1,31 @@
|
||||
import { expect, test } from "bun:test";
|
||||
|
||||
import {
|
||||
androidNdkProvider,
|
||||
androidProvider,
|
||||
appleProvider,
|
||||
appleReleasesProvider,
|
||||
chromeDevtoolsProtocolProvider,
|
||||
cmakeProvider,
|
||||
cocoaPodsProvider,
|
||||
expoProvider,
|
||||
flowProvider,
|
||||
glideProvider,
|
||||
jetbrainsIssuesProvider,
|
||||
metroProvider,
|
||||
okHttpProvider,
|
||||
reactNativeGestureHandlerProvider,
|
||||
reactNativeProvider,
|
||||
reactNativeReanimatedProvider,
|
||||
reactNativeScreensProvider,
|
||||
reactNativeWorkletsProvider,
|
||||
reactProvider,
|
||||
resolveAllowedUrl,
|
||||
resolveAllowedRequestUrl,
|
||||
sdWebImageProvider,
|
||||
swiftEvolutionProvider,
|
||||
typescriptProvider,
|
||||
webStandardsProvider,
|
||||
} from "../../research-mcp/providers.js";
|
||||
|
||||
test("Apple provider accepts only HTTPS documentation paths on the exact host", () => {
|
||||
@@ -180,6 +189,7 @@ test("React Native ecosystem providers stay on their exact documentation hosts a
|
||||
reactNativeWorkletsProvider,
|
||||
"https://docs.swmansion.com/react-native-worklets/docs/fundamentals/getting-started/",
|
||||
],
|
||||
[metroProvider, "https://metrobundler.dev/docs/configuration"],
|
||||
] as const) {
|
||||
expect(resolveAllowedUrl(provider, url).hostname).toBe(new URL(url).hostname);
|
||||
}
|
||||
@@ -193,4 +203,53 @@ test("React Native ecosystem providers stay on their exact documentation hosts a
|
||||
expect(() => resolveAllowedUrl(expoProvider, "https://expo.dev/accounts")).toThrow(
|
||||
/outside the expo documentation allowlist/,
|
||||
);
|
||||
expect(() =>
|
||||
resolveAllowedUrl(metroProvider, "https://metrobundler.dev.evil.example/docs/configuration"),
|
||||
).toThrow(/outside the metro documentation allowlist/);
|
||||
});
|
||||
|
||||
test("React Native contract providers admit only their fixed authoritative corpora", () => {
|
||||
for (const [provider, url] of [
|
||||
[reactProvider, "https://react.dev/reference/react/useEffect"],
|
||||
[flowProvider, "https://flow.org/en/docs/types/unions/"],
|
||||
[typescriptProvider, "https://www.typescriptlang.org/docs/handbook/declaration-files/"],
|
||||
[androidNdkProvider, "https://developer.android.com/ndk/guides/cpp-support"],
|
||||
[cmakeProvider, "https://cmake.org/cmake/help/v3.31/command/add_library.html"],
|
||||
[cocoaPodsProvider, "https://guides.cocoapods.org/syntax/podfile.html"],
|
||||
[webStandardsProvider, "https://fetch.spec.whatwg.org/"],
|
||||
[webStandardsProvider, "https://w3c.github.io/IntersectionObserver/"],
|
||||
[
|
||||
chromeDevtoolsProtocolProvider,
|
||||
"https://chromedevtools.github.io/devtools-protocol/tot/Runtime/",
|
||||
],
|
||||
] as const) {
|
||||
expect(resolveAllowedUrl(provider, url).hostname).toBe(new URL(url).hostname);
|
||||
}
|
||||
|
||||
for (const [provider, url] of [
|
||||
[reactProvider, "https://react.dev/reference/react-dom/components/form"],
|
||||
[reactProvider, "https://react.dev/learn/escape-hatches"],
|
||||
[flowProvider, "https://flow.org/blog/"],
|
||||
[typescriptProvider, "https://www.typescriptlang.org/play"],
|
||||
[androidNdkProvider, "https://developer.android.com/studio/projects/install-ndk"],
|
||||
[cmakeProvider, "https://cmake.org/community/"],
|
||||
[cocoaPodsProvider, "https://guides.cocoapods.org/blog/"],
|
||||
[webStandardsProvider, "https://w3c.github.io/permissions/"],
|
||||
[webStandardsProvider, "https://whatwg.org/"],
|
||||
[chromeDevtoolsProtocolProvider, "https://chromedevtools.github.io/debugger-protocol-viewer/"],
|
||||
] as const) {
|
||||
expect(() => resolveAllowedUrl(provider, url)).toThrow(
|
||||
/outside the .* documentation allowlist/,
|
||||
);
|
||||
}
|
||||
|
||||
expect(() =>
|
||||
resolveAllowedUrl(webStandardsProvider, "https://user:[email protected]/"),
|
||||
).toThrow(/outside the web-standards documentation allowlist/);
|
||||
expect(() =>
|
||||
resolveAllowedUrl(androidNdkProvider, "https://developer.android.com.evil.example/ndk/guides"),
|
||||
).toThrow(/outside the android-ndk documentation allowlist/);
|
||||
expect(() =>
|
||||
resolveAllowedUrl(reactNativeProvider, "https://reactnative.dev/contributing/overview"),
|
||||
).toThrow(/outside the react-native documentation allowlist/);
|
||||
});
|
||||
@@ -299,3 +299,67 @@ test("Apple release searches select the platform-specific fixed scope", async ()
|
||||
expect(queries[0]).toContain("site:developer.apple.com/documentation/xcode-release-notes");
|
||||
expect(queries[1]).toContain("site:developer.apple.com/documentation/ios-ipados-release-notes");
|
||||
});
|
||||
|
||||
test("web standards discovery uses fixed WHATWG and W3C scopes with standards provenance", async () => {
|
||||
let searchRequests = 0;
|
||||
const fetchImplementation = async (input: string | URL | Request): Promise<Response> => {
|
||||
const url = new URL(input instanceof Request ? input.url : input.toString());
|
||||
if (url.hostname === "api.search.brave.com") {
|
||||
searchRequests++;
|
||||
const query = url.searchParams.get("q") ?? "";
|
||||
expect(query).toContain("site:whatwg.org");
|
||||
expect(query).toContain("site:w3c.github.io");
|
||||
return jsonResponse({
|
||||
web: {
|
||||
results: [
|
||||
{ title: "Fetch Standard", url: "https://fetch.spec.whatwg.org/" },
|
||||
{ title: "Unrelated W3C draft", url: "https://w3c.github.io/permissions/" },
|
||||
],
|
||||
},
|
||||
});
|
||||
}
|
||||
expect(url.href).toBe("https://fetch.spec.whatwg.org/");
|
||||
return new Response(
|
||||
"<!doctype html><html><head><title>Fetch Standard</title></head><body><main>" +
|
||||
"<h1>Fetch</h1><p>The AbortSignal controls cancellation of a fetch request and " +
|
||||
"causes the fetch promise to reject when the operation is terminated.</p>" +
|
||||
"</main></body></html>",
|
||||
{ headers: { "content-type": "text/html; charset=utf-8" } },
|
||||
);
|
||||
};
|
||||
|
||||
const response = await searchRemoteDocumentation(
|
||||
"web-standards",
|
||||
"AbortSignal fetch cancellation",
|
||||
2,
|
||||
{ apiKey: "test-key", fetchImplementation, sourceKinds: ["standard"] },
|
||||
);
|
||||
|
||||
expect(searchRequests).toBe(1);
|
||||
expect(response.warnings).toEqual([]);
|
||||
expect(response.results).toHaveLength(1);
|
||||
expect(response.results[0]).toMatchObject({
|
||||
provider: "web-standards",
|
||||
sourceKind: "standard",
|
||||
url: "https://fetch.spec.whatwg.org/",
|
||||
});
|
||||
});
|
||||
|
||||
test("provenance filtering skips a standards provider before network access", async () => {
|
||||
let requests = 0;
|
||||
const response = await searchRemoteDocumentation(
|
||||
"chrome-devtools-protocol",
|
||||
"Runtime evaluate",
|
||||
1,
|
||||
{
|
||||
apiKey: "test-key",
|
||||
sourceKinds: ["official-api"],
|
||||
fetchImplementation: async () => {
|
||||
requests++;
|
||||
return jsonResponse({ web: { results: [] } });
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(requests).toBe(0);
|
||||
expect(response).toEqual({ results: [], warnings: [] });
|
||||
});
|
||||
@@ -149,6 +149,40 @@ test("search filters named corpora and provenance classes", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
test("search filters external standards separately from implementation documentation", () => {
|
||||
const index = buildSearchIndex([
|
||||
{
|
||||
id: "standard:fetch",
|
||||
platform: "react-native",
|
||||
provider: "web-standards",
|
||||
sourceKind: "standard",
|
||||
title: "Fetch Standard",
|
||||
url: "https://fetch.spec.whatwg.org/",
|
||||
passage: "AbortSignal controls cancellation of a fetch request.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
{
|
||||
id: "rn:fetch",
|
||||
platform: "react-native",
|
||||
provider: "react-native",
|
||||
sourceKind: "official-api",
|
||||
title: "React Native global fetch",
|
||||
url: "https://reactnative.dev/docs/global-fetch",
|
||||
passage: "React Native exposes fetch as a global API.",
|
||||
indexedAt: "2026-08-06T00:00:00.000Z",
|
||||
},
|
||||
]);
|
||||
|
||||
const results = searchDocumentation(index, "fetch AbortSignal", {
|
||||
platform: "react-native",
|
||||
sourceKinds: ["standard"],
|
||||
limit: 5,
|
||||
});
|
||||
expect(results.map(({ provider, sourceKind }) => ({ provider, sourceKind }))).toEqual([
|
||||
{ provider: "web-standards", sourceKind: "standard" },
|
||||
]);
|
||||
});
|
||||
|
||||
test("a language filter excludes otherwise matching untagged documents", () => {
|
||||
const index = buildSearchIndex([
|
||||
...chunks,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { test, expect, beforeAll, afterAll } from "bun:test";
|
||||
import { TypeSafeClient } from "@typesafe-ai/sdk";
|
||||
import { mkdtempSync, writeFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
@@ -82,6 +83,34 @@ const finding = (over: Partial<Finding>): Finding => ({
|
||||
// (keep the finding). Grounded non-criticals never reach the LLM at all.
|
||||
const handle = {} as OpencodeHandle;
|
||||
|
||||
const contradictedJev = new TypeSafeClient({
|
||||
apiKey: "test-key",
|
||||
retry: { maxRetries: 0 },
|
||||
fetch: async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
model: "jev-1.13.0",
|
||||
answers: {
|
||||
support: {
|
||||
type: "choice",
|
||||
choice: "contradicted",
|
||||
confidence: 0.97,
|
||||
probabilities: { supported: 0.01, needs_reasoning: 0.02, contradicted: 0.97 },
|
||||
},
|
||||
},
|
||||
usage: { input_tokens: 20, output_tokens: 0 },
|
||||
}),
|
||||
{ status: 200, headers: { "content-type": "application/json" } },
|
||||
),
|
||||
});
|
||||
|
||||
const jevConfig = {
|
||||
model: "jev-1.13.0",
|
||||
minConfidence: 0.9,
|
||||
timeoutMs: 10_000,
|
||||
maxContextChars: 30_000,
|
||||
};
|
||||
|
||||
test("grounded non-critical is kept without an LLM call", async () => {
|
||||
const res = await verifyFindings(
|
||||
handle,
|
||||
@@ -92,6 +121,40 @@ test("grounded non-critical is kept without an LLM call", async () => {
|
||||
expect(res.dropped).toEqual([]);
|
||||
});
|
||||
|
||||
test("Jev actively drops a confidently contradicted ordinary finding", async () => {
|
||||
const res = await verifyFindings(
|
||||
handle,
|
||||
[finding({ title: "contradicted", evidence: "return items[next++]!;" })],
|
||||
"/",
|
||||
undefined,
|
||||
[],
|
||||
{ config: jevConfig, client: contradictedJev },
|
||||
);
|
||||
expect(res.kept).toEqual([]);
|
||||
expect(res.dropped[0]?.reason).toContain("Jev directly contradicted");
|
||||
expect(res.jev?.evaluated).toBe(1);
|
||||
});
|
||||
|
||||
test("Jev defers a contradicted security finding to the reasoning verifier", async () => {
|
||||
const res = await verifyFindings(
|
||||
handle,
|
||||
[
|
||||
finding({
|
||||
title: "protected",
|
||||
category: "security",
|
||||
evidence: "return items[next++]!;",
|
||||
}),
|
||||
],
|
||||
"/",
|
||||
undefined,
|
||||
[],
|
||||
{ config: jevConfig, client: contradictedJev },
|
||||
);
|
||||
// The dummy reasoning handle fails, and the existing fail-open rule keeps it.
|
||||
expect(res.kept.map((item) => item.title)).toEqual(["protected"]);
|
||||
expect(res.dropped).toEqual([]);
|
||||
});
|
||||
|
||||
test("absent evidence is ESCALATED, not hard-dropped (fail-open keeps it here)", async () => {
|
||||
// The key regression guard: an imperfect/absent quote must not be a silent drop.
|
||||
// With the dummy handle the escalated LLM call throws → fail-open → kept.
|
||||
|
||||
+18
-3
@@ -4,7 +4,7 @@ import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
import {
|
||||
CONFIG_DIRNAME,
|
||||
configDirFor,
|
||||
hasScopeConfig,
|
||||
loadAuthFromRoot,
|
||||
loadReviewConfig,
|
||||
@@ -495,7 +495,7 @@ function resolveStackConfirm(
|
||||
* `.expo-code-review/.runs/reviews.jsonl` from the workspace as an artifact.
|
||||
*/
|
||||
function workspaceRunsDir(cwd: string): string {
|
||||
return path.join(cwd, CONFIG_DIRNAME, ".runs");
|
||||
return path.join(configDirFor(cwd), ".runs");
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -563,6 +563,7 @@ async function runLegacyCi(
|
||||
breakGlassMarker: config.breakGlassMarker,
|
||||
cwd,
|
||||
feedback: config.feedback,
|
||||
inline: config.inline,
|
||||
headSha,
|
||||
});
|
||||
|
||||
@@ -685,6 +686,10 @@ async function runLegacyCi(
|
||||
findings: [],
|
||||
summary: `⚠️ The AI reviewer failed to run, so this change was **not** reviewed:\n\n> ${publicFailureReason(error)}`,
|
||||
incomplete: [],
|
||||
// A failed run's empty findings list proves nothing: this flag is what keeps
|
||||
// the inline sync additive-only (no thread teardown) — and it renders the
|
||||
// accurate "No review" decision label.
|
||||
couldNotComplete: true,
|
||||
});
|
||||
} catch (postError) {
|
||||
process.stderr.write(
|
||||
@@ -701,6 +706,10 @@ function failureReview(scopeName: string, reason: string): CoordinatorOutput {
|
||||
findings: [],
|
||||
summary: `⚠️ The AI reviewer failed to run for scope **${scopeName}**, so those changes were **not** reviewed:\n\n> ${reason}`,
|
||||
incomplete: [],
|
||||
// Empty-but-not-clean: keeps the inline sync additive-only for the whole
|
||||
// aggregate (a failed scope's findings may still be live on the PR) and renders
|
||||
// the accurate "No review" label in the scope table.
|
||||
couldNotComplete: true,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -889,6 +898,7 @@ async function runRoutedCi(
|
||||
// Root-only feedback config (see loadScopeConfig): lets a reporter posting with
|
||||
// no explicit records match replies itself (annotate mode) at report time.
|
||||
feedback: rootConfig.feedback,
|
||||
inline: rootConfig.inline,
|
||||
headSha,
|
||||
});
|
||||
|
||||
@@ -1123,7 +1133,12 @@ async function runRoutedCi(
|
||||
aggState?.pins,
|
||||
)
|
||||
: undefined;
|
||||
await aggregate.reportAggregate(finalResults, resolution.unmatched, aggFeedback);
|
||||
// A --scopes partial run carries the other scopes out of (possibly truncated)
|
||||
// embedded state, so its target set is incomplete: inline teardown must not
|
||||
// run (failed scopes are handled inside reportAggregate itself).
|
||||
await aggregate.reportAggregate(finalResults, resolution.unmatched, aggFeedback, {
|
||||
inlineTeardown: !scopesFilter,
|
||||
});
|
||||
}
|
||||
// Clean up any per-scope comments from a previous per-scope run. A partial run
|
||||
// only ever touches the named scopes' comments.
|
||||
|
||||
@@ -98,6 +98,8 @@ export async function dismissCommand(argv: string[], mode: "add" | "remove"): Pr
|
||||
// feedback policy in force now — without it a reply-cleared finding would be
|
||||
// un-hidden by an unrelated /dismiss.
|
||||
feedback: config.feedback,
|
||||
// Lets the dismissal stub/revive the finding's inline thread (additive-only).
|
||||
inline: config.inline,
|
||||
});
|
||||
const result = await reporter.applyDismissal(
|
||||
mode === "add" ? args.ids : [],
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
loadScopeConfig,
|
||||
loadAuthFromRoot,
|
||||
hasConfig,
|
||||
CONFIG_DIRNAME,
|
||||
resolveConfigDir,
|
||||
tokenEnvMismatch,
|
||||
} from "../config/load.js";
|
||||
@@ -246,7 +247,7 @@ export async function doctorCommand(argv: string[] = []): Promise<void> {
|
||||
}
|
||||
|
||||
if (!hasConfig(root)) {
|
||||
line(false, `no ${".expo-code-review"}/config.jsonc (run \`ecr init\`)`);
|
||||
line(false, `no ${CONFIG_DIRNAME}/config.jsonc (run \`ecr init\`)`);
|
||||
} else {
|
||||
try {
|
||||
// Reuse the engine-detection peek above; re-load only if that failed so
|
||||
@@ -479,9 +480,7 @@ async function listScopes(root: string): Promise<void> {
|
||||
return;
|
||||
}
|
||||
if (!manifest) {
|
||||
process.stdout.write(
|
||||
`No ${".expo-code-review"}/routing.jsonc — run \`ecr init --monorepo\`.\n`,
|
||||
);
|
||||
process.stdout.write(`No ${CONFIG_DIRNAME}/routing.jsonc — run \`ecr init --monorepo\`.\n`);
|
||||
process.exitCode = 0;
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@ import { existsSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { CONFIG_DIRNAME } from "../config/load.js";
|
||||
import { CONFIG_DIRNAME, configDirFor } from "../config/load.js";
|
||||
import { ROUTING_FILENAME } from "../config/routing.js";
|
||||
import { RoutingScopeSchema, type RoutingScope } from "../config/schema.js";
|
||||
import { FORBIDDEN_TOKEN_ENVS } from "../core/auth.js";
|
||||
@@ -13,7 +13,7 @@ import { errorMessage } from "../core/util.js";
|
||||
|
||||
const TEMPLATES_DIR = fileURLToPath(new URL("../../templates/", import.meta.url));
|
||||
|
||||
const USAGE = `ecr init — scaffold .expo-code-review/ in the current repo
|
||||
const USAGE = `ecr init — scaffold .expo-agents/code-review/ in the current repo (a legacy .expo-code-review/ is reused)
|
||||
|
||||
Usage:
|
||||
ecr init [--no-workflow] [--force] Scaffold the root config (+ CI workflow)
|
||||
@@ -22,8 +22,8 @@ Usage:
|
||||
register it in the root routing.jsonc
|
||||
|
||||
Options:
|
||||
--monorepo Also write .expo-code-review/routing.jsonc (routing manifest)
|
||||
--scope <dir> Scaffold <dir>/.expo-code-review/ (no auth) + add a scope entry
|
||||
--monorepo Also write routing.jsonc in the setup dir (routing manifest)
|
||||
--scope <dir> Scaffold <dir>/.expo-agents/code-review/ (no auth) + add a scope entry
|
||||
--no-workflow Skip writing the CI workflows (review, command, and dismiss
|
||||
under .github/workflows/)
|
||||
--token-env <name[,name…]>
|
||||
@@ -83,7 +83,7 @@ async function scaffold(argv: string[]): Promise<void> {
|
||||
}
|
||||
|
||||
const root = (await repoRoot()) ?? process.cwd();
|
||||
const configDir = path.join(root, CONFIG_DIRNAME);
|
||||
const configDir = configDirFor(root); // an existing legacy .expo-code-review/ is kept, never doubled
|
||||
|
||||
// A non-default --token-env only takes effect through the review workflows,
|
||||
// but existing workflow files are skipped (not rewritten) without --force —
|
||||
@@ -286,7 +286,7 @@ async function scaffoldScope(argv: string[], scopeDirRaw: string): Promise<void>
|
||||
);
|
||||
}
|
||||
|
||||
const configDir = path.join(root, scopeDir, CONFIG_DIRNAME);
|
||||
const configDir = configDirFor(path.join(root, scopeDir));
|
||||
await mkdir(configDir, { recursive: true });
|
||||
|
||||
const created: string[] = [];
|
||||
|
||||
@@ -147,6 +147,7 @@ export async function postReviewCommand(argv: string[]): Promise<void> {
|
||||
breakGlassMarker: config.breakGlassMarker,
|
||||
cwd,
|
||||
feedback: config.feedback,
|
||||
inline: config.inline,
|
||||
headSha,
|
||||
});
|
||||
const result = await publishDeferredReview(reporter, artifact, async () => {
|
||||
|
||||
@@ -276,6 +276,8 @@ export async function reviewCommand(argv: string[]): Promise<void> {
|
||||
cwd,
|
||||
// Root-only feedback config (loadScopeConfig inherits it from the root).
|
||||
feedback: config.feedback,
|
||||
// Root-only too (loadScopeConfig inherits it from the root).
|
||||
inline: config.inline,
|
||||
headSha: await reviewedHeadSha(source),
|
||||
})
|
||||
: null;
|
||||
@@ -361,6 +363,7 @@ export async function reviewCommand(argv: string[]): Promise<void> {
|
||||
breakGlassMarker: config.breakGlassMarker,
|
||||
cwd,
|
||||
feedback: config.feedback,
|
||||
inline: config.inline,
|
||||
headSha,
|
||||
})
|
||||
: null;
|
||||
|
||||
@@ -2,7 +2,13 @@
|
||||
import { readdir, readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
import { CONFIG_DIRNAME, stripJsonComments, stripTrailingCommas } from "../config/load.js";
|
||||
import {
|
||||
CONFIG_DIRNAME,
|
||||
configDirFor,
|
||||
isConfigDirPath,
|
||||
stripJsonComments,
|
||||
stripTrailingCommas,
|
||||
} from "../config/load.js";
|
||||
import { ROUTING_FILENAME } from "../config/routing.js";
|
||||
import { repoRoot } from "../core/exec.js";
|
||||
import { errorMessage } from "../core/util.js";
|
||||
@@ -76,11 +82,7 @@ async function discoverConfigFiles(root: string): Promise<string[]> {
|
||||
continue;
|
||||
}
|
||||
await walk(path.join(dir, entry.name));
|
||||
} else if (
|
||||
entry.isFile() &&
|
||||
path.basename(dir) === CONFIG_DIRNAME &&
|
||||
CONFIG_FILENAMES.has(entry.name)
|
||||
) {
|
||||
} else if (entry.isFile() && isConfigDirPath(dir) && CONFIG_FILENAMES.has(entry.name)) {
|
||||
found.push(path.join(dir, entry.name));
|
||||
}
|
||||
}
|
||||
@@ -145,7 +147,7 @@ export async function verifyConfig(
|
||||
options: { expected?: string } = {},
|
||||
): Promise<VerifyResult> {
|
||||
const findings: VerifyFinding[] = [];
|
||||
const rootConfigDir = path.join(root, CONFIG_DIRNAME);
|
||||
const rootConfigDir = configDirFor(root);
|
||||
const rel = (file: string): string => path.relative(root, file) || path.basename(file);
|
||||
|
||||
const files = await discoverConfigFiles(root);
|
||||
|
||||
+53
-10
@@ -17,7 +17,25 @@ import type {
|
||||
} from "./schema.js";
|
||||
import { toolMap } from "../core/tools.js";
|
||||
|
||||
export const CONFIG_DIRNAME = ".expo-code-review";
|
||||
/**
|
||||
* The setup directory. `.expo-agents/code-review/` is the home from 0.15.0 —
|
||||
* one `.expo-agents/` directory holds every Expo agent tool's per-repo files
|
||||
* (code-review, verify) instead of one dot directory per tool at the root.
|
||||
* The pre-0.15 `.expo-code-review/` keeps working unchanged: every lookup
|
||||
* takes whichever of the two exists (new first), and scaffolding writes into
|
||||
* an existing legacy directory rather than starting a second one.
|
||||
*/
|
||||
export const CONFIG_DIRNAME = path.join(".expo-agents", "code-review");
|
||||
export const LEGACY_CONFIG_DIRNAME = ".expo-code-review";
|
||||
export const CONFIG_DIRNAMES: readonly string[] = [CONFIG_DIRNAME, LEGACY_CONFIG_DIRNAME];
|
||||
|
||||
/** Does this directory path name a setup dir (either spelling)? */
|
||||
export function isConfigDirPath(dir: string): boolean {
|
||||
const normalized = path.normalize(dir);
|
||||
return CONFIG_DIRNAMES.some(
|
||||
(name) => normalized === name || normalized.endsWith(path.sep + name),
|
||||
);
|
||||
}
|
||||
|
||||
/** Stack config for a scope load (where `stack` is schema-rejected and absent). */
|
||||
const STACK_CONFIG_DEFAULTS: LoadedConfig["stack"] = {
|
||||
@@ -39,6 +57,12 @@ const FEEDBACK_CONFIG_DEFAULTS: LoadedConfig["feedback"] = {
|
||||
maxAdjudications: 10,
|
||||
};
|
||||
|
||||
/** Inline-comment config for a scope load (where `inline` is schema-rejected). */
|
||||
const INLINE_CONFIG_DEFAULTS: LoadedConfig["inline"] = {
|
||||
enabled: false,
|
||||
maxComments: 20,
|
||||
};
|
||||
|
||||
/** Research defaults for a scope load (where `research` is schema-rejected). */
|
||||
const RESEARCH_CONFIG_DEFAULTS: LoadedConfig["research"] = {
|
||||
enabled: false,
|
||||
@@ -50,22 +74,34 @@ const RESEARCH_CONFIG_DEFAULTS: LoadedConfig["research"] = {
|
||||
/** Default OpenCode tool toggles for a reviewer: read the repo, never mutate it. */
|
||||
const DEFAULT_AGENT_TOOLS = toolMap(["read", "grep", "glob", "list"]);
|
||||
|
||||
export function configDirFor(repoRoot: string): string {
|
||||
return path.join(repoRoot, CONFIG_DIRNAME);
|
||||
/**
|
||||
* The setup dir under `base`: `.expo-agents/code-review/` when it exists, else
|
||||
* the legacy `.expo-code-review/` when THAT exists, else the new name (for
|
||||
* messages and for scaffolding a fresh repo). Works for the repo root and for
|
||||
* scope directories alike.
|
||||
*/
|
||||
export function configDirFor(base: string): string {
|
||||
for (const name of CONFIG_DIRNAMES) {
|
||||
const candidate = path.join(base, name);
|
||||
if (existsSync(candidate)) {
|
||||
return candidate;
|
||||
}
|
||||
}
|
||||
return path.join(base, CONFIG_DIRNAME);
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the config directory: an explicit override (absolute or repo-relative)
|
||||
* → the ECR_CONFIG_DIR env var → the repo's default `.expo-code-review/`. This is
|
||||
* the escape hatch (graft 1); with neither override present the result is exactly
|
||||
* `configDirFor(repoRoot)`, so the default behavior is byte-identical.
|
||||
* → the ECR_CONFIG_DIR env var → the repo's default setup dir (new name, then
|
||||
* legacy). This is the escape hatch (graft 1); with neither override present the
|
||||
* result is exactly `configDirFor(repoRoot)`.
|
||||
*/
|
||||
export function resolveConfigDir(repoRoot: string, override?: string): string {
|
||||
const chosen = override ?? process.env.ECR_CONFIG_DIR;
|
||||
if (chosen) {
|
||||
return path.isAbsolute(chosen) ? chosen : path.join(repoRoot, chosen);
|
||||
}
|
||||
return path.join(repoRoot, CONFIG_DIRNAME);
|
||||
return configDirFor(repoRoot);
|
||||
}
|
||||
|
||||
export interface LoadConfigOptions {
|
||||
@@ -76,7 +112,7 @@ export interface LoadConfigOptions {
|
||||
/** Parsed config with the centrally-locked keys optional (scope configs omit them). */
|
||||
type ParsedConfig = Omit<
|
||||
RawReviewConfig,
|
||||
"auth" | "breakGlass" | "commentTag" | "stack" | "feedback" | "research"
|
||||
"auth" | "breakGlass" | "commentTag" | "stack" | "feedback" | "research" | "jev" | "inline"
|
||||
> & {
|
||||
auth?: RawReviewConfig["auth"];
|
||||
breakGlass?: RawReviewConfig["breakGlass"];
|
||||
@@ -84,6 +120,8 @@ type ParsedConfig = Omit<
|
||||
stack?: RawReviewConfig["stack"];
|
||||
feedback?: RawReviewConfig["feedback"];
|
||||
research?: RawReviewConfig["research"];
|
||||
jev?: RawReviewConfig["jev"];
|
||||
inline?: RawReviewConfig["inline"];
|
||||
};
|
||||
|
||||
export function hasConfig(repoRoot: string, options: LoadConfigOptions = {}): boolean {
|
||||
@@ -203,6 +241,7 @@ async function loadConfigDir(
|
||||
// Root-only: scope schemas reject research configuration, so an untrusted
|
||||
// subtree cannot select the index or alter the network-facing runtime.
|
||||
research: parsed.research ?? RESEARCH_CONFIG_DEFAULTS,
|
||||
...(parsed.jev ? { jev: parsed.jev } : {}),
|
||||
// parsed.breakGlass/auth are always present for the root schema (defaults) and
|
||||
// absent for the scope schema; loadScopeConfig overrides both afterwards.
|
||||
breakGlassMarker: parsed.breakGlass?.marker ?? "/skip-review",
|
||||
@@ -219,6 +258,8 @@ async function loadConfigDir(
|
||||
// for a scope config and the defaults stand in (unused — the command layer
|
||||
// reads the ROOT config's feedback values; the comment lifecycle is global).
|
||||
feedback: parsed.feedback ?? FEEDBACK_CONFIG_DEFAULTS,
|
||||
// Root-only, same reasoning as feedback.
|
||||
inline: parsed.inline ?? INLINE_CONFIG_DEFAULTS,
|
||||
};
|
||||
return { config, raw: rawObject };
|
||||
}
|
||||
@@ -311,7 +352,7 @@ export function hasScopeConfig(root: string, scope: RoutingScope): boolean {
|
||||
if (scope.config === ".") {
|
||||
return true;
|
||||
}
|
||||
const dir = path.join(root, scope.config, CONFIG_DIRNAME);
|
||||
const dir = configDirFor(path.join(root, scope.config));
|
||||
return existsSync(path.join(dir, "config.jsonc")) || existsSync(path.join(dir, "config.json"));
|
||||
}
|
||||
|
||||
@@ -345,7 +386,7 @@ export async function loadScopeConfig(
|
||||
`scope "${scope.name}": config "${scope.config}" resolves outside the repo checkout`,
|
||||
);
|
||||
}
|
||||
const dir = path.join(root, scope.config, CONFIG_DIRNAME);
|
||||
const dir = configDirFor(path.join(root, scope.config));
|
||||
const { config } = await loadConfigDir(dir, ScopeReviewConfigSchema);
|
||||
base = config;
|
||||
// Non-default scopes never carry their own marker (the scope schema rejects
|
||||
@@ -386,7 +427,9 @@ export async function loadScopeConfig(
|
||||
// run the default policy instead of the repo's real one.
|
||||
stack: rootConfig.stack,
|
||||
feedback: rootConfig.feedback,
|
||||
inline: rootConfig.inline,
|
||||
research: rootConfig.research,
|
||||
jev: rootConfig.jev,
|
||||
scopeName: scope.name,
|
||||
};
|
||||
}
|
||||
|
||||
+54
-1
@@ -87,6 +87,19 @@ export const ReviewConfigSchema = z.object({
|
||||
resultsPerQuery: 2,
|
||||
timeoutMs: 30_000,
|
||||
}),
|
||||
// Optional Jev verification cascade. Presence enables it: confident local
|
||||
// judgments act immediately, while ambiguous or protected findings defer to
|
||||
// the reasoning verifier. ROOT-ONLY because it sends bounded source context
|
||||
// to a separate provider.
|
||||
// @ref LLP 0014#active-selective-cascade [implements] — narrow decisions act; uncertainty defers
|
||||
jev: z
|
||||
.object({
|
||||
model: z.string().min(1).default("jev-1.13.0"),
|
||||
minConfidence: z.number().min(0.5).max(1).default(0.9),
|
||||
timeoutMs: z.number().int().min(1000).max(60_000).default(10_000),
|
||||
maxContextChars: z.number().int().min(1000).max(30_000).default(30_000),
|
||||
})
|
||||
.optional(),
|
||||
breakGlass: z
|
||||
.object({ marker: z.string().default("/skip-review") })
|
||||
.default({ marker: "/skip-review" }),
|
||||
@@ -224,6 +237,20 @@ export const ReviewConfigSchema = z.object({
|
||||
protectedCategories: ["secrets", "security"],
|
||||
maxAdjudications: 10,
|
||||
}),
|
||||
// Inline PR review comments for findings anchored to a line in the diff: the
|
||||
// main comment stays the durable state store and renders such findings short,
|
||||
// linking to the inline thread. ROOT-ONLY (the comment lifecycle is global) and
|
||||
// off by default — a feature that mutates N comments per run earns trust with
|
||||
// field data first. Turning it off later leaves existing threads up (documented);
|
||||
// clear() still sweeps them on comment-mode switches.
|
||||
inline: z
|
||||
.object({
|
||||
enabled: z.boolean().default(false),
|
||||
// Cap on inline comments PER REPORTER: per PR in single/legacy comment mode,
|
||||
// per scope in per-scope mode. Bounds create-notification fan-out and API use.
|
||||
maxComments: z.number().int().positive().default(20),
|
||||
})
|
||||
.default({ enabled: false, maxComments: 20 }),
|
||||
});
|
||||
export type RawReviewConfig = z.infer<typeof ReviewConfigSchema>;
|
||||
|
||||
@@ -320,7 +347,7 @@ export type RoutingDefaults = RoutingManifest["defaults"];
|
||||
* Scope config = root config MINUS the centrally locked keys. Allowlist of
|
||||
* scope-overridable keys (Turborepo-style, graft 6): model, policy, chunk,
|
||||
* noise (+ the prompt files living beside it: shared.md, coordinator.md,
|
||||
* agents/). NEVER auth, breakGlass, or research — declaring one fails parsing at the
|
||||
* agents/). NEVER auth, breakGlass, research, or jev — declaring one fails parsing at the
|
||||
* Zod level so IDE/doctor catch it before CI. commentTag is also locked: a
|
||||
* scope's comment marker is always DERIVED (`<rootTag>:<scope>`; the default
|
||||
* scope keeps the root tag) so `ecr ci`'s post/clear/reconcile paths and a
|
||||
@@ -335,6 +362,8 @@ export const ScopeReviewConfigSchema = ReviewConfigSchema.omit({
|
||||
stack: true,
|
||||
feedback: true,
|
||||
research: true,
|
||||
jev: true,
|
||||
inline: true,
|
||||
}).extend({
|
||||
auth: z
|
||||
.never({ error: "auth is locked to the root config; remove it from this scope config" })
|
||||
@@ -364,6 +393,18 @@ export const ScopeReviewConfigSchema = ReviewConfigSchema.omit({
|
||||
"research is locked to the root config because it starts a trusted host process; remove it from this scope config",
|
||||
})
|
||||
.optional(),
|
||||
jev: z
|
||||
.never({
|
||||
error:
|
||||
"jev is locked to the root config because it sends bounded source context to an external provider; remove it from this scope config",
|
||||
})
|
||||
.optional(),
|
||||
inline: z
|
||||
.never({
|
||||
error:
|
||||
"inline is locked to the root config (the comment lifecycle is global); remove it from this scope config",
|
||||
})
|
||||
.optional(),
|
||||
});
|
||||
export type RawScopeReviewConfig = z.infer<typeof ScopeReviewConfigSchema>;
|
||||
|
||||
@@ -441,6 +482,13 @@ export interface LoadedConfig {
|
||||
resultsPerQuery: number;
|
||||
timeoutMs: number;
|
||||
};
|
||||
/** Root-only Jev selective-verification settings; absent means disabled. */
|
||||
jev?: {
|
||||
model: string;
|
||||
minConfidence: number;
|
||||
timeoutMs: number;
|
||||
maxContextChars: number;
|
||||
};
|
||||
breakGlassMarker: string;
|
||||
commentTag: string;
|
||||
/** Every configured provider credential (one entry for the legacy single shape). */
|
||||
@@ -468,4 +516,9 @@ export interface LoadedConfig {
|
||||
protectedCategories: Category[];
|
||||
maxAdjudications: number;
|
||||
};
|
||||
/** Root-only inline-comment settings (see ReviewConfigSchema.inline). */
|
||||
inline: {
|
||||
enabled: boolean;
|
||||
maxComments: number;
|
||||
};
|
||||
}
|
||||
@@ -50,6 +50,7 @@ export const FORBIDDEN_TOKEN_ENVS = new Set([
|
||||
"GITHUB_TOKEN",
|
||||
"GH_TOKEN",
|
||||
"BRAVE_SEARCH_API_KEY",
|
||||
"TYPESAFE_API_KEY",
|
||||
"ACTIONS_RUNTIME_TOKEN",
|
||||
"ACTIONS_ID_TOKEN_REQUEST_TOKEN",
|
||||
"AWS_ACCESS_KEY_ID",
|
||||
|
||||
@@ -582,6 +582,9 @@ export function parseClaudeResult(stdout: string): ClaudeResult {
|
||||
const tokens: TokenUsage = {
|
||||
input: num(usage.input_tokens),
|
||||
output: num(usage.output_tokens),
|
||||
reasoning: num(
|
||||
(usage.output_tokens_details as Record<string, unknown> | null | undefined)?.thinking_tokens,
|
||||
),
|
||||
cache: {
|
||||
write: num(usage.cache_creation_input_tokens),
|
||||
read: num(usage.cache_read_input_tokens),
|
||||
|
||||
+18
-4
@@ -4,7 +4,13 @@
|
||||
import { readdir, readFile, stat } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
import { CONFIG_DIRNAME, stripJsonComments, stripTrailingCommas } from "../config/load.js";
|
||||
import {
|
||||
CONFIG_DIRNAME,
|
||||
LEGACY_CONFIG_DIRNAME,
|
||||
configDirFor,
|
||||
stripJsonComments,
|
||||
stripTrailingCommas,
|
||||
} from "../config/load.js";
|
||||
import { ROUTING_FILENAME } from "../config/routing.js";
|
||||
import { git, pathInside } from "./exec.js";
|
||||
import { matchesIgnore } from "./noise.js";
|
||||
@@ -518,10 +524,18 @@ export async function discoverSetupDirs(root: string): Promise<string[]> {
|
||||
continue;
|
||||
}
|
||||
const child = path.join(dir, entry.name);
|
||||
if (entry.name === CONFIG_DIRNAME) {
|
||||
if (entry.name === LEGACY_CONFIG_DIRNAME) {
|
||||
found.push(child);
|
||||
continue; // a setup dir never nests another
|
||||
}
|
||||
if (entry.name === ".expo-agents") {
|
||||
// The shared Expo agent-tools dir: only its code-review/ child is ours.
|
||||
const setup = path.join(dir, CONFIG_DIRNAME);
|
||||
if ((await pathKind(setup)) === "dir") {
|
||||
found.push(setup);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// Other dot dirs are skipped wholesale: `.claude/worktrees/` holds checkouts of
|
||||
// this same repo, whose setup dirs would otherwise be swept as if they were scopes.
|
||||
if (entry.name === "node_modules" || entry.name.startsWith(".")) {
|
||||
@@ -617,7 +631,7 @@ async function checkRoutingManifest(
|
||||
}
|
||||
const configDir = typeof entry.config === "string" ? entry.config : null;
|
||||
if (configDir) {
|
||||
const setupDir = path.resolve(index.root, configDir, CONFIG_DIRNAME);
|
||||
const setupDir = configDirFor(path.resolve(index.root, configDir));
|
||||
if (!pathInside(setupDir, index.root)) {
|
||||
problems.push({
|
||||
file: relative,
|
||||
@@ -630,7 +644,7 @@ async function checkRoutingManifest(
|
||||
file: relative,
|
||||
line: 1,
|
||||
kind: "structural",
|
||||
problem: `scope "${name}" has no ${configDir}/${CONFIG_DIRNAME}/ directory`,
|
||||
problem: `scope "${name}" has no ${configDir}/${CONFIG_DIRNAME}/ (or ${LEGACY_CONFIG_DIRNAME}/) directory`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -56,6 +56,10 @@ export function reviewPostingConfigFingerprint(config: LoadedConfig): string {
|
||||
commentTag: config.commentTag,
|
||||
breakGlassMarker: config.breakGlassMarker,
|
||||
feedback: config.feedback,
|
||||
// Inline comments change what posting does (N review-comment mutations), so a
|
||||
// saved review must not post under a different inline policy than it was
|
||||
// saved with.
|
||||
inline: config.inline,
|
||||
}),
|
||||
)
|
||||
.digest("hex");
|
||||
|
||||
+265
@@ -0,0 +1,265 @@
|
||||
// @ref LLP 0014#active-selective-cascade [implements] — Jev handles narrow judgments and defers uncertainty
|
||||
import { TypeSafeClient, choice } from "@typesafe-ai/sdk";
|
||||
import { z } from "zod";
|
||||
|
||||
import type { LoadedConfig } from "../config/schema.js";
|
||||
import type { Finding } from "./schema.js";
|
||||
import { fingerprintFinding } from "./schema.js";
|
||||
|
||||
const TYPESAFE_API_URL = "https://api.typesafe.ai";
|
||||
const CONCURRENCY = 4;
|
||||
const MAX_FILE_CHARS = 1_000;
|
||||
const MAX_TITLE_CHARS = 1_000;
|
||||
const MAX_RATIONALE_CHARS = 6_000;
|
||||
const MAX_EVIDENCE_CHARS = 6_000;
|
||||
const INPUT_COST_PER_MILLION = 0.042;
|
||||
|
||||
const SUPPORT_CRITERIA = {
|
||||
supported: "The supplied source context directly demonstrates the exact reported problem.",
|
||||
needs_reasoning:
|
||||
"The context is insufficient, or the claim needs multi-file, runtime, arithmetic, temporal, or indirect reasoning.",
|
||||
contradicted:
|
||||
"The supplied source context directly demonstrates that the reported problem is not present.",
|
||||
} as const;
|
||||
|
||||
const probability = z.number().min(0).max(1);
|
||||
const JevResponseSchema = z.object({
|
||||
model: z.string().min(1),
|
||||
answers: z.object({
|
||||
support: z.object({
|
||||
type: z.literal("choice"),
|
||||
choice: z.enum(["supported", "needs_reasoning", "contradicted"]),
|
||||
confidence: probability,
|
||||
probabilities: z.object({
|
||||
supported: probability,
|
||||
needs_reasoning: probability,
|
||||
contradicted: probability,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
usage: z.object({
|
||||
input_tokens: z.number().int().nonnegative(),
|
||||
output_tokens: z.number().int().nonnegative(),
|
||||
}),
|
||||
});
|
||||
|
||||
export interface JevCandidate {
|
||||
finding: Finding;
|
||||
sourceContext: string;
|
||||
}
|
||||
|
||||
export interface JevEvaluation {
|
||||
fingerprint: string;
|
||||
model: string;
|
||||
support: keyof typeof SUPPORT_CRITERIA;
|
||||
confidence: number;
|
||||
probabilities: Record<keyof typeof SUPPORT_CRITERIA, number>;
|
||||
}
|
||||
|
||||
export interface JevVerificationSummary {
|
||||
configuredModel: string;
|
||||
actualModel?: string;
|
||||
durationMs: number;
|
||||
evaluated: number;
|
||||
failed: number;
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
cost: number;
|
||||
unavailable?: string;
|
||||
}
|
||||
|
||||
export interface JevVerificationResult {
|
||||
evaluations: Map<string, JevEvaluation>;
|
||||
summary: JevVerificationSummary;
|
||||
}
|
||||
|
||||
export interface EvaluateWithJevOptions {
|
||||
config: NonNullable<LoadedConfig["jev"]>;
|
||||
candidates: readonly JevCandidate[];
|
||||
env?: NodeJS.ProcessEnv;
|
||||
apiKey?: string;
|
||||
client?: TypeSafeClient;
|
||||
}
|
||||
|
||||
/** Run a reviewer-engine spawn without exposing the unrelated Jev credential. */
|
||||
export async function withoutJevCredential<T>(
|
||||
run: () => Promise<T>,
|
||||
env: NodeJS.ProcessEnv = process.env,
|
||||
): Promise<T> {
|
||||
const apiKey = env.TYPESAFE_API_KEY;
|
||||
try {
|
||||
delete env.TYPESAFE_API_KEY;
|
||||
return await run();
|
||||
} finally {
|
||||
if (apiKey === undefined) delete env.TYPESAFE_API_KEY;
|
||||
else env.TYPESAFE_API_KEY = apiKey;
|
||||
}
|
||||
}
|
||||
|
||||
/** Classify candidates with one atomic question; failures become missing evaluations. */
|
||||
export async function evaluateFindingsWithJev({
|
||||
config,
|
||||
candidates,
|
||||
env = process.env,
|
||||
apiKey: explicitApiKey,
|
||||
client,
|
||||
}: EvaluateWithJevOptions): Promise<JevVerificationResult> {
|
||||
const started = Date.now();
|
||||
const apiKey = (explicitApiKey ?? env.TYPESAFE_API_KEY)?.trim();
|
||||
if (!client && !apiKey) return emptyResult(config, started, "TYPESAFE_API_KEY is not set");
|
||||
|
||||
let resolvedClient: TypeSafeClient;
|
||||
try {
|
||||
resolvedClient =
|
||||
client ??
|
||||
new TypeSafeClient({
|
||||
apiKey,
|
||||
baseURL: TYPESAFE_API_URL,
|
||||
defaultModel: config.model,
|
||||
timeout: config.timeoutMs,
|
||||
retry: { maxRetries: 1 },
|
||||
logLevel: "off",
|
||||
});
|
||||
} catch {
|
||||
return emptyResult(config, started, "Jev client could not be initialized");
|
||||
}
|
||||
|
||||
const evaluations = new Map<string, JevEvaluation>();
|
||||
let failed = 0;
|
||||
let inputTokens = 0;
|
||||
let outputTokens = 0;
|
||||
let actualModel: string | undefined;
|
||||
let next = 0;
|
||||
|
||||
const worker = async (): Promise<void> => {
|
||||
while (next < candidates.length) {
|
||||
const candidate = candidates[next++];
|
||||
if (!candidate) continue;
|
||||
try {
|
||||
const response = JevResponseSchema.parse(
|
||||
await resolvedClient.systemOne({
|
||||
model: config.model,
|
||||
state: buildJevState(
|
||||
candidate.finding,
|
||||
candidate.sourceContext,
|
||||
config.maxContextChars,
|
||||
),
|
||||
questions: {
|
||||
support: choice(
|
||||
"Does this source context directly support or contradict this exact code-review finding? Select needs_reasoning whenever the answer depends on omitted context or non-local reasoning.",
|
||||
SUPPORT_CRITERIA,
|
||||
),
|
||||
},
|
||||
}),
|
||||
);
|
||||
inputTokens += response.usage.input_tokens;
|
||||
outputTokens += response.usage.output_tokens;
|
||||
actualModel = response.model;
|
||||
const fingerprint = fingerprintFinding(candidate.finding);
|
||||
evaluations.set(fingerprint, {
|
||||
fingerprint,
|
||||
model: response.model,
|
||||
support: response.answers.support.choice,
|
||||
confidence: response.answers.support.confidence,
|
||||
probabilities: response.answers.support.probabilities,
|
||||
});
|
||||
} catch {
|
||||
failed += 1;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
await Promise.all(
|
||||
Array.from({ length: Math.min(CONCURRENCY, candidates.length) }, () => worker()),
|
||||
);
|
||||
|
||||
return {
|
||||
evaluations,
|
||||
summary: {
|
||||
configuredModel: config.model,
|
||||
...(actualModel ? { actualModel } : {}),
|
||||
durationMs: Date.now() - started,
|
||||
evaluated: evaluations.size,
|
||||
failed,
|
||||
inputTokens,
|
||||
outputTokens,
|
||||
cost: (inputTokens * INPUT_COST_PER_MILLION) / 1_000_000,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** High-confidence local answers act; sensitive or uncertain answers defer. */
|
||||
export function jevDisposition(
|
||||
finding: Finding,
|
||||
evaluation: JevEvaluation | undefined,
|
||||
minConfidence: number,
|
||||
): "keep" | "drop" | "defer" {
|
||||
if (!evaluation || evaluation.confidence < minConfidence) return "defer";
|
||||
// Jev sees repository source, not the audited external passage. The reasoning
|
||||
// verifier remains responsible for citation support in either direction.
|
||||
if (finding.sources?.length) return "defer";
|
||||
if (evaluation.support === "supported") return "keep";
|
||||
if (evaluation.support === "needs_reasoning") return "defer";
|
||||
if (
|
||||
finding.severity === "critical" ||
|
||||
finding.category === "security" ||
|
||||
finding.category === "secrets"
|
||||
) {
|
||||
return "defer";
|
||||
}
|
||||
return "drop";
|
||||
}
|
||||
|
||||
/** Build the only state sent to TypeSafe: finding metadata plus bounded local source. */
|
||||
export function buildJevState(finding: Finding, sourceContext: string, maxContextChars: number) {
|
||||
return {
|
||||
finding: {
|
||||
severity: finding.severity,
|
||||
category: finding.category,
|
||||
file: boundText(finding.file, MAX_FILE_CHARS),
|
||||
line: finding.line ?? null,
|
||||
title: boundText(finding.title, MAX_TITLE_CHARS),
|
||||
rationale: boundText(finding.rationale, MAX_RATIONALE_CHARS),
|
||||
evidence: finding.evidence ? boundText(finding.evidence, MAX_EVIDENCE_CHARS) : null,
|
||||
},
|
||||
sourceContext: boundContext(sourceContext, finding.evidence, maxContextChars),
|
||||
};
|
||||
}
|
||||
|
||||
function boundText(value: string, maxChars: number): string {
|
||||
return value.length <= maxChars ? value : `${value.slice(0, maxChars - 1)}…`;
|
||||
}
|
||||
|
||||
function boundContext(context: string, evidence: string | undefined, maxChars: number): string {
|
||||
if (context.length <= maxChars) return context;
|
||||
const omitted = "\n... source omitted by ecr ...\n";
|
||||
const budget = Math.max(1, maxChars - omitted.length);
|
||||
const evidenceAt = evidence ? context.indexOf(evidence) : -1;
|
||||
if (evidenceAt >= 0) {
|
||||
const start = Math.max(0, evidenceAt - Math.floor(budget / 2));
|
||||
return context.slice(start, start + budget) + omitted;
|
||||
}
|
||||
const first = Math.floor(budget / 2);
|
||||
const last = budget - first;
|
||||
return context.slice(0, first) + omitted + context.slice(-last);
|
||||
}
|
||||
|
||||
function emptyResult(
|
||||
config: NonNullable<LoadedConfig["jev"]>,
|
||||
started: number,
|
||||
unavailable: string,
|
||||
): JevVerificationResult {
|
||||
return {
|
||||
evaluations: new Map(),
|
||||
summary: {
|
||||
configuredModel: config.model,
|
||||
durationMs: Date.now() - started,
|
||||
evaluated: 0,
|
||||
failed: 0,
|
||||
inputTokens: 0,
|
||||
outputTokens: 0,
|
||||
cost: 0,
|
||||
unavailable,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import type { CoordinatorOutput, Finding, ReviewMetadata, ReviewTrace } from "./
|
||||
import type { FilteredFile } from "./noise.js";
|
||||
import type { TokenUsage } from "./opencode.js";
|
||||
import type { ResearchProvenance } from "./research.js";
|
||||
import type { JevVerificationSummary } from "./jev.js";
|
||||
|
||||
export interface RunLogRecord {
|
||||
timestamp: string;
|
||||
@@ -21,6 +22,8 @@ export interface RunLogRecord {
|
||||
filteredFiles: FilteredFile[];
|
||||
/** Bounded queries/results plus grounded research decisions and final usefulness metrics. */
|
||||
research?: ResearchProvenance;
|
||||
/** Active Jev cascade usage and availability; individual drops use verifierDropped. */
|
||||
jevVerification?: JevVerificationSummary;
|
||||
agentCosts: Record<string, number>;
|
||||
totalCost: number;
|
||||
// Aggregate token usage across all agent + coordinator requests, for cache
|
||||
|
||||
+46
-8
@@ -20,10 +20,10 @@ import type { ResearchMcpRuntime } from "./research.js";
|
||||
export const CLAUDE_CODE_ENGINE = "claude-code" as const;
|
||||
|
||||
/** Fixed so repo config cannot redirect the Meta credential. */
|
||||
// @ref LLP 0003#muse-spark-via-meta-model-api [implements] — fixed endpoint and model allowlist
|
||||
// @ref LLP 0003#muse-spark-via-meta-model-api [implements] — fixed endpoint, provider routing, and known-model metadata
|
||||
export const META_MODEL_API_BASE_URL = "https://api.meta.ai/v1";
|
||||
|
||||
/** Supported public Muse models. Unknown ids remain visible to preflight. */
|
||||
/** Known public Muse models. Other explicit `meta/...` ids use neutral metadata. */
|
||||
export const META_MUSE_MODELS: Record<string, Record<string, unknown>> = {
|
||||
"muse-spark-1.2": {
|
||||
name: "Muse Spark 1.2",
|
||||
@@ -55,6 +55,36 @@ export const META_MUSE_MODELS: Record<string, Record<string, unknown>> = {
|
||||
include: ["reasoning.encrypted_content"],
|
||||
},
|
||||
},
|
||||
"muse-spark-1.3": {
|
||||
name: "Muse Spark 1.3",
|
||||
reasoning: true,
|
||||
limit: { context: 1_048_576, output: 1_048_576 },
|
||||
modalities: {
|
||||
input: ["text", "image", "pdf", "video"],
|
||||
output: ["text"],
|
||||
},
|
||||
options: {
|
||||
store: false,
|
||||
reasoningEffort: "high",
|
||||
reasoningSummary: "auto",
|
||||
include: ["reasoning.encrypted_content"],
|
||||
},
|
||||
},
|
||||
"muse-spark-1.3-contributor": {
|
||||
name: "Muse Spark 1.3 Contributor",
|
||||
reasoning: true,
|
||||
limit: { context: 1_048_576, output: 1_048_576 },
|
||||
modalities: {
|
||||
input: ["text", "image", "pdf", "video"],
|
||||
output: ["text"],
|
||||
},
|
||||
options: {
|
||||
store: false,
|
||||
reasoningEffort: "high",
|
||||
reasoningSummary: "auto",
|
||||
include: ["reasoning.encrypted_content"],
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -265,7 +295,19 @@ export function buildOpencodeConfig(
|
||||
entry.tokenEnv &&
|
||||
!entry.upstream
|
||||
) {
|
||||
// Muse needs the Responses adapter; openai-compatible uses Chat Completions.
|
||||
// Meta's Model API uses the Responses adapter; openai-compatible uses Chat
|
||||
// Completions. Register every explicitly referenced `meta/…` model so a new
|
||||
// Meta family/name does not require an ECR release merely to select the right
|
||||
// provider. Known Muse ids get their published capabilities; unknown ids stay
|
||||
// neutral and are validated by Meta on their first request.
|
||||
const models: Record<string, unknown> = {};
|
||||
for (const id of referencedModels) {
|
||||
const slash = id.indexOf("/");
|
||||
if (slash > 0 && id.slice(0, slash) === "meta" && id.length > slash + 1) {
|
||||
const model = id.slice(slash + 1);
|
||||
models[model] = META_MUSE_MODELS[model] ?? {};
|
||||
}
|
||||
}
|
||||
provider.meta = {
|
||||
npm: "@ai-sdk/openai",
|
||||
name: "Meta Model API",
|
||||
@@ -273,11 +315,7 @@ export function buildOpencodeConfig(
|
||||
baseURL: META_MODEL_API_BASE_URL,
|
||||
apiKey: `{env:${entry.tokenEnv}}`,
|
||||
},
|
||||
models: Object.fromEntries(
|
||||
Object.entries(META_MUSE_MODELS).filter(([model]) =>
|
||||
referencedModels.includes(`meta/${model}`),
|
||||
),
|
||||
),
|
||||
models,
|
||||
};
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -35,6 +35,16 @@ export interface LinkContext {
|
||||
repo: string; // owner/repo
|
||||
prNumber: number;
|
||||
diffLines?: Map<string, Set<number>>;
|
||||
/**
|
||||
* Permalinks to this run's live inline finding comments, keyed by the SAME id the
|
||||
* comment renders (plain fingerprint, or scope-namespaced for an aggregate). A
|
||||
* finding with an entry renders in SHORT form — title, links, one-line rationale —
|
||||
* because its full text lives in the inline thread; a finding without one renders
|
||||
* in full exactly as before. Callers must pass a per-render COPY of a shared
|
||||
* LinkContext (`{ ...link, inlineUrls }`), never mutate the shared object: the
|
||||
* routed ci flow shares one context across every scope reporter.
|
||||
*/
|
||||
inlineUrls?: Map<string, string>;
|
||||
/**
|
||||
* The PR base commit SHA (the branch the PR targets, e.g. `main`). When a finding
|
||||
* points at code NOT in the diff — a caller or helper the PR merely references,
|
||||
@@ -121,6 +131,29 @@ export function commentMarker(tag: string): string {
|
||||
return `<!-- ${tag} -->`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-finding identity marker of an inline (PR review) comment, always rendered at
|
||||
* byte 0 of the body. Substring-safe against `commentMarker(tag)` and any derived
|
||||
* scope tag (`<tag>:<scope>`) in both directions — the closing ` -->` differs —
|
||||
* including a scope literally named `inline`.
|
||||
*/
|
||||
export function inlineCommentMarker(tag: string, fp: string): string {
|
||||
return `<!-- ${tag}:inline:fp=${fp} -->`;
|
||||
}
|
||||
|
||||
/**
|
||||
* The fingerprint an inline comment of OURS carries, or null. Anchored at byte 0
|
||||
* and constrained to the fingerprint alphabet, so a marker echoed later in a body
|
||||
* (untrusted prose, a quote-reply) can never parse as identity — mechanical
|
||||
* safety, not just the incidental "our renderer emits it first". Identity itself
|
||||
* is author + this marker (see the reporter); the parse alone proves nothing.
|
||||
*/
|
||||
export function parseInlineMarkerFp(body: string, tag: string): string | null {
|
||||
const escapedTag = tag.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
const match = body.match(new RegExp(`^<!-- ${escapedTag}:inline:fp=([a-f0-9]{6,64}) -->`));
|
||||
return match ? match[1]! : null;
|
||||
}
|
||||
|
||||
// @ref LLP 0011#forged-state-markers [implements] — the first-match parsers make an earlier forged marker win, so untrusted prose never keeps a raw `<!--`
|
||||
/**
|
||||
* Neutralize anything that could impersonate this reviewer's embedded state
|
||||
@@ -348,12 +381,49 @@ function sourceLabel(value: string): string {
|
||||
.replace(/([\\[\]])/g, "\\$1");
|
||||
}
|
||||
|
||||
/** Longest one-line rationale excerpt the short form keeps in the main comment. */
|
||||
const SHORT_RATIONALE_CHARS = 160;
|
||||
|
||||
/**
|
||||
* A rationale reduced to one plain line for the short (inlined) form: whitespace
|
||||
* collapsed, every `<` escaped so a truncated HTML block (`<details>`) can never
|
||||
* leave an unclosed tag in the comment, cut at a word boundary with an ellipsis.
|
||||
* This excerpt is the audit trail that survives the PR author collapsing or
|
||||
* resolving the inline thread, so it must always be non-empty when the rationale is.
|
||||
*/
|
||||
export function oneLineRationale(text: string): string {
|
||||
const flat = text.replace(/</g, "<").replace(/\s+/g, " ").trim();
|
||||
if (flat.length <= SHORT_RATIONALE_CHARS) {
|
||||
return flat;
|
||||
}
|
||||
const cut = flat.slice(0, SHORT_RATIONALE_CHARS);
|
||||
const atWord = cut.slice(0, cut.lastIndexOf(" "));
|
||||
return `${atWord.length > 0 ? atWord : cut}…`;
|
||||
}
|
||||
|
||||
/** Only ever link to github.com from a stored/returned URL (see REPLY_URL_RE). */
|
||||
function validInlineUrl(link: LinkContext | undefined, id: string): string | null {
|
||||
const url = link?.inlineUrls?.get(id);
|
||||
return url && REPLY_URL_RE.test(url) ? url : null;
|
||||
}
|
||||
|
||||
function renderFindingLines(
|
||||
finding: Finding,
|
||||
link?: LinkContext,
|
||||
id: string = fingerprintFinding(finding),
|
||||
reply?: FeedbackRecord,
|
||||
): string[] {
|
||||
const inlineUrl = validInlineUrl(link, id);
|
||||
if (inlineUrl) {
|
||||
// Short form: the full text lives in the inline thread on the flagged line.
|
||||
// The one-line rationale stays here as the durable audit trail (the author
|
||||
// can collapse/resolve the thread; they cannot edit this comment).
|
||||
return [
|
||||
`- **${stripStateMarkers(finding.title)}** — ${location(finding, link)} _(${finding.category})_ · \`id:${id}\` · 💬 [inline comment](${inlineUrl})${replyAnnotation(reply)}`,
|
||||
...indentContinuation(oneLineRationale(stripStateMarkers(finding.rationale))),
|
||||
"",
|
||||
];
|
||||
}
|
||||
const out = [
|
||||
`- **${stripStateMarkers(finding.title)}** — ${location(finding, link)} _(${finding.category})_ · \`id:${id}\`${replyAnnotation(reply)}`,
|
||||
...indentContinuation(stripStateMarkers(finding.rationale)),
|
||||
@@ -376,6 +446,53 @@ function renderFindingLines(
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Full body of one inline (PR review) finding comment: identity marker at byte 0,
|
||||
* then the finding rendered whole — everything model-written passes
|
||||
* stripStateMarkers, same as the main comment. The footer teaches the clearing
|
||||
* flow (an UNQUOTED id citation), since the main comment only shows the short form.
|
||||
* The body must never contain `:state=`/`:fingerprints=` (it would be dropped as
|
||||
* our own main comment by the reply matcher's backstop) — it doesn't.
|
||||
*/
|
||||
export function renderInlineCommentBody(finding: Finding, tag: string, fp: string): string {
|
||||
const lines = [
|
||||
inlineCommentMarker(tag, fp),
|
||||
`**${severityHeading(finding.severity)}: ${stripStateMarkers(finding.title)}** _(${finding.category})_ · \`id:${fp}\``,
|
||||
"",
|
||||
stripStateMarkers(finding.rationale),
|
||||
];
|
||||
if (finding.sources?.length) {
|
||||
const sources = finding.sources
|
||||
.map((source) => `[${sourceLabel(source.title)}](<${source.url}>)`)
|
||||
.join(", ");
|
||||
lines.push("", `**Sources:** ${sources}`);
|
||||
}
|
||||
if (finding.suggestion) {
|
||||
lines.push("", `**Suggestion:** ${stripStateMarkers(finding.suggestion)}`);
|
||||
}
|
||||
lines.push(
|
||||
"",
|
||||
"---",
|
||||
`_🤖 AI review finding — reply here to respond; write \`id:${fp}\` (outside any quote) in your reply to formally answer it. Full review in the main PR comment._`,
|
||||
);
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* What a no-longer-tracked inline comment is patched to when its thread has human
|
||||
* replies (a bare one is deleted instead). Deliberately NEUTRAL: a finding leaves
|
||||
* the inline set for many reasons that are not resolution — capped out, its line
|
||||
* left the diff, aggregate-state truncation — so this text must never assert
|
||||
* "resolved" or "dismissed". The marker stays, so a returning finding revives the
|
||||
* same thread instead of opening a new one.
|
||||
*/
|
||||
export function inlineStubBody(tag: string, fp: string): string {
|
||||
return [
|
||||
inlineCommentMarker(tag, fp),
|
||||
"_🤖 This finding is no longer tracked inline — see the main review comment for current status._",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
// @ref LLP 0012#run-points-command-and-review [implements] — setup advice renders outside the findings list, so it never blocks
|
||||
/** Advice about the reviewer's own setup (stale refs, cited code this PR moves). */
|
||||
function setupNote(notes: string[] = []): string[] {
|
||||
|
||||
+50
-2
@@ -3,6 +3,9 @@ import type { FeedbackRecord, Finding } from "./schema.js";
|
||||
|
||||
/** One human comment on the PR, as the reporter hands it over. */
|
||||
export interface ReplyComment {
|
||||
/** Issue-comment id, or the NEGATED PR-review-comment id for an inline reply.
|
||||
* The two GitHub id sequences are independent, so negation keeps them
|
||||
* collision-free in every stored/compared FeedbackRecord.commentId. */
|
||||
id: number;
|
||||
body: string;
|
||||
login: string;
|
||||
@@ -12,6 +15,15 @@ export interface ReplyComment {
|
||||
* Absent/false is fail-closed — an unresolved author can never clear a finding. */
|
||||
author?: boolean;
|
||||
url?: string;
|
||||
/**
|
||||
* For an inline (PR review) reply: the fingerprint of OUR finding whose thread it
|
||||
* answers, resolved by the reporter from `in_reply_to_id` chaining to an
|
||||
* author-verified, marker-anchored inline comment of ours. A structural signal
|
||||
* chosen by the replier (they replied to that thread), so it matches in EVERY
|
||||
* `match` mode — but it never sets `citedId`: clearing still requires the
|
||||
* replier's own unquoted `id:` token.
|
||||
*/
|
||||
threadFp?: string;
|
||||
}
|
||||
|
||||
/** Blockquote lines read per comment. A reply that quotes half the review is
|
||||
@@ -35,6 +47,23 @@ const FINDING_ID_RE = /\bid:([a-f0-9]{6,64})\b/gi;
|
||||
*/
|
||||
const OWN_COMMENT_RE = /<!--[^\n]*:(?:state|fingerprints)=/;
|
||||
|
||||
/** The tag-independent shape of an inline finding comment's identity marker. */
|
||||
const INLINE_MARKER_RE = /<!--[^\n]*:inline:fp=/;
|
||||
|
||||
/**
|
||||
* Tag-independent backstop for OUR inline finding comments, which carry an
|
||||
* UNQUOTED `id:<fp>` token in the body (so a maintainer can copy it into a
|
||||
* reply). If the author/tag filters ever miss one — a bot-login fallback, a
|
||||
* commentTag rename, a second posting identity — matching it as a reply would
|
||||
* let the review cite (and, with maintainer association, CLEAR) its own
|
||||
* finding. Only UNQUOTED lines count: GitHub's "Quote reply" copies our marker
|
||||
* verbatim behind `> `, and that comment is a genuine human reply that must
|
||||
* still be matched.
|
||||
*/
|
||||
export function hasUnquotedInlineMarker(body: string): boolean {
|
||||
return body.split("\n").some((line) => !/^\s*>/.test(line) && INLINE_MARKER_RE.test(line));
|
||||
}
|
||||
|
||||
/**
|
||||
* Markdown → comparable text: link text without the target, no backticks or
|
||||
* emphasis marks, collapsed whitespace, lowercase, no trailing punctuation. A
|
||||
@@ -82,6 +111,20 @@ export function extractQuotedLines(body: string): string[] {
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Is `candidate` a more recent reply than `incumbent`? Issue comments carry
|
||||
* positive ids; inline (PR review) replies carry NEGATED ids, so within the
|
||||
* inline stream recency is the raw id's magnitude, not the signed value. The two
|
||||
* sequences are independent, so cross-stream recency is undecidable — the issue
|
||||
* comment wins deterministically (the main thread is the formal reply channel).
|
||||
*/
|
||||
export function replyIsNewer(candidate: number, incumbent: number): boolean {
|
||||
if (candidate >= 0 === incumbent >= 0) {
|
||||
return Math.abs(candidate) > Math.abs(incumbent);
|
||||
}
|
||||
return candidate >= 0;
|
||||
}
|
||||
|
||||
/** The `id:<hex>` tokens a comment cites, lowercased and deduped. */
|
||||
export function extractFindingIds(body: string): string[] {
|
||||
const ids = [...body.matchAll(FINDING_ID_RE)].map((match) => match[1]!.toLowerCase());
|
||||
@@ -137,7 +180,7 @@ export function matchReplies(
|
||||
|
||||
const newest = new Map<string, FeedbackRecord>();
|
||||
for (const comment of comments) {
|
||||
if (OWN_COMMENT_RE.test(comment.body)) {
|
||||
if (OWN_COMMENT_RE.test(comment.body) || hasUnquotedInlineMarker(comment.body)) {
|
||||
continue;
|
||||
}
|
||||
const matched = new Set<string>();
|
||||
@@ -145,6 +188,11 @@ export function matchReplies(
|
||||
// quote-matched reply still records WHETHER it cites the finding, because that is
|
||||
// what decides clearing.
|
||||
const cited = new Set(extractCitedFindingIds(comment.body));
|
||||
// An inline reply names its finding structurally — the replier chose which
|
||||
// thread to answer — so it matches in every `match` mode. Never sets `citedId`.
|
||||
if (comment.threadFp && known.has(comment.threadFp)) {
|
||||
matched.add(comment.threadFp);
|
||||
}
|
||||
if (opts.match !== "quote") {
|
||||
for (const id of extractFindingIds(comment.body)) {
|
||||
if (known.has(id)) {
|
||||
@@ -168,7 +216,7 @@ export function matchReplies(
|
||||
}
|
||||
for (const fp of matched) {
|
||||
const previous = newest.get(fp);
|
||||
if (previous && previous.commentId >= comment.id) {
|
||||
if (previous && !replyIsNewer(comment.id, previous.commentId)) {
|
||||
continue;
|
||||
}
|
||||
newest.set(fp, {
|
||||
|
||||
+25
-1
@@ -84,6 +84,8 @@ import {
|
||||
summarizeResearchUsefulness,
|
||||
} from "./research.js";
|
||||
import type { ResearchEvidence, ResearchMcpRuntime, ResearchProvenance } from "./research.js";
|
||||
import { withoutJevCredential } from "./jev.js";
|
||||
import type { JevVerificationSummary } from "./jev.js";
|
||||
|
||||
export interface ReviewRunOptions {
|
||||
config: LoadedConfig;
|
||||
@@ -297,6 +299,7 @@ export async function runReview(
|
||||
|
||||
let researchEvidence: ResearchEvidence[] = [];
|
||||
let researchRecord: ResearchProvenance | undefined;
|
||||
let jevRecord: JevVerificationSummary | undefined;
|
||||
|
||||
// Materialize the PR-head tree (not the current checkout) when the source can, so
|
||||
// the agents' surrounding-source reads and the verifier's re-reads see the versions
|
||||
@@ -387,9 +390,16 @@ export async function runReview(
|
||||
// close it. Two separate try blocks keep the precise per-engine error messages.
|
||||
let opencodeHandle: OpencodeHandle | null = null;
|
||||
let claudeHandle: ClaudeCodeHandle | null = null;
|
||||
// OpenCode's SDK copies process.env into its server. Withhold the unrelated Jev
|
||||
// credential at spawn time, then restore the parent immediately; Claude uses an
|
||||
// allowlisted child env already. The captured value is passed directly to Jev.
|
||||
// @ref LLP 0014#active-selective-cascade [constrained-by] — reviewer engines never receive the Jev credential
|
||||
const jevApiKey = process.env.TYPESAFE_API_KEY;
|
||||
try {
|
||||
if (usesOpencode) {
|
||||
opencodeHandle = await startOpencode(buildOpencodeConfig(config, researchRuntime));
|
||||
opencodeHandle = await withoutJevCredential(() =>
|
||||
startOpencode(buildOpencodeConfig(config, researchRuntime)),
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
await auth.cleanup();
|
||||
@@ -1089,6 +1099,9 @@ export async function runReview(
|
||||
process.cwd(),
|
||||
progress,
|
||||
researchEvidence,
|
||||
config.jev
|
||||
? { config: config.jev, ...(jevApiKey ? { apiKey: jevApiKey } : {}) }
|
||||
: undefined,
|
||||
);
|
||||
agentCosts["verifier"] = verification.cost;
|
||||
trackTokens("verifier", verification.tokens);
|
||||
@@ -1098,6 +1111,15 @@ export async function runReview(
|
||||
config.agents[0]?.model ?? config.coordinator.model,
|
||||
verification.model,
|
||||
);
|
||||
if (verification.jev) {
|
||||
jevRecord = verification.jev;
|
||||
agentCosts["jev-verifier"] = verification.jev.cost;
|
||||
trackTokens("jev-verifier", {
|
||||
input: verification.jev.inputTokens,
|
||||
output: verification.jev.outputTokens,
|
||||
});
|
||||
trackModel("jev-verifier", verification.jev.configuredModel, verification.jev.actualModel);
|
||||
}
|
||||
verifierDropped = verification.dropped;
|
||||
citationStrips = verification.citationStripped;
|
||||
if (verification.dropped.length > 0 || verification.citationStripped.length > 0) {
|
||||
@@ -1370,6 +1392,7 @@ export async function runReview(
|
||||
await safeLog(logPath, {
|
||||
...baseRecord,
|
||||
...(researchRecord ? { research: researchRecord } : {}),
|
||||
...(jevRecord ? { jevVerification: jevRecord } : {}),
|
||||
agentCosts,
|
||||
totalCost: sum(agentCosts),
|
||||
tokens: tokenTotals,
|
||||
@@ -1408,6 +1431,7 @@ export async function runReview(
|
||||
await safeLog(logPath, {
|
||||
...baseRecord,
|
||||
...(researchRecord ? { research: researchRecord } : {}),
|
||||
...(jevRecord ? { jevVerification: jevRecord } : {}),
|
||||
agentCosts,
|
||||
totalCost: sum(agentCosts),
|
||||
tokens: tokenTotals,
|
||||
|
||||
+10
-1
@@ -428,7 +428,16 @@ export function applyPins(
|
||||
pin.commentId === undefined ||
|
||||
!records.some(
|
||||
(record) =>
|
||||
record.fp === pin.fp && record.maintainer === true && record.commentId > pin.commentId!,
|
||||
record.fp === pin.fp &&
|
||||
record.maintainer === true &&
|
||||
// "Posted after the pin" is only decidable within one id sequence. Issue
|
||||
// comments carry positive ids and inline (PR review) replies carry NEGATED
|
||||
// ones (see the reporter's replyComments); the two sequences are
|
||||
// independent, so a cross-stream comparison says nothing about time — keep
|
||||
// the pin. Within a stream, recency is the raw id's magnitude (negation
|
||||
// inverts the signed order for inline ids).
|
||||
record.commentId >= 0 === pin.commentId! >= 0 &&
|
||||
Math.abs(record.commentId) > Math.abs(pin.commentId!),
|
||||
),
|
||||
);
|
||||
const pinnedFps = new Set(kept.map((pin) => pin.fp));
|
||||
|
||||
+75
-10
@@ -2,11 +2,15 @@
|
||||
// @ref LLP 0005#verifier-confinement-and-fail-open
|
||||
import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import type { TypeSafeClient } from "@typesafe-ai/sdk";
|
||||
|
||||
import { pathInside } from "./exec.js";
|
||||
import type { LoadedConfig } from "../config/schema.js";
|
||||
import { evaluateFindingsWithJev, jevDisposition } from "./jev.js";
|
||||
import type { JevVerificationSummary } from "./jev.js";
|
||||
import type { ResearchEvidence } from "./research.js";
|
||||
import type { Finding } from "./schema.js";
|
||||
import { parseVerdict } from "./schema.js";
|
||||
import { fingerprintFinding, parseVerdict } from "./schema.js";
|
||||
import { addTokenUsage, promptAndParse, VERIFIER_AGENT } from "./opencode.js";
|
||||
import type { OpencodeHandle, TokenUsage } from "./opencode.js";
|
||||
import { buildVerifierSystem, buildVerifierTask } from "./prompts.js";
|
||||
@@ -34,6 +38,8 @@ export interface VerificationResult {
|
||||
tokens: TokenUsage;
|
||||
/** provider/model that actually answered the verify calls (see PromptResult.model). */
|
||||
model?: string;
|
||||
/** Active selective-verification usage; absent when Jev is not configured. */
|
||||
jev?: JevVerificationSummary;
|
||||
}
|
||||
|
||||
/** The audited passages behind a finding's grounded citations, bounded per source. */
|
||||
@@ -94,10 +100,13 @@ export function matchEvidence(evidence: string, content: string): "present" | "a
|
||||
|
||||
// @ref LLP 0005#verifier-confinement-and-fail-open [implements] — pathInside gate: out-of-tree reads (and their present/absent verdict) refused
|
||||
/** Read the cited file and grade the evidence against it (see matchEvidence). */
|
||||
async function evidencePresence(
|
||||
async function inspectEvidence(
|
||||
finding: Finding,
|
||||
cwd: string,
|
||||
): Promise<"present" | "absent" | "unknown"> {
|
||||
): Promise<{
|
||||
presence: "present" | "absent" | "unknown";
|
||||
sourceContext: string;
|
||||
}> {
|
||||
// finding.file is an unconstrained, LLM-authored string produced over untrusted PR
|
||||
// content, so a prompt-injected finding could point it at a host secret. path.resolve
|
||||
// IGNORES cwd when finding.file is already absolute (e.g. ~/.claude/.credentials.json),
|
||||
@@ -107,15 +116,15 @@ async function evidencePresence(
|
||||
// materialized PR-head tree); anything outside is uncheckable, never read.
|
||||
const resolved = path.resolve(cwd, finding.file);
|
||||
if (!pathInside(resolved, cwd)) {
|
||||
return "unknown";
|
||||
return { presence: "unknown", sourceContext: "" };
|
||||
}
|
||||
let content: string;
|
||||
try {
|
||||
content = await readFile(resolved, "utf8");
|
||||
} catch {
|
||||
return "unknown";
|
||||
return { presence: "unknown", sourceContext: "" };
|
||||
}
|
||||
return matchEvidence(finding.evidence ?? "", content);
|
||||
return { presence: matchEvidence(finding.evidence ?? "", content), sourceContext: content };
|
||||
}
|
||||
|
||||
// @ref LLP 0005#verifier-confinement-and-fail-open [constrained-by] — fails open: a verify error/timeout keeps the finding, never drops it
|
||||
@@ -145,6 +154,12 @@ export async function verifyFindings(
|
||||
onProgress?: (message: string) => void,
|
||||
/** This run's audited research evidence, for findings that cite documentation. */
|
||||
researchEvidence: ResearchEvidence[] = [],
|
||||
jev?: {
|
||||
config: NonNullable<LoadedConfig["jev"]>;
|
||||
apiKey?: string;
|
||||
/** Test seam; production constructs the pinned client inside jev.ts. */
|
||||
client?: TypeSafeClient;
|
||||
},
|
||||
): Promise<VerificationResult> {
|
||||
const dropped: Array<{ finding: Finding; reason: string }> = [];
|
||||
const citationStripped: Array<{ finding: Finding; reason: string }> = [];
|
||||
@@ -156,9 +171,29 @@ export async function verifyFindings(
|
||||
|
||||
// Phase 1 — deterministic quote-grounding for every finding.
|
||||
const checked = await Promise.all(
|
||||
findings.map(async (finding) => ({ finding, presence: await evidencePresence(finding, cwd) })),
|
||||
findings.map(async (finding) => ({ finding, ...(await inspectEvidence(finding, cwd)) })),
|
||||
);
|
||||
|
||||
// Phase 2 — active selective classification. A confident local answer can
|
||||
// avoid the slower verifier; uncertainty is explicitly handed to it.
|
||||
const jevResult = jev
|
||||
? await evaluateFindingsWithJev({
|
||||
config: jev.config,
|
||||
candidates: checked.map(({ finding, sourceContext }) => ({ finding, sourceContext })),
|
||||
...(jev.apiKey ? { apiKey: jev.apiKey } : {}),
|
||||
...(jev.client ? { client: jev.client } : {}),
|
||||
})
|
||||
: undefined;
|
||||
if (jevResult?.summary.unavailable) {
|
||||
onProgress?.(
|
||||
` Jev unavailable (${jevResult.summary.unavailable}); using normal verification.`,
|
||||
);
|
||||
} else if (jevResult) {
|
||||
onProgress?.(
|
||||
` Jev classified ${jevResult.summary.evaluated} finding(s); ${jevResult.summary.failed} fell back.`,
|
||||
);
|
||||
}
|
||||
|
||||
// Decide which findings need an LLM check vs. can be kept directly. A finding
|
||||
// that cites documentation always gets an LLM check: the repo alone cannot
|
||||
// confirm an external-behavior claim, and the verifier must judge whether the
|
||||
@@ -171,14 +206,36 @@ export async function verifyFindings(
|
||||
}> = [];
|
||||
for (const { finding, presence } of checked) {
|
||||
const citedSources = citedSourcesFor(finding, researchEvidence);
|
||||
if (presence === "absent" || finding.severity === "critical" || citedSources) {
|
||||
const evaluation = jevResult?.evaluations.get(fingerprintFinding(finding));
|
||||
const disposition = evaluation
|
||||
? jevDisposition(finding, evaluation, jev?.config.minConfidence ?? 1)
|
||||
: undefined;
|
||||
if (disposition === "keep") {
|
||||
verdicts.set(finding, "keep");
|
||||
continue;
|
||||
}
|
||||
if (disposition === "drop") {
|
||||
verdicts.set(finding, "drop");
|
||||
dropped.push({
|
||||
finding,
|
||||
reason: `Jev directly contradicted the finding (${Math.round(evaluation!.confidence * 100)}% confidence)`,
|
||||
});
|
||||
onProgress?.(` Jev: dropped ${finding.severity} "${finding.title}" as contradicted.`);
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
disposition === "defer" ||
|
||||
presence === "absent" ||
|
||||
finding.severity === "critical" ||
|
||||
citedSources
|
||||
) {
|
||||
toVerify.push({ finding, presence, ...(citedSources ? { citedSources } : {}) });
|
||||
} else {
|
||||
verdicts.set(finding, "keep"); // grounded (or uncheckable) non-critical
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 2 — LLM verify (parallel). Refuted → drop; verified or errored → keep.
|
||||
// Phase 3 — reasoning verify (parallel). Refuted → drop; verified or errored → keep.
|
||||
await Promise.all(
|
||||
toVerify.map(async ({ finding, presence, citedSources }, index) => {
|
||||
try {
|
||||
@@ -242,5 +299,13 @@ export async function verifyFindings(
|
||||
const kept = findings
|
||||
.filter((finding) => verdicts.get(finding) === "keep")
|
||||
.map((finding) => replacements.get(finding) ?? finding);
|
||||
return { kept, dropped, citationStripped, cost, tokens, model };
|
||||
return {
|
||||
kept,
|
||||
dropped,
|
||||
citationStripped,
|
||||
cost,
|
||||
tokens,
|
||||
model,
|
||||
...(jevResult ? { jev: jevResult.summary } : {}),
|
||||
};
|
||||
}
|
||||
+724
-5
@@ -4,12 +4,16 @@ import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
import { resolveTrustedTool, run } from "../core/exec.js";
|
||||
import { errorMessage } from "../core/util.js";
|
||||
import { parseUnifiedDiff } from "../core/diff.js";
|
||||
import {
|
||||
buildDiffLineIndex,
|
||||
commentMarker,
|
||||
inlineStubBody,
|
||||
parseInlineMarkerFp,
|
||||
parseReviewState,
|
||||
renderAggregateMarkdown,
|
||||
renderInlineCommentBody,
|
||||
renderMarkdown,
|
||||
} from "../core/render.js";
|
||||
import type { LinkContext, ReviewState, ScopeReviewResult } from "../core/render.js";
|
||||
@@ -17,7 +21,13 @@ import { matchReplies } from "../core/responses.js";
|
||||
import type { ReplyComment } from "../core/responses.js";
|
||||
import { dropStaleVerdict, feedbackApplied, feedbackNeedsPrAuthor } from "../core/adjudicate.js";
|
||||
import type { AdjudicationItem } from "../core/adjudicate.js";
|
||||
import { applyPins, collectPins, fingerprintFinding, scopedFingerprint } from "../core/schema.js";
|
||||
import {
|
||||
applyPins,
|
||||
collectPins,
|
||||
fingerprintFinding,
|
||||
scopedFingerprint,
|
||||
SEVERITY_RANK,
|
||||
} from "../core/schema.js";
|
||||
import type {
|
||||
CoordinatorOutput,
|
||||
DismissalRecord,
|
||||
@@ -52,6 +62,13 @@ export interface GitHubReporterOptions {
|
||||
* the reporter behaves exactly as before until a caller opts in.
|
||||
*/
|
||||
feedback?: LoadedConfig["feedback"];
|
||||
/**
|
||||
* Root-only inline-comment config (see LoadedConfig.inline). Absent ⇒ the
|
||||
* reporter never touches PR review comments at all (zero extra API calls);
|
||||
* present with `enabled: false` ⇒ no sync, but `clear()` still sweeps this
|
||||
* tag's inline comments so a comment-mode switch or a disable cleans up.
|
||||
*/
|
||||
inline?: LoadedConfig["inline"];
|
||||
/**
|
||||
* The head commit this run reviews (the PR's immutable head OID, from the source's
|
||||
* metadata). Feedback merging binds a stored verdict to the source it judged, so a
|
||||
@@ -148,6 +165,188 @@ export interface IssueComment {
|
||||
html_url?: string;
|
||||
}
|
||||
|
||||
/** One PR review (inline/diff) comment, as the pulls comments API returns it. */
|
||||
export interface ReviewCommentApi {
|
||||
id: number;
|
||||
body?: string;
|
||||
path?: string;
|
||||
line?: number | null;
|
||||
user?: { login?: string };
|
||||
html_url?: string;
|
||||
/** Present on replies; points at the thread's root comment. */
|
||||
in_reply_to_id?: number;
|
||||
author_association?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Burst-collapsing cache over the paginated review-comments listing, shared across
|
||||
* reporter INSTANCES per PR (same motivation and key shape as prAuthorByPr): routed
|
||||
* single-mode CI builds a fresh reporter per scope for its clear() reconciliation,
|
||||
* and per-instance caching would re-fetch the identical list once per scope. The
|
||||
* short TTL and mutation invalidation mirror fetchAllComments.
|
||||
*/
|
||||
const reviewCommentsByPr = new Map<string, { at: number; comments: Promise<ReviewCommentApi[]> }>();
|
||||
|
||||
/** An existing inline comment of OURS (author + anchored marker), as the planner sees it. */
|
||||
export interface ExistingInlineComment {
|
||||
id: number;
|
||||
fp: string;
|
||||
body: string;
|
||||
url?: string;
|
||||
/** Any listed comment replies to it (`in_reply_to_id` === its id). */
|
||||
hasReplies: boolean;
|
||||
}
|
||||
|
||||
/** What one inline sync run will do. Bodies are rendered by the executor. */
|
||||
export interface InlineSyncPlan {
|
||||
/** Findings to POST a new inline comment for (needs a commit id). */
|
||||
create: Array<{ fp: string; finding: Finding }>;
|
||||
/** Live threads whose body converges to the finding's current render. */
|
||||
patch: Array<{ comment: ExistingInlineComment; finding: Finding }>;
|
||||
/** No-longer-tracked threads WITH replies: patch to the neutral stub. */
|
||||
stub: ExistingInlineComment[];
|
||||
/** No-longer-tracked threads with no replies (and older duplicates): delete. */
|
||||
remove: ExistingInlineComment[];
|
||||
}
|
||||
|
||||
// @ref LLP 0011#suppression-is-never-silent [constrained-by] — leaving the inline set is
|
||||
// not resolution: teardown only stubs/deletes when the caller proved the target set is
|
||||
// complete, and the stub text (see inlineStubBody) never claims resolved/dismissed
|
||||
/**
|
||||
* Decide the inline mutations for one sync, purely. `targets` are the ACTIVE,
|
||||
* diff-anchored findings; `existing` are OUR live inline comments.
|
||||
*
|
||||
* The cap is STICKY and deterministic: candidates order by severity rank then fp,
|
||||
* and fps that already have a live thread fill slots before new ones are admitted —
|
||||
* a live thread is never evicted (and its replies destroyed) just to create a
|
||||
* different one, and a tie at the boundary cannot flip membership run to run.
|
||||
*
|
||||
* `teardown: false` (a failed/partial/carried run whose target set may be
|
||||
* incomplete) makes the plan additive-only: no stubs, no deletes of live threads —
|
||||
* except older DUPLICATES of a kept fp (a crash window between create and list),
|
||||
* which are ours and redundant whatever the target set says.
|
||||
*/
|
||||
export function planInlineSync(opts: {
|
||||
targets: Array<{ fp: string; finding: Finding }>;
|
||||
existing: ExistingInlineComment[];
|
||||
maxComments: number;
|
||||
teardown: boolean;
|
||||
}): InlineSyncPlan {
|
||||
const ordered = [...opts.targets].sort(
|
||||
(a, b) =>
|
||||
SEVERITY_RANK[a.finding.severity] - SEVERITY_RANK[b.finding.severity] ||
|
||||
a.fp.localeCompare(b.fp),
|
||||
);
|
||||
|
||||
// Newest comment per fp is the live thread; older ones are duplicates.
|
||||
const liveByFp = new Map<string, ExistingInlineComment>();
|
||||
const duplicates: ExistingInlineComment[] = [];
|
||||
for (const comment of [...opts.existing].sort((a, b) => a.id - b.id)) {
|
||||
const previous = liveByFp.get(comment.fp);
|
||||
if (previous) {
|
||||
duplicates.push(previous);
|
||||
}
|
||||
liveByFp.set(comment.fp, comment);
|
||||
}
|
||||
|
||||
const selected: Array<{ fp: string; finding: Finding }> = [];
|
||||
for (const entry of ordered) {
|
||||
if (selected.length >= opts.maxComments) {
|
||||
break;
|
||||
}
|
||||
if (liveByFp.has(entry.fp)) {
|
||||
selected.push(entry);
|
||||
}
|
||||
}
|
||||
for (const entry of ordered) {
|
||||
if (selected.length >= opts.maxComments) {
|
||||
break;
|
||||
}
|
||||
if (!liveByFp.has(entry.fp)) {
|
||||
selected.push(entry);
|
||||
}
|
||||
}
|
||||
|
||||
const plan: InlineSyncPlan = { create: [], patch: [], stub: [], remove: [] };
|
||||
const selectedFps = new Set(selected.map((entry) => entry.fp));
|
||||
for (const entry of selected) {
|
||||
const live = liveByFp.get(entry.fp);
|
||||
if (live) {
|
||||
plan.patch.push({ comment: live, finding: entry.finding });
|
||||
} else {
|
||||
plan.create.push(entry);
|
||||
}
|
||||
}
|
||||
for (const [fp, live] of liveByFp) {
|
||||
if (selectedFps.has(fp)) {
|
||||
continue;
|
||||
}
|
||||
if (!opts.teardown) {
|
||||
continue;
|
||||
}
|
||||
if (live.hasReplies) {
|
||||
plan.stub.push(live);
|
||||
} else {
|
||||
plan.remove.push(live);
|
||||
}
|
||||
}
|
||||
// A duplicate with human replies is still a conversation: stub it like any other
|
||||
// retired thread rather than deleting what people wrote in.
|
||||
for (const duplicate of duplicates) {
|
||||
if (duplicate.hasReplies) {
|
||||
plan.stub.push(duplicate);
|
||||
} else {
|
||||
plan.remove.push(duplicate);
|
||||
}
|
||||
}
|
||||
return plan;
|
||||
}
|
||||
|
||||
/** `\r\n`-insensitive body comparison: the API returns CRLF-normalized bodies. */
|
||||
export function inlineBodiesEqual(a: string, b: string): boolean {
|
||||
return a.replace(/\r\n/g, "\n") === b.replace(/\r\n/g, "\n");
|
||||
}
|
||||
|
||||
/** Does a body's UNQUOTED text carry this marker? Quoted (`> `) lines are the
|
||||
* replier quoting US, which is a genuine reply, not our comment. */
|
||||
export function hasUnquotedMarker(body: string, marker: string): boolean {
|
||||
return body.split("\n").some((line) => !/^\s*>/.test(line) && line.includes(marker));
|
||||
}
|
||||
|
||||
/**
|
||||
* The findings the main comment renders in its ACTIVE list — not dismissed, not
|
||||
* cleared by an applied reply, not requalified — which is exactly the inline sync's
|
||||
* target set. Mirrors the isDropped/kept split in renderMarkdown and
|
||||
* renderAggregateMarkdown (first record per fp wins, like matchedFeedback there).
|
||||
* Exported for tests.
|
||||
*/
|
||||
export function activeFindings(
|
||||
withFp: Array<{ finding: Finding; fp: string }>,
|
||||
dismissed: DismissalRecord[],
|
||||
records: FeedbackRecord[],
|
||||
): Array<{ finding: Finding; fp: string }> {
|
||||
const dismissedFps = new Set(dismissed.map((record) => record.fp));
|
||||
const appliedFps = new Set<string>();
|
||||
const seen = new Set<string>();
|
||||
for (const record of records) {
|
||||
if (!seen.has(record.fp)) {
|
||||
seen.add(record.fp);
|
||||
if (record.applied) {
|
||||
appliedFps.add(record.fp);
|
||||
}
|
||||
}
|
||||
}
|
||||
return withFp.filter(
|
||||
({ finding, fp }) => !dismissedFps.has(fp) && !appliedFps.has(fp) && !finding.requalifiedBy,
|
||||
);
|
||||
}
|
||||
|
||||
/** A per-render COPY of the link context carrying this run's inline thread URLs —
|
||||
* the routed ci flow shares one LinkContext across reporters, so never mutate it. */
|
||||
function withInlineUrls(link: LinkContext, inlineUrls: Map<string, string>): LinkContext {
|
||||
return inlineUrls.size > 0 ? { ...link, inlineUrls } : link;
|
||||
}
|
||||
|
||||
// @ref LLP 0011#suppression-is-never-silent [implements] — a verdict carries only while BOTH the reply and the reviewed source are unchanged
|
||||
/**
|
||||
* Carry a decision already recorded in the prior comment state onto the freshly
|
||||
@@ -490,16 +689,38 @@ export class GitHubReporter implements Reporter {
|
||||
? applyPins(feedback, pinsIn)
|
||||
: await this.computeFeedback(withFp, priorRecords, pinsIn);
|
||||
const link = await this.linkContextAsync();
|
||||
// Inline threads converge to the ACTIVE findings before the body renders, so
|
||||
// the short form can link each one. A failed run (couldNotComplete) posts an
|
||||
// empty findings list that proves nothing — additive-only, no teardown.
|
||||
const inlineUrls = await this.syncInline(activeFindings(withFp, dismissed, records), link, {
|
||||
teardown: review.couldNotComplete !== true,
|
||||
});
|
||||
await this.upsertComment(
|
||||
renderMarkdown(review, this.options.commentTag, dismissed, link, records, pins, inputHash),
|
||||
renderMarkdown(
|
||||
review,
|
||||
this.options.commentTag,
|
||||
dismissed,
|
||||
withInlineUrls(link, inlineUrls),
|
||||
records,
|
||||
pins,
|
||||
inputHash,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Post/update the aggregate multi-scope comment (comment:'single' mode). */
|
||||
/**
|
||||
* Post/update the aggregate multi-scope comment (comment:'single' mode).
|
||||
* `opts.inlineTeardown: false` marks a run whose result set is NOT the complete
|
||||
* truth about the PR — a `--scopes` partial run carries the other scopes out of
|
||||
* (possibly truncated) embedded state — so the inline sync must not stub/delete
|
||||
* threads it merely cannot see. Any failed scope forces the same, since its
|
||||
* findings list is empty without being clean.
|
||||
*/
|
||||
async reportAggregate(
|
||||
results: ScopeReviewResult[],
|
||||
unmatchedFiles: string[],
|
||||
feedback?: FeedbackRecord[],
|
||||
opts?: { inlineTeardown?: boolean },
|
||||
): Promise<void> {
|
||||
const existing = await this.findExistingComment();
|
||||
const state = existing ? parseReviewState(existing.body, this.options.commentTag) : null;
|
||||
@@ -518,12 +739,18 @@ export class GitHubReporter implements Reporter {
|
||||
? applyPins(feedback, pinsIn)
|
||||
: await this.computeFeedback(withFp, priorRecords, pinsIn);
|
||||
const link = await this.linkContextAsync();
|
||||
const teardown =
|
||||
(opts?.inlineTeardown ?? true) &&
|
||||
results.every((result) => result.review.couldNotComplete !== true);
|
||||
const inlineUrls = await this.syncInline(activeFindings(withFp, dismissed, records), link, {
|
||||
teardown,
|
||||
});
|
||||
await this.upsertComment(
|
||||
renderAggregateMarkdown(
|
||||
results,
|
||||
this.options.commentTag,
|
||||
dismissed,
|
||||
link,
|
||||
withInlineUrls(link, inlineUrls),
|
||||
{ unmatchedFiles },
|
||||
records,
|
||||
pins,
|
||||
@@ -698,10 +925,17 @@ export class GitHubReporter implements Reporter {
|
||||
* failed lookup gives, and under any other `dismiss` value nothing reads the flag.
|
||||
*/
|
||||
private async replyComments(): Promise<ReplyComment[]> {
|
||||
const [comments, ownLogin, prAuthor] = await Promise.all([
|
||||
// Inline replies are read only when the reporter carries an ENABLED inline
|
||||
// config: the `ecr feedback` crawl and every non-inline repo keep exactly the
|
||||
// API footprint they had.
|
||||
const inlineEnabled = this.options.inline?.enabled === true;
|
||||
const [comments, ownLogin, prAuthor, reviewComments] = await Promise.all([
|
||||
this.fetchAllComments(),
|
||||
this.resolveOwnLogin(),
|
||||
feedbackNeedsPrAuthor(this.options.feedback) ? this.resolvePrAuthor() : null,
|
||||
inlineEnabled
|
||||
? this.fetchAllReviewComments().catch(() => [] as ReviewCommentApi[])
|
||||
: ([] as ReviewCommentApi[]),
|
||||
]);
|
||||
const out: ReplyComment[] = [];
|
||||
for (const comment of comments) {
|
||||
@@ -723,6 +957,64 @@ export class GitHubReporter implements Reporter {
|
||||
...(comment.html_url ? { url: comment.html_url } : {}),
|
||||
});
|
||||
}
|
||||
if (inlineEnabled && reviewComments.length > 0) {
|
||||
// fp per OUR top-level inline comment (author + anchored marker), so a reply
|
||||
// threading to it names its finding structurally.
|
||||
const fpByRootId = new Map<number, string>();
|
||||
if (ownLogin) {
|
||||
for (const comment of reviewComments) {
|
||||
if (comment.in_reply_to_id != null || comment.user?.login !== ownLogin) {
|
||||
continue;
|
||||
}
|
||||
const fp = parseInlineMarkerFp(comment.body ?? "", this.options.commentTag);
|
||||
if (fp) {
|
||||
fpByRootId.set(comment.id, fp);
|
||||
}
|
||||
}
|
||||
}
|
||||
const byId = new Map(reviewComments.map((comment) => [comment.id, comment]));
|
||||
// `in_reply_to_id` points at the thread root, but chase a short chain anyway
|
||||
// so an unexpected reply-to-reply shape still resolves.
|
||||
const threadFpOf = (comment: ReviewCommentApi): string | undefined => {
|
||||
let parentId = comment.in_reply_to_id;
|
||||
for (let hop = 0; parentId != null && hop < 5; hop++) {
|
||||
const fp = fpByRootId.get(parentId);
|
||||
if (fp) {
|
||||
return fp;
|
||||
}
|
||||
parentId = byId.get(parentId)?.in_reply_to_id;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
for (const comment of reviewComments) {
|
||||
const login = comment.user?.login;
|
||||
if (!login || (ownLogin && login === ownLogin)) {
|
||||
continue;
|
||||
}
|
||||
const body = comment.body ?? "";
|
||||
// Unquoted-line marker tests only: a GitHub "Quote reply" copies our marker
|
||||
// behind `> ` and is a genuine human reply that must still be matched.
|
||||
if (
|
||||
hasUnquotedMarker(body, this.marker) ||
|
||||
body.split("\n").some((line) => !/^\s*>/.test(line) && line.includes(":inline:fp="))
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const threadFp = threadFpOf(comment);
|
||||
out.push({
|
||||
// NEGATED: PR-review-comment ids are an independent sequence from issue
|
||||
// comments; negation keeps stored commentIds collision-free across streams
|
||||
// (and applyPins refuses cross-stream pin-lift comparisons).
|
||||
id: -comment.id,
|
||||
body,
|
||||
login,
|
||||
maintainer: MAINTAINER_ASSOCIATIONS.has(comment.author_association ?? ""),
|
||||
author: prAuthor != null && login === prAuthor,
|
||||
...(comment.html_url ? { url: comment.html_url } : {}),
|
||||
...(threadFp ? { threadFp } : {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -748,6 +1040,35 @@ export class GitHubReporter implements Reporter {
|
||||
for (const comment of await this.ownComments()) {
|
||||
await this.deleteComment(comment.id);
|
||||
}
|
||||
// Sweep this tag's inline threads too, gated on the config KEY being present
|
||||
// (not on `enabled`), so a comment-mode switch or a later disable still cleans
|
||||
// up — while callers that never wire `inline` (break-glass reporter, terminal
|
||||
// failure notes) keep their zero-review-API footprint. A thread humans replied
|
||||
// in is stubbed, never deleted. Fail-soft per thread.
|
||||
if (this.options.inline !== undefined) {
|
||||
let own: ExistingInlineComment[] = [];
|
||||
try {
|
||||
own = await this.ownInlineComments();
|
||||
} catch {
|
||||
return; // fail-soft: the main-comment clear above already happened
|
||||
}
|
||||
for (const comment of own) {
|
||||
try {
|
||||
if (comment.hasReplies) {
|
||||
const stub = inlineStubBody(this.options.commentTag, comment.fp);
|
||||
if (!inlineBodiesEqual(comment.body, stub)) {
|
||||
await this.patchReviewComment(comment.id, stub);
|
||||
}
|
||||
} else {
|
||||
await this.deleteReviewComment(comment.id);
|
||||
}
|
||||
} catch (error) {
|
||||
process.stderr.write(
|
||||
`inline sync: clear failed for one inline comment (skipped): ${errorMessage(error)}\n`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -793,6 +1114,365 @@ export class GitHubReporter implements Reporter {
|
||||
return link;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch ALL PR review (inline) comments, paginating manually like
|
||||
* fetchAllCommentsUncached. Unlike the issue-comments endpoint this one honors
|
||||
* sort/direction, but a full scan is needed anyway (we reconcile every own
|
||||
* thread), so the same oldest-first walk keeps the two fetchers uniform.
|
||||
*/
|
||||
private async fetchAllReviewCommentsUncached(): Promise<ReviewCommentApi[]> {
|
||||
const all: ReviewCommentApi[] = [];
|
||||
const gh = await resolveTrustedTool("gh");
|
||||
for (let page = 1; page <= GitHubReporter.MAX_COMMENT_PAGES; page++) {
|
||||
const { stdout } = await run(
|
||||
gh,
|
||||
[
|
||||
"api",
|
||||
"-X",
|
||||
"GET",
|
||||
`repos/${this.options.repo}/pulls/${this.options.prNumber}/comments`,
|
||||
"-f",
|
||||
"per_page=100",
|
||||
"-f",
|
||||
`page=${page}`,
|
||||
],
|
||||
{ cwd: this.options.cwd },
|
||||
);
|
||||
let batch: ReviewCommentApi[];
|
||||
try {
|
||||
batch = JSON.parse(stdout) as ReviewCommentApi[];
|
||||
} catch {
|
||||
break;
|
||||
}
|
||||
if (!Array.isArray(batch) || batch.length === 0) {
|
||||
break;
|
||||
}
|
||||
all.push(...batch);
|
||||
if (batch.length < 100) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
return all;
|
||||
}
|
||||
|
||||
/** Shared-per-PR, TTL'd review-comments listing (see reviewCommentsByPr). */
|
||||
private fetchAllReviewComments(): Promise<ReviewCommentApi[]> {
|
||||
const key = prAuthorCacheKey(this.options.repo, this.options.prNumber, this.options.cwd);
|
||||
const now = Date.now();
|
||||
const cached = reviewCommentsByPr.get(key);
|
||||
if (cached && now - cached.at <= GitHubReporter.COMMENTS_CACHE_TTL_MS) {
|
||||
return cached.comments;
|
||||
}
|
||||
const comments = this.fetchAllReviewCommentsUncached().catch((error: unknown) => {
|
||||
// Never cache a failure.
|
||||
if (reviewCommentsByPr.get(key)?.comments === comments) {
|
||||
reviewCommentsByPr.delete(key);
|
||||
}
|
||||
throw error;
|
||||
});
|
||||
reviewCommentsByPr.set(key, { at: now, comments });
|
||||
return comments;
|
||||
}
|
||||
|
||||
private invalidateReviewComments(): void {
|
||||
reviewCommentsByPr.delete(
|
||||
prAuthorCacheKey(this.options.repo, this.options.prNumber, this.options.cwd),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* OUR live inline finding comments under THIS tag: top-level, authored by our
|
||||
* own login (unspoofable), carrying the anchored byte-0 marker. Identity is
|
||||
* author + marker exactly like selectOwnComments; an unresolved login returns
|
||||
* nothing (fail closed). `hasReplies` is derived from the same listed snapshot.
|
||||
*/
|
||||
private async ownInlineComments(): Promise<ExistingInlineComment[]> {
|
||||
const [comments, ownLogin] = await Promise.all([
|
||||
this.fetchAllReviewComments(),
|
||||
this.resolveOwnLogin(),
|
||||
]);
|
||||
if (!ownLogin) {
|
||||
return [];
|
||||
}
|
||||
const repliedTo = new Set(
|
||||
comments
|
||||
.filter((comment) => comment.in_reply_to_id != null)
|
||||
.map((comment) => comment.in_reply_to_id!),
|
||||
);
|
||||
const own: ExistingInlineComment[] = [];
|
||||
for (const comment of comments) {
|
||||
if (comment.in_reply_to_id != null || comment.user?.login !== ownLogin) {
|
||||
continue;
|
||||
}
|
||||
const fp = parseInlineMarkerFp(comment.body ?? "", this.options.commentTag);
|
||||
if (!fp) {
|
||||
continue;
|
||||
}
|
||||
own.push({
|
||||
id: comment.id,
|
||||
fp,
|
||||
body: comment.body ?? "",
|
||||
...(comment.html_url ? { url: comment.html_url } : {}),
|
||||
hasReplies: repliedTo.has(comment.id),
|
||||
});
|
||||
}
|
||||
return own;
|
||||
}
|
||||
|
||||
/** The PR's CURRENT head OID, fetched live — or null. Used to refuse creates
|
||||
* whose commit_id (the head this run reviewed) is no longer the head. */
|
||||
private async liveHeadOid(): Promise<string | null> {
|
||||
try {
|
||||
const gh = await resolveTrustedTool("gh");
|
||||
const { stdout } = await run(
|
||||
gh,
|
||||
[
|
||||
"pr",
|
||||
"view",
|
||||
String(this.options.prNumber),
|
||||
"--repo",
|
||||
this.options.repo,
|
||||
"--json",
|
||||
"headRefOid",
|
||||
"--jq",
|
||||
".headRefOid",
|
||||
],
|
||||
{ cwd: this.options.cwd },
|
||||
);
|
||||
return stdout.trim() || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Secondary-rate-limit shapes (403/429): abort the rest of the sync, not just
|
||||
* this mutation — hammering on would dig the hole deeper. */
|
||||
private static isRateLimitError(error: unknown): boolean {
|
||||
return /HTTP 403|HTTP 429|rate limit|secondary rate/i.test(errorMessage(error));
|
||||
}
|
||||
|
||||
// @ref LLP 0008#github-reporter-identity [constrained-by] — inline threads are
|
||||
// patched/deleted only when author + anchored marker both match; creates also require
|
||||
// a resolved own login, or every run would duplicate what it cannot recognize
|
||||
/**
|
||||
* Converge this reporter's inline comments to `targets` (the ACTIVE, in-diff
|
||||
* findings), returning fp → thread permalink for the live full-form threads.
|
||||
* Never throws and never blocks the main comment: every failure degrades to
|
||||
* "that finding renders full-form in the main comment".
|
||||
*
|
||||
* `teardown: false` marks a run whose target set may be INCOMPLETE — a failed
|
||||
* review, a `--scopes` partial run, carried/truncated aggregate state — and makes
|
||||
* the sync additive-only (see planInlineSync): a finding this run cannot see must
|
||||
* not have its thread torn down.
|
||||
*
|
||||
* Creates require the reviewed head (`options.headSha`) to still BE the live head:
|
||||
* after a mid-review push the diff we validated lines against no longer matches
|
||||
* the commit we'd anchor to — a stale commit_id can silently pin the comment to
|
||||
* the wrong code (or 422 wholesale after a force-push) — so creates are skipped
|
||||
* and logged; patches/stubs/deletes are body-only and safe either way.
|
||||
*/
|
||||
private async syncInline(
|
||||
targets: Array<{ finding: Finding; fp: string }>,
|
||||
link: LinkContext,
|
||||
opts: { teardown: boolean },
|
||||
): Promise<Map<string, string>> {
|
||||
const urls = new Map<string, string>();
|
||||
const config = this.options.inline;
|
||||
if (!config?.enabled || !link.diffLines) {
|
||||
return urls;
|
||||
}
|
||||
const eligible = targets.filter(
|
||||
({ finding }) =>
|
||||
finding.line != null && link.diffLines!.get(finding.file)?.has(finding.line) === true,
|
||||
);
|
||||
let existing: ExistingInlineComment[];
|
||||
try {
|
||||
existing = await this.ownInlineComments();
|
||||
if (eligible.length > 0 && (await this.resolveOwnLogin()) == null) {
|
||||
// Without an own login we cannot recognize our previous comments, so a
|
||||
// create now would duplicate on every future run. Fail closed.
|
||||
process.stderr.write(
|
||||
"inline sync: could not resolve the posting identity; skipping inline comments this run.\n",
|
||||
);
|
||||
return urls;
|
||||
}
|
||||
} catch (error) {
|
||||
process.stderr.write(
|
||||
`inline sync: could not list review comments (findings render in the main comment): ${errorMessage(error)}\n`,
|
||||
);
|
||||
return urls;
|
||||
}
|
||||
|
||||
const plan = planInlineSync({
|
||||
targets: eligible,
|
||||
existing,
|
||||
maxComments: config.maxComments,
|
||||
teardown: opts.teardown,
|
||||
});
|
||||
|
||||
// One live-head check per sync, only when creating. A mismatch (or an unknown
|
||||
// reviewed head) skips creates; everything body-only still runs.
|
||||
let creates = plan.create;
|
||||
if (creates.length > 0) {
|
||||
const reviewedHead = this.options.headSha;
|
||||
const live = reviewedHead ? await this.liveHeadOid() : null;
|
||||
if (!reviewedHead || live !== reviewedHead) {
|
||||
process.stderr.write(
|
||||
`inline sync: PR head ${reviewedHead ? "moved during the review" : "is unknown"}; ` +
|
||||
`skipping ${creates.length} new inline comment(s) this run.\n`,
|
||||
);
|
||||
creates = [];
|
||||
}
|
||||
}
|
||||
|
||||
const counts = { created: 0, patched: 0, stubbed: 0, deleted: 0, failed: 0 };
|
||||
let aborted = false;
|
||||
const attempt = async (fn: () => Promise<void>): Promise<void> => {
|
||||
if (aborted) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await fn();
|
||||
} catch (error) {
|
||||
counts.failed++;
|
||||
if (GitHubReporter.isRateLimitError(error)) {
|
||||
aborted = true;
|
||||
process.stderr.write(
|
||||
`inline sync: rate-limited; aborting remaining inline mutations this run: ${errorMessage(error)}\n`,
|
||||
);
|
||||
} else {
|
||||
process.stderr.write(`inline sync: mutation failed (skipped): ${errorMessage(error)}\n`);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Sequential on purpose: parallel content creation is what trips GitHub's
|
||||
// secondary rate limit.
|
||||
for (const { comment, finding } of plan.patch) {
|
||||
if (comment.url) {
|
||||
urls.set(comment.fp, comment.url);
|
||||
}
|
||||
const body = renderInlineCommentBody(finding, this.options.commentTag, comment.fp);
|
||||
if (inlineBodiesEqual(comment.body, body)) {
|
||||
continue;
|
||||
}
|
||||
await attempt(async () => {
|
||||
await this.patchReviewComment(comment.id, body);
|
||||
counts.patched++;
|
||||
});
|
||||
}
|
||||
for (const entry of creates) {
|
||||
await attempt(async () => {
|
||||
const created = await this.createReviewComment(
|
||||
renderInlineCommentBody(entry.finding, this.options.commentTag, entry.fp),
|
||||
entry.finding.file,
|
||||
entry.finding.line!,
|
||||
);
|
||||
counts.created++;
|
||||
if (created?.html_url) {
|
||||
urls.set(entry.fp, created.html_url);
|
||||
}
|
||||
});
|
||||
}
|
||||
for (const comment of plan.stub) {
|
||||
const body = inlineStubBody(this.options.commentTag, comment.fp);
|
||||
if (inlineBodiesEqual(comment.body, body)) {
|
||||
continue;
|
||||
}
|
||||
await attempt(async () => {
|
||||
await this.patchReviewComment(comment.id, body);
|
||||
counts.stubbed++;
|
||||
});
|
||||
}
|
||||
for (const comment of plan.remove) {
|
||||
await attempt(async () => {
|
||||
await this.deleteReviewComment(comment.id);
|
||||
counts.deleted++;
|
||||
});
|
||||
}
|
||||
|
||||
const total = counts.created + counts.patched + counts.stubbed + counts.deleted + counts.failed;
|
||||
if (total > 0) {
|
||||
const summary =
|
||||
`inline sync: ${counts.created} created, ${counts.patched} patched, ` +
|
||||
`${counts.stubbed} stubbed, ${counts.deleted} deleted, ${counts.failed} failed`;
|
||||
process.stderr.write(`${summary}\n`);
|
||||
// Inline comments are mutated in place like the main comment, so this line is
|
||||
// the only per-run record of what changed (mirrors upsertComment's mirror).
|
||||
await appendStepSummary(`🤖 AI review — ${summary}`);
|
||||
}
|
||||
return urls;
|
||||
}
|
||||
|
||||
private async createReviewComment(
|
||||
body: string,
|
||||
path_: string,
|
||||
line: number,
|
||||
): Promise<{ html_url?: string } | null> {
|
||||
this.invalidateReviewComments();
|
||||
const gh = await resolveTrustedTool("gh");
|
||||
const payload = JSON.stringify({
|
||||
body,
|
||||
commit_id: this.options.headSha,
|
||||
path: path_,
|
||||
line,
|
||||
side: "RIGHT",
|
||||
});
|
||||
const dir = await mkdtemp(path.join(tmpdir(), "ecr-"));
|
||||
const jsonPath = path.join(dir, "review-comment.json");
|
||||
try {
|
||||
await writeFile(jsonPath, payload, "utf8");
|
||||
const { stdout } = await run(
|
||||
gh,
|
||||
[
|
||||
"api",
|
||||
"-X",
|
||||
"POST",
|
||||
`repos/${this.options.repo}/pulls/${this.options.prNumber}/comments`,
|
||||
"--input",
|
||||
jsonPath,
|
||||
],
|
||||
{ cwd: this.options.cwd },
|
||||
);
|
||||
try {
|
||||
return JSON.parse(stdout) as { html_url?: string };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
private async patchReviewComment(commentId: number, body: string): Promise<void> {
|
||||
this.invalidateReviewComments();
|
||||
const gh = await resolveTrustedTool("gh");
|
||||
await this.withBodyFile(body, (jsonPath) =>
|
||||
run(
|
||||
gh,
|
||||
[
|
||||
"api",
|
||||
"-X",
|
||||
"PATCH",
|
||||
`repos/${this.options.repo}/pulls/comments/${commentId}`,
|
||||
"--input",
|
||||
jsonPath,
|
||||
],
|
||||
{ cwd: this.options.cwd },
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
private async deleteReviewComment(commentId: number): Promise<void> {
|
||||
this.invalidateReviewComments();
|
||||
const gh = await resolveTrustedTool("gh");
|
||||
await run(
|
||||
gh,
|
||||
["api", "-X", "DELETE", `repos/${this.options.repo}/pulls/comments/${commentId}`],
|
||||
{ cwd: this.options.cwd },
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Add or remove per-PR finding dismissals in the reviewer's comment and re-render
|
||||
* it in place — no re-review needed (the comment embeds the full review state). The
|
||||
@@ -841,6 +1521,45 @@ export class GitHubReporter implements Reporter {
|
||||
state.inputHash,
|
||||
);
|
||||
await this.patchComment(existing.id, body);
|
||||
// Additive-only inline updates for exactly the fps this action named — never a
|
||||
// full stale sweep: the embedded state an aggregate stores may be TRUNCATED
|
||||
// (renderAggregateMarkdown trims kept findings to fit GitHub's body cap), so
|
||||
// "not in state" here does not mean "gone" and must not tear a thread down.
|
||||
// Fail-soft: a dismissal must never fail on the inline layer.
|
||||
if (this.options.inline?.enabled) {
|
||||
try {
|
||||
const own = await this.ownInlineComments();
|
||||
const byFp = new Map(own.map((comment) => [comment.fp, comment]));
|
||||
const findingById = stateFindingsById(state);
|
||||
for (const fp of next.matched) {
|
||||
const comment = byFp.get(fp);
|
||||
if (!comment) {
|
||||
continue;
|
||||
}
|
||||
const stub = inlineStubBody(this.options.commentTag, fp);
|
||||
if (!inlineBodiesEqual(comment.body, stub)) {
|
||||
await this.patchReviewComment(comment.id, stub);
|
||||
}
|
||||
}
|
||||
for (const fp of remove) {
|
||||
const comment = byFp.get(fp);
|
||||
const finding = findingById.get(fp);
|
||||
// A finding truncated out of the state (or a thread that was deleted) just
|
||||
// renders full-form in the main comment until the next review re-creates it.
|
||||
if (!comment || !finding) {
|
||||
continue;
|
||||
}
|
||||
const full = renderInlineCommentBody(finding, this.options.commentTag, fp);
|
||||
if (!inlineBodiesEqual(comment.body, full)) {
|
||||
await this.patchReviewComment(comment.id, full);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
process.stderr.write(
|
||||
`inline sync: dismissal update skipped (main comment already updated): ${errorMessage(error)}\n`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return {
|
||||
dismissedCount: next.dismissed.length,
|
||||
matched: next.matched,
|
||||
|
||||
@@ -22,17 +22,26 @@ const DIRECT_PROVIDER_ORDER: readonly ProviderId[] = [
|
||||
"android-releases",
|
||||
"media3",
|
||||
"agp",
|
||||
"android-ndk",
|
||||
"android",
|
||||
"glide",
|
||||
"okhttp",
|
||||
"kotlin-coroutines",
|
||||
"gradle",
|
||||
"jetbrains-issues",
|
||||
"react",
|
||||
"web-standards",
|
||||
"chrome-devtools-protocol",
|
||||
"flow",
|
||||
"typescript",
|
||||
"cmake",
|
||||
"cocoapods",
|
||||
"react-native-reanimated",
|
||||
"react-native-gesture-handler",
|
||||
"react-native-screens",
|
||||
"react-native-worklets",
|
||||
"react-native",
|
||||
"metro",
|
||||
"expo",
|
||||
];
|
||||
|
||||
@@ -49,13 +58,22 @@ const DIRECT_SOURCE_KIND: Record<ProviderId, SourceKind> = {
|
||||
"kotlin-coroutines": "official-guide",
|
||||
gradle: "official-guide",
|
||||
agp: "release-notes",
|
||||
"android-ndk": "official-guide",
|
||||
"jetbrains-issues": "issue-tracker",
|
||||
expo: "official-api",
|
||||
react: "official-api",
|
||||
"react-native": "official-api",
|
||||
"web-standards": "standard",
|
||||
"chrome-devtools-protocol": "standard",
|
||||
flow: "official-guide",
|
||||
typescript: "official-guide",
|
||||
cmake: "official-guide",
|
||||
cocoapods: "official-guide",
|
||||
"react-native-reanimated": "official-guide",
|
||||
"react-native-gesture-handler": "official-guide",
|
||||
"react-native-screens": "official-guide",
|
||||
"react-native-worklets": "official-guide",
|
||||
metro: "official-guide",
|
||||
};
|
||||
|
||||
export interface DirectDocumentationTarget {
|
||||
|
||||
@@ -345,6 +345,10 @@ export const expoProvider = htmlProvider("expo", "react-native", "Expo documenta
|
||||
{ hostname: "docs.expo.dev", prefixes: [""] },
|
||||
]);
|
||||
|
||||
export const reactProvider = htmlProvider("react", "react-native", "React documentation", [
|
||||
{ hostname: "react.dev", prefixes: ["/reference/react", "/reference/rules"] },
|
||||
]);
|
||||
|
||||
export const reactNativeProvider = htmlProvider(
|
||||
"react-native",
|
||||
"react-native",
|
||||
@@ -357,6 +361,66 @@ export const reactNativeProvider = htmlProvider(
|
||||
],
|
||||
);
|
||||
|
||||
export const webStandardsProvider = htmlProvider(
|
||||
"web-standards",
|
||||
"react-native",
|
||||
"Web platform standards",
|
||||
[
|
||||
{ hostname: "fetch.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "xhr.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "websockets.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "dom.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "streams.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "url.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "encoding.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "console.spec.whatwg.org", prefixes: [""] },
|
||||
{ hostname: "html.spec.whatwg.org", prefixes: ["/multipage"] },
|
||||
{
|
||||
hostname: "w3c.github.io",
|
||||
prefixes: [
|
||||
"/FileAPI",
|
||||
"/hr-time",
|
||||
"/performance-timeline",
|
||||
"/resource-timing",
|
||||
"/IntersectionObserver",
|
||||
],
|
||||
},
|
||||
],
|
||||
);
|
||||
|
||||
export const chromeDevtoolsProtocolProvider = htmlProvider(
|
||||
"chrome-devtools-protocol",
|
||||
"react-native",
|
||||
"Chrome DevTools Protocol documentation",
|
||||
[{ hostname: "chromedevtools.github.io", prefixes: ["/devtools-protocol"] }],
|
||||
);
|
||||
|
||||
export const flowProvider = htmlProvider("flow", "react-native", "Flow documentation", [
|
||||
{ hostname: "flow.org", prefixes: ["/en/docs"] },
|
||||
]);
|
||||
|
||||
export const typescriptProvider = htmlProvider(
|
||||
"typescript",
|
||||
"react-native",
|
||||
"TypeScript documentation",
|
||||
[{ hostname: "www.typescriptlang.org", prefixes: ["/docs"] }],
|
||||
);
|
||||
|
||||
export const androidNdkProvider = htmlProvider(
|
||||
"android-ndk",
|
||||
"android",
|
||||
"Android NDK documentation",
|
||||
[{ hostname: "developer.android.com", prefixes: ["/ndk"] }],
|
||||
);
|
||||
|
||||
export const cmakeProvider = htmlProvider("cmake", "android", "CMake documentation", [
|
||||
{ hostname: "cmake.org", prefixes: ["/cmake/help"] },
|
||||
]);
|
||||
|
||||
export const cocoaPodsProvider = htmlProvider("cocoapods", "apple", "CocoaPods guides", [
|
||||
{ hostname: "guides.cocoapods.org", prefixes: ["/syntax", "/using"] },
|
||||
]);
|
||||
|
||||
export const reactNativeReanimatedProvider = htmlProvider(
|
||||
"react-native-reanimated",
|
||||
"react-native",
|
||||
@@ -431,6 +495,10 @@ export const reactNativeWorkletsProvider = htmlProvider(
|
||||
true,
|
||||
);
|
||||
|
||||
export const metroProvider = htmlProvider("metro", "react-native", "Metro bundler documentation", [
|
||||
{ hostname: "metrobundler.dev", prefixes: [""] },
|
||||
]);
|
||||
|
||||
const providers: Record<ProviderId, DocumentationProvider> = {
|
||||
apple: appleProvider,
|
||||
"apple-releases": appleReleasesProvider,
|
||||
@@ -446,11 +514,20 @@ const providers: Record<ProviderId, DocumentationProvider> = {
|
||||
agp: agpProvider,
|
||||
"jetbrains-issues": jetbrainsIssuesProvider,
|
||||
expo: expoProvider,
|
||||
react: reactProvider,
|
||||
"react-native": reactNativeProvider,
|
||||
"web-standards": webStandardsProvider,
|
||||
"chrome-devtools-protocol": chromeDevtoolsProtocolProvider,
|
||||
flow: flowProvider,
|
||||
typescript: typescriptProvider,
|
||||
"android-ndk": androidNdkProvider,
|
||||
cmake: cmakeProvider,
|
||||
cocoapods: cocoaPodsProvider,
|
||||
"react-native-reanimated": reactNativeReanimatedProvider,
|
||||
"react-native-gesture-handler": reactNativeGestureHandlerProvider,
|
||||
"react-native-screens": reactNativeScreensProvider,
|
||||
"react-native-worklets": reactNativeWorkletsProvider,
|
||||
metro: metroProvider,
|
||||
};
|
||||
|
||||
export function getProvider(provider: ProviderId): DocumentationProvider {
|
||||
|
||||
@@ -71,10 +71,42 @@ const providerSearchDefinitions: Record<Exclude<ProviderId, "expo">, ProviderSea
|
||||
scopes: ["youtrack.jetbrains.com/issue"],
|
||||
sourceKind: "issue-tracker",
|
||||
},
|
||||
react: {
|
||||
scopes: ["react.dev/reference/react", "react.dev/reference/rules"],
|
||||
sourceKind: "official-api",
|
||||
},
|
||||
"react-native": {
|
||||
scopes: ["reactnative.dev"],
|
||||
sourceKind: "official-api",
|
||||
},
|
||||
"web-standards": {
|
||||
scopes: ["whatwg.org", "w3c.github.io"],
|
||||
sourceKind: "standard",
|
||||
},
|
||||
"chrome-devtools-protocol": {
|
||||
scopes: ["chromedevtools.github.io/devtools-protocol"],
|
||||
sourceKind: "standard",
|
||||
},
|
||||
flow: {
|
||||
scopes: ["flow.org/en/docs"],
|
||||
sourceKind: "official-guide",
|
||||
},
|
||||
typescript: {
|
||||
scopes: ["typescriptlang.org/docs"],
|
||||
sourceKind: "official-guide",
|
||||
},
|
||||
"android-ndk": {
|
||||
scopes: ["developer.android.com/ndk"],
|
||||
sourceKind: "official-guide",
|
||||
},
|
||||
cmake: {
|
||||
scopes: ["cmake.org/cmake/help/latest"],
|
||||
sourceKind: "official-guide",
|
||||
},
|
||||
cocoapods: {
|
||||
scopes: ["guides.cocoapods.org/syntax", "guides.cocoapods.org/using"],
|
||||
sourceKind: "official-guide",
|
||||
},
|
||||
"react-native-reanimated": {
|
||||
scopes: ["docs.swmansion.com/react-native-reanimated/docs"],
|
||||
sourceKind: "official-api",
|
||||
@@ -91,6 +123,10 @@ const providerSearchDefinitions: Record<Exclude<ProviderId, "expo">, ProviderSea
|
||||
scopes: ["docs.swmansion.com/react-native-worklets/docs"],
|
||||
sourceKind: "official-api",
|
||||
},
|
||||
metro: {
|
||||
scopes: ["metrobundler.dev/docs"],
|
||||
sourceKind: "official-guide",
|
||||
},
|
||||
};
|
||||
|
||||
export interface RemoteSearchOptions {
|
||||
|
||||
@@ -25,7 +25,7 @@ const queryGuidance =
|
||||
"Formulate short documentation queries from exact API symbols plus one behavior or constraint term. Good: `CameraView barcodeScannerSettings`, `NWPathMonitor pathUpdateHandler`, `GestureDetector simultaneous gestures`. Avoid questions, prose, package/import names, code snippets, literals, paths, credentials, and other sensitive context. If the first result is broad, retry with a narrower symbol or member name.";
|
||||
|
||||
const providerGuidance =
|
||||
"Provider map: apple=Apple SDK APIs and Human Interface Guidelines; apple-releases=Xcode and Apple platform release notes; swift-evolution=Swift Evolution proposals; sdwebimage=SDWebImage APIs and caching/loading behavior; android=Android, Jetpack, Compose, and Google Play services APIs; android-releases=Android platform releases and behavior changes; media3=Jetpack Media3; glide=Glide; okhttp=OkHttp; kotlin-coroutines=Kotlin coroutines; gradle=Gradle; agp=Android Gradle Plugin; jetbrains-issues=JetBrains YouTrack context; expo=Expo documentation; react-native=React Native core; react-native-reanimated=Reanimated; react-native-gesture-handler=Gesture Handler; react-native-screens=Screens; react-native-worklets=Worklets. Native source retains platform context: use apple/android for OS contracts and add the dependency provider for dependency-owned behavior; an Expo package path does not make a native API an Expo-docs query. Issue-tracker results are context, not API contracts.";
|
||||
"Provider map: apple=Apple SDK APIs and Human Interface Guidelines; apple-releases=Xcode and Apple platform release notes; swift-evolution=Swift Evolution proposals; sdwebimage=SDWebImage APIs and caching/loading behavior; android=Android, Jetpack, Compose, and Google Play services APIs; android-releases=Android platform releases and behavior changes; android-ndk=Android NDK; media3=Jetpack Media3; glide=Glide; okhttp=OkHttp; kotlin-coroutines=Kotlin coroutines; gradle=Gradle; agp=Android Gradle Plugin; jetbrains-issues=JetBrains YouTrack context; expo=Expo documentation; react=React APIs and semantics; react-native=React Native core documentation; web-standards=WHATWG and W3C web platform standards; chrome-devtools-protocol=Chrome DevTools Protocol; flow=Flow type-system documentation; typescript=TypeScript declaration semantics; cmake=CMake build-system documentation; cocoapods=CocoaPods integration guides; react-native-reanimated=Reanimated; react-native-gesture-handler=Gesture Handler; react-native-screens=Screens; react-native-worklets=Worklets; metro=Metro bundler. Native source retains platform context: use apple/android for OS contracts and add the dependency, build-tool, type-system, or web-contract provider for behavior that source owns; repository ownership does not make a native API a React Native or Expo documentation query. Standards describe compatibility targets; confirm the reviewed implementation claims that target before applying them. Issue-tracker results are context, not API contracts.";
|
||||
|
||||
export interface DocumentationServerOptions {
|
||||
braveApiKey?: string;
|
||||
@@ -84,7 +84,9 @@ export async function createDocumentationServer(options: DocumentationServerOpti
|
||||
.min(1)
|
||||
.max(SOURCE_KINDS.length)
|
||||
.optional()
|
||||
.describe("Optional provenance classes to search"),
|
||||
.describe(
|
||||
"Optional provenance classes to search; standard denotes an external compatibility specification rather than an implementation-specific API page",
|
||||
),
|
||||
},
|
||||
annotations: {
|
||||
readOnlyHint: true,
|
||||
|
||||
@@ -18,11 +18,20 @@ export const PROVIDERS = [
|
||||
"agp",
|
||||
"jetbrains-issues",
|
||||
"expo",
|
||||
"react",
|
||||
"react-native",
|
||||
"web-standards",
|
||||
"chrome-devtools-protocol",
|
||||
"flow",
|
||||
"typescript",
|
||||
"android-ndk",
|
||||
"cmake",
|
||||
"cocoapods",
|
||||
"react-native-reanimated",
|
||||
"react-native-gesture-handler",
|
||||
"react-native-screens",
|
||||
"react-native-worklets",
|
||||
"metro",
|
||||
] as const;
|
||||
|
||||
export type ProviderId = (typeof PROVIDERS)[number];
|
||||
@@ -30,6 +39,7 @@ export type ProviderId = (typeof PROVIDERS)[number];
|
||||
export const SOURCE_KINDS = [
|
||||
"official-api",
|
||||
"official-guide",
|
||||
"standard",
|
||||
"release-notes",
|
||||
"issue-tracker",
|
||||
] as const;
|
||||
|
||||
@@ -114,6 +114,8 @@ jobs:
|
||||
# META_API_KEY: ${{ secrets.META_API_KEY }}
|
||||
# Optional search-only credential for trusted platform documentation research.
|
||||
BRAVE_SEARCH_API_KEY: ${{ secrets.BRAVE_SEARCH_API_KEY }}
|
||||
# Optional Jev verification credential (used when the root config has `jev`).
|
||||
TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
|
||||
# Optional: override the model for every agent.
|
||||
REVIEWER_MODEL: ${{ vars.REVIEWER_MODEL }}
|
||||
run: |
|
||||
@@ -128,6 +130,8 @@ jobs:
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: review-run-log-pr${{ github.event.issue.number }}
|
||||
path: .expo-code-review/.runs/reviews.jsonl
|
||||
path: |
|
||||
.expo-agents/code-review/.runs/reviews.jsonl
|
||||
.expo-code-review/.runs/reviews.jsonl
|
||||
if-no-files-found: ignore
|
||||
retention-days: 14
|
||||
@@ -152,6 +152,8 @@ jobs:
|
||||
# META_API_KEY: ${{ secrets.META_API_KEY }}
|
||||
# Optional search-only credential for trusted platform documentation research.
|
||||
BRAVE_SEARCH_API_KEY: ${{ secrets.BRAVE_SEARCH_API_KEY }}
|
||||
# Optional Jev verification credential (used when the root config has `jev`).
|
||||
TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
|
||||
# Optional: override the model for every agent.
|
||||
REVIEWER_MODEL: ${{ vars.REVIEWER_MODEL }}
|
||||
AGENTS: ${{ steps.cmd.outputs.agents }}
|
||||
@@ -183,6 +185,8 @@ jobs:
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: review-run-log-pr${{ github.event.issue.number }}
|
||||
path: .expo-code-review/.runs/reviews.jsonl
|
||||
path: |
|
||||
.expo-agents/code-review/.runs/reviews.jsonl
|
||||
.expo-code-review/.runs/reviews.jsonl
|
||||
if-no-files-found: ignore
|
||||
retention-days: 14
|
||||
+29
-1
@@ -48,6 +48,18 @@
|
||||
// "timeoutMs": 30000
|
||||
// },
|
||||
|
||||
// Optional Jev verification cascade (ROOT-ONLY). Adding this block activates it:
|
||||
// confident local support keeps a finding, confident contradiction drops an
|
||||
// ordinary finding, and uncertainty or protected findings defer to the reasoning
|
||||
// verifier. Pin the model because moving aliases can invalidate the threshold.
|
||||
// Set TYPESAFE_API_KEY in the environment before adding the block.
|
||||
// "jev": {
|
||||
// "model": "jev-1.13.0",
|
||||
// "minConfidence": 0.9,
|
||||
// "timeoutMs": 10000,
|
||||
// "maxContextChars": 30000
|
||||
// },
|
||||
|
||||
// Large diffs are split into focused chunks by changed-line count, plus a
|
||||
// cross-cutting pass for multi-file issues. Diffs under maxChangedLines are one
|
||||
// full-context pass. Defaults shown; raise/lower per your model + PR sizes.
|
||||
@@ -87,7 +99,7 @@
|
||||
// "auth": { "mode": "api-key", "provider": "openai", "tokenEnv": "OPENAI_API_KEY" }
|
||||
//
|
||||
// Muse Spark via the public Meta Model API Responses endpoint:
|
||||
// set models to `meta/muse-spark-1.2` and configure:
|
||||
// set models to `meta/muse-spark-1.3` and configure:
|
||||
// "auth": { "providers": {
|
||||
// "meta": { "mode": "api-key", "tokenEnv": "META_API_KEY" }
|
||||
// } }
|
||||
@@ -160,4 +172,20 @@
|
||||
// "protectedCategories": ["secrets", "security"],
|
||||
// "maxAdjudications": 10 // cap on model calls per run
|
||||
// }
|
||||
|
||||
// Inline PR comments (ROOT-ONLY, off by default). When enabled, each finding
|
||||
// anchored to a line in the PR diff is posted as an inline review comment on
|
||||
// that line (full rationale/suggestion there), and the main comment renders it
|
||||
// short — title, links, a one-line rationale — with a link to the thread. The
|
||||
// main comment stays the single source of truth (state, /dismiss, replies).
|
||||
// Threads update in place across re-reviews; a retired thread with human
|
||||
// replies is stubbed, never deleted. Replies on inline threads are matched to
|
||||
// their finding like main-comment replies (same clearing rules).
|
||||
// NOTE: the first enabled run on a PR sends one notification per created
|
||||
// inline comment — `maxComments` bounds that. Disabling later leaves existing
|
||||
// threads up until a comment-mode switch (or manual cleanup) clears them.
|
||||
// "inline": {
|
||||
// "enabled": false,
|
||||
// "maxComments": 20 // per PR (single comment mode) / per scope (per-scope mode)
|
||||
// }
|
||||
}
|
||||
@@ -118,6 +118,8 @@ jobs:
|
||||
# META_API_KEY: ${{ secrets.META_API_KEY }}
|
||||
# Optional search-only credential for trusted platform documentation research.
|
||||
BRAVE_SEARCH_API_KEY: ${{ secrets.BRAVE_SEARCH_API_KEY }}
|
||||
# Optional Jev verification credential (used when the root config has `jev`).
|
||||
TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
|
||||
# Optional: override the model for every agent.
|
||||
REVIEWER_MODEL: ${{ vars.REVIEWER_MODEL }}
|
||||
|
||||
@@ -133,6 +135,8 @@ jobs:
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: review-run-log-pr${{ github.event.pull_request.number }}
|
||||
path: .expo-code-review/.runs/reviews.jsonl
|
||||
path: |
|
||||
.expo-agents/code-review/.runs/reviews.jsonl
|
||||
.expo-code-review/.runs/reviews.jsonl
|
||||
if-no-files-found: ignore
|
||||
retention-days: 14
|
||||
Reference in new issue
Block a user