feat(browser): trace captured website source-map locations (#897)

* refactor(browser): share captured-script acquisition boundaries

Reuse stable capture/source acquisition and identity validation for module
and source-map analysis. Preserve host I/O, malformed input, unavailable
source and digest mismatch diagnostics.

Refs #851

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)

* feat(browser): trace captured website source-map locations

Refs #851. Preserve selected capture/map byte identity, duplicate mappings and embedded source evidence through the pinned upstream codec.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)

* test(browser): verify source-map artifacts and synchronize generated contracts

Cover byte-preserving map reads and actionable filesystem constraints. Refresh generated public schemas and metadata on current main.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)

* fix(browser): preserve inert source-map URL context

The selected URL is local codec context and grants no transport authority. Preserve its reported text and verify normal/userinfo contexts through real public interfaces, alongside independent manifest byte identity.

Assisted-by: Yukari (gensokyo-skills:yukari-boundary-analysis)
Assisted-by: Kogasa (gensokyo-skills:kogasa-surprise-testing)
This commit is contained in:
ysopwnable
2026-10-07 16:21:18 +08:00
committed by GitHub
parent 4801a24089
commit f430c416c7
58 changed files with 3420 additions and 282 deletions
+50
View File
@@ -0,0 +1,50 @@
name: Real website source-map verification
on:
pull_request:
paths:
- ".github/workflows/real-web-source-map.yml"
- "scripts/verify-browser-source-maps.mjs"
- "scripts/lib/browser-source-map-e2e.mjs"
- "scripts/fixtures/browser-source-map-site.mjs"
- "src/javascript/sourceMaps/**"
- "src/browser/assets/**"
- "src/domain/webSource*.ts"
- "src/domain/webScript*.ts"
- "src/application/WebSourceLocation*.ts"
- "src/application/ValidateSelectedWebScript.ts"
- "src/composition/webSourceLocations.ts"
- "src/server/registerWebSourceLocationTool.ts"
- "src/server/createServer.ts"
- "src/contracts/webScriptToolContracts.ts"
- "src/process/**"
- "src/cli/webSourceLocationCommands.ts"
- "src/cli.ts"
- "src/artifacts/readStableArtifact.ts"
- "package.json"
- "package-lock.json"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: real-web-source-map-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
HUSKY: "0"
jobs:
real-web-source-map:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24
cache: npm
- run: npm ci
- name: Install the pinned compiler into an isolated fixture directory
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false [email protected]
- run: npm run verify:browser:source-maps
env:
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
REA_BROWSER_NO_SANDBOX: "true"
REA_WEB_SOURCE_MAP_COMPILER: ${{ runner.temp }}/rea-web-source-map-compiler/node_modules/esbuild/lib/main.js
+6 -1
View File
@@ -411,7 +411,7 @@ See [native investigation](docs/native-investigation.md) for keyed archives, ins
| Browser observation | 9 | page structure, network metadata, scripts, source maps, WebMCP discovery, screenshots, and capture comparisons |
| Electron analysis | 5 | renderer observation, static app mapping, and static/runtime reconciliation |
| JavaScript runtime | 2 | Node/Electron Inspector target discovery, script locations, and execution-context events |
| Application workflows | 12 | captured website script export; Android/Apple inventory projections; cross-layer feature traces, build comparisons, historical source mapping, static return-shape comparison, and reconstruction checks |
| Application workflows | 13 | captured website script export; Android/Apple inventory projections; cross-layer feature traces, build comparisons, historical source mapping, static return-shape comparison, and reconstruction checks |
| Workspace and observation | 21 | sessions, evidence bundles, navigation context, process/artifact/function comparisons, and open-question tracking |
The public interface describes what the agent is trying to learn. Providers decide how to answer. macOS utilities handle common semantic inspection without launching Hopper; Hopper handles deeper native analysis; the process harness records direct behavioral captures.
@@ -436,6 +436,11 @@ Static Android APK inspection is verified on Linux with headless JADX; see [Andr
`rea trace-web-module-imports`, preserving query/fragment identity and optional
import-map context. Requires caller-supplied Chromium via
`REA_BROWSER_EXECUTABLE`; see [module relationships](docs/website-module-trace.md).
- **Source locations:** Trace one retained website script point through a selected
local source map with `rea trace-web-source-location`. See
[captured website source locations](docs/web-source-location.md) for byte
identities, embedded original content and coverage limits.
Recover readable modules from selected local bundles with
`recover_javascript_sources` / `rea recover-javascript-sources`, then pass
the returned `analysis_input` to static analysis. This optional Linux x64
+1 -1
View File
@@ -223,7 +223,7 @@ rea setup
| مراقبة المتصفح | 9 | بنية الصفحات وبيانات الشبكة والبرامج النصية وخرائط المصدر واكتشاف WebMCP والصور ومقارنة الالتقاط |
| تحليل Electron | 5 | مراقبة العارضين وخريطة التطبيق الثابتة والمواءمة بين نتائج الثبات والتشغيل |
| وقت تشغيل JavaScript | 2 | اكتشاف أهداف Node/Electron Inspector ومواضع البرامج وأحداث سياقات التنفيذ |
| سير عمل التطبيقات | 12 | تصدير النصوص البرمجية الملتقطة، تتبع الميزات عبر الطبقات ومقارنة الإصدارات ومواءمة المصدر التاريخي ومقارنة أشكال القيم المعادة والتحقق من إعادة الإنشاء |
| سير عمل التطبيقات | 13 | تصدير النصوص البرمجية الملتقطة، تتبع الميزات عبر الطبقات ومقارنة الإصدارات ومواءمة المصدر التاريخي ومقارنة أشكال القيم المعادة والتحقق من إعادة الإنشاء |
| مساحة العمل والملاحظة | 21 | الجلسات وحزم الأدلة وسياق التنقل ومقارنة العمليات والمخرجات والدوال وتسجيل الأسئلة غير المحسومة |
## خطة العمل
+1 -1
View File
@@ -212,7 +212,7 @@ REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェ
| ブラウザー観察 | 9 | ページ構成、ネットワークメタデータ、スクリプト、ソースマップ、WebMCP 検出、スクリーンショット、キャプチャ比較 |
| Electron 解析 | 5 | レンダラー観察、静的アプリマッピング、静的/実行時結果の対応付け |
| JavaScript 実行時 | 2 | Node/Electron Inspector 対象の検出、スクリプト位置、実行コンテキストイベント |
| アプリワークフロー | 12 | 取得した Web スクリプトのエクスポート、層をまたぐ機能トレース、ビルド比較、履歴ソース対応、静的戻り値形状の比較、再実装検証 |
| アプリワークフロー | 13 | 取得した Web スクリプトのエクスポート、層をまたぐ機能トレース、ビルド比較、履歴ソース対応、静的戻り値形状の比較、再実装検証 |
| ワークスペースと観察 | 21 | セッション、証跡バンドル、ナビゲーション文脈、プロセス・成果物・関数比較、未解決事項の記録 |
## ロードマップ
+1 -1
View File
@@ -213,7 +213,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
| 브라우저 관찰 | 9 | 페이지 구조, 네트워크 메타데이터, 스크립트, 소스 맵, WebMCP 탐색, 스크린샷, 캡처 비교 |
| Electron 분석 | 5 | 렌더러 관찰, 정적 앱 매핑, 정적·런타임 결과 연결 |
| JavaScript 런타임 | 2 | Node/Electron Inspector 대상 탐색, 스크립트 위치, 실행 컨텍스트 이벤트 |
| 앱 워크플로 | 12 | 캡처한 웹 스크립트 내보내기, 계층 간 기능 추적, 빌드 비교, 히스토리 소스 매핑, 정적 반환 구조 비교, 재구현 검증 |
| 앱 워크플로 | 13 | 캡처한 웹 스크립트 내보내기, 계층 간 기능 추적, 빌드 비교, 히스토리 소스 매핑, 정적 반환 구조 비교, 재구현 검증 |
| 작업 공간과 관찰 | 21 | 세션, 근거 번들, 탐색 컨텍스트, 프로세스·아티팩트·함수 비교, 미해결 항목 기록 |
## 로드맵
+1 -1
View File
@@ -212,7 +212,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
| 浏览器观察 | 9 | 页面结构、网络元数据、脚本、来源映射、WebMCP 发现、截图与捕获比较 |
| Electron 分析 | 5 | 渲染进程观察、静态应用映射、静态/运行时结果关联 |
| JavaScript 运行时 | 2 | Node/Electron Inspector 目标发现、脚本位置与执行上下文事件 |
| 应用工作流 | 12 | 捕获的网站脚本导出、跨层功能追踪、构建比较、历史源码映射、静态返回结构比较与重建验证 |
| 应用工作流 | 13 | 捕获的网站脚本导出、跨层功能追踪、构建比较、历史源码映射、静态返回结构比较与重建验证 |
| 工作区与观察 | 21 | 会话、证据包、导航上下文、进程/产物/函数比较与待解决问题跟踪 |
## 路线图
+4
View File
@@ -28,6 +28,7 @@ graph TD
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
AUX["src/artifacts/ + src/native/<br/>disjoint artifact and native capabilities"]
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution"]
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
DOTNET["src/dotnet/<br/>Managed static metadata provider<br/>PE/CLI metadata inventory, members, IL"]
FIRMWARE["src/firmware/<br/>Binwalk / Unblob adapters<br/>owned processes, report normalization, private extraction"]
@@ -95,6 +96,9 @@ graph TD
SERVER --> CONTRACTS
SERVER --> APP
APP --> BROWSER
APP -->|source-map codec port| JS_CODEC
JS_CODEC --> PROCESS
JS_CODEC --> DOMAIN
APP -->|runtime observation port| INSPECTOR
INSPECTOR -->|shared CDP and file-location helpers| BROWSER
INSPECTOR --> DOMAIN
+16 -10
View File
@@ -10,7 +10,7 @@
"skill_version": "26"
},
"tools": {
"total": 130,
"total": 131,
"families": [
{
"id": "direct",
@@ -179,7 +179,7 @@
{
"id": "application",
"surface": "application-workflow",
"count": 12,
"count": 13,
"tools": [
"build_reconstruction_obligation_ledger",
"compare_application_versions",
@@ -192,7 +192,8 @@
"recover_javascript_sources",
"trace_application_feature",
"trace_javascript_semantics",
"trace_web_module_imports"
"trace_web_module_imports",
"trace_web_source_location"
]
},
{
@@ -471,7 +472,11 @@
"id": "rea-web-script-export",
"name": "REA captured script export",
"version": "1",
"capabilities": ["export_web_scripts", "trace_web_module_imports"]
"capabilities": [
"export_web_scripts",
"trace_web_module_imports",
"trace_web_source_location"
]
},
{
"id": "unblob",
@@ -561,7 +566,7 @@
}
],
"cli": {
"primary_count": 85,
"primary_count": 86,
"commands": [
"address-to-file-offset",
"analyze",
@@ -644,6 +649,7 @@
"trace-native-ui-action",
"trace-native-values",
"trace-web-module-imports",
"trace-web-source-location",
"uninstall",
"update",
"verify-managed-native-boundaries",
@@ -658,15 +664,15 @@
},
"runtime_catalog": {
"counts": {
"cli_commands": 85,
"mcp_tools": 130,
"cli_commands": 86,
"mcp_tools": 131,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "676a0ae5ff16933e3a9b2db0e5a95fcecf57ac926d18ce8b7a31909e040a9972",
"tools_sha256": "7d34beb174889c873bad4bae9668558fd72ee6e67f610e06127554dae875ea7e",
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
"combined_sha256": "c32a78528d109b2c25582a74c02d25ccb779cb511f6e2477017a2e8805c07bac",
"providers_sha256": "67251a90abf28e1cd9020cf68f579bf2460922d90dc113798a391039279db009"
"combined_sha256": "2ea72f46db5355f3427fd7b2d7f733bacec0d7c334f2e60fe2da78e8323b92ba",
"providers_sha256": "a7d885531ee25d42c51385444faec544d34cd7eba6250a7dc75c4faedd4454a3"
}
}
}
+22
View File
@@ -486,3 +486,25 @@ and compares a finite set of known fixture results. It does not establish
arbitrary recovered-application equivalence. CI installs these prerequisites only
in `.github/workflows/real-javascript-recovery.yml`; the existing `real-browser`
lane uses real Chrome for browser capture and website workflows.
### Captured website source-map lane
`npm run verify:browser:source-maps` checks actual Chromium capture/export and
source-map point tracing through CLI and stdio MCP. It requires absolute
`REA_BROWSER_EXECUTABLE` and `REA_WEB_SOURCE_MAP_COMPILER` pointing to esbuild
0.25.10's `lib/main.js` in a caller-owned isolated installation. Preflight reports
missing prerequisites for this lane. The compiler is used only to generate the
source-owned fixture. No Hopper, Ghidra or application dependency installation
is required.
```bash
REA_BROWSER_EXECUTABLE=/absolute/path/to/chromium \
REA_WEB_SOURCE_MAP_COMPILER=/absolute/path/to/fixture-tools/node_modules/esbuild/lib/main.js \
npm run verify:browser:source-maps
```
`scripts/verify-browser-source-maps.mjs /absolute/path/to/installed/rea.mjs`
checks an installed package after building the verifier dependencies. The separate
conditional `real-web-source-map` CI job supplies Chrome and an isolated pinned
fixture compiler; static/unit checks do not acquire a browser. See
[the source location guide](web-source-location.md) for the verified decoder profile.
@@ -1,5 +1,5 @@
{
"manifest_id": "ecm_df760a39c0bf1144d883fe0b80e22cc93364d23289b9a9a14d0efff72a1031bb",
"manifest_id": "ecm_ce4060c0134f89240982703dc2ed56d182edc7bb35ea40ae9c0efba3373be81a",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -13,7 +13,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "e6e0645e84a6514c7d7d3e42d5b0a424bfb0110951eff41471f901de39dbec17"
"sha256": "deb344fea1b58757709540664e8ab1d3f941f36161a0bb597cd32f9cce53d0ee"
}
],
"claims": [
+97
View File
@@ -0,0 +1,97 @@
# Captured website source locations
`trace_web_source_location` maps one selected position in a script retained by
`export_web_scripts` through one explicitly paired local source map. The shared
CLI/MCP workflow verifies actual manifest, script and map byte identities and
returns the original position and embedded source inline. It acquires no browser,
fetches no URL and executes no application code.
```bash
rea trace-web-source-location \
/analysis/scripts/manifest.json 0 \
/analysis/app.js.map 'https://example.test/maps/app.js.map?v=7#context' \
1 25 --json
```
The six positional arguments select the manifest, zero-based script index,
absolute map path, absolute map URL context, one-based generated line and
zero-based UTF-16 column. The MCP arguments are:
```json
{
"manifest_path": "/analysis/scripts/manifest.json",
"script_index": 0,
"source_map": {
"path": "/analysis/app.js.map",
"url": "https://example.test/maps/app.js.map?v=7#context"
},
"generated_position": { "line": 1, "column": 25 }
}
```
The URL supplies the context for resolving `sources` and `sourceRoot`; it is not
an instruction to fetch. Absolute virtual or file contexts can be selected as
well. The map/script association is caller-selected: a declaration or local map
does not prove deployment authenticity or execution. Use
[import tracing](website-module-trace.md) to inspect outgoing native module edges.
## Meaning of the evidence
Lookup follows the greatest generated position less than or equal to the
selected point, across lines, and returns **every** segment at that position.
It includes generated-only mappings. This follows
[ECMA-426 GetOriginalPositions](https://tc39.es/ecma426/#sec-getoriginalpositions),
with one-based lines in REA's public contract. A point result does not establish
coverage of a source range, execution or UI causality.
Mapped entries preserve the raw source name/root, section path, section-local
source index and flattened index. Duplicate URLs remain distinct, including
entries with different embedded contents. Missing source names retain a null
resolved URL. Original coordinates outside embedded text retain the reported
coordinates and a null offset.
Embedded content hashes identify the UTF-8 encoding of the decoded
`sourcesContent` string, including its line endings. They do not identify an
independently captured original file. Raw map text is retained in Evidence's
`raw_result`, including unknown extensions; these extensions are not interpreted.
Capture context and source Evidence links accompany independently verified
selected file identities.
## Adapter and resource boundaries
The adapter uses the locked upstream
[`@jridgewell/trace-mapping` 0.3.31](https://github.com/jridgewell/trace-mapping)
without modifying its code. It validates map/VLQ structure before allocation,
normalizes nested indexed offsets before upstream flattening, checks section
boundaries and indexes, and retains raw declaration identity. It supports regular
maps and inline indexed sections. External section URLs return an actionable
unsupported-profile error. This is a defined decoder profile, not a claim to
implement every current source-map extension.
One owned Node subprocess decodes the map. Its private request directory is
removed before success is returned. Only this child's inherited `NODE_OPTIONS`
is removed; explicit V8 flags set a 192 MiB old-generation and 8 MiB semi-space
budget, one V8 worker and a 20-second deadline. Reported actual V8 heap limits
are checked. These are heap limits, not an aggregate process RSS limit. Cleanup
may extend the deadline and an unconfirmed cleanup overrides success with
`cleanup_incomplete` and the owning operation/resources.
Complete evidence is bounded by a 4 MiB map, 32 MiB codec reply, 262144 decoded
rows/segments and 64 nested section levels. Exceeding a budget returns an explicit
failure, without partial results. The existing 32 MiB manifest and 16 MiB selected
script boundaries also apply. No global toolchain or browser configuration changes
are made.
## Verification
Focused tests cover duplicate mappings, indexed offsets/overlap/local indexes,
32-bit VLQ overflow, nullable content, UTF-16/CRLF positions, immutable artifact
identity, cancellation, late output closure, cleanup failures, SDK schemas and
public CLI behavior. The actual codec process is exercised on the test host.
The optional `verify:browser:source-maps` lane compiles a source-owned TypeScript
fixture with isolated esbuild 0.25.10, serves it to real Chromium, captures and
exports the actual response, then checks the original compiler point and exact
retained content through CLI and stdio MCP. It can target an installed package.
Only this lane requires Chromium and the fixture compiler. Linux x64 is the real
workflow covered by the dedicated CI job; other host workflows remain unverified.
+2 -1
View File
@@ -180,7 +180,8 @@
"verify:browser:modules": "npm run build:cached && node scripts/verify-browser-modules.mjs",
"verify:test-discovery": "node scripts/verify-test-discovery.mjs",
"verify:javascript:recovery": "node scripts/verify/javascript/recovery.mjs",
"verify:module-boundaries": "node scripts/verify-module-boundaries.mjs"
"verify:module-boundaries": "node scripts/verify-module-boundaries.mjs",
"verify:browser:source-maps": "npm run build:cached && node scripts/verify-browser-source-maps.mjs"
},
"lint-staged": {
"*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}": [
@@ -0,0 +1,77 @@
import { createServer } from "node:http";
import { pathToFileURL } from "node:url";
import { access } from "node:fs/promises";
import { isAbsolute } from "node:path";
/** Generate a real compiler map and serve only source-owned fixture code. */
export async function startBrowserSourceMapSite() {
const command = process.env.REA_WEB_SOURCE_MAP_COMPILER;
if (!command || !isAbsolute(command))
throw new Error(
"verify:browser:source-maps requires absolute REA_WEB_SOURCE_MAP_COMPILER pointing to isolated esbuild 0.25.10 lib/main.js",
);
await access(command);
const compiler = await import(pathToFileURL(command).href);
if (compiler.version !== "0.25.10")
throw new Error(`Expected esbuild 0.25.10; observed ${compiler.version}`);
const original =
'// 🎋 UTF-16 and CRLF fixture\r\nfunction boot(): void {\r\n const marker: string = "source-map proof";\r\n document.body.dataset.ready = marker;\r\n}\r\nboot();\r\n';
const compiled = await compiler
.transform(original, {
loader: "ts",
sourcefile: "../src/fixture.ts?v=7#original",
sourcemap: "external",
minify: true,
format: "iife",
target: "es2022",
})
.finally(() => compiler.stop());
const token = '"source-map proof"';
const generatedOffset = compiled.code.indexOf(token);
const originalOffset = original.indexOf(token);
if (generatedOffset < 0 || originalOffset < 0)
throw new Error("Compiler fixture marker missing");
const position = textPosition(compiled.code, generatedOffset);
const originalPosition = textPosition(original, originalOffset);
let mapRequests = 0;
const server = createServer((request, response) => {
if (request.url === "/")
return response
.writeHead(200, { "Content-Type": "text/html" })
.end('<body><script src="/app.js?build=7"></script></body>');
if (request.url === "/app.js?build=7")
return response
.writeHead(200, { "Content-Type": "text/javascript" })
.end(compiled.code);
if (request.url?.startsWith("/maps/app.js.map")) {
mapRequests += 1;
return response
.writeHead(200, { "Content-Type": "application/json" })
.end(compiled.map);
}
response.writeHead(404).end("Unselected");
});
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (!address || typeof address === "string")
throw new Error("Source map fixture has no TCP address");
return {
origin: `http://127.0.0.1:${address.port}`,
original,
generated: compiled.code,
map: compiled.map,
position,
originalPosition,
originalOffset,
mapRequests: () => mapRequests,
compiler: { id: "esbuild", version: compiler.version },
close: () =>
new Promise((resolve, reject) =>
server.close((error) => (error ? reject(error) : resolve())),
),
};
}
const textPosition = (text, offset) => {
const preceding = text.slice(0, offset).split(/\r\n?|\n|\u2028|\u2029/u);
return { line: preceding.length, column: preceding.at(-1).length };
};
+193
View File
@@ -0,0 +1,193 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseEvidence } from "../../dist/domain/evidence.js";
import { webSourceLocationResultSchema } from "../../dist/domain/webSourceLocation.js";
import { startBrowserSourceMapSite } from "../fixtures/browser-source-map-site.mjs";
import { mcpTextValue } from "./mcp-verifier-results.mjs";
/** Verify real Chrome capture → export → original compiler point through packaged CLI/MCP. */
export async function verifyBrowserSourceMap(
executable,
entrypoint = fileURLToPath(new URL("../rea.mjs", import.meta.url)),
) {
const site = await startBrowserSourceMapSite();
const root = await mkdtemp(join(tmpdir(), "rea-browser-source-map-"));
const env = {
...process.env,
REA_LOG_LEVEL: "silent",
HOPPER_LAUNCHER_PATH: "/rea-unconfigured-provider/hopper",
};
const cli = async (...args) => {
const { stdout } = await promisify(execFile)(
process.execPath,
[entrypoint, ...args, "--json"],
{ env, timeout: 45000, maxBuffer: 16 * 1024 * 1024 },
);
return parseEvidence(JSON.parse(stdout));
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env,
stderr: "pipe",
});
const client = new Client({ name: "source-map-real-e2e", version: "1" });
try {
const capture = await cli(
"capture-browser-scenario",
JSON.stringify({
browser: { mode: "launch", executable_path: executable },
start_url: { url: site.origin },
actions: [
{
step_id: "ready",
action: "wait_for",
locator: {
kind: "css",
selector: '[data-ready="source-map proof"]',
},
state: "visible",
timeout_ms: 10000,
},
],
capture: { network: { response_body: true } },
}),
);
const capturePath = join(root, "capture.json");
await writeFile(capturePath, JSON.stringify(capture));
const exported = await cli(
"export-web-scripts",
capturePath,
join(root, "export"),
);
const manifest = exported.normalized_result;
const manifestBytes = await readFile(manifest.manifest.path);
const identity = {
sha256: sha(manifestBytes),
bytes: manifestBytes.length,
};
const scriptIndex = manifest.scripts.findIndex(
(script) => script.url === `${site.origin}/app.js?build=7`,
);
assert.ok(
scriptIndex >= 0,
"Actual browser capture omitted compiled source",
);
const mapPath = join(root, "app.js.map");
await writeFile(mapPath, site.map);
const before = site.mapRequests();
await client.connect(transport);
let expected;
for (const context of [
site.origin,
site.origin.replace("://", "://selected-user:selected-value@"),
]) {
const input = {
manifest_path: manifest.manifest.path,
script_index: scriptIndex,
source_map: {
path: mapPath,
url: `${context}/maps/app.js.map?v=7#context`,
},
generated_position: site.position,
};
expected = await assertPublicPair(cli, client, {
site,
input,
capture,
identity,
});
}
assert.equal(
site.mapRequests(),
before,
"Point tracing fetched the selected map URL",
);
return {
compiler: site.compiler,
generated_sha256: sha(site.generated),
source_map_sha256: sha(site.map),
original_sha256: sha(site.original),
public_cli: true,
public_stdio_mcp: true,
actual_chrome_capture: true,
source_map_refetch: false,
original_point: site.originalPosition,
codec: expected.engine,
runtime: expected.runtime,
selected_inert_userinfo_context_preserved: true,
public_point_cases: 4,
};
} finally {
await client.close();
await transport.close();
await site.close();
await rm(root, { recursive: true, force: true });
}
}
function assertPoint(evidence, { site, input, capture, identity }) {
const result = webSourceLocationResultSchema.parse(
evidence.normalized_result,
);
assert.equal(result.manifest.sha256, identity.sha256);
assert.equal(result.manifest.bytes, identity.bytes);
assert.equal(result.source.bytes, Buffer.byteLength(site.generated));
assert.equal(result.source.sha256, sha(site.generated));
assert.equal(result.source_map.sha256, sha(site.map));
assert.equal(result.source_map.url, input.source_map.url);
assert.equal(result.manifest.source_evidence_id, capture.evidence_id);
assert.equal(result.execution, "unknown");
assert.equal(result.source_authenticity, "unknown");
assert.equal(result.matches.length, 1);
const match = result.matches[0];
assert.equal(match.state, "mapped");
assert.deepEqual(match.original_position, {
...site.originalPosition,
offset: site.originalOffset,
content_position: "verified-in-embedded-text",
});
assert.equal(
match.resolved_url,
new URL("../src/fixture.ts?v=7#original", input.source_map.url).href,
);
assert.equal(match.content.state, "embedded");
assert.equal(match.content.text, site.original);
assert.equal(match.content.utf8_sha256, sha(site.original));
assert.ok(result.runtime.v8_heap_limit_bytes <= 256 * 1024 * 1024);
assert.deepEqual(evidence.raw_result, { source_map_text: site.map });
return result;
}
const sha = (text) => createHash("sha256").update(text).digest("hex");
async function assertPublicPair(cli, client, oracle) {
const { input } = oracle;
const evidence = await cli(
"trace-web-source-location",
input.manifest_path,
String(input.script_index),
input.source_map.path,
input.source_map.url,
String(input.generated_position.line),
String(input.generated_position.column),
);
const expected = assertPoint(evidence, oracle);
const response = await client.callTool({
name: "trace_web_source_location",
arguments: input,
});
assert.notEqual(response.isError, true, mcpTextValue(response));
const mcpEvidence = parseEvidence(
JSON.parse(mcpTextValue(response)).evidence,
);
assert.deepEqual(assertPoint(mcpEvidence, oracle), expected);
assert.equal(evidence.parameters.source_map.url, input.source_map.url);
return expected;
}
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env node
import { access } from "node:fs/promises";
import { isAbsolute } from "node:path";
import { verifyBrowserSourceMap } from "./lib/browser-source-map-e2e.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
const executable = process.env.REA_BROWSER_EXECUTABLE;
if (!executable || !isAbsolute(executable))
throw new Error(
"verify:browser:source-maps requires absolute REA_BROWSER_EXECUTABLE pointing to caller-supplied Chromium",
);
await access(executable);
const run = createVerifierRun();
const proof = await verifyBrowserSourceMap(executable, process.argv[2]);
process.stdout.write(
`${JSON.stringify({ ...proof, verifier_run: await completeVerifierRun(run), verified: true })}\n`,
);
+2 -2
View File
@@ -3,8 +3,8 @@ name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "26"
tool_count: 130
catalog_digest: "c32a78528d109b2c25582a74c02d25ccb779cb511f6e2477017a2e8805c07bac"
tool_count: 131
catalog_digest: "2ea72f46db5355f3427fd7b2d7f733bacec0d7c334f2e60fe2da78e8323b92ba"
---
# REA
@@ -0,0 +1,48 @@
import { createHash } from "node:crypto";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../domain/result.js";
import type {
SelectedWebScriptArtifacts,
SelectedWebScriptInput,
} from "../domain/webScriptArtifacts.js";
import type { ExportedWebScript } from "../domain/webScriptExport.js";
/** Bind actual source text to the selected manifest entry before analysis. */
export const validateSelectedWebScript = (
input: SelectedWebScriptInput,
data: SelectedWebScriptArtifacts,
operation: string,
): Result<ExportedWebScript, AnalysisError> => {
const selected = data.manifest.scripts[input.script_index];
if (
selected === undefined ||
selected.content.state !== "exported" ||
selected.content.sha256 !== data.sourceFile.sha256 ||
selected.content.bytes !== data.sourceFile.bytes ||
Buffer.byteLength(data.source) !== data.sourceFile.bytes ||
createHash("sha256").update(data.source, "utf8").digest("hex") !==
data.sourceFile.sha256 ||
data.manifestFile.path !== input.manifest_path
)
return err(
new AnalysisOutputError(
operation,
"Artifact port changed the selected script/source identity.",
),
);
return ok(selected);
};
/** Carry reported capture context alongside independently verified manifest bytes. */
export const capturedWebManifestIdentity = (
data: SelectedWebScriptArtifacts,
) => ({
...data.manifestFile,
reported_output_directory: data.manifest.output_directory,
capture_sha256: data.manifest.capture_sha256,
capture_path: data.manifest.capture_path,
capture_kind: data.manifest.capture_kind,
capture_completeness: data.manifest.capture_completeness,
source_evidence_id: data.manifest.source_evidence_id,
});
+2 -7
View File
@@ -6,18 +6,13 @@ import type {
WebModuleResolution,
WebModuleTraceInput,
} from "../domain/webModuleTrace.js";
import type { webScriptExportManifestSchema } from "../domain/webScriptExport.js";
import type { z } from "zod";
import type { SelectedWebScriptArtifacts } from "../domain/webScriptArtifacts.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
type JsonObject = Record<string, JsonValue>;
/** Verified selected source and capture metadata; provider protocols stay outside. */
export interface WebModuleArtifacts {
readonly manifest: z.output<typeof webScriptExportManifestSchema>;
readonly manifestFile: WebModuleFile;
readonly sourceFile: WebModuleFile;
readonly source: string;
export interface WebModuleArtifacts extends SelectedWebScriptArtifacts {
readonly importMap: {
readonly file: WebModuleFile;
readonly baseUrl: string;
+10 -33
View File
@@ -1,5 +1,4 @@
import { isAbsolute } from "node:path";
import { createHash } from "node:crypto";
import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
@@ -24,7 +23,11 @@ import {
type WebModuleResolution,
type WebModuleTraceResult,
} from "../domain/webModuleTrace.js";
import { webScriptExportManifestSchema } from "../domain/webScriptExport.js";
import { selectedWebScriptArtifactsSchema } from "../domain/webScriptArtifacts.js";
import {
capturedWebManifestIdentity,
validateSelectedWebScript,
} from "./ValidateSelectedWebScript.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
import type {
WebModuleArtifactPort,
@@ -33,11 +36,7 @@ import type {
} from "./WebModulePorts.js";
const OPERATION = "trace_web_module_imports";
const artifactsSchema = z.strictObject({
manifest: webScriptExportManifestSchema,
manifestFile: webModuleFileSchema,
sourceFile: webModuleFileSchema,
source: z.string(),
const artifactsSchema = selectedWebScriptArtifactsSchema.extend({
importMap: z
.strictObject({
file: webModuleFileSchema,
@@ -125,23 +124,9 @@ export class WebModuleTraceService {
"Artifact port changed the selected import-map identity/context.",
),
);
const selected = data.manifest.scripts[input.script_index];
if (
selected === undefined ||
selected.content.state !== "exported" ||
selected.content.sha256 !== data.sourceFile.sha256 ||
selected.content.bytes !== data.sourceFile.bytes ||
Buffer.byteLength(data.source) !== data.sourceFile.bytes ||
createHash("sha256").update(data.source, "utf8").digest("hex") !==
data.sourceFile.sha256 ||
data.manifestFile.path !== input.manifest_path
)
return err(
new AnalysisOutputError(
OPERATION,
"Artifact port did not bind the selected source and manifest identities.",
),
);
const bound = validateSelectedWebScript(input, data, OPERATION);
if (!bound.ok) return bound;
const selected = bound.value;
const importerUrl = input.importer_url ?? selected.url;
const importerAdmission = admitContext(importerUrl, ["importer_url"]);
if (!importerAdmission.ok) return importerAdmission;
@@ -215,15 +200,7 @@ export class WebModuleTraceService {
});
}
const normalized = webModuleTraceResultSchema.safeParse({
manifest: {
...data.manifestFile,
reported_output_directory: data.manifest.output_directory,
capture_sha256: data.manifest.capture_sha256,
capture_path: data.manifest.capture_path,
capture_kind: data.manifest.capture_kind,
capture_completeness: data.manifest.capture_completeness,
source_evidence_id: data.manifest.source_evidence_id,
},
manifest: capturedWebManifestIdentity(data),
source: {
...data.sourceFile,
script_index: input.script_index,
+15
View File
@@ -0,0 +1,15 @@
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { Result } from "../domain/result.js";
import type {
SelectedWebScriptArtifacts,
SelectedWebScriptInput,
} from "../domain/webScriptArtifacts.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
/** Read the caller-selected capture export and verify the selected source identity. */
export interface WebScriptArtifactPort {
load(
input: SelectedWebScriptInput,
options?: ExecutionOptions,
): Promise<Result<SelectedWebScriptArtifacts, AnalysisError>>;
}
+40
View File
@@ -0,0 +1,40 @@
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { Result } from "../domain/result.js";
import type {
WebSourceLocationInput,
WebSourceMapReport,
} from "../domain/webSourceLocation.js";
import type { SelectedWebScriptArtifacts } from "../domain/webScriptArtifacts.js";
import type { z } from "zod";
import type { webArtifactFileSchema } from "../domain/webScriptArtifacts.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
/** Actual selected source and local map bytes; no provider URL fetching. */
export interface WebSourceLocationArtifacts extends SelectedWebScriptArtifacts {
readonly sourceMap: {
readonly file: z.output<typeof webArtifactFileSchema>;
readonly url: string;
readonly text: string;
};
}
/** Replaceable artifact acquisition boundary for a caller-selected pairing. */
export interface WebSourceLocationArtifactPort {
load(
input: WebSourceLocationInput,
options?: ExecutionOptions,
): Promise<Result<WebSourceLocationArtifacts, AnalysisError>>;
}
/** Replaceable codec receives map data and one explicit generated point. */
export interface WebSourceMapPort {
trace(
input: {
readonly text: string;
readonly path: string;
readonly url: string;
readonly position: WebSourceLocationInput["generated_position"];
},
options?: ExecutionOptions,
): Promise<Result<WebSourceMapReport, AnalysisError>>;
}
@@ -0,0 +1,151 @@
import { expect, it } from "vitest";
import {
webSourceLocationArgs as args,
webSourceLocationFixture,
webSourceMapDecoderFixture as decoder,
} from "../../tests/fixtures/webSourceLocation.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ok } from "../domain/result.js";
import { webSourceLocationResultSchema } from "../domain/webSourceLocation.js";
import { WebSourceLocationService } from "./WebSourceLocationService.js";
import type { WebSourceMapPort } from "./WebSourceLocationPorts.js";
const service = (data = webSourceLocationFixture(), port = decoder) =>
new WebSourceLocationService({ load: () => Promise.resolve(ok(data)) }, port);
it("joins byte identity and historical capture context while keeping selected authenticity/execution unknown", async () => {
const data = webSourceLocationFixture();
data.manifest.source_evidence_id = `ev_${"a".repeat(64)}`;
const response = await service(data).trace(args);
if (!response.ok) throw response.error;
const result = webSourceLocationResultSchema.parse(
response.value.normalized_result,
);
expect(result).toMatchObject({
source: { sha256: data.sourceFile.sha256 },
source_map: {
association: "caller-selected",
sha256: data.sourceMap.file.sha256,
},
matches: [{ state: "mapped", content: { text: "original" } }],
execution: "unknown",
source_authenticity: "unknown",
});
expect(response.value.raw_result).toEqual({
source_map_text: data.sourceMap.text,
});
});
it.each([
{ ...args, manifest_path: "relative.json" },
{ ...args, script_index: -1 },
{ ...args, source_map: { ...args.source_map, path: "relative.map" } },
{ ...args, source_map: { ...args.source_map, url: "relative.map" } },
{ ...args, generated_position: { line: 0, column: 0 } },
])(
"rejects invalid selected context before acquiring artifacts: %j",
async (input) => {
const response = await new WebSourceLocationService(
{
load: () => {
throw new Error("must not acquire");
},
},
decoder,
).trace(input);
if (response.ok) throw new Error("expected invalid input");
expect(projectAnalysisError(response.error).code).toBe("invalid_request");
},
);
it.each([
"file:///analysis/app.map",
"webpack:///bundle/app.map",
"https://selected-user:[email protected]/maps/app.map",
])(
"retains an absolute inert map context without fetching: %s",
async (url) => {
const fixture = webSourceLocationFixture();
const data = { ...fixture, sourceMap: { ...fixture.sourceMap, url } };
const response = await service(data).trace({
...args,
source_map: { ...args.source_map, url },
});
if (!response.ok) throw response.error;
expect(
webSourceLocationResultSchema.parse(response.value.normalized_result)
.url_context,
).toBe(url);
},
);
it.each(["source", "map", "map-path", "map-url"])(
"rejects changed artifact identity before decoding: %s",
async (variant) => {
const fixture = webSourceLocationFixture();
const data = {
...fixture,
source: variant === "source" ? "different" : fixture.source,
sourceMap: { ...fixture.sourceMap },
};
if (variant === "map") data.sourceMap.text += " ";
if (variant === "map-path") data.sourceMap.file.path = "/other.map";
if (variant === "map-url") data.sourceMap.url = "https://other.test/";
const response = await service(data, {
trace: () => {
throw new Error("must not decode");
},
}).trace(args);
if (response.ok) throw new Error("expected changed identity failure");
expect(response.error._tag).toBe("AnalysisOutputError");
},
);
it.each(["digest", "url", "point"])(
"rejects changed codec identity: %s",
async (variant) => {
const port: WebSourceMapPort = {
trace: async (input) => {
const decoded = await decoder.trace(input);
if (!decoded.ok) return decoded;
const report = decoded.value;
if (variant === "digest") report.source_map_sha256 = "f".repeat(64);
if (variant === "url") report.url_context = "https://other.test/";
if (variant === "point") report.lookup.requested.column += 1;
return Promise.resolve(ok(report));
},
};
const response = await service(undefined, port).trace(args);
if (response.ok) throw new Error("expected changed codec identity failure");
expect(response.error._tag).toBe("AnalysisOutputError");
},
);
it("rejects a generated point outside retained text before decoding", async () => {
const response = await service(undefined, {
trace: () => {
throw new Error("must not decode");
},
}).trace({ ...args, generated_position: { line: 2, column: 0 } });
if (response.ok) throw new Error("expected invalid point");
expect(projectAnalysisError(response.error)).toMatchObject({
code: "invalid_request",
details: { issues: [{ message: expect.stringContaining("outside") }] },
});
});
it("honors cancellation both before acquisition and across decoder completion", async () => {
const first = new AbortController();
first.abort();
const response = await new WebSourceLocationService(
{
load: () => {
throw new Error("must not acquire");
},
},
decoder,
).trace(args, { signal: first.signal });
expect(response.ok).toBe(false);
const controller = new AbortController();
const late = await service(undefined, {
trace: (input) => {
controller.abort();
return decoder.trace(input);
},
}).trace(args, { signal: controller.signal });
if (late.ok) throw new Error("expected cancellation");
expect(late.error._tag).toBe("AnalysisCancelledError");
});
+239
View File
@@ -0,0 +1,239 @@
import { createHash } from "node:crypto";
import { isAbsolute } from "node:path";
import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import { createEvidence, type Evidence } from "../domain/evidence.js";
import { projectInputIssues } from "../domain/inputIssueProjection.js";
import { jsonObjectSchema } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import {
selectedWebScriptArtifactsSchema,
webArtifactFileSchema,
} from "../domain/webScriptArtifacts.js";
import {
WEB_SOURCE_MAP_LIMITS,
webSourceLocationInputSchema,
webSourceLocationResultSchema,
webSourceMapReportSchema,
type WebSourceLocationInput,
type WebSourceMapReport,
} from "../domain/webSourceLocation.js";
import { webSourceOffset } from "../domain/webSourcePosition.js";
import type { ExecutionOptions } from "./AnalysisProvider.js";
import {
capturedWebManifestIdentity,
validateSelectedWebScript,
} from "./ValidateSelectedWebScript.js";
import type {
WebSourceLocationArtifactPort,
WebSourceMapPort,
} from "./WebSourceLocationPorts.js";
const OPERATION = "trace_web_source_location";
const artifactsSchema = selectedWebScriptArtifactsSchema.extend({
sourceMap: z.strictObject({
file: webArtifactFileSchema,
url: z.string().min(1),
text: z.string(),
}),
});
const LIMITATIONS = [
"The source/map pairing and source-map URL context are caller-selected. Deployment authenticity and execution remain unknown.",
"Capture metadata and its Evidence reference are historical context; this trace independently verifies only the selected manifest, script and map bytes.",
"Source names and sourceRoot are reported declarations. Resolved URLs are derived by the pinned upstream codec; no original source is fetched or executed.",
"Embedded source hashes identify the UTF-8 encoding of decoded sourcesContent text, not an independently retained original file. Missing or out-of-range original content positions remain explicit.",
"Lookup returns all segments at the greatest generated position not greater than the selected point, including earlier lines and generated-only segments. A mapped point does not establish an executed range or UI causality.",
"Regular maps and inline indexed sections are supported. External section URLs are unsupported; unknown extensions are retained in raw map text and are not interpreted.",
"Complete evidence has a 4 MiB map, 32 MiB reply, 262144 decoded rows/segments and 64-level section budget. The owned codec has a 20-second deadline, 192 MiB old-generation heap and one V8 worker; cleanup may extend the deadline. V8 heap limits do not establish aggregate RSS limits.",
];
/** Join verified captured bytes with an explicitly selected source-map point. */
export class WebSourceLocationService {
constructor(
readonly artifacts: WebSourceLocationArtifactPort,
readonly decoder: WebSourceMapPort,
) {}
/** Shared CLI/MCP workflow preserves raw declarations and every equal-position match. */
async trace(
rawInput: unknown,
options?: ExecutionOptions,
): Promise<Result<Evidence, AnalysisError>> {
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION));
const parsed = webSourceLocationInputSchema.safeParse(rawInput);
if (!parsed.success)
return err(
new AnalysisInputError(
OPERATION,
{ cause: parsed.error },
projectInputIssues(parsed.error.issues, rawInput),
),
);
const input = parsed.data;
const admitted = admitContext(input);
if (!admitted.ok) return admitted;
const loaded = await this.artifacts.load(input, options);
if (!loaded.ok) return loaded;
const verified = artifactsSchema.safeParse(loaded.value);
if (!verified.success)
return invalidOutput(
"Artifact port returned malformed source/map evidence.",
);
const data = verified.data;
const bound = validateSelectedWebScript(input, data, OPERATION);
if (!bound.ok) return bound;
const map = data.sourceMap;
const pairing = verifyPairing(input, data);
if (!pairing.ok) return pairing;
const generatedOffset = pairing.value;
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION));
const traced = await this.decoder.trace(
{
text: map.text,
path: map.file.path,
url: map.url,
position: input.generated_position,
},
options,
);
if (!traced.ok) return traced;
const report = verifyReport(input, map.file.sha256, traced.value);
if (!report.ok) return report;
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION));
const normalized = webSourceLocationResultSchema.safeParse({
...report.value,
manifest: capturedWebManifestIdentity(data),
source: {
...data.sourceFile,
script_index: input.script_index,
script: bound.value,
},
source_map: { ...map.file, url: map.url, association: "caller-selected" },
generated_offset: generatedOffset,
source_authenticity: "unknown",
execution: "unknown",
limitations: LIMITATIONS,
});
if (!normalized.success) return invalidOutput(normalized.error.message);
return ok(
createEvidence(
{
path: data.sourceFile.path,
sha256: data.sourceFile.sha256,
format: "file",
},
{
id: "rea-web-source-location",
name: "REA captured source-map point trace",
version: "1",
},
{
operation: OPERATION,
parameters: jsonObjectSchema.parse(input),
result: normalized.data,
rawResult: { source_map_text: map.text },
confidence: "derived",
authority: "historical-reference",
limitations: LIMITATIONS,
locations: [
data.sourceFile.path,
data.manifestFile.path,
map.file.path,
].map((path) => ({ kind: "artifact-path", path })),
evidenceLinks:
data.manifest.source_evidence_id === null
? []
: [data.manifest.source_evidence_id],
},
),
);
}
}
const invalidOutput = (message: string): Result<never, AnalysisError> =>
err(new AnalysisOutputError(OPERATION, message));
const invalid = (
path: readonly string[],
message: string,
): Result<never, AnalysisError> =>
err(
new AnalysisInputError(OPERATION, undefined, [
{ path, reason: "invalid_format", message },
]),
);
const admitContext = (
input: WebSourceLocationInput,
): Result<null, AnalysisError> => {
for (const [path, value] of [
["manifest_path", input.manifest_path],
["source_map.path", input.source_map.path],
] as const)
if (!isAbsolute(value))
return invalid(
path.split("."),
"Expected an absolute filesystem path on this host.",
);
if (!URL.canParse(input.source_map.url))
return invalid(
["source_map", "url"],
"Expected an absolute source-map URL context. Select its deployment context explicitly; no URL is fetched.",
);
return ok(null);
};
const verifyPairing = (
input: WebSourceLocationInput,
data: z.output<typeof artifactsSchema>,
): Result<number, AnalysisError> => {
const map = data.sourceMap;
if (
map.file.path !== input.source_map.path ||
map.url !== input.source_map.url ||
Buffer.byteLength(map.text) !== map.file.bytes ||
createHash("sha256").update(map.text).digest("hex") !== map.file.sha256 ||
map.file.bytes > WEB_SOURCE_MAP_LIMITS.mapBytes
)
return invalidOutput(
"Artifact port changed the selected source-map byte identity or URL context.",
);
const generatedOffset = webSourceOffset(
data.source,
input.generated_position,
);
if (generatedOffset === undefined)
return invalid(
["generated_position"],
"Selected generated position is outside the retained script text (one-based lines; zero-based UTF-16 columns).",
);
return ok(generatedOffset);
};
const verifyReport = (
input: WebSourceLocationInput,
sha256: string,
value: unknown,
): Result<WebSourceMapReport, AnalysisError> => {
const report = webSourceMapReportSchema.safeParse(value);
if (!report.success)
return invalidOutput(
"Codec port returned malformed source-map point evidence.",
);
if (
report.data.source_map_sha256 !== sha256 ||
report.data.url_context !== input.source_map.url ||
report.data.lookup.requested.line !== input.generated_position.line ||
report.data.lookup.requested.column !== input.generated_position.column
)
return invalidOutput(
"Codec port changed the selected map, URL context or generated point.",
);
return ok(report.data);
};
@@ -0,0 +1,139 @@
import { lstat, realpath } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, sep } from "node:path";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
import { err, ok } from "../../domain/result.js";
import type { SelectedWebScriptInput } from "../../domain/webScriptArtifacts.js";
import { webScriptExportManifestSchema } from "../../domain/webScriptExport.js";
import {
decodeWebArtifact,
readWebArtifactJson,
webArtifactReadError,
WebArtifactFormatFailure,
type WebArtifactReadContext,
} from "./WebArtifactReads.js";
/** Read one digest-verified captured source under the manifest's current location. */
export class LocalWebScriptArtifacts implements WebScriptArtifactPort {
constructor(readonly operation: WebArtifactReadContext["operation"]) {}
/** Read the selected source; relocated exports retain their reported original root. */
async load(input: SelectedWebScriptInput, options?: ExecutionOptions) {
let field: readonly (string | number)[] = ["manifest_path"];
let targetPath = input.manifest_path;
try {
const manifestBytes = await readWebArtifactJson(
input.manifest_path,
32 * 1024 * 1024,
options?.signal,
);
const manifest = webScriptExportManifestSchema.parse(manifestBytes.value);
field = ["script_index"];
const selected = manifest.scripts[input.script_index];
if (selected === undefined)
throw new WebArtifactFormatFailure(
`No script at index ${String(input.script_index)}; manifest contains ${String(manifest.scripts.length)} scripts.`,
);
if (selected.content.state !== "exported")
return err(
new AnalysisCapabilityUnavailableError(
"web-module-artifacts",
this.operation,
"selected_source_unavailable",
{
userMessage: `Selected script ${String(input.script_index)} has unavailable source bytes (${selected.content.reason}): ${selected.content.message} Capture source bytes before tracing this script.`,
},
),
);
const portable = normalizeArtifactPath(selected.content.relative_path);
if (portable !== selected.content.relative_path)
throw new ArtifactReaderFailure(
"path",
`Selected source path changes during normalization: ${selected.content.relative_path}`,
);
const root = dirname(input.manifest_path);
const sourcePath = join(root, "files", portable);
targetPath = sourcePath;
await assertContained(root, sourcePath);
const sourceBytes = await readStableArtifact(
sourcePath,
16 * 1024 * 1024,
options?.signal,
);
await assertContained(root, sourcePath);
if (
sourceBytes.sha256 !== selected.content.sha256 ||
sourceBytes.bytes.length !== selected.content.bytes
)
throw new ArtifactReaderFailure(
"integrity",
`Selected source identity mismatch: ${sourcePath}; expected sha256=${selected.content.sha256}, bytes=${String(selected.content.bytes)}; observed sha256=${sourceBytes.sha256}, bytes=${String(sourceBytes.bytes.length)}.`,
undefined,
{
logicalPath: selected.content.relative_path,
declaredSha256: selected.content.sha256,
calculatedSha256: sourceBytes.sha256,
unpacked: false,
},
);
const source = decodeWebArtifact(sourceBytes.bytes);
return ok({
manifest,
manifestFile: {
path: input.manifest_path,
sha256: manifestBytes.sha256,
bytes: manifestBytes.bytes.length,
},
sourceFile: {
path: sourcePath,
sha256: sourceBytes.sha256,
bytes: sourceBytes.bytes.length,
},
source,
});
} catch (cause: unknown) {
return err(
webArtifactReadError(
cause,
{ operation: this.operation, field, targetPath },
options?.signal,
),
);
}
}
}
const assertContained = async (root: string, path: string): Promise<void> => {
const canonicalRoot = await realpath(root);
const canonicalFile = await realpath(path);
const fromRoot = relative(canonicalRoot, canonicalFile);
if (
fromRoot === ".." ||
fromRoot.startsWith(`..${sep}`) ||
isAbsolute(fromRoot)
)
throw new ArtifactReaderFailure(
"path",
`Selected source escapes its manifest directory: ${path}`,
);
let directory = dirname(path);
while (directory !== root) {
const stat = await lstat(directory);
if (!stat.isDirectory() || stat.isSymbolicLink())
throw new ArtifactReaderFailure(
"path",
`Selected source directory is a symlink or not a directory: ${directory}`,
);
const parent = dirname(directory);
if (parent === directory)
throw new ArtifactReaderFailure(
"path",
"Selected source has no containing manifest directory.",
);
directory = parent;
}
};
+93
View File
@@ -0,0 +1,93 @@
import { z } from "zod";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { safeParseJson } from "../../domain/safeJson.js";
/** The selected artifact and input field owning a reader failure. */
export interface WebArtifactReadContext {
readonly operation: "trace_web_module_imports" | "trace_web_source_location";
readonly field: readonly (string | number)[];
readonly targetPath: string;
}
/** A malformed producer document, distinct from host I/O failures. */
export class WebArtifactFormatFailure extends Error {}
/** Decode exact UTF-8 content, preserving a BOM for digest verification. */
export const decodeWebArtifact = (bytes: Buffer): string => {
try {
return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
bytes,
);
} catch (cause: unknown) {
throw new WebArtifactFormatFailure(
"Selected artifact is not valid UTF-8.",
{ cause },
);
}
};
/** Read one stable JSON artifact with a documented byte budget. */
export const readWebArtifactJson = async (
path: string,
maximumBytes: number,
signal?: AbortSignal,
) => {
const data = await readStableArtifact(path, maximumBytes, signal);
const value = safeParseJson(decodeWebArtifact(data.bytes));
if (!value.ok) throw new WebArtifactFormatFailure(value.error);
return { ...data, value: value.value };
};
/** Preserve artifact format, integrity, OS I/O, and cancellation reasons. */
export const webArtifactReadError = (
cause: unknown,
context: WebArtifactReadContext,
signal?: AbortSignal,
): AnalysisError => {
if (signal?.aborted === true)
return new AnalysisCancelledError(context.operation);
if (cause instanceof ArtifactReaderFailure)
return new ArtifactOperationError(
context.operation,
cause.reason,
cause.details,
cause.message,
);
if (
cause instanceof Error &&
"code" in cause &&
typeof cause.code === "string" &&
"syscall" in cause &&
typeof cause.syscall === "string"
)
return new ArtifactOperationError(
context.operation,
"io",
undefined,
`${context.field.join(".")}: ${cause.code}: ${cause.message}`,
);
if (cause instanceof WebArtifactFormatFailure || cause instanceof z.ZodError)
return new AnalysisInputError(context.operation, { cause }, [
{
path: context.field,
reason: "invalid_format",
message: `${context.targetPath}: ${cause.message}`,
},
]);
return new ProviderAdapterError("captured-web-artifacts", context.operation, {
cause,
diagnostics: {
target_path: context.targetPath,
field: context.field.join("."),
error_message: cause instanceof Error ? cause.message : String(cause),
},
});
};
@@ -0,0 +1,57 @@
import type { WebSourceLocationArtifactPort } from "../../application/WebSourceLocationPorts.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import {
WEB_SOURCE_MAP_LIMITS,
type WebSourceLocationInput,
} from "../../domain/webSourceLocation.js";
import { err, ok } from "../../domain/result.js";
import { LocalWebScriptArtifacts } from "./LocalWebScriptArtifacts.js";
import { decodeWebArtifact, webArtifactReadError } from "./WebArtifactReads.js";
/** Read one verified capture source and the caller-selected map, without fetching. */
export class LocalWebSourceLocationArtifacts implements WebSourceLocationArtifactPort {
constructor(
readonly scripts: WebScriptArtifactPort = new LocalWebScriptArtifacts(
"trace_web_source_location",
),
) {}
/** Preserve actual map identity separately from the selected deployment URL. */
async load(input: WebSourceLocationInput, options?: ExecutionOptions) {
const loaded = await this.scripts.load(input, options);
if (!loaded.ok) return loaded;
try {
const data = await readStableArtifact(
input.source_map.path,
WEB_SOURCE_MAP_LIMITS.mapBytes,
options?.signal,
);
return ok({
...loaded.value,
sourceMap: {
file: {
path: input.source_map.path,
sha256: data.sha256,
bytes: data.bytes.length,
},
url: input.source_map.url,
text: decodeWebArtifact(data.bytes),
},
});
} catch (cause: unknown) {
return err(
webArtifactReadError(
cause,
{
operation: "trace_web_source_location",
field: ["source_map", "path"],
targetPath: input.source_map.path,
},
options?.signal,
),
);
}
}
}
+37 -206
View File
@@ -1,230 +1,61 @@
import { lstat, realpath } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, sep } from "node:path";
import { z } from "zod";
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
import type {
WebModuleArtifactPort,
WebModuleArtifacts,
} from "../../application/WebModulePorts.js";
import type { WebScriptArtifactPort } from "../../application/WebScriptArtifactPort.js";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisCapabilityUnavailableError,
} from "../../domain/analysisErrorCore.js";
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { jsonObjectSchema } from "../../domain/jsonValue.js";
import { safeParseJson } from "../../domain/safeJson.js";
import { err, ok } from "../../domain/result.js";
import type { WebModuleTraceInput } from "../../domain/webModuleTrace.js";
import { webScriptExportManifestSchema } from "../../domain/webScriptExport.js";
import { LocalWebScriptArtifacts } from "../assets/LocalWebScriptArtifacts.js";
import {
readWebArtifactJson,
webArtifactReadError,
} from "../assets/WebArtifactReads.js";
/** Load verified exported source bytes relative to the current manifest location. */
/** Add an explicitly selected import map to a verified captured-script source. */
export class LocalWebModuleArtifacts implements WebModuleArtifactPort {
/** Verify exactly the selected manifest, source and optional local map. */
constructor(
readonly scripts: WebScriptArtifactPort = new LocalWebScriptArtifacts(
"trace_web_module_imports",
),
) {}
/** Read selected source and optional map without website requests. */
async load(input: WebModuleTraceInput, options?: ExecutionOptions) {
let field: readonly (string | number)[] = ["manifest_path"];
let targetPath = input.manifest_path;
const loaded = await this.scripts.load(input, options);
if (!loaded.ok) return loaded;
if (input.import_map === undefined)
return ok({ ...loaded.value, importMap: null });
const selected = input.import_map;
try {
const manifestBytes = await readStableArtifact(
input.manifest_path,
32 * 1024 * 1024,
const data = await readWebArtifactJson(
selected.path,
4 * 1024 * 1024,
options?.signal,
);
const manifest = webScriptExportManifestSchema.parse(
parseJson(manifestBytes.bytes),
);
field = ["script_index"];
const selected = manifest.scripts[input.script_index];
if (selected === undefined)
throw new ModuleArtifactFormatFailure(
`No script at index ${String(input.script_index)}; manifest contains ${String(manifest.scripts.length)} scripts.`,
);
if (selected.content.state !== "exported")
return err(
new AnalysisCapabilityUnavailableError(
"web-module-artifacts",
"trace_web_module_imports",
"selected_source_unavailable",
{
userMessage: `Selected script ${String(input.script_index)} has unavailable source bytes (${selected.content.reason}): ${selected.content.message} Capture source bytes before tracing this script.`,
},
),
);
const portable = normalizeArtifactPath(selected.content.relative_path);
if (portable !== selected.content.relative_path)
throw new ArtifactReaderFailure(
"path",
`Selected source path changes during normalization: ${selected.content.relative_path}`,
);
const root = dirname(input.manifest_path);
const sourcePath = join(root, "files", portable);
targetPath = sourcePath;
await assertContained(root, sourcePath);
const sourceBytes = await readStableArtifact(
sourcePath,
16 * 1024 * 1024,
options?.signal,
);
await assertContained(root, sourcePath);
if (
sourceBytes.sha256 !== selected.content.sha256 ||
sourceBytes.bytes.length !== selected.content.bytes
)
throw new ArtifactReaderFailure(
"integrity",
`Selected source identity mismatch: ${sourcePath}; expected sha256=${selected.content.sha256}, bytes=${String(selected.content.bytes)}; observed sha256=${sourceBytes.sha256}, bytes=${String(sourceBytes.bytes.length)}.`,
undefined,
{
logicalPath: selected.content.relative_path,
declaredSha256: selected.content.sha256,
calculatedSha256: sourceBytes.sha256,
unpacked: false,
},
);
const source = decode(sourceBytes.bytes);
let importMap: WebModuleArtifacts["importMap"] = null;
if (input.import_map !== undefined) {
field = ["import_map", "path"];
targetPath = input.import_map.path;
const bytes = await readStableArtifact(
input.import_map.path,
4 * 1024 * 1024,
options?.signal,
);
importMap = {
file: {
path: input.import_map.path,
sha256: bytes.sha256,
bytes: bytes.bytes.length,
},
baseUrl: input.import_map.base_url,
value: jsonObjectSchema.parse(parseJson(bytes.bytes)),
};
}
return ok({
manifest,
manifestFile: {
path: input.manifest_path,
sha256: manifestBytes.sha256,
bytes: manifestBytes.bytes.length,
const importMap: WebModuleArtifacts["importMap"] = {
file: {
path: selected.path,
sha256: data.sha256,
bytes: data.bytes.length,
},
sourceFile: {
path: sourcePath,
sha256: sourceBytes.sha256,
bytes: sourceBytes.bytes.length,
},
source,
importMap,
});
baseUrl: selected.base_url,
value: jsonObjectSchema.parse(data.value),
};
return ok({ ...loaded.value, importMap });
} catch (cause: unknown) {
if (options?.signal?.aborted === true)
return err(new AnalysisCancelledError("trace_web_module_imports"));
if (cause instanceof ArtifactReaderFailure)
return err(
new ArtifactOperationError(
"trace_web_module_imports",
cause.reason,
cause.details,
cause.message,
),
);
if (
cause instanceof Error &&
"code" in cause &&
typeof cause.code === "string" &&
"syscall" in cause &&
typeof cause.syscall === "string"
)
return err(
new ArtifactOperationError(
"trace_web_module_imports",
"io",
undefined,
`${field.join(".")}: ${cause.code}: ${cause.message}`,
),
);
if (
cause instanceof ModuleArtifactFormatFailure ||
cause instanceof z.ZodError
)
return err(
new AnalysisInputError("trace_web_module_imports", { cause }, [
{
path: field,
reason: "invalid_format",
message: `${targetPath}: ${cause.message}`,
},
]),
);
return err(
new ProviderAdapterError(
"web-module-artifacts",
"trace_web_module_imports",
webArtifactReadError(
cause,
{
cause,
diagnostics: {
target_path: targetPath,
field: field.join("."),
error_message:
cause instanceof Error ? cause.message : String(cause),
},
operation: "trace_web_module_imports",
field: ["import_map", "path"],
targetPath: selected.path,
},
options?.signal,
),
);
}
}
}
class ModuleArtifactFormatFailure extends Error {}
const decode = (bytes: Buffer): string => {
try {
return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
bytes,
);
} catch (cause: unknown) {
throw new ModuleArtifactFormatFailure(
"Selected artifact is not valid UTF-8.",
{ cause },
);
}
};
const parseJson = (bytes: Buffer): unknown => {
const parsed = safeParseJson(decode(bytes));
if (!parsed.ok) throw new ModuleArtifactFormatFailure(parsed.error);
return parsed.value;
};
const assertContained = async (root: string, path: string): Promise<void> => {
const canonicalRoot = await realpath(root);
const canonicalFile = await realpath(path);
const fromRoot = relative(canonicalRoot, canonicalFile);
if (
fromRoot === ".." ||
fromRoot.startsWith(`..${sep}`) ||
isAbsolute(fromRoot)
)
throw new ArtifactReaderFailure(
"path",
`Selected source escapes its manifest directory: ${path}`,
);
let directory = dirname(path);
while (directory !== root) {
const stat = await lstat(directory);
if (!stat.isDirectory() || stat.isSymbolicLink())
throw new ArtifactReaderFailure(
"path",
`Selected source directory is a symlink or not a directory: ${directory}`,
);
const parent = dirname(directory);
if (parent === directory)
throw new ArtifactReaderFailure(
"path",
"Selected source has no containing manifest directory.",
);
directory = parent;
}
};
+2
View File
@@ -14,6 +14,7 @@ import { registerProcessCommands } from "./cli/processCommands.js";
import { registerBrowserCommands } from "./cli/browserCommands.js";
import { registerWebScriptCommands } from "./cli/webScriptCommands.js";
import { registerWebModuleCommands } from "./cli/webModuleCommands.js";
import { registerWebSourceLocationCommands } from "./cli/webSourceLocationCommands.js";
import { registerJavaScriptRecoveryCommands } from "./cli/javascriptRecoveryCommands.js";
import { registerAdvancedBrowserCommands } from "./cli/browserAdvancedCommands.js";
import { registerBrowserScenarioCommands } from "./cliBrowserScenarioCommands.js";
@@ -59,6 +60,7 @@ export const createCli = (
registerBrowserCommands(cli, logger);
registerWebScriptCommands(cli, logger);
registerWebModuleCommands(cli, logger, environment);
registerWebSourceLocationCommands(cli, logger, environment);
registerJavaScriptRecoveryCommands(cli, logger, environment);
registerAdvancedBrowserCommands(cli, logger);
registerBrowserScenarioCommands(cli, logger);
+56
View File
@@ -0,0 +1,56 @@
import { z } from "incur";
import { createWebSourceLocationService } from "../composition/webSourceLocations.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "../logger.js";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
/** Expose the same explicit selected map/point contract through the CLI. */
export const registerWebSourceLocationCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
): void => {
const service = createWebSourceLocationService(environment);
cli.command(CLI_COMMANDS.traceWebSourceLocation, {
description:
"Trace one retained script position through an explicitly selected local source map",
args: z.object({
manifestPath: z
.string()
.describe("Absolute export_web_scripts manifest path"),
scriptIndex: z.coerce
.number()
.int()
.min(0)
.describe("Zero-based manifest script index"),
sourceMapPath: z.string().describe("Absolute local source-map path"),
sourceMapUrl: z
.string()
.describe("Absolute source-map URL context; no fetch"),
line: z.coerce.number().int().min(1).describe("One-based generated line"),
column: z.coerce
.number()
.int()
.min(0)
.describe("Zero-based UTF-16 generated column"),
}),
run: ({ args }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.traceWebSourceLocation, async () => {
const result = await service.trace(
{
manifest_path: args.manifestPath,
script_index: args.scriptIndex,
source_map: { path: args.sourceMapPath, url: args.sourceMapUrl },
generated_position: { line: args.line, column: args.column },
},
{ signal },
);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
});
};
+1
View File
@@ -1,5 +1,6 @@
/** Canonical names used by every one-shot CLI command registration. */
export const CLI_COMMANDS = Object.freeze({
traceWebSourceLocation: "trace-web-source-location",
traceWebModuleImports: "trace-web-module-imports",
analyze: "analyze",
inspect: "inspect",
+12
View File
@@ -0,0 +1,12 @@
import { WebSourceLocationService } from "../application/WebSourceLocationService.js";
import { LocalWebSourceLocationArtifacts } from "../browser/assets/WebSourceLocationArtifacts.js";
import { SourceMapDecoder } from "../javascript/sourceMaps/SourceMapDecoder.js";
/** Compose local source/map analysis without import-time reads or process acquisition. */
export const createWebSourceLocationService = (
environment: Readonly<NodeJS.ProcessEnv> = process.env,
): WebSourceLocationService =>
new WebSourceLocationService(
new LocalWebSourceLocationArtifacts(),
new SourceMapDecoder({ environment }),
);
+1
View File
@@ -84,6 +84,7 @@ const PUBLISHED_TOOL_NAME_FLOOR = [
"extract_artifact",
"export_web_scripts",
"trace_web_module_imports",
"trace_web_source_location",
"find_changed_behavior",
"find_xrefs_to_name",
"get_call_graph",
+5
View File
@@ -41,6 +41,11 @@ const sessionEvidence = effects({ mutatesSession: true });
/** Explicit effect audit for every public tool. */
export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
trace_web_source_location: effects({
mutatesSession: true,
writesFilesystem: true,
launchesProcess: true,
}),
trace_web_module_imports: effects({
mutatesSession: true,
writesFilesystem: true,
+28
View File
@@ -9,9 +9,37 @@ import {
} from "../domain/webModuleTrace.js";
import { toolContractMetadata } from "./toolEffects.js";
import { evidenceResultOf } from "./toolOutputSchemas.js";
import {
webSourceLocationInputSchema,
webSourceLocationResultSchema,
} from "../domain/webSourceLocation.js";
/** Local captured-script publication contracts shared by CLI and MCP. */
export const WEB_SCRIPT_TOOL_CONTRACTS = [
{
name: "trace_web_source_location",
...toolContractMetadata("trace_web_source_location"),
description:
"Trace one generated position in an exported website script through an explicitly paired local source map. Verifies manifest, selected script and map SHA-256/byte identities; returns every equal-position mapping, reported source/root, resolved source URL, embedded sourcesContent and original position inline. One-based lines and zero-based UTF-16 columns; greatest generated position <= requested across lines. Duplicate sources and generated-only mappings remain distinct. Caller-selected pairing does not establish deployment authenticity or execution. Regular maps and inline indexed sections only; external section URLs are unsupported. No network access or application execution; uses the pinned upstream codec in an owned Node process with private temporary files, independent 192 MiB old-generation heap and 20-second deadline. 4 MiB map, 32 MiB reply; complete evidence or actionable resource failure. Cleanup may extend the deadline.",
kind: "application",
inputSchema: webSourceLocationInputSchema,
outputSchema: evidenceResultOf(webSourceLocationResultSchema),
examples: [
{
title:
"Trace a selected captured JavaScript point to embedded original source",
input: {
manifest_path: "/analysis/scripts/manifest.json",
script_index: 0,
source_map: {
path: "/analysis/app.js.map",
url: "https://example.test/app.js.map",
},
generated_position: { line: 1, column: 25 },
},
},
],
},
{
name: "trace_web_module_imports",
...toolContractMetadata("trace_web_module_imports"),
+1
View File
@@ -14,6 +14,7 @@ export class ArtifactOperationError extends AnalysisError {
| "inspect_keyed_archive"
| "export_web_scripts"
| "trace_web_module_imports"
| "trace_web_source_location"
| "analyze_javascript_application",
readonly reason:
| "cancelled"
+19
View File
@@ -0,0 +1,19 @@
import { expect, it } from "vitest";
import { ProviderCleanupError } from "./providerCleanupError.js";
import { projectAnalysisError } from "./analysisErrorProjection.js";
it("preserves the binary cleanup default and an explicitly owned nonbinary operation", () => {
expect(new ProviderCleanupError("ghidra", ["database"], {}).operation).toBe(
"close_binary",
);
const error = new ProviderCleanupError(
"source-map-decoder",
["runtime"],
{ reason: "cleanup failed" },
{ operation: "trace_web_source_location" },
);
expect(error.operation).toBe("trace_web_source_location");
expect(projectAnalysisError(error)).toMatchObject({
code: "cleanup_incomplete",
details: { operation: "trace_web_source_location" },
});
});
+4 -2
View File
@@ -12,9 +12,11 @@ export class ProviderCleanupError extends ProviderAdapterError {
providerId: string,
resources: readonly string[],
diagnostics: Readonly<Record<string, JsonValue>>,
options?: ErrorOptions,
/** Identify the operation owning these resources; binary callers retain their default. */
options?: ErrorOptions & { readonly operation?: string },
) {
super(providerId, "close_binary", { ...options, diagnostics });
const { operation = "close_binary", ...errorOptions } = options ?? {};
super(providerId, operation, { ...errorOptions, diagnostics });
this.cleanupResources = [...resources];
}
}
+49
View File
@@ -0,0 +1,49 @@
import { z } from "zod";
import { digestSchema } from "./digests.js";
import { webScriptExportManifestSchema } from "./webScriptExport.js";
/** Select one source from an existing captured-script export. */
export const selectedWebScriptInputSchema = z.strictObject({
manifest_path: z
.string()
.min(1)
.describe("Absolute export_web_scripts manifest path"),
script_index: z
.number()
.int()
.min(0)
.describe("Zero-based index in manifest.scripts"),
});
export type SelectedWebScriptInput = z.output<
typeof selectedWebScriptInputSchema
>;
/** Exact identity of one local file, independently read as bytes. */
export const webArtifactFileSchema = z.strictObject({
path: z.string().min(1),
sha256: digestSchema,
bytes: z.number().int().min(0),
});
/** Artifact-port report shared by capture-derived analyses. */
export const selectedWebScriptArtifactsSchema = z.strictObject({
manifest: webScriptExportManifestSchema,
manifestFile: webArtifactFileSchema,
sourceFile: webArtifactFileSchema,
source: z.string(),
});
export type SelectedWebScriptArtifacts = z.output<
typeof selectedWebScriptArtifactsSchema
>;
/** Preserve original capture authority and completeness alongside the actual manifest identity. */
export const capturedWebManifestIdentitySchema = webArtifactFileSchema.extend({
reported_output_directory: z.string(),
...webScriptExportManifestSchema.pick({
capture_path: true,
capture_sha256: true,
capture_kind: true,
capture_completeness: true,
source_evidence_id: true,
}).shape,
});
+127
View File
@@ -0,0 +1,127 @@
import { z } from "zod";
import { digestSchema } from "./digests.js";
import { exportedWebScriptSchema } from "./webScriptExport.js";
import {
capturedWebManifestIdentitySchema,
selectedWebScriptInputSchema,
webArtifactFileSchema,
} from "./webScriptArtifacts.js";
/** JavaScript source-map positions use UTF-16 columns and one-based lines. */
export const webSourcePositionSchema = z.strictObject({
line: z.number().int().min(1),
column: z.number().int().min(0),
});
/** Trace one explicit generated position through an explicitly paired local map. */
export const webSourceLocationInputSchema = selectedWebScriptInputSchema.extend(
{
source_map: z.strictObject({
path: z
.string()
.min(1)
.describe("Absolute local source-map path; no fetch"),
url: z
.string()
.min(1)
.describe(
"Explicit source-map URL context used to resolve original source names",
),
}),
generated_position: webSourcePositionSchema,
},
);
export type WebSourceLocationInput = z.output<
typeof webSourceLocationInputSchema
>;
const originalContentSchema = z.discriminatedUnion("state", [
z.strictObject({
state: z.literal("embedded"),
text: z.string(),
utf8_sha256: digestSchema,
utf8_bytes: z.number().int().min(0),
representation: z.literal("UTF-8 encoding of decoded sourcesContent text"),
}),
z.strictObject({
state: z.literal("unavailable"),
reason: z.literal("sourcesContent-not-retained"),
}),
]);
/** Keep every matching segment, including anonymous mappings and duplicate URLs. */
export const webSourceMapMatchSchema = z.discriminatedUnion("state", [
z.strictObject({ state: z.literal("generated-only") }),
z.strictObject({
state: z.literal("mapped"),
flattened_source_index: z.number().int().min(0),
section_path: z.array(z.number().int().min(0)),
source_index: z.number().int().min(0),
reported_source: z.string().nullable(),
reported_source_root: z.string().nullable(),
resolved_url: z.string().nullable(),
ignored: z.boolean(),
content: originalContentSchema,
original_position: webSourcePositionSchema.extend({
offset: z.number().int().min(0).nullable(),
content_position: z.enum([
"verified-in-embedded-text",
"outside-embedded-text",
"content-unavailable",
]),
}),
name: z.string().nullable(),
}),
]);
/** Codec observations; map association and deployment authenticity remain selected context. */
export const webSourceMapReportSchema = z.strictObject({
source_map_sha256: digestSchema,
url_context: z.string().min(1),
engine: z.strictObject({ id: z.string().min(1), version: z.string().min(1) }),
runtime: z.strictObject({
id: z.string().min(1),
version: z.string().min(1),
v8_heap_limit_bytes: z.number().int().min(1),
}),
format: z.enum(["regular", "indexed"]),
reported_file: z.string().nullable(),
lookup: z.strictObject({
requested: webSourcePositionSchema,
matched_generated_position: webSourcePositionSchema.nullable(),
semantics: z.literal(
"greatest generated position <= requested, across lines; all equal-position segments",
),
location_units: z.literal("one-based lines; zero-based UTF-16 columns"),
}),
matches: z.array(webSourceMapMatchSchema),
diagnostics: z.array(z.string()),
});
/** Bounds for complete source-map evidence and upstream decoded layouts. */
export const WEB_SOURCE_MAP_LIMITS = {
mapBytes: 4 * 1024 * 1024,
outputBytes: 32 * 1024 * 1024,
decodedRows: 262144,
decodedSegments: 262144,
sectionDepth: 64,
decodeTimeoutMs: 20000,
} as const;
export type WebSourceMapReport = z.output<typeof webSourceMapReportSchema>;
/** Selected capture/map byte identities and complete point attribution inline. */
export const webSourceLocationResultSchema = webSourceMapReportSchema.extend({
manifest: capturedWebManifestIdentitySchema,
source: webArtifactFileSchema.extend({
script_index: z.number().int().min(0),
script: exportedWebScriptSchema,
}),
source_map: webArtifactFileSchema.extend({
url: z.string().min(1),
association: z.literal("caller-selected"),
}),
generated_offset: z.number().int().min(0),
source_authenticity: z.literal("unknown"),
execution: z.literal("unknown"),
limitations: z.array(z.string()),
});
+25
View File
@@ -0,0 +1,25 @@
import { expect, it } from "vitest";
import { webSourceOffset } from "./webSourcePosition.js";
it("uses UTF-16 columns and actual JavaScript line terminators", () => {
const text = "🎋a\r\nb\rc\nd\u2028e\u2029";
expect(webSourceOffset(text, { line: 1, column: 2 })).toBe(2);
expect(webSourceOffset(text, { line: 1, column: 3 })).toBe(3);
expect(webSourceOffset(text, { line: 1, column: 4 })).toBeUndefined();
expect(webSourceOffset(text, { line: 2, column: 0 })).toBe(5);
expect(webSourceOffset(text, { line: 6, column: 0 })).toBe(13);
expect(webSourceOffset(text, { line: 7, column: 0 })).toBeUndefined();
});
it.each([
{ line: 0, column: 0 },
{ line: 1, column: -1 },
{ line: 1.5, column: 0 },
])("rejects invalid source positions: %j", (position) => {
expect(webSourceOffset("", position)).toBeUndefined();
});
it("retains the empty source and the final empty line", () => {
expect(webSourceOffset("", { line: 1, column: 0 })).toBe(0);
expect(webSourceOffset("a\r\n", { line: 2, column: 0 })).toBe(3);
});
+32
View File
@@ -0,0 +1,32 @@
/** JavaScript positions use one-based lines and zero-based UTF-16 columns. */
export interface WebSourcePosition {
readonly line: number;
readonly column: number;
}
/** Verify a selected position against actual text without splitting the whole source. */
export const webSourceOffset = (
text: string,
position: WebSourcePosition,
): number | undefined => {
if (
!Number.isSafeInteger(position.line) ||
position.line < 1 ||
!Number.isSafeInteger(position.column) ||
position.column < 0
)
return undefined;
let line = 1;
let start = 0;
for (const boundary of text.matchAll(/\r\n?|\n|\u2028|\u2029/gu)) {
if (line === position.line)
return position.column <= boundary.index - start
? start + position.column
: undefined;
line += 1;
start = boundary.index + boundary[0].length;
}
return line === position.line && position.column <= text.length - start
? start + position.column
: undefined;
};
File diff suppressed because one or more lines are too long
@@ -0,0 +1,59 @@
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { WEB_SOURCE_MAP_LIMITS } from "../../domain/webSourceLocation.js";
import { traceSourceMap } from "./TraceSourceMap.js";
import { SourceMapFormatFailure } from "./SourceMapFormat.js";
import {
sourceMapCodecInputSchema,
sourceMapCodecReplySchema,
} from "./SourceMapCodecProtocol.js";
import { z } from "zod";
/** Dedicated process entrypoint invoked only for an explicitly selected map. */
const execute = async (): Promise<string> => {
try {
const path = process.argv[2];
if (path === undefined)
throw new SourceMapFormatFailure(
"format",
"Codec request path is missing.",
);
const data = await readStableArtifact(
path,
WEB_SOURCE_MAP_LIMITS.outputBytes,
);
const text = new TextDecoder("utf-8", { fatal: true }).decode(data.bytes);
const value: unknown = JSON.parse(text);
const input = sourceMapCodecInputSchema.parse(value);
if (Buffer.byteLength(input.text) > WEB_SOURCE_MAP_LIMITS.mapBytes)
throw new SourceMapFormatFailure(
"limit",
"Source map exceeds the 4 MiB codec input budget.",
);
const report = traceSourceMap(input.text, input.url, input.position);
const reply = JSON.stringify(
sourceMapCodecReplySchema.parse({ state: "success", report }),
);
if (Buffer.byteLength(reply) > WEB_SOURCE_MAP_LIMITS.outputBytes)
throw new SourceMapFormatFailure(
"limit",
"Complete source-map point evidence exceeds the 32 MiB output budget; no partial evidence was returned.",
);
return reply;
} catch (cause: unknown) {
if (
!(
cause instanceof SourceMapFormatFailure ||
cause instanceof z.ZodError ||
cause instanceof SyntaxError
)
)
throw cause;
return JSON.stringify({
state: "failure",
reason: cause instanceof SourceMapFormatFailure ? cause.reason : "format",
message: cause.message,
});
}
};
process.stdout.write(await execute());
@@ -0,0 +1,25 @@
import { z } from "zod";
import {
webSourceMapReportSchema,
webSourcePositionSchema,
} from "../../domain/webSourceLocation.js";
/** Trusted codec process input; captured application code is never evaluated. */
export const sourceMapCodecInputSchema = z.strictObject({
text: z.string(),
url: z.string().min(1),
position: webSourcePositionSchema,
});
/** Codec success or a specific producer/resource constraint. */
export const sourceMapCodecReplySchema = z.discriminatedUnion("state", [
z.strictObject({
state: z.literal("success"),
report: webSourceMapReportSchema,
}),
z.strictObject({
state: z.literal("failure"),
reason: z.enum(["format", "unsupported", "limit"]),
message: z.string(),
}),
]);
@@ -0,0 +1,242 @@
import { randomUUID } from "node:crypto";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisOutputError,
AnalysisTimeoutError,
} from "../../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { err, ok, type Result } from "../../domain/result.js";
import { WEB_SOURCE_MAP_LIMITS } from "../../domain/webSourceLocation.js";
import {
PrivateRuntimeRoot,
PrivateRuntimeRootUnavailableError,
} from "../../process/PrivateRuntimeRoot.js";
import { ProviderStartupDeadline } from "../../process/ProviderDeadline.js";
import { cleanupOwnedProcessGroup } from "../../process/ProcessOwnership.js";
import {
ProviderProcessSupervisor,
spawnOwnedProviderProcess,
type OwnedProviderProcessSpawnOptions,
type SpawnedOwnedProviderProcess,
} from "../../process/ProviderProcess.js";
const OPERATION = "trace_web_source_location";
const PROVIDER = "source-map-decoder";
/** Launcher seam retains the actual codec protocol and ownership supervisor. */
export type SourceMapCodecLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/** Runtime implementations establish privacy before returning an owned directory. */
export interface SourceMapCodecRuntime {
readonly path: string;
close(): Promise<void>;
}
/** Replaceable acquisition boundaries for an owned codec operation. */
export interface SourceMapDecoderDependencies {
readonly launcher?: SourceMapCodecLauncher;
readonly createRuntime?: () => Promise<SourceMapCodecRuntime>;
readonly environment?: Readonly<NodeJS.ProcessEnv>;
}
/** Run a trusted codec command and release its process/root before returning output. */
export const runSourceMapCommand = async (
request: string,
path: string,
options: ExecutionOptions | undefined,
dependencies: SourceMapDecoderDependencies,
): Promise<Result<string, AnalysisError>> => {
const deadline = new ProviderStartupDeadline(
WEB_SOURCE_MAP_LIMITS.decodeTimeoutMs,
options?.signal,
);
const command = new OwnedCodec(path, dependencies, deadline);
let result: Result<string, AnalysisError>;
try {
await command.start(request);
result = ok(await command.collect());
} catch (cause: unknown) {
result = err(command.failure(cause));
} finally {
deadline.dispose();
}
const cleanup = await command.close(result.ok ? null : result.error.message);
if (cleanup !== undefined) return err(cleanup);
return result.ok && options?.signal?.aborted
? err(new AnalysisCancelledError(OPERATION))
: result;
};
class OwnedCodec {
readonly #runId = `rea-source-map-${randomUUID()}`;
#runtime: SourceMapCodecRuntime | undefined;
#supervisor: ProviderProcessSupervisor | undefined;
#oversized = false;
#processError: string | undefined;
constructor(
readonly path: string,
readonly dependencies: SourceMapDecoderDependencies,
readonly deadline: ProviderStartupDeadline,
) {}
async start(request: string): Promise<void> {
this.check();
this.#runtime = await (
this.dependencies.createRuntime ??
(() => PrivateRuntimeRoot.create({ prefix: "rea-source-map-" }))
)();
this.check();
const requestPath = join(this.#runtime.path, "request.json");
await writeFile(requestPath, request, {
flag: "wx",
mode: 0o600,
signal: this.deadline.signal,
});
this.check();
const environment: NodeJS.ProcessEnv = {
...(this.dependencies.environment ?? process.env),
UV_THREADPOOL_SIZE: "1",
};
delete environment.NODE_OPTIONS;
const spawned = await (
this.dependencies.launcher ?? spawnOwnedProviderProcess
)({
command: process.execPath,
arguments: [
"--max-old-space-size=192",
"--max-semi-space-size=8",
"--v8-pool-size=1",
"--single-threaded",
"--stack_size=2048",
fileURLToPath(new URL("./SourceMapCodecProcess.js", import.meta.url)),
requestPath,
],
runId: this.#runId,
cwd: this.#runtime.path,
hostEnvironment: environment,
});
this.#supervisor = new ProviderProcessSupervisor(
{
...spawned,
ownsProcessLifetime: true,
cleanup:
spawned.cleanup ??
(() => cleanupOwnedProcessGroup(spawned.ownership)),
},
{
onDiagnostic: (event) => {
if (event.type === "error") this.#processError = event.message;
if (
event.type === "output" &&
event.totalBytes >
(event.stream === "stdout"
? WEB_SOURCE_MAP_LIMITS.outputBytes
: 1024 * 1024)
)
this.#oversized = true;
},
},
);
}
async collect(): Promise<string> {
const supervisor = this.#supervisor;
if (supervisor === undefined)
throw new Error("Codec process was not acquired");
while (!(await supervisor.waitForOutputClose(50))) this.check();
this.check();
const snapshot = supervisor.snapshot();
if (snapshot.exitCode !== 0 || snapshot.signal !== null)
throw new ProviderAdapterError(PROVIDER, OPERATION, {
diagnostics: {
map_path: this.path,
exit_code: snapshot.exitCode ?? null,
signal: snapshot.signal ?? null,
stderr: snapshot.stderr.text,
},
});
return snapshot.stdout.text;
}
check(): void {
if (this.deadline.interruption === "cancelled")
throw new AnalysisCancelledError(OPERATION);
if (this.deadline.interruption === "timeout")
throw new AnalysisTimeoutError(
OPERATION,
WEB_SOURCE_MAP_LIMITS.decodeTimeoutMs,
);
if (this.#oversized)
throw new AnalysisOutputError(
OPERATION,
"Codec output exceeds its complete-evidence or diagnostic budget.",
);
if (this.#processError !== undefined)
throw new ProviderAdapterError(PROVIDER, OPERATION, {
diagnostics: { map_path: this.path, error_message: this.#processError },
});
}
failure(cause: unknown): AnalysisError {
if (this.deadline.interruption === "cancelled")
return new AnalysisCancelledError(OPERATION);
if (this.deadline.interruption === "timeout")
return new AnalysisTimeoutError(
OPERATION,
WEB_SOURCE_MAP_LIMITS.decodeTimeoutMs,
);
if (cause instanceof AnalysisError) return cause;
return cause instanceof PrivateRuntimeRootUnavailableError
? new AnalysisCapabilityUnavailableError(
PROVIDER,
OPERATION,
cause.message,
{ cause },
)
: new ProviderAdapterError(PROVIDER, OPERATION, {
cause,
diagnostics: { map_path: this.path, error_message: message(cause) },
});
}
async close(
previousError: string | null,
): Promise<ProviderCleanupError | undefined> {
const failures: string[] = [];
if (this.#supervisor !== undefined) {
const stopped = await this.#supervisor.stop();
if (stopped.status === "incomplete") failures.push(stopped.reason);
}
if (this.#runtime !== undefined) {
try {
await this.#runtime.close();
} catch (cause: unknown) {
failures.push(message(cause));
}
}
return failures.length === 0
? undefined
: new ProviderCleanupError(
PROVIDER,
[
this.#runId,
...(this.#runtime === undefined ? [] : [this.#runtime.path]),
],
{
map_path: this.path,
cleanup_failures: failures,
previous_error: previousError,
codec_pid: this.#supervisor?.launch.process.pid ?? null,
},
{ operation: OPERATION },
);
}
}
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
@@ -0,0 +1,124 @@
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import type { WebSourceMapPort } from "../../application/WebSourceLocationPorts.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
} from "../../domain/analysisErrorCore.js";
import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
WEB_SOURCE_MAP_LIMITS,
type WebSourceMapReport,
} from "../../domain/webSourceLocation.js";
import {
sourceMapCodecInputSchema,
sourceMapCodecReplySchema,
} from "./SourceMapCodecProtocol.js";
import {
runSourceMapCommand,
type SourceMapDecoderDependencies,
} from "./SourceMapCommand.js";
export type { SourceMapDecoderDependencies } from "./SourceMapCommand.js";
const OPERATION = "trace_web_source_location";
/** Integrate the pinned upstream codec through an independently bounded owned process. */
export class SourceMapDecoder implements WebSourceMapPort {
constructor(readonly dependencies: SourceMapDecoderDependencies = {}) {}
/** Return complete point evidence after process and private-root cleanup. */
async trace(
input: Parameters<WebSourceMapPort["trace"]>[0],
options?: ExecutionOptions,
): Promise<Result<WebSourceMapReport, AnalysisError>> {
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION));
const request = sourceMapCodecInputSchema.safeParse({
text: input.text,
url: input.url,
position: input.position,
});
if (!request.success)
return err(new AnalysisInputError(OPERATION, { cause: request.error }));
const serialized = JSON.stringify(request.data);
if (
Buffer.byteLength(input.text) > WEB_SOURCE_MAP_LIMITS.mapBytes ||
Buffer.byteLength(serialized) > WEB_SOURCE_MAP_LIMITS.outputBytes
)
return err(
new ArtifactOperationError(
OPERATION,
"limit",
undefined,
`${input.path}: Source-map input exceeds the 4 MiB map or 32 MiB encoded request budget.`,
),
);
const response = await runSourceMapCommand(
serialized,
input.path,
options,
this.dependencies,
);
if (!response.ok) return response;
let value: unknown;
try {
value = JSON.parse(response.value);
} catch (cause: unknown) {
return err(
new AnalysisOutputError(
OPERATION,
`Malformed codec JSON reply: ${cause instanceof Error ? cause.message : String(cause)}`,
),
);
}
const reply = sourceMapCodecReplySchema.safeParse(value);
if (!reply.success)
return err(
new AnalysisOutputError(
OPERATION,
`Malformed codec reply: ${reply.error.message}`,
),
);
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION));
if (reply.data.state === "failure")
return codecFailure(input.path, reply.data.reason, reply.data.message);
if (reply.data.report.runtime.v8_heap_limit_bytes > 256 * 1024 * 1024)
return err(
new AnalysisOutputError(
OPERATION,
"Codec reported a V8 heap limit exceeding its independent process budget.",
),
);
return ok(reply.data.report);
}
}
const codecFailure = (
path: string,
reason: "format" | "unsupported" | "limit",
detail: string,
): Result<never, AnalysisError> =>
err(
reason === "limit"
? new ArtifactOperationError(
OPERATION,
"limit",
undefined,
`${path}: ${detail}`,
)
: reason === "unsupported"
? new AnalysisCapabilityUnavailableError(
"source-map-decoder",
OPERATION,
"unsupported_source_map_profile",
{ userMessage: `${path}: ${detail}` },
)
: new AnalysisInputError(OPERATION, undefined, [
{
path: ["source_map", "path"],
reason: "invalid_format",
message: `${path}: ${detail}`,
},
]),
);
@@ -0,0 +1,269 @@
import { z } from "zod";
import { WEB_SOURCE_MAP_LIMITS } from "../../domain/webSourceLocation.js";
const leafSchema = z.object({
version: z.literal(3),
file: z.string().nullable().optional(),
sourceRoot: z.string().nullable().optional(),
sources: z.array(z.string().nullable()),
sourcesContent: z.array(z.string().nullable()).nullable().optional(),
names: z.array(z.string()).default([]),
mappings: z.string().regex(/^[A-Za-z0-9+/,;]*$/u),
ignoreList: z.array(z.number().int().min(0)).optional(),
x_google_ignoreList: z.array(z.number().int().min(0)).optional(),
});
const mapHeaderSchema = z.object({
version: z.literal(3),
file: z.string().nullable().optional(),
sections: z.unknown().optional(),
});
const sectionSchema = z.object({
offset: z.object({
line: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
column: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER),
}),
map: z.unknown().optional(),
url: z.string().optional(),
});
const indexedSchema = z.object({
version: z.literal(3),
sections: z.array(sectionSchema),
});
type Offset = { readonly line: number; readonly column: number };
interface PendingMap {
readonly value: unknown;
readonly path: readonly number[];
readonly offset: Offset;
readonly stop: Offset | null;
}
/** A decoder-profile failure with an actionable producer constraint. */
export class SourceMapFormatFailure extends Error {
constructor(
readonly reason: "format" | "unsupported" | "limit",
message: string,
) {
super(message);
}
}
/** Raw declaration identity, separate from flattened engine URL identity. */
export interface SourceMapSourceDeclaration {
readonly sectionPath: readonly number[];
readonly sourceIndex: number;
readonly source: string | null;
readonly root: string | null;
readonly content: string | null;
readonly ignored: boolean;
}
/** Validated leaf with absolute first-line offsets and its enclosing section boundary. */
export interface SourceMapLeaf {
readonly map: z.output<typeof leafSchema>;
readonly offset: Offset;
readonly stop: Offset | null;
}
const compare = (a: Offset, b: Offset): number =>
a.line - b.line || a.column - b.column;
const addOffset = (base: Offset, offset: Offset): Offset => {
const result = {
line: base.line + offset.line,
column: offset.column + (offset.line === 0 ? base.column : 0),
};
if (
!Number.isSafeInteger(result.line) ||
!Number.isSafeInteger(result.column) ||
result.line >= WEB_SOURCE_MAP_LIMITS.decodedRows
)
throw new SourceMapFormatFailure(
"limit",
"Indexed offset exceeds the 262144 generated-row decoder budget.",
);
return result;
};
const addSections = (node: PendingMap, pending: PendingMap[]): void => {
const sections = indexedSchema.parse(node.value).sections;
for (const [index, section] of sections.entries()) {
const previous = sections[index - 1];
if (previous !== undefined && compare(section.offset, previous.offset) <= 0)
throw new SourceMapFormatFailure(
"format",
"Indexed section offsets must be strictly increasing.",
);
if (section.url !== undefined)
throw new SourceMapFormatFailure(
"unsupported",
`External source-map section URL is not fetched: ${section.url}`,
);
if (section.map === undefined)
throw new SourceMapFormatFailure(
"format",
"Indexed section requires an embedded map.",
);
}
for (let index = sections.length - 1; index >= 0; index -= 1) {
const section = sections[index];
if (section === undefined) continue;
const next = sections[index + 1];
const nextOffset =
next === undefined ? null : addOffset(node.offset, next.offset);
const stop =
nextOffset === null
? node.stop
: node.stop === null || compare(nextOffset, node.stop) < 0
? nextOffset
: node.stop;
const offset = addOffset(node.offset, section.offset);
if (node.stop !== null && compare(offset, node.stop) >= 0)
throw new SourceMapFormatFailure(
"format",
"Nested source-map section overlaps its enclosing boundary.",
);
pending.push({
value: section.map,
path: [...node.path, index],
offset,
stop,
});
}
};
const inspectMappingLayout = (
mappings: string,
): { lines: number; segments: number } => {
let lines = 1;
let segments = 0;
let components = 0;
let inSegment = false;
let continued = false;
let digits = 0;
const finish = (): void => {
if (continued)
throw new SourceMapFormatFailure(
"format",
"Source-map VLQ ends with an unfinished continuation.",
);
if (inSegment && ![1, 4, 5].includes(components))
throw new SourceMapFormatFailure(
"format",
"Source-map segment requires one, four or five VLQ components.",
);
};
for (const character of mappings) {
if (character === "," || character === ";") {
finish();
inSegment = false;
components = 0;
if (character === ";") lines += 1;
} else {
digits += 1;
if (
(digits === 7 && !/[A-Dg-j]/u.test(character)) ||
(digits > 7 && character !== "A" && character !== "g")
)
throw new SourceMapFormatFailure(
"format",
"Source-map VLQ exceeds the format's 32-bit quantity representation.",
);
if (!inSegment) segments += 1;
inSegment = true;
continued = /[g-z0-9+/]/u.test(character);
if (!continued) {
components += 1;
digits = 0;
}
}
if (
segments > WEB_SOURCE_MAP_LIMITS.decodedSegments ||
lines > WEB_SOURCE_MAP_LIMITS.decodedRows
)
throw new SourceMapFormatFailure(
"limit",
"Source-map layout exceeds the 262144 segment/generated-row decoder budget.",
);
}
finish();
return { lines, segments };
};
/** Validate structure before allocation, retaining raw declarations and section boundaries. */
export const inspectSourceMap = (
text: string,
): {
readonly jsonText: string;
readonly format: "regular" | "indexed";
readonly reportedFile: string | null;
readonly declarations: readonly SourceMapSourceDeclaration[];
readonly leaves: readonly SourceMapLeaf[];
} => {
const withoutBom = text.replace(/^\uFEFF/u, "");
const jsonText = withoutBom.startsWith(")]}'")
? withoutBom.slice(withoutBom.indexOf("\n") + 1)
: withoutBom;
const value: unknown = JSON.parse(jsonText);
const root = mapHeaderSchema.parse(value);
const declarations: SourceMapSourceDeclaration[] = [];
const leaves: SourceMapLeaf[] = [];
const pending: PendingMap[] = [
{ value, path: [], offset: { line: 0, column: 0 }, stop: null },
];
let segmentCount = 0;
while (pending.length > 0) {
const node = pending.pop();
if (node === undefined) break;
if (node.path.length > WEB_SOURCE_MAP_LIMITS.sectionDepth)
throw new SourceMapFormatFailure(
"limit",
"Indexed source-map nesting exceeds the 64-level decoder stack budget.",
);
if (mapHeaderSchema.parse(node.value).sections !== undefined) {
addSections(node, pending);
continue;
}
const leaf = leafSchema.parse(node.value);
const layout = inspectMappingLayout(leaf.mappings);
segmentCount += layout.segments;
if (
segmentCount > WEB_SOURCE_MAP_LIMITS.decodedSegments ||
node.offset.line + layout.lines > WEB_SOURCE_MAP_LIMITS.decodedRows
)
throw new SourceMapFormatFailure(
"limit",
"Source-map layout exceeds the 262144 segment/generated-row decoder budget.",
);
leaves.push({ map: leaf, offset: node.offset, stop: node.stop });
for (const [sourceIndex, source] of leaf.sources.entries())
declarations.push({
sectionPath: node.path,
sourceIndex,
source,
root: leaf.sourceRoot ?? null,
content: leaf.sourcesContent?.[sourceIndex] ?? null,
ignored: (leaf.ignoreList ?? leaf.x_google_ignoreList ?? []).includes(
sourceIndex,
),
});
}
// Absolute leaves adapt nested first-line column offsets before upstream AnyMap.
// Raw map bytes and the original section/source identities are retained separately.
const normalized =
root.sections === undefined
? leaves[0]?.map
: {
version: 3,
sections: leaves.map((leaf) => ({
offset: leaf.offset,
map: leaf.map,
})),
};
return {
jsonText: JSON.stringify(normalized),
format: root.sections === undefined ? "regular" : "indexed",
reportedFile: root.file ?? null,
declarations,
leaves,
};
};
@@ -0,0 +1,227 @@
import { describe, expect, it } from "vitest";
import { traceSourceMap } from "./TraceSourceMap.js";
const url = "https://app.test/maps/app.js.map?v=2#map";
const point = { line: 1, column: 0 };
const leaf = {
version: 3,
names: [],
mappings: "AAAA",
sources: ["app.ts"],
sourcesContent: ["original"],
};
const trace = (map: unknown, position = point) =>
traceSourceMap(JSON.stringify(map), url, position);
describe("upstream source-map point evidence", () => {
it("preserves every duplicate-column/source-URL candidate and its own embedded bytes", () => {
const report = trace({
...leaf,
sources: ["same.ts", "same.ts"],
sourcesContent: ["first", "second"],
mappings: "AAAA,ACAA",
});
expect(report.matches).toHaveLength(2);
expect(report.matches).toMatchObject([
{
flattened_source_index: 0,
source_index: 0,
resolved_url: "https://app.test/maps/same.ts",
content: { text: "first" },
},
{
flattened_source_index: 1,
source_index: 1,
resolved_url: "https://app.test/maps/same.ts",
content: { text: "second" },
},
]);
});
it("keeps sourceRoot, query/fragment identity and decoded-text UTF-8 digest", () => {
const report = trace({
...leaf,
sourceRoot: "../src",
sources: ["a.ts?v=2#source"],
sourcesContent: ["🎋a\r\nb"],
names: ["answer"],
mappings: "AAAEA",
});
expect(report.matches[0]).toMatchObject({
reported_source: "a.ts?v=2#source",
reported_source_root: "../src",
resolved_url: "https://app.test/src/a.ts?v=2#source",
name: "answer",
original_position: {
line: 1,
column: 2,
offset: 2,
content_position: "verified-in-embedded-text",
},
content: { utf8_bytes: 8 },
});
});
it("retains generated-only mappings and no-match positions explicitly", () => {
expect(
trace({ ...leaf, mappings: "AAAA,K" }, { line: 1, column: 5 }).matches,
).toEqual([{ state: "generated-only" }]);
expect(trace({ ...leaf, mappings: "KAAA" }).matches).toEqual([]);
expect(
trace({ ...leaf, mappings: "" }).lookup.matched_generated_position,
).toBeNull();
});
it("uses cross-line greatest-position lookup without inventing a mapping", () => {
expect(
trace({ ...leaf, mappings: "AAAA;;" }, { line: 3, column: 2 }).lookup
.matched_generated_position,
).toEqual({ line: 1, column: 0 });
});
it("preserves unknown source identity while retaining embedded content", () => {
expect(trace({ ...leaf, sources: [null] }).matches[0]).toMatchObject({
reported_source: null,
resolved_url: null,
content: { text: "original" },
});
});
it("keeps indexed original content aligned when a preceding section has partial contents", () => {
const report = trace(
{
version: 3,
sections: [
{
offset: { line: 0, column: 0 },
map: { ...leaf, sources: ["a", "b"], sourcesContent: ["A"] },
},
{
offset: { line: 1, column: 0 },
map: { ...leaf, sources: ["c"], sourcesContent: ["C"] },
},
],
},
{ line: 2, column: 0 },
);
expect(report.matches[0]).toMatchObject({
section_path: [1],
source_index: 0,
flattened_source_index: 2,
reported_source: "c",
content: { text: "C" },
});
});
it("supports nested indexed line offsets with first-line-only column shifts", () => {
const report = trace(
{
version: 3,
sections: [
{
offset: { line: 1, column: 2 },
map: {
version: 3,
sections: [{ offset: { line: 1, column: 3 }, map: leaf }],
},
},
],
},
{ line: 3, column: 3 },
);
expect(report.lookup.matched_generated_position).toEqual({
line: 3,
column: 3,
});
expect(report.matches[0]).toMatchObject({
section_path: [0, 0],
original_position: { line: 1, column: 0 },
});
});
it("reports original positions outside embedded contents without fabricating an offset", () => {
expect(trace({ ...leaf, mappings: "AAUA" }).matches[0]).toMatchObject({
original_position: {
line: 11,
column: 0,
offset: null,
content_position: "outside-embedded-text",
},
});
});
});
describe("source-map layout admission", () => {
it.each(["ACAA", "AAAAC", "g", "!", "AA", "D"])(
"rejects malformed VLQ/components/indexes: %s",
(mappings) => {
expect(() => trace({ ...leaf, mappings })).toThrow();
},
);
it("rejects huge indexed offsets before upstream row allocation", () => {
expect(() =>
trace({
version: 3,
sections: [{ offset: { line: 1000000000, column: 0 }, map: leaf }],
}),
).toThrow(/decoder budget/u);
});
it("reports the actual malformed section offset constraint", () => {
expect(() =>
trace({
version: 3,
sections: [{ offset: { line: -1, column: 0 }, map: leaf }],
}),
).toThrow(/offset/u);
});
it("rejects overlapping section mappings rather than silently clipping evidence", () => {
expect(() =>
trace({
version: 3,
sections: [
{
offset: { line: 0, column: 0 },
map: { ...leaf, mappings: "AAAA,UAAA" },
},
{ offset: { line: 0, column: 5 }, map: leaf },
],
}),
).toThrow(/overlap/u);
});
it("accepts absent optional names and null contents without changing reported identity", () => {
expect(
trace({
version: 3,
sources: ["app.ts"],
sourcesContent: null,
mappings: "AAAA",
}).matches[0],
).toMatchObject({
reported_source: "app.ts",
content: { state: "unavailable" },
});
});
});
it.each(["ACAA", "AAAAC"])(
"rejects invalid section-local source/name indexes before flattening: %s",
(mappings) => {
expect(() =>
traceSourceMap(
JSON.stringify({
version: 3,
sections: [
{ offset: { line: 0, column: 0 }, map: { ...leaf, mappings } },
{
offset: { line: 1, column: 0 },
map: { ...leaf, names: ["other"] },
},
],
}),
url,
point,
),
).toThrow(/index/u);
},
);
it.each(["ggggggEAAA", "gggggggBAAA"])(
"rejects overflowing VLQ quantities before upstream bitwise decoding: %s",
(mappings) => {
expect(() =>
traceSourceMap(JSON.stringify({ ...leaf, mappings }), url, point),
).toThrow(/32-bit/u);
},
);
+203
View File
@@ -0,0 +1,203 @@
import { createHash } from "node:crypto";
import { createRequire } from "node:module";
import { getHeapStatistics } from "node:v8";
import {
AnyMap,
TraceMap,
decodedMappings,
type SourceMapSegment,
} from "@jridgewell/trace-mapping";
import { z } from "zod";
import { webSourceOffset } from "../../domain/webSourcePosition.js";
import {
webSourceMapReportSchema,
type WebSourceMapReport,
WEB_SOURCE_MAP_LIMITS,
} from "../../domain/webSourceLocation.js";
import type { WebSourcePosition } from "../../domain/webSourcePosition.js";
import {
inspectSourceMap,
SourceMapFormatFailure,
type SourceMapSourceDeclaration,
} from "./SourceMapFormat.js";
/** Decode with the pinned upstream codec; preserve every equal-position candidate. */
export const traceSourceMap = (
text: string,
url: string,
position: WebSourcePosition,
): WebSourceMapReport => {
const inspected = inspectSourceMap(text);
const { map, rows } = decodeValidatedMap(inspected, url);
const matched = greatestPosition(rows, position);
const segments =
matched === null
? []
: (rows[matched.line - 1] ?? []).filter(
(segment) => segment[0] === matched.column,
);
let bytes = 0;
const matches: WebSourceMapReport["matches"] = [];
for (const segment of segments) {
const match = originalMatch(segment, map, inspected.declarations);
bytes += Buffer.byteLength(JSON.stringify(match)) + 1;
if (bytes > WEB_SOURCE_MAP_LIMITS.outputBytes)
throw new SourceMapFormatFailure(
"limit",
"Complete source-map point evidence exceeds the 32 MiB output budget; no partial evidence was returned.",
);
matches.push(match);
}
const metadata: unknown = createRequire(import.meta.url)(
"@jridgewell/trace-mapping/package.json",
);
const version = z
.object({ version: z.string().min(1) })
.parse(metadata).version;
return webSourceMapReportSchema.parse({
source_map_sha256: createHash("sha256").update(text).digest("hex"),
url_context: url,
engine: { id: "jridgewell-trace-mapping", version },
runtime: {
id: "node-v8",
version: process.version,
v8_heap_limit_bytes: getHeapStatistics().heap_size_limit,
},
format: inspected.format,
reported_file: inspected.reportedFile,
lookup: {
requested: position,
matched_generated_position: matched,
semantics:
"greatest generated position <= requested, across lines; all equal-position segments",
location_units: "one-based lines; zero-based UTF-16 columns",
},
matches,
diagnostics: [],
});
};
const decodeValidatedMap = (
inspected: ReturnType<typeof inspectSourceMap>,
url: string,
) => {
for (const leaf of inspected.leaves) {
const decoded = decodedMappings(
new TraceMap(JSON.stringify(leaf.map), url),
);
for (const [line, row] of decoded.entries())
for (const segment of row) {
if (
segment.some((value) => !Number.isSafeInteger(value) || value < 0) ||
(segment.length !== 1 && segment[1] >= leaf.map.sources.length) ||
(segment.length === 5 && segment[4] >= leaf.map.names.length)
)
throw new SourceMapFormatFailure(
"format",
"Decoded section-local position, source index or name index is invalid.",
);
const generatedLine = leaf.offset.line + line;
const generatedColumn =
segment[0] + (line === 0 ? leaf.offset.column : 0);
if (
leaf.stop !== null &&
(generatedLine > leaf.stop.line ||
(generatedLine === leaf.stop.line &&
generatedColumn >= leaf.stop.column))
)
throw new SourceMapFormatFailure(
"format",
"Embedded source-map mappings overlap the following section boundary.",
);
}
}
const map = new AnyMap(inspected.jsonText, url);
const rows = decodedMappings(map);
for (const row of rows)
for (const segment of row) {
if (
![1, 4, 5].includes(segment.length) ||
segment.some((v) => !Number.isSafeInteger(v) || v < 0) ||
(segment.length !== 1 && segment[1] >= inspected.declarations.length) ||
(segment.length === 5 && segment[4] >= map.names.length)
)
throw new SourceMapFormatFailure(
"format",
"Decoded mapping has invalid component count, position, source index or name index.",
);
}
return { map, rows };
};
const greatestPosition = (
rows: ReturnType<typeof decodedMappings>,
position: WebSourcePosition,
): WebSourcePosition | null => {
for (
let line = Math.min(position.line - 1, rows.length - 1);
line >= 0;
line -= 1
) {
let column: number | undefined;
for (const segment of rows[line] ?? [])
if (line < position.line - 1 || segment[0] <= position.column)
column = segment[0];
if (column !== undefined) return { line: line + 1, column };
}
return null;
};
const originalMatch = (
segment: Readonly<SourceMapSegment>,
map: TraceMap,
declarations: readonly SourceMapSourceDeclaration[],
): WebSourceMapReport["matches"][number] => {
if (segment.length === 1) return { state: "generated-only" };
const declaration = declarations[segment[1]];
if (declaration === undefined)
throw new SourceMapFormatFailure(
"format",
"Mapping references an absent source declaration.",
);
const original = { line: segment[2] + 1, column: segment[3] };
const offset =
declaration.content === null
? undefined
: webSourceOffset(declaration.content, original);
return {
state: "mapped",
flattened_source_index: segment[1],
section_path: [...declaration.sectionPath],
source_index: declaration.sourceIndex,
reported_source: declaration.source,
reported_source_root: declaration.root,
resolved_url:
declaration.source === null
? null
: (map.resolvedSources[segment[1]] ?? null),
ignored: declaration.ignored,
content:
declaration.content === null
? { state: "unavailable", reason: "sourcesContent-not-retained" }
: {
state: "embedded",
text: declaration.content,
utf8_sha256: createHash("sha256")
.update(declaration.content)
.digest("hex"),
utf8_bytes: Buffer.byteLength(declaration.content),
representation: "UTF-8 encoding of decoded sourcesContent text",
},
original_position: {
...original,
offset: offset ?? null,
content_position:
declaration.content === null
? "content-unavailable"
: offset === undefined
? "outside-embedded-text"
: "verified-in-embedded-text",
},
name: segment.length === 5 ? (map.names[segment[4]] ?? null) : null,
};
};
+38 -13
View File
@@ -187,7 +187,7 @@ export const spawnOwnedProviderProcess = async (
/**
* Supervises only generic process resources; wire protocol remains adapter-owned.
*
* Output retention is bounded per stream while byte totals remain exact. Stop
* Output retention is complete per stream; adapters own resource budgets. Stop
* first honors token-verified group cleanup when supplied, otherwise it uses a
* bounded TERM-to-KILL escalation for a directly owned child.
*/
@@ -199,6 +199,8 @@ export class ProviderProcessSupervisor {
readonly code: number | null;
readonly signal: NodeJS.Signals | null;
}>();
readonly #outputClose = deferred<void>();
#outputClosed = false;
readonly #onStdout = (chunk: Buffer | string): void => {
this.#capture("stdout", this.#stdout, chunk);
};
@@ -216,6 +218,8 @@ export class ProviderProcessSupervisor {
signal: NodeJS.Signals | null,
): void => {
this.#recordExit(code, signal);
this.#outputClosed = true;
this.#outputClose.resolve(undefined);
};
readonly #onError = (cause: Error): void => {
this.#options.onDiagnostic?.({ type: "error", message: cause.message });
@@ -252,18 +256,20 @@ export class ProviderProcessSupervisor {
/** Wait for process exit up to a caller-owned bounded interval. */
async waitForExit(timeoutMs: number): Promise<boolean> {
if (this.#exitObservation !== undefined) return true;
let timer: NodeJS.Timeout | undefined;
try {
return await Promise.race([
this.#exit.promise.then(() => true),
new Promise<boolean>((resolve) => {
timer = setTimeout(() => resolve(false), Math.max(0, timeoutMs));
}),
]);
} finally {
if (timer !== undefined) clearTimeout(timer);
}
return waitForProcessEvent(
this.#exitObservation !== undefined,
this.#exit.promise,
timeoutMs,
);
}
/** Wait until the producer has closed its output streams; exit alone does not prove complete output. */
async waitForOutputClose(timeoutMs: number): Promise<boolean> {
return waitForProcessEvent(
this.#outputClosed,
this.#outputClose.promise,
timeoutMs,
);
}
/** Stop an owned process once; concurrent callers share the same escalation. */
@@ -442,3 +448,22 @@ const waitForSpawn = (child: ChildProcess): Promise<void> =>
child.once("spawn", onSpawn);
child.once("error", onError);
});
const waitForProcessEvent = async (
observed: boolean,
event: Promise<unknown>,
timeoutMs: number,
): Promise<boolean> => {
if (observed) return true;
let timer: NodeJS.Timeout | undefined;
try {
return await Promise.race([
event.then(() => true),
new Promise<boolean>((resolve) => {
timer = setTimeout(() => resolve(false), Math.max(0, timeoutMs));
}),
]);
} finally {
if (timer !== undefined) clearTimeout(timer);
}
};
+35
View File
@@ -0,0 +1,35 @@
import { EventEmitter } from "node:events";
import { PassThrough } from "node:stream";
import { expect, it } from "vitest";
import { ProviderProcessSupervisor } from "./ProviderProcess.js";
it("exit does not establish complete output when a producer closes its streams later", async () => {
const producer = Object.assign(new EventEmitter(), {
stdout: new PassThrough(),
stderr: new PassThrough(),
exitCode: null,
signalCode: null,
kill: () => false,
});
const supervisor = new ProviderProcessSupervisor({
process: producer,
ownsProcessLifetime: false,
});
try {
producer.stdout.write("first");
producer.emit("exit", 0, null);
expect(await supervisor.waitForExit(0)).toBe(true);
expect(await supervisor.waitForOutputClose(0)).toBe(false);
producer.stdout.end("last");
producer.stderr.end("diagnostic");
producer.emit("close", 0, null);
expect(await supervisor.waitForOutputClose(0)).toBe(true);
expect(supervisor.snapshot()).toMatchObject({
stdout: { text: "firstlast" },
stderr: { text: "diagnostic" },
exitCode: 0,
});
} finally {
supervisor.dispose();
}
});
+10
View File
@@ -19,6 +19,9 @@ import { registerWebScriptTool } from "./registerWebScriptTool.js";
import { registerWebModuleTool } from "./registerWebModuleTool.js";
import type { WebModuleTraceService } from "../application/WebModuleTraceService.js";
import { createWebModuleTraceService } from "../composition/webModules.js";
import { createWebSourceLocationService } from "../composition/webSourceLocations.js";
import { registerWebSourceLocationTool } from "./registerWebSourceLocationTool.js";
import type { WebSourceLocationService } from "../application/WebSourceLocationService.js";
import { registerJavaScriptRecoveryTool } from "./registerJavaScriptRecoveryTool.js";
import { JavaScriptRecoveryService } from "../application/javascript/JavaScriptRecoveryService.js";
import type { JavaScriptRecoveryPort } from "../application/javascript/JavaScriptRecoveryPort.js";
@@ -54,6 +57,7 @@ export interface CreateServerOptions {
readonly firmwareAnalysis?: FirmwareAnalysisPort;
readonly javascriptRecovery?: JavaScriptRecoveryPort;
readonly webModuleTrace?: WebModuleTraceService;
readonly webSourceLocation?: WebSourceLocationService;
readonly androidAnalysis?: AndroidAnalysisPort;
readonly browserObservation?: BrowserObservationPort;
readonly browserScenarioCapture?: BrowserScenarioCapturePort;
@@ -178,6 +182,12 @@ export const createServer = (
toolLogger,
recordEvidence,
);
registerWebSourceLocationTool(
server,
options.webSourceLocation ?? createWebSourceLocationService(),
toolLogger,
recordEvidence,
);
registerJavaScriptRecoveryTool(
server,
new JavaScriptRecoveryService(
@@ -0,0 +1,32 @@
import type { McpServer } from "@modelcontextprotocol/server";
import type { WebSourceLocationService } from "../application/WebSourceLocationService.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import { toolContract } from "../contracts/toolContracts.js";
import type { Logger } from "../logger.js";
import { logToolExecution } from "./toolLogging.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
import { toCallToolResult } from "./toolResult.js";
/** Bind source point tracing to its exact named contract and session Evidence owner. */
export const registerWebSourceLocationTool = (
server: McpServer,
service: WebSourceLocationService,
logger: Logger,
recordEvidence?: EvidenceWriter["recordEvidence"],
): void => {
const contract = toolContract("trace_web_source_location");
server.registerTool(
contract.name,
toolRegistrationOptions(contract),
async (input, context) => {
const result = await logToolExecution(logger, contract.name, () =>
service.trace(input, { signal: context.mcpReq.signal }),
);
if (!result.ok) return toCallToolResult(result, contract);
const recorded = recordEvidence?.(result.value);
return recorded !== undefined && !recorded.ok
? toCallToolResult(recorded, contract)
: toCallToolResult(result, contract);
},
);
};
@@ -0,0 +1,67 @@
import { expect } from "vitest";
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { webSourceLocationFixture } from "../fixtures/webSourceLocation.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
import { cliTest } from "../support/cli/cliFixture.js";
cliTest(
"accepts numeric UTF-16 coordinates and runs the actual owned source-map codec without a browser",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-source-map-cli-");
const fixture = webSourceLocationFixture();
await mkdir(join(root, "files", "modules"), { recursive: true });
await writeFile(join(root, "files", "modules", "main.js"), fixture.source);
const manifest = join(root, "manifest.json");
const map = join(root, "app.js.map");
await writeFile(manifest, JSON.stringify(fixture.manifest));
await writeFile(map, fixture.sourceMap.text);
const response = await cli.run({
arguments: [
"trace-web-source-location",
manifest,
"0",
map,
fixture.sourceMap.url,
"1",
"0",
"--json",
],
environment: {
REA_LOG_LEVEL: "silent",
REA_BROWSER_EXECUTABLE: "relative-unconfigured-browser",
},
timeoutMs: 15000,
});
expect(response.exitCode).toBe(0);
expect(response.json).toMatchObject({
normalized_result: {
source: { script_index: 0 },
source_map: { association: "caller-selected" },
generated_offset: 0,
execution: "unknown",
matches: [{ content: { text: "original" } }],
runtime: { id: "node-v8" },
},
});
const invalid = await cli.run({
arguments: [
"trace-web-source-location",
manifest,
"0",
map,
fixture.sourceMap.url,
"2",
"0",
"--json",
],
environment: { REA_LOG_LEVEL: "silent" },
timeoutMs: 15000,
});
expect(invalid.exitCode).toBe(1);
expect(invalid.json).toMatchObject({
code: "invalid_request",
details: { issues: [{ path: ["generated_position"] }] },
});
},
);
@@ -0,0 +1,78 @@
import { createHash } from "node:crypto";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import { LocalWebSourceLocationArtifacts } from "../../../src/browser/assets/WebSourceLocationArtifacts.js";
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
import { ok } from "../../../src/domain/result.js";
import { WEB_SOURCE_MAP_LIMITS } from "../../../src/domain/webSourceLocation.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import {
webSourceLocationArgs,
webSourceLocationFixture,
} from "../../fixtures/webSourceLocation.js";
const reader = () => {
const { sourceMap: _sourceMap, ...script } = webSourceLocationFixture();
return new LocalWebSourceLocationArtifacts({
load: () => Promise.resolve(ok(script)),
});
};
it("reads exact selected map bytes, retaining a BOM and explicitly selected URL context", async () => {
const root = await createTestTempDirectory("rea-map-artifacts-");
const path = join(root, "app.map");
const text = '\uFEFF{"version":3,"sources":[],"names":[],"mappings":""}';
await writeFile(path, text);
const response = await reader().load({
...webSourceLocationArgs,
source_map: { ...webSourceLocationArgs.source_map, path },
});
if (!response.ok) throw response.error;
expect(response.value.sourceMap).toEqual({
file: {
path,
sha256: createHash("sha256").update(text).digest("hex"),
bytes: Buffer.byteLength(text),
},
url: webSourceLocationArgs.source_map.url,
text,
});
});
it.each(["unavailable", "invalid-utf8", "oversized"])(
"preserves the selected map reader failure: %s",
async (variant) => {
const root = await createTestTempDirectory("rea-map-artifacts-");
const path = join(root, "app.map");
if (variant === "invalid-utf8")
await writeFile(path, Buffer.from([255, 254]));
if (variant === "oversized")
await writeFile(
path,
Buffer.alloc(WEB_SOURCE_MAP_LIMITS.mapBytes + 1, 32),
);
const response = await reader().load({
...webSourceLocationArgs,
source_map: { ...webSourceLocationArgs.source_map, path },
});
if (response.ok) throw new Error("expected map failure");
const projection = projectAnalysisError(response.error);
if (variant === "invalid-utf8")
expect(projection).toMatchObject({
code: "invalid_request",
details: {
issues: [
{
path: ["source_map", "path"],
message: expect.stringContaining("UTF-8"),
},
],
},
});
else
expect(response.error).toMatchObject({
_tag: "ArtifactOperationError",
reason: variant === "oversized" ? "limit" : "io",
detail: expect.stringContaining(path),
});
},
);
@@ -0,0 +1,206 @@
import { fileURLToPath } from "node:url";
import type { SourceMapDecoderDependencies } from "../../../src/javascript/sourceMaps/SourceMapDecoder.js";
import { access } from "node:fs/promises";
import { expect, it } from "vitest";
import { SourceMapDecoder } from "../../../src/javascript/sourceMaps/SourceMapDecoder.js";
import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js";
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
import { waitForProviderProcessReady } from "../../fixtures/providerProcess.js";
import {
webSourceLocationArgs,
webSourceLocationFixture,
} from "../../fixtures/webSourceLocation.js";
const realDecoder = (dependencies: SourceMapDecoderDependencies = {}) =>
new SourceMapDecoder({
...dependencies,
launcher:
dependencies.launcher ??
((options) =>
spawnOwnedProviderProcess({
...options,
arguments: options.arguments.map((argument) =>
argument.endsWith("/SourceMapCodecProcess.js") ||
argument.endsWith("\\SourceMapCodecProcess.js")
? fileURLToPath(
new URL(
"../../../dist/javascript/sourceMaps/SourceMapCodecProcess.js",
import.meta.url,
),
)
: argument,
),
})),
});
const fixture = webSourceLocationFixture();
const input = {
text: fixture.sourceMap.text,
url: fixture.sourceMap.url,
path: fixture.sourceMap.file.path,
position: webSourceLocationArgs.generated_position,
};
it("runs the real pinned codec with an independent heap and removes its private request directory", async () => {
let path: string | undefined;
const environment = {
...process.env,
NODE_OPTIONS: "--require=/rea-file-that-does-not-exist",
};
const decoder = realDecoder({
environment,
createRuntime: async () => {
const root = await PrivateRuntimeRoot.create();
path = root.path;
return root;
},
});
const response = await decoder.trace(input);
if (!response.ok) throw response.error;
expect(response.value).toMatchObject({
engine: { id: "jridgewell-trace-mapping", version: "0.3.31" },
runtime: { id: "node-v8" },
matches: [{ content: { text: "original" } }],
});
expect(response.value.runtime.v8_heap_limit_bytes).toBeLessThanOrEqual(
256 * 1024 * 1024,
);
expect(environment.NODE_OPTIONS).toBe(
"--require=/rea-file-that-does-not-exist",
);
if (path === undefined) throw new Error("runtime not acquired");
await expect(access(path)).rejects.toMatchObject({ code: "ENOENT" });
});
it.each([
{ text: "{", tag: "AnalysisInputError" },
{
text: JSON.stringify({
version: 3,
sections: [
{
offset: { line: 0, column: 0 },
url: "https://app.test/external.map",
},
],
}),
tag: "AnalysisCapabilityUnavailableError",
},
{
text: JSON.stringify({
version: 3,
sections: [{ offset: { line: 1000000000, column: 0 }, map: {} }],
}),
tag: "ArtifactOperationError",
},
])("preserves actual producer failures: $tag", async ({ text, tag }) => {
const response = await realDecoder().trace({ ...input, text });
if (response.ok) throw new Error("expected failed map");
expect(response.error._tag).toBe(tag);
});
it("reports cleanup failure with the actual owning operation", async () => {
const response = await realDecoder({
createRuntime: async () => {
const root = await PrivateRuntimeRoot.create();
return {
path: root.path,
close: async () => {
await root.close();
throw new Error("observed cleanup failure");
},
};
},
}).trace(input);
if (response.ok) throw new Error("expected cleanup failure");
expect(response.error).toMatchObject({
cleanupIncomplete: true,
operation: "trace_web_source_location",
diagnostics: { cleanup_failures: ["observed cleanup failure"] },
});
});
it("cancels a real owned process and removes the runtime request", async () => {
const controller = new AbortController();
let path: string | undefined;
const response = await realDecoder({
createRuntime: async () => {
const root = await PrivateRuntimeRoot.create();
path = root.path;
return root;
},
launcher: async (options) => {
const child = await spawnOwnedProviderProcess({
...options,
arguments: [
"-e",
'process.stdout.write("ready\\n"); setInterval(() => undefined, 1000)',
],
});
await waitForProviderProcessReady(child.process);
setImmediate(() => controller.abort());
return child;
},
}).trace(input, { signal: controller.signal });
if (response.ok) throw new Error("expected cancellation");
expect(response.error._tag).toBe("AnalysisCancelledError");
if (path === undefined) throw new Error("runtime not acquired");
await expect(access(path)).rejects.toMatchObject({ code: "ENOENT" });
});
it("retains final diagnostic bytes for an actual failed codec process", async () => {
const response = await realDecoder({
launcher: (options) =>
spawnOwnedProviderProcess({
...options,
arguments: [
"-e",
'process.stderr.write("final diagnostic"); process.exitCode = 7',
],
}),
}).trace(input);
if (response.ok) throw new Error("expected exit failure");
expect(response.error).toMatchObject({
diagnostics: { exit_code: 7, stderr: "final diagnostic" },
});
});
it("reports complete evidence amplification as a resource failure before serializing an oversized reply", async () => {
const text = JSON.stringify({
version: 3,
sources: ["large.ts"],
sourcesContent: ["a".repeat(600000)],
names: [],
mappings: Array.from({ length: 60 }, () => "AAAA").join(","),
});
const response = await realDecoder().trace({ ...input, text });
if (response.ok) throw new Error("expected complete-evidence budget failure");
expect(response.error).toMatchObject({
_tag: "ArtifactOperationError",
reason: "limit",
detail: expect.stringContaining("no partial"),
});
});
it("rejects a malformed actual subprocess reply", async () => {
const response = await realDecoder({
launcher: (options) =>
spawnOwnedProviderProcess({
...options,
arguments: ["-e", 'process.stdout.write("not JSON")'],
}),
}).trace(input);
if (response.ok) throw new Error("expected malformed reply");
expect(response.error._tag).toBe("AnalysisOutputError");
});
it("removes the private request when launching the codec fails", async () => {
let path: string | undefined;
const response = await realDecoder({
createRuntime: async () => {
const root = await PrivateRuntimeRoot.create();
path = root.path;
return root;
},
launcher: () => Promise.reject(new Error("observed launch failure")),
}).trace(input);
if (response.ok) throw new Error("expected launch failure");
expect(response.error).toMatchObject({
diagnostics: { error_message: "observed launch failure" },
});
if (path === undefined) throw new Error("runtime not acquired");
await expect(access(path)).rejects.toMatchObject({ code: "ENOENT" });
});
@@ -0,0 +1,71 @@
import { traceSourceMap } from "../../../src/javascript/sourceMaps/TraceSourceMap.js";
import { webSourceLocationArgs as args } from "../../fixtures/webSourceLocation.js";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it, onTestFinished } from "vitest";
import { WebSourceLocationService } from "../../../src/application/WebSourceLocationService.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { ok } from "../../../src/domain/result.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { webSourceLocationFixture } from "../../fixtures/webSourceLocation.js";
it("publishes valid SDK schemas and records the named module trace without a binary target", async () => {
const service = new WebSourceLocationService(
{
load: () => Promise.resolve(ok(webSourceLocationFixture())),
},
{
trace: (input) =>
Promise.resolve(
ok(traceSourceMap(input.text, input.url, input.position)),
),
},
);
const session = createTestBinarySession(() => {
throw new Error("binary provider must not start");
});
const server = createServer(session, session, { webSourceLocation: service });
const client = new Client({ name: "module-contract", version: "1" });
onTestFinished(async () => {
await client.close();
await server.close();
await session.close();
});
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
await server.connect(serverTransport);
await client.connect(clientTransport);
const advertised = (await client.listTools()).tools.find(
(tool) => tool.name === "trace_web_source_location",
);
if (advertised?.outputSchema === undefined)
throw new Error("module trace must advertise both schemas");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const inputSchema: Record<string, unknown> = advertised.inputSchema;
const outputSchema: Record<string, unknown> = advertised.outputSchema;
expect(ajv.validateSchema(inputSchema)).toBe(true);
expect(ajv.validateSchema(outputSchema)).toBe(true);
expect(ajv.validate(inputSchema, args)).toBe(true);
const response = await client.callTool({
name: "trace_web_source_location",
arguments: args,
});
expect(response.isError).not.toBe(true);
expect(
ajv.validate(outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
).toBe(true);
const parsed = toolContract("trace_web_source_location").outputSchema.parse(
response.structuredContent,
);
const evidence = parseEvidence(parsed.evidence);
expect(parsed.result).toEqual(evidence.normalized_result);
expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence);
const invalid = await client.callTool({
name: "trace_web_source_location",
arguments: { ...args, manifest_path: "relative.json" },
});
expect(invalid.isError).toBe(true);
});
+49
View File
@@ -0,0 +1,49 @@
import { createHash } from "node:crypto";
import type { WebSourceLocationArtifacts } from "../../src/application/WebSourceLocationPorts.js";
import { webModuleArtifactsFixture } from "./webModuleTrace.js";
import { traceSourceMap } from "../../src/javascript/sourceMaps/TraceSourceMap.js";
import type { WebSourceMapPort } from "../../src/application/WebSourceLocationPorts.js";
import { ok } from "../../src/domain/result.js";
/** Inert codec seam for workflow tests; real process verification is separate. */
export const webSourceMapDecoderFixture: WebSourceMapPort = {
trace: (input) =>
Promise.resolve(ok(traceSourceMap(input.text, input.url, input.position))),
};
/** Source-owned map fixture with independent byte identity and retained capture context. */
export const webSourceLocationFixture = (
text = JSON.stringify({
version: 3,
sources: ["original.ts"],
sourcesContent: ["original"],
names: [],
mappings: "AAAA",
}),
): WebSourceLocationArtifacts => {
const { importMap: _importMap, ...script } =
webModuleArtifactsFixture("generated");
return {
...script,
sourceMap: {
file: {
path: "/analysis/app.js.map",
sha256: createHash("sha256").update(text).digest("hex"),
bytes: Buffer.byteLength(text),
},
url: "https://app.test/app.js.map?v=1#context",
text,
},
};
};
/** Explicit fixture pairing; no capture authenticity or execution is asserted. */
export const webSourceLocationArgs = {
manifest_path: "/analysis/manifest.json",
script_index: 0,
source_map: {
path: "/analysis/app.js.map",
url: "https://app.test/app.js.map?v=1#context",
},
generated_position: { line: 1, column: 0 },
};