mirror of
https://github.com/morluto/rea.git
synced 2026-10-12 02:19:24 +00:00
chore: merge main into feat/dylib-resolution
main now carries #964 (skill version 29, macOS native row 7 -> 8): - README_ja/ar/ko/zh: take main's tables and count trace_dylib_resolution in the artifact graph row (5 -> 6). - Skill: move to version 30. - Regenerate the product catalog, conformance manifest, skill digest and package metadata (136 tools). Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
+1
-1
@@ -217,7 +217,7 @@ rea setup
|
||||
| -------------------- | ----: | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| الفحص الأصلي | 41 | الدوال والشيفرة شبه المصدرية والتعليمات والسلاسل والرموز والاستدعاءات والمراجع والتعليقات وقراءة البايتات ومواضع الملفات |
|
||||
| سير عمل التحقيق | 14 | نظرات عامة على التطبيقات وملفات الدوال وواجهات API الأصلية والتوزيع وفك الترجمة الدفعي وتتبع الميزات ومسارات الاستدعاء ومخططاته واكتشاف Swift وObjective-C |
|
||||
| أدوات macOS الأصلية | 7 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift دون تشغيل Hopper |
|
||||
| أدوات macOS الأصلية | 8 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift ومراقبة الاستدعاءات عبر LLDB دون تشغيل Hopper |
|
||||
| مخطط المخرجات | 6 | جرد الدلائل والحزم وملفات Interface Builder المترجمة وكتالوجات موارد Apple وحل تحميل مكتبات Mach-O الديناميكية والاستخراج |
|
||||
| PE/CLI المُدار | 7 | هوية تجميعات .NET وبياناتها الوصفية وتعليمات CIL والاعتماديات الأصلية واستيراد إعادة البناء ومقارنة الإصدارات |
|
||||
| البرامج الثابتة | 2 | فحص مناطق البرامج الثابتة على Linux واستخراجها الصريح |
|
||||
|
||||
+1
-1
@@ -240,7 +240,7 @@ TypeScript と SQLite を使って私のプロジェクト向けに実装して
|
||||
| -------------------- | --: | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ネイティブ検査 | 41 | 関数、疑似コード、アセンブリ、文字列、シンボル、呼び出し、参照、注釈、バイト読み取り、ファイルオフセット |
|
||||
| 調査ワークフロー | 14 | アプリ概要、関数ドシエ、ネイティブ API とディスパッチ、一括逆コンパイル、機能トレース、コールパス、コールグラフ、Swift と Objective-C の検出 |
|
||||
| macOS ネイティブ | 7 | Hopper を起動せずに扱える Mach-O メタデータ、コード署名、plist、アーキテクチャ、Swift シンボルのデマングル |
|
||||
| macOS ネイティブ | 8 | Hopper を起動せずに扱える Mach-O メタデータ、コード署名、plist、アーキテクチャ、Swift シンボルのデマングル、LLDB による呼び出し観測 |
|
||||
| 成果物グラフ | 6 | ディレクトリとパッケージの目録、コンパイル済み Interface Builder ファイル、Apple アセットカタログ、Mach-O dylib 読み込みの解決、抽出 |
|
||||
| マネージド PE/CLI | 7 | .NET の識別情報、メタデータ、CIL 命令、ネイティブ依存関係、再構築結果のインポート、ビルド比較 |
|
||||
| ファームウェア | 2 | Linux ファームウェア領域の検査と明示的な展開 |
|
||||
|
||||
+1
-1
@@ -207,7 +207,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
|
||||
| ----------------- | --: | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 네이티브 검사 | 41 | 함수, 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조, 주석, 바이트 읽기, 파일 오프셋 |
|
||||
| 조사 워크플로 | 14 | 앱 개요, 함수 기록서, 네이티브 API와 디스패치, 일괄 디컴파일, 기능 추적, 호출 경로, 호출 그래프, Swift와 Objective-C 탐색 |
|
||||
| macOS 네이티브 | 7 | Hopper 실행 없이 처리하는 Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원 |
|
||||
| macOS 네이티브 | 8 | Hopper 실행 없이 처리하는 Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원, LLDB 호출 관찰 |
|
||||
| 아티팩트 그래프 | 6 | 디렉터리와 패키지 목록, 컴파일된 Interface Builder 파일, Apple 애셋 카탈로그, Mach-O dylib 로드 해석, 추출 |
|
||||
| 관리 PE/CLI | 7 | .NET 식별, 메타데이터, CIL 명령어, 네이티브 종속성, 재구성 가져오기, 빌드 비교 |
|
||||
| 펌웨어 | 2 | Linux 펌웨어 영역 검사 및 명시적 추출 |
|
||||
|
||||
+1
-1
@@ -206,7 +206,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
|
||||
| ----------------- | ---: | ------------------------------------------------------------------------------------------------------ |
|
||||
| 原生检查 | 41 | 函数、伪代码、汇编、字符串、符号、调用、引用、注释、字节读取与文件偏移 |
|
||||
| 调查工作流 | 14 | 应用概览、函数档案、原生 API 与分发、批量反编译、功能追踪、调用路径与调用图、Swift 与 Objective-C 发现 |
|
||||
| macOS 原生工具 | 7 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原,无需启动 Hopper |
|
||||
| macOS 原生工具 | 8 | Mach-O 元数据、代码签名、plist、架构、Swift 符号还原与 LLDB 调用观察,无需启动 Hopper |
|
||||
| 产物图 | 6 | 目录与软件包清单、编译后的 Interface Builder 文件、Apple 资源目录、Mach-O dylib 加载解析与提取 |
|
||||
| 托管 PE/CLI | 7 | .NET 程序集身份、元数据、CIL 指令、原生依赖声明、重建导入与构建比较 |
|
||||
| 固件 | 2 | Linux 固件区域检查与显式提取 |
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
"""Bounded native call observation executed inside LLDB's embedded Python.
|
||||
|
||||
REA imports this module into `lldb --batch` and runs `rea_trace <config.json>`.
|
||||
It launches one owned process, stops at the requested functions and
|
||||
Objective-C methods, records each entry from registers and memory reads only,
|
||||
continues, and kills the process once it exits, the event limit is reached or
|
||||
the duration elapses. No expression is evaluated and no target code runs on
|
||||
the tracer's behalf. The result is written as JSON to the configured path.
|
||||
"""
|
||||
|
||||
import json
|
||||
import time
|
||||
|
||||
import lldb
|
||||
|
||||
ARGUMENT_REGISTERS = {
|
||||
"arm64": ["x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7"],
|
||||
"arm64e": ["x0", "x1", "x2", "x3", "x4", "x5", "x6", "x7"],
|
||||
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
|
||||
}
|
||||
MAX_SELECTOR_BYTES = 1024
|
||||
MAX_REPORTED_LOCATIONS = 64
|
||||
MAX_OTHER_STOPS = 256
|
||||
LISTENER_NAME = "rea-native-call-observation"
|
||||
|
||||
|
||||
def _hex(value):
|
||||
return "0x%x" % value
|
||||
|
||||
|
||||
def _location(address, target):
|
||||
"""Load and file address, module and symbol of one code address."""
|
||||
module = address.GetModule()
|
||||
symbol = address.GetSymbol()
|
||||
file_address = address.GetFileAddress()
|
||||
return {
|
||||
"load_address": _hex(address.GetLoadAddress(target)),
|
||||
"file_address": None
|
||||
if file_address == lldb.LLDB_INVALID_ADDRESS
|
||||
else _hex(file_address),
|
||||
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
|
||||
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
|
||||
"symbol": symbol.GetName() if symbol.IsValid() else None,
|
||||
}
|
||||
|
||||
|
||||
def _no_files():
|
||||
return lldb.SBFileSpecList()
|
||||
|
||||
|
||||
def _create_breakpoints(target, spec):
|
||||
"""LLDB breakpoints for one request; a class-qualified method may need two."""
|
||||
if spec["kind"] == "function":
|
||||
modules = lldb.SBFileSpecList()
|
||||
if spec.get("module"):
|
||||
modules.Append(lldb.SBFileSpec(spec["module"], False))
|
||||
return [
|
||||
target.BreakpointCreateByName(
|
||||
spec["name"], lldb.eFunctionNameTypeAuto, modules, _no_files()
|
||||
)
|
||||
]
|
||||
class_name = spec.get("class_name")
|
||||
method_type = spec.get("method_type", "any")
|
||||
if class_name is None:
|
||||
return [
|
||||
target.BreakpointCreateByName(
|
||||
spec["selector"], lldb.eFunctionNameTypeSelector, _no_files(), _no_files()
|
||||
)
|
||||
]
|
||||
prefixes = {"instance": ["-"], "class": ["+"], "any": ["-", "+"]}[method_type]
|
||||
return [
|
||||
target.BreakpointCreateByName(
|
||||
"%s[%s %s]" % (prefix, class_name, spec["selector"]),
|
||||
lldb.eFunctionNameTypeFull,
|
||||
_no_files(),
|
||||
_no_files(),
|
||||
)
|
||||
for prefix in prefixes
|
||||
]
|
||||
|
||||
|
||||
def _dynamic_class(target, pointer):
|
||||
"""Class of an Objective-C receiver from LLDB's runtime reader, without running code."""
|
||||
if pointer == 0:
|
||||
return None
|
||||
data = lldb.SBData.CreateDataFromUInt64Array(
|
||||
target.GetByteOrder(), target.GetAddressByteSize(), [pointer]
|
||||
)
|
||||
value = target.CreateValueFromData(
|
||||
"receiver", data, target.GetBasicType(lldb.eBasicTypeObjCID)
|
||||
)
|
||||
dynamic = value.GetDynamicValue(lldb.eDynamicDontRunTarget)
|
||||
name = dynamic.GetTypeName() if dynamic.IsValid() else None
|
||||
if not name or name in ("id", "objc_object *"):
|
||||
return None
|
||||
return name[:-2] if name.endswith(" *") else name
|
||||
|
||||
|
||||
def _wanted(spec, symbol):
|
||||
"""Selector-only requests may admit only instance or only class methods."""
|
||||
if spec["kind"] != "objc-method" or spec.get("class_name") is not None:
|
||||
return True
|
||||
method_type = spec.get("method_type", "any")
|
||||
if method_type == "any":
|
||||
return True
|
||||
return symbol.startswith("-[" if method_type == "instance" else "+[")
|
||||
|
||||
|
||||
def _record(context, thread, index, sequence):
|
||||
target, process, config = context["target"], context["process"], context["config"]
|
||||
frame = thread.GetFrameAtIndex(0)
|
||||
location = _location(frame.GetPCAddress(), target)
|
||||
symbol = location["symbol"] or ""
|
||||
if not _wanted(config["breakpoints"][index], symbol):
|
||||
return None
|
||||
names = ARGUMENT_REGISTERS.get(config["architecture"], [])
|
||||
registers = [
|
||||
{"name": name, "value": _hex(frame.FindRegister(name).GetValueAsUnsigned())}
|
||||
for name in names[: config["argument_registers"]]
|
||||
]
|
||||
receiver_class = None
|
||||
selector = None
|
||||
if symbol.startswith("-[") or symbol.startswith("+["):
|
||||
error = lldb.SBError()
|
||||
text = process.ReadCStringFromMemory(
|
||||
frame.FindRegister(names[1]).GetValueAsUnsigned(),
|
||||
MAX_SELECTOR_BYTES,
|
||||
error,
|
||||
)
|
||||
selector = text if error.Success() and text else None
|
||||
if symbol.startswith("-["):
|
||||
receiver_class = _dynamic_class(
|
||||
target, frame.FindRegister(names[0]).GetValueAsUnsigned()
|
||||
)
|
||||
frames = min(thread.GetNumFrames(), 1 + config["backtrace_frames"])
|
||||
record = {
|
||||
"sequence": sequence,
|
||||
"elapsed_ms": round((time.monotonic() - context["started"]) * 1000, 3),
|
||||
"thread_id": thread.GetThreadID(),
|
||||
"breakpoint_index": index,
|
||||
"receiver_class": receiver_class,
|
||||
"selector": selector,
|
||||
"registers": registers,
|
||||
"backtrace": [
|
||||
_location(thread.GetFrameAtIndex(depth).GetPCAddress(), target)
|
||||
for depth in range(1, frames)
|
||||
],
|
||||
}
|
||||
record.update(location)
|
||||
return record
|
||||
|
||||
|
||||
def _handle_stop(context, events, other_stops):
|
||||
"""Record breakpoint hits of one stop; return an outcome when tracing must end."""
|
||||
process = context["process"]
|
||||
for thread in process:
|
||||
reason = thread.GetStopReason()
|
||||
if reason == lldb.eStopReasonBreakpoint:
|
||||
index = context["breakpoint_index"].get(thread.GetStopReasonDataAtIndex(0))
|
||||
if index is None:
|
||||
continue
|
||||
record = _record(context, thread, index, len(events))
|
||||
if record is not None:
|
||||
events.append(record)
|
||||
if len(events) >= context["config"]["max_events"]:
|
||||
return "event-limit"
|
||||
elif reason in (lldb.eStopReasonSignal, lldb.eStopReasonException):
|
||||
other_stops.append(thread.GetStopDescription(256))
|
||||
if len(other_stops) >= MAX_OTHER_STOPS:
|
||||
return "stop-limit"
|
||||
return None
|
||||
|
||||
|
||||
def _breakpoint_report(target, created):
|
||||
report = []
|
||||
for index, breakpoints in enumerate(created):
|
||||
locations = []
|
||||
count = 0
|
||||
for breakpoint in breakpoints:
|
||||
count += breakpoint.GetNumLocations()
|
||||
for position in range(breakpoint.GetNumLocations()):
|
||||
if len(locations) < MAX_REPORTED_LOCATIONS:
|
||||
locations.append(
|
||||
_location(breakpoint.GetLocationAtIndex(position).GetAddress(), target)
|
||||
)
|
||||
report.append({"index": index, "location_count": count, "locations": locations})
|
||||
return report
|
||||
|
||||
|
||||
def _launch_info(target, config, listener):
|
||||
launch = lldb.SBLaunchInfo(config["arguments"])
|
||||
launch.SetListener(listener)
|
||||
environment = target.GetEnvironment()
|
||||
for key, value in config["environment"].items():
|
||||
environment.Set(key, value, True)
|
||||
launch.SetEnvironment(environment, False)
|
||||
if config.get("working_directory"):
|
||||
launch.SetWorkingDirectory(config["working_directory"])
|
||||
launch.AddOpenFileAction(0, "/dev/null", True, False)
|
||||
launch.AddOpenFileAction(1, config["stdout_path"], False, True)
|
||||
launch.AddOpenFileAction(2, config["stderr_path"], False, True)
|
||||
return launch
|
||||
|
||||
|
||||
def _wait_for_exit(listener, process, seconds):
|
||||
event = lldb.SBEvent()
|
||||
deadline = time.monotonic() + seconds
|
||||
while process.GetState() != lldb.eStateExited and time.monotonic() < deadline:
|
||||
listener.WaitForEvent(1, event)
|
||||
|
||||
|
||||
def trace(debugger, config):
|
||||
debugger.SetAsync(True)
|
||||
# Stop at the symbol itself, where argument registers still hold the ABI
|
||||
# arguments, even when line tables would let LLDB skip the prologue.
|
||||
lldb.SBDebugger.SetInternalVariable(
|
||||
"target.skip-prologue", "false", debugger.GetInstanceName()
|
||||
)
|
||||
error = lldb.SBError()
|
||||
target = debugger.CreateTarget(
|
||||
config["executable"], config["architecture"], None, False, error
|
||||
)
|
||||
if not target.IsValid():
|
||||
return {"status": "target-error", "error": error.GetCString()}
|
||||
created = [_create_breakpoints(target, spec) for spec in config["breakpoints"]]
|
||||
breakpoint_index = {}
|
||||
for index, breakpoints in enumerate(created):
|
||||
for breakpoint in breakpoints:
|
||||
breakpoint_index[breakpoint.GetID()] = index
|
||||
listener = lldb.SBListener(LISTENER_NAME)
|
||||
started = time.monotonic()
|
||||
process = target.Launch(_launch_info(target, config, listener), error)
|
||||
if not error.Success() or not process.IsValid():
|
||||
return {"status": "launch-error", "error": error.GetCString()}
|
||||
with open(config["pid_path"], "w") as handle:
|
||||
handle.write(str(process.GetProcessID()))
|
||||
context = {
|
||||
"target": target,
|
||||
"process": process,
|
||||
"config": config,
|
||||
"started": started,
|
||||
"breakpoint_index": breakpoint_index,
|
||||
}
|
||||
events = []
|
||||
other_stops = []
|
||||
outcome = None
|
||||
deadline = started + config["duration_ms"] / 1000
|
||||
event = lldb.SBEvent()
|
||||
while outcome is None:
|
||||
if time.monotonic() >= deadline:
|
||||
outcome = "duration-elapsed"
|
||||
break
|
||||
if not listener.WaitForEvent(1, event):
|
||||
continue
|
||||
if not lldb.SBProcess.EventIsProcessEvent(event):
|
||||
continue
|
||||
state = lldb.SBProcess.GetStateFromEvent(event)
|
||||
if state == lldb.eStateExited:
|
||||
outcome = "exited"
|
||||
elif state == lldb.eStateStopped and not lldb.SBProcess.GetRestartedFromEvent(event):
|
||||
outcome = _handle_stop(context, events, other_stops)
|
||||
if outcome is None:
|
||||
process.Continue()
|
||||
pid = process.GetProcessID()
|
||||
exit_status = None
|
||||
exit_description = None
|
||||
if outcome == "exited":
|
||||
exit_status = process.GetExitStatus()
|
||||
exit_description = process.GetExitDescription()
|
||||
else:
|
||||
process.Kill()
|
||||
_wait_for_exit(listener, process, 5)
|
||||
return {
|
||||
"status": "traced",
|
||||
"version": lldb.SBDebugger.GetVersionString().splitlines()[0],
|
||||
"pid": pid,
|
||||
"outcome": outcome,
|
||||
"exit_status": exit_status,
|
||||
"exit_description": exit_description,
|
||||
"killed": outcome != "exited",
|
||||
"terminated": process.GetState() == lldb.eStateExited,
|
||||
"elapsed_ms": round((time.monotonic() - started) * 1000, 3),
|
||||
"breakpoints": _breakpoint_report(target, created),
|
||||
"events": events,
|
||||
"other_stops": other_stops,
|
||||
}
|
||||
|
||||
|
||||
def run(debugger, command, result, internal_dict):
|
||||
"""LLDB command entry point: `rea_trace <config.json>`."""
|
||||
with open(command.strip()) as handle:
|
||||
config = json.load(handle)
|
||||
try:
|
||||
output = trace(debugger, config)
|
||||
except Exception as error: # Reported as a tracer failure, never re-raised into LLDB.
|
||||
output = {"status": "tracer-error", "error": "%s: %s" % (type(error).__name__, error)}
|
||||
with open(config["result_path"], "w") as handle:
|
||||
json.dump(output, handle)
|
||||
|
||||
|
||||
def __lldb_init_module(debugger, internal_dict):
|
||||
debugger.HandleCommand("command script add -f rea_lldb_tracer.run rea_trace")
|
||||
@@ -226,3 +226,22 @@ URI schemes are classified independently of local file names: a reference such
|
||||
as `web3:app.js` remains external even when an artifact has that literal name.
|
||||
Scheme characters may include digits after the initial letter; a relative path
|
||||
such as `./web3:app.js` still names a local artifact.
|
||||
|
||||
Package `exports` fallback arrays are supported both at the top level and under
|
||||
the root `"."` entry. The resolver selects targets in declared order using the
|
||||
same conditional and invalid-entry handling as nested exports arrays.
|
||||
|
||||
ESM relative module paths and selected package exports targets use URL suffix handling
|
||||
and one percent-decoding pass. CommonJS relative paths and legacy package main
|
||||
fields retain literal filename punctuation. Exports targets must start with
|
||||
`./` and exclude `.`, `..`, and `node_modules` path segments, including their
|
||||
encoded forms, before URL normalization. Arrays skip invalid exports targets
|
||||
in declared order; URL decoding and file lookup failures for a selected target
|
||||
do not select a later entry. Rejected references retain the package metadata
|
||||
path, original target, failed constraint, and importing source location.
|
||||
|
||||
HTML script references resolve to exact inventoried files after applying the
|
||||
document base and query/fragment rules. CommonJS module lookups retain extension
|
||||
and directory resolution.
|
||||
Unresolved HTML references retain their declaration, source range, and resolution
|
||||
reason in the renderer observations.
|
||||
|
||||
@@ -166,6 +166,47 @@ owned helper per scenario, removes its temporary compiler cache and stops its
|
||||
helper on cancellation. It does not launch or own the selected application. UI
|
||||
actions may change application data or trigger network activity.
|
||||
|
||||
## Native call observation
|
||||
|
||||
`observe_native_calls` / `rea observe-native-calls <app-or-mach-o> <input-json>`
|
||||
launches the active Mach-O as a new, owned process under LLDB. It stops at
|
||||
caller-selected entries, records them and continues:
|
||||
|
||||
- functions by exact symbol name, optionally limited to one image;
|
||||
- Objective-C methods by selector, optionally limited to one class and to
|
||||
instance or class methods.
|
||||
|
||||
```sh
|
||||
rea observe-native-calls /Applications/Example.app '{"breakpoints":[{"kind":"objc-method","class_name":"NSURLSession","selector":"dataTaskWithRequest:completionHandler:"},{"kind":"function","name":"open","module":"libsystem_kernel.dylib"}],"duration_ms":5000,"backtrace_frames":4}' --json
|
||||
```
|
||||
|
||||
Each event is an observation: thread, image, symbol, load and file address,
|
||||
the raw integer argument registers, optional caller frames and, for an
|
||||
Objective-C method, the selector read from `_cmd` and the receiver's dynamic
|
||||
class. Breakpoints stop at the symbol itself, so the registers hold the ABI
|
||||
arguments. The receiver class comes from LLDB's Objective-C runtime reader
|
||||
without running target code; no expression is evaluated, and REA never
|
||||
attaches to a process it did not launch.
|
||||
|
||||
The run ends when the process exits, `max_events` entries are recorded, or
|
||||
`duration_ms` elapses. In the last two cases the process is killed, and REA
|
||||
confirms that it is gone. The result reports:
|
||||
|
||||
- the outcome, exit status and captured stdout/stderr (1 MiB kept per stream);
|
||||
- signal or exception stops;
|
||||
- every breakpoint's resolved locations; a breakpoint that matched no loaded
|
||||
code makes coverage partial;
|
||||
- limitations.
|
||||
|
||||
Only entries are observed: return values, floating-point and stack arguments,
|
||||
and inlined or `objc_direct` calls are not. The target runs with the current
|
||||
user's permissions and may change files, show UI or use the network.
|
||||
|
||||
Debugging must be allowed for the user (Developer Tools access). A
|
||||
hardened-runtime target without `com.apple.security.get-task-allow` cannot be
|
||||
debugged; `inspect_signature` reports this as the debugger-attach facet, and
|
||||
the tool fails with a `debugger-attach-denied` reason.
|
||||
|
||||
## Provider and verification boundaries
|
||||
|
||||
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
|
||||
|
||||
@@ -7,10 +7,10 @@
|
||||
"client": "2.3.1",
|
||||
"core": "2.3.1"
|
||||
},
|
||||
"skill_version": "29"
|
||||
"skill_version": "30"
|
||||
},
|
||||
"tools": {
|
||||
"total": 135,
|
||||
"total": 136,
|
||||
"families": [
|
||||
{
|
||||
"id": "direct",
|
||||
@@ -84,7 +84,7 @@
|
||||
{
|
||||
"id": "native",
|
||||
"surface": "native-provider",
|
||||
"count": 7,
|
||||
"count": 8,
|
||||
"tools": [
|
||||
"capture_native_ui_scenario",
|
||||
"demangle_swift",
|
||||
@@ -92,6 +92,7 @@
|
||||
"inspect_plist",
|
||||
"inspect_signature",
|
||||
"list_architectures",
|
||||
"observe_native_calls",
|
||||
"observe_native_ui"
|
||||
]
|
||||
},
|
||||
@@ -368,6 +369,7 @@
|
||||
"inspect_plist",
|
||||
"inspect_signature",
|
||||
"list_architectures",
|
||||
"observe_native_calls",
|
||||
"observe_native_ui"
|
||||
]
|
||||
},
|
||||
@@ -585,7 +587,7 @@
|
||||
}
|
||||
],
|
||||
"cli": {
|
||||
"primary_count": 90,
|
||||
"primary_count": 91,
|
||||
"commands": [
|
||||
"address-to-file-offset",
|
||||
"analyze",
|
||||
@@ -650,6 +652,7 @@
|
||||
"list-electron-targets",
|
||||
"list-javascript-runtime-targets",
|
||||
"observe-javascript-runtime",
|
||||
"observe-native-calls",
|
||||
"observe-native-ui",
|
||||
"observe-web-execution",
|
||||
"observe-web-session",
|
||||
@@ -687,15 +690,15 @@
|
||||
},
|
||||
"runtime_catalog": {
|
||||
"counts": {
|
||||
"cli_commands": 90,
|
||||
"mcp_tools": 135,
|
||||
"cli_commands": 91,
|
||||
"mcp_tools": 136,
|
||||
"mcp_prompts": 6
|
||||
},
|
||||
"digests": {
|
||||
"tools_sha256": "6bf6a68f630e8664f83735264f3e687aaaaf0237a7c2f928b2e9a40ef2b4830e",
|
||||
"tools_sha256": "6d195c42333ecfa46e9b253f15174876b34ed1fb226047d1327e3be960680f26",
|
||||
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
|
||||
"combined_sha256": "e6fc5093be3b5c7f9dd0903c197beede8fc9e0295a3cf78211f619367e838910",
|
||||
"providers_sha256": "f30e396bb97eb22e7872b718aef0dd03e0df931db11c03e799d85b0aa9d237fb"
|
||||
"combined_sha256": "047ba77689d22d3b3dac6f51fae06a1a306aaab24fa3be7c2007b396a5fd26f7",
|
||||
"providers_sha256": "96cfc3247384603286780f35bc403870bbca53a648959cbe36bc28dbb43700e5"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -529,6 +529,17 @@ conformance/vtable fixture.
|
||||
`positive_e2e: false`; it must not be reported as capture/action proof.
|
||||
Both commands reject a changed executable digest and clean up the fixture
|
||||
process and helper. These lanes require an interactive macOS desktop. See [native investigation](native-investigation.md)
|
||||
|
||||
`npm run verify:native-calls` needs only macOS with Command Line Tools (`clang`,
|
||||
`lldb`, `codesign`, `nm`). It compiles `tests/conformance/native/calls.m` and
|
||||
runs `observe-native-calls` through the CLI and stdio MCP. It checks:
|
||||
|
||||
- the receiver class, selector and argument registers of every entry, and that
|
||||
breakpoint addresses equal `nm`'s symbol addresses;
|
||||
- captured stdout and an environment override;
|
||||
- the event-limit and duration outcomes, with the process confirmed gone;
|
||||
- that a hardened-runtime copy is refused with `debugger-attach-denied`, and
|
||||
that the same copy signed with `get-task-allow` is traced.
|
||||
for the exact ABI, authority, graph and observation boundaries.
|
||||
|
||||
### Firmware adapters
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"manifest_id": "ecm_60794753dc2b47951640d0e37043b536292167523ba6d02bc87e5973f96d8c01",
|
||||
"manifest_id": "ecm_5d2fd13510a46479a522d904504fdf959a0feb9d7773cbffba50d3ae6e3e9a79",
|
||||
"verifier": {
|
||||
"id": "rea-managed-conformance",
|
||||
"version": "1"
|
||||
@@ -13,7 +13,7 @@
|
||||
"skill_digests": [
|
||||
{
|
||||
"skill_id": "reverse-engineer-anything",
|
||||
"sha256": "5ca360882454778b5a98b99f54d0f9712350b566514a68c6afae4490f59f33ac"
|
||||
"sha256": "8d4c905cac2ea90440a072cdfefc6d6830ea33a3688855c745d2e870bc40960f"
|
||||
}
|
||||
],
|
||||
"claims": [
|
||||
|
||||
Generated
+25
@@ -31,6 +31,7 @@
|
||||
"isomorphic-git": "1.38.7",
|
||||
"jsonc-parser": "3.3.1",
|
||||
"lossless-json": "4.3.1",
|
||||
"parse5": "8.0.1",
|
||||
"pino": "10.4.0",
|
||||
"playwright-core": "1.58.2",
|
||||
"plist": "5.0.0",
|
||||
@@ -6339,6 +6340,30 @@
|
||||
"integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
|
||||
"license": "(MIT AND Zlib)"
|
||||
},
|
||||
"node_modules/parse5": {
|
||||
"version": "8.0.1",
|
||||
"resolved": "https://registry.npmjs.org/parse5/-/parse5-8.0.1.tgz",
|
||||
"integrity": "sha512-z1e/HMG90obSGeidlli3hj7cbocou0/wa5HacvI3ASx34PecNjNQeaHNo5WIZpWofN9kgkqV1q5YvXe3F0FoPw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"entities": "^8.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/inikulin/parse5?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/parse5/node_modules/entities": {
|
||||
"version": "8.1.0",
|
||||
"resolved": "https://registry.npmjs.org/entities/-/entities-8.1.0.tgz",
|
||||
"integrity": "sha512-kxL7msIffSuh9aaFAMD7rxAIuTRMAHMeBtgHW2yUdWw732ZNh4MehkF2gdjvtdmikkaIP9bFDDJOPlsvm7avrA==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">=20.19.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/entities?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/path-key": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz",
|
||||
|
||||
@@ -44,6 +44,7 @@
|
||||
"bridge/hopper_bridge.py",
|
||||
"bridge/native/NativeUIChildren.swift",
|
||||
"bridge/native/ReaNativeUI.swift",
|
||||
"bridge/native/rea_lldb_tracer.py",
|
||||
"bridge/process/ProcessRunTokenReader.swift",
|
||||
"bridge/process/ProcessRunTokenReaderMain.swift",
|
||||
"bridge/ghidra/ReaGhidraBridge.java",
|
||||
@@ -176,6 +177,7 @@
|
||||
"prepack": "node scripts/prepack.mjs",
|
||||
"prepare": "node scripts/prepare.mjs",
|
||||
"verify:asset-catalog": "npm run build:cached && node scripts/verify/apple/asset-catalog.mjs",
|
||||
"verify:native-calls": "npm run build:cached && node scripts/verify-native-calls.mjs",
|
||||
"verify:native-ui:permissions": "npm run build:cached && node scripts/verify-native-ui.mjs --allow-permission-denial",
|
||||
"test:focused": "node scripts/run-development-tests.mjs focused",
|
||||
"build:windows-native": "node scripts/build-windows-native.mjs",
|
||||
@@ -225,6 +227,7 @@
|
||||
"isomorphic-git": "1.38.7",
|
||||
"jsonc-parser": "3.3.1",
|
||||
"lossless-json": "4.3.1",
|
||||
"parse5": "8.0.1",
|
||||
"pino": "10.4.0",
|
||||
"playwright-core": "1.58.2",
|
||||
"plist": "5.0.0",
|
||||
|
||||
@@ -0,0 +1,240 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { basename, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import {
|
||||
artifactCli,
|
||||
artifactMcpResult,
|
||||
withArtifactMcp,
|
||||
} from "./lib/artifact-e2e.mjs";
|
||||
|
||||
const exec = promisify(execFile);
|
||||
|
||||
if (process.platform !== "darwin")
|
||||
throw new Error(
|
||||
"Native call observation verification requires macOS with Command Line Tools",
|
||||
);
|
||||
for (const tool of ["clang", "lldb", "codesign", "nm"])
|
||||
try {
|
||||
await exec("/usr/bin/xcrun", ["--find", tool]);
|
||||
} catch (cause) {
|
||||
throw new Error(
|
||||
`Native call observation verification requires ${tool} from Command Line Tools (xcrun --find ${tool} failed)`,
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
|
||||
const SOURCE = fileURLToPath(
|
||||
new URL("../tests/conformance/native/calls.m", import.meta.url),
|
||||
);
|
||||
const GREET = "-[ReaCallGreeter greet:times:]";
|
||||
const INPUT = {
|
||||
breakpoints: [
|
||||
{
|
||||
kind: "objc-method",
|
||||
class_name: "ReaCallGreeter",
|
||||
selector: "greet:times:",
|
||||
method_type: "instance",
|
||||
},
|
||||
{ kind: "function", name: "rea_call_add" },
|
||||
],
|
||||
environment: { REA_CALLS_MARKER: "observed" },
|
||||
argument_registers: 4,
|
||||
backtrace_frames: 1,
|
||||
};
|
||||
|
||||
/** Pointer-free facts that two runs of the same scenario must share. */
|
||||
const stable = (result) =>
|
||||
result.events.map((event) => ({
|
||||
symbol: event.symbol,
|
||||
file_address: event.file_address,
|
||||
selector: event.selector,
|
||||
receiver_class: event.receiver_class,
|
||||
counter:
|
||||
event.symbol === GREET
|
||||
? event.registers[3].value
|
||||
: event.registers[0].value,
|
||||
}));
|
||||
|
||||
const symbolAddress = async (binary, symbol) => {
|
||||
const { stdout } = await exec("/usr/bin/xcrun", ["nm", "-n", binary]);
|
||||
const line = stdout.split("\n").find((entry) => entry.endsWith(` ${symbol}`));
|
||||
assert.ok(line, `nm lists ${symbol}`);
|
||||
return `0x${BigInt(`0x${line.split(" ")[0]}`).toString(16)}`;
|
||||
};
|
||||
|
||||
const checkScenario = async (binary) => {
|
||||
const result = await artifactCli("observe-native-calls", binary, [
|
||||
JSON.stringify(INPUT),
|
||||
]);
|
||||
assert.equal(result.process.outcome, "exited");
|
||||
assert.equal(result.process.exit_status, 0);
|
||||
assert.equal(result.process.terminated, true);
|
||||
assert.match(result.process.stdout.text, /^world x0 40\n/u);
|
||||
assert.match(result.process.stdout.text, /marker observed\n$/u);
|
||||
assert.deepEqual(result.coverage, {
|
||||
status: "complete",
|
||||
event_limit_reached: false,
|
||||
unresolved_breakpoints: [],
|
||||
});
|
||||
const greet = result.events.filter(({ symbol }) => symbol === GREET);
|
||||
const add = result.events.filter(({ symbol }) => symbol === "rea_call_add");
|
||||
assert.equal(greet.length, 3);
|
||||
assert.equal(add.length, 3);
|
||||
for (const [index, event] of greet.entries()) {
|
||||
assert.equal(event.receiver_class, "ReaCallGreeter");
|
||||
assert.equal(event.selector, "greet:times:");
|
||||
assert.equal(event.registers[3].value, `0x${index}`);
|
||||
assert.equal(event.backtrace[0].symbol, "main");
|
||||
assert.equal(event.module, basename(binary));
|
||||
}
|
||||
for (const [index, event] of add.entries()) {
|
||||
assert.equal(event.registers[0].value, `0x${index}`);
|
||||
assert.equal(event.registers[1].value, "0x28");
|
||||
}
|
||||
// Breakpoints stop at the symbol itself, which Apple's nm reports.
|
||||
assert.equal(
|
||||
add[0].file_address,
|
||||
await symbolAddress(binary, "_rea_call_add"),
|
||||
);
|
||||
assert.equal(greet[0].file_address, await symbolAddress(binary, GREET));
|
||||
return result;
|
||||
};
|
||||
|
||||
const alive = (pid) => {
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
|
||||
const checkBounds = async (binary) => {
|
||||
const limited = await artifactCli("observe-native-calls", binary, [
|
||||
JSON.stringify({
|
||||
breakpoints: [{ kind: "function", name: "nanosleep" }],
|
||||
arguments: ["--wait"],
|
||||
max_events: 2,
|
||||
duration_ms: 30_000,
|
||||
}),
|
||||
]);
|
||||
assert.equal(limited.process.outcome, "event-limit");
|
||||
assert.equal(limited.events.length, 2);
|
||||
assert.equal(limited.process.terminated, true);
|
||||
assert.equal(alive(limited.process.pid), false);
|
||||
const elapsed = await artifactCli("observe-native-calls", binary, [
|
||||
JSON.stringify({
|
||||
breakpoints: [{ kind: "function", name: "rea_call_add" }],
|
||||
arguments: ["--wait"],
|
||||
duration_ms: 1_500,
|
||||
}),
|
||||
]);
|
||||
assert.equal(elapsed.process.outcome, "duration-elapsed");
|
||||
assert.equal(elapsed.events.length, 0);
|
||||
assert.equal(alive(elapsed.process.pid), false);
|
||||
};
|
||||
|
||||
const cliFailure = async (binary) => {
|
||||
try {
|
||||
await exec(
|
||||
process.execPath,
|
||||
[
|
||||
fileURLToPath(new URL("rea.mjs", import.meta.url)),
|
||||
"observe-native-calls",
|
||||
binary,
|
||||
JSON.stringify(INPUT),
|
||||
"--json",
|
||||
],
|
||||
{ env: { ...process.env, REA_LOG_LEVEL: "silent" }, timeout: 120_000 },
|
||||
);
|
||||
} catch (cause) {
|
||||
return JSON.parse(cause.stdout);
|
||||
}
|
||||
throw new Error("observe-native-calls unexpectedly succeeded");
|
||||
};
|
||||
|
||||
/** Hardened runtime blocks the debugger unless get-task-allow is granted. */
|
||||
const checkHardenedRuntime = async (directory, binary) => {
|
||||
const blocked = join(directory, "hardened");
|
||||
await exec("/bin/cp", [binary, blocked]);
|
||||
await exec("/usr/bin/xcrun", [
|
||||
"codesign",
|
||||
"-f",
|
||||
"-s",
|
||||
"-",
|
||||
"-o",
|
||||
"runtime",
|
||||
blocked,
|
||||
]);
|
||||
const denied = await cliFailure(blocked);
|
||||
assert.equal(denied.code, "capability_unavailable");
|
||||
assert.match(denied.details.reason, /^debugger-attach-denied: /u);
|
||||
const allowed = join(directory, "debuggable");
|
||||
const entitlements = join(directory, "get-task-allow.plist");
|
||||
await writeFile(
|
||||
entitlements,
|
||||
'<?xml version="1.0" encoding="UTF-8"?><plist version="1.0"><dict><key>com.apple.security.get-task-allow</key><true/></dict></plist>',
|
||||
);
|
||||
await exec("/bin/cp", [binary, allowed]);
|
||||
await exec("/usr/bin/xcrun", [
|
||||
"codesign",
|
||||
"-f",
|
||||
"-s",
|
||||
"-",
|
||||
"-o",
|
||||
"runtime",
|
||||
"--entitlements",
|
||||
entitlements,
|
||||
allowed,
|
||||
]);
|
||||
const traced = await checkScenario(allowed);
|
||||
assert.equal(traced.process.exit_status, 0);
|
||||
};
|
||||
|
||||
const directory = await mkdtemp(join(tmpdir(), "rea-native-calls-"));
|
||||
try {
|
||||
const binary = join(directory, "ReaCalls");
|
||||
await exec("/usr/bin/xcrun", [
|
||||
"clang",
|
||||
"-fobjc-arc",
|
||||
"-framework",
|
||||
"Foundation",
|
||||
SOURCE,
|
||||
"-o",
|
||||
binary,
|
||||
]);
|
||||
const viaCli = await checkScenario(binary);
|
||||
await withArtifactMcp(binary, async (client) => {
|
||||
const viaMcp = await artifactMcpResult(
|
||||
client,
|
||||
"observe_native_calls",
|
||||
INPUT,
|
||||
);
|
||||
assert.deepEqual(stable(viaMcp), stable(viaCli));
|
||||
assert.equal(viaMcp.process.stdout.text, viaCli.process.stdout.text);
|
||||
});
|
||||
await checkBounds(binary);
|
||||
await checkHardenedRuntime(directory, binary);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
ok: true,
|
||||
mocked: false,
|
||||
cli: true,
|
||||
stdio_mcp: true,
|
||||
debugger: viaCli.debugger.version,
|
||||
events: viaCli.events.length,
|
||||
outcomes: ["exited", "event-limit", "duration-elapsed"],
|
||||
hardened_runtime: {
|
||||
without_get_task_allow: "denied",
|
||||
with_get_task_allow: "traced",
|
||||
},
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
@@ -2,9 +2,9 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "29"
|
||||
tool_count: 135
|
||||
catalog_digest: "e6fc5093be3b5c7f9dd0903c197beede8fc9e0295a3cf78211f619367e838910"
|
||||
version: "30"
|
||||
tool_count: 136
|
||||
catalog_digest: "047ba77689d22d3b3dac6f51fae06a1a306aaab24fa3be7c2007b396a5fd26f7"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
@@ -9,6 +9,12 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
|
||||
not original source. Provider unavailability and unsupported metadata remain
|
||||
unknown rather than false.
|
||||
|
||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default `result.tool_availability` includes the
|
||||
complete tool inventory with availability reasons and remediation. When choosing
|
||||
|
||||
@@ -25,6 +25,7 @@ import type {
|
||||
JavaScriptSourceMapOriginal,
|
||||
} from "./JavaScriptArtifactAnalysisTypes.js";
|
||||
import { analyzeJavaScriptJsonModule } from "./JavaScriptJsonModules.js";
|
||||
import { htmlArtifactReferences } from "../../domain/javascript/htmlArtifactReferences.js";
|
||||
|
||||
interface MutableArtifactAnalysis {
|
||||
readonly files: AnalyzedJavaScriptArtifactFile[];
|
||||
@@ -205,26 +206,13 @@ const parseHtmlScripts = (
|
||||
path: string,
|
||||
text: string,
|
||||
): JavaScriptHtmlScriptObservation[] => {
|
||||
const scripts: JavaScriptHtmlScriptObservation[] = [];
|
||||
const baseHref = htmlBaseHref(text);
|
||||
const pattern = /<script\b[^>]*\bsrc\s*=\s*(["'])([^"']+)\1[^>]*>/giu;
|
||||
for (const match of text.matchAll(pattern)) {
|
||||
const script = match[2];
|
||||
if (script === undefined) continue;
|
||||
const start = match.index;
|
||||
scripts.push({
|
||||
html_path: path,
|
||||
script_path: script,
|
||||
base_href: baseHref,
|
||||
location: rangeForOffsets(text, start, start + match[0].length),
|
||||
});
|
||||
}
|
||||
return scripts;
|
||||
};
|
||||
|
||||
const htmlBaseHref = (text: string): string | null => {
|
||||
const match = /<base\b[^>]*\bhref\s*=\s*(["'])([^"']+)\1[^>]*>/iu.exec(text);
|
||||
return match?.[2] ?? null;
|
||||
const { scripts, baseHref } = htmlArtifactReferences(text);
|
||||
return scripts.map(({ scriptPath, startOffset, endOffset }) => ({
|
||||
html_path: path,
|
||||
script_path: scriptPath,
|
||||
base_href: baseHref,
|
||||
location: rangeForOffsets(text, startOffset, endOffset),
|
||||
}));
|
||||
};
|
||||
|
||||
const parseSourceMap = (
|
||||
|
||||
@@ -5,7 +5,10 @@ import {
|
||||
linkElectronRoleToAsset,
|
||||
type JavaScriptArtifactGraphContext,
|
||||
} from "./JavaScriptArtifactGraphContext.js";
|
||||
import { artifactObservationEvidence } from "./JavaScriptArtifactGraphEvidence.js";
|
||||
import {
|
||||
artifactObservationEvidence,
|
||||
staticInferenceEvidence,
|
||||
} from "./JavaScriptArtifactGraphEvidence.js";
|
||||
import { completeApplicationCoverage } from "../../domain/javascript/javascriptApplicationEvidenceSchemas.js";
|
||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||
|
||||
@@ -57,7 +60,32 @@ export const addJavaScriptHtmlRoles = (
|
||||
htmlBaseHref: script.base_href,
|
||||
});
|
||||
const resolvedPath = resolution.resolved_path;
|
||||
if (resolvedPath === null) continue;
|
||||
if (resolvedPath === null) {
|
||||
context.accumulator.addNode({
|
||||
kind: role.kind,
|
||||
identity: role.identity,
|
||||
observations: [
|
||||
{
|
||||
label: script.script_path,
|
||||
properties: {
|
||||
mechanism: "html-script-reference",
|
||||
script_path: script.script_path,
|
||||
base_href: script.base_href,
|
||||
...resolution,
|
||||
},
|
||||
evidence: staticInferenceEvidence({
|
||||
sha256: html.sha256,
|
||||
path: html.path,
|
||||
range: script.location,
|
||||
operation: "resolve-html-script-reference",
|
||||
coverage: completeApplicationCoverage(),
|
||||
limitations: resolution.limitations,
|
||||
}),
|
||||
},
|
||||
],
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const resolved = context.fileNodes.get(resolvedPath);
|
||||
if (resolved === undefined) continue;
|
||||
addStaticInferenceEdge(context, {
|
||||
|
||||
@@ -676,3 +676,52 @@ const file = (
|
||||
? { included: false, reason: "invalid-utf8" }
|
||||
: { included: true, value: text },
|
||||
});
|
||||
|
||||
describe("exact HTML resource resolution", () => {
|
||||
it.each([
|
||||
["extension", ["app.js"]],
|
||||
["directory index", ["app/index.js"]],
|
||||
["directory package", ["app/package.json", "app/entry.js"]],
|
||||
] as const)("does not infer a file from %s fallback", (_name, paths) => {
|
||||
const files = fileMap([
|
||||
file("index.html", "root"),
|
||||
...paths.map((path) =>
|
||||
file(
|
||||
path,
|
||||
"root",
|
||||
path.endsWith("package.json") ? '{"main":"entry.js"}' : "",
|
||||
),
|
||||
),
|
||||
]);
|
||||
expect(
|
||||
resolve({
|
||||
declaredPath: "./app",
|
||||
sourcePath: "index.html",
|
||||
context: "html-reference",
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ resolution_status: "not-found", resolved_path: null });
|
||||
});
|
||||
|
||||
it("prefers an exact extensionless resource and retains URL suffix handling", () => {
|
||||
const files = fileMap([
|
||||
file("renderer/index.html", "root"),
|
||||
file("assets/app", "root"),
|
||||
file("assets/app.js", "root"),
|
||||
file("assets/app/index.js", "root"),
|
||||
]);
|
||||
expect(
|
||||
resolve({
|
||||
declaredPath: "./app?cache=1#v2",
|
||||
sourcePath: "renderer/index.html",
|
||||
context: "html-reference",
|
||||
htmlBaseHref: "/assets/",
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({
|
||||
declared_path: "./app?cache=1#v2",
|
||||
resolution_status: "resolved",
|
||||
resolved_path: "assets/app",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -86,13 +86,25 @@ const resolvePackagePath = (
|
||||
input: ResolveArtifactPathInput,
|
||||
packageChain: ReadonlySet<string>,
|
||||
mode: "full" | "files-and-index" = "full",
|
||||
exportsTargetUrl = false,
|
||||
): ArtifactPathResolution => {
|
||||
const rejected = rejectDeclaration(input);
|
||||
if (rejected !== null) return rejected;
|
||||
const candidate = contextualCandidate(input);
|
||||
const candidate = contextualCandidate(input, exportsTargetUrl);
|
||||
if (typeof candidate !== "string") return candidate;
|
||||
const confined = confineCandidate(input, candidate);
|
||||
if (typeof confined !== "string") return confined;
|
||||
if (input.context === "html-reference")
|
||||
return outcome(
|
||||
input,
|
||||
resolveFileCandidates(input, [confined]) ?? {
|
||||
resolvedPath: null,
|
||||
status: "not-found",
|
||||
limitations: [
|
||||
`The exact HTML resource ${confined} was not found among the selected application files.`,
|
||||
],
|
||||
},
|
||||
);
|
||||
const resolved =
|
||||
mode === "files-and-index"
|
||||
? (resolveFileCandidates(input, [
|
||||
@@ -124,11 +136,13 @@ const rejectDeclaration = (
|
||||
|
||||
const contextualCandidate = (
|
||||
input: ResolveArtifactPathInput,
|
||||
exportsTargetUrl: boolean,
|
||||
): string | ArtifactPathResolution => {
|
||||
const { context } = input;
|
||||
const declared =
|
||||
let declared =
|
||||
context === "html-reference" ||
|
||||
context === "url-reference" ||
|
||||
exportsTargetUrl ||
|
||||
(context === "module-specifier" && input.moduleKind !== "require")
|
||||
? stripQueryAndFragment(input.declaredPath)
|
||||
: input.declaredPath;
|
||||
@@ -146,6 +160,24 @@ const contextualCandidate = (
|
||||
return unresolvedOutcome(input, "external", [
|
||||
"URL schemes and protocol-relative URLs are outside this local artifact path context.",
|
||||
]);
|
||||
// Exports targets use URL paths for both loaders; legacy main fields remain literal.
|
||||
if (
|
||||
exportsTargetUrl ||
|
||||
(context === "module-specifier" && input.moduleKind !== "require")
|
||||
) {
|
||||
try {
|
||||
declared = decodeURIComponent(declared);
|
||||
} catch (cause: unknown) {
|
||||
void cause;
|
||||
return unresolvedOutcome(input, "rejected", [
|
||||
"The module URL path contains malformed percent encoding.",
|
||||
]);
|
||||
}
|
||||
if (declared.includes("\0"))
|
||||
return unresolvedOutcome(input, "rejected", [
|
||||
"The decoded module URL path contains NUL.",
|
||||
]);
|
||||
}
|
||||
const relative = declared.startsWith("/") ? declared.slice(1) : declared;
|
||||
return declared.startsWith("/")
|
||||
? relative
|
||||
@@ -317,6 +349,12 @@ const resolveCandidate = (
|
||||
`Directory package metadata ${packagePath} is not valid package JSON.`,
|
||||
],
|
||||
};
|
||||
if (main.status === "rejected")
|
||||
return {
|
||||
resolvedPath: null,
|
||||
status: "rejected",
|
||||
limitations: [`Package metadata ${packagePath}: ${main.limitation}`],
|
||||
};
|
||||
if (main.status === "unmatched")
|
||||
return {
|
||||
resolvedPath: null,
|
||||
@@ -353,7 +391,9 @@ const resolveCandidate = (
|
||||
if (index !== null) return index;
|
||||
}
|
||||
// Preserve recursive artifact package lookup only after legacy file/index fallbacks.
|
||||
return candidateOutcome(resolvePackagePath(entryInput, chain));
|
||||
return candidateOutcome(
|
||||
resolvePackagePath(entryInput, chain, "full", main.source === "exports"),
|
||||
);
|
||||
};
|
||||
|
||||
const candidateOutcome = (
|
||||
@@ -408,6 +448,7 @@ const packageEntry = (
|
||||
}
|
||||
| { readonly status: "missing" }
|
||||
| { readonly status: "invalid" }
|
||||
| { readonly status: "rejected"; readonly limitation: string }
|
||||
| { readonly status: "unmatched"; readonly declared: readonly string[] } => {
|
||||
try {
|
||||
const value: unknown = JSON.parse(text);
|
||||
@@ -439,26 +480,53 @@ const packageExport = (
|
||||
value: unknown,
|
||||
moduleKind: ResolveArtifactPathInput["moduleKind"],
|
||||
): PackageExportOutcome => {
|
||||
if (typeof value === "string") return packagePathValue(value);
|
||||
if (typeof value !== "object" || value === null || Array.isArray(value))
|
||||
return { status: "invalid" };
|
||||
const root = Reflect.get(value, ".") ?? value;
|
||||
if (typeof root === "string") return packagePathValue(root);
|
||||
if (typeof root !== "object" || root === null || Array.isArray(root))
|
||||
return { status: "invalid" };
|
||||
const root =
|
||||
typeof value === "object" && value !== null && Object.hasOwn(value, ".")
|
||||
? Reflect.get(value, ".")
|
||||
: value;
|
||||
const flattened = exportTargets(root, packageExportConditions(moduleKind));
|
||||
if (flattened.kind === "invalid") return { status: "invalid" };
|
||||
if (flattened.kind === "invalid")
|
||||
return { status: "rejected", limitation: flattened.limitation };
|
||||
const first =
|
||||
flattened.kind === "unmatched" ? undefined : flattened.values[0];
|
||||
return first === undefined
|
||||
? { status: "unmatched", declared: Object.keys(root) }
|
||||
? {
|
||||
status: "unmatched",
|
||||
declared:
|
||||
typeof root === "object" && root !== null ? Object.keys(root) : [],
|
||||
}
|
||||
: { status: "value", value: first };
|
||||
};
|
||||
|
||||
type ExportTargets =
|
||||
| { readonly kind: "targets"; readonly values: readonly string[] }
|
||||
| { readonly kind: "unmatched" }
|
||||
| { readonly kind: "invalid" };
|
||||
| { readonly kind: "invalid"; readonly limitation: string };
|
||||
|
||||
const invalidExportTarget = (
|
||||
value: unknown,
|
||||
reason: string,
|
||||
): ExportTargets => ({
|
||||
kind: "invalid",
|
||||
limitation: `The package exports target ${JSON.stringify(value) ?? String(value)} ${reason}.`,
|
||||
});
|
||||
|
||||
// Node checks raw target segments before URL normalization. Compare encoded
|
||||
// segment names here; full URL decoding follows array target selection.
|
||||
const forbiddenExportSegment = (value: string): string | undefined =>
|
||||
value
|
||||
.slice(2)
|
||||
.split(/[\\/]/u)
|
||||
.find((raw) => {
|
||||
const segment = raw.replace(/%[0-9a-f]{2}/giu, (encoded) =>
|
||||
String.fromCharCode(Number.parseInt(encoded.slice(1), 16)),
|
||||
);
|
||||
return (
|
||||
segment === "." ||
|
||||
segment === ".." ||
|
||||
segment.toLowerCase() === "node_modules"
|
||||
);
|
||||
});
|
||||
|
||||
/**
|
||||
* Resolve a target in Node's declared order. Explicit null blocks an active
|
||||
@@ -470,26 +538,39 @@ const exportTargets = (
|
||||
value: unknown,
|
||||
conditions: ReadonlySet<string>,
|
||||
): ExportTargets => {
|
||||
if (typeof value === "string")
|
||||
return value.startsWith("./")
|
||||
if (typeof value === "string") {
|
||||
if (!value.startsWith("./"))
|
||||
return invalidExportTarget(value, 'must start with "./"');
|
||||
const forbidden = forbiddenExportSegment(value);
|
||||
return forbidden === undefined
|
||||
? { kind: "targets", values: [value] }
|
||||
: { kind: "invalid" };
|
||||
: invalidExportTarget(
|
||||
value,
|
||||
`contains forbidden path segment ${JSON.stringify(forbidden)} before URL normalization`,
|
||||
);
|
||||
}
|
||||
if (value === null) return { kind: "targets", values: [] };
|
||||
if (Array.isArray(value)) {
|
||||
let invalid = false;
|
||||
let invalid: string | null = null;
|
||||
for (const entry of value) {
|
||||
const nested = exportTargets(entry, conditions);
|
||||
if (nested.kind === "invalid") {
|
||||
invalid = true;
|
||||
invalid = nested.limitation;
|
||||
continue;
|
||||
}
|
||||
if (nested.kind === "unmatched") continue;
|
||||
invalid = false;
|
||||
invalid = null;
|
||||
if (nested.values.length > 0) return nested;
|
||||
}
|
||||
return invalid ? { kind: "invalid" } : { kind: "targets", values: [] };
|
||||
return invalid === null
|
||||
? { kind: "targets", values: [] }
|
||||
: { kind: "invalid", limitation: invalid };
|
||||
}
|
||||
if (typeof value !== "object") return { kind: "invalid" };
|
||||
if (typeof value !== "object")
|
||||
return invalidExportTarget(
|
||||
value,
|
||||
"must be a relative string, object, array, or null",
|
||||
);
|
||||
for (const [condition, target] of Object.entries(value)) {
|
||||
if (!conditions.has(condition)) continue;
|
||||
const nested = exportTargets(target, conditions);
|
||||
@@ -516,7 +597,7 @@ const packageExportConditions = (
|
||||
|
||||
type PackageExportOutcome =
|
||||
| { readonly status: "value"; readonly value: string }
|
||||
| { readonly status: "invalid" }
|
||||
| { readonly status: "rejected"; readonly limitation: string }
|
||||
| { readonly status: "unmatched"; readonly declared: readonly string[] };
|
||||
|
||||
const packagePathValue = (
|
||||
|
||||
@@ -1,7 +1,14 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
import { resolveArtifactPathByContext } from "./JavaScriptArtifactPathResolution.js";
|
||||
import { analyzeJavaScriptApplication } from "./JavaScriptApplicationService.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../domain/javascript/javascriptApplicationAnalysis.js";
|
||||
|
||||
const filesFor = (paths: readonly string[]) =>
|
||||
new Map<string, JavaScriptArtifactFile>(
|
||||
@@ -84,3 +91,241 @@ describe("module specifier punctuation", () => {
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("ESM module URL decoding", () => {
|
||||
it.each([
|
||||
["./plain.mjs", "plain.mjs"],
|
||||
["./space%20name.mjs", "space name.mjs"],
|
||||
["./pr%C3%A9load.mjs", "préload.mjs"],
|
||||
["./name%3F%23.mjs", "name?#.mjs"],
|
||||
["./percent%2520.mjs?cache=1#v2", "percent%20.mjs"],
|
||||
])(
|
||||
"points %s imports to the file loaded by Node",
|
||||
async (specifier, target) => {
|
||||
const root = await createTestTempDirectory("rea-module-url-");
|
||||
const main = join(root, "main.mjs");
|
||||
await writeFile(
|
||||
main,
|
||||
`import value from ${JSON.stringify(specifier)}; console.log(value);`,
|
||||
);
|
||||
await writeFile(
|
||||
join(root, target),
|
||||
`export default ${JSON.stringify(target)};`,
|
||||
);
|
||||
const encoded = specifier.slice(2).split("?", 1)[0]?.split("#", 1)[0];
|
||||
if (encoded !== undefined && encoded !== target)
|
||||
await writeFile(join(root, encoded), 'export default "encoded decoy";');
|
||||
const native = await promisify(execFile)(process.execPath, [main], {
|
||||
timeout: 5_000,
|
||||
});
|
||||
expect(native.stdout.trim()).toBe(target);
|
||||
|
||||
const result = await analyzeJavaScriptApplication({
|
||||
input_path: root,
|
||||
format: "directory",
|
||||
});
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) throw result.error;
|
||||
const { graph } = javascriptApplicationAnalysisResultSchema.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
const imports = graph.edges.filter(
|
||||
(edge) =>
|
||||
edge.relation === "imports" &&
|
||||
edge.properties.specifier === specifier,
|
||||
);
|
||||
expect(imports.length).toBeGreaterThan(0);
|
||||
for (const edge of imports)
|
||||
expect(edge.properties.resolved_path).toBe(target);
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves CommonJS literal percent names", () => {
|
||||
expect(
|
||||
resolveArtifactPathByContext({
|
||||
declaredPath: "./space%20name.cjs",
|
||||
sourcePath: "app/main.js",
|
||||
context: "module-specifier",
|
||||
moduleKind: "require",
|
||||
files: filesFor(["app/space%20name.cjs", "app/space name.cjs"]),
|
||||
}),
|
||||
).toMatchObject({
|
||||
resolved_path: "app/space%20name.cjs",
|
||||
resolution_status: "resolved",
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["./invalid%.mjs", "./invalid%C3.mjs", "./nul%00.mjs"])(
|
||||
"rejects malformed or NUL URL %s",
|
||||
(declaredPath) => {
|
||||
expect(
|
||||
resolveArtifactPathByContext({
|
||||
declaredPath,
|
||||
sourcePath: "app/main.js",
|
||||
context: "module-specifier",
|
||||
moduleKind: "import",
|
||||
files: filesFor([`app/${declaredPath.slice(2)}`]),
|
||||
}),
|
||||
).toMatchObject({ resolved_path: null, resolution_status: "rejected" });
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
const packageEntryUrlCases = [
|
||||
["exports", "./actual%20file.cjs", "actual file.cjs"],
|
||||
["exports", "./actual.cjs?variant", "actual.cjs"],
|
||||
["exports", "./actual.cjs#variant", "actual.cjs"],
|
||||
["exports", "./actual%23file.cjs", "actual#file.cjs"],
|
||||
["exports", "./actual%3Ffile.cjs", "actual?file.cjs"],
|
||||
["exports", "./actual%252Ffile.cjs", "actual%2Ffile.cjs"],
|
||||
["exports", "./actual.cjs", "actual.cjs"],
|
||||
["main", "./actual%20file.cjs", "actual%20file.cjs"],
|
||||
["main", "./actual?file.cjs", "actual?file.cjs"],
|
||||
["main", "./actual#file.cjs", "actual#file.cjs"],
|
||||
] as const;
|
||||
|
||||
describe("package exports URL paths", () => {
|
||||
it.each(
|
||||
(["import", "require"] as const).flatMap((moduleKind) =>
|
||||
packageEntryUrlCases.map(([field, declaredPath, target]) => ({
|
||||
moduleKind,
|
||||
field,
|
||||
declaredPath,
|
||||
target,
|
||||
})),
|
||||
),
|
||||
)(
|
||||
"matches Node $moduleKind for $field $declaredPath",
|
||||
async ({ moduleKind, field, declaredPath, target }) => {
|
||||
const root = await createTestTempDirectory("rea-package-export-url-");
|
||||
const packageRoot = join(root, "node_modules", "fixture");
|
||||
await mkdir(packageRoot, { recursive: true });
|
||||
await writeFile(
|
||||
join(packageRoot, "package.json"),
|
||||
JSON.stringify({
|
||||
name: "fixture",
|
||||
[field]: declaredPath,
|
||||
}),
|
||||
);
|
||||
await writeFile(
|
||||
join(packageRoot, target),
|
||||
`module.exports=${JSON.stringify(target)};`,
|
||||
);
|
||||
const literal = declaredPath.slice(2).split("?", 1)[0]?.split("#", 1)[0];
|
||||
if (literal !== undefined && literal !== target)
|
||||
await writeFile(
|
||||
join(packageRoot, literal),
|
||||
'module.exports="literal decoy";',
|
||||
);
|
||||
const main = join(
|
||||
root,
|
||||
moduleKind === "import" ? "main.mjs" : "main.cjs",
|
||||
);
|
||||
await writeFile(
|
||||
main,
|
||||
moduleKind === "import"
|
||||
? 'import value from "fixture"; console.log(value);'
|
||||
: 'console.log(require("fixture"));',
|
||||
);
|
||||
const native = await promisify(execFile)(process.execPath, [main], {
|
||||
timeout: 5_000,
|
||||
});
|
||||
expect(native.stdout.trim()).toBe(target);
|
||||
const result = await analyzeJavaScriptApplication({
|
||||
input_path: root,
|
||||
format: "directory",
|
||||
});
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) throw result.error;
|
||||
const { graph } = javascriptApplicationAnalysisResultSchema.parse(
|
||||
result.value.normalized_result,
|
||||
);
|
||||
const imports = graph.edges.filter(
|
||||
(edge) =>
|
||||
edge.relation === "imports" &&
|
||||
edge.properties.specifier === "fixture",
|
||||
);
|
||||
expect(imports.length).toBeGreaterThan(0);
|
||||
for (const edge of imports)
|
||||
expect(edge.properties.resolved_path).toBe(
|
||||
`node_modules/fixture/${target}`,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it.each(
|
||||
(["import", "require"] as const).flatMap((moduleKind) =>
|
||||
[
|
||||
"./actual%file.cjs",
|
||||
"./actual%E0%A4file.cjs",
|
||||
"./actual%00file.cjs",
|
||||
"./actual%2Ffile.cjs",
|
||||
"./actual%5Cfile.cjs",
|
||||
"./actual%2Efile.cjs",
|
||||
"./../../../../escape.cjs",
|
||||
].map((declaredPath) => ({ moduleKind, declaredPath })),
|
||||
),
|
||||
)(
|
||||
"rejects $moduleKind URL $declaredPath without borrowing a literal decoy",
|
||||
({ moduleKind, declaredPath }) => {
|
||||
const packagePath = "app/node_modules/fixture/package.json";
|
||||
const files = filesFor([
|
||||
packagePath,
|
||||
`app/node_modules/fixture/${declaredPath.slice(2)}`,
|
||||
]);
|
||||
const metadata = files.get(packagePath);
|
||||
if (metadata === undefined) throw new Error("Missing fixture metadata");
|
||||
files.set(packagePath, {
|
||||
...metadata,
|
||||
kind: "package-json",
|
||||
text: {
|
||||
included: true,
|
||||
value: JSON.stringify({ exports: declaredPath }),
|
||||
},
|
||||
});
|
||||
expect(
|
||||
resolveArtifactPathByContext({
|
||||
declaredPath: "fixture",
|
||||
sourcePath: "app/main.js",
|
||||
context: "module-specifier",
|
||||
moduleKind,
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ resolved_path: null, resolution_status: "rejected" });
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["import", "require"] as const)(
|
||||
"keeps %s exports inside their inventoried container",
|
||||
(moduleKind) => {
|
||||
const packagePath = "app/node_modules/fixture/package.json";
|
||||
const target = "app/node_modules/fixture/actual file.cjs";
|
||||
const files = filesFor([packagePath, target]);
|
||||
const metadata = files.get(packagePath);
|
||||
const crossContainer = files.get(target);
|
||||
if (metadata === undefined || crossContainer === undefined)
|
||||
throw new Error("Missing fixture files");
|
||||
files.set(packagePath, {
|
||||
...metadata,
|
||||
kind: "package-json",
|
||||
text: {
|
||||
included: true,
|
||||
value: '{"exports":"./actual%20file.cjs"}',
|
||||
},
|
||||
});
|
||||
files.set(target, {
|
||||
...crossContainer,
|
||||
container_sha256: "c".repeat(64),
|
||||
});
|
||||
expect(
|
||||
resolveArtifactPathByContext({
|
||||
declaredPath: "fixture",
|
||||
sourcePath: "app/main.js",
|
||||
context: "module-specifier",
|
||||
moduleKind,
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ resolved_path: null, resolution_status: "not-found" });
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
import { z } from "incur";
|
||||
|
||||
import { runProviderAnalysis } from "../composition/directAnalysis.js";
|
||||
import { CLI_COMMANDS } from "../cliCommandNames.js";
|
||||
import { parseCliJsonInput } from "../cliJsonInput.js";
|
||||
import { logCliCommand } from "../cliLogging.js";
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { projectInputIssues } from "../domain/inputIssueProjection.js";
|
||||
import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||
import { nativeCallObservationInputSchema } from "../domain/native/nativeCallObservation.js";
|
||||
import type { Logger } from "../logger.js";
|
||||
import type { CliInstance } from "./types.js";
|
||||
|
||||
const OPERATION = "observe_native_calls";
|
||||
|
||||
/** `observe-native-calls <path> <input-json>`: the MCP input object as JSON. */
|
||||
export const registerNativeCallCommands = (
|
||||
cli: CliInstance,
|
||||
logger: Logger,
|
||||
): void => {
|
||||
cli.command(CLI_COMMANDS.observeNativeCalls, {
|
||||
description:
|
||||
"Launch a Mach-O under LLDB and record entries into selected functions or Objective-C methods",
|
||||
args: z.object({
|
||||
path: z.string().describe("Mach-O or app path"),
|
||||
inputJson: z
|
||||
.string()
|
||||
.describe(
|
||||
"Inline JSON or a JSON file path: breakpoints, arguments, environment, duration_ms, max_events, argument_registers, backtrace_frames",
|
||||
),
|
||||
}),
|
||||
run: ({ args }) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.observeNativeCalls, async () => {
|
||||
const input = await parseCliJsonInput(args.inputJson, OPERATION);
|
||||
if (!input.ok) return input.error;
|
||||
const parsed = nativeCallObservationInputSchema.safeParse(input.value);
|
||||
if (!parsed.success)
|
||||
return {
|
||||
error: "Analysis failed",
|
||||
...projectAnalysisError(
|
||||
new AnalysisInputError(
|
||||
OPERATION,
|
||||
undefined,
|
||||
projectInputIssues(parsed.error.issues, input.value),
|
||||
),
|
||||
),
|
||||
};
|
||||
return runProviderAnalysis(
|
||||
args.path,
|
||||
OPERATION,
|
||||
z.record(z.string(), jsonValueSchema).parse(parsed.data),
|
||||
logger,
|
||||
);
|
||||
}),
|
||||
});
|
||||
};
|
||||
@@ -20,6 +20,7 @@ import { swiftSymbolsSchema } from "../contracts/native/nativeToolContracts.js";
|
||||
import type { Logger } from "../logger.js";
|
||||
import { isReferenceSourceImportCliFailure } from "./referenceSourceImportStatus.js";
|
||||
import type { CliInstance } from "./types.js";
|
||||
import { registerNativeCallCommands } from "./nativeCallCommands.js";
|
||||
|
||||
export const registerUtilityCommands = (
|
||||
cli: CliInstance,
|
||||
@@ -28,6 +29,7 @@ export const registerUtilityCommands = (
|
||||
): void => {
|
||||
registerCapabilityCommands(cli, logger);
|
||||
registerNativeCommands(cli, logger);
|
||||
registerNativeCallCommands(cli, logger);
|
||||
for (const [command, operation] of [
|
||||
[CLI_COMMANDS.observeNativeUi, "observe_native_ui"],
|
||||
[CLI_COMMANDS.captureNativeUiScenario, "capture_native_ui_scenario"],
|
||||
|
||||
@@ -52,6 +52,7 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
importManagedReconstruction: "import-managed-reconstruction",
|
||||
projectManagedApplicationGraph: "project-managed-application-graph",
|
||||
observeNativeUi: "observe-native-ui",
|
||||
observeNativeCalls: "observe-native-calls",
|
||||
captureNativeUiScenario: "capture-native-ui-scenario",
|
||||
inspectMacho: "inspect-macho",
|
||||
inspectSignature: "inspect-signature",
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { z } from "zod";
|
||||
import { nativeCallObservationInputSchema } from "../../domain/native/nativeCallObservation.js";
|
||||
import {
|
||||
nativeUiObservationInputSchema,
|
||||
nativeUiScenarioInputSchema,
|
||||
@@ -22,6 +23,19 @@ const examples: Readonly<Record<string, Readonly<Record<string, unknown>>>> = {
|
||||
steps: [{ kind: "wait", milliseconds: 100 }],
|
||||
},
|
||||
demangle_swift: { symbols: ["$s4Test3fooyyF"] },
|
||||
observe_native_calls: {
|
||||
breakpoints: [
|
||||
{
|
||||
kind: "objc-method",
|
||||
class_name: "NSURLSession",
|
||||
selector: "dataTaskWithRequest:completionHandler:",
|
||||
},
|
||||
{ kind: "function", name: "open", module: "libsystem_kernel.dylib" },
|
||||
],
|
||||
arguments: ["--help"],
|
||||
duration_ms: 5000,
|
||||
backtrace_frames: 4,
|
||||
},
|
||||
};
|
||||
|
||||
const native = <Name extends string, Schema extends z.ZodObject>(
|
||||
@@ -71,6 +85,11 @@ export const NATIVE_TOOL_CONTRACTS = [
|
||||
"Run selected-element press, increment/decrement scroll, AXValue text entry and bounded wait steps in one exact native app window. Returns ordered before/after captures and action/capture failures. Actions can change app data, network activity and persistent state; application state is left as-is and restoration is not attempted.",
|
||||
nativeUiScenarioInputSchema,
|
||||
),
|
||||
native(
|
||||
"observe_native_calls",
|
||||
"Launch the active Mach-O as an owned process under LLDB and record each entry into caller-selected functions or Objective-C methods: thread, module, symbol, load and file address, raw argument registers, selector, receiver class and optional caller frames. Every hit auto-continues. No expression is evaluated and no other process is attached. The process is killed when max_events or duration_ms is reached; exit status, captured stdout/stderr, unresolved breakpoints and limitations are returned inline. The process runs with the current user's permissions and may change files, show UI or use the network. A hardened-runtime target needs the get-task-allow entitlement.",
|
||||
nativeCallObservationInputSchema,
|
||||
),
|
||||
native(
|
||||
"inspect_macho",
|
||||
"Inspect Mach-O slices, load commands, imports, exports, dependencies, build metadata, segments, sections, permissions, and exact command provenance without launching Hopper.",
|
||||
|
||||
@@ -180,6 +180,15 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
|
||||
mayDiscardData: true,
|
||||
idempotent: false,
|
||||
}),
|
||||
// The owned process can change files, show UI and use the network.
|
||||
observe_native_calls: effects({
|
||||
mutatesSession: true,
|
||||
writesFilesystem: true,
|
||||
launchesProcess: true,
|
||||
accessesNetwork: true,
|
||||
changesUiState: true,
|
||||
idempotent: false,
|
||||
}),
|
||||
inspect_macho: nativeEvidence,
|
||||
inspect_signature: nativeEvidence,
|
||||
inspect_plist: nativeEvidence,
|
||||
|
||||
@@ -4,6 +4,7 @@ import { nativeLoadImageSchema } from "../domain/native/nativeLoadImage.js";
|
||||
import { nativeUiResultSchema } from "../domain/native/nativeUiObservation.js";
|
||||
import { nativeValueTraceSchema } from "../domain/native/nativeValueTrace.js";
|
||||
import { nativeDataTypeSchema } from "../domain/native/nativeDataType.js";
|
||||
import { nativeCallObservationResultSchema } from "../domain/native/nativeCallObservation.js";
|
||||
import {
|
||||
nativeInstructionSchema,
|
||||
nativeCallTargetsSchema,
|
||||
@@ -323,6 +324,7 @@ export const enhancedOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
|
||||
export const nativeOutputSchemas: Readonly<Record<string, z.ZodObject>> = {
|
||||
observe_native_ui: resultOf(nativeUiResultSchema),
|
||||
capture_native_ui_scenario: resultOf(nativeUiResultSchema),
|
||||
observe_native_calls: resultOf(nativeCallObservationResultSchema),
|
||||
inspect_macho: resultOf(inspectMachoSchema),
|
||||
inspect_signature: resultOf(inspectSignatureSchema),
|
||||
inspect_plist: resultOf(inspectPlistSchema),
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { htmlArtifactReferences } from "./htmlArtifactReferences.js";
|
||||
|
||||
it.each([
|
||||
['<script data-src="decoy.js"></script>', []],
|
||||
['<script x:src="decoy.js"></script>', []],
|
||||
[`<script title='src="decoy.js"'></script>`, []],
|
||||
["<script src=actual.js></script>", ["actual.js"]],
|
||||
['<script title=">" src="actual.js"></script>', ["actual.js"]],
|
||||
['<SCRIPT SRC="actual.js"></SCRIPT>', ["actual.js"]],
|
||||
["<script src='actual.js'></script>", ["actual.js"]],
|
||||
['<script data-src="decoy.js" src="actual.js"></script>', ["actual.js"]],
|
||||
['<script src="actual.js" src="decoy.js"></script>', ["actual.js"]],
|
||||
['<script src="" src="decoy.js"></script>', []],
|
||||
['<script src="actual.js"></script>', ["actual.js"]],
|
||||
['<!-- <script src="decoy.js"></script> -->', []],
|
||||
['<textarea><script src="decoy.js"></script></textarea>', []],
|
||||
['<style><script src="decoy.js"></script></style>', []],
|
||||
[`<script>const text = '<script src="decoy.js">';</script>`, []],
|
||||
[
|
||||
'<template><base href="decoy/"><script src="decoy.js"></script></template>',
|
||||
[],
|
||||
],
|
||||
['<svg><script src="decoy.js"></script></svg>', []],
|
||||
[
|
||||
'<svg><foreignObject><script src="actual.js"></script></foreignObject></svg>',
|
||||
["actual.js"],
|
||||
],
|
||||
// The existing extraction contract does not classify script MIME types.
|
||||
['<script type="application/json" src="actual.js"></script>', ["actual.js"]],
|
||||
])("extracts document references from %s", (text, expected) => {
|
||||
expect(
|
||||
htmlArtifactReferences(text).scripts.map(({ scriptPath }) => scriptPath),
|
||||
).toEqual(expected);
|
||||
});
|
||||
|
||||
it("retains exact opening-tag offsets rather than the script body", () => {
|
||||
const opening = '<script\n title=">" src=actual.js>';
|
||||
const prefix = "<!doctype html>\n\n";
|
||||
const text = `${prefix}${opening}ignored body</script>`;
|
||||
expect(htmlArtifactReferences(text)).toEqual({
|
||||
baseHref: null,
|
||||
scripts: [
|
||||
{
|
||||
scriptPath: "actual.js",
|
||||
startOffset: prefix.length,
|
||||
endOffset: prefix.length + opening.length,
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
['<base data-href="decoy/"><base href=actual/>', "actual/"],
|
||||
['<base href="actual/" href="decoy/"><base href="later/">', "actual/"],
|
||||
['<base href=""><base href="decoy/">', ""],
|
||||
['<template><base href="decoy/"></template><base href="actual/">', "actual/"],
|
||||
['<!-- <base href="decoy/"> --><base title=">" href="actual/">', "actual/"],
|
||||
])("selects the first real base href from %s", (text, expected) => {
|
||||
expect(htmlArtifactReferences(text).baseHref).toBe(expected);
|
||||
});
|
||||
|
||||
it("traverses deep valid document content without recursive stack growth", () => {
|
||||
const depth = 12_000;
|
||||
const text = `${"<div>".repeat(depth)}<script src=actual.js></script>${"</div>".repeat(depth)}`;
|
||||
expect(
|
||||
htmlArtifactReferences(text).scripts.map(({ scriptPath }) => scriptPath),
|
||||
).toEqual(["actual.js"]);
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
import {
|
||||
defaultTreeAdapter,
|
||||
parse,
|
||||
type DefaultTreeAdapterTypes,
|
||||
} from "parse5";
|
||||
|
||||
interface HtmlScriptReference {
|
||||
readonly scriptPath: string;
|
||||
readonly startOffset: number;
|
||||
readonly endOffset: number;
|
||||
}
|
||||
|
||||
/** Extract inert HTML script references and the first document base href. */
|
||||
export const htmlArtifactReferences = (
|
||||
text: string,
|
||||
): {
|
||||
readonly baseHref: string | null;
|
||||
readonly scripts: readonly HtmlScriptReference[];
|
||||
} => {
|
||||
const pending: DefaultTreeAdapterTypes.Node[] = [
|
||||
parse(text, { sourceCodeLocationInfo: true }),
|
||||
];
|
||||
const scripts: HtmlScriptReference[] = [];
|
||||
let baseHref: string | null = null;
|
||||
while (pending.length > 0) {
|
||||
const node = pending.pop();
|
||||
if (node === undefined) continue;
|
||||
if (
|
||||
defaultTreeAdapter.isElementNode(node) &&
|
||||
node.namespaceURI === "http://www.w3.org/1999/xhtml"
|
||||
) {
|
||||
if (node.tagName === "base" && baseHref === null) {
|
||||
const href = node.attrs.find(({ name }) => name === "href");
|
||||
if (href !== undefined) baseHref = href.value;
|
||||
}
|
||||
if (node.tagName === "script") {
|
||||
const src = node.attrs.find(({ name }) => name === "src");
|
||||
const location = node.sourceCodeLocation?.startTag;
|
||||
if (src !== undefined && src.value !== "" && location !== undefined) {
|
||||
scripts.push({
|
||||
scriptPath: src.value,
|
||||
startOffset: location.startOffset,
|
||||
endOffset: location.endOffset,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
// Template content belongs to a separate inert document fragment. Only
|
||||
// traverse the actual document children, without recursion on deep input.
|
||||
if ("childNodes" in node) {
|
||||
for (const child of node.childNodes.toReversed()) pending.push(child);
|
||||
}
|
||||
}
|
||||
return { baseHref, scripts };
|
||||
};
|
||||
@@ -0,0 +1,198 @@
|
||||
import { z } from "zod";
|
||||
|
||||
const hexSchema = z.string().regex(/^0x[0-9a-f]+$/u);
|
||||
/** Objective-C class names and selectors never contain spaces or brackets. */
|
||||
const objcName = z
|
||||
.string()
|
||||
.min(1)
|
||||
.regex(/^[^\s[\]]+$/u, "Use the bare name without spaces or brackets");
|
||||
|
||||
const functionBreakpoint = {
|
||||
kind: z.literal("function"),
|
||||
name: z
|
||||
.string()
|
||||
.min(1)
|
||||
.describe(
|
||||
"Exact symbol name, such as `open`, a C++ or mangled Swift symbol, or `-[NSString length]`.",
|
||||
),
|
||||
};
|
||||
|
||||
const objcMethodBreakpoint = {
|
||||
kind: z.literal("objc-method"),
|
||||
selector: objcName.describe(
|
||||
"Selector such as `dataTaskWithRequest:completionHandler:`.",
|
||||
),
|
||||
};
|
||||
|
||||
/** One function or Objective-C method whose entries are observed. */
|
||||
export const nativeCallBreakpointSchema = z.discriminatedUnion("kind", [
|
||||
z.strictObject({
|
||||
...functionBreakpoint,
|
||||
module: z
|
||||
.string()
|
||||
.min(1)
|
||||
.optional()
|
||||
.describe(
|
||||
"Image file name that defines the symbol, such as `libsystem_kernel.dylib`; omit to match every image.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...objcMethodBreakpoint,
|
||||
class_name: objcName
|
||||
.optional()
|
||||
.describe("Implementing class; omit to match every class."),
|
||||
method_type: z.enum(["instance", "class", "any"]).default("any"),
|
||||
}),
|
||||
]);
|
||||
|
||||
/**
|
||||
* Launch the active Mach-O as an owned process under LLDB and observe calls.
|
||||
* Duration and event bounds keep runtime observation finite.
|
||||
*/
|
||||
export const nativeCallObservationInputSchema = z.strictObject({
|
||||
breakpoints: z.array(nativeCallBreakpointSchema).min(1),
|
||||
arguments: z.array(z.string()).default([]),
|
||||
environment: z
|
||||
.record(
|
||||
z
|
||||
.string()
|
||||
.min(1)
|
||||
.regex(/^[^=\0]+$/u, "Environment names cannot contain '=' or NUL"),
|
||||
z.string().regex(/^[^\0]*$/u, "Environment values cannot contain NUL"),
|
||||
)
|
||||
.default({})
|
||||
.describe("Overrides on top of the environment REA runs with."),
|
||||
working_directory: z.string().min(1).optional(),
|
||||
duration_ms: z
|
||||
.number()
|
||||
.int()
|
||||
.min(100)
|
||||
.max(600_000)
|
||||
.default(10_000)
|
||||
.describe(
|
||||
"Observation window; the process is killed if it is still running.",
|
||||
),
|
||||
max_events: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(100_000)
|
||||
.default(1_000)
|
||||
.describe(
|
||||
"Stop observing and kill the process after this many recorded entries.",
|
||||
),
|
||||
argument_registers: z
|
||||
.number()
|
||||
.int()
|
||||
.min(0)
|
||||
.max(8)
|
||||
.default(4)
|
||||
.describe(
|
||||
"Integer argument registers to record per entry (arm64 has 8, x86_64 has 6).",
|
||||
),
|
||||
backtrace_frames: z
|
||||
.number()
|
||||
.int()
|
||||
.min(0)
|
||||
.max(64)
|
||||
.default(0)
|
||||
.describe("Caller frames to record per entry."),
|
||||
});
|
||||
|
||||
export type NativeCallObservationInput = z.infer<
|
||||
typeof nativeCallObservationInputSchema
|
||||
>;
|
||||
|
||||
/** A code address with the image and symbol LLDB attributes to it. */
|
||||
export const nativeCodeLocationSchema = z.strictObject({
|
||||
load_address: hexSchema,
|
||||
file_address: hexSchema.nullable(),
|
||||
module: z.string().nullable(),
|
||||
module_path: z.string().nullable(),
|
||||
symbol: z.string().nullable(),
|
||||
});
|
||||
|
||||
/** One observed entry into a requested function or method. */
|
||||
export const nativeCallEventSchema = z.strictObject({
|
||||
sequence: z.number().int().nonnegative(),
|
||||
elapsed_ms: z.number().nonnegative(),
|
||||
thread_id: z.number().int().nonnegative(),
|
||||
breakpoint_index: z.number().int().nonnegative(),
|
||||
...nativeCodeLocationSchema.shape,
|
||||
/** Dynamic class of an instance-method receiver, read without running code. */
|
||||
receiver_class: z.string().nullable(),
|
||||
/** Selector string read from the `_cmd` register of an Objective-C method. */
|
||||
selector: z.string().nullable(),
|
||||
registers: z.array(z.strictObject({ name: z.string(), value: hexSchema })),
|
||||
backtrace: z.array(nativeCodeLocationSchema),
|
||||
});
|
||||
|
||||
const capturedOutputSchema = z.strictObject({
|
||||
text: z.string(),
|
||||
bytes: z.number().int().nonnegative(),
|
||||
truncated: z.boolean(),
|
||||
});
|
||||
|
||||
/** Observed calls, the process lifecycle, and breakpoint resolution. */
|
||||
export const nativeCallObservationResultSchema = z.strictObject({
|
||||
target: z.strictObject({
|
||||
path: z.string(),
|
||||
sha256: z.string().regex(/^[0-9a-f]{64}$/u),
|
||||
architecture: z.string(),
|
||||
arguments: z.array(z.string()),
|
||||
working_directory: z.string().nullable(),
|
||||
}),
|
||||
debugger: z.strictObject({
|
||||
path: z.string(),
|
||||
sha256: z.string().regex(/^[0-9a-f]{64}$/u),
|
||||
version: z.string().nullable(),
|
||||
}),
|
||||
process: z.strictObject({
|
||||
pid: z.number().int().positive(),
|
||||
outcome: z.enum([
|
||||
"exited",
|
||||
"duration-elapsed",
|
||||
"event-limit",
|
||||
"stop-limit",
|
||||
]),
|
||||
exit_status: z.number().int().nullable(),
|
||||
exit_description: z.string().nullable(),
|
||||
/** REA confirmed the process is gone when observation ended. */
|
||||
terminated: z.boolean(),
|
||||
elapsed_ms: z.number().nonnegative(),
|
||||
stdout: capturedOutputSchema,
|
||||
stderr: capturedOutputSchema,
|
||||
/** Signal and exception stops LLDB reported; the process continued past each. */
|
||||
other_stops: z.array(z.string()),
|
||||
}),
|
||||
breakpoints: z.array(
|
||||
z.strictObject({
|
||||
index: z.number().int().nonnegative(),
|
||||
request: z.discriminatedUnion("kind", [
|
||||
z.strictObject({
|
||||
...functionBreakpoint,
|
||||
module: z.string().nullable(),
|
||||
}),
|
||||
z.strictObject({
|
||||
...objcMethodBreakpoint,
|
||||
class_name: z.string().nullable(),
|
||||
method_type: z.enum(["instance", "class", "any"]),
|
||||
}),
|
||||
]),
|
||||
location_count: z.number().int().nonnegative(),
|
||||
locations: z.array(nativeCodeLocationSchema),
|
||||
}),
|
||||
),
|
||||
events: z.array(nativeCallEventSchema),
|
||||
coverage: z.strictObject({
|
||||
status: z.enum(["complete", "partial"]),
|
||||
event_limit_reached: z.boolean(),
|
||||
/** Requests that matched no code in any image loaded while observing. */
|
||||
unresolved_breakpoints: z.array(z.number().int().nonnegative()),
|
||||
}),
|
||||
limitations: z.array(z.string()),
|
||||
});
|
||||
|
||||
export type NativeCallObservationResult = z.infer<
|
||||
typeof nativeCallObservationResultSchema
|
||||
>;
|
||||
@@ -5,5 +5,5 @@ export const PACKAGE_METADATA = {
|
||||
serverSdkVersion: "2.3.1",
|
||||
clientSdkVersion: "2.3.1",
|
||||
coreSdkVersion: "2.3.1",
|
||||
skillVersion: "29",
|
||||
skillVersion: "30",
|
||||
} as const;
|
||||
|
||||
@@ -124,7 +124,8 @@ const ALLOWED_TOOLS = new Set([
|
||||
"vtool",
|
||||
]);
|
||||
|
||||
const resolveXcrunTool = async (
|
||||
/** Locate an Xcode tool through `xcrun --find` and pin its executable digest. */
|
||||
export const resolveXcrunTool = async (
|
||||
tool: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<ResolvedTool, NativeCommandFailure>> => {
|
||||
|
||||
@@ -0,0 +1,400 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { open, readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { z } from "zod";
|
||||
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisTimeoutError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import {
|
||||
nativeCallEventSchema,
|
||||
nativeCodeLocationSchema,
|
||||
type NativeCallObservationInput,
|
||||
} from "../domain/native/nativeCallObservation.js";
|
||||
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import { safeParseJson } from "../domain/safeJson.js";
|
||||
import { execFileOutput } from "../process/ExecFileOutput.js";
|
||||
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
|
||||
import { resolveXcrunTool, type ResolvedTool } from "./CommandRunner.js";
|
||||
|
||||
const OPERATION = "observe_native_calls";
|
||||
const PROVIDER = "native-macos";
|
||||
/** Captured target output kept per stream; the rest is counted, not stored. */
|
||||
const MAX_OUTPUT_BYTES = 1024 * 1024;
|
||||
/** LLDB startup, symbol loading and teardown beyond the observation window. */
|
||||
const DEADLINE_GRACE_MS = 60_000;
|
||||
const DIAGNOSTIC_BYTES = 16 * 1024;
|
||||
|
||||
const BRIDGE = fileURLToPath(
|
||||
new URL("../../bridge/native/rea_lldb_tracer.py", import.meta.url),
|
||||
);
|
||||
|
||||
/** JSON the LLDB bridge writes after a completed run. */
|
||||
const tracedSchema = z.strictObject({
|
||||
status: z.literal("traced"),
|
||||
version: z.string(),
|
||||
pid: z.number().int().positive(),
|
||||
outcome: z.enum(["exited", "duration-elapsed", "event-limit", "stop-limit"]),
|
||||
exit_status: z.number().int().nullable(),
|
||||
exit_description: z.string().nullable(),
|
||||
killed: z.boolean(),
|
||||
terminated: z.boolean(),
|
||||
elapsed_ms: z.number().nonnegative(),
|
||||
breakpoints: z.array(
|
||||
z.strictObject({
|
||||
index: z.number().int().nonnegative(),
|
||||
location_count: z.number().int().nonnegative(),
|
||||
locations: z.array(nativeCodeLocationSchema),
|
||||
}),
|
||||
),
|
||||
events: z.array(nativeCallEventSchema),
|
||||
other_stops: z.array(z.string()),
|
||||
});
|
||||
|
||||
const tracerOutputSchema = z.discriminatedUnion("status", [
|
||||
tracedSchema,
|
||||
z.strictObject({
|
||||
status: z.enum(["target-error", "launch-error", "tracer-error"]),
|
||||
error: z.string().nullable(),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Bounded text of one target output stream. */
|
||||
export interface CapturedOutput {
|
||||
readonly text: string;
|
||||
readonly bytes: number;
|
||||
readonly truncated: boolean;
|
||||
}
|
||||
|
||||
/** One completed LLDB observation, before projection into the result contract. */
|
||||
export interface NativeCallTrace {
|
||||
readonly debugger: ResolvedTool & { readonly version: string | null };
|
||||
readonly run: z.infer<typeof tracedSchema>;
|
||||
readonly stdout: CapturedOutput;
|
||||
readonly stderr: CapturedOutput;
|
||||
/** REA confirmed the traced process no longer exists. */
|
||||
readonly terminated: boolean;
|
||||
}
|
||||
|
||||
/** Launch one owned process under a debugger and record calls. */
|
||||
export interface NativeCallTracer {
|
||||
trace(
|
||||
request: {
|
||||
readonly executable: string;
|
||||
readonly architecture: string;
|
||||
readonly input: NativeCallObservationInput;
|
||||
},
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<NativeCallTrace, AnalysisError>>;
|
||||
}
|
||||
|
||||
/** Remediation for a target LLDB may not debug. */
|
||||
const ATTACH_REMEDIATION =
|
||||
"Allow debugging for this user (Developer Tools access: `DevToolsSecurity -enable`, or membership in the _developer group). A hardened-runtime target is debuggable only with the com.apple.security.get-task-allow entitlement; see inspect_signature's debugger-attach facet.";
|
||||
|
||||
/** Production tracer: `lldb --batch` running the REA LLDB bridge. */
|
||||
export class LldbCallTracer implements NativeCallTracer {
|
||||
async trace(
|
||||
request: Parameters<NativeCallTracer["trace"]>[0],
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<NativeCallTrace, AnalysisError>> {
|
||||
const lldb = await resolveXcrunTool("lldb", signal);
|
||||
if (!lldb.ok)
|
||||
return err(
|
||||
lldb.error.reason === "cancelled"
|
||||
? new AnalysisCancelledError(OPERATION)
|
||||
: new AnalysisCapabilityUnavailableError(
|
||||
PROVIDER,
|
||||
OPERATION,
|
||||
"LLDB from Xcode or the Command Line Tools is not available",
|
||||
),
|
||||
);
|
||||
const root = await PrivateRuntimeRoot.create({ prefix: "rea-lldb-" });
|
||||
try {
|
||||
return await this.#run(lldb.value, root.path, request, signal);
|
||||
} finally {
|
||||
await root.close();
|
||||
}
|
||||
}
|
||||
|
||||
async #run(
|
||||
lldb: ResolvedTool,
|
||||
directory: string,
|
||||
request: Parameters<NativeCallTracer["trace"]>[0],
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<NativeCallTrace, AnalysisError>> {
|
||||
const paths = {
|
||||
config: join(directory, "config.json"),
|
||||
result: join(directory, "result.json"),
|
||||
pid: join(directory, "pid"),
|
||||
stdout: join(directory, "stdout"),
|
||||
stderr: join(directory, "stderr"),
|
||||
};
|
||||
const { input } = request;
|
||||
await writeFile(
|
||||
paths.config,
|
||||
JSON.stringify({
|
||||
executable: request.executable,
|
||||
architecture: request.architecture,
|
||||
arguments: input.arguments,
|
||||
environment: input.environment,
|
||||
working_directory: input.working_directory ?? null,
|
||||
breakpoints: input.breakpoints.map((breakpoint) =>
|
||||
breakpoint.kind === "function"
|
||||
? { ...breakpoint, module: breakpoint.module ?? null }
|
||||
: { ...breakpoint, class_name: breakpoint.class_name ?? null },
|
||||
),
|
||||
duration_ms: input.duration_ms,
|
||||
max_events: input.max_events,
|
||||
argument_registers: input.argument_registers,
|
||||
backtrace_frames: input.backtrace_frames,
|
||||
stdout_path: paths.stdout,
|
||||
stderr_path: paths.stderr,
|
||||
result_path: paths.result,
|
||||
pid_path: paths.pid,
|
||||
}),
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
const exited = await runLldb(
|
||||
lldb.path,
|
||||
[
|
||||
"--batch",
|
||||
"--no-lldbinit",
|
||||
"--no-use-colors",
|
||||
"-o",
|
||||
`command script import ${quote(BRIDGE)}`,
|
||||
"-o",
|
||||
`rea_trace ${paths.config}`,
|
||||
],
|
||||
{
|
||||
deadlineMs: input.duration_ms + DEADLINE_GRACE_MS,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
},
|
||||
);
|
||||
const terminated = await ensureTerminated(paths.pid, request.executable);
|
||||
if (exited.kind === "cancelled")
|
||||
return err(new AnalysisCancelledError(OPERATION));
|
||||
if (exited.kind === "timeout")
|
||||
return err(
|
||||
new AnalysisTimeoutError(
|
||||
OPERATION,
|
||||
input.duration_ms + DEADLINE_GRACE_MS,
|
||||
),
|
||||
);
|
||||
const output = await readTracerOutput(paths.result);
|
||||
if (output === undefined)
|
||||
return err(
|
||||
new ProviderAdapterError(PROVIDER, OPERATION, {
|
||||
diagnostics: {
|
||||
reason: "LLDB exited without a tracer result",
|
||||
exit_code: exited.exitCode,
|
||||
lldb_output: exited.output,
|
||||
},
|
||||
}),
|
||||
);
|
||||
if (output.status !== "traced") return err(tracerFailure(output));
|
||||
return ok({
|
||||
debugger: { ...lldb, version: output.version },
|
||||
run: output,
|
||||
stdout: await capturedOutput(paths.stdout),
|
||||
stderr: await capturedOutput(paths.stderr),
|
||||
terminated,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/** LLDB command arguments are quoted; the bridge path is REA's own install path. */
|
||||
const quote = (value: string): string =>
|
||||
`"${value.replaceAll("\\", "\\\\").replaceAll('"', '\\"')}"`;
|
||||
|
||||
const tracerFailure = (output: {
|
||||
readonly status: "target-error" | "launch-error" | "tracer-error";
|
||||
readonly error: string | null;
|
||||
}): AnalysisError => {
|
||||
const message = output.error ?? "no LLDB error text";
|
||||
if (output.status === "tracer-error")
|
||||
return new ProviderAdapterError(PROVIDER, OPERATION, {
|
||||
diagnostics: { reason: `LLDB bridge failed: ${message}` },
|
||||
});
|
||||
if (output.status === "target-error")
|
||||
return new AnalysisCapabilityUnavailableError(
|
||||
PROVIDER,
|
||||
OPERATION,
|
||||
`lldb-target-unloadable: ${message}`,
|
||||
);
|
||||
// debugserver reports denied task access as an attach failure.
|
||||
return /attach|not allowed|permission|denied|debugserver/iu.test(message)
|
||||
? new AnalysisCapabilityUnavailableError(
|
||||
PROVIDER,
|
||||
OPERATION,
|
||||
`debugger-attach-denied: ${message}`,
|
||||
{ userMessage: ATTACH_REMEDIATION },
|
||||
)
|
||||
: new AnalysisCapabilityUnavailableError(
|
||||
PROVIDER,
|
||||
OPERATION,
|
||||
`launch-failed: ${message}`,
|
||||
);
|
||||
};
|
||||
|
||||
const readTracerOutput = async (
|
||||
path: string,
|
||||
): Promise<z.infer<typeof tracerOutputSchema> | undefined> => {
|
||||
let text: string;
|
||||
try {
|
||||
text = await readFile(path, "utf8");
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
const parsed = safeParseJson(text);
|
||||
if (!parsed.ok) return undefined;
|
||||
const output = tracerOutputSchema.safeParse(parsed.value);
|
||||
return output.success ? output.data : undefined;
|
||||
};
|
||||
|
||||
const capturedOutput = async (path: string): Promise<CapturedOutput> => {
|
||||
let handle;
|
||||
try {
|
||||
handle = await open(path, "r");
|
||||
} catch {
|
||||
return { text: "", bytes: 0, truncated: false };
|
||||
}
|
||||
try {
|
||||
const { size } = await handle.stat();
|
||||
const buffer = Buffer.alloc(Math.min(size, MAX_OUTPUT_BYTES));
|
||||
const { bytesRead } = await handle.read(buffer, 0, buffer.length, 0);
|
||||
return {
|
||||
text: buffer.subarray(0, bytesRead).toString("utf8"),
|
||||
bytes: size,
|
||||
truncated: size > MAX_OUTPUT_BYTES,
|
||||
};
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
};
|
||||
|
||||
type LldbExit =
|
||||
| {
|
||||
readonly kind: "exited";
|
||||
readonly exitCode: number | null;
|
||||
readonly output: string;
|
||||
}
|
||||
| { readonly kind: "cancelled" }
|
||||
| { readonly kind: "timeout" };
|
||||
|
||||
/** Run LLDB in its own process group so cancellation can stop it and debugserver. */
|
||||
const runLldb = (
|
||||
executable: string,
|
||||
arguments_: readonly string[],
|
||||
options: { readonly signal?: AbortSignal; readonly deadlineMs: number },
|
||||
): Promise<LldbExit> =>
|
||||
new Promise((resolve) => {
|
||||
if (options.signal?.aborted === true) {
|
||||
resolve({ kind: "cancelled" });
|
||||
return;
|
||||
}
|
||||
const child = spawn(executable, [...arguments_], {
|
||||
shell: false,
|
||||
detached: true,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
let output = "";
|
||||
let stopped: "cancelled" | "timeout" | undefined;
|
||||
const keep = (chunk: Buffer): void => {
|
||||
output = (output + chunk.toString("utf8")).slice(-DIAGNOSTIC_BYTES);
|
||||
};
|
||||
child.stdout.on("data", keep);
|
||||
child.stderr.on("data", keep);
|
||||
const kill = (reason: "cancelled" | "timeout"): void => {
|
||||
stopped ??= reason;
|
||||
if (child.pid !== undefined)
|
||||
try {
|
||||
process.kill(-child.pid, "SIGKILL");
|
||||
} catch {
|
||||
// The group already exited.
|
||||
}
|
||||
};
|
||||
const timer = setTimeout(() => {
|
||||
kill("timeout");
|
||||
}, options.deadlineMs);
|
||||
const onAbort = (): void => {
|
||||
kill("cancelled");
|
||||
};
|
||||
options.signal?.addEventListener("abort", onAbort, { once: true });
|
||||
const finish = (exitCode: number | null): void => {
|
||||
clearTimeout(timer);
|
||||
options.signal?.removeEventListener("abort", onAbort);
|
||||
resolve(
|
||||
stopped === undefined
|
||||
? { kind: "exited", exitCode, output }
|
||||
: { kind: stopped },
|
||||
);
|
||||
};
|
||||
child.on("error", () => {
|
||||
finish(null);
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
finish(code);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* debugserver kills the traced process when LLDB exits. Confirm it, and kill a
|
||||
* survivor only when its executable is still the traced one.
|
||||
*/
|
||||
const ensureTerminated = async (
|
||||
pidPath: string,
|
||||
executable: string,
|
||||
): Promise<boolean> => {
|
||||
let pid: number;
|
||||
try {
|
||||
pid = Number.parseInt(await readFile(pidPath, "utf8"), 10);
|
||||
} catch {
|
||||
// LLDB never launched the target.
|
||||
return true;
|
||||
}
|
||||
if (!Number.isSafeInteger(pid) || pid <= 0) return false;
|
||||
for (let attempt = 0; attempt < 20; attempt++) {
|
||||
if (!alive(pid)) return true;
|
||||
if (attempt === 10 && (await commandOf(pid)) === executable)
|
||||
try {
|
||||
process.kill(pid, "SIGKILL");
|
||||
} catch {
|
||||
// It exited between the checks.
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
return !alive(pid);
|
||||
};
|
||||
|
||||
const alive = (pid: number): boolean => {
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch (cause: unknown) {
|
||||
return !(
|
||||
cause instanceof Error &&
|
||||
"code" in cause &&
|
||||
cause.code === "ESRCH"
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const commandOf = async (pid: number): Promise<string | undefined> => {
|
||||
try {
|
||||
const { stdout } = await execFileOutput(
|
||||
"/bin/ps",
|
||||
["-o", "comm=", "-p", String(pid)],
|
||||
{ timeout: 5_000 },
|
||||
);
|
||||
return stdout.trim();
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,182 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { open } from "node:fs/promises";
|
||||
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisInputError,
|
||||
} from "../domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../domain/analysisErrorBase.js";
|
||||
import type { BinaryTarget } from "../domain/binaryTarget.js";
|
||||
import { EvidenceIntegrityError } from "../domain/evidenceErrors.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import {
|
||||
nativeCallObservationInputSchema,
|
||||
nativeCallObservationResultSchema,
|
||||
type NativeCallObservationInput,
|
||||
type NativeCallObservationResult,
|
||||
} from "../domain/native/nativeCallObservation.js";
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
import type { NativeCallTrace, NativeCallTracer } from "./LldbCallTracer.js";
|
||||
|
||||
const OPERATION = "observe_native_calls";
|
||||
|
||||
/** What every call observation leaves unobserved or approximated. */
|
||||
export const NATIVE_CALL_OBSERVATION_LIMITATIONS = [
|
||||
"Only entries are observed; return values, floating-point and stack-passed arguments are not captured.",
|
||||
"Registers are the raw integer argument registers at entry. For Objective-C methods the first two hold the receiver and the selector.",
|
||||
"Receiver classes come from LLDB's Objective-C runtime reader without running target code; they are null for class methods and for receivers LLDB cannot classify.",
|
||||
"Calls that never reach a symbol's entry, such as inlined code or objc_direct methods called directly, are not observed.",
|
||||
"The target is launched directly under LLDB, not through LaunchServices; LLDB disables ASLR for it, so load addresses can equal file addresses.",
|
||||
"The observation window is checked about once a second, and recording pauses the process at every entry, which slows it.",
|
||||
] as const;
|
||||
|
||||
/** Launch the active Mach-O under LLDB, record requested call entries, then stop it. */
|
||||
export const observeNativeCalls = async (
|
||||
target: BinaryTarget,
|
||||
parameters: Readonly<Record<string, JsonValue>>,
|
||||
tracer: NativeCallTracer,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<NativeCallObservationResult, AnalysisError>> => {
|
||||
if (target.kind !== "executable" || target.format !== "mach-o")
|
||||
return err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
"native-macos",
|
||||
OPERATION,
|
||||
"Native call observation launches a Mach-O executable; the active target is not one",
|
||||
),
|
||||
);
|
||||
const parsed = nativeCallObservationInputSchema.safeParse(parameters);
|
||||
if (!parsed.success)
|
||||
return err(new AnalysisInputError(OPERATION, { cause: parsed.error }));
|
||||
// Observe the bytes the session's other Evidence describes.
|
||||
if ((await fileSha256(target.path, signal)) !== target.sha256)
|
||||
return err(
|
||||
new EvidenceIntegrityError(
|
||||
"Native call observation target digest changed after session binding",
|
||||
),
|
||||
);
|
||||
if (signal?.aborted === true)
|
||||
return err(new AnalysisCancelledError(OPERATION));
|
||||
const traced = await tracer.trace(
|
||||
{
|
||||
executable: target.path,
|
||||
architecture: target.architecture,
|
||||
input: parsed.data,
|
||||
},
|
||||
signal,
|
||||
);
|
||||
if (!traced.ok) return traced;
|
||||
return ok(projectNativeCalls(target, parsed.data, traced.value));
|
||||
};
|
||||
|
||||
/** Join the request, the tracer's run and REA's own checks into the result contract. */
|
||||
export const projectNativeCalls = (
|
||||
target: Pick<BinaryTarget, "path" | "sha256"> & {
|
||||
readonly architecture: string;
|
||||
},
|
||||
input: NativeCallObservationInput,
|
||||
trace: NativeCallTrace,
|
||||
): NativeCallObservationResult => {
|
||||
const { run } = trace;
|
||||
const unresolved = run.breakpoints
|
||||
.filter(({ location_count: count }) => count === 0)
|
||||
.map(({ index }) => index);
|
||||
const eventLimit = run.outcome === "event-limit";
|
||||
const partial =
|
||||
eventLimit || run.outcome === "stop-limit" || unresolved.length > 0;
|
||||
return nativeCallObservationResultSchema.parse({
|
||||
target: {
|
||||
path: target.path,
|
||||
sha256: target.sha256,
|
||||
architecture: target.architecture,
|
||||
arguments: input.arguments,
|
||||
working_directory: input.working_directory ?? null,
|
||||
},
|
||||
debugger: trace.debugger,
|
||||
process: {
|
||||
pid: run.pid,
|
||||
outcome: run.outcome,
|
||||
exit_status: run.exit_status,
|
||||
exit_description: run.exit_description,
|
||||
terminated: run.terminated && trace.terminated,
|
||||
elapsed_ms: run.elapsed_ms,
|
||||
stdout: trace.stdout,
|
||||
stderr: trace.stderr,
|
||||
other_stops: run.other_stops,
|
||||
},
|
||||
breakpoints: input.breakpoints.map((request, index) => ({
|
||||
index,
|
||||
request:
|
||||
request.kind === "function"
|
||||
? { ...request, module: request.module ?? null }
|
||||
: { ...request, class_name: request.class_name ?? null },
|
||||
location_count: run.breakpoints[index]?.location_count ?? 0,
|
||||
locations: run.breakpoints[index]?.locations ?? [],
|
||||
})),
|
||||
events: run.events,
|
||||
coverage: {
|
||||
status: partial ? "partial" : "complete",
|
||||
event_limit_reached: eventLimit,
|
||||
unresolved_breakpoints: unresolved,
|
||||
},
|
||||
limitations: [
|
||||
...NATIVE_CALL_OBSERVATION_LIMITATIONS,
|
||||
...(eventLimit
|
||||
? [
|
||||
`Observation stopped at max_events (${input.max_events}); later calls were not recorded and the process was killed.`,
|
||||
]
|
||||
: []),
|
||||
...(run.outcome === "duration-elapsed"
|
||||
? [
|
||||
`The process was still running after duration_ms (${input.duration_ms}) and was killed.`,
|
||||
]
|
||||
: []),
|
||||
...(run.outcome === "stop-limit"
|
||||
? [
|
||||
"Observation stopped after repeated signal or exception stops; the process was killed.",
|
||||
]
|
||||
: []),
|
||||
...(unresolved.length === 0
|
||||
? []
|
||||
: [
|
||||
`Breakpoints ${unresolved.join(", ")} matched no code in any image loaded while observing; their calls could not be observed.`,
|
||||
]),
|
||||
...(run.breakpoints.some(
|
||||
({ location_count: count, locations }) => locations.length < count,
|
||||
)
|
||||
? [
|
||||
"Breakpoints with many matching locations list only the first 64; location_count gives the total.",
|
||||
]
|
||||
: []),
|
||||
...(trace.stdout.truncated || trace.stderr.truncated
|
||||
? ["Target output beyond 1 MiB per stream is counted but not kept."]
|
||||
: []),
|
||||
...(run.terminated && trace.terminated
|
||||
? []
|
||||
: [
|
||||
`REA could not confirm that process ${run.pid} exited; check for it before relying on host state.`,
|
||||
]),
|
||||
],
|
||||
});
|
||||
};
|
||||
|
||||
const fileSha256 = async (
|
||||
path: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string> => {
|
||||
const handle = await open(path, "r");
|
||||
try {
|
||||
const hash = createHash("sha256");
|
||||
const buffer = new Uint8Array(1024 * 1024);
|
||||
for (;;) {
|
||||
signal?.throwIfAborted();
|
||||
const { bytesRead } = await handle.read(buffer, 0, buffer.length, null);
|
||||
if (bytesRead === 0) break;
|
||||
hash.update(buffer.subarray(0, bytesRead));
|
||||
}
|
||||
return hash.digest("hex");
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
};
|
||||
@@ -5,6 +5,8 @@ import {
|
||||
export { NATIVE_MACOS_PROVIDER_IDENTITY } from "./NativeMacOSProviderMetadata.js";
|
||||
import { inspectAppleDispatchMetadata } from "./AppleDispatchMetadata.js";
|
||||
import { observeNativeUi } from "./NativeUiObservation.js";
|
||||
import { LldbCallTracer, type NativeCallTracer } from "./LldbCallTracer.js";
|
||||
import { observeNativeCalls } from "./NativeCallObservation.js";
|
||||
import { dirname, isAbsolute, resolve } from "node:path";
|
||||
import { open, realpath, stat } from "node:fs/promises";
|
||||
|
||||
@@ -72,6 +74,7 @@ export class NativeMacOSProvider implements AnalysisProvider {
|
||||
constructor(
|
||||
private readonly runner: NativeCommandRunner = new XcrunCommandRunner(),
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
private readonly tracer: NativeCallTracer = new LldbCallTracer(),
|
||||
) {
|
||||
this.#capabilities = nativeMacOSCapabilities(platform);
|
||||
}
|
||||
@@ -85,7 +88,7 @@ export class NativeMacOSProvider implements AnalysisProvider {
|
||||
}
|
||||
|
||||
createClient(target: BinaryTarget): AnalysisClient {
|
||||
return new NativeMacOSClient(target, this.runner);
|
||||
return new NativeMacOSClient(target, this.runner, this.tracer);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -93,6 +96,7 @@ class NativeMacOSClient implements AnalysisClient {
|
||||
constructor(
|
||||
private readonly target: BinaryTarget,
|
||||
private readonly runner: NativeCommandRunner,
|
||||
private readonly tracer: NativeCallTracer,
|
||||
) {}
|
||||
|
||||
async execute(
|
||||
@@ -157,6 +161,22 @@ class NativeMacOSClient implements AnalysisClient {
|
||||
"Operation is not implemented by native macOS tools.",
|
||||
),
|
||||
);
|
||||
if (operation === "observe_native_calls") {
|
||||
const result = await observeNativeCalls(
|
||||
this.target,
|
||||
parameters,
|
||||
this.tracer,
|
||||
options?.signal,
|
||||
);
|
||||
return result.ok
|
||||
? ok(
|
||||
createAnalysisExecution(result.value, IDENTITY, {
|
||||
limitations: result.value.limitations,
|
||||
locations: [{ kind: "artifact-path", path: this.target.path }],
|
||||
}),
|
||||
)
|
||||
: result;
|
||||
}
|
||||
if (
|
||||
operation === "observe_native_ui" ||
|
||||
operation === "capture_native_ui_scenario"
|
||||
@@ -208,6 +228,7 @@ class NativeMacOSClient implements AnalysisClient {
|
||||
switch (operation) {
|
||||
case "observe_native_ui":
|
||||
case "capture_native_ui_scenario":
|
||||
case "observe_native_calls":
|
||||
return Promise.resolve(
|
||||
err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
|
||||
@@ -12,6 +12,12 @@ export const NATIVE_MACOS_PROVIDER_IDENTITY: ProviderIdentity = Object.freeze({
|
||||
});
|
||||
const IDENTITY = NATIVE_MACOS_PROVIDER_IDENTITY;
|
||||
|
||||
/** Operations whose target process can show UI, use the network and write files. */
|
||||
const OWNED_PROCESS_OPERATIONS: ReadonlySet<string> = new Set([
|
||||
"capture_native_ui_scenario",
|
||||
"observe_native_calls",
|
||||
]);
|
||||
|
||||
/** Declare native host coverage without creating a command runner. */
|
||||
export const nativeMacOSCapabilities = (
|
||||
platform: NodeJS.Platform = process.platform,
|
||||
@@ -36,10 +42,10 @@ export const nativeMacOSCapabilities = (
|
||||
effects: Object.freeze({
|
||||
mutatesArtifact: false,
|
||||
launchesProcess: contract.name !== "inspect_native_dispatch_metadata",
|
||||
mayShowUi: contract.name === "capture_native_ui_scenario",
|
||||
mayAccessNetwork: contract.name === "capture_native_ui_scenario",
|
||||
mayShowUi: OWNED_PROCESS_OPERATIONS.has(contract.name),
|
||||
mayAccessNetwork: OWNED_PROCESS_OPERATIONS.has(contract.name),
|
||||
mayWriteFilesystem:
|
||||
contract.name === "capture_native_ui_scenario" ||
|
||||
OWNED_PROCESS_OPERATIONS.has(contract.name) ||
|
||||
contract.name === "observe_native_ui",
|
||||
changesPermissions: false,
|
||||
requiresRoot: false,
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { reconstructJavaScriptArtifact } from "../../../src/application/javascript/JavaScriptArtifactReconstruction.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const cases = [
|
||||
['<script data-src="decoy.js"></script>', []],
|
||||
['<script x:src="decoy.js"></script>', []],
|
||||
[`<script title='src="decoy.js"'></script>`, []],
|
||||
["<script src=actual.js></script>", ["actual.js"]],
|
||||
['<script title=">" src="actual.js"></script>', ["actual.js"]],
|
||||
['<script src="actual.js"></script>', ["actual.js"]],
|
||||
["<script src='actual.js'></script>", ["actual.js"]],
|
||||
['<script data-src="decoy.js" src="actual.js"></script>', ["actual.js"]],
|
||||
['<!-- <script src="decoy.js"></script> -->', []],
|
||||
['<textarea><script src="decoy.js"></script></textarea>', []],
|
||||
['<template><script src="decoy.js"></script></template>', []],
|
||||
['<script src="actual.js" src="decoy.js"></script>', ["actual.js"]],
|
||||
] as const;
|
||||
|
||||
it.each(cases)(
|
||||
"projects local loads from actual HTML attributes in %s",
|
||||
async (html, expected) => {
|
||||
const root = await createTestTempDirectory("rea-html-entrypoints-");
|
||||
await Promise.all([
|
||||
writeFile(join(root, "index.html"), html),
|
||||
writeFile(join(root, "actual.js"), "globalThis.ownedActual = true;"),
|
||||
writeFile(join(root, "decoy.js"), "globalThis.ownedDecoy = true;"),
|
||||
]);
|
||||
const result = await reconstructJavaScriptArtifact({ input_path: root });
|
||||
const loads = result.graph.edges.filter(
|
||||
({ relation, properties }) =>
|
||||
relation === "loads" && properties.script_path !== undefined,
|
||||
);
|
||||
expect(loads.map(({ properties }) => properties.resolved_path)).toEqual(
|
||||
expected,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves source digest, opening-tag range and decoded local base resolution", async () => {
|
||||
const root = await createTestTempDirectory("rea-html-entrypoint-evidence-");
|
||||
const opening = '<script title=">" src="actual.js">';
|
||||
const prefix = '<base data-href="decoy/"><base href=assets/>\n';
|
||||
const html = `${prefix}${opening}</script>`;
|
||||
await mkdir(join(root, "assets"));
|
||||
await Promise.all([
|
||||
writeFile(join(root, "index.html"), html),
|
||||
writeFile(
|
||||
join(root, "assets", "actual.js"),
|
||||
"globalThis.ownedActual = true;",
|
||||
),
|
||||
]);
|
||||
const result = await reconstructJavaScriptArtifact({ input_path: root });
|
||||
const loads = result.graph.edges.filter(
|
||||
({ relation, properties }) =>
|
||||
relation === "loads" && properties.script_path !== undefined,
|
||||
);
|
||||
expect(loads).toHaveLength(1);
|
||||
expect(loads[0]).toMatchObject({
|
||||
properties: {
|
||||
script_path: "actual.js",
|
||||
base_href: "assets/",
|
||||
resolved_path: "assets/actual.js",
|
||||
},
|
||||
evidence: {
|
||||
authority: "static-relationship-inference",
|
||||
artifact: {
|
||||
available: true,
|
||||
sha256: createHash("sha256").update(html).digest("hex"),
|
||||
},
|
||||
location: {
|
||||
available: true,
|
||||
value: {
|
||||
kind: "source-range",
|
||||
source: "index.html",
|
||||
start: { line: 2, column: 0 },
|
||||
end: { line: 2, column: opening.length },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,133 @@
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { createRequire } from "node:module";
|
||||
import { dirname, join, relative, sep } from "node:path";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { reconstructJavaScriptArtifact } from "../../../src/application/javascript/JavaScriptArtifactReconstruction.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const cases: readonly {
|
||||
name: string;
|
||||
html: string;
|
||||
files: Readonly<Record<string, string>>;
|
||||
expected: readonly string[];
|
||||
}[] = [
|
||||
{
|
||||
name: "extension fallback",
|
||||
html: '<script src="./app"></script>',
|
||||
files: { "app.js": "globalThis.owned = true;" },
|
||||
expected: [],
|
||||
},
|
||||
{
|
||||
name: "directory index",
|
||||
html: '<script src="./app"></script>',
|
||||
files: { "app/index.js": "globalThis.owned = true;" },
|
||||
expected: [],
|
||||
},
|
||||
{
|
||||
name: "directory package",
|
||||
html: '<script src="./app"></script>',
|
||||
files: {
|
||||
"app/package.json": '{"main":"entry.js"}',
|
||||
"app/entry.js": "globalThis.owned = true;",
|
||||
},
|
||||
expected: [],
|
||||
},
|
||||
{
|
||||
name: "base directory",
|
||||
html: '<base href="assets/"><script src="./app"></script>',
|
||||
files: { "assets/app.js": "globalThis.owned = true;" },
|
||||
expected: [],
|
||||
},
|
||||
{
|
||||
name: "exact base resource",
|
||||
html: '<base href="assets/"><script src="./app.js?cache=1#v2"></script>',
|
||||
files: {
|
||||
"assets/app.js": "globalThis.owned = true;",
|
||||
"app.js": "globalThis.decoy = true;",
|
||||
},
|
||||
expected: ["assets/app.js"],
|
||||
},
|
||||
{
|
||||
name: "exact script resource",
|
||||
html: '<script src="./app.js?cache=1#v2"></script>',
|
||||
files: { "app.js": "globalThis.owned = true;" },
|
||||
expected: ["app.js"],
|
||||
},
|
||||
];
|
||||
|
||||
it.each(cases)(
|
||||
"uses exact HTML resource identity for $name",
|
||||
async ({ html, files, expected }) => {
|
||||
const root = await createTestTempDirectory("rea-html-resources-");
|
||||
await Promise.all(
|
||||
Object.entries({ ...files, "index.html": html }).map(
|
||||
async ([path, text]) => {
|
||||
await mkdir(dirname(join(root, path)), { recursive: true });
|
||||
await writeFile(join(root, path), text);
|
||||
},
|
||||
),
|
||||
);
|
||||
const result = await reconstructJavaScriptArtifact({ input_path: root });
|
||||
const loads = result.graph.edges.filter(
|
||||
({ relation, properties }) =>
|
||||
relation === "loads" && properties.script_path !== undefined,
|
||||
);
|
||||
expect(loads.map(({ properties }) => properties.resolved_path)).toEqual(
|
||||
expected,
|
||||
);
|
||||
if (expected.length === 0) {
|
||||
const reference = result.graph.nodes
|
||||
.flatMap(({ observations }) => observations)
|
||||
.find(
|
||||
({ properties }) => properties.mechanism === "html-script-reference",
|
||||
);
|
||||
expect(reference).toMatchObject({
|
||||
properties: {
|
||||
script_path: "./app",
|
||||
resolution_context: "html-reference",
|
||||
resolution_status: "not-found",
|
||||
resolved_path: null,
|
||||
},
|
||||
evidence: {
|
||||
authority: "static-relationship-inference",
|
||||
location: {
|
||||
available: true,
|
||||
value: { kind: "source-range", source: "index.html" },
|
||||
},
|
||||
limitations: expect.arrayContaining([
|
||||
expect.stringContaining("exact HTML resource"),
|
||||
]),
|
||||
},
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("retains CommonJS extension resolution against Node", async () => {
|
||||
const root = await createTestTempDirectory("rea-html-node-control-");
|
||||
await Promise.all([
|
||||
writeFile(
|
||||
join(root, "main.cjs"),
|
||||
'const value = require("./app"); module.exports = value;',
|
||||
),
|
||||
writeFile(join(root, "app.js"), 'module.exports = "actual";'),
|
||||
]);
|
||||
const expected = relative(
|
||||
root,
|
||||
createRequire(join(root, "main.cjs")).resolve("./app"),
|
||||
)
|
||||
.split(sep)
|
||||
.join("/");
|
||||
const result = await reconstructJavaScriptArtifact({ input_path: root });
|
||||
const edge = result.graph.edges.find(
|
||||
({ properties }) =>
|
||||
properties.module_link_kind === "require" &&
|
||||
properties.specifier === "./app",
|
||||
);
|
||||
expect(edge?.properties).toMatchObject({
|
||||
resolution_status: "resolved",
|
||||
resolved_path: expected,
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,231 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, join } from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { reconstructJavaScriptArtifact } from "../../../src/application/javascript/JavaScriptArtifactReconstruction.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const cases: readonly {
|
||||
name: string;
|
||||
exports: unknown;
|
||||
expected: string | null;
|
||||
nodeError?: string;
|
||||
rejectedTarget?: string;
|
||||
}[] = [
|
||||
...[
|
||||
"./../outside.cjs",
|
||||
"./lib/../actual.cjs",
|
||||
"././actual.cjs",
|
||||
"./node_modules/actual.cjs",
|
||||
"./NODE_MODULES/actual.cjs",
|
||||
"./%2e%2e/outside.cjs",
|
||||
"./n%6fde_modules/actual.cjs",
|
||||
"./lib\\..\\actual.cjs",
|
||||
"../outside.cjs",
|
||||
"actual.cjs",
|
||||
"/actual.cjs",
|
||||
"file:./actual.cjs",
|
||||
"",
|
||||
].map((target) => ({
|
||||
name: `invalid target ${JSON.stringify(target)}`,
|
||||
exports: target,
|
||||
expected: null,
|
||||
nodeError: "ERR_INVALID_PACKAGE_TARGET",
|
||||
rejectedTarget: target,
|
||||
})),
|
||||
{
|
||||
name: "root subpath rejects a parent segment",
|
||||
exports: { ".": "./../outside.cjs" },
|
||||
expected: null,
|
||||
nodeError: "ERR_INVALID_PACKAGE_TARGET",
|
||||
rejectedTarget: "./../outside.cjs",
|
||||
},
|
||||
{
|
||||
name: "active condition rejects instead of selecting default",
|
||||
exports: { node: "./../outside.cjs", default: "./actual.cjs" },
|
||||
expected: null,
|
||||
nodeError: "ERR_INVALID_PACKAGE_TARGET",
|
||||
rejectedTarget: "./../outside.cjs",
|
||||
},
|
||||
{
|
||||
name: "top-level array skips an invalid target",
|
||||
exports: ["./../outside.cjs", "./actual.cjs"],
|
||||
expected: "node_modules/fixture/actual.cjs",
|
||||
},
|
||||
{
|
||||
name: "root array skips encoded invalid segments",
|
||||
exports: { ".": ["./%2e%2e/outside.cjs", "./actual.cjs"] },
|
||||
expected: "node_modules/fixture/actual.cjs",
|
||||
},
|
||||
{
|
||||
name: "nested arrays skip invalid package-local segments",
|
||||
exports: { node: [["./n%6fde_modules/actual.cjs", "./actual.cjs"]] },
|
||||
expected: "node_modules/fixture/actual.cjs",
|
||||
},
|
||||
{
|
||||
name: "last invalid target retains its reason",
|
||||
exports: ["./../outside.cjs", "././actual.cjs"],
|
||||
expected: null,
|
||||
nodeError: "ERR_INVALID_PACKAGE_TARGET",
|
||||
rejectedTarget: "././actual.cjs",
|
||||
},
|
||||
{
|
||||
name: "null following an invalid target blocks exports",
|
||||
exports: ["./../outside.cjs", null],
|
||||
expected: null,
|
||||
nodeError: "ERR_PACKAGE_PATH_NOT_EXPORTED",
|
||||
},
|
||||
{
|
||||
name: "an invalid target following null remains invalid",
|
||||
exports: [null, "./../outside.cjs"],
|
||||
expected: null,
|
||||
nodeError: "ERR_INVALID_PACKAGE_TARGET",
|
||||
rejectedTarget: "./../outside.cjs",
|
||||
},
|
||||
{
|
||||
name: "inactive invalid condition does not block default",
|
||||
exports: { browser: "./../outside.cjs", default: "./actual.cjs" },
|
||||
expected: "node_modules/fixture/actual.cjs",
|
||||
},
|
||||
...["./bad%.cjs", "./bad%C3.cjs"].map((target) => ({
|
||||
name: `malformed URL ${target} is selected before URL decoding`,
|
||||
exports: [target, "./actual.cjs"],
|
||||
expected: null,
|
||||
nodeError: "URIError",
|
||||
})),
|
||||
{
|
||||
name: "encoded separator is selected before final path refusal",
|
||||
exports: ["./lib%2Factual.cjs", "./actual.cjs"],
|
||||
expected: null,
|
||||
nodeError: "ERR_INVALID_MODULE_SPECIFIER",
|
||||
},
|
||||
{
|
||||
name: "missing first target does not select a second file",
|
||||
exports: ["./missing.cjs", "./actual.cjs"],
|
||||
expected: null,
|
||||
nodeError: "MODULE_NOT_FOUND",
|
||||
},
|
||||
{
|
||||
name: "nested directory is valid",
|
||||
exports: "./lib/actual.cjs",
|
||||
expected: "node_modules/fixture/lib/actual.cjs",
|
||||
},
|
||||
{
|
||||
name: "node_modules substring is not a forbidden segment",
|
||||
exports: "./node_modules.cjs",
|
||||
expected: "node_modules/fixture/node_modules.cjs",
|
||||
},
|
||||
{
|
||||
name: "encoded underscore and letters do not borrow a forbidden directory",
|
||||
exports: ["./%6eode%5fmodules/actual.cjs", "./actual.cjs"],
|
||||
expected: "node_modules/fixture/actual.cjs",
|
||||
},
|
||||
{
|
||||
name: "percent encoding is applied once",
|
||||
exports: "./literal%252e.cjs",
|
||||
expected: "node_modules/fixture/literal%2e.cjs",
|
||||
},
|
||||
];
|
||||
|
||||
const nativeResultSchema = z.object({
|
||||
value: z.string().nullable(),
|
||||
code: z.string().nullable(),
|
||||
});
|
||||
|
||||
it.each(
|
||||
(["import", "require"] as const).flatMap((moduleKind) =>
|
||||
cases.map((testCase) => ({ ...testCase, moduleKind })),
|
||||
),
|
||||
)(
|
||||
"matches Node $moduleKind target selection for $name",
|
||||
async ({
|
||||
moduleKind,
|
||||
name,
|
||||
exports,
|
||||
expected,
|
||||
nodeError,
|
||||
rejectedTarget,
|
||||
}) => {
|
||||
const root = await createTestTempDirectory("rea-package-target-");
|
||||
const mainPath = moduleKind === "import" ? "main.mjs" : "main.cjs";
|
||||
const files = {
|
||||
[mainPath]:
|
||||
moduleKind === "import"
|
||||
? 'import value from "fixture"; export default value;'
|
||||
: 'module.exports = require("fixture");',
|
||||
"node_modules/fixture/package.json": JSON.stringify({ exports }),
|
||||
...Object.fromEntries(
|
||||
[
|
||||
"node_modules/fixture/actual.cjs",
|
||||
"node_modules/outside.cjs",
|
||||
"node_modules/fixture/lib/actual.cjs",
|
||||
"node_modules/fixture/node_modules/actual.cjs",
|
||||
"node_modules/fixture/node_modules.cjs",
|
||||
"node_modules/fixture/literal%2e.cjs",
|
||||
].map((path) => [path, `module.exports = ${JSON.stringify(path)};`]),
|
||||
),
|
||||
};
|
||||
await Promise.all(
|
||||
Object.entries(files).map(async ([path, text]) => {
|
||||
await mkdir(dirname(join(root, path)), { recursive: true });
|
||||
await writeFile(join(root, path), text);
|
||||
}),
|
||||
);
|
||||
const runner =
|
||||
moduleKind === "import"
|
||||
? 'const load = async () => (await import("fixture")).default;'
|
||||
: 'const load = async () => require("fixture");';
|
||||
const native = await promisify(execFile)(
|
||||
process.execPath,
|
||||
[
|
||||
"--input-type",
|
||||
moduleKind === "import" ? "module" : "commonjs",
|
||||
"--eval",
|
||||
`${runner} load().then(value => console.log(JSON.stringify({value,code:null})), error => console.log(JSON.stringify({value:null,code:error.code ?? error.name})));`,
|
||||
],
|
||||
{ cwd: root, timeout: 5_000 },
|
||||
);
|
||||
const oracle = nativeResultSchema.parse(JSON.parse(native.stdout));
|
||||
expect(oracle.value, name).toBe(expected);
|
||||
if (nodeError !== undefined)
|
||||
expect(oracle.code).toBe(
|
||||
moduleKind === "import" && nodeError === "MODULE_NOT_FOUND"
|
||||
? "ERR_MODULE_NOT_FOUND"
|
||||
: nodeError,
|
||||
);
|
||||
const result = await reconstructJavaScriptArtifact({
|
||||
input_path: root,
|
||||
format: "directory",
|
||||
});
|
||||
const edge = result.graph.edges.find(
|
||||
({ properties }) =>
|
||||
properties.specifier === "fixture" &&
|
||||
properties.resolution_status !== undefined &&
|
||||
properties.resolution_status !== null,
|
||||
);
|
||||
expect(edge).toBeDefined();
|
||||
expect(edge?.properties.resolved_path).toBe(expected);
|
||||
if (expected !== null) {
|
||||
expect(edge?.properties.resolution_status).toBe("resolved");
|
||||
} else {
|
||||
expect(edge?.properties.resolution_status).not.toBe("resolved");
|
||||
}
|
||||
if (rejectedTarget !== undefined) {
|
||||
expect(edge?.properties.resolution_status).toBe("rejected");
|
||||
expect(edge?.evidence.limitations.join(" ")).toContain(
|
||||
JSON.stringify(rejectedTarget),
|
||||
);
|
||||
expect(edge?.evidence.limitations.join(" ")).toContain("exports target");
|
||||
expect(edge?.evidence.limitations.join(" ")).not.toContain(
|
||||
"not valid package JSON",
|
||||
);
|
||||
expect(edge?.evidence.location).toMatchObject({
|
||||
value: { kind: "source-range", source: mainPath },
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
@@ -25,6 +25,26 @@ const cases: readonly {
|
||||
metadata: { main: "actual.cjs" },
|
||||
index: false,
|
||||
},
|
||||
{
|
||||
name: "top-level exports array",
|
||||
metadata: { exports: ["./actual.cjs"] },
|
||||
index: true,
|
||||
},
|
||||
{
|
||||
name: "root subpath exports array",
|
||||
metadata: { exports: { ".": ["./actual.cjs"] } },
|
||||
index: true,
|
||||
},
|
||||
{
|
||||
name: "exports array skips invalid and unmatched entries",
|
||||
metadata: { exports: [42, { browser: "./browser.js" }, "./actual.cjs"] },
|
||||
index: true,
|
||||
},
|
||||
{
|
||||
name: "nested root exports array",
|
||||
metadata: { exports: { ".": [[null, { require: "./actual.cjs" }]] } },
|
||||
index: true,
|
||||
},
|
||||
{ name: "index fallback", metadata: {}, index: true },
|
||||
{
|
||||
name: "missing legacy main fallback",
|
||||
|
||||
@@ -237,18 +237,23 @@ describe("unmatched package export conditions", () => {
|
||||
},
|
||||
);
|
||||
it("does not report valid metadata as malformed", async () => {
|
||||
const { outcome } = await compareWithNode({ default: 7 }, "import");
|
||||
expect(outcome.resolution_status).toBe("unavailable");
|
||||
expect(outcome.limitations[0]).toContain("not valid package JSON");
|
||||
const { node_error, outcome } = await compareWithNode(
|
||||
{ default: 7 },
|
||||
"import",
|
||||
);
|
||||
expect(node_error).toBe("ERR_INVALID_PACKAGE_TARGET");
|
||||
expect(outcome.resolution_status).toBe("rejected");
|
||||
expect(outcome.limitations[0]).toContain("exports target 7");
|
||||
expect(outcome.limitations[0]).not.toContain("not valid package JSON");
|
||||
});
|
||||
});
|
||||
|
||||
describe("package exports fallback refusal reasons", () => {
|
||||
it.each([
|
||||
[{ import: [42, 7] }, "ERR_INVALID_PACKAGE_TARGET", "unavailable"],
|
||||
[{ import: [42, 7] }, "ERR_INVALID_PACKAGE_TARGET", "rejected"],
|
||||
[{ import: [null, null] }, "ERR_PACKAGE_PATH_NOT_EXPORTED", "external"],
|
||||
[{ import: [42, null] }, "ERR_PACKAGE_PATH_NOT_EXPORTED", "external"],
|
||||
[{ import: [null, 42] }, "ERR_INVALID_PACKAGE_TARGET", "unavailable"],
|
||||
[{ import: [null, 42] }, "ERR_INVALID_PACKAGE_TARGET", "rejected"],
|
||||
[
|
||||
{ import: { node: null }, default: "./default.cjs" },
|
||||
"ERR_PACKAGE_PATH_NOT_EXPORTED",
|
||||
@@ -265,6 +270,10 @@ describe("package exports fallback refusal reasons", () => {
|
||||
expect(node_error).toBe(code);
|
||||
expect(outcome.resolved_path).toBeNull();
|
||||
expect(outcome.resolution_status).toBe(status);
|
||||
if (code === "ERR_INVALID_PACKAGE_TARGET") {
|
||||
expect(outcome.limitations[0]).toContain("exports target");
|
||||
expect(outcome.limitations[0]).not.toContain("not valid package JSON");
|
||||
}
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { expect, it } from "vitest";
|
||||
import { z } from "zod";
|
||||
|
||||
import { nativeCallObservationResultSchema } from "../../../src/domain/native/nativeCallObservation.js";
|
||||
import { ok } from "../../../src/domain/result.js";
|
||||
import type { NativeCallTracer } from "../../../src/native/LldbCallTracer.js";
|
||||
import { NativeMacOSProvider } from "../../../src/native/NativeMacOSProvider.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { createTestBinarySession } from "../../fixtures/binarySession.js";
|
||||
import { NativeFixtureRunner } from "../../fixtures/nativeCommands.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
/** A thin little-endian arm64 MH_EXECUTE header with no load commands. */
|
||||
const machoHeader = (): Buffer => {
|
||||
const header = Buffer.alloc(32);
|
||||
header.writeUInt32LE(0xfeedfacf, 0);
|
||||
header.writeUInt32LE(0x0100000c, 4);
|
||||
header.writeUInt32LE(2, 12);
|
||||
return header;
|
||||
};
|
||||
|
||||
const tracer: NativeCallTracer & { calls: number } = {
|
||||
calls: 0,
|
||||
trace(request) {
|
||||
this.calls++;
|
||||
return Promise.resolve(
|
||||
ok({
|
||||
debugger: {
|
||||
path: "/usr/bin/lldb",
|
||||
sha256: "b".repeat(64),
|
||||
version: "lldb-fixture",
|
||||
},
|
||||
run: {
|
||||
status: "traced",
|
||||
version: "lldb-fixture",
|
||||
pid: 99,
|
||||
outcome: "exited",
|
||||
exit_status: 0,
|
||||
exit_description: null,
|
||||
killed: false,
|
||||
terminated: true,
|
||||
elapsed_ms: 1,
|
||||
breakpoints: request.input.breakpoints.map((_, index) => ({
|
||||
index,
|
||||
location_count: 1,
|
||||
locations: [],
|
||||
})),
|
||||
events: [],
|
||||
other_stops: [],
|
||||
},
|
||||
stdout: { text: "", bytes: 0, truncated: false },
|
||||
stderr: { text: "", bytes: 0, truncated: false },
|
||||
terminated: true,
|
||||
}),
|
||||
);
|
||||
},
|
||||
};
|
||||
|
||||
it("routes observe_native_calls through MCP with schema-checked input and output", async () => {
|
||||
const directory = await createTestTempDirectory("rea-native-calls-mcp-");
|
||||
const path = join(directory, "Tool");
|
||||
await writeFile(path, machoHeader());
|
||||
const session = createTestBinarySession(
|
||||
new NativeMacOSProvider(new NativeFixtureRunner(), "darwin", tracer),
|
||||
);
|
||||
const server = createServer(session, session);
|
||||
const client = new Client({ name: "native-calls-mcp-test", version: "1" });
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
try {
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
const opened = await client.callTool({
|
||||
name: "open_binary",
|
||||
arguments: { path },
|
||||
});
|
||||
expect(opened.isError, JSON.stringify(opened)).not.toBe(true);
|
||||
const called = await client.callTool({
|
||||
name: "observe_native_calls",
|
||||
arguments: {
|
||||
breakpoints: [{ kind: "function", name: "open" }],
|
||||
duration_ms: 1000,
|
||||
},
|
||||
});
|
||||
expect(called.isError, JSON.stringify(called.structuredContent)).not.toBe(
|
||||
true,
|
||||
);
|
||||
const result = nativeCallObservationResultSchema.parse(
|
||||
z.object({ result: z.unknown() }).parse(called.structuredContent).result,
|
||||
);
|
||||
expect(result.process).toMatchObject({ pid: 99, outcome: "exited" });
|
||||
expect(result.breakpoints[0]?.request).toEqual({
|
||||
kind: "function",
|
||||
name: "open",
|
||||
module: null,
|
||||
});
|
||||
const rejected = await client.callTool({
|
||||
name: "observe_native_calls",
|
||||
arguments: {
|
||||
breakpoints: [{ kind: "function", name: "open" }],
|
||||
duration_ms: 1,
|
||||
},
|
||||
});
|
||||
expect(rejected.isError).toBe(true);
|
||||
expect(tracer.calls).toBe(1);
|
||||
} finally {
|
||||
await client.close();
|
||||
await server.close();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,293 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
} from "../../../../src/domain/analysisErrorCore.js";
|
||||
import type { AnalysisError } from "../../../../src/domain/analysisErrorBase.js";
|
||||
import { EvidenceIntegrityError } from "../../../../src/domain/evidenceErrors.js";
|
||||
import { nativeCallObservationResultSchema } from "../../../../src/domain/native/nativeCallObservation.js";
|
||||
import { err, ok, type Result } from "../../../../src/domain/result.js";
|
||||
import type {
|
||||
NativeCallTrace,
|
||||
NativeCallTracer,
|
||||
} from "../../../../src/native/LldbCallTracer.js";
|
||||
import { NativeMacOSProvider } from "../../../../src/native/NativeMacOSProvider.js";
|
||||
import {
|
||||
NativeFixtureRunner,
|
||||
nativeMachoTarget,
|
||||
} from "../../../fixtures/nativeCommands.js";
|
||||
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
|
||||
|
||||
const LOCATION = {
|
||||
load_address: "0x100000928",
|
||||
file_address: "0x100000928",
|
||||
module: "Fixture",
|
||||
module_path: "/tmp/Fixture",
|
||||
symbol: "-[Greeter greet:times:]",
|
||||
};
|
||||
|
||||
const run = (
|
||||
overrides: Partial<NativeCallTrace["run"]> = {},
|
||||
): NativeCallTrace["run"] => ({
|
||||
status: "traced",
|
||||
version: "lldb-2100.0.17.203",
|
||||
pid: 4242,
|
||||
outcome: "exited",
|
||||
exit_status: 0,
|
||||
exit_description: null,
|
||||
killed: false,
|
||||
terminated: true,
|
||||
elapsed_ms: 12.5,
|
||||
breakpoints: [
|
||||
{ index: 0, location_count: 1, locations: [LOCATION] },
|
||||
{ index: 1, location_count: 1, locations: [LOCATION] },
|
||||
],
|
||||
events: [
|
||||
{
|
||||
sequence: 0,
|
||||
elapsed_ms: 3,
|
||||
thread_id: 7,
|
||||
breakpoint_index: 0,
|
||||
...LOCATION,
|
||||
receiver_class: "Greeter",
|
||||
selector: "greet:times:",
|
||||
registers: [
|
||||
{ name: "x0", value: "0x6000" },
|
||||
{ name: "x1", value: "0x7000" },
|
||||
],
|
||||
backtrace: [],
|
||||
},
|
||||
],
|
||||
other_stops: [],
|
||||
...overrides,
|
||||
});
|
||||
|
||||
/** Records the request and returns a canned run or failure. */
|
||||
class FixtureTracer implements NativeCallTracer {
|
||||
readonly requests: Parameters<NativeCallTracer["trace"]>[0][] = [];
|
||||
|
||||
constructor(
|
||||
private readonly outcome: Result<NativeCallTrace, AnalysisError> = ok({
|
||||
debugger: {
|
||||
path: "/usr/bin/lldb",
|
||||
sha256: "a".repeat(64),
|
||||
version: "lldb-2100.0.17.203",
|
||||
},
|
||||
run: run(),
|
||||
stdout: { text: "hello\n", bytes: 6, truncated: false },
|
||||
stderr: { text: "", bytes: 0, truncated: false },
|
||||
terminated: true,
|
||||
}),
|
||||
) {}
|
||||
|
||||
trace(request: Parameters<NativeCallTracer["trace"]>[0]) {
|
||||
this.requests.push(request);
|
||||
return Promise.resolve(this.outcome);
|
||||
}
|
||||
}
|
||||
|
||||
const BREAKPOINTS = [
|
||||
{ kind: "objc-method", class_name: "Greeter", selector: "greet:times:" },
|
||||
{ kind: "function", name: "rea_add" },
|
||||
];
|
||||
|
||||
const fixtureTarget = async () => {
|
||||
const directory = await createTestTempDirectory("rea-native-calls-");
|
||||
const path = join(directory, "Fixture");
|
||||
await writeFile(path, "fixture executable");
|
||||
return {
|
||||
...nativeMachoTarget(path),
|
||||
sha256: createHash("sha256").update("fixture executable").digest("hex"),
|
||||
};
|
||||
};
|
||||
|
||||
const observe = async (
|
||||
tracer: NativeCallTracer,
|
||||
parameters: Record<string, unknown>,
|
||||
target?: Awaited<ReturnType<typeof fixtureTarget>>,
|
||||
signal?: AbortSignal,
|
||||
) =>
|
||||
new NativeMacOSProvider(new NativeFixtureRunner(), "darwin", tracer)
|
||||
.createClient(target ?? (await fixtureTarget()))
|
||||
.execute(
|
||||
"observe_native_calls",
|
||||
// The provider parses JSON-like parameters itself.
|
||||
JSON.parse(JSON.stringify(parameters)),
|
||||
signal === undefined ? undefined : { signal },
|
||||
);
|
||||
|
||||
describe("observe_native_calls projection", () => {
|
||||
it("launches the active Mach-O through the tracer and projects its run", async () => {
|
||||
const tracer = new FixtureTracer();
|
||||
const target = await fixtureTarget();
|
||||
const observed = await observe(
|
||||
tracer,
|
||||
{ breakpoints: BREAKPOINTS, arguments: ["--flag"] },
|
||||
target,
|
||||
);
|
||||
if (!observed.ok) throw observed.error;
|
||||
expect(tracer.requests[0]).toMatchObject({
|
||||
executable: target.path,
|
||||
architecture: "arm64",
|
||||
input: {
|
||||
arguments: ["--flag"],
|
||||
duration_ms: 10_000,
|
||||
max_events: 1_000,
|
||||
argument_registers: 4,
|
||||
backtrace_frames: 0,
|
||||
},
|
||||
});
|
||||
const result = nativeCallObservationResultSchema.parse(
|
||||
observed.value.result,
|
||||
);
|
||||
expect(result.breakpoints.map(({ request }) => request)).toEqual([
|
||||
{
|
||||
kind: "objc-method",
|
||||
class_name: "Greeter",
|
||||
selector: "greet:times:",
|
||||
method_type: "any",
|
||||
},
|
||||
{ kind: "function", name: "rea_add", module: null },
|
||||
]);
|
||||
expect(result.process).toMatchObject({
|
||||
pid: 4242,
|
||||
outcome: "exited",
|
||||
exit_status: 0,
|
||||
terminated: true,
|
||||
stdout: { text: "hello\n" },
|
||||
});
|
||||
expect(result.events[0]).toMatchObject({
|
||||
receiver_class: "Greeter",
|
||||
selector: "greet:times:",
|
||||
});
|
||||
expect(result.coverage).toEqual({
|
||||
status: "complete",
|
||||
event_limit_reached: false,
|
||||
unresolved_breakpoints: [],
|
||||
});
|
||||
expect(observed.value.locations).toEqual([
|
||||
{ kind: "artifact-path", path: target.path },
|
||||
]);
|
||||
});
|
||||
|
||||
it("marks event limits, unresolved breakpoints and unconfirmed exits", async () => {
|
||||
const tracer = new FixtureTracer(
|
||||
ok({
|
||||
debugger: {
|
||||
path: "/usr/bin/lldb",
|
||||
sha256: "a".repeat(64),
|
||||
version: null,
|
||||
},
|
||||
run: run({
|
||||
outcome: "event-limit",
|
||||
exit_status: null,
|
||||
killed: true,
|
||||
terminated: false,
|
||||
breakpoints: [
|
||||
{ index: 0, location_count: 1, locations: [LOCATION] },
|
||||
{ index: 1, location_count: 0, locations: [] },
|
||||
],
|
||||
}),
|
||||
stdout: { text: "", bytes: 0, truncated: false },
|
||||
stderr: { text: "x".repeat(16), bytes: 2_000_000, truncated: true },
|
||||
terminated: true,
|
||||
}),
|
||||
);
|
||||
const observed = await observe(tracer, {
|
||||
breakpoints: BREAKPOINTS,
|
||||
max_events: 1,
|
||||
});
|
||||
if (!observed.ok) throw observed.error;
|
||||
const result = nativeCallObservationResultSchema.parse(
|
||||
observed.value.result,
|
||||
);
|
||||
expect(result.coverage).toEqual({
|
||||
status: "partial",
|
||||
event_limit_reached: true,
|
||||
unresolved_breakpoints: [1],
|
||||
});
|
||||
expect(result.process.terminated).toBe(false);
|
||||
expect(result.limitations).toEqual(
|
||||
expect.arrayContaining([
|
||||
"Observation stopped at max_events (1); later calls were not recorded and the process was killed.",
|
||||
"Breakpoints 1 matched no code in any image loaded while observing; their calls could not be observed.",
|
||||
"Target output beyond 1 MiB per stream is counted but not kept.",
|
||||
"REA could not confirm that process 4242 exited; check for it before relying on host state.",
|
||||
]),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("observe_native_calls failures", () => {
|
||||
it("rejects malformed input before launching anything", async () => {
|
||||
for (const parameters of [
|
||||
{ breakpoints: [] },
|
||||
{ breakpoints: [{ kind: "objc-method", selector: "-[A b]" }] },
|
||||
{ breakpoints: BREAKPOINTS, environment: { "A=B": "x" } },
|
||||
{ breakpoints: BREAKPOINTS, argument_registers: 9 },
|
||||
]) {
|
||||
const tracer = new FixtureTracer();
|
||||
const observed = await observe(tracer, parameters);
|
||||
expect(observed.ok).toBe(false);
|
||||
if (!observed.ok)
|
||||
expect(observed.error).toBeInstanceOf(AnalysisInputError);
|
||||
expect(tracer.requests).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses a changed or non-Mach-O target", async () => {
|
||||
const tracer = new FixtureTracer();
|
||||
const target = await fixtureTarget();
|
||||
const changed = await observe(
|
||||
tracer,
|
||||
{ breakpoints: BREAKPOINTS },
|
||||
{ ...target, sha256: "f".repeat(64) },
|
||||
);
|
||||
expect(!changed.ok && changed.error).toBeInstanceOf(EvidenceIntegrityError);
|
||||
const database = await new NativeMacOSProvider(
|
||||
new NativeFixtureRunner(),
|
||||
"darwin",
|
||||
tracer,
|
||||
)
|
||||
.createClient({
|
||||
path: target.path,
|
||||
sha256: target.sha256,
|
||||
kind: "database",
|
||||
format: "analysis-database",
|
||||
})
|
||||
.execute("observe_native_calls", { breakpoints: [] });
|
||||
expect(!database.ok && database.error).toBeInstanceOf(
|
||||
AnalysisCapabilityUnavailableError,
|
||||
);
|
||||
expect(tracer.requests).toEqual([]);
|
||||
});
|
||||
|
||||
it("keeps the tracer's failure reason and caller cancellation", async () => {
|
||||
const denied = new AnalysisCapabilityUnavailableError(
|
||||
"native-macos",
|
||||
"observe_native_calls",
|
||||
"debugger-attach-denied: attach failed",
|
||||
);
|
||||
const failed = await observe(new FixtureTracer(err(denied)), {
|
||||
breakpoints: BREAKPOINTS,
|
||||
});
|
||||
expect(!failed.ok && failed.error).toBe(denied);
|
||||
const controller = new AbortController();
|
||||
controller.abort();
|
||||
const cancelled = await observe(
|
||||
new FixtureTracer(),
|
||||
{ breakpoints: BREAKPOINTS },
|
||||
undefined,
|
||||
controller.signal,
|
||||
);
|
||||
expect(!cancelled.ok && cancelled.error).toBeInstanceOf(
|
||||
AnalysisCancelledError,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,33 @@
|
||||
#import <Foundation/Foundation.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
|
||||
// Observed by verify:native-calls; `--wait` keeps the process running.
|
||||
@interface ReaCallGreeter : NSObject
|
||||
- (NSString *)greet:(NSString *)name times:(long)count;
|
||||
@end
|
||||
@implementation ReaCallGreeter
|
||||
- (NSString *)greet:(NSString *)name times:(long)count {
|
||||
return [NSString stringWithFormat:@"%@ x%ld", name, count];
|
||||
}
|
||||
@end
|
||||
|
||||
__attribute__((noinline)) int rea_call_add(int left, int right) {
|
||||
return left + right;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
@autoreleasepool {
|
||||
if (argc > 1 && strcmp(argv[1], "--wait") == 0)
|
||||
for (;;) sleep(1);
|
||||
ReaCallGreeter *greeter = [ReaCallGreeter new];
|
||||
for (long index = 0; index < 3; index++)
|
||||
printf("%s %d\n", [[greeter greet:@"world" times:index] UTF8String],
|
||||
rea_call_add((int)index, 40));
|
||||
const char *marker = getenv("REA_CALLS_MARKER");
|
||||
if (marker != NULL)
|
||||
printf("marker %s\n", marker);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
Reference in New Issue
Block a user