chore: merge main into feat/dylib-resolution

main now carries #1037 (skill version 30, reworded native binary row):
- README_ja/ar/ko/zh: take main's tables and count
  trace_dylib_resolution in the artifact graph row (5 -> 6).
- Skill: move to version 31.
- Regenerate the product catalog, conformance manifest, skill digest and
  package metadata (137 tools).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Kaoru0822
2026-10-08 06:31:43 +00:00
co-authored by Claude Opus 5.5
59 changed files with 5068 additions and 45 deletions
+78
View File
@@ -0,0 +1,78 @@
name: Real offline binary layout
on:
workflow_dispatch:
inputs:
large_mcp:
description: "Also verify the complete 65,281-section result over MCP (slow)"
type: boolean
default: false
pull_request:
paths:
- ".github/workflows/real-binary-layout.yml"
- "bridge/pwntools/**"
- "src/native/pwntools/**"
- "src/domain/native/binaryLayout*"
- "src/application/binaryDiagnostics/**"
- "src/contracts/native/binaryDiagnosticsToolContracts.ts"
- "src/composition/binaryDiagnostics.ts"
- "src/server/registerBinaryDiagnosticsTools.ts"
- "src/server/createServer.ts"
- "src/cli/binaryDiagnosticsCommands.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/process/OwnedCommand.ts"
- "src/process/ProviderProcess.ts"
- "src/process/ProviderDeadline.ts"
- "src/process/ProcessOwnership*.ts"
- "src/process/PrivateRuntimeRoot.ts"
- "src/artifacts/readStableArtifact*.ts"
- "src/artifacts/ArtifactReader.ts"
- "src/domain/analysisError*.ts"
- "scripts/verify-binary-layout.mjs"
- "scripts/lib/elf-section-name-fixtures.mjs"
- "scripts/lib/large-result-mcp.mjs"
- "scripts/fixtures/binary-layout*"
- "package.json"
- "package-lock.json"
permissions:
contents: read
concurrency:
group: real-binary-layout-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
REA_VERIFY_LARGE_ELF_MCP: ${{ inputs.large_mcp && '1' || '0' }}
jobs:
real-binary-layout:
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- name: Acquire exact upstream libraries in an isolated runner environment
shell: bash
run: |
layout_python_root="${RUNNER_TEMP}/rea-pwntools-layout"
python3 -m venv "${layout_python_root}"
"${layout_python_root}/bin/python" -m pip install --disable-pip-version-check \
'pwntools==4.15.0' 'pyelftools==0.33' 'unicorn==2.1.2'
command -v gcc
command -v ld
command -v strip
command -v strace
echo "REA_PWNTOOLS_PYTHON=${layout_python_root}/bin/python" >> "${GITHUB_ENV}"
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
- name: Verify source-owned fixtures through public CLI and MCP
run: npm run verify:binary:layout
+1
View File
@@ -150,6 +150,7 @@ Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](https://morluto.github.io/rea/guides/native/) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; [application analysis](https://morluto.github.io/rea/guides/javascript/) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](https://morluto.github.io/rea/guides/browser/) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR; mitmdump on Linux for native mitmproxy captures; [capture guide](docs/web-network-captures.md) |
+14 -14
View File
@@ -213,20 +213,20 @@ rea setup
## منصة أدوات التحقيق
| عائلة الأدوات | العدد | الاستخدام |
| -------------------- | ----: | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| الفحص الأصلي | 41 | الدوال والشيفرة شبه المصدرية والتعليمات والسلاسل والرموز والاستدعاءات والمراجع والتعليقات وقراءة البايتات ومواضع الملفات |
| سير عمل التحقيق | 14 | نظرات عامة على التطبيقات وملفات الدوال وواجهات API الأصلية والتوزيع وفك الترجمة الدفعي وتتبع الميزات ومسارات الاستدعاء ومخططاته واكتشاف Swift وObjective-C |
| أدوات macOS الأصلية | 8 | بيانات Mach-O والتوقيعات وplist والمعماريات واستعادة أسماء Swift ومراقبة الاستدعاءات عبر LLDB دون تشغيل Hopper |
| مخطط المخرجات | 6 | جرد الدلائل والحزم وملفات Interface Builder المترجمة وكتالوجات موارد Apple وحل تحميل مكتبات Mach-O الديناميكية والاستخراج |
| PE/CLI المُدار | 7 | هوية تجميعات .NET وبياناتها الوصفية وتعليمات CIL والاعتماديات الأصلية واستيراد إعادة البناء ومقارنة الإصدارات |
| البرامج الثابتة | 2 | فحص مناطق البرامج الثابتة على Linux واستخراجها الصريح |
| Android APK | 5 | بيانات الحزمة وmanifest والبحث عن الأصناف وقوائم الأعضاء وفك ترجمة الأساليب والمراجع الثابتة |
| مراقبة المتصفح | 12 | بنية الصفحات وبيانات الشبكة والبرامج النصية وخرائط المصدر واكتشاف WebMCP والصور ومقارنة الالتقاط |
| تحليل Electron | 5 | مراقبة العارضين وخريطة التطبيق الثابتة والمواءمة بين نتائج الثبات والتشغيل |
| وقت تشغيل JavaScript | 2 | اكتشاف أهداف Node/Electron Inspector ومواضع البرامج وأحداث سياقات التنفيذ |
| سير عمل التطبيقات | 13 | تصدير النصوص البرمجية الملتقطة، تتبع الميزات عبر الطبقات ومقارنة الإصدارات ومواءمة المصدر التاريخي ومقارنة أشكال القيم المعادة والتحقق من إعادة الإنشاء |
| مساحة العمل والملاحظة | 21 | الجلسات وحزم الأدلة وسياق التنقل ومقارنة العمليات والمخرجات والدوال وتسجيل الأسئلة غير المحسومة |
| عائلة الأدوات | العدد | الاستخدام |
| ---------------------- | ----: | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| الفحص الأصلي | 41 | الدوال والشيفرة شبه المصدرية والتعليمات والسلاسل والرموز والاستدعاءات والمراجع والتعليقات وقراءة البايتات ومواضع الملفات |
| سير عمل التحقيق | 14 | نظرات عامة على التطبيقات وملفات الدوال وواجهات API الأصلية والتوزيع وفك الترجمة الدفعي وتتبع الميزات ومسارات الاستدعاء ومخططاته واكتشاف Swift وObjective-C |
| أدوات الملفات الثنائية | 9 | بيانات Mach-O والتوقيعات وplist والمعماريات وأسماء Swift ومراقبة الاستدعاءات عبر LLDB على macOS؛ تخطيط ELF وأدلة الحماية الثابتة على Linux |
| مخطط المخرجات | 6 | جرد الدلائل والحزم وملفات Interface Builder المترجمة وكتالوجات موارد Apple وحل تحميل مكتبات Mach-O الديناميكية والاستخراج |
| PE/CLI المُدار | 7 | هوية تجميعات .NET وبياناتها الوصفية وتعليمات CIL والاعتماديات الأصلية واستيراد إعادة البناء ومقارنة الإصدارات |
| البرامج الثابتة | 2 | فحص مناطق البرامج الثابتة على Linux واستخراجها الصريح |
| Android APK | 5 | بيانات الحزمة وmanifest والبحث عن الأصناف وقوائم الأعضاء وفك ترجمة الأساليب والمراجع الثابتة |
| مراقبة المتصفح | 12 | بنية الصفحات وبيانات الشبكة والبرامج النصية وخرائط المصدر واكتشاف WebMCP والصور ومقارنة الالتقاط |
| تحليل Electron | 5 | مراقبة العارضين وخريطة التطبيق الثابتة والمواءمة بين نتائج الثبات والتشغيل |
| وقت تشغيل JavaScript | 2 | اكتشاف أهداف Node/Electron Inspector ومواضع البرامج وأحداث سياقات التنفيذ |
| سير عمل التطبيقات | 13 | تصدير النصوص البرمجية الملتقطة، تتبع الميزات عبر الطبقات ومقارنة الإصدارات ومواءمة المصدر التاريخي ومقارنة أشكال القيم المعادة والتحقق من إعادة الإنشاء |
| مساحة العمل والملاحظة | 21 | الجلسات وحزم الأدلة وسياق التنقل ومقارنة العمليات والمخرجات والدوال وتسجيل الأسئلة غير المحسومة |
## خطة العمل
+1 -1
View File
@@ -240,7 +240,7 @@ TypeScript と SQLite を使って私のプロジェクト向けに実装して
| -------------------- | --: | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ネイティブ検査 | 41 | 関数、疑似コード、アセンブリ、文字列、シンボル、呼び出し、参照、注釈、バイト読み取り、ファイルオフセット |
| 調査ワークフロー | 14 | アプリ概要、関数ドシエ、ネイティブ API とディスパッチ、一括逆コンパイル、機能トレース、コールパス、コールグラフ、Swift と Objective-C の検出 |
| macOS ネイティブ | 8 | Hopper を起動せずに扱える Mach-O メタデータ、コード署名、plist、アーキテクチャ、Swift シンボルのデマングル、LLDB による呼び出し観測 |
| ネイティブバイナリ | 9 | macOS の Mach-O メタデータ、署名、plist、アーキテクチャ、Swift シンボル、LLDB による呼び出し観測;Linux の ELF レイアウトと静的な保護機構の証拠 |
| 成果物グラフ | 6 | ディレクトリとパッケージの目録、コンパイル済み Interface Builder ファイル、Apple アセットカタログ、Mach-O dylib 読み込みの解決、抽出 |
| マネージド PE/CLI | 7 | .NET の識別情報、メタデータ、CIL 命令、ネイティブ依存関係、再構築結果のインポート、ビルド比較 |
| ファームウェア | 2 | Linux ファームウェア領域の検査と明示的な展開 |
+1 -1
View File
@@ -207,7 +207,7 @@ REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이
| ----------------- | --: | ------------------------------------------------------------------------------------------------------------------------- |
| 네이티브 검사 | 41 | 함수, 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조, 주석, 바이트 읽기, 파일 오프셋 |
| 조사 워크플로 | 14 | 앱 개요, 함수 기록서, 네이티브 API와 디스패치, 일괄 디컴파일, 기능 추적, 호출 경로, 호출 그래프, Swift와 Objective-C 탐색 |
| macOS 네이티브 | 8 | Hopper 실행 없이 처리하는 Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 이름 복원, LLDB 호출 관찰 |
| 네이티브 바이너리 | 9 | macOS의 Mach-O 메타데이터, 서명, plist, 아키텍처, Swift 심볼과 LLDB 호출 관찰; Linux의 ELF 레이아웃과 정적 보호 증거 |
| 아티팩트 그래프 | 6 | 디렉터리와 패키지 목록, 컴파일된 Interface Builder 파일, Apple 애셋 카탈로그, Mach-O dylib 로드 해석, 추출 |
| 관리 PE/CLI | 7 | .NET 식별, 메타데이터, CIL 명령어, 네이티브 종속성, 재구성 가져오기, 빌드 비교 |
| 펌웨어 | 2 | Linux 펌웨어 영역 검사 및 명시적 추출 |
+1 -1
View File
@@ -206,7 +206,7 @@ REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常
| ----------------- | ---: | ------------------------------------------------------------------------------------------------------ |
| 原生检查 | 41 | 函数、伪代码、汇编、字符串、符号、调用、引用、注释、字节读取与文件偏移 |
| 调查工作流 | 14 | 应用概览、函数档案、原生 API 与分发、批量反编译、功能追踪、调用路径与调用图、Swift 与 Objective-C 发现 |
| macOS 原生工具 | 8 | Mach-O 元数据、代码签名、plist、架构、Swift 符号还原与 LLDB 调用观察,无需启动 Hopper |
| 原生二进制工具 | 9 | macOS 的 Mach-O 元数据、签名、plist、架构、Swift 符号与 LLDB 调用观察;Linux 的 ELF 布局和静态保护证据 |
| 产物图 | 6 | 目录与软件包清单、编译后的 Interface Builder 文件、Apple 资源目录、Mach-O dylib 加载解析与提取 |
| 托管 PE/CLI | 7 | .NET 程序集身份、元数据、CIL 指令、原生依赖声明、重建导入与构建比较 |
| 固件 | 2 | Linux 固件区域检查与显式提取 |
+39
View File
@@ -0,0 +1,39 @@
"""Catch typed resource failures while loading or running the owned ELF adapter.
Only this package's fixed sibling is executed; caller input stays in request.json.
Interpreter startup failures before this boundary remain unclassified.
"""
import os
import sys
def resource_exit(status, marker):
try:
if len(sys.argv) > 2:
descriptor = os.open(sys.argv[1], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
os.write(descriptor, marker)
os.close(descriptor)
except (OSError, MemoryError):
# Without a complete marker the parent keeps this failure unclassified.
pass
os._exit(status)
try:
implementation = os.path.join(os.path.dirname(__file__), "layout_impl.py")
with open(implementation, "rb") as handle:
source = handle.read()
exec(compile(source, implementation, "exec"), {
"__file__": implementation,
"__name__": "__main__",
})
except MemoryError:
# Reserved status: an observed allocation failure, including imports/compile.
resource_exit(75, b"M")
except OSError as error:
# Linux ENOMEM is an observed allocation failure, independent of error text.
if error.errno == 12:
resource_exit(75, b"M")
# Linux EFBIG is observed directly, including a partial limits/reply write.
if error.errno == 27:
resource_exit(76, b"F")
raise
+356
View File
@@ -0,0 +1,356 @@
"""Offline adapter to unchanged pwntools 4.15.0; never use libs/maps/libc.
Owned request/snapshot/reply paths are supplied by REA. No host target-process launch,
user configuration initialization, debugger startup, or dependency installation.
"""
import base64
import importlib.metadata
import io
import json
import os
from pathlib import Path
import resource
import sys
PROFILE = "[email protected];[email protected];[email protected]"
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
OUTPUT_BYTES = 64 * 1024 * 1024
class LayoutFailure(Exception):
def __init__(self, reason, message):
super().__init__(message)
self.reason = reason
def address(value):
return hex(int(value))
def file_range(offset, size, length):
return 0 <= offset <= length and 0 <= size <= length - offset
def location(offset, size, length):
if not file_range(offset, size, length):
raise LayoutFailure("format", "ELF contains a file-backed range outside the selected snapshot.")
return {"offset": address(offset), "bytes": address(size)}
def name_reference(display, section, offset, content):
header = getattr(section, "header", None)
if header is None or header.sh_type != "SHT_STRTAB":
return {"display": display, "bytes_base64": None, "location": None,
"unknown_reason": "Referenced string table could not be resolved."}
return name_in_table(display, header.sh_offset, header.sh_size, offset, content)
def name_in_table(display, table_start, table_size, offset, content):
location(table_start, table_size, len(content))
if not 0 <= offset < table_size:
raise LayoutFailure("format", f"ELF name offset {offset} lies outside its declared string table of {table_size} bytes at file offset {table_start}.")
start = table_start + offset
limit = table_start + table_size
end = content.find(b"\0", start, limit)
if end < 0:
raise LayoutFailure("format", "ELF name lacks a terminator within its reported string table.")
raw = content[start:end]
return {"display": raw.decode("utf-8", "replace") if display is None else display, "bytes_base64": base64.b64encode(raw).decode("ascii"),
"location": location(start, len(raw) + 1, len(content)), "unknown_reason": None}
def dynamic_name_reference(tags, offset, image, content):
starts = [tag.d_val for tag in tags if tag.d_tag == "DT_STRTAB"]
sizes = [tag.d_val for tag in tags if tag.d_tag == "DT_STRSZ"]
if not starts or not sizes:
raise LayoutFailure("format", "ELF dependency requires DT_STRTAB and DT_STRSZ.")
if len(starts) != 1 or len(sizes) != 1:
raise LayoutFailure("unsupported", "ELF dependency string-table tags are ambiguous: " + repr({"DT_STRTAB": starts, "DT_STRSZ": sizes}))
if not 0 <= offset < sizes[0]:
raise LayoutFailure("format", f"ELF dependency name offset {offset} lies outside declared DT_STRSZ {sizes[0]}.")
offsets = set()
if len(starts) == 1 and len(sizes) == 1:
for segment in image.iter_segments_by_type("PT_LOAD"):
h = segment.header
if starts[0] >= h.p_vaddr and starts[0] + sizes[0] <= h.p_vaddr + h.p_filesz:
offsets.add(h.p_offset + starts[0] - h.p_vaddr)
if len(offsets) != 1:
raise LayoutFailure("unsupported", "Dynamic string table lacks a unique file-backed mapping: " + repr({"DT_STRTAB": starts, "DT_STRSZ": sizes, "candidate_file_offsets": sorted(offsets)}))
return name_in_table(None, offsets.pop(), sizes[0], offset, content)
def symbol_value_meaning(raw, image_type, sections):
index = raw.st_shndx
if raw.st_info.type == "STT_FILE": return "no-address"
if index == "SHN_UNDEF": return "undefined"
if index == "SHN_COMMON": return "alignment"
if index == "SHN_ABS": return "absolute-value"
if not isinstance(index, int) or not 0 < index < len(sections) or index >= 0xff00:
return "unknown-section-index"
if sections[index].header.sh_type == "SHT_NULL":
return "unknown-section-index"
if image_type == "ET_REL": return "section-offset"
if raw.st_info.type == "STT_TLS": return "tls-offset"
return "linked-virtual-address"
def relocation_target(image_type, section_index, offset, sections):
if image_type != "ET_REL":
return {"kind": "linked-virtual-address", "address": address(offset)}
if section_index == 0 or sections[section_index].header.sh_type == "SHT_NULL":
return {"kind": "unknown-section", "reported_section_index": section_index,
"offset": address(offset), "unknown_reason": "undefined-section-reference" if section_index == 0 else "inactive-section-header"}
return {"kind": "section-offset", "section_index": section_index, "offset": address(offset)}
def inspect_elf(path, cache):
for package, expected in PACKAGES.items():
try:
version = importlib.metadata.version(package)
except importlib.metadata.PackageNotFoundError as error:
raise LayoutFailure("unavailable", "Selected Python lacks " + package + " " + expected + ".") from error
if version != expected:
raise LayoutFailure("unsupported", "Expected " + package + " " + expected + "; selected Python reports " + version + ".")
content = path.read_bytes()
if len(content) < 4 or content[:4] != b"\x7fELF":
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
if len(content) < 16:
raise LayoutFailure("format", "Selected ELF identification is truncated.")
if content[4] != 2 or content[5] != 1:
raise LayoutFailure("unsupported", "Initial layout profile requires ELF64 little-endian objects.")
if len(content) < 64:
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
if int.from_bytes(content[18:20], "little") != 62:
raise LayoutFailure("unsupported", "Initial layout profile requires x86-64 ELF objects.")
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
raise LayoutFailure("unsupported", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
from pwnlib.context import context
with context.local(cache_dir=str(cache), log_level="warning"):
from elftools.common.exceptions import ELFError
from elftools.elf.elffile import ELFFile
from pwnlib.elf import ELF
try:
headers = ELFFile(io.BytesIO(content))
section_count = headers.num_sections()
if headers.header.e_shstrndx == 0xffff and section_count == 0:
raise LayoutFailure("format", "ELF extended section-name index requires a declared section-zero header.")
raw_name_index = headers.header.e_shstrndx
if 0xff00 <= raw_name_index < 0xffff:
raise LayoutFailure("format", f"ELF section-name index {raw_name_index} is reserved and cannot directly reference a section.")
section_name_index = headers.get_shstrndx()
if raw_name_index == 0xffff and section_name_index < 0xff00:
raise LayoutFailure("format", f"ELF SHN_XINDEX resolves to ordinary index {section_name_index}; the escape requires an index at least SHN_LORESERVE (0xff00).")
if section_name_index != 0:
if section_name_index >= section_count:
raise LayoutFailure("format", f"ELF section-name table index {section_name_index} lies outside the declared section count {section_count}.")
names_header = headers._get_section_header(section_name_index)
if names_header.sh_type != "SHT_STRTAB":
raise LayoutFailure("format", f"ELF section-name table {section_name_index} is not SHT_STRTAB.")
location(names_header.sh_offset, names_header.sh_size, len(content))
except ELFError as error:
raise LayoutFailure("format", str(error)) from error
class LayoutELF(ELF):
def _get_section_name(self, header):
# SHN_UNDEF declares no names; upstream would otherwise read
# strings at the inactive section-zero offset.
return "" if section_name_index == 0 else super()._get_section_name(header)
# pyelftools 0.33 calls these documented filters internally;
# pwntools 4.15.0's cached overrides do not accept the argument.
# Forward the filter on this adapter instance, without modifying
# either upstream package or process-global methods.
def iter_segments(self, type=None):
return (segment for segment in super().iter_segments()
if type is None or segment.header.p_type == type)
def iter_sections(self, type=None):
return (section for section in super().iter_sections()
if type is None or section.header.sh_type == type)
try:
image = LayoutELF(str(path), checksec=False)
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
raise LayoutFailure("unsupported", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
raise LayoutFailure("unsupported", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
length = len(content)
all_sections = list(image.iter_sections())
shstrings = image.get_section(section_name_index) if all_sections and section_name_index != 0 else None
sections, segments, symbols, relocations, packed_relatives = [], [], [], [], []
needed, interpreters = [], []
for index, section in enumerate(all_sections):
h = section.header
backing = h.sh_type not in ("SHT_NOBITS", "SHT_NULL")
if backing: location(h.sh_offset, h.sh_size, length)
sections.append({
"index": index, "name": name_reference(section.name, shstrings, h.sh_name, content) if shstrings is not None else {"display": "", "bytes_base64": None, "location": None, "unknown_reason": "ELF declares no section-name table (SHN_UNDEF index 0)."}, "name_offset": address(h.sh_name), "type": h.sh_type,
"header_location": location(image.header.e_shoff + index * image.header.e_shentsize, image.header.e_shentsize, length),
"address": address(h.sh_addr), "offset": address(h.sh_offset),
"size": address(h.sh_size), "alignment": address(h.sh_addralign),
"flags": address(h.sh_flags), "link": h.sh_link, "info": h.sh_info,
"entry_size": address(h.sh_entsize), "file_backing": "file" if backing else "none",
})
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
strings = all_sections[h.sh_link]
for entry, symbol in enumerate(section.iter_symbols()):
raw = symbol.entry
symbols.append({
"table_index": index, "entry_index": entry,
"name": name_reference(symbol.name, strings, raw.st_name, content), "name_offset": address(raw.st_name),
"location": location(h.sh_offset + entry * h.sh_entsize, h.sh_entsize, length),
"value": address(raw.st_value), "value_meaning": symbol_value_meaning(raw, image.header.e_type, all_sections),
"size": address(raw.st_size), "binding": raw.st_info.bind, "type": raw.st_info.type,
"visibility": raw.st_other.visibility, "section_index": raw.st_shndx,
})
if h.sh_type in ("SHT_REL", "SHT_RELA"):
if image.header.e_type == "ET_REL" and h.sh_info >= len(all_sections):
raise LayoutFailure("format", f"ELF relocation section {index} references missing target section {h.sh_info}.")
symbol_table = None
if h.sh_link != 0:
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type not in ("SHT_SYMTAB", "SHT_DYNSYM"):
raise LayoutFailure("format", f"ELF relocation section {index} links to invalid symbol table {h.sh_link}.")
symbol_table = all_sections[h.sh_link]
for entry, relocation in enumerate(section.iter_relocations()):
raw = relocation.entry
if raw.r_info_sym != 0 and (symbol_table is None or raw.r_info_sym >= symbol_table.num_symbols()):
raise LayoutFailure("format", f"ELF relocation section {index} entry {entry} references missing symbol {raw.r_info_sym} in table {h.sh_link}.")
relocations.append({
"section_index": index, "entry_index": entry, "reported_offset": address(raw.r_offset),
"target": relocation_target(image.header.e_type, h.sh_info, raw.r_offset, all_sections),
"location": location(h.sh_offset + entry * h.sh_entsize, h.sh_entsize, length),
"type": raw.r_info_type, "symbol_table_index": h.sh_link, "symbol_index": raw.r_info_sym,
"symbol_reference_meaning": "zero-symbol-value" if raw.r_info_sym == 0 else "symbol-table-entry",
"addend": None if "r_addend" not in raw else str(raw.r_addend),
})
if h.sh_type == "SHT_RELR":
packed_relatives.append({
"section_index": index,
"location": location(h.sh_offset, h.sh_size, length),
"encoded_bytes_base64": base64.b64encode(content[h.sh_offset:h.sh_offset + h.sh_size]).decode("ascii"),
"entries": [{"decoded_index": decoded, "reported_offset": address(relocation.entry.r_offset)} for decoded, relocation in enumerate(section.iter_relocations())],
"offset_meaning": "unknown" if image.header.e_type == "ET_REL" else "linked-virtual-address",
"evidence_kind": "derived", "entry_source_locations": None, "addends": None,
})
for index, segment in enumerate(image.iter_segments()):
h = segment.header
backing = h.p_type != "PT_NULL" and h.p_filesz != 0
if backing: location(h.p_offset, h.p_filesz, length)
if h.p_type == "PT_LOAD" and h.p_memsz < h.p_filesz:
raise LayoutFailure("format", "ELF load segment memory size is smaller than its file size.")
segments.append({
"index": index, "type": h.p_type, "offset": address(h.p_offset),
"header_location": location(image.header.e_phoff + index * image.header.e_phentsize, image.header.e_phentsize, length),
"file_size": address(h.p_filesz), "memory_size": address(h.p_memsz),
"virtual_address": address(h.p_vaddr), "physical_address": address(h.p_paddr),
"alignment": address(h.p_align), "flags": address(h.p_flags),
"file_backing": "file" if backing else "none",
"permissions": None if h.p_type == "PT_NULL" else {"read": bool(h.p_flags & 4), "write": bool(h.p_flags & 2), "execute": bool(h.p_flags & 1)},
})
if h.p_type == "PT_DYNAMIC":
tag_size = image.structs.Elf_Dyn.sizeof()
tags = []
for tag_index in range(h.p_filesz // tag_size):
tag = segment._get_tag(tag_index)
tags.append(tag)
if tag.d_tag == "DT_NULL":
break
else:
raise LayoutFailure("format", f"ELF dynamic segment {index} has no complete DT_NULL terminator within its reported file range.")
for tag in tags:
if tag.d_tag == "DT_NEEDED":
needed.append(dynamic_name_reference(tags, tag.d_val, image, content))
if h.p_type == "PT_INTERP":
raw = content[h.p_offset:h.p_offset + h.p_filesz]
end = raw.find(b"\0")
if end < 0:
raise LayoutFailure("format", "ELF interpreter lacks a reported terminator.")
interpreters.append({"display": raw[:end].decode("utf-8", "replace"), "bytes_base64": base64.b64encode(raw[:end]).decode("ascii"), "location": location(h.p_offset, end + 1, length), "unknown_reason": None})
return {
"format": "elf", "architecture": {"machine": image.header.e_machine, "bits": image.bits, "byte_order": image.endian},
"image_type": image.header.e_type,
"entry_point": {"reported_value": address(image.header.e_entry), "meaning": "not-applicable" if image.header.e_type == "ET_REL" else "absent" if image.header.e_entry == 0 else "linked-virtual-address", "execution_status": "unknown"},
"runtime_load_base": None, "sections": sections, "segments": segments, "symbols": symbols, "relocations": relocations,
"packed_relative_relocations": packed_relatives, "relocation_inventory_completeness": "unknown",
"linkage": {"needed_libraries": needed, "interpreters": interpreters,
"got": [{"display_name": name, "address": address(value)} for name, value in image.got.items()],
"plt": [{"display_name": name, "address": address(value)} for name, value in image.plt.items()],
"convenience_maps_completeness": "unknown", "runtime_library_paths": None},
"mitigations": {"evidence_kind": "inferred", "position_independent": image.pie, "nx_indicator": image.nx, "executable_stack_indicator": image.execstack, "stack_canary_indicator": image.canary, "relro": image.relro},
"limitations": [
"PT_NULL payload fields are unused and retain their reported numbers without file-byte or permission claims. Zero-file-size segments provide no file bytes.",
"SHN_XINDEX symbol values remain unresolved in this upstream representation; an external index table does not establish a resolved index in this report.",
"Reported values preserve linked addresses, section offsets, TLS offsets, alignment and absolute values separately. Runtime load base and library paths are unknown.",
"Names are display strings plus raw bytes and file ranges where resolvable; section/table/entry indices preserve identity and duplicates. Name ranges include their terminating NUL; raw name bytes exclude it.",
"Ordinary relocation rows cover REL/RELA. RELR tables retain complete encoded bytes and upstream-decoded offsets as derived evidence; per-offset packed-word locations and implicit addends remain unknown. Relocation inventory completeness is unknown, including other encodings and missing section tables.",
"Relocation symbol index zero uses a zero symbol value without a table lookup. Positive indices reference validated original symbol-table entries; raw table and symbol indices remain reported.",
"Relocatable relocation targets with SHN_UNDEF or inactive SHT_NULL headers retain their reported section index/offset as unknown; they do not establish an associated target section.",
"GOT/PLT are derived upstream convenience maps; PLT inference can emulate selected instructions in Unicorn. They do not establish target runtime behavior; aliases may collapse and warnings may indicate incomplete coverage. Their completeness is unknown.",
"Mitigations are upstream static heuristics, not runtime protection. ET_DYN does not prove an executable; absent canary symbols do not prove every function unprotected.",
"This profile inspects ELF EXEC/DYN/REL layout only. It does not analyze recorded cores, resolve loaded libraries, launch the target as a host process or start a debugger.",
"Symbol/relocation inventories reflect original section tables. A sectionless image can still report dynamic dependency names; missing tables do not prove absence of dynamic symbols or relocations.",
*(["Absent section-name table: section-dependent RELRO/canary heuristics may be incomplete; protection coverage is unknown."] if all_sections and section_name_index == 0 else []),
*(["Sectionless images lack the .dynamic section and symbol tables used by upstream heuristics. RELRO/canary indicators may be incomplete; their values remain reported static candidates, with protection coverage unknown."] if not all_sections else []),
],
}
except LayoutFailure:
raise
except (ELFError, ValueError, AssertionError, IndexError) as error:
raise LayoutFailure("format", "Selected ELF failed unchanged upstream structural parsing: " + str(error)) from error
def lower_resource_limits():
limits = {}
for name, kind, maximum in (
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
("cpu_seconds", resource.RLIMIT_CPU, 30),
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
):
soft, hard = resource.getrlimit(kind)
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
effective = min(finite)
# Preserve the inherited hard boundary and never raise a caller's soft limit.
resource.setrlimit(kind, (effective, hard))
limits[name] = effective
return limits
def main(request_path):
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
os.environ["PWNLIB_NOTERM"] = "1"
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
limits = None
# Release bounded emergency headroom before constructing a MemoryError reply.
# The outer exit-status contract also covers allocation failure during serialization.
memory_reserve = bytearray(1024 * 1024)
try:
limits = lower_resource_limits()
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(limits, handle)
value = inspect_elf(Path(request["snapshot_path"]), Path(request_path).parent / "cache")
value["limitations"].append("Effective owned Python resource soft limits: " + json.dumps(limits, sort_keys=True) + ". Inherited tighter limits are retained.")
reply = {"ok": True, "profile": PROFILE, "value": value}
except LayoutFailure as error:
reply = {"ok": False, "reason": error.reason, "message": str(error)}
except MemoryError:
memory_reserve = None
reply = {"ok": False, "reason": "resource-limit", "message": "pwntools memory allocation failed under the effective resource limits; the exact allocation cause is unknown.", "reported_limits": limits}
except Exception as error:
if isinstance(error, OSError) and error.errno == 12:
memory_reserve = None
reply = {"ok": False, "reason": "resource-limit", "message": "pwntools reported OSError(ENOMEM); the exact allocation cause is unknown.", "reported_limits": limits}
else:
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete ELF layout exceeds the 64 MiB reply budget."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
if __name__ == "__main__":
main(sys.argv[-1])
+4
View File
@@ -26,6 +26,7 @@ graph TD
end
subgraph "Provider Layers"
PWNTOOLS["src/native/pwntools/<br/>BYO offline ELF adapter<br/>owned Python process, unchanged upstream tables/heuristics"]
HOPPER["src/hopper/<br/>Hopper launch & Unix-socket protocol<br/>BridgeLauncher, HopperClient, protocol"]
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
@@ -92,6 +93,9 @@ graph TD
APP -->|record callbacks| RECORDS
SERVER -->|narrow record ports| RECORDS
RECORDS --> DOMAIN
COMPOSITION --> PWNTOOLS
PWNTOOLS --> PROCESS
PWNTOOLS --> DOMAIN
COMPOSITION --> HOPPER
COMPOSITION --> GHIDRA
COMPOSITION --> IDA
+131
View File
@@ -0,0 +1,131 @@
# Offline binary diagnostics
`inspect_binary_layout` / `inspect-binary-layout` answers what an explicit
file contains: section/segment layout, original symbols/relocations, static
linkage names and mitigation indicators. It works without an active Hopper,
Ghidra or IDA target. CTF is one possible use; crash triage, compatibility and
ordinary binary inspection use the same modular capability.
## Bring your own engine
Initial real verification covers **Linux x64, ELF64 x86-64 little-endian
EXEC/DYN/REL**. Provide an absolute Python executable whose environment already
contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
```sh
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
rea inspect-binary-layout ./selected.elf --json
```
```json
{
"name": "inspect_binary_layout",
"arguments": { "path": "/artifacts/selected.elf" }
}
```
REA never installs Python, packages or GDB, changes a user init file, launches
the selected object as a host process, or requests runtime library resolution. The Python process
uses isolated mode and an owned cache. Exact upstream profiles are recorded in
[upstream provenance](../third_party/pwntools/README.md).
## Interpreting results
- Artifact path, SHA-256 and size identify original selected bytes. All parsing
reads an owned stable snapshot; the original is unchanged.
- Addresses, offsets, lengths and flags are hexadecimal strings. A linked
address is not a runtime address. Runtime load base remains null. A zero
EXEC/DYN entry means absent; a relocatable entry is not applicable.
- Each symbol retains its table and entry index. Its reported value can mean
undefined, alignment, absolute value, no address, unknown section index,
section offset, TLS offset or linked virtual address. Duplicate names remain
separate entries. SHN_XINDEX remains unresolved in the selected upstream
representation; its external index table is not silently treated as resolved.
- Relocatable objects have section-relative relocations and no executable entry
claim. Signed REL/RELA relocation addends are decimal strings. Packed RELR
tables retain full original bytes and upstream-decoded offsets as derived
evidence, with per-offset packed-word locations and implicit addends unknown.
Overall relocation inventory completeness remains unknown.
A relocatable SHN_UNDEF target or inactive SHT_NULL header remains an explicit
unknown with the original reported section index and offset.
- REL/RELA symbol references retain their original table and entry indices.
Positive indices resolve through a validated symbol table; symbol index zero
means a zero symbol value without a table lookup, including when no table is
linked. Malformed references fail with the affected section and symbol index.
- Section-name and symbol tables require declared SHT_STRTAB links.
SHN_UNDEF explicitly means no section names: display is empty, raw name and
location unknown, with the original name offsets retained. Extended section-name
indices resolve through section zero and undergo the same validation.
- Symbol tables require a declared SHT_STRTAB link. Name offsets and terminators
must stay inside the declared string table; malformed references fail before
unrelated bytes can become names. Dynamic dependencies use DT_STRTAB/DT_STRSZ
with a unique file-backed mapping; ambiguous or unbacked mappings are
explicitly unsupported.
- Name display strings use UTF-8 replacement for opaque bytes, including
sectionless dependency names and interpreter paths. Raw name
bytes and string-table ranges retain observed identity where resolvable;
ranges include the terminating NUL and base64 bytes exclude it.
- NOBITS and NULL sections provide no file bytes. Reported file-backed ranges
are checked against snapshot size again at the public boundary. PT_NULL payload
fields are unused: reported numbers remain, file backing is none and interpreted
permissions are null. Segments with zero file size also have no file bytes.
- Sectionless dynamic images still report dependency names with raw bytes and
file locations where uniquely mapped. Missing symbol/relocation section tables
do not establish that those runtime tables are absent.
Upstream RELRO/canary heuristics rely on sections and can be incomplete in
this case; their reported candidates retain an explicit coverage limitation.
- DT_NEEDED/PT_INTERP are reported names, not resolved runtime paths. GOT/PLT
maps are derived convenience views and may collapse aliases. PLT inference
can emulate selected instructions in Unicorn; its outputs are static candidates.
Exec syscall tracing observes host process launches. Map completeness
remains unknown and original upstream warnings are returned inline.
- Canary, PIE, NX, executable-stack and RELRO are static inferences. NX and
executable-stack are separate upstream indicators: on x86-64 an executable
stack can accompany an unknown (`null`) NX indicator. ET_DYN can be a shared
library, and an absent canary symbol does not prove every function unprotected.
Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics
and 30-second owned command deadline. The reply budget applies to the decoder
record; the CLI/MCP Evidence envelope adds copies and encoding overhead. The
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
a caller-configured `StdioClientTransport({ maxBufferSize: ... })`; the large
extended-index verification uses 256 MiB and an explicit five-minute request
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
retained, and the effective values are returned in the result limitations. Virtual address space is not RSS;
Unicorn needs a large virtual map. Limits fail with no partial success, and
owned process/root cleanup completes independently of cancellation.
Core files, GDB session/state/control, optional pwndbg enrichment and instruction
inspection are separate increments tracked by #969. This increment makes no
real support claim for those features or other operating systems/architectures.
Typed decoder failures retain bounded `captured_output` (stdout, stderr and an
explicit truncation flag) alongside the original failure category and reason.
Successful `diagnostics` also exposes the supervisor's `truncated` flag. Output
beyond the complete diagnostic budget is rejected, including when the decoder
writes a valid reply; cleanup and late cancellation reuse the observed flag.
Memory allocation failures use `resource_constraint`, with effective resource
limits (or an explicit unknown) and memory-specific recovery guidance. A
MemoryError alone does not establish the exact failed allocation or that a
particular budget was exhausted. Invalid undersized symbol entries are rejected
before REA reports source ranges for them.
The bridge reserves 1 MiB to report allocation failures. If even error reporting
or serialization fails with MemoryError, its reserved exit status preserves the
resource classification with unknown effective limits.
Reserved exits require a matching private marker written by the bridge's actual
failure branch. Bare launcher exits or missing/unwritable markers retain process
diagnostics with the resource cause unverified.
Signal termination alone is never classified as observed memory exhaustion.
The small bootstrap also covers catchable MemoryError during adapter imports,
compilation and initialization. Python interpreter startup failures before the
bootstrap runs retain their observed process diagnostics without guessed causes.
SIGXCPU retains a CPU resource diagnostic and CPU-specific recovery advice.
Observed EFBIG writes or SIGXFSZ termination retain a file-size resource
diagnostic and file-size-specific recovery advice. If a tight file limit also
prevents writing the limit record, effective values remain unknown.
The bridge records actual limits in owned storage before analysis; missing or
malformed limit reports remain unknown with their read failure preserved. A
received signal alone does not establish its exact cause. Dynamic tags require
a complete DT_NULL inside PT_DYNAMIC; interpreter names and ranges end at the
first NUL, with any padding still represented by the original segment range.
+53
View File
@@ -14,6 +14,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -48,6 +49,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -82,6 +84,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -116,6 +119,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -150,6 +154,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -184,6 +189,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -218,6 +224,42 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
},
"message": { "type": "string", "minLength": 1 },
"retryable": { "type": "boolean" },
"remediation": {
"type": "object",
"properties": { "action": { "type": "string", "minLength": 1 } },
"required": ["action"],
"additionalProperties": false
},
"details": {
"type": "object",
"propertyNames": { "type": "string" },
"additionalProperties": {}
}
},
"required": ["code", "category", "message", "retryable", "remediation"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"code": { "type": "string", "const": "resource_constraint" },
"category": {
"type": "string",
"enum": [
"invalid_input",
"unsupported_provider",
"integrity_mismatch",
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -252,6 +294,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -286,6 +329,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -342,6 +386,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -376,6 +421,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -410,6 +456,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -444,6 +491,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -478,6 +526,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -512,6 +561,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -546,6 +596,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -580,6 +631,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
@@ -614,6 +666,7 @@
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure"
]
+17 -9
View File
@@ -7,10 +7,10 @@
"client": "2.3.1",
"core": "2.3.1"
},
"skill_version": "30"
"skill_version": "31"
},
"tools": {
"total": 136,
"total": 137,
"families": [
{
"id": "direct",
@@ -84,10 +84,11 @@
{
"id": "native",
"surface": "native-provider",
"count": 8,
"count": 9,
"tools": [
"capture_native_ui_scenario",
"demangle_swift",
"inspect_binary_layout",
"inspect_macho",
"inspect_plist",
"inspect_signature",
@@ -373,6 +374,12 @@
"observe_native_ui"
]
},
{
"id": "pwntools-elf",
"name": "REA pwntools ELF adapter",
"version": "[email protected];[email protected];[email protected]",
"capabilities": ["inspect_binary_layout"]
},
{
"id": "rea-android-application",
"name": "REA Android application workflows",
@@ -587,7 +594,7 @@
}
],
"cli": {
"primary_count": 91,
"primary_count": 92,
"commands": [
"address-to-file-offset",
"analyze",
@@ -629,6 +636,7 @@
"inspect-android-package",
"inspect-artifact",
"inspect-asset-catalog",
"inspect-binary-layout",
"inspect-electron-page",
"inspect-firmware-regions",
"inspect-keyed-archive",
@@ -690,15 +698,15 @@
},
"runtime_catalog": {
"counts": {
"cli_commands": 91,
"mcp_tools": 136,
"cli_commands": 92,
"mcp_tools": 137,
"mcp_prompts": 6
},
"digests": {
"tools_sha256": "6d195c42333ecfa46e9b253f15174876b34ed1fb226047d1327e3be960680f26",
"tools_sha256": "1f9455a546d76d63e9ceba7bfdc6045f6abde8ead8492c55bd0cdfa726061a91",
"prompts_sha256": "ef2de8814390b340623c32001610aa2709d90bed51537b6e562ab3abf144ab4d",
"combined_sha256": "047ba77689d22d3b3dac6f51fae06a1a306aaab24fa3be7c2007b396a5fd26f7",
"providers_sha256": "96cfc3247384603286780f35bc403870bbca53a648959cbe36bc28dbb43700e5"
"combined_sha256": "ce23d159fbfc07b338a5e2d51569a71597842f0cf3a3c09986c799512752ab02",
"providers_sha256": "6a96ebafaa7c9284661db6c2036524dcba0407e85278c6bb5ac12bbad0d85a6f"
}
}
}
+17
View File
@@ -626,6 +626,23 @@ compiler. Ordinary unit/static gates acquire no browser. See
[website runtime attribution](web-runtime.md) for effects, resource bounds and
coverage limits.
### Offline binary layout
`npm run verify:binary:layout` requires Linux x64, GCC/binutils, absolute
`REA_PWNTOOLS_PYTHON` with pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2 and
absolute `REA_VERIFY_STRACE_COMMAND`. It compiles ephemeral source-owned ELF
fixtures and checks public CLI/MCP, lossless addresses/names, file ranges,
mitigation inferences, malformed/unsupported input, original file hashes and
released process ownership. Exec syscall tracing must identify only the declared
Node/Python launchers; no target binary is executed. Core/debugger claims need
separate verification lanes. Pass an installed package entrypoint as the script's
first argument to verify packaging independently of the checkout.
The valid SHN_XINDEX fixture has 65,281 full section rows; CLI is checked in
the ordinary lane. Its large MCP transfer is opt-in with
`REA_VERIFY_LARGE_ELF_MCP=1` (or the workflow dispatch `large_mcp` input), an
explicit 256 MiB SDK receive buffer and five-minute request timeout. Ordinary
MCP fixtures retain the pinned SDK defaults.
## Agent evaluation and conformance records
Evaluate native, JavaScript, managed and browser investigation tasks through a
@@ -1,5 +1,5 @@
{
"manifest_id": "ecm_5d2fd13510a46479a522d904504fdf959a0feb9d7773cbffba50d3ae6e3e9a79",
"manifest_id": "ecm_2821fc2b686037a57ed7e5847a3927cc95b598d222a54983b34d87402e18bd30",
"verifier": {
"id": "rea-managed-conformance",
"version": "1"
@@ -13,7 +13,7 @@
"skill_digests": [
{
"skill_id": "reverse-engineer-anything",
"sha256": "8d4c905cac2ea90440a072cdfefc6d6830ea33a3688855c745d2e870bc40960f"
"sha256": "21157bdb015b42cf2fafc15fe150605ca89cc564761bb24e87b5bd6a00da7f19"
}
],
"claims": [
+6 -2
View File
@@ -63,7 +63,10 @@
"LICENSE",
"native/windows/build/manifest.json",
"native/windows/build/rea-windows-x64.node",
"scripts/verify-windows-native-artifact.mjs"
"scripts/verify-windows-native-artifact.mjs",
"bridge/pwntools/layout.py",
"bridge/pwntools/layout_impl.py",
"third_party/pwntools/README.md"
],
"scripts": {
"deps:check": "node scripts/check-dependency-install.mjs",
@@ -195,7 +198,8 @@
"verify:module-boundaries": "node scripts/verify-module-boundaries.mjs",
"verify:browser:source-maps": "npm run build:cached && node scripts/verify-browser-source-maps.mjs",
"verify:browser:runtime": "npm run build:cached && node scripts/verify-browser-runtime.mjs",
"verify:web:network-captures": "npm run build:cached && node scripts/verify-web-network-captures.mjs"
"verify:web:network-captures": "npm run build:cached && node scripts/verify-web-network-captures.mjs",
"verify:binary:layout": "npm run build:cached && node scripts/verify-binary-layout.mjs"
},
"lint-staged": {
"*.{cjs,cts,js,jsx,mjs,mts,ts,tsx}": [
+10
View File
@@ -0,0 +1,10 @@
.globl _start
.globl absolute_value
.set absolute_value, 0x20000000000007
.bss
.lcomm zero_backing, 8192
.text
_start:
mov $60, %eax
xor %edi, %edi
syscall
+17
View File
@@ -0,0 +1,17 @@
#include <stdio.h>
#include <stdlib.h>
int stored_value = 7;
char zero_backing[8192];
__thread int tls_value;
int common_value;
__attribute__((noinline)) int read_values(const char *text) {
char buffer[32];
snprintf(buffer, sizeof(buffer), "%s", text);
return buffer[0] + stored_value + tls_value + common_value;
}
int main(int argc, char **argv) {
return read_values(argc > 1 ? argv[1] : "source-owned");
}
+9 -1
View File
@@ -16,7 +16,10 @@ export const toolFamilyCatalog = (sources) => {
{
id: "native",
surface: "native-provider",
contracts: sources.nativeContracts.NATIVE_TOOL_CONTRACTS,
contracts: [
...sources.nativeContracts.NATIVE_TOOL_CONTRACTS,
...sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS,
],
},
{
id: "artifact",
@@ -143,6 +146,11 @@ export const providerCatalog = (sources) => {
identity: sources.nativeProvider.NATIVE_MACOS_PROVIDER_IDENTITY,
contracts: sources.nativeContracts.NATIVE_TOOL_CONTRACTS,
},
{
identity: sources.pwntoolsProvider.PWNTOOLS_PROVIDER_IDENTITY,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS,
},
{
identity: sources.artifactProviders.ARTIFACT_GRAPH_PROVIDER,
contracts: sources.artifactContracts.ARTIFACT_TOOL_CONTRACTS,
+3
View File
@@ -40,6 +40,9 @@ const SOURCE_PATHS = {
officialContracts: "dist/contracts/officialToolContracts.js",
enhancedContracts: "dist/contracts/enhancedToolContracts.js",
sessionContracts: "dist/contracts/sessionToolContracts.js",
binaryDiagnosticsContracts:
"dist/contracts/native/binaryDiagnosticsToolContracts.js",
pwntoolsProvider: "dist/native/pwntools/PwntoolsRelease.js",
nativeContracts: "dist/contracts/native/nativeToolContracts.js",
artifactContracts: "dist/contracts/artifactToolContracts.js",
managedContracts: "dist/contracts/managed/managedToolContracts.js",
+88
View File
@@ -0,0 +1,88 @@
import assert from "node:assert/strict";
/** Source-owned ELF64 mutations for declared, absent and escaped name-table references. */
export const sectionNameFixtures = (original, report) => {
// ELF gABI: https://gabi.xinuos.com/elf/02-eheader.html#e-shstrndx
// SHN_XINDEX must carry an actual index >= SHN_LORESERVE, not a small index.
const sourceTable = report.sections[original.readUInt16LE(62)];
assert.equal(sourceTable.type, "SHT_STRTAB");
const sourceHeader = Number(BigInt(sourceTable.header_location.offset));
const fixtures = [];
for (const problem of [
"out-of-range",
"wrong-type",
"undeclared-header",
"extended-undeclared-header",
"extended-without-sections",
"extended-absent",
"extended-small",
]) {
let bytes = Buffer.from(original);
let selected = report.sections.length + 100;
if (problem === "wrong-type")
selected = report.sections.find(
(section) => section.type === "SHT_PROGBITS",
).index;
else if (problem.includes("undeclared-header")) {
const end =
Number(bytes.readBigUInt64LE(40)) +
report.sections.length * bytes.readUInt16LE(58);
const extended = Buffer.alloc(Math.max(bytes.length, end + 64));
bytes.copy(extended);
bytes.copy(extended, end, sourceHeader, sourceHeader + 64);
bytes = extended;
selected = report.sections.length;
}
if (problem === "extended-without-sections") {
bytes.writeBigUInt64LE(0n, 40);
bytes.writeUInt16LE(0, 60);
bytes.writeUInt16LE(0xffff, 62);
} else if (problem.startsWith("extended-")) {
bytes.writeUInt16LE(0xffff, 62);
if (problem === "extended-absent") selected = 0;
if (problem === "extended-small") selected = sourceTable.index;
bytes.writeUInt32LE(selected, Number(bytes.readBigUInt64LE(40)) + 40);
} else bytes.writeUInt16LE(selected, 62);
fixtures.push({ name: problem, bytes, expectation: "invalid_input" });
}
const absent = Buffer.from(original);
absent.writeUInt16LE(0, 62);
fixtures.push({ name: "absent", bytes: absent, expectation: "absent" });
// A genuine escape requires a large table. Relocate the original table, keep
// its referenced indices and move only the name table to index SHN_LORESERVE.
const selected = 0xff00,
count = selected + 1;
const start = Math.ceil(original.length / 8) * 8;
const large = Buffer.alloc(start + count * 64);
original.copy(large);
const oldStart = Number(original.readBigUInt64LE(40));
original.copy(large, start, oldStart, oldStart + report.sections.length * 64);
large.fill(
0,
start + sourceTable.index * 64,
start + (sourceTable.index + 1) * 64,
);
original.copy(large, start + selected * 64, sourceHeader, sourceHeader + 64);
large.writeBigUInt64LE(BigInt(start), 40);
large.writeUInt16LE(0, 60);
large.writeUInt16LE(0xffff, 62);
large.writeBigUInt64LE(BigInt(count), start + 32);
large.writeUInt32LE(selected, start + 40);
fixtures.push({
name: "extended-large",
bytes: large,
expectation: "resolved",
tableIndex: selected,
sectionCount: count,
});
const reserved = Buffer.from(large);
reserved.writeUInt16LE(selected, 62);
reserved.writeUInt32LE(0, start + 40);
fixtures.push({
name: "direct-reserved",
bytes: reserved,
expectation: "invalid_input",
});
return fixtures;
};
+27
View File
@@ -0,0 +1,27 @@
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
/** Own one SDK client with an explicit receive budget for a large complete result. */
export const withLargeResultMcp = async (
{ entrypoint, environment, maxBufferSize },
useClient,
) => {
const client = new Client({ name: "large-result-verifier", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
env: environment,
stderr: "inherit",
maxBufferSize,
});
try {
await client.connect(transport);
return await useClient(client);
} finally {
try {
await client.close();
} finally {
await transport.close();
}
}
};
File diff suppressed because it is too large Load Diff
+8 -3
View File
@@ -2,9 +2,9 @@
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "30"
tool_count: 136
catalog_digest: "047ba77689d22d3b3dac6f51fae06a1a306aaab24fa3be7c2007b396a5fd26f7"
version: "31"
tool_count: 137
catalog_digest: "ce23d159fbfc07b338a5e2d51569a71597842f0cf3a3c09986c799512752ab02"
---
# REA
@@ -99,6 +99,11 @@ explicit path or endpoint and do not need it.
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
profile and credential exclusions.
- User-owned Electron runtime already open: `list_electron_targets`.
- Explicit Linux ELF file for offline layout, symbols, relocations or static
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
caller-supplied pwntools without opening a disassembler database. Preserve its
raw locations and inference/coverage limits; see the
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
- Native executable, library, or analysis database: `open_binary`, then
use focused analysis tools directly; call `binary_overview` when metadata or
inventory context is useful and available from the selected provider.
+9
View File
@@ -33,6 +33,7 @@ type ProviderDescriptor = ProviderCapability;
export type AvailabilityPolicy = {
readonly processCaptureEnabled: boolean;
readonly optionalProviderLoadFailures?: OptionalProviderLoadFailures;
readonly binaryLayoutEnabled?: boolean;
readonly firmwareInspectionEnabled?: boolean;
readonly firmwareExtractionEnabled?: boolean;
readonly androidAnalysisEnabled?: boolean;
@@ -280,6 +281,14 @@ const workflowAvailabilityFor = ({
remediation:
"On Linux x64, provide an absolute REA_WAKARU_COMMAND for Wakaru 1.13.0 and util-linux prlimit. No binary target is required.",
};
if (name === "inspect_binary_layout")
return policy.binaryLayoutEnabled === true
? { reason: "available", remediation: null }
: {
reason: "provider_missing",
remediation:
"On Linux x64, set absolute REA_PWNTOOLS_PYTHON to caller-supplied Python with pwntools 4.15.0. No active binary target is required.",
};
if (kind === "firmware-provider") {
const enabled =
name === "inspect_firmware_regions"
@@ -0,0 +1,17 @@
import type { ExecutionOptions } from "../AnalysisProvider.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type {
BinaryLayout,
InspectBinaryLayoutInput,
} from "../../domain/native/binaryLayout.js";
import type { Result } from "../../domain/result.js";
import type { ProviderIdentity } from "../AnalysisProvider.js";
/** Replaceable offline binary-layout provider; owns its parser and process protocol. */
export interface BinaryLayoutPort {
readonly identity: ProviderIdentity;
inspect(
input: InspectBinaryLayoutInput,
options?: ExecutionOptions,
): Promise<Result<BinaryLayout, AnalysisError>>;
}
@@ -0,0 +1,131 @@
import { expect, it } from "vitest";
import { BinaryLayoutService } from "./BinaryLayoutService.js";
import {
binaryLayoutFixture,
BINARY_LAYOUT_TEST_PROVIDER,
} from "../../../tests/fixtures/binaryDiagnostics/layout.js";
import { parseEvidence } from "../../domain/evidence.js";
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
import { err, ok } from "../../domain/result.js";
it("binds original bytes, inline observations, unknowns and producing identity", async () => {
const fixture = binaryLayoutFixture();
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => Promise.resolve(ok(fixture)),
});
const result = await service.inspect({ path: fixture.artifact.path });
if (!result.ok) throw result.error;
const evidence = parseEvidence(result.value);
expect(evidence.normalized_result).toEqual(fixture);
expect(evidence.raw_result).toEqual(fixture);
expect(evidence.subject?.digest.sha256).toBe(fixture.artifact.sha256);
expect(evidence.provider).toEqual(BINARY_LAYOUT_TEST_PROVIDER);
expect(evidence.confidence).toBe("observed");
});
it.each([
{ path: "relative.elf" },
{ path: "/selected.elf", approval: true },
{ path: 123 },
{},
])(
"rejects malformed selection before starting the provider: %j",
async (input) => {
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => {
throw new Error("must not run");
},
});
expect(await service.inspect(input)).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
},
);
it("preserves a provider's actionable unsupported reason", async () => {
const failure = new AnalysisCapabilityUnavailableError(
"fixture-layout",
"inspect_binary_layout",
"ELF core belongs to recorded-crash analysis.",
);
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => Promise.resolve(err(failure)),
});
expect(await service.inspect({ path: "/source.elf" })).toEqual(err(failure));
});
it("cancels before invocation and after an acquired observation", async () => {
const controller = new AbortController();
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => {
controller.abort();
return Promise.resolve(ok(binaryLayoutFixture()));
},
});
expect(
await service.inspect(
{ path: "/artifacts/source-owned.elf" },
{ signal: controller.signal },
),
).toMatchObject({
ok: false,
error: {
_tag: "AnalysisCancelledError",
capturedOutput: binaryLayoutFixture().diagnostics,
},
});
const untouched = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => {
throw new Error("must not run");
},
});
expect(
await untouched.inspect(
{ path: "/source.elf" },
{ signal: controller.signal },
),
).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } });
});
it.each(["different-path", "invalid-digest", "outside-range"])(
"rejects provider evidence at the next boundary: %s",
async (problem) => {
const fixture = binaryLayoutFixture();
if (problem === "different-path") fixture.artifact.path = "/different.elf";
if (problem === "invalid-digest") fixture.artifact.sha256 = "unobserved";
if (problem === "outside-range")
fixture.segments.push({
index: 0,
type: "PT_LOAD",
offset: "0x80",
header_location: { offset: "0x0", bytes: "0x38" },
file_backing: "file",
file_size: "0x1",
memory_size: "0x1",
virtual_address: "0x20000000000001",
physical_address: "0x0",
alignment: "0x1",
flags: "0x5",
permissions: { read: true, write: false, execute: true },
});
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => Promise.resolve(ok(fixture)),
});
expect(
await service.inspect({ path: "/artifacts/source-owned.elf" }),
).toMatchObject({
ok: false,
error: {
_tag: "AnalysisOutputError",
capturedOutput: fixture.diagnostics,
},
});
},
);
@@ -0,0 +1,91 @@
import { isAbsolute } from "node:path";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import type { BinaryLayoutPort } from "./BinaryLayoutPort.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../../domain/analysisErrorCore.js";
import { createEvidence, type Evidence } from "../../domain/evidence.js";
import {
inspectBinaryLayoutInputSchema,
binaryLayoutSchema,
} from "../../domain/native/binaryLayout.js";
import { projectInputIssues } from "../../domain/inputIssueProjection.js";
import { err, ok, type Result } from "../../domain/result.js";
const OPERATION = "inspect_binary_layout";
/** Shared CLI/MCP validation and Evidence composition for offline layout inspection. */
export class BinaryLayoutService {
constructor(readonly provider: BinaryLayoutPort) {}
/** Bind complete observations to the caller-selected artifact before creating Evidence. */
async inspect(
rawInput: unknown,
options?: ExecutionOptions,
): Promise<Result<Evidence, AnalysisError>> {
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION));
const input = inspectBinaryLayoutInputSchema.safeParse(rawInput);
if (!input.success)
return err(
new AnalysisInputError(
OPERATION,
{ cause: input.error },
projectInputIssues(input.error.issues, rawInput),
),
);
if (!isAbsolute(input.data.path))
return err(
new AnalysisInputError(OPERATION, undefined, [
{
path: ["path"],
reason: "invalid_format",
message: "Expected an absolute filesystem path on this host.",
},
]),
);
const inspected = await this.provider.inspect(input.data, options);
if (!inspected.ok) return inspected;
const diagnostics = binaryLayoutSchema.shape.diagnostics.safeParse(
inspected.value.diagnostics,
);
const outputOptions = diagnostics.success
? { capturedOutput: diagnostics.data }
: undefined;
if (options?.signal?.aborted)
return err(new AnalysisCancelledError(OPERATION, outputOptions));
const report = binaryLayoutSchema.safeParse(inspected.value);
if (!report.success || report.data.artifact.path !== input.data.path)
return err(
new AnalysisOutputError(
OPERATION,
"Binary layout provider returned malformed evidence or changed the selected artifact identity.",
outputOptions,
),
);
const value = report.data;
return ok(
createEvidence(
{
path: value.artifact.path,
format: "elf",
architecture: "x86_64",
sha256: value.artifact.sha256,
},
this.provider.identity,
{
operation: OPERATION,
parameters: input.data,
result: value,
rawResult: value,
confidence: "observed",
limitations: value.limitations,
locations: [{ kind: "artifact-path", path: value.artifact.path }],
},
),
);
}
}
+2
View File
@@ -1,3 +1,4 @@
import { registerBinaryDiagnosticsCommands } from "./cli/binaryDiagnosticsCommands.js";
import { Cli } from "incur";
import { createLogger, parseLogLevel } from "./logger.js";
@@ -59,6 +60,7 @@ export const createCli = (
registerManagedCommands(cli, logger);
registerAndroidCommands(cli, logger, environment);
registerFirmwareCommands(cli, logger, environment);
registerBinaryDiagnosticsCommands(cli, logger, environment);
registerEvidenceCommands(cli, logger);
registerProcessCommands(cli, logger, environment);
registerBrowserCommands(cli, logger);
+35
View File
@@ -0,0 +1,35 @@
import { z } from "incur";
import { resolve } from "node:path";
import { createBinaryLayoutService } from "../composition/binaryDiagnostics.js";
import { CLI_COMMANDS } from "../cliCommandNames.js";
import { logCliCommand } from "../cliLogging.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { Logger } from "../logger.js";
import type { CliInstance } from "./types.js";
import { withCommandCancellation } from "./commandCancellation.js";
/** Inspect operator-selected files through the same shared workflow as MCP. */
export const registerBinaryDiagnosticsCommands = (
cli: CliInstance,
logger: Logger,
environment: Readonly<NodeJS.ProcessEnv>,
): void => {
const service = createBinaryLayoutService(environment);
cli.command(CLI_COMMANDS.inspectBinaryLayout, {
description:
"Inspect binary file/linked layout without executing the selected object",
args: z.object({
path: z.string().describe("Local ELF binary or relocatable object"),
}),
run: ({ args }) =>
withCommandCancellation((signal) =>
logCliCommand(logger, CLI_COMMANDS.inspectBinaryLayout, async () => {
const result = await service.inspect(
{ path: resolve(args.path) },
{ signal },
);
return result.ok ? result.value : projectAnalysisError(result.error);
}),
),
});
};
+1
View File
@@ -38,6 +38,7 @@ export const CLI_COMMANDS = Object.freeze({
resolveNativeCallTargets: "resolve-native-call-targets",
inspectAssetCatalog: "inspect-asset-catalog",
inspectManagedArtifact: "inspect-managed-artifact",
inspectBinaryLayout: "inspect-binary-layout",
inspectFirmwareRegions: "inspect-firmware-regions",
extractFirmware: "extract-firmware",
inspectAndroidPackage: "inspect-android-package",
+8
View File
@@ -0,0 +1,8 @@
import { BinaryLayoutService } from "../application/binaryDiagnostics/BinaryLayoutService.js";
import { PwntoolsLayoutProvider } from "../native/pwntools/PwntoolsLayoutProvider.js";
/** Compose the offline adapter without import-time I/O or dependency acquisition. */
export const createBinaryLayoutService = (
environment: Readonly<NodeJS.ProcessEnv> = process.env,
): BinaryLayoutService =>
new BinaryLayoutService(new PwntoolsLayoutProvider(environment));
+2
View File
@@ -1,3 +1,4 @@
import { BINARY_DIAGNOSTICS_TOOL_CONTRACTS } from "./native/binaryDiagnosticsToolContracts.js";
import { describe, expect, it } from "vitest";
import { z } from "zod";
@@ -69,6 +70,7 @@ describe("tool contract surface", () => {
...OFFICIAL_TOOL_CONTRACTS,
...ENHANCED_TOOL_CONTRACTS,
...NATIVE_TOOL_CONTRACTS,
...BINARY_DIAGNOSTICS_TOOL_CONTRACTS,
...ARTIFACT_TOOL_CONTRACTS,
...ANDROID_TOOL_CONTRACTS,
...FIRMWARE_TOOL_CONTRACTS,
+2
View File
@@ -9,6 +9,7 @@ const categorySchema = z.enum([
"truncated",
"cancelled",
"timeout",
"resource_constraint",
"unavailable",
"execution_failure",
]);
@@ -38,6 +39,7 @@ export const analysisErrorProjectionSchema = z.discriminatedUnion("code", [
generic("capability_unavailable"),
generic("provider_unavailable"),
generic("provider_timeout"),
generic("resource_constraint"),
generic("cancelled"),
z
.object({
@@ -0,0 +1,26 @@
import {
inspectBinaryLayoutInputSchema,
binaryLayoutSchema,
} from "../../domain/native/binaryLayout.js";
import type { ToolContract } from "../toolContractTypes.js";
import { toolContractMetadata } from "../toolEffects.js";
import { evidenceResultOf } from "../toolOutputSchemas.js";
/** Offline binary diagnostics are independent of macOS process/UI providers. */
export const BINARY_DIAGNOSTICS_TOOL_CONTRACTS = [
{
name: "inspect_binary_layout",
...toolContractMetadata("inspect_binary_layout"),
kind: "native-provider",
description:
"Inspect an explicit local binary without launching it as a host process. Returns file-backed section/segment ranges, linked addresses, original symbol/REL/RELA table identities, complete encoded RELR tables and upstream-derived offsets with unknown per-word locations/addends, raw name bytes, dependency/interpreter names and static mitigation inferences inline with artifact SHA-256 Evidence. Initial profile: ELF64 x86-64 little-endian EXEC/DYN/REL on Linux x64 via caller-supplied Python with unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. Runtime addresses/library paths, overall relocation inventory completeness and complete derived GOT/PLT coverage remain unknown. Zero executable entry values mean absence; relocatable entries are not applicable. Upstream can emulate PLT instructions with Unicorn for derived static maps. Uses bounded complete output and owned temporary files/processes; malformed, unsupported or oversized objects return no partial success.",
inputSchema: inspectBinaryLayoutInputSchema,
outputSchema: evidenceResultOf(binaryLayoutSchema),
examples: [
{
title: "Inspect ELF linked and file layout",
input: { path: "/artifacts/application.elf" },
},
],
},
] as const satisfies readonly ToolContract[];
+2
View File
@@ -1,3 +1,4 @@
import { BINARY_DIAGNOSTICS_TOOL_CONTRACTS } from "./native/binaryDiagnosticsToolContracts.js";
import { describe, expect, it } from "vitest";
import { ENHANCED_TOOL_CONTRACTS } from "./enhancedToolContracts.js";
@@ -23,6 +24,7 @@ const GROUPS = {
official: OFFICIAL_TOOL_CONTRACTS,
enhanced: ENHANCED_TOOL_CONTRACTS,
native: NATIVE_TOOL_CONTRACTS,
binary_diagnostics: BINARY_DIAGNOSTICS_TOOL_CONTRACTS,
artifact: ARTIFACT_TOOL_CONTRACTS,
managed: MANAGED_TOOL_CONTRACTS,
android: ANDROID_TOOL_CONTRACTS,
+2
View File
@@ -1,3 +1,4 @@
import { BINARY_DIAGNOSTICS_TOOL_CONTRACTS } from "./native/binaryDiagnosticsToolContracts.js";
import { NATIVE_TOOL_CONTRACTS } from "./native/nativeToolContracts.js";
import { ARTIFACT_TOOL_CONTRACTS } from "./artifactToolContracts.js";
import { MANAGED_TOOL_CONTRACTS } from "./managed/managedToolContracts.js";
@@ -23,6 +24,7 @@ export const TOOL_CONTRACTS = [
...OFFICIAL_TOOL_CONTRACTS,
...ENHANCED_TOOL_CONTRACTS,
...NATIVE_TOOL_CONTRACTS,
...BINARY_DIAGNOSTICS_TOOL_CONTRACTS,
...ARTIFACT_TOOL_CONTRACTS,
...MANAGED_TOOL_CONTRACTS,
...ANDROID_TOOL_CONTRACTS,
+5
View File
@@ -65,6 +65,11 @@ export const TOOL_EFFECTS: Readonly<Record<string, ToolEffects>> = {
launchesProcess: true,
accessesNetwork: true,
}),
inspect_binary_layout: effects({
mutatesSession: true,
writesFilesystem: true,
launchesProcess: true,
}),
inspect_firmware_regions: effects({
mutatesSession: true,
launchesProcess: true,
+21
View File
@@ -8,6 +8,7 @@ const ANALYSIS_ERROR_TAGS = [
"AnalysisCapabilityUnavailableError",
"AnalysisCancelledError",
"AnalysisTimeoutError",
"AnalysisResourceConstraintError",
"ProviderSelectionError",
"ProviderAdapterError",
"BrowserObservationError",
@@ -30,8 +31,28 @@ const ANALYSIS_ERROR_TAGS = [
/** Stable tag for an expected analysis failure. */
export type AnalysisErrorTag = (typeof ANALYSIS_ERROR_TAGS)[number];
/** Retained command output associated with a typed failure; truncation stays explicit. */
export interface AnalysisCapturedOutput {
readonly stdout: string;
readonly stderr: string;
readonly truncated: boolean;
}
/** Optional provider-neutral context that must survive typed error projection. */
export interface AnalysisErrorOptions extends ErrorOptions {
readonly capturedOutput?: AnalysisCapturedOutput;
}
/** Base class for expected analysis, provider, and session failures. */
export abstract class AnalysisError extends Error {
readonly capturedOutput: AnalysisCapturedOutput | undefined;
constructor(message: string, options?: AnalysisErrorOptions) {
super(message, options);
this.capturedOutput =
options?.capturedOutput === undefined
? undefined
: { ...options.capturedOutput };
}
abstract readonly _tag: AnalysisErrorTag;
readonly userMessage: string | undefined = undefined;
readonly userCategory: "cancelled" | undefined = undefined;
+29 -6
View File
@@ -1,5 +1,23 @@
import type { JsonValue } from "./jsonValue.js";
import { AnalysisError } from "./analysisErrorBase.js";
import {
AnalysisError,
type AnalysisErrorOptions,
} from "./analysisErrorBase.js";
/** Provider reported a resource failure, distinct from malformed input or unsupported coverage. */
export class AnalysisResourceConstraintError extends AnalysisError {
readonly _tag = "AnalysisResourceConstraintError";
constructor(
readonly operation: string,
readonly resource: "memory" | "cpu" | "file-size",
readonly reason: string,
readonly reportedLimits: Readonly<Record<string, JsonValue>> | null,
options?: AnalysisErrorOptions,
) {
super(`Resource constraint during ${operation}: ${reason}`, options);
}
}
/** Provider-neutral invalid analysis input or output at an application boundary. */
export class AnalysisProtocolError extends AnalysisError {
@@ -12,7 +30,7 @@ export class AnalysisInputError extends AnalysisError {
constructor(
readonly operation: string,
options?: ErrorOptions,
options?: AnalysisErrorOptions,
readonly issues: readonly AnalysisInputIssue[] = [],
) {
super(`Invalid analysis input for ${operation}`, options);
@@ -43,7 +61,7 @@ export class AnalysisOutputError extends AnalysisError {
constructor(
readonly operation: string,
readonly reason: string,
options?: ErrorOptions,
options?: AnalysisErrorOptions,
) {
super(`Invalid analysis output for ${operation}: ${reason}`, options);
}
@@ -58,7 +76,7 @@ export class AnalysisCapabilityUnavailableError extends AnalysisError {
readonly providerId: string,
readonly operation: string,
readonly reason: string,
options?: ErrorOptions & { readonly userMessage?: string },
options?: AnalysisErrorOptions & { readonly userMessage?: string },
) {
super(
`Provider ${providerId} cannot execute ${operation}: ${reason}`,
@@ -72,8 +90,11 @@ export class AnalysisCapabilityUnavailableError extends AnalysisError {
export class AnalysisCancelledError extends AnalysisError {
readonly _tag = "AnalysisCancelledError";
constructor(readonly operation: string) {
super(`Analysis operation was cancelled: ${operation}`);
constructor(
readonly operation: string,
options?: AnalysisErrorOptions,
) {
super(`Analysis operation was cancelled: ${operation}`, options);
}
}
@@ -84,9 +105,11 @@ export class AnalysisTimeoutError extends AnalysisError {
constructor(
readonly operation: string,
readonly timeoutMs: number,
options?: AnalysisErrorOptions,
) {
super(
`Analysis operation timed out after ${String(timeoutMs)}ms: ${operation}`,
options,
);
}
}
+10
View File
@@ -2,6 +2,7 @@ import {
AnalysisAccessDeniedError,
AnalysisArtifactChangedError,
AnalysisInputError,
AnalysisResourceConstraintError,
} from "./analysisErrorCore.js";
import { ArtifactOperationError } from "./artifactOperationError.js";
import {
@@ -32,6 +33,12 @@ import { type AnalysisErrorProjection } from "./analysisErrorProjection.js";
export const analysisErrorRemediationAction = (
error: AnalysisError,
): string => {
if (error instanceof AnalysisResourceConstraintError)
return error.resource === "cpu"
? "Review the reported worker CPU limits and observed signal. Retry with sufficient CPU time or a smaller artifact; REA retains tighter inherited limits."
: error.resource === "file-size"
? "Review the reported worker file-size limits and write failure. Retry with a sufficient file-size allowance for the evidence reply; REA retains tighter inherited limits."
: "Review the reported worker memory limits and available host memory. Retry with sufficient memory or a smaller artifact; REA retains tighter inherited limits.";
if (error instanceof HopperTimeoutError)
return error.providerState === "busy"
? "Check binary_session.analysis_activity, wait for the active Hopper request to finish, then retry."
@@ -123,12 +130,14 @@ const STATIC_ERROR_CATEGORIES: Readonly<
AnalysisCancelledError: "cancelled",
HopperCancelledError: "cancelled",
AnalysisTimeoutError: "timeout",
AnalysisResourceConstraintError: "resource_constraint",
HopperTimeoutError: "timeout",
NoBinaryOpenError: "unavailable",
BinaryTargetError: "unavailable",
};
export const analysisErrorUserMessage = (error: AnalysisError): string => {
if (error instanceof AnalysisResourceConstraintError) return error.reason;
if (error instanceof AnalysisAccessDeniedError)
return "Host filesystem permissions denied read access to the selected path.";
if (error instanceof AnalysisArtifactChangedError)
@@ -263,6 +272,7 @@ const KNOWN_ERROR_TAGS = {
AnalysisCapabilityUnavailableError: true,
AnalysisCancelledError: true,
AnalysisTimeoutError: true,
AnalysisResourceConstraintError: true,
ProviderSelectionError: true,
ProviderAdapterError: true,
BrowserObservationError: true,
@@ -3,6 +3,9 @@ import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
AnalysisCancelledError,
AnalysisTimeoutError,
AnalysisResourceConstraintError,
} from "./analysisErrorCore.js";
import { ArtifactOperationError } from "./artifactOperationError.js";
import { BinaryTargetError } from "./configurationErrors.js";
@@ -16,6 +19,87 @@ import {
import { ProviderAdapterError } from "./providerAdapterError.js";
import { UnknownRegistryError } from "./unknownRegistryError.js";
import { projectAnalysisError } from "./analysisErrorProjection.js";
import { ProviderSelectionError } from "./providerSelectionError.js";
import { analysisErrorProjectionSchema } from "../contracts/errorSchemas.js";
const retainedOutput = {
stdout: "selected output\u0000",
stderr: "upstream warning",
truncated: true,
};
it.each([
["memory", null],
["memory", { address_space_bytes: 67108864 }],
["cpu", null],
["cpu", { cpu_seconds: 1 }],
["file-size", null],
["file-size", { file_size_bytes: 1024 }],
] as const)(
"projects reported %s constraints with observed or unknown limits: %j",
(resource, limits) => {
const projected = projectAnalysisError(
new AnalysisResourceConstraintError(
"inspect_binary_layout",
resource,
resource === "memory"
? "Memory allocation failed; exact cause unknown."
: "Observed SIGXCPU; exact signal cause unknown.",
limits,
{ capturedOutput: retainedOutput },
),
);
expect(projected).toMatchObject({
code: "resource_constraint",
category: "resource_constraint",
retryable: false,
details: {
resource,
reported_limits: limits,
captured_output: retainedOutput,
},
});
expect(projected.remediation.action).toContain(
resource === "cpu"
? "CPU"
: resource === "file-size"
? "file-size"
: "memory",
);
expect(projected.remediation.action).not.toContain("doctor");
expect(analysisErrorProjectionSchema.safeParse(projected).success).toBe(
true,
);
},
);
it.each([
new AnalysisInputError("inspect", { capturedOutput: retainedOutput }),
new AnalysisOutputError("inspect", "invalid reply", {
capturedOutput: retainedOutput,
}),
new AnalysisCapabilityUnavailableError(
"provider",
"inspect",
"unsupported profile",
{ capturedOutput: retainedOutput },
),
new AnalysisCancelledError("inspect", { capturedOutput: retainedOutput }),
new AnalysisTimeoutError("inspect", 30, { capturedOutput: retainedOutput }),
new ProviderSelectionError({
operation: "inspect",
reason: "provider_unavailable",
requestedProviderId: "provider",
candidateIds: [],
capturedOutput: retainedOutput,
}),
])(
"retains captured output without changing the typed primary failure: $._tag",
(failure) => {
expect(projectAnalysisError(failure).details?.captured_output).toEqual(
retainedOutput,
);
},
);
describe("analysis error projection: provider failures", () => {
it("projects the primary browser failure alongside incomplete cleanup", () => {
+16 -1
View File
@@ -14,6 +14,7 @@ import {
AnalysisInputError,
AnalysisOutputError,
AnalysisTimeoutError,
AnalysisResourceConstraintError,
} from "./analysisErrorCore.js";
import { ArtifactOperationError } from "./artifactOperationError.js";
import { BinaryTargetError } from "./configurationErrors.js";
@@ -39,7 +40,11 @@ export const projectAnalysisError = (
): AnalysisErrorProjection => {
assertKnownAnalysisErrorTag(error._tag);
const code = errorCode(error);
const details = errorDetails(error);
const primaryDetails = errorDetails(error);
const details =
error.capturedOutput === undefined
? primaryDetails
: { ...primaryDetails, captured_output: { ...error.capturedOutput } };
return {
code,
category: analysisErrorCategory(error),
@@ -138,6 +143,7 @@ const STATIC_ERROR_CODES = {
AnalysisCancelledError: "cancelled",
HopperCancelledError: "cancelled",
AnalysisTimeoutError: "provider_timeout",
AnalysisResourceConstraintError: "resource_constraint",
HopperTimeoutError: "provider_timeout",
HopperProcessError: "provider_unavailable",
HopperStartError: "provider_unavailable",
@@ -181,6 +187,13 @@ const errorDetails = (
const requestErrorDetails = (
error: AnalysisError,
): Readonly<Record<string, JsonValue>> | undefined => {
if (error instanceof AnalysisResourceConstraintError)
return {
operation: error.operation,
resource: error.resource,
reason: error.reason,
reported_limits: error.reportedLimits,
};
if (error instanceof AnalysisArtifactChangedError)
return {
operation: error.operation,
@@ -409,6 +422,7 @@ export interface AnalysisErrorProjection extends Readonly<
| "capability_unavailable"
| "provider_unavailable"
| "provider_timeout"
| "resource_constraint"
| "cancelled"
| "artifact_integrity_mismatch"
| "artifact_operation_failed"
@@ -427,6 +441,7 @@ export interface AnalysisErrorProjection extends Readonly<
| "truncated"
| "cancelled"
| "timeout"
| "resource_constraint"
| "unavailable"
| "execution_failure";
readonly message: string;
+362
View File
@@ -0,0 +1,362 @@
import { expect, it } from "vitest";
import { binaryLayoutSchema, type BinaryLayout } from "./binaryLayout.js";
const report = () => ({
artifact: { path: "/fixture.elf", sha256: "a".repeat(64), bytes: 128 },
format: "elf",
architecture: { machine: "EM_X86_64", bits: 64, byte_order: "little" },
image_type: "ET_EXEC",
entry_point: {
reported_value: "0x20000000000001",
meaning: "linked-virtual-address",
execution_status: "unknown",
},
runtime_load_base: null,
sections: [
{
index: 0,
name: {
display: "opaque",
bytes_base64: "b3BhcXVl",
location: { offset: "0x40", bytes: "0x7" },
unknown_reason: null,
},
name_offset: "0x0",
type: "SHT_NOBITS",
header_location: { offset: "0x0", bytes: "0x40" },
address: "0x20000000000001",
offset: "0xffffffffffffffff",
size: "0x2000",
alignment: "0x1",
flags: "0x3",
link: 0,
info: 0,
entry_size: "0x0",
file_backing: "none",
},
],
segments: [],
symbols: [],
relocations: [],
packed_relative_relocations: [],
relocation_inventory_completeness: "unknown",
linkage: {
needed_libraries: [],
interpreters: [],
got: [],
plt: [],
convenience_maps_completeness: "unknown",
runtime_library_paths: null,
},
mitigations: {
evidence_kind: "inferred",
position_independent: false,
nx_indicator: null,
executable_stack_indicator: false,
stack_canary_indicator: false,
relro: null,
},
diagnostics: { stdout: "", stderr: "", truncated: false },
limitations: [],
});
it("retains high linked values and NOBITS without inventing file backing", () => {
const value = binaryLayoutSchema.parse(report());
expect(value.entry_point.reported_value).toBe("0x20000000000001");
expect(value.sections[0]?.offset).toBe("0xffffffffffffffff");
expect(value.runtime_load_base).toBeNull();
});
it.each([
"file-backed-outside",
"header-outside",
"wrong-index",
"noncanonical-base64",
"wrong-machine",
"outside-elf64",
])("rejects a malformed source representation: %s", (problem) => {
const value = report();
const section = value.sections[0];
if (section === undefined) throw new Error("section missing");
if (problem === "file-backed-outside") section.file_backing = "file";
if (problem === "header-outside") section.header_location.offset = "0x80";
if (problem === "wrong-index") section.index = 1;
if (problem === "wrong-machine") value.architecture.machine = "EM_AARCH64";
if (problem === "outside-elf64")
value.entry_point.reported_value = "0x10000000000000000";
if (problem === "noncanonical-base64") section.name.bytes_base64 = "AR==";
expect(binaryLayoutSchema.safeParse(value).success).toBe(false);
});
it.each([
["PT_NULL", "0xffffffffffffffff", "none", null, true],
["PT_LOAD", "0x0", "none", { read: true, write: true, execute: false }, true],
[
"PT_LOAD",
"0x1",
"file",
{ read: true, write: false, execute: false },
false,
],
["PT_NULL", "0x1", "file", null, false],
[
"PT_NULL",
"0x1",
"none",
{ read: false, write: false, execute: false },
false,
],
[
"PT_LOAD",
"0x1",
"none",
{ read: true, write: false, execute: false },
false,
],
])(
"validates actual segment backing/meaning for %s size %s",
(type, fileSize, backing, permissions, valid) => {
const value = {
...report(),
segments: [
{
index: 0,
type,
header_location: { offset: "0x0", bytes: "0x38" },
offset: "0xffffffffffffffff",
file_size: fileSize,
memory_size: fileSize,
virtual_address: "0x0",
physical_address: "0x0",
alignment: "0x0",
flags: "0x0",
file_backing: backing,
permissions,
},
],
};
expect(binaryLayoutSchema.safeParse(value).success).toBe(valid);
},
);
it.each([
["ET_EXEC", "0x0", "absent", true],
["ET_DYN", "0x0", "linked-virtual-address", false],
["ET_REL", "0x0", "not-applicable", true],
["ET_EXEC", "0x1", "absent", false],
])(
"validates actual entry meaning: %s %s %s",
(imageType, entry, meaning, valid) => {
expect(
binaryLayoutSchema.safeParse({
...report(),
image_type: imageType,
entry_point: {
reported_value: entry,
meaning,
execution_status: "unknown",
},
}).success,
).toBe(valid);
},
);
it.each([
"valid",
"byte-length",
"section-identity",
"decoded-order",
"noncanonical-bytes",
])(
"retains original packed bytes and derived offset provenance: %s",
(problem) => {
const value = report();
const section = value.sections[0];
if (section === undefined) throw new Error("section missing");
section.type = "SHT_RELR";
section.file_backing = "file";
section.offset = "0x60";
section.size = "0x8";
const table = {
section_index: 0,
location: { offset: "0x60", bytes: "0x8" },
encoded_bytes_base64: "AAAAAAAAAAA=",
entries: [{ decoded_index: 0, reported_offset: "0x20000000000001" }],
offset_meaning: "linked-virtual-address",
evidence_kind: "derived",
entry_source_locations: null,
addends: null,
};
if (problem === "byte-length") table.location.bytes = "0x7";
if (problem === "section-identity") table.section_index = 1;
if (problem === "decoded-order")
table.entries[0] = { decoded_index: 1, reported_offset: "0x0" };
if (problem === "noncanonical-bytes") table.encoded_bytes_base64 = "AB==";
expect(
binaryLayoutSchema.safeParse({
...value,
packed_relative_relocations: [table],
}).success,
).toBe(problem === "valid");
},
);
it.each([
"valid",
"zero-symbol",
"zero-without-table",
"missing-table",
"wrong-table-kind",
"missing-symbol",
"symbol-outside-table",
"positive-without-table",
"owner-link-mismatch",
"wrong-meaning",
"valid-section-target",
"missing-target",
"wrong-target-owner",
"wrong-target-offset",
"undefined-target",
"inactive-target",
"false-resolved-zero-target",
"wrong-unknown-target",
"missing-string-table",
"wrong-string-table-kind",
])("validates relocation symbol reference semantics: %s", (problem) => {
const value = binaryLayoutSchema.parse(report());
value.artifact.bytes = 512;
const base = value.sections[0];
if (base === undefined) throw new Error("section missing");
value.sections.push(
{
...base,
index: 1,
type: "SHT_SYMTAB",
link: 3,
header_location: { offset: "0x40", bytes: "0x40" },
offset: "0x100",
size: "0x30",
entry_size: "0x18",
file_backing: "file",
},
{
...base,
index: 2,
type: "SHT_RELA",
header_location: { offset: "0x80", bytes: "0x40" },
offset: "0x130",
size: "0x18",
entry_size: "0x18",
link: 1,
file_backing: "file",
},
{
...base,
index: 3,
type: "SHT_STRTAB",
offset: "0x40",
size: "0x7",
file_backing: "file",
},
);
value.symbols.push({
table_index: 1,
entry_index: 1,
name: base.name,
name_offset: "0x0",
location: { offset: "0x118", bytes: "0x18" },
value: "0x0",
value_meaning: "undefined",
size: "0x0",
binding: "STB_GLOBAL",
type: "STT_NOTYPE",
visibility: "STV_DEFAULT",
section_index: "SHN_UNDEF",
});
const relocation: BinaryLayout["relocations"][number] = {
section_index: 2,
entry_index: 0,
reported_offset: "0x0",
target: { kind: "linked-virtual-address", address: "0x0" },
location: { offset: "0x130", bytes: "0x18" },
type: 1,
symbol_table_index: 1,
symbol_index: 1,
symbol_reference_meaning: "symbol-table-entry",
addend: "0",
};
const table = value.sections[1];
const owner = value.sections[2];
if (table === undefined || owner === undefined)
throw new Error("table or relocation owner missing");
if (problem.startsWith("zero")) {
relocation.symbol_index = 0;
relocation.symbol_reference_meaning = "zero-symbol-value";
}
if (problem === "zero-without-table" || problem === "positive-without-table")
owner.link = relocation.symbol_table_index = 0;
if (problem === "missing-table")
owner.link = relocation.symbol_table_index = 99;
if (problem === "wrong-table-kind") table.type = "SHT_PROGBITS";
if (problem === "missing-string-table") table.link = 99;
if (problem === "wrong-string-table-kind") table.link = 0;
if (problem === "missing-symbol") value.symbols = [];
if (problem === "symbol-outside-table") table.size = "0x18";
if (problem === "owner-link-mismatch") owner.link = 0;
if (problem === "wrong-meaning")
relocation.symbol_reference_meaning = "zero-symbol-value";
if (problem.includes("target")) {
value.image_type = "ET_REL";
value.entry_point.meaning = "not-applicable";
owner.info = 1;
relocation.target = {
kind: "section-offset",
section_index:
problem === "missing-target"
? 99
: problem === "wrong-target-owner"
? 2
: 1,
offset: problem === "wrong-target-offset" ? "0x1" : "0x0",
};
if (problem === "missing-target") owner.info = 99;
if (problem === "false-resolved-zero-target") {
owner.info = 0;
relocation.target.section_index = 0;
}
if (
["undefined-target", "inactive-target", "wrong-unknown-target"].includes(
problem,
)
) {
if (problem === "inactive-target")
value.sections.push({ ...base, index: 4, type: "SHT_NULL" });
owner.info =
problem === "undefined-target"
? 0
: problem === "inactive-target"
? 4
: 1;
relocation.target = {
kind: "unknown-section",
reported_section_index: owner.info,
offset: "0x0",
unknown_reason:
problem === "undefined-target"
? "undefined-section-reference"
: "inactive-section-header",
};
}
}
value.relocations.push(relocation);
expect(binaryLayoutSchema.safeParse(value).success).toBe(
[
"valid",
"zero-symbol",
"zero-without-table",
"valid-section-target",
"undefined-target",
"inactive-target",
].includes(problem),
);
});
+443
View File
@@ -0,0 +1,443 @@
import { z } from "zod";
const unsignedHex = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/);
const index = z.number().int().nonnegative();
const scalar = z.union([z.string(), z.number().int()]);
const range = z.strictObject({ offset: unsignedHex, bytes: unsignedHex });
const canonicalBase64 = z
.string()
.regex(
/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/][AQgw]==|[A-Za-z0-9+/]{2}[AEIMQUYcgkosw048]=)?$/,
);
const name = z.strictObject({
display: z.string(),
bytes_base64: canonicalBase64.nullable(),
location: range.nullable(),
unknown_reason: z.string().nullable(),
});
/** Explicit local object selection, independent of an active disassembler target. */
export const inspectBinaryLayoutInputSchema = z.strictObject({
path: z
.string()
.min(1)
.describe("Absolute filesystem path to the selected binary"),
});
/** File and linked-image evidence; numeric ELF values never pass through unsafe JSON numbers. */
const binaryLayoutObjectSchema = z.strictObject({
artifact: z.strictObject({
path: z.string().min(1),
sha256: z.string().regex(/^[a-f0-9]{64}$/),
bytes: index,
}),
format: z.literal("elf"),
architecture: z.strictObject({
machine: z.literal("EM_X86_64"),
bits: z.literal(64),
byte_order: z.literal("little"),
}),
image_type: z.enum(["ET_EXEC", "ET_DYN", "ET_REL"]),
entry_point: z.strictObject({
reported_value: unsignedHex,
meaning: z.enum(["linked-virtual-address", "not-applicable", "absent"]),
execution_status: z.literal("unknown"),
}),
runtime_load_base: z.null(),
sections: z.array(
z.strictObject({
index,
name,
name_offset: unsignedHex,
type: scalar,
header_location: range,
address: unsignedHex,
offset: unsignedHex,
size: unsignedHex,
alignment: unsignedHex,
flags: unsignedHex,
link: index,
info: index,
entry_size: unsignedHex,
file_backing: z.enum(["file", "none"]),
}),
),
segments: z.array(
z.strictObject({
index,
type: scalar,
header_location: range,
offset: unsignedHex,
file_size: unsignedHex,
memory_size: unsignedHex,
virtual_address: unsignedHex,
physical_address: unsignedHex,
alignment: unsignedHex,
flags: unsignedHex,
file_backing: z.enum(["file", "none"]),
permissions: z
.strictObject({
read: z.boolean(),
write: z.boolean(),
execute: z.boolean(),
})
.nullable(),
}),
),
symbols: z.array(
z.strictObject({
table_index: index,
entry_index: index,
name,
name_offset: unsignedHex,
location: range,
value: unsignedHex,
value_meaning: z.enum([
"undefined",
"alignment",
"absolute-value",
"no-address",
"unknown-section-index",
"section-offset",
"tls-offset",
"linked-virtual-address",
]),
size: unsignedHex,
binding: scalar,
type: scalar,
visibility: scalar,
section_index: scalar,
}),
),
relocations: z.array(
z.strictObject({
section_index: index,
entry_index: index,
reported_offset: unsignedHex,
target: z.discriminatedUnion("kind", [
z.strictObject({
kind: z.literal("section-offset"),
section_index: index,
offset: unsignedHex,
}),
z.strictObject({
kind: z.literal("linked-virtual-address"),
address: unsignedHex,
}),
z.strictObject({
kind: z.literal("unknown-section"),
reported_section_index: index,
offset: unsignedHex,
unknown_reason: z.enum([
"undefined-section-reference",
"inactive-section-header",
]),
}),
]),
location: range,
type: index,
symbol_table_index: index,
symbol_index: index,
symbol_reference_meaning: z.enum([
"zero-symbol-value",
"symbol-table-entry",
]),
addend: z
.string()
.regex(/^-?(?:0|[1-9][0-9]{0,18})$/)
.nullable(),
}),
),
packed_relative_relocations: z.array(
z.strictObject({
section_index: index,
location: range,
encoded_bytes_base64: canonicalBase64,
entries: z.array(
z.strictObject({ decoded_index: index, reported_offset: unsignedHex }),
),
offset_meaning: z.enum(["linked-virtual-address", "unknown"]),
evidence_kind: z.literal("derived"),
entry_source_locations: z.null(),
addends: z.null(),
}),
),
relocation_inventory_completeness: z.literal("unknown"),
linkage: z.strictObject({
needed_libraries: z.array(name),
interpreters: z.array(name),
got: z.array(
z.strictObject({ display_name: z.string(), address: unsignedHex }),
),
plt: z.array(
z.strictObject({ display_name: z.string(), address: unsignedHex }),
),
convenience_maps_completeness: z.literal("unknown"),
runtime_library_paths: z.null(),
}),
mitigations: z.strictObject({
evidence_kind: z.literal("inferred"),
position_independent: z.boolean(),
nx_indicator: z.boolean().nullable(),
executable_stack_indicator: z.boolean(),
stack_canary_indicator: z.boolean(),
relro: z.enum(["Partial", "Full"]).nullable(),
}),
diagnostics: z.strictObject({
stdout: z.string(),
stderr: z.string(),
truncated: z.boolean(),
}),
limitations: z.array(z.string()),
});
/** Decoder payload excludes the artifact identity observed by the owning process boundary. */
export const binaryLayoutPayloadSchema = binaryLayoutObjectSchema.omit({
artifact: true,
diagnostics: true,
});
/** Check source ranges again at the public boundary, independently of upstream parsing. */
export const binaryLayoutSchema = binaryLayoutObjectSchema.superRefine(
(value, context) => {
const size = BigInt(value.artifact.bytes);
const check = (
offset: string,
length: string,
path: (string | number)[],
): void => {
const start = BigInt(offset);
const bytes = BigInt(length);
if (start > size || bytes > size - start)
context.addIssue({
code: "custom",
path,
message:
"Reported file-backed range lies outside the selected artifact.",
});
};
const checkLocation = (
where: z.infer<typeof range>,
path: (string | number)[],
): void => check(where.offset, where.bytes, path);
const checkName = (
reported: z.infer<typeof name>,
path: (string | number)[],
): void => {
if (reported.location !== null)
checkLocation(reported.location, [...path, "location"]);
};
const expectedEntryMeaning =
value.image_type === "ET_REL"
? "not-applicable"
: value.entry_point.reported_value === "0x0"
? "absent"
: "linked-virtual-address";
if (value.entry_point.meaning !== expectedEntryMeaning)
context.addIssue({
code: "custom",
path: ["entry_point", "meaning"],
message:
"Entry meaning must distinguish zero/unused fields from an actual linked address.",
});
for (const [index, section] of value.sections.entries()) {
if (
(section.type === "SHT_SYMTAB" || section.type === "SHT_DYNSYM") &&
value.sections[section.link]?.type !== "SHT_STRTAB"
)
context.addIssue({
code: "custom",
path: ["sections", index, "link"],
message:
"Symbol tables must link to an existing string-table section.",
});
if (section.index !== index)
context.addIssue({
code: "custom",
path: ["sections", index, "index"],
message:
"Complete section order must preserve original producer indices.",
});
checkLocation(section.header_location, [
"sections",
index,
"header_location",
]);
checkName(section.name, ["sections", index, "name"]);
if (section.file_backing === "file")
check(section.offset, section.size, ["sections", index]);
}
for (const [index, segment] of value.segments.entries()) {
if (segment.index !== index)
context.addIssue({
code: "custom",
path: ["segments", index, "index"],
message:
"Complete segment order must preserve original producer indices.",
});
checkLocation(segment.header_location, [
"segments",
index,
"header_location",
]);
const hasFileBytes =
segment.type !== "PT_NULL" && BigInt(segment.file_size) !== 0n;
if (segment.file_backing !== (hasFileBytes ? "file" : "none"))
context.addIssue({
code: "custom",
path: ["segments", index, "file_backing"],
message:
"File backing must follow the reported segment type and file size.",
});
if ((segment.type === "PT_NULL") !== (segment.permissions === null))
context.addIssue({
code: "custom",
path: ["segments", index, "permissions"],
message:
"Unused PT_NULL flags have no permission meaning; other segment flags retain their interpretation.",
});
if (hasFileBytes)
check(segment.offset, segment.file_size, ["segments", index]);
}
for (const [index, symbol] of value.symbols.entries()) {
checkLocation(symbol.location, ["symbols", index, "location"]);
checkName(symbol.name, ["symbols", index, "name"]);
}
for (const [index, relocation] of value.relocations.entries())
checkLocation(relocation.location, ["relocations", index, "location"]);
validateRelocationReferences(value, context);
validatePackedRelativeTables(value, context, checkLocation);
for (const facet of ["needed_libraries", "interpreters"] as const)
for (const [index, reported] of value.linkage[facet].entries())
checkName(reported, ["linkage", facet, index]);
},
);
export type InspectBinaryLayoutInput = z.infer<
typeof inspectBinaryLayoutInputSchema
>;
export type BinaryLayout = z.infer<typeof binaryLayoutSchema>;
const validateRelocationReferences = (
value: z.output<typeof binaryLayoutObjectSchema>,
context: z.RefinementCtx,
): void => {
const symbols = new Set(
value.symbols.map(
(symbol) => `${symbol.table_index}:${symbol.entry_index}`,
),
);
for (const [index, relocation] of value.relocations.entries()) {
const owner = value.sections[relocation.section_index];
const target = relocation.target;
const targetMatches =
value.image_type === "ET_REL"
? target.kind === "section-offset"
? target.section_index !== 0 &&
target.section_index === owner?.info &&
value.sections[target.section_index] !== undefined &&
value.sections[target.section_index]?.type !== "SHT_NULL" &&
target.offset === relocation.reported_offset
: target.kind === "unknown-section" &&
target.reported_section_index === owner?.info &&
target.offset === relocation.reported_offset &&
(target.reported_section_index === 0
? target.unknown_reason === "undefined-section-reference"
: value.sections[target.reported_section_index]?.type ===
"SHT_NULL" &&
target.unknown_reason === "inactive-section-header")
: target.kind === "linked-virtual-address" &&
target.address === relocation.reported_offset;
if (!targetMatches)
context.addIssue({
code: "custom",
path: ["relocations", index, "target"],
message:
"Relocation target must preserve its reported offset and owner reference; SHN_UNDEF and inactive headers remain unknown rather than resolved sections.",
});
const table = value.sections[relocation.symbol_table_index];
const tableExists =
relocation.symbol_table_index !== 0 &&
(table?.type === "SHT_SYMTAB" || table?.type === "SHT_DYNSYM");
const positiveReferenceExists =
tableExists &&
BigInt(table.entry_size) >= 24n &&
BigInt(relocation.symbol_index) <
BigInt(table.size) / BigInt(table.entry_size) &&
symbols.has(
`${relocation.symbol_table_index}:${relocation.symbol_index}`,
);
if (
(owner?.type !== "SHT_REL" && owner?.type !== "SHT_RELA") ||
owner.link !== relocation.symbol_table_index ||
(relocation.symbol_table_index !== 0 && !tableExists) ||
(relocation.symbol_index !== 0 && !positiveReferenceExists) ||
relocation.symbol_reference_meaning !==
(relocation.symbol_index === 0
? "zero-symbol-value"
: "symbol-table-entry")
)
context.addIssue({
code: "custom",
path: ["relocations", index],
message:
"Relocation must preserve its REL/RELA owner and resolve positive symbol indices through the linked symbol table; index zero uses a zero symbol value.",
});
}
};
const validatePackedRelativeTables = (
value: z.output<typeof binaryLayoutObjectSchema>,
context: z.RefinementCtx,
checkLocation: (
location: z.output<typeof range>,
path: (string | number)[],
) => void,
): void => {
for (const [
tableIndex,
table,
] of value.packed_relative_relocations.entries()) {
checkLocation(table.location, [
"packed_relative_relocations",
tableIndex,
"location",
]);
const encodedBytes =
(table.encoded_bytes_base64.length * 3) / 4 -
(table.encoded_bytes_base64.endsWith("==")
? 2
: table.encoded_bytes_base64.endsWith("=")
? 1
: 0);
if (
!Number.isInteger(encodedBytes) ||
BigInt(encodedBytes) !== BigInt(table.location.bytes)
)
context.addIssue({
code: "custom",
path: ["packed_relative_relocations", tableIndex],
message:
"Complete packed table bytes must match its original file range.",
});
const section = value.sections[table.section_index];
if (
section?.type !== "SHT_RELR" ||
section.offset !== table.location.offset ||
section.size !== table.location.bytes
)
context.addIssue({
code: "custom",
path: ["packed_relative_relocations", tableIndex, "section_index"],
message:
"Packed relative evidence must retain its original RELR section identity and range.",
});
for (const [decoded, entry] of table.entries.entries())
if (entry.decoded_index !== decoded)
context.addIssue({
code: "custom",
path: ["packed_relative_relocations", tableIndex, "entries", decoded],
message: "Decoded offsets must preserve upstream iterator order.",
});
}
};
+11 -2
View File
@@ -1,5 +1,8 @@
import type { JsonValue } from "./jsonValue.js";
import { AnalysisError } from "./analysisErrorBase.js";
import {
AnalysisError,
type AnalysisCapturedOutput,
} from "./analysisErrorBase.js";
/** Stable failure classes for target-bound deep-provider selection. */
export type ProviderSelectionFailureReason =
@@ -19,6 +22,7 @@ export interface ProviderSelectionRejection {
/** Complete typed context for one failed deep-provider selection. */
export interface ProviderSelectionErrorOptions {
readonly capturedOutput?: AnalysisCapturedOutput;
readonly operation?: string;
readonly reason: ProviderSelectionFailureReason;
readonly requestedProviderId: string;
@@ -38,7 +42,12 @@ export class ProviderSelectionError extends AnalysisError {
constructor(options: ProviderSelectionErrorOptions) {
const operation = options.operation ?? "open_binary";
super(providerSelectionDiagnostic(options));
super(
providerSelectionDiagnostic(options),
options.capturedOutput === undefined
? undefined
: { capturedOutput: options.capturedOutput },
);
this.operation = operation;
this.reason = options.reason;
this.requestedProviderId = options.requestedProviderId;
+1 -1
View File
@@ -5,5 +5,5 @@ export const PACKAGE_METADATA = {
serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1",
skillVersion: "30",
skillVersion: "31",
} as const;
+245
View File
@@ -0,0 +1,245 @@
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
import {
AnalysisError,
type AnalysisCapturedOutput,
} from "../../domain/analysisErrorBase.js";
import type { ProviderProcessSnapshot } from "../../process/ProviderProcess.js";
import {
AnalysisAccessDeniedError,
AnalysisArtifactChangedError,
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
AnalysisTimeoutError,
AnalysisResourceConstraintError,
} from "../../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { ProviderSelectionError } from "../../domain/providerSelectionError.js";
import { OwnedCommandFailure } from "../../process/OwnedCommand.js";
import type { PwntoolsFailureEvidence } from "./PwntoolsResourceLimits.js";
import {
PWNTOOLS_PROVIDER_IDENTITY,
PWNTOOLS_LIMITS,
PWNTOOLS_MEMORY_FAILURE_EXIT,
PWNTOOLS_FILE_SIZE_FAILURE_EXIT,
} from "./PwntoolsRelease.js";
const OPERATION = "inspect_binary_layout";
/** Keep read, engine, output and owned-lifecycle failure reasons distinct. */
export const pwntoolsLayoutFailure = (
cause: unknown,
phase: string,
path: string,
executablePath = path,
failureEvidence: PwntoolsFailureEvidence = {},
): AnalysisError => {
const limitReport = failureEvidence.limits;
if (cause instanceof AnalysisError) return cause;
if (cause instanceof OwnedCommandFailure) {
const outputOptions =
cause.snapshot === null
? undefined
: { capturedOutput: capturedPwntoolsOutput(cause.snapshot) };
if (cause.cleanupFailure !== null)
return new ProviderCleanupError(
PWNTOOLS_PROVIDER_IDENTITY.id,
cause.resources,
{
reason: cause.cleanupFailure,
...(cause.snapshot === null
? {}
: {
captured_output: { ...capturedPwntoolsOutput(cause.snapshot) },
}),
previous_error: {
failure_kind: cause.reason,
message: cause.message,
exit_code: cause.snapshot?.exitCode ?? null,
signal: cause.snapshot?.signal ?? null,
stdout: cause.snapshot?.stdout.text ?? null,
stderr: cause.snapshot?.stderr.text ?? null,
},
},
{ operation: OPERATION, cause },
);
if (cause.reason === "cancelled")
return new AnalysisCancelledError(OPERATION, outputOptions);
if (cause.reason === "timeout")
return new AnalysisTimeoutError(
OPERATION,
PWNTOOLS_LIMITS.timeoutMs,
outputOptions,
);
if (cause.reason === "output-limit")
return new AnalysisOutputError(OPERATION, cause.message, outputOptions);
if (
cause.reason === "process" &&
(cause.snapshot?.signal === "SIGXFSZ" ||
(failureEvidence.marker?.resource === "file-size" &&
cause.snapshot?.exitCode === PWNTOOLS_FILE_SIZE_FAILURE_EXIT &&
cause.snapshot.signal === null))
)
return new AnalysisResourceConstraintError(
OPERATION,
"file-size",
(cause.snapshot.signal === "SIGXFSZ"
? "Owned Python terminated with SIGXFSZ; the exact signal cause is unknown."
: "The owned Python bridge reported EFBIG while writing a file; the exact write failure cause is unknown.") +
(limitReport?.failure === null || limitReport === undefined
? ""
: ` Effective limit report unavailable: ${limitReport.failure}`),
limitReport?.limits ?? null,
outputOptions,
);
if (cause.reason === "process" && cause.snapshot?.signal === "SIGXCPU")
return new AnalysisResourceConstraintError(
OPERATION,
"cpu",
"Owned Python terminated with SIGXCPU; the exact signal cause is unknown." +
(limitReport?.failure === null || limitReport === undefined
? ""
: ` Effective limit report unavailable: ${limitReport.failure}`),
limitReport?.limits ?? null,
outputOptions,
);
if (
cause.reason === "process" &&
failureEvidence.marker?.resource === "memory" &&
cause.snapshot?.exitCode === PWNTOOLS_MEMORY_FAILURE_EXIT &&
cause.snapshot.signal === null
)
return new AnalysisResourceConstraintError(
OPERATION,
"memory",
"The owned Python bridge reported a memory allocation failure without a structured reply; the exact allocation cause is unknown." +
(limitReport?.failure === null || limitReport === undefined
? ""
: ` Effective limit report unavailable: ${limitReport.failure}`),
limitReport?.limits ?? null,
outputOptions,
);
}
if (
cause instanceof Error &&
"code" in cause &&
"syscall" in cause &&
typeof cause.syscall === "string" &&
cause.syscall.startsWith("spawn ")
)
return pwntoolsUnavailable(
`Selected Python could not launch (${String(cause.code)}): ${executablePath}. Check the configured executable, its interpreter and host execute access.`,
executablePath,
String(cause.code),
);
if (cause instanceof ArtifactReaderFailure) {
if (cause.reason === "integrity")
return new AnalysisArtifactChangedError(OPERATION, path, cause.message, {
cause,
});
if (cause.reason === "cancelled")
return new AnalysisCancelledError(OPERATION);
if (
cause.reason === "limit" ||
cause.reason === "path" ||
cause.reason === "format"
)
return new AnalysisInputError(OPERATION, { cause }, [
{
path: ["path"],
reason: cause.reason === "limit" ? "out_of_range" : "invalid_format",
message: cause.message,
},
]);
}
if (cause instanceof Error && "code" in cause) {
if (phase === "configuration")
return pwntoolsUnavailable(
`Selected Python executable is unavailable (${String(cause.code)}): ${path}. Check the caller-selected path and execute access.`,
path,
String(cause.code),
);
if (
phase === "artifact-read" &&
(cause.code === "EACCES" || cause.code === "EPERM")
)
return new AnalysisAccessDeniedError(OPERATION, path, cause.code, {
cause,
});
if (
phase === "artifact-read" &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR")
)
return new AnalysisInputError(OPERATION, { cause }, [
{
path: ["path"],
reason: "invalid_value",
message: `Selected object could not be read (${String(cause.code)}): ${path}.`,
},
]);
}
return new ProviderAdapterError(PWNTOOLS_PROVIDER_IDENTITY.id, OPERATION, {
cause,
diagnostics: {
phase,
path,
reason: cause instanceof Error ? cause.message : String(cause),
...(failureEvidence.marker === undefined
? {}
: { resource_failure_marker: { ...failureEvidence.marker } }),
...(cause instanceof OwnedCommandFailure
? {
failure_kind: cause.reason,
exit_code: cause.snapshot?.exitCode ?? null,
signal: cause.snapshot?.signal ?? null,
stdout: cause.snapshot?.stdout.text ?? null,
stderr: cause.snapshot?.stderr.text ?? null,
...(cause.snapshot === null
? {}
: {
captured_output: {
...capturedPwntoolsOutput(cause.snapshot),
},
}),
}
: {}),
},
});
};
/** Report caller-selected engine absence with its actual configuration and host constraint. */
export const pwntoolsUnavailable = (
reason: string,
path: string,
systemCode?: string,
capturedOutput?: AnalysisCapturedOutput,
): ProviderSelectionError =>
new ProviderSelectionError({
...(capturedOutput === undefined ? {} : { capturedOutput }),
operation: OPERATION,
reason: "provider_unavailable",
requestedProviderId: PWNTOOLS_PROVIDER_IDENTITY.id,
candidateIds: [PWNTOOLS_PROVIDER_IDENTITY.id],
rejections: [
{
providerId: PWNTOOLS_PROVIDER_IDENTITY.id,
code: "provider_unavailable",
reason,
diagnostics: {
configuration_key: "REA_PWNTOOLS_PYTHON",
executable_path: path,
...(systemCode === undefined ? {} : { system_code: systemCode }),
},
},
],
});
/** Retain bounded text output and the shared supervisor's observed truncation flag. */
export const capturedPwntoolsOutput = (
snapshot: ProviderProcessSnapshot,
): AnalysisCapturedOutput => ({
stdout: snapshot.stdout.text,
stderr: snapshot.stderr.text,
truncated: snapshot.diagnosticTruncated === true,
});
@@ -0,0 +1,381 @@
import {
pwntoolsLayoutFailure,
pwntoolsUnavailable,
capturedPwntoolsOutput,
} from "./PwntoolsFailures.js";
import { randomUUID } from "node:crypto";
import { access, stat, writeFile } from "node:fs/promises";
import { constants } from "node:fs";
import { isAbsolute, join } from "node:path";
import { fileURLToPath } from "node:url";
import { z } from "zod";
import type { ExecutionOptions } from "../../application/AnalysisProvider.js";
import type { BinaryLayoutPort } from "../../application/binaryDiagnostics/BinaryLayoutPort.js";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
AnalysisResourceConstraintError,
} from "../../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
binaryLayoutPayloadSchema,
binaryLayoutSchema,
type BinaryLayout,
type InspectBinaryLayoutInput,
} from "../../domain/native/binaryLayout.js";
import { runOwnedCommand } from "../../process/OwnedCommand.js";
import { OwnedCommandFailure } from "../../process/OwnedCommand.js";
import {
pwntoolsResourceLimitsSchema,
type PwntoolsLimitReport,
type PwntoolsFailureEvidence,
} from "./PwntoolsResourceLimits.js";
import { PrivateRuntimeRoot } from "../../process/PrivateRuntimeRoot.js";
import {
PWNTOOLS_PROVIDER_IDENTITY,
PWNTOOLS_LIMITS,
PWNTOOLS_FILE_SIZE_FAILURE_EXIT,
PWNTOOLS_MEMORY_FAILURE_EXIT,
} from "./PwntoolsRelease.js";
const OPERATION = "inspect_binary_layout";
const decoded = binaryLayoutPayloadSchema;
const replySchema = z.discriminatedUnion("ok", [
z.strictObject({
ok: z.literal(true),
profile: z.literal(PWNTOOLS_PROVIDER_IDENTITY.version),
value: decoded,
}),
z.strictObject({
ok: z.literal(false),
reason: z.enum([
"format",
"unsupported",
"unavailable",
"resource-limit",
"output-limit",
"decoder",
]),
message: z.string(),
reported_limits: pwntoolsResourceLimitsSchema.nullable().optional(),
}),
]);
type Launcher = NonNullable<Parameters<typeof runOwnedCommand>[2]>["launcher"];
/** Thin owned-process adapter; all ELF parsing and heuristics come from unchanged upstream code. */
export class PwntoolsLayoutProvider implements BinaryLayoutPort {
readonly identity = PWNTOOLS_PROVIDER_IDENTITY;
constructor(
readonly environment: Readonly<NodeJS.ProcessEnv>,
readonly launcher?: Launcher,
readonly createRuntime: () => Promise<
Pick<PrivateRuntimeRoot, "path" | "close">
> = () => PrivateRuntimeRoot.create({ prefix: "rea-elf-layout-" }),
) {}
/** Snapshot one explicit artifact and return complete static observations without launching the selected object as a host process. */
async inspect(
input: InspectBinaryLayoutInput,
options?: ExecutionOptions,
): Promise<Result<BinaryLayout, AnalysisError>> {
let root: Pick<PrivateRuntimeRoot, "path" | "close"> | undefined;
let result: Result<BinaryLayout, AnalysisError>;
let phase: "configuration" | "artifact-read" | "decoder" = "configuration";
const executablePath = this.environment.REA_PWNTOOLS_PYTHON ?? "";
let selectedPath = executablePath;
try {
if (options?.signal?.aborted) throw new AnalysisCancelledError(OPERATION);
if (process.platform !== "linux" || process.arch !== "x64")
throw new AnalysisCapabilityUnavailableError(
this.identity.id,
OPERATION,
"The real-verified pwntools layout profile currently supports Linux x64 only.",
);
if (!isAbsolute(selectedPath))
throw pwntoolsUnavailable(
"Set REA_PWNTOOLS_PYTHON to an absolute caller-supplied Python executable with pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2; REA never installs Python or packages.",
selectedPath,
);
await access(selectedPath, constants.X_OK);
if (!(await stat(selectedPath)).isFile())
throw pwntoolsUnavailable(
"Selected Python executable must be a regular executable file, not a directory: " +
selectedPath,
selectedPath,
);
phase = "artifact-read";
selectedPath = input.path;
const snapshot = await readStableArtifact(
input.path,
PWNTOOLS_LIMITS.inputBytes,
options?.signal,
);
phase = "decoder";
root = await this.createRuntime();
const requestPath = join(root.path, "request.json");
const snapshotPath = join(root.path, "object.snapshot");
const replyPath = join(root.path, "reply.json");
await writeFile(snapshotPath, snapshot.bytes, {
flag: "wx",
mode: 0o600,
...(options?.signal === undefined ? {} : { signal: options.signal }),
});
await writeFile(
requestPath,
JSON.stringify({ snapshot_path: snapshotPath, reply_path: replyPath }),
{ flag: "wx", mode: 0o600 },
);
const execution = await runOwnedCommand(
{
command: this.environment.REA_PWNTOOLS_PYTHON ?? "",
arguments: [
"-I",
fileURLToPath(
new URL("../../../bridge/pwntools/layout.py", import.meta.url),
),
join(root.path, "resource.failure"),
requestPath,
],
cwd: root.path,
runId: `rea-elf-layout-${randomUUID()}`,
hostEnvironment: {
...this.environment,
PWNLIB_NOTERM: "1",
OPENBLAS_NUM_THREADS: "1",
OMP_NUM_THREADS: "1",
},
},
{
timeoutMs: PWNTOOLS_LIMITS.timeoutMs,
diagnosticBytes: PWNTOOLS_LIMITS.diagnosticBytes,
},
{
...(options?.signal === undefined ? {} : { signal: options.signal }),
...(this.launcher === undefined ? {} : { launcher: this.launcher }),
},
);
const capturedOutput = capturedPwntoolsOutput(execution);
let reply: z.output<typeof replySchema>;
try {
const file = await readStableArtifact(
replyPath,
PWNTOOLS_LIMITS.outputBytes,
options?.signal,
);
reply = replySchema.parse(JSON.parse(file.bytes.toString("utf8")));
} catch (cause: unknown) {
if (options?.signal?.aborted)
throw new AnalysisCancelledError(OPERATION, { capturedOutput });
throw new AnalysisOutputError(
OPERATION,
`Owned ELF decoder reply failed for ${input.path}: ${cause instanceof z.ZodError ? cause.issues[0]?.message + " at " + (cause.issues[0]?.path.map(String).join(".") ?? "root") : cause instanceof Error ? cause.message : String(cause)}`,
{ cause, capturedOutput },
);
}
if (!reply.ok) {
if (reply.reason === "resource-limit")
throw new AnalysisResourceConstraintError(
OPERATION,
"memory",
reply.message,
reply.reported_limits ?? null,
{ capturedOutput },
);
if (reply.reason === "format")
throw new AnalysisInputError(OPERATION, { capturedOutput }, [
{
path: ["path"],
reason: "invalid_format",
message: reply.message,
},
]);
if (reply.reason === "unavailable")
throw pwntoolsUnavailable(
reply.message,
this.environment.REA_PWNTOOLS_PYTHON ?? "",
undefined,
capturedOutput,
);
if (reply.reason === "unsupported")
throw new AnalysisCapabilityUnavailableError(
this.identity.id,
OPERATION,
reply.message,
{ userMessage: reply.message, capturedOutput },
);
if (reply.reason === "output-limit")
throw new AnalysisOutputError(OPERATION, reply.message, {
capturedOutput,
});
throw new ProviderAdapterError(this.identity.id, OPERATION, {
diagnostics: {
phase,
failure_kind: reply.reason,
reason: reply.message,
stdout: execution.stdout.text,
stderr: execution.stderr.text,
captured_output: { ...capturedOutput },
},
});
}
const validated = binaryLayoutSchema.safeParse({
...reply.value,
artifact: {
path: input.path,
sha256: snapshot.sha256,
bytes: snapshot.bytes.length,
},
diagnostics: {
stdout: execution.stdout.text,
stderr: execution.stderr.text,
truncated: capturedOutput.truncated,
},
});
if (!validated.success)
throw new AnalysisOutputError(
OPERATION,
`ELF reply contains invalid source ranges or value meanings: ${validated.error.issues[0]?.message ?? "schema mismatch"}`,
{ capturedOutput },
);
for (const table of validated.data.packed_relative_relocations) {
const start = Number(BigInt(table.location.offset));
const length = Number(BigInt(table.location.bytes));
if (
!Buffer.from(table.encoded_bytes_base64, "base64").equals(
snapshot.bytes.subarray(start, start + length),
)
)
throw new AnalysisOutputError(
OPERATION,
"Reported packed relocation bytes differ from the selected snapshot at their original file range.",
{ capturedOutput },
);
}
result = ok(validated.data);
} catch (cause: unknown) {
let limitReport: PwntoolsLimitReport | undefined;
let marker: PwntoolsFailureEvidence["marker"];
if (
cause instanceof OwnedCommandFailure &&
cause.snapshot?.signal === null &&
(cause.snapshot.exitCode === PWNTOOLS_MEMORY_FAILURE_EXIT ||
cause.snapshot.exitCode === PWNTOOLS_FILE_SIZE_FAILURE_EXIT) &&
root !== undefined
) {
try {
const reported = await readStableArtifact(
join(root.path, "resource.failure"),
1,
);
const expected =
cause.snapshot.exitCode === PWNTOOLS_MEMORY_FAILURE_EXIT
? "M"
: "F";
if (!reported.bytes.equals(Buffer.from(expected, "ascii")))
throw new Error(
"Private bridge failure marker does not match the observed exit status.",
);
marker = {
resource: expected === "M" ? "memory" : "file-size",
failure: null,
};
} catch (markerFailure: unknown) {
marker = {
resource: null,
failure:
markerFailure instanceof Error
? markerFailure.message
: String(markerFailure),
};
}
}
if (
cause instanceof OwnedCommandFailure &&
(cause.snapshot?.signal === "SIGXCPU" ||
cause.snapshot?.signal === "SIGXFSZ" ||
((cause.snapshot?.exitCode === PWNTOOLS_FILE_SIZE_FAILURE_EXIT ||
cause.snapshot?.exitCode === PWNTOOLS_MEMORY_FAILURE_EXIT) &&
cause.snapshot.signal === null)) &&
root !== undefined
) {
try {
const reported = await readStableArtifact(
join(root.path, "limits.json"),
4096,
);
limitReport = {
limits: pwntoolsResourceLimitsSchema.parse(
JSON.parse(reported.bytes.toString("utf8")),
),
failure: null,
};
} catch (reportFailure: unknown) {
limitReport = {
limits: null,
failure:
reportFailure instanceof Error
? reportFailure.message
: String(reportFailure),
};
}
}
result = err(
options?.signal?.aborted &&
(cause === options.signal.reason ||
(cause instanceof Error && cause.name === "AbortError"))
? new AnalysisCancelledError(OPERATION)
: pwntoolsLayoutFailure(cause, phase, selectedPath, executablePath, {
...(limitReport === undefined ? {} : { limits: limitReport }),
...(marker === undefined ? {} : { marker }),
}),
);
}
if (root !== undefined) {
try {
await root.close();
} catch (cause: unknown) {
return err(
new ProviderCleanupError(
this.identity.id,
[root.path],
{
reason: cause instanceof Error ? cause.message : String(cause),
previous_error: result.ok
? null
: projectAnalysisError(result.error),
...(result.ok
? {
captured_output: {
stdout: result.value.diagnostics.stdout,
stderr: result.value.diagnostics.stderr,
truncated: result.value.diagnostics.truncated,
},
}
: {}),
},
{ operation: OPERATION },
),
);
}
}
return result.ok && options?.signal?.aborted
? err(
new AnalysisCancelledError(OPERATION, {
capturedOutput: {
stdout: result.value.diagnostics.stdout,
stderr: result.value.diagnostics.stderr,
truncated: result.value.diagnostics.truncated,
},
}),
)
: result;
}
}
+19
View File
@@ -0,0 +1,19 @@
/** Verified unchanged upstream profile for the Linux ELF adapter. */
export const PWNTOOLS_PROVIDER_IDENTITY = {
id: "pwntools-elf",
name: "REA pwntools ELF adapter",
version: "[email protected];[email protected];[email protected]",
} as const;
/** Bridge-reserved status for MemoryError when no structured reply can be written. */
export const PWNTOOLS_MEMORY_FAILURE_EXIT = 75;
/** Bridge-reserved status for an observed EFBIG failure without a complete reply. */
export const PWNTOOLS_FILE_SIZE_FAILURE_EXIT = 76;
/** Complete evidence budgets; address-space is separate from resident memory. */
export const PWNTOOLS_LIMITS = {
inputBytes: 32 * 1024 * 1024,
outputBytes: 64 * 1024 * 1024,
diagnosticBytes: 1024 * 1024,
timeoutMs: 30_000,
addressSpaceBytes: 3 * 1024 * 1024 * 1024,
cpuSeconds: 30,
} as const;
@@ -0,0 +1,32 @@
import { z } from "zod";
import { PWNTOOLS_LIMITS } from "./PwntoolsRelease.js";
/** Actual limits reported by the owned bridge after lowering inherited soft limits. */
export const pwntoolsResourceLimitsSchema = z.strictObject({
address_space_bytes: z
.number()
.int()
.nonnegative()
.max(PWNTOOLS_LIMITS.addressSpaceBytes),
cpu_seconds: z.number().int().nonnegative().max(PWNTOOLS_LIMITS.cpuSeconds),
file_size_bytes: z
.number()
.int()
.nonnegative()
.max(PWNTOOLS_LIMITS.outputBytes),
});
/** A missing or invalid report leaves limits unknown and retains its read failure. */
export interface PwntoolsLimitReport {
readonly limits: z.output<typeof pwntoolsResourceLimitsSchema> | null;
readonly failure: string | null;
}
/** Private failure-branch evidence; a bare launcher exit code is insufficient. */
export interface PwntoolsFailureEvidence {
readonly limits?: PwntoolsLimitReport;
readonly marker?: {
readonly resource: "memory" | "file-size" | null;
readonly failure: string | null;
};
}
+15
View File
@@ -1,3 +1,6 @@
import { registerBinaryDiagnosticsTools } from "./registerBinaryDiagnosticsTools.js";
import { createBinaryLayoutService } from "../composition/binaryDiagnostics.js";
import type { BinaryLayoutService } from "../application/binaryDiagnostics/BinaryLayoutService.js";
import { McpServer } from "@modelcontextprotocol/server";
import { isAbsolute } from "node:path";
@@ -60,6 +63,7 @@ const ACTIVE_TARGET_INSTRUCTIONS =
export interface CreateServerOptions {
readonly logger?: Logger;
readonly binaryLayout?: BinaryLayoutService;
readonly firmwareAnalysis?: FirmwareAnalysisPort;
readonly javascriptRecovery?: JavaScriptRecoveryPort;
readonly webModuleTrace?: WebModuleTraceService;
@@ -88,6 +92,11 @@ const installSessionToolAvailability = (
webModuleResolutionEnabled:
options.webModuleTrace !== undefined ||
isAbsolute(process.env.REA_BROWSER_EXECUTABLE ?? ""),
binaryLayoutEnabled:
options.binaryLayout !== undefined ||
(process.platform === "linux" &&
process.arch === "x64" &&
isAbsolute(process.env.REA_PWNTOOLS_PYTHON ?? "")),
firmwareInspectionEnabled:
options.firmwareAnalysis !== undefined ||
(process.platform === "linux" &&
@@ -190,6 +199,12 @@ export const createServer = (
toolLogger,
recordEvidence,
);
registerBinaryDiagnosticsTools(
server,
options.binaryLayout ?? createBinaryLayoutService(),
toolLogger,
recordEvidence,
);
registerFirmwareTools(
server,
new FirmwareAnalysisService(
@@ -0,0 +1,32 @@
import type { McpServer } from "@modelcontextprotocol/server";
import type { BinaryLayoutService } from "../application/binaryDiagnostics/BinaryLayoutService.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import { toolContract } from "../contracts/toolContracts.js";
import type { Logger } from "../logger.js";
import { logToolExecution } from "./toolLogging.js";
import { toolRegistrationOptions } from "./toolRegistrationOptions.js";
import { toCallToolResult } from "./toolResult.js";
/** Bind offline layout inspection to its named contract and session Evidence owner. */
export const registerBinaryDiagnosticsTools = (
server: McpServer,
service: BinaryLayoutService,
logger: Logger,
recordEvidence?: EvidenceWriter["recordEvidence"],
): void => {
const contract = toolContract("inspect_binary_layout");
server.registerTool(
contract.name,
toolRegistrationOptions(contract),
async (input, context) => {
const result = await logToolExecution(logger, contract.name, () =>
service.inspect(input, { signal: context.mcpReq.signal }),
);
if (!result.ok) return toCallToolResult(result, contract);
const recorded = recordEvidence?.(result.value);
return recorded !== undefined && !recorded.ok
? toCallToolResult(recorded, contract)
: toCallToolResult(result, contract);
},
);
};
+3
View File
@@ -18,6 +18,7 @@ export interface SessionAvailabilityDefaults {
| "androidAnalysisEnabled"
| "javascriptRecoveryEnabled"
| "webModuleResolutionEnabled"
| "binaryLayoutEnabled"
| "firmwareInspectionEnabled"
| "firmwareExtractionEnabled"
>;
@@ -32,6 +33,8 @@ export const sessionAvailabilityPolicy = (
configured ??
(() => ({
processCaptureEnabled: platform !== "win32",
binaryLayoutEnabled:
defaults.optionalFeatures?.binaryLayoutEnabled ?? false,
firmwareInspectionEnabled:
defaults.optionalFeatures?.firmwareInspectionEnabled ?? false,
firmwareExtractionEnabled:
+1 -1
View File
@@ -9,7 +9,7 @@ import type { Result } from "../domain/result.js";
/**
* Serialize an application result as MCP text and structured content.
* Shared error projection preserves actionable local diagnostics while omitting
* raw causes and captured process output.
* raw cause objects; explicitly retained bounded command output stays visible.
*/
export const toCallToolResult = (
result: Result<JsonValue, AnalysisError>,
@@ -0,0 +1,691 @@
import {
createTestWorkspace,
removeTestWorkspace,
} from "../../support/workspace/workspaceFixture.js";
import { access, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { z } from "zod";
import { expect, it, onTestFinished } from "vitest";
import { PwntoolsLayoutProvider } from "../../../src/native/pwntools/PwntoolsLayoutProvider.js";
import { PWNTOOLS_PROVIDER_IDENTITY } from "../../../src/native/pwntools/PwntoolsRelease.js";
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
import { waitForProviderProcessReady } from "../../fixtures/providerProcess.js";
import { PrivateRuntimeRoot } from "../../../src/process/PrivateRuntimeRoot.js";
import { binaryLayoutFixture } from "../../fixtures/binaryDiagnostics/layout.js";
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
const unsupportedHost = process.platform !== "linux" || process.arch !== "x64";
const requestSchema = z.strictObject({
snapshot_path: z.string(),
reply_path: z.string(),
});
it.runIf(!unsupportedHost).each([
["SIGXCPU", "reported", "cpu"],
["SIGXCPU", "missing", "cpu"],
["SIGXCPU", "malformed", "cpu"],
["SIGXFSZ", "reported", "file-size"],
["EFBIG", "reported", "file-size"],
["EFBIG", "malformed", "file-size"],
] as const)(
"preserves observed %s with %s limit evidence",
async (failure, scenario, resource) => {
const { path } = await fixture();
let ownedPath = "";
const limits = {
address_space_bytes: 3221225472,
cpu_seconds: 1,
file_size_bytes: 1024,
};
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
ownedPath = spawn.cwd ?? "";
if (failure === "EFBIG")
await writeFile(join(ownedPath, "resource.failure"), "F");
if (scenario !== "missing")
await writeFile(
join(ownedPath, "limits.json"),
scenario === "reported"
? JSON.stringify(limits)
: '{"cpu_seconds":999}',
);
return spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: [
"-e",
failure === "EFBIG"
? 'process.stdout.write("resource diagnostic", () => { process.exitCode = 76; })'
: failure === "SIGXFSZ"
? 'process.on("SIGXFSZ", () => { process.removeAllListeners("SIGXFSZ"); process.kill(process.pid, "SIGXFSZ"); }); process.stdout.write("resource diagnostic", () => process.kill(process.pid, "SIGXFSZ")); setTimeout(() => {}, 1000);'
: `process.stdout.write("resource diagnostic", () => process.kill(process.pid, "${failure}"))`,
],
});
},
);
const result = await provider.inspect({ path });
if (result.ok) throw new Error("Expected observed resource failure");
const projected = projectAnalysisError(result.error);
expect(projected).toMatchObject({
code: "resource_constraint",
details: {
resource,
reported_limits: scenario === "reported" ? limits : null,
captured_output: { stdout: "resource diagnostic", truncated: false },
},
});
expect(result.error.message).toContain(
failure === "EFBIG"
? "exact write failure cause is unknown"
: "exact signal cause is unknown",
);
if (scenario !== "reported")
expect(result.error.message).toContain("limit report unavailable");
expect(projected.remediation.action).toContain(
resource === "cpu" ? "CPU" : "file-size",
);
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(!unsupportedHost)(
"retains actual diagnostic truncation when output overflow and process cleanup both fail",
async () => {
const { path } = await fixture();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
ownedPath = spawn.cwd ?? "";
const launched = await spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: ["-e", 'process.stdout.write("x".repeat(1048577))'],
});
return {
...launched,
cleanup: async () => {
await launched.cleanup?.();
throw new Error("injected post-cleanup reporting failure");
},
};
},
);
const result = await provider.inspect({ path });
if (result.ok) throw new Error("Expected compounded lifecycle failure");
expect(result.error).toMatchObject({
cleanupIncomplete: true,
diagnostics: {
reason: "injected post-cleanup reporting failure",
previous_error: { failure_kind: "output-limit" },
captured_output: { truncated: true, stderr: "" },
},
});
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it
.runIf(!unsupportedHost)
.each([
"reserved-memory-status",
"unmarked-memory-status",
"unmarked-file-status",
"wrong-marker",
"ordinary-exit",
"signal",
])(
"requires matching private bridge failure evidence, without guessing from stderr: %s",
async (scenario) => {
const { path } = await fixture();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
ownedPath = spawn.cwd ?? "";
if (
scenario === "reserved-memory-status" ||
scenario === "wrong-marker"
)
await writeFile(
join(ownedPath, "resource.failure"),
scenario === "wrong-marker" ? Buffer.from([0xcd]) : "M",
);
return spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: [
"-e",
'process.stderr.write("MemoryError-like diagnostic\\n");' +
(scenario === "signal"
? 'process.kill(process.pid,"SIGKILL")'
: `process.exit(${scenario === "unmarked-file-status" ? "76" : ["reserved-memory-status", "unmarked-memory-status", "wrong-marker"].includes(scenario) ? "75" : "7"})`),
],
});
},
);
const result = await provider.inspect({ path });
if (result.ok) throw new Error("Expected explicit process failure");
const projected = projectAnalysisError(result.error);
expect(projected.code).toBe(
scenario === "reserved-memory-status"
? "resource_constraint"
: "execution_failure",
);
expect(JSON.stringify(projected)).toContain("MemoryError-like diagnostic");
if (scenario === "reserved-memory-status")
expect(projected.details).toMatchObject({
resource: "memory",
reported_limits: null,
captured_output: { truncated: false },
});
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(!unsupportedHost).each([1048576, 1048577])(
"keeps complete diagnostics or fails explicitly when a valid reply emits %s bytes",
async (bytes) => {
const { path } = await fixture();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
ownedPath = spawn.cwd ?? "";
const requestPath = spawn.arguments.at(-1);
if (requestPath === undefined) throw new Error("Request path missing");
const request = requestSchema.parse(
JSON.parse(await readFile(requestPath, "utf8")),
);
const {
artifact: _artifact,
diagnostics: _diagnostics,
...value
} = binaryLayoutFixture(path);
await writeFile(
request.reply_path,
JSON.stringify({
ok: true,
profile: PWNTOOLS_PROVIDER_IDENTITY.version,
value,
}),
);
return spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: [
"-e",
`process.stdout.write("x".repeat(${String(bytes)}))`,
],
});
},
);
const result = await provider.inspect({ path });
if (bytes === 1048576) {
if (!result.ok) throw result.error;
expect(result.value.diagnostics.stdout).toHaveLength(bytes);
expect(result.value.diagnostics.truncated).toBe(false);
} else {
if (result.ok)
throw new Error("Excess diagnostics must not return partial success");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "unreadable_output",
details: { captured_output: { truncated: true } },
});
}
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(!unsupportedHost).each([
null,
{
address_space_bytes: 67108864,
cpu_seconds: 20,
file_size_bytes: 33554432,
},
])(
"preserves reported memory failure, budgets, diagnostics and cleanup: %j",
async (limits) => {
const { path } = await fixture();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
ownedPath = spawn.cwd ?? "";
const requestPath = spawn.arguments.at(-1);
if (requestPath === undefined) throw new Error("Request path missing");
const request = requestSchema.parse(
JSON.parse(await readFile(requestPath, "utf8")),
);
await writeFile(
request.reply_path,
JSON.stringify({
ok: false,
reason: "resource-limit",
message: "Memory allocation failed; exact cause unknown.",
reported_limits: limits,
}),
);
return spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: [
"-e",
"process.stderr.write('upstream allocation diagnostic')",
],
});
},
);
const result = await provider.inspect({ path });
if (result.ok) throw new Error("Expected resource failure");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "resource_constraint",
category: "resource_constraint",
details: {
resource: "memory",
reported_limits: limits,
captured_output: {
stderr: "upstream allocation diagnostic",
truncated: false,
},
},
});
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
expect(await readFile(path, "utf8")).toBe("source-owned-seam-bytes");
},
);
it.runIf(!unsupportedHost).each(["cancelled", "output-limit", "process"])(
"releases an acquired decoder and its snapshot after %s",
async (scenario) => {
const { path } = await fixture();
const controller = new AbortController();
let ownedPath = "";
let pid: number | undefined;
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
ownedPath = spawn.cwd ?? "";
const launched = await spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: [
"-e",
scenario === "cancelled"
? 'process.stdout.write("ready\\n"); setInterval(() => undefined, 1000)'
: scenario === "output-limit"
? 'process.stdout.write("x".repeat(1048577))'
: 'process.stderr.write("decoder exited unexpectedly"); process.exit(7)',
],
});
pid = launched.process.pid;
if (scenario === "cancelled") {
await waitForProviderProcessReady(launched.process);
setImmediate(() => controller.abort());
}
return launched;
},
);
const result = await provider.inspect(
{ path },
{ signal: controller.signal },
);
expect(result).toMatchObject({
ok: false,
error: {
_tag:
scenario === "cancelled"
? "AnalysisCancelledError"
: scenario === "output-limit"
? "AnalysisOutputError"
: "ProviderAdapterError",
...(scenario === "process"
? {
diagnostics: {
exit_code: 7,
stderr: "decoder exited unexpectedly",
},
}
: {}),
},
});
if (pid === undefined) throw new Error("Decoder was not acquired");
const acquiredPid = pid;
expect(() => process.kill(acquiredPid, 0)).toThrow();
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
expect(await readFile(path, "utf8")).toBe("source-owned-seam-bytes");
},
);
it.runIf(!unsupportedHost)(
"reports private storage denial as an adapter failure rather than selected-input read denial",
async () => {
const { path } = await fixture();
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
undefined,
() =>
Promise.reject(
Object.assign(new Error("private root denied"), { code: "EACCES" }),
),
);
expect(await provider.inspect({ path })).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
diagnostics: { phase: "decoder", reason: "private root denied" },
},
});
},
);
it.runIf(!unsupportedHost)(
"preserves a decoder failure when private-root cleanup also fails",
async () => {
const { path, root } = await fixture();
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
const requestPath = spawn.arguments.at(-1);
if (requestPath === undefined) throw new Error("Request path missing");
const request = requestSchema.parse(
JSON.parse(await readFile(requestPath, "utf8")),
);
await writeFile(
request.reply_path,
JSON.stringify({
ok: false,
reason: "format",
message: "Malformed original ELF table.",
}),
);
return spawnOwnedProviderProcess({
...spawn,
arguments: ["-e", "process.exit(0)"],
});
},
() =>
Promise.resolve({
path: root,
close: () => Promise.reject(new Error("root cleanup failed")),
}),
);
expect(await provider.inspect({ path })).toMatchObject({
ok: false,
error: {
cleanupIncomplete: true,
cleanupResources: [root],
diagnostics: {
reason: "root cleanup failed",
previous_error: { category: "invalid_input" },
},
},
});
},
);
const fixture = async () => {
const workspace = await createTestWorkspace("rea-layout-boundary-");
const root = workspace.root;
onTestFinished(() => removeTestWorkspace(root));
const path = join(root, "selected.elf");
await writeFile(path, "source-owned-seam-bytes");
return { root, path };
};
it.runIf(!unsupportedHost)(
"preserves cancellation during the actual signal-aware snapshot write and closes its acquired root",
async () => {
const { path } = await fixture();
const controller = new AbortController();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
() => {
throw new Error("Decoder must not launch");
},
async () => {
const root = await PrivateRuntimeRoot.create({
prefix: "rea-layout-cancel-",
});
ownedPath = root.path;
controller.abort();
return root;
},
);
expect(
await provider.inspect({ path }, { signal: controller.signal }),
).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } });
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it
.runIf(!unsupportedHost)
.each([
"success",
"format",
"unavailable",
"unsupported",
"output-limit",
"wrong-profile",
"malformed-reply",
])(
"preserves process/reply boundaries and cleans private snapshots: %s",
async (scenario) => {
const { root, path } = await fixture();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
const requestPath = spawn.arguments.at(-1);
if (requestPath === undefined) throw new Error("request path missing");
ownedPath = spawn.cwd ?? "";
expect(spawn.arguments[0]).toBe("-I");
const request = requestSchema.parse(
JSON.parse(await readFile(requestPath, "utf8")),
);
expect(await readFile(request.snapshot_path, "utf8")).toBe(
"source-owned-seam-bytes",
);
const {
artifact: _artifact,
diagnostics: _diagnostics,
...value
} = binaryLayoutFixture(path);
await writeFile(
request.reply_path,
scenario === "malformed-reply"
? "broken-json"
: JSON.stringify(
[
"format",
"unavailable",
"unsupported",
"output-limit",
].includes(scenario)
? {
ok: false,
reason: scenario,
message: "ELF table is truncated.",
}
: {
ok: true,
profile:
scenario === "wrong-profile"
? "pwntools@other"
: PWNTOOLS_PROVIDER_IDENTITY.version,
value,
},
),
);
return spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: ["-e", "process.stderr.write('upstream warning')"],
});
},
);
const result = await provider.inspect({ path });
if (scenario === "success") {
if (!result.ok) throw result.error;
expect(result.value.artifact).toMatchObject({
path,
bytes: Buffer.byteLength("source-owned-seam-bytes"),
});
expect(result.value.artifact.sha256).toMatch(/^[a-f0-9]{64}$/);
expect(result.value.diagnostics.stderr).toBe("upstream warning");
} else
expect(result).toMatchObject({
ok: false,
error: {
_tag:
scenario === "format"
? "AnalysisInputError"
: scenario === "unavailable"
? "ProviderSelectionError"
: scenario === "unsupported"
? "AnalysisCapabilityUnavailableError"
: "AnalysisOutputError",
},
});
if (!result.ok)
expect(projectAnalysisError(result.error).details).toMatchObject({
captured_output: {
stdout: "",
stderr: "upstream warning",
truncated: false,
},
});
expect(await readFile(path, "utf8")).toBe("source-owned-seam-bytes");
expect(ownedPath.startsWith(root)).toBe(false);
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(!unsupportedHost)(
"rejects missing configuration, absent input and prelaunch cancellation distinctly",
async () => {
const { path } = await fixture();
expect(
await new PwntoolsLayoutProvider({}).inspect({ path }),
).toMatchObject({
ok: false,
error: { _tag: "ProviderSelectionError", reason: "provider_unavailable" },
});
const provider = new PwntoolsLayoutProvider({
REA_PWNTOOLS_PYTHON: process.execPath,
});
expect(await provider.inspect({ path: path + ".absent" })).toMatchObject({
ok: false,
error: { _tag: "AnalysisInputError" },
});
const controller = new AbortController();
controller.abort();
expect(
await provider.inspect({ path }, { signal: controller.signal }),
).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } });
},
);
it.runIf(unsupportedHost)(
"reports unverified host coverage before acquiring a provider",
async () => {
expect(
await new PwntoolsLayoutProvider({}).inspect({ path: "/selected.elf" }),
).toMatchObject({
ok: false,
error: { _tag: "AnalysisCapabilityUnavailableError" },
});
},
);
it.runIf(!unsupportedHost)(
"retains decoder warnings when cancellation arrives during successful root cleanup",
async () => {
const { path } = await fixture();
const controller = new AbortController();
let ownedPath = "";
const provider = new PwntoolsLayoutProvider(
{ REA_PWNTOOLS_PYTHON: process.execPath },
async (spawn) => {
const requestPath = spawn.arguments.at(-1);
if (requestPath === undefined) throw new Error("Request path missing");
const request = requestSchema.parse(
JSON.parse(await readFile(requestPath, "utf8")),
);
const {
artifact: _artifact,
diagnostics: _diagnostics,
...value
} = binaryLayoutFixture(path);
await writeFile(
request.reply_path,
JSON.stringify({
ok: true,
profile: PWNTOOLS_PROVIDER_IDENTITY.version,
value,
}),
);
return spawnOwnedProviderProcess({
...spawn,
command: process.execPath,
arguments: ["-e", 'process.stderr.write("retained late warning")'],
});
},
async () => {
const root = await PrivateRuntimeRoot.create({
prefix: "rea-layout-late-cancel-",
});
ownedPath = root.path;
return {
path: root.path,
close: async () => {
await root.close();
controller.abort();
},
};
},
);
const result = await provider.inspect(
{ path },
{ signal: controller.signal },
);
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
if (result.ok) throw new Error("Late cancellation must be preserved");
expect(projectAnalysisError(result.error).details?.captured_output).toEqual(
{ stdout: "", stderr: "retained late warning", truncated: false },
);
await expect(access(ownedPath)).rejects.toMatchObject({ code: "ENOENT" });
},
);
it.runIf(!unsupportedHost).each(["directory", "missing-interpreter"])(
"identifies the configured executable on actual launch/preflight failure: %s",
async (scenario) => {
const { path, root } = await fixture();
const executable =
scenario === "directory" ? root : join(root, "broken-python");
if (scenario === "missing-interpreter")
await writeFile(
executable,
"#!/rea-missing-interpreter-for-boundary-test\n",
{ mode: 0o700 },
);
const result = await new PwntoolsLayoutProvider({
REA_PWNTOOLS_PYTHON: executable,
}).inspect({ path });
if (result.ok) throw new Error("Expected unavailable executable");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "provider_unavailable",
details: {
rejections: [{ diagnostics: { executable_path: executable } }],
},
});
expect(result.error._tag).toBe("ProviderSelectionError");
expect(await readFile(path, "utf8")).toBe("source-owned-seam-bytes");
},
);
@@ -1,3 +1,4 @@
import { PWNTOOLS_PROVIDER_IDENTITY } from "../../../src/native/pwntools/PwntoolsRelease.js";
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
@@ -115,6 +116,7 @@ describe("canonical product catalog", () => {
GHIDRA_PROVIDER_IDENTITY,
IDA_PROVIDER_IDENTITY,
NATIVE_MACOS_PROVIDER_IDENTITY,
PWNTOOLS_PROVIDER_IDENTITY,
ARTIFACT_GRAPH_PROVIDER,
ANDROID_APPLICATION_PROVIDER,
APPLE_APPLICATION_PROVIDER,
@@ -0,0 +1,76 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it, onTestFinished } from "vitest";
import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { ok } from "../../../src/domain/result.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import {
binaryLayoutFixture,
BINARY_LAYOUT_TEST_PROVIDER,
} from "../../fixtures/binaryDiagnostics/layout.js";
it("advertises exact valid schemas and records inline layout without a binary target", async () => {
const value = binaryLayoutFixture();
const service = new BinaryLayoutService({
identity: BINARY_LAYOUT_TEST_PROVIDER,
inspect: () => Promise.resolve(ok(value)),
});
const session = createTestBinarySession(() => {
throw new Error("deep provider must not start");
});
const server = createServer(session, session, { binaryLayout: service });
const client = new Client({ name: "binary-layout-contract", version: "1" });
onTestFinished(async () => {
await client.close();
await server.close();
await session.close();
});
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
await server.connect(serverTransport);
await client.connect(clientTransport);
const advertised = (await client.listTools()).tools.find(
(tool) => tool.name === "inspect_binary_layout",
);
if (advertised?.outputSchema === undefined)
throw new Error("layout must publish both schemas");
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const inputSchema: Record<string, unknown> = advertised.inputSchema;
const outputSchema: Record<string, unknown> = advertised.outputSchema;
expect(ajv.validateSchema(inputSchema)).toBe(true);
expect(ajv.validateSchema(outputSchema)).toBe(true);
expect(ajv.validate(inputSchema, { path: value.artifact.path })).toBe(true);
expect(
ajv.validate(inputSchema, {
path: value.artifact.path,
approval: true,
}),
).toBe(false);
const response = await client.callTool({
name: "inspect_binary_layout",
arguments: { path: value.artifact.path },
});
expect(response.isError).not.toBe(true);
expect(
ajv.validate(outputSchema, response.structuredContent),
JSON.stringify(ajv.errors),
).toBe(true);
const parsed = toolContract("inspect_binary_layout").outputSchema.parse(
response.structuredContent,
);
const evidence = parseEvidence(parsed.evidence);
expect(parsed.result).toEqual(value);
expect(parsed.result).toEqual(evidence.normalized_result);
expect(session.evidenceById(evidence.evidence_id)).toEqual(evidence);
expect(
(
await client.callTool({
name: "inspect_binary_layout",
arguments: { path: "relative.elf" },
})
).isError,
).toBe(true);
});
+48
View File
@@ -0,0 +1,48 @@
import type { BinaryLayout } from "../../../src/domain/native/binaryLayout.js";
/** Small source-owned provider observation for validation and transport tests. */
export const binaryLayoutFixture = (
path = "/artifacts/source-owned.elf",
): BinaryLayout => ({
artifact: { path, sha256: "a".repeat(64), bytes: 128 },
format: "elf",
architecture: { machine: "EM_X86_64", bits: 64, byte_order: "little" },
image_type: "ET_REL",
entry_point: {
reported_value: "0x0",
meaning: "not-applicable",
execution_status: "unknown",
},
runtime_load_base: null,
sections: [],
segments: [],
symbols: [],
relocations: [],
packed_relative_relocations: [],
relocation_inventory_completeness: "unknown",
linkage: {
needed_libraries: [],
interpreters: [],
got: [],
plt: [],
convenience_maps_completeness: "unknown",
runtime_library_paths: null,
},
mitigations: {
evidence_kind: "inferred",
position_independent: false,
nx_indicator: null,
executable_stack_indicator: false,
stack_canary_indicator: false,
relro: null,
},
diagnostics: { stdout: "producer warning", stderr: "", truncated: false },
limitations: ["Runtime addresses remain unknown."],
});
/** Producer identity belongs to the test seam, not a simulated installed engine claim. */
export const BINARY_LAYOUT_TEST_PROVIDER = {
id: "fixture-layout",
name: "Source-owned layout seam",
version: "1",
} as const;
+40
View File
@@ -0,0 +1,40 @@
# Offline ELF upstream profile
REA uses caller-supplied, unchanged Python packages. It vendors no pwntools,
pyelftools or Unicorn implementation and never installs these dependencies.
| Upstream | Exact verified profile | License | Role |
| --- | --- | --- | --- |
| [pwntools](https://github.com/Gallopsled/pwntools) | 4.15.0; tag commit `3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c` | MIT | ELF analysis and static mitigation/GOT/PLT heuristics |
| [pyelftools](https://github.com/eliben/pyelftools) | 0.33 | Public domain | Original ELF sections, segments, symbols and relocations |
| [Unicorn](https://github.com/unicorn-engine/unicorn) | 2.1.2 | GPL-2.0 | Upstream PLT emulation; distinct from executing the target as a host process |
The adapter checks all three runtime versions before importing the ELF engine.
Original upstream code/licenses remain in the caller-selected environment.
No target or core binaries are committed. Fixtures are compiled in owned
temporary storage from repository source and are never executed.
`bridge/pwntools/layout.py` is an owned allocation-failure bootstrap;
`layout_impl.py` is the adapter. It reads original tables through
unchanged upstream APIs, preserves raw name bytes/ranges, and serializes uint64
values as strings. It never accesses `ELF.libs`, `ELF.maps`, `ELF.libc` or
`process.corefile`, which have materially different execution authority.
The real verification lane traces host exec syscalls, checks original artifact
hashes and observes released owned processes. No negative claim relies only on
an unmodified target's sentinel: upstream can execute a patched copy when its
runtime-library accessors are used.
The instance-local adapter subclass forwards the documented pyelftools 0.33
`iter_segments(type=...)` / `iter_sections(type=...)` filters missing from
pwntools 4.15.0 cached overrides. No upstream file or global method is changed.
Sectionless dynamic string tables use unique DT_STRTAB/DT_STRSZ mappings to
file-backed PT_LOAD bytes; ambiguous or non-file-backed mappings are explicitly
unsupported. Section-name table escapes follow the [ELF ABI](https://gabi.xinuos.com/elf/02-eheader.html):
SHN_XINDEX represents actual indices at least 0xff00; ordinary absence uses index
zero. Raw name offsets remain reported when the name table is absent.
PLT convenience inference uses the unchanged upstream Unicorn instruction
emulator. REA labels those maps as derived static evidence. The no-execution
verification claim concerns launching the selected object as a host process,
not the absence of static instruction emulation inside an analysis engine.