Compare commits

..
17 changed files with 147 additions and 1030 deletions

No files matched your search

-5
View File
@@ -358,11 +358,6 @@ export const documentationSections = [
source: "design/limits-and-recovery.md",
route: "/design/limits-and-recovery",
},
{
text: "REA limit inventory",
source: "design/rea-limit-inventory.md",
route: "/design/rea-limit-inventory",
},
{
text: "MCP tool design",
source: "development/tool-design.md",
-1
View File
@@ -111,7 +111,6 @@ product pages. The VitePress documentation source is this `docs/` tree.
## Development and verification
- [Settings and recovery](design/limits-and-recovery.md)
- [REA limit inventory](design/rea-limit-inventory.md)
- [Flutter budget calibration](verification/flutter-budget-calibration.md)
- [Source-map budget calibration](verification/source-map-budget-calibration.md)
- [Source-location delivery qualification](verification/source-location-delivery.md)
+1 -3
View File
@@ -5,9 +5,7 @@ who owns each value, when an update takes effect, how providers enforce it, and
how to extend the system. The public tool contract is in
[MCP contracts](../reference/mcp-contracts.md#editable-analysis-settings); recorded runtime
checks and their limitations are in the
[verification report](../verification/runtime-analysis-settings.md). The
[limit inventory](rea-limit-inventory.md) lists every numeric `REA_` limit, which
of them are runtime settings, and which stay restart-only.
[verification report](../verification/runtime-analysis-settings.md).
## Problem and design boundary
-225
View File
@@ -1,225 +0,0 @@
# REA limit inventory
Audit for [#9881](https://github.com/morluto/rea/issues/9881). Counts below are
from this revision of `src`. A numeric `REA_` limit is a name under `src` (tests
excluded) that carries a byte, time, count, heap, budget, deadline, or maximum.
`scripts/verify/rea-numeric-limits.mjs` fails the static checks when one of those
names is neither an environment seed of
[`ANALYSIS_SETTINGS`](https://github.com/morluto/rea/blob/main/src/domain/analysisSettingDefinitions.ts) nor an
entry in [`numericLimitRegistry.ts`](https://github.com/morluto/rea/blob/main/src/domain/numericLimitRegistry.ts)
with a reason.
`effect` is not unified yet. The restart column is the structured field a
failure uses today. Prose that only says to restart REA is marked `prose`.
`mcp_server_restart_required` is a server-identity alignment state in
`src/serverIdentity.ts`, not a setting flag.
Three numeric limits are absent from this tree and are being moved into the
runtime system on their own PRs. This audit does not change them:
| Name | Tracking |
| -------------------------------------------- | -------- |
| `REA_NTFS_MFT_MAX_FILE_BYTES` | #9868 |
| `REA_REGF_HIVE_MAX_FILE_BYTES` | #9865 |
| `REA_WEB_BUNDLE_ENDPOINT_LITERAL_CANDIDATES` | #9836 |
When those names land in `src`, the static check requires them to be analysis
settings. They are not on the restart-only list.
## Counts
| Category | Count |
| --------------------------------------------------------------------- | ---------------------------------------------------------------------------: |
| Runtime analysis settings | 17 (14 have an `REA_` seed; 3 are per-call parameters only) |
| Per-call numeric `REA_` limits still read from the launch environment | 11 |
| Restart-only numeric `REA_` limits | 15 |
| Numeric `REA_` names discovered in `src` | 40 |
| Non-numeric `REA_` settings kept restart-only | 53 caller settings, plus process-protocol names that are not caller settings |
| Hardcoded ceilings left for follow-up | grouped below; none converted in the inventory change |
## Runtime analysis settings
Already adjustable with `update_analysis_settings`. Each call reads the snapshot
accepted for that operation. Launch source is `default` or `setting`; a runtime
override is `source: "runtime"`. The structured restart field is
`rea_restart_required: false`. `applies_at` is the finer point (`next_operation`,
`next_worker`, or `next_startup`).
| Setting | Environment seed | Owner | Default | Floor | Maximum | Read |
| -------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------- | ----------------------------------------- | ---------------: | -------------------------------------: | --------------------- |
| `asar.max_header_bytes` | `REA_ASAR_MAX_HEADER_BYTES` | `src/domain/analysisResourceLimits.ts` | 16777216 | 1 | 268435456 | per operation |
| `asar.max_listing_bytes` | `REA_ASAR_MAX_LISTING_BYTES` | `src/domain/analysisResourceLimits.ts` | 67108864 | 1 | `Number.MAX_SAFE_INTEGER` | per operation |
| `ghidra.value_flow_max_operations` | `REA_GHIDRA_VALUE_FLOW_MAX_OPERATIONS` | `src/domain/analysisResourceLimits.ts` | 3000 | 1 | 2147483647 | per operation |
| `ghidra.value_flow_max_inputs_per_operation` | `REA_GHIDRA_VALUE_FLOW_MAX_INPUTS_PER_OPERATION` | `src/domain/analysisResourceLimits.ts` | 64 | 1 | 2147483647 | per operation |
| `ghidra.value_flow_max_edges` | `REA_GHIDRA_VALUE_FLOW_MAX_EDGES` | `src/domain/analysisResourceLimits.ts` | 12000 | 1 | 2147483647 | per operation |
| `ghidra.decompile_max_payload_mib` | `REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB` | `src/domain/analysisResourceLimits.ts` | unset (`null`, unbounded payload policy) | 1 | 2047 | per operation |
| `ghidra.startup_timeout_ms` | `REA_GHIDRA_STARTUP_TIMEOUT_MS` | `src/domain/analysisResourceLimits.ts` | 330000 | 1 | 2147483647 | next provider startup |
| `hopper.startup_timeout_ms` | `REA_HOPPER_STARTUP_TIMEOUT_MS` | `src/domain/analysisResourceLimits.ts` | 120000 | 1 | 2147483647 | next provider startup |
| `ghidra.decompile_timeout_seconds` | `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` | `src/domain/analysisResourceLimits.ts` | unset (`null`, decompile stays unbounded) | 1 | `MAX_GHIDRA_DECOMPILE_TIMEOUT_SECONDS` | per operation |
| `jadx.operation_timeout_ms` | `REA_JADX_OPERATION_TIMEOUT_MS` | `src/domain/analysisResourceLimits.ts` | 120000 | 1 | 2147483647 | per operation |
| `regex.matching_deadline_ms` | `REA_REGEX_MATCHING_DEADLINE_MS` | `src/domain/analysisResourceLimits.ts` | 5000 | 1 | 2147483647 | per operation |
| `regex.worker_heap_request_mib` | `REA_REGEX_WORKER_HEAP_REQUEST_MIB` | `src/domain/analysisResourceLimits.ts` | 128 | 7 | 16384 | next worker |
| `capture.decoder_old_generation_mib` | `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` | `src/domain/analysisResourceLimits.ts` | 192 | semi-space floor | 16384 | next worker |
| `javascript.analysis_timeout_ms` | none (`analysis_timeout_ms`) | `src/domain/javascript/javascriptAnalysisResourceControls.ts` | owner default | 1 | 2147483647 | per operation |
| `javascript.max_heap_mb` | none (`max_heap_mb`) | `src/domain/javascript/javascriptAnalysisResourceControls.ts` | owner default | owner floor | owner maximum | next worker |
| `wasm.command_timeout_ms` | none (`command_timeout_ms`) | `src/domain/wasm/wasmArtifact.ts` | owner default | 1 | owner maximum | per operation |
| `native.matching_representation_bytes` | `REA_NATIVE_MATCHING_REPRESENTATION_BYTES` | `src/domain/analysisResourceLimits.ts` | 100663296 | 1 | 1073741824 | per operation |
An out-of-range `update_analysis_settings` value is rejected by the setting's
schema. Invalid launch seeds that the global environment parser accepts as
provider-local stay inspectable with `source: "invalid"`.
## Per-call limits still outside the runtime system
These are checked when the work runs. They keep the `REA_` variable, default,
floor, and maximum, and the migration reads the accepted snapshot instead of a
value copied at process start. Until that lands they are `policy: "per-call"` in
the registry so the static check still requires a reason.
| Environment | Owner | Default | Floor | Maximum | Read today | Restart field |
| ------------------------------------------ | ------------------------------------------- | ----------------------: | ----: | ---------: | ------------------------------------------------------------------------------------- | -------------------------------------------------- |
| `REA_BRIDGE_MAX_REQUEST_BYTES` | `src/config/bridgeMaxRequestBytes.ts` | 16777216 | 7464 | 536870887 | each bridge request, from the launch value | prose |
| `REA_DEBUGGER_COMMAND_DEADLINE_MS` | `src/domain/analysisResourceLimits.ts` | 120000 | 1 | 2147483647 | copied when the debugger manager is constructed; applied to each command | `restart_required: true` |
| `REA_DEBUGGER_MAX_COMMAND_OUTPUT_BYTES` | `src/domain/analysisResourceLimits.ts` | 16777216 | 1 | 89303718 | copied when the debugger manager is constructed; applied to each command | `restart_required: true` |
| `REA_HOPPER_CALLER_SCAN_BUDGET_MS` | `src/domain/analysisResourceLimits.ts` | 2000 | 1 | 2147483647 | each caller-scan operation, from config resolved at launch | prose |
| `REA_JEB_MCP_TIMEOUT_MS` | `src/jeb/JebMcpConnection.ts` | 60000 (MCP SDK default) | 1 | 2147483647 | captured when the JEB connection is created; applied to discovery and each tools/call | prose |
| `REA_KUNA_PROJECT_TIMEOUT_MS` | `src/kuna/KunaConfiguration.ts` | 1800000 | 1 | 2147483647 | project export and whole-program scans, from the launch configuration | prose |
| `REA_KUNA_TIMEOUT_MS` | `src/kuna/KunaConfiguration.ts` | 120000 | 1 | 2147483647 | each function decompile, from the launch configuration | prose |
| `REA_PWNTOOLS_MAX_INPUT_BYTES` | `src/native/pwntools/PwntoolsInputLimit.ts` | 33554432 | 1 | 1073741824 | each pwntools decode, unless the call passes `producer_limits.input_bytes` | prose |
| `REA_WINDOWS_CHANGE_TIME_DRIFT_MS` | `src/config/windowsChangeTimeSettle.ts` | 2000 | 1 | 60000 | each ChangeTime observation | `restartRequired: true` on an invalid launch value |
| `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS` | `src/config/windowsChangeTimeSettle.ts` | 25 | 1 | 1000 | each ChangeTime observation | `restartRequired: true` on an invalid launch value |
| `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS` | `src/config/windowsChangeTimeSettle.ts` | 40 | 1 | 1000 | each ChangeTime observation | `restartRequired: true` on an invalid launch value |
## Restart-only numeric limits
These stay out of `update_analysis_settings`.
| Environment | Owner | Default | Floor | Maximum | Read | Why restart-only |
| ----------------------------------- | -------------------------------------- | ------------------: | -------: | ------------------------: | ------------------------------------------- | ------------------------------------------------------------------------------------- |
| `REA_DEBUGGER_MAX_PENDING_COMMANDS` | `src/domain/analysisResourceLimits.ts` | 16 | 1 | 256 | debugger manager construction | Queue capacity of the manager created at startup. Field: `restart_required: true`. |
| `REA_DEBUGGER_MAX_SESSIONS` | `src/domain/analysisResourceLimits.ts` | 8 | 1 | 64 | debugger manager construction | Session-pool size of the manager created at startup. Field: `restart_required: true`. |
| `REA_HOPPER_DEMO_DIALOG_TIMEOUT_MS` | `src/domain/hopperStartupFailure.ts` | 10000 | 1 | 2147483647 | Linux private-display adapter start | The helper reads the process environment. Prose restart. |
| `REA_JADX_ACTIVE_PROCESSOR_COUNT` | `src/android/JadxConfiguration.ts` | unset (JVM default) | 1 | 2147483647 | JADX JVM launch | `-XX:ActiveProcessorCount` is fixed for that JVM. Prose on invalid values. |
| `REA_JADX_HEAP_MIB` | `src/android/JadxConfiguration.ts` | unset (JVM default) | 1 | `Number.MAX_SAFE_INTEGER` | JADX JVM launch | `-Xmx` is fixed for that JVM. Prose on invalid values. |
| `REA_KUNA_CACHE_MAX_BYTES` | `src/kuna/KunaConfiguration.ts` | 4294967296 | 1048576 | `Number.MAX_SAFE_INTEGER` | Kuna cache construction | The cache is sized once for the provider. Prose restart. |
| `REA_KUNA_JOBS` | `src/kuna/KunaConfiguration.ts` | 1 | 1 | 32 | Kuna run-slot construction | Worker-pool width for the provider. Prose restart. |
| `REA_MCP_DOCTOR_DEADLINE_MS` | `src/config/mcpStartupBudgets.ts` | 30000 | 1 | 2147483647 | MCP server startup | Startup phase of the MCP server itself. Prose. |
| `REA_MCP_FIRST_CATALOG_BUDGET_MS` | `src/config/mcpStartupBudgets.ts` | 20000 | 1 | 2147483647 | MCP server startup | Startup phase of the MCP server itself. Prose. |
| `REA_MCP_INITIALIZE_BUDGET_MS` | `src/config/mcpStartupBudgets.ts` | 15000 | 1 | 2147483647 | MCP server startup | Startup phase of the MCP server itself. Prose. |
| `REA_MCP_MAX_INPUT_ELEMENTS` | `src/config/mcpInputBudget.ts` | 1024 | 1 | `Number.MAX_SAFE_INTEGER` | MCP server creation | Argument admission for the server process. Prose restart. |
| `REA_MCP_MAX_RESPONSE_BYTES` | `src/config/mcpResponseBudget.ts` | 10485760 when unset | 10485760 | `Number.MAX_SAFE_INTEGER` | MCP server creation | Stdio receive buffer and the paired client buffer. Prose restart. |
| `REA_MCP_TOOLS_LIST_BUDGET_MS` | `src/config/mcpStartupBudgets.ts` | 5000 | 1 | 2147483647 | MCP server startup | Startup phase of the MCP server itself. Prose. |
| `REA_RECONNECT_TIMEOUT_SECONDS` | `src/hopper/BridgeLauncher.ts` | 30 | 30 | 30 | written into the Hopper bridge helper | Not a caller setting. It is `HOPPER_SHUTDOWN_TIMEOUT_MS / 1000`. |
| `REA_SWIFTC_COMPILE_TIMEOUT_MS` | `src/config/providerStartupTimeout.ts` | 120000 | 1 | 2147483647 | Darwin ownership-helper reader construction | Captured at process start for that helper compile. Prose restart. |
## Restart-only non-numeric settings
No floor or maximum. They are read from the launch environment (startup). None
of them use `rea_restart_required`, `restart_required`, or `restartRequired`;
recovery text tells the caller to change the launch environment. They stay
restart-only because they choose a command, a binary, a filesystem root, the
browser sandbox, secret patterns, the provider, or the shape of the MCP server
or logger.
| Name | Owner | Why |
| ------------------------------------ | -------------------------------------------------- | ------------------------------------------- |
| `REA_ADB_PATH` | `src/adb/AdbConfiguration.ts` | Caller-selected executable path. |
| `REA_AGENT_ENV_COMMAND` | `src/application/DesktopBuildDoctor.ts` | Caller-selected command. |
| `REA_ANALYSIS_PROVIDER` | `src/config/environment.ts` | Provider choice for the process. |
| `REA_APKTOOL_COMMAND` | `src/apktool/ApktoolConfiguration.ts` | Caller-selected command. |
| `REA_BINWALK_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_BROWSER_EXECUTABLE` | `src/application/CapabilityInventory.ts` | Caller-selected browser binary. |
| `REA_BROWSER_NO_SANDBOX` | `src/application/SetupRegistrationEnvironment.ts` | Browser sandbox policy. |
| `REA_CAPINFOS_COMMAND` | `src/pcap/PcapConfiguration.ts` | Caller-selected command. |
| `REA_CUTTER_BRIDGE_DIR` | `src/application/SetupRegistrationEnvironment.ts` | Filesystem root for the Cutter bridge. |
| `REA_DARWIN_MODULE_CACHE_PATH` | `src/application/SetupRegistrationEnvironment.ts` | Filesystem path for the Swift module cache. |
| `REA_DARWIN_SWIFTC_PATH` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected compiler path. |
| `REA_DCONF_COMMAND` | `src/desktop/DconfDatabaseReader.ts` | Caller-selected command. |
| `REA_DOCKER_COMMAND` | `src/application/DesktopBuildDoctor.ts` | Caller-selected command. |
| `REA_ELECTRON_CAPTURE_ARGV` | `src/browser/PlaywrightElectronActiveProvider.ts` | Caller-selected process arguments. |
| `REA_EVM_PRLIMIT_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_FIRMWARE_PRLIMIT_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_GDB_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected debugger binary. |
| `REA_GHIDRA_CHECKPOINT_DIR` | `src/config/environment.ts` | Filesystem root. |
| `REA_GHIDRA_COMPILER_SPEC_ID` | `src/config/environment.ts` | Provider analysis choice. |
| `REA_GHIDRA_DISABLED_ANALYZERS` | `src/config/environment.ts` | Provider analysis choice. |
| `REA_GHIDRA_LANGUAGE_ID` | `src/config/environment.ts` | Provider analysis choice. |
| `REA_GHIDRA_NATIVEAOT_JAR` | `src/config/environment.ts` | Caller-selected binary path. |
| `REA_GHIDRA_PROGRAM_NAME` | `src/config/environment.ts` | Provider project selection. |
| `REA_GHIDRA_PROJECT_PATH` | `src/config/environment.ts` | Filesystem root. |
| `REA_GHIDRA_SEED_FILE` | `src/config/environment.ts` | Filesystem path. |
| `REA_IDA_MCP_CONFIG` | `src/config/environment.ts` | Filesystem path of the IDA MCP config. |
| `REA_ILSPY_CMD_PATH` | `src/config/environment.ts` | Caller-selected executable path. |
| `REA_JADX_MCP_JAR` | `src/android/JadxConfiguration.ts` | Caller-selected binary path. |
| `REA_JAVASCRIPT_PRLIMIT_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_JEB_MCP_URL` | `src/config/environment.ts` | Provider endpoint. |
| `REA_KUNA_CACHE_DIR` | `src/kuna/KunaConfiguration.ts` | Filesystem root. |
| `REA_KUNA_COMMAND` | `src/kuna/KunaConfiguration.ts` | Caller-selected executable path. |
| `REA_LOG_LEVEL` | `src/config/environment.ts` | Logger configuration for the process. |
| `REA_MANAGED_LINUX_HOPPER_LAUNCHER` | `src/application/HopperLauncherDiscovery.ts` | Caller-selected launcher path. |
| `REA_MCP_INPUT_SCHEMA_PROFILE` | `src/config/environment.ts` | MCP server schema shape. |
| `REA_MCP_OUTPUT_SCHEMA_PROFILE` | `src/config/environment.ts` | MCP server schema shape. |
| `REA_MCP_TOOL_CATALOG_PROFILE` | `src/config/environment.ts` | MCP server catalog shape. |
| `REA_MCP_TOOL_FAMILIES` | `src/config/environment.ts` | Which tools the MCP server registers. |
| `REA_MITMDUMP_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_OBJDUMP_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_PWNDBG_GDB` | `src/native/pwndbg/PwndbgCoreContext.ts` | Caller-selected debugger binary. |
| `REA_PWNDBG_GDBINIT` | `src/native/pwndbg/PwndbgCoreContext.ts` | Caller-selected file path. |
| `REA_PWNDBG_VENV_PATH` | `src/native/pwndbg/PwndbgCoreContext.ts` | Filesystem root. |
| `REA_PWNTOOLS_PYTHON` | `src/native/pwntools/PwntoolsDecoder.ts` | Caller-selected interpreter path. |
| `REA_REFERENCE_SECRET_PATTERNS_JSON` | `src/config/environment.ts` | Secret-pattern policy. |
| `REA_RIZIN_COMMAND` | `src/rizin/RizinDebugSessionManager.ts` | Caller-selected debugger binary. |
| `REA_SIGROK_COMMAND` | `src/signals/SigrokRecordingProvider.ts` | Caller-selected command. |
| `REA_SIGROK_PRLIMIT_COMMAND` | `src/signals/SigrokRecordingProvider.ts` | Caller-selected command. |
| `REA_TSHARK_COMMAND` | `src/pcap/PcapConfiguration.ts` | Caller-selected command. |
| `REA_TSHARK_PRLIMIT_COMMAND` | `src/pcap/PcapNativeBound.ts` | Caller-selected command. |
| `REA_UNBLOB_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_WABT_BIN_DIRECTORY` | `src/wasm/WabtArtifactProvider.ts` | Filesystem root of the WABT tools. |
| `REA_WAKARU_COMMAND` | `src/application/SetupRegistrationEnvironment.ts` | Caller-selected command. |
| `REA_WORKFLOW_PROVIDER` | `src/application/DirectAnalysis.ts` | Provider choice. |
| `REA_XPROP_COMMAND` | `src/application/desktop/DesktopCaptureFailure.ts` | Caller-selected command. |
Names that look like `REA_` but are process protocol, not caller settings:
`REA_BUILD_COMMIT`, `REA_BUILD_COMPUTER_REA_ARCHIVE`, `REA_INSTALL_SCRIPT`,
`REA_OWNS_PROCESS_LIFETIME`, `REA_PAYLOAD`, `REA_PROCESS_RUN_ID`, `REA_RUN_ID`,
`REA_SOCKET`, `REA_STATE_FORMAT`, `REA_TARGET_PATH`, `REA_TOKEN`, bridge markers
such as `REA_COMMAND_COMPLETE_`, and the `ERR_REA_WINDOWS_NATIVE` code.
`GHIDRA_INSTALL_DIR`, `JAVA_HOME`, `HOPPER_LAUNCHER_PATH`, `HOPPER_TARGET_PATH`,
`HOPPER_TARGET_KIND`, and `HOPPER_LOADER_ARGS_JSON` are launch configuration
without the `REA_` prefix. Paths and the provider choice among them stay
restart-only for the same reason as the table above.
## Hardcoded ceilings left for follow-up
These are checked per call and tell the caller this release has no runtime
override. They are not `REA_` settings, so the numeric-limit check does not
list them. Each family has its own owner. Turning one into a setting needs that
owner's default, floor, maximum, and an error that carries the setting id,
value, source, and range. Format and protocol ceilings that the
[limits guide](limits-and-recovery.md) already calls fixed stay fixed. In
particular, JSON nesting stops at `MAX_JSON_DEPTH` (1000) and the debugger
control-reply budget stays separate from
`REA_DEBUGGER_MAX_COMMAND_OUTPUT_BYTES`.
| Family | Examples | Follow-up |
| ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| Native structural input ceilings | PE resources, imports, load config, certificates, exports, initializers, binary hardening, Windows shortcuts | One setting per ceiling, or a shared input budget only where the owners match. |
| Native plist | `MAX_NATIVE_PLIST_BYTES`, decode bytes, nesting depth | Separate from JSON depth. |
| SQLite inspection | concurrent inspections, input bytes, diagnostic bytes | Pool width stays restart-only if it sizes the worker set; byte budgets can become settings. |
| Producer command allowances | fixed command deadline and retained output where the text says there is no runtime override | Distinct from per-call `producer_limits`. |
| Hopper caller-scan retention and page allowances | `HOPPER_CALLER_SCAN_RETENTION_BYTES`, `HOPPER_CALLER_PAGE_ALLOWANCE_BYTES` | Explicitly not raised by the scan time budget. |
| Rizin frame protocol | `MAX_FRAME_BYTES`, queued frames | Protocol bounds, not caller budgets. |
| Debugger detach and control replies | `DETACH_TIMEOUT_MS`, `DEBUGGER_CONTROL_REPLY_BYTES` | Ownership deadlines, not the shared command deadline. |
| Web network capture nesting | `max_nesting_depth` already has a per-call parameter | The fixed depth maximum is the follow-up, not the parameter. |
## Static check
`npm run lint` runs `scripts/verify/rea-numeric-limits.mjs` after
dependencies are installed. `scripts/verify/architecture-guards.mjs` runs the
same check when `typescript6` is already installed; the pre-install CI step
cannot. The check scans `src` excluding tests,
keeps names that are numeric limits, and requires each name to be an
`environment` on an analysis setting or a registry entry with a reason.
`policy: "per-call"` is temporary. The migration that moves the limit deletes
the entry and rejects any new `per-call` policy so the durable exceptions are
only `restart-only`.
+1 -1
View File
@@ -148,7 +148,7 @@
"build:termux:run": "npm run compile:termux && npm run artifacts:skills",
"format": "oxfmt .",
"format:check": "oxfmt --check .",
"lint": "node scripts/ci/static-check-report.mjs lint && npm run verify:module-boundaries && node scripts/verify/rea-numeric-limits.mjs",
"lint": "node scripts/ci/static-check-report.mjs lint && npm run verify:module-boundaries",
"lint:details": "oxlint && npm run verify:module-boundaries",
"lint:fix": "oxlint --fix && npm run verify:module-boundaries",
"knip": "knip --cache",
-21
View File
@@ -1,5 +1,3 @@
import { spawnSync } from "node:child_process";
import { existsSync } from "node:fs";
import { readFile, readdir } from "node:fs/promises";
import { join, relative, sep } from "node:path";
import { fileURLToPath } from "node:url";
@@ -195,23 +193,4 @@ if (failures.length > 0) {
console.error(`Architecture guards failed:\n\n${failures.join("\n\n")}\n`);
process.exit(1);
}
// CI runs this file before `npm ci`. The numeric-limit parser needs
// typescript6, so that check waits until dependencies exist. `npm run lint`
// runs it after install.
const typescript6 = new URL(
"../../node_modules/typescript6/package.json",
import.meta.url,
);
if (!existsSync(typescript6)) {
console.log(
"Numeric REA_ limit check waits until dependencies are installed.",
);
} else {
const numericLimits = spawnSync(
process.execPath,
[fileURLToPath(new URL("./rea-numeric-limits.mjs", import.meta.url))],
{ stdio: "inherit" },
);
if (numericLimits.status !== 0) process.exit(numericLimits.status ?? 1);
}
console.log(`Architecture guards passed across ${files.length} source files.`);
-23
View File
@@ -1,23 +0,0 @@
/** Relative path of the registry. Its text is a declaration, not a producer. */
export const NUMERIC_LIMIT_DECLARATION: "src/domain/numericLimitRegistry.ts";
/** Limits that must become analysis settings and cannot be registry exceptions. */
export const REQUIRED_RUNTIME_LIMITS: readonly [
"REA_NTFS_MFT_MAX_FILE_BYTES",
"REA_REGF_HIVE_MAX_FILE_BYTES",
"REA_WEB_BUNDLE_ENDPOINT_LITERAL_CANDIDATES",
];
/** True for a numeric `REA_` limit, including the documented endpoint-candidate count. */
export function isNumericLimit(name: string): boolean;
/** Producer paths for numeric limits. Registry text, comments, and allowlists are omitted. */
export function discoverNumericLimitUses(
files: readonly { readonly path: string; readonly source: string }[],
): Map<string, string>;
/** Why a tracked limit is not an analysis setting, when it is present. */
export function requiredRuntimeLimitFailure(
name: string,
presence: { readonly inRuntime: boolean; readonly inRegistry: boolean },
): string | undefined;
-319
View File
@@ -1,319 +0,0 @@
import { readFile, readdir } from "node:fs/promises";
import { join, relative, resolve, sep } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import ts from "typescript6";
// Every numeric REA_ limit under src must be an analysis setting or a
// documented exception with a reason. restart-only is durable. per-call is a
// launch-captured limit whose runtime migration deletes the registry entry.
// The three limits moving on #9868, #9865, and #9836 are not in this tree;
// once their names appear here they must join ANALYSIS_SETTINGS.
const root = fileURLToPath(new URL("../../", import.meta.url));
const definitionsPath = join(root, "src/domain/analysisSettingDefinitions.ts");
const registryPath = join(root, "src/domain/numericLimitRegistry.ts");
const listFiles = async (dir) => {
const output = [];
for (const entry of await readdir(dir, { withFileTypes: true })) {
const path = join(dir, entry.name);
if (entry.isDirectory()) {
if (entry.name === "node_modules" || entry.name === "dist") continue;
output.push(...(await listFiles(path)));
} else if (
entry.name.endsWith(".ts") &&
!entry.name.endsWith(".test.ts") &&
!entry.name.endsWith(".fixture.ts")
)
output.push(path);
}
return output;
};
/**
* Count-shaped limits whose names do not use a byte, time, or maximum suffix.
* `REA_WEB_BUNDLE_ENDPOINT_LITERAL_CANDIDATES` is the runtime setting on #9836.
*/
const DOCUMENTED_COUNT_LIMITS = new Set([
"REA_WEB_BUNDLE_ENDPOINT_LITERAL_CANDIDATES",
]);
/** Registry text is a declaration, not a producer that uses the limit. */
export const NUMERIC_LIMIT_DECLARATION = "src/domain/numericLimitRegistry.ts";
/**
* These limits are moving onto runtime analysis settings on their own PRs.
* A producer or a registry entry cannot satisfy that requirement.
*/
export const REQUIRED_RUNTIME_LIMITS = [
"REA_NTFS_MFT_MAX_FILE_BYTES",
"REA_REGF_HIVE_MAX_FILE_BYTES",
"REA_WEB_BUNDLE_ENDPOINT_LITERAL_CANDIDATES",
];
const unwrapExpression = (node) => {
let current = node;
while (
ts.isAsExpression(current) ||
ts.isSatisfiesExpression(current) ||
ts.isParenthesizedExpression(current) ||
ts.isTypeAssertionExpression(current) ||
ts.isNonNullExpression(current)
)
current = current.expression;
return current;
};
const literalText = (node) =>
ts.isStringLiteral(node) || ts.isNoSubstitutionTemplateLiteral(node)
? node.text
: undefined;
/** Same-file const strings and `setting` fields, so `environment[name]` still resolves. */
const stringBindings = (sourceFile) => {
const bindings = new Map();
const visit = (node) => {
if (
ts.isVariableDeclaration(node) &&
ts.isIdentifier(node.name) &&
node.initializer !== undefined
) {
const initializer = unwrapExpression(node.initializer);
const literal = literalText(initializer);
if (literal !== undefined) bindings.set(node.name.text, literal);
if (ts.isObjectLiteralExpression(initializer)) {
for (const property of initializer.properties) {
if (
!ts.isPropertyAssignment(property) ||
(!ts.isIdentifier(property.name) &&
!ts.isStringLiteral(property.name))
)
continue;
const value = literalText(unwrapExpression(property.initializer));
if (value !== undefined)
bindings.set(`${node.name.text}.${property.name.text}`, value);
}
}
}
ts.forEachChild(node, visit);
};
visit(sourceFile);
return bindings;
};
const boundLimitName = (node, bindings) => {
const expression = unwrapExpression(node);
const literal = literalText(expression);
if (literal !== undefined) return literal;
if (ts.isIdentifier(expression)) return bindings.get(expression.text);
if (
ts.isPropertyAccessExpression(expression) &&
ts.isIdentifier(expression.expression) &&
(ts.isIdentifier(expression.name) || ts.isStringLiteral(expression.name))
)
return bindings.get(
`${expression.expression.text}.${expression.name.text}`,
);
return undefined;
};
export const isNumericLimit = (name) =>
DOCUMENTED_COUNT_LIMITS.has(name) ||
/(?:_BYTES|_MS|_MIB|_SECONDS|_POLLS|_JOBS|_ELEMENTS|_COUNT|_SESSIONS|_COMMANDS)$/u.test(
name,
) ||
/_MAX_/u.test(name) ||
/_(?:TIMEOUT|DEADLINE|BUDGET)(?:_|$)/u.test(name) ||
/_HEAP(?:_|$)/u.test(name);
const limitName = (text) =>
text !== undefined && /^REA_[A-Z0-9_]+$/u.test(text) && isNumericLimit(text)
? text
: undefined;
const remember = (discovered, path, name) => {
const limit = limitName(name);
if (limit === undefined || discovered.has(limit)) return;
discovered.set(limit, path);
};
/**
* Producers are syntax, not text. A read is a property or element access,
* including a const string used as the element, a string argument to a call,
* an object-binding key (rename and default included), an `in` check, or an
* object key that a spread can copy. Comments and array literals are not reads.
*/
export const discoverNumericLimitUses = (files) => {
const discovered = new Map();
for (const file of files) {
const relativePath = file.path.split(sep).join("/");
if (relativePath === NUMERIC_LIMIT_DECLARATION) continue;
const sourceFile = ts.createSourceFile(
relativePath,
file.source,
ts.ScriptTarget.Latest,
true,
ts.ScriptKind.TS,
);
const bindings = stringBindings(sourceFile);
const visit = (node) => {
if (
ts.isPropertyAccessExpression(node) ||
ts.isPropertyAccessChain?.(node)
)
remember(discovered, relativePath, node.name.text);
if (ts.isElementAccessExpression(node))
remember(
discovered,
relativePath,
boundLimitName(node.argumentExpression, bindings),
);
if (ts.isCallExpression(node) || ts.isNewExpression(node)) {
for (const argument of node.arguments ?? [])
remember(
discovered,
relativePath,
boundLimitName(argument, bindings),
);
}
if (
ts.isBinaryExpression(node) &&
node.operatorToken.kind === ts.SyntaxKind.InKeyword
)
remember(discovered, relativePath, boundLimitName(node.left, bindings));
if (
ts.isBindingElement(node) &&
node.dotDotDotToken === undefined &&
ts.isObjectBindingPattern(node.parent)
) {
const key = node.propertyName ?? node.name;
if (
ts.isIdentifier(key) ||
ts.isStringLiteral(key) ||
ts.isNumericLiteral(key)
)
remember(discovered, relativePath, key.text);
}
if (
(ts.isPropertyAssignment(node) ||
ts.isShorthandPropertyAssignment(node)) &&
(ts.isIdentifier(node.name) ||
ts.isStringLiteral(node.name) ||
ts.isNumericLiteral(node.name))
)
remember(discovered, relativePath, node.name.text);
if (ts.isVariableDeclaration(node) && node.initializer !== undefined)
remember(
discovered,
relativePath,
literalText(unwrapExpression(node.initializer)),
);
if (ts.isPropertyAssignment(node))
remember(
discovered,
relativePath,
literalText(unwrapExpression(node.initializer)),
);
if (
ts.isTemplateExpression(node) ||
ts.isNoSubstitutionTemplateLiteral(node)
) {
const chunks = ts.isNoSubstitutionTemplateLiteral(node)
? [node.text]
: [
node.head.text,
...node.templateSpans.map((span) => span.literal.text),
];
for (const chunk of chunks)
for (const token of chunk.match(/REA_[A-Z0-9_]+/gu) ?? [])
remember(discovered, relativePath, token);
}
ts.forEachChild(node, visit);
};
visit(sourceFile);
}
return discovered;
};
/** Tracked limits join ANALYSIS_SETTINGS. The exception registry cannot classify them. */
export const requiredRuntimeLimitFailure = (
name,
{ inRuntime, inRegistry },
) => {
if (!REQUIRED_RUNTIME_LIMITS.includes(name)) return undefined;
if (inRuntime && !inRegistry) return undefined;
return `${name} must join ANALYSIS_SETTINGS; numericLimitRegistry cannot classify it`;
};
const namesIn = (source, pattern) =>
new Set([...source.matchAll(pattern)].map((match) => match[1]));
const direct =
process.argv[1] !== undefined &&
import.meta.url === pathToFileURL(resolve(process.argv[1])).href;
if (direct) {
const failures = [];
const definitions = await readFile(definitionsPath, "utf8");
const registry = await readFile(registryPath, "utf8");
const runtime = namesIn(definitions, /environment:\s*"(REA_[A-Z0-9_]+)"/gu);
const entries = [
...registry.matchAll(
/name:\s*"(REA_[A-Z0-9_]+)"\s*,\s*policy:\s*"(restart-only|per-call)"\s*,\s*read:\s*"(startup|per-call)"\s*,\s*reason:\s*"((?:\\.|[^"\\])*)"/gu,
),
];
const listed = new Map();
for (const match of entries) {
const [, name, policy, read, reason] = match;
if (listed.has(name)) failures.push(`${name} is listed twice`);
listed.set(name, { policy, read, reason });
if (reason.trim().length < 40)
failures.push(`${name} needs a reason that states why`);
if (runtime.has(name))
failures.push(
`${name} is both a runtime setting and on the exception list`,
);
if (policy === "per-call" && read !== "per-call")
failures.push(`${name} is per-call policy but is not read per call`);
if (policy === "restart-only" && read !== "startup")
failures.push(`${name} is restart-only but is not read at startup`);
}
const discovered = discoverNumericLimitUses(
await Promise.all(
(await listFiles(join(root, "src"))).map(async (path) => ({
path: relative(root, path),
source: await readFile(path, "utf8"),
})),
),
);
for (const [name, file] of [...discovered].sort(([left], [right]) =>
left < right ? -1 : left > right ? 1 : 0,
)) {
if (!runtime.has(name) && !listed.has(name))
failures.push(
`${name} in ${file} is neither an analysis setting nor docs/design/rea-limit-inventory.md via src/domain/numericLimitRegistry.ts`,
);
}
for (const name of listed.keys()) {
if (!discovered.has(name))
failures.push(`${name} is registered but no longer appears in src`);
}
const present = new Set([...discovered.keys(), ...listed.keys(), ...runtime]);
for (const name of REQUIRED_RUNTIME_LIMITS) {
if (!present.has(name)) continue;
const failure = requiredRuntimeLimitFailure(name, {
inRuntime: runtime.has(name),
inRegistry: listed.has(name),
});
if (failure !== undefined) failures.push(failure);
}
if (failures.length > 0) {
console.error(`Numeric REA_ limit check failed:\n${failures.join("\n")}`);
process.exit(1);
}
console.log(
`Numeric REA_ limits: ${runtime.size} runtime, ${[...listed.values()].filter((entry) => entry.policy === "restart-only").length} restart-only, ${[...listed.values()].filter((entry) => entry.policy === "per-call").length} per-call pending migration, ${discovered.size} discovered.`,
);
}
@@ -20,6 +20,11 @@ it("reads a Python 3.9 reference-flagged PYZ table", () => {
]);
});
it("reads marshal one-byte string tags as Latin-1", () => {
for (const hex of ["6101000000e9", "4101000000e9", "7a01e9", "5a01e9"])
expect(readMarshalSubset(bytes(hex))).toEqual({ ok: true, value: "é" });
});
it("decodes CPython marshal Unicode with surrogatepass", () => {
expect(
readMarshalSubset(
+2 -8
View File
@@ -168,14 +168,8 @@ const readAsciiBytes = (
): Result<string, PyInstallerFailure> => {
const bytes = readBytes(cursor, length);
if (!bytes.ok) return bytes;
if (bytes.value.some((byte) => byte > 0x7f))
return err(
pyInstallerFailure(
"format",
"PYZ marshal ASCII string contains a non-ASCII byte",
),
);
return ok(bytes.value.toString("ascii"));
// These tags use CPython's one-byte Unicode constructor, not an ASCII check.
return ok(bytes.value.toString("latin1"));
};
const readUtf8 = (
-36
View File
@@ -1,36 +0,0 @@
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { expect, it } from "vitest";
import { ANALYSIS_SETTINGS } from "./analysisSettingDefinitions.js";
import { NUMERIC_LIMIT_REGISTRY } from "./numericLimitRegistry.js";
const inventory = readFileSync(
join(import.meta.dirname, "../../docs/design/rea-limit-inventory.md"),
"utf8",
);
it("gives every numeric exception a reason and a matching inventory row", () => {
const names = NUMERIC_LIMIT_REGISTRY.map((entry) => entry.name);
expect(new Set(names).size).toBe(names.length);
const runtimeEnvironments = new Set<string>(
Object.values(ANALYSIS_SETTINGS).flatMap((definition) =>
"environment" in definition && definition.environment !== undefined
? [definition.environment]
: [],
),
);
for (const entry of NUMERIC_LIMIT_REGISTRY) {
expect(entry.reason.trim().length).toBeGreaterThanOrEqual(40);
expect(entry.read).toBe(
entry.policy === "per-call" ? "per-call" : "startup",
);
expect(runtimeEnvironments.has(entry.name)).toBe(false);
expect(inventory).toContain(`\`${entry.name}\``);
}
expect(
NUMERIC_LIMIT_REGISTRY.filter((entry) => entry.policy === "restart-only"),
).toHaveLength(15);
expect(
NUMERIC_LIMIT_REGISTRY.filter((entry) => entry.policy === "per-call"),
).toHaveLength(11);
});
-203
View File
@@ -1,203 +0,0 @@
/**
* Numeric `REA_` limits that are not runtime analysis settings.
*
* The static check in `scripts/verify/rea-numeric-limits.mjs` fails when a
* numeric `REA_` limit in `src` is absent from both this list and
* `ANALYSIS_SETTINGS`. `restart-only` is the durable exception. `per-call`
* names a limit still read from the launch environment on each use; the
* runtime migration deletes that entry instead of leaving it here.
*
* Narrative bounds, owners, and the non-numeric settings live in
* `docs/design/rea-limit-inventory.md`.
*/
export interface NumericLimitRegistryEntry {
readonly name: string;
readonly policy: "restart-only" | "per-call";
readonly read: "startup" | "per-call";
readonly reason: string;
}
export const NUMERIC_LIMIT_REGISTRY = [
{
name: "REA_BRIDGE_MAX_REQUEST_BYTES",
policy: "per-call",
read: "per-call",
reason:
"The Hopper, Ghidra, and Cutter clients measure each request against this ceiling. The value is still taken from the launch environment, so a running process does not see a new value until the runtime migration removes this entry.",
},
{
name: "REA_DEBUGGER_COMMAND_DEADLINE_MS",
policy: "per-call",
read: "per-call",
reason:
"GDB and Rizin apply this deadline to each debugger command. The session manager currently copies it at construction from the launch environment; the runtime migration reads the accepted operation snapshot instead.",
},
{
name: "REA_DEBUGGER_MAX_COMMAND_OUTPUT_BYTES",
policy: "per-call",
read: "per-call",
reason:
"Each debugger command retains output up to this ceiling. The session manager currently copies it at construction from the launch environment; the runtime migration reads the accepted operation snapshot instead.",
},
{
name: "REA_DEBUGGER_MAX_PENDING_COMMANDS",
policy: "restart-only",
read: "startup",
reason:
"Caps the pending-command queue created with the debugger session manager. It is pool capacity for that manager, not a budget checked independently on each command.",
},
{
name: "REA_DEBUGGER_MAX_SESSIONS",
policy: "restart-only",
read: "startup",
reason:
"Caps how many debugger sessions the session manager created at startup will admit. Changing the pool size of a manager that already exists requires a new REA process.",
},
{
name: "REA_HOPPER_CALLER_SCAN_BUDGET_MS",
policy: "per-call",
read: "per-call",
reason:
"Forward caller reconciliation reads this budget for each scan. It is still resolved from the launch environment; the runtime migration reads the accepted operation snapshot.",
},
{
name: "REA_HOPPER_DEMO_DIALOG_TIMEOUT_MS",
policy: "restart-only",
read: "startup",
reason:
"The Linux private-display adapter reads this watch from the process environment when that helper starts. A running adapter does not observe a value stored only in the REA process.",
},
{
name: "REA_JADX_ACTIVE_PROCESSOR_COUNT",
policy: "restart-only",
read: "startup",
reason:
"Passed to the JVM as -XX:ActiveProcessorCount when the JADX worker process starts. A running JVM does not adopt a new processor count.",
},
{
name: "REA_JADX_HEAP_MIB",
policy: "restart-only",
read: "startup",
reason:
"Passed to the JVM as -Xmx when the JADX worker process starts. A running JVM does not adopt a new heap size.",
},
{
name: "REA_JEB_MCP_TIMEOUT_MS",
policy: "per-call",
read: "per-call",
reason:
"Bounds each JEB tools/call and catalog discovery. The connection currently captures the launch value when it is created; the runtime migration reads the accepted snapshot for each request.",
},
{
name: "REA_KUNA_CACHE_MAX_BYTES",
policy: "restart-only",
read: "startup",
reason:
"Sizes the Kuna project cache when that cache is constructed for the provider. An existing cache does not resize from a later value.",
},
{
name: "REA_KUNA_JOBS",
policy: "restart-only",
read: "startup",
reason:
"Selects the Kuna worker-pool width when the provider builds its run slots. The pool is not resized for a process that is already running.",
},
{
name: "REA_KUNA_PROJECT_TIMEOUT_MS",
policy: "per-call",
read: "per-call",
reason:
"Bounds a Kuna project export and the whole-program scans that use that deadline. The configuration object still comes from the launch environment; the runtime migration reads the accepted snapshot at each use.",
},
{
name: "REA_KUNA_TIMEOUT_MS",
policy: "per-call",
read: "per-call",
reason:
"Bounds one Kuna function decompile. The configuration object still comes from the launch environment; the runtime migration reads the accepted snapshot at each use.",
},
{
name: "REA_MCP_DOCTOR_DEADLINE_MS",
policy: "restart-only",
read: "startup",
reason:
"Bounds the MCP doctor phase while the server is starting. It shapes server startup, not a later analysis call.",
},
{
name: "REA_MCP_FIRST_CATALOG_BUDGET_MS",
policy: "restart-only",
read: "startup",
reason:
"Bounds the first MCP catalog phase while the server is starting. It shapes server startup, not a later analysis call.",
},
{
name: "REA_MCP_INITIALIZE_BUDGET_MS",
policy: "restart-only",
read: "startup",
reason:
"Bounds the MCP initialize phase while the server is starting. It shapes server startup, not a later analysis call.",
},
{
name: "REA_MCP_MAX_INPUT_ELEMENTS",
policy: "restart-only",
read: "startup",
reason:
"Sizes argument admission for the MCP server created at process start. A running server does not rebuild that parser for a new allowance.",
},
{
name: "REA_MCP_MAX_RESPONSE_BYTES",
policy: "restart-only",
read: "startup",
reason:
"Sizes the MCP stdio receive buffer when the server process starts. A running transport and the paired client buffer do not adopt a new ceiling.",
},
{
name: "REA_MCP_TOOLS_LIST_BUDGET_MS",
policy: "restart-only",
read: "startup",
reason:
"Bounds the MCP tools/list phase while the server is starting. It shapes server startup, not a later analysis call.",
},
{
name: "REA_PWNTOOLS_MAX_INPUT_BYTES",
policy: "per-call",
read: "per-call",
reason:
"Default artifact-size admission for a pwntools decode. A per-call producer_limits.input_bytes override already exists; the launch default is still read from the environment and the runtime migration moves that default into the snapshot.",
},
{
name: "REA_RECONNECT_TIMEOUT_SECONDS",
policy: "restart-only",
read: "startup",
reason:
"Fixed 30-second value written into the Hopper bridge helper from HOPPER_SHUTDOWN_TIMEOUT_MS. It is not a caller-selected launch setting.",
},
{
name: "REA_SWIFTC_COMPILE_TIMEOUT_MS",
policy: "restart-only",
read: "startup",
reason:
"Captured when the Darwin ownership-helper reader is constructed at process start. That helper compile uses the captured deadline, so a later value needs a new REA process.",
},
{
name: "REA_WINDOWS_CHANGE_TIME_DRIFT_MS",
policy: "per-call",
read: "per-call",
reason:
"Each Windows ChangeTime observation compares the file against this drift window. The observation still parses the launch environment; the runtime migration reads the accepted operation snapshot.",
},
{
name: "REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS",
policy: "per-call",
read: "per-call",
reason:
"Each Windows ChangeTime settle loop stops after this many polls. The observation still parses the launch environment; the runtime migration reads the accepted operation snapshot.",
},
{
name: "REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS",
policy: "per-call",
read: "per-call",
reason:
"Each Windows ChangeTime settle loop waits this long between reads. The observation still parses the launch environment; the runtime migration reads the accepted operation snapshot.",
},
] as const satisfies readonly NumericLimitRegistryEntry[];
-20
View File
@@ -1,4 +1,3 @@
import { readFile } from "node:fs/promises";
import { describe, expect, it } from "vitest";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -1120,25 +1119,6 @@ describe("Ghidra native instruction decoder context for ARM TMode and MIPS ISA_M
});
});
// Kept on the file rather than in its own suite. The full source run collected
// that suite with no tests and failed the file before this assertion could run.
it("binds TMode to ARM and ISA_MODE to MIPS in the Ghidra bridge script", async () => {
const bridgePath = new URL(
"../../bridge/ghidra/ReaGhidraBridge.java",
import.meta.url,
);
const content = await readFile(bridgePath, "utf8");
expect(content).toContain(
'currentProgram.getLanguageID().getIdAsString().startsWith("ARM:")',
);
expect(content).toContain(
'currentProgram.getLanguageID().getIdAsString().startsWith("MIPS:")',
);
expect(content).toContain('"TMode"');
expect(content).toContain('"ISA_MODE"');
expect(content).toContain('"ghidra-program-context"');
});
describe("Ghidra dossier nested boundary composition", () => {
const identityLocations = [
"caller",
+54
View File
@@ -407,6 +407,30 @@ it("charges long digits to the string budget and keeps large previews bounded",
);
});
it("reads marshal one-byte string tags as Latin-1 with Unicode identity", () => {
for (const tag of [0x61, 0x41, 0x7a, 0x5a]) {
const encoded = Buffer.concat([
Buffer.from([tag]),
tag === 0x61 || tag === 0x41 ? u32(1) : Buffer.from([1]),
Buffer.from([0xe9]),
]);
const sameUnicode = marshalUnicode(Buffer.from("é", "utf8"));
const set = Buffer.concat([
Buffer.from([0x3c]),
u32(2),
encoded,
sameUnicode,
]);
const report = root(parse(withConsts([encoded, set])));
expect(report.consts[0]).toMatchObject({
kind: "str",
value: "é",
bytes: 1,
});
expect(report.consts[1]).toEqual({ kind: "set", length: 1 });
}
});
it("decodes marshal strings with CPython surrogatepass rules", () => {
const report = parse(
withConsts([marshalUnicode(Buffer.from("efbbbf73eda080efbbbf", "hex"))]),
@@ -1534,6 +1558,36 @@ it("hashes a long TYPE_REF tuple chain without recursing", () => {
expect(pythonPycSchema.safeParse(report).success).toBe(true);
});
it("hashes shallow TYPE_REF code chains without using the JavaScript stack", () => {
const links = 3000;
const codes = Array.from({ length: links }, (_, index) =>
marshalCode("py311", {
flagged: true,
name: "f",
consts: index === 0 ? [Buffer.from([0x4e])] : [marshalRef(index - 1)],
}),
);
const bytes = Buffer.concat([
pythonPycHeader(3531),
marshalCode("py311", {
constsObject: Buffer.concat([
Buffer.from([0x28]),
u32(links + 1),
...codes,
Buffer.from([0x3c]),
u32(1),
marshalRef(links - 1),
]),
}),
]);
const report = parseWith(bytes, { max_objects: 65_536 });
expect(root(report).consts.at(-1)).toEqual({ kind: "set", length: 1 });
expect(report.coverage).toMatchObject({
status: "complete",
code_objects: links + 1,
});
});
it("rejects code CPython treats as corrupt instead of complete coverage", () => {
const file = (
layout: "py37" | "py38" | "py311",
+83 -17
View File
@@ -332,7 +332,7 @@ class MarshalReader {
readonly #codeKeys = new WeakMap<CodeNode, string>();
readonly #codeIdentities = new WeakMap<CodeNode, number>();
readonly #codeEquality = new WeakMap<CodeNode, CodeEquality>();
/** Python hashability per code object. False while that check is in progress. */
/** Completed Python hashability checks per code object. */
readonly #codeHashable = new WeakMap<CodeNode, boolean>();
/** Tuples whose items are still being read; CPython cannot hash them yet. */
readonly #readingTuples = new WeakSet<MarshalValue>();
@@ -436,13 +436,19 @@ class MarshalReader {
};
case TYPE_INTERNED:
case TYPE_UNICODE:
return this.utf8(this.nonNegativeInt32("Marshal string length"), false);
return this.string(
this.nonNegativeInt32("Marshal string length"),
"utf8",
);
case TYPE_ASCII:
case TYPE_ASCII_INTERNED:
return this.utf8(this.nonNegativeInt32("Marshal string length"), true);
return this.string(
this.nonNegativeInt32("Marshal string length"),
"latin1",
);
case TYPE_SHORT_ASCII:
case TYPE_SHORT_ASCII_INTERNED:
return this.utf8(this.u8(), true);
return this.string(this.u8(), "latin1");
case TYPE_TUPLE:
return this.sequenceValue(
"tuple",
@@ -1309,13 +1315,70 @@ class MarshalReader {
#isCodeHashable(node: CodeNode): boolean {
const known = this.#codeHashable.get(node);
if (known !== undefined) return known;
this.#codeHashable.set(node, false);
const material =
this.#codeEquality.get(node) ??
malformed("Code object is missing equality material.");
const hashable = this.#equalityKey(material.constsTuple, "hash") !== null;
this.#codeHashable.set(node, hashable);
return hashable;
const frames: { value: MarshalValue; finish: boolean }[] = [
{ value: { kind: "code", value: node }, finish: false },
];
const gray = new Set<MarshalValue | CodeNode>();
const complete = new Set<MarshalValue>();
const reject = (): false => {
this.#codeHashable.set(node, false);
return false;
};
while (frames.length > 0) {
this.#takeKeyStep();
const frame = frames.pop()!;
const value = frame.value;
const identity = value.kind === "code" ? value.value : value;
if (frame.finish) {
gray.delete(identity);
if (value.kind === "code") {
const material =
this.#codeEquality.get(value.value) ??
malformed("Code object is missing equality material.");
// Child code checks are now complete, so the existing tuple key
// walker preserves its equality and expansion budgets without recursion.
const hashable =
this.#equalityKey(material.constsTuple, "hash") !== null;
this.#codeHashable.set(value.value, hashable);
if (!hashable) return reject();
} else complete.add(value);
continue;
}
if (value.kind === "code") {
const cached = this.#codeHashable.get(value.value);
if (cached === false) return reject();
if (cached === true) continue;
} else {
const cached = this.#keyCache.get(value);
if (cached === null) return reject();
if (cached !== undefined || complete.has(value)) continue;
}
if (
gray.has(identity) ||
this.#readingTuples.has(value) ||
value.kind === "list" ||
value.kind === "set" ||
value.kind === "dict" ||
value.kind === "null"
)
return reject();
const children =
value.kind === "code"
? [
(
this.#codeEquality.get(value.value) ??
malformed("Code object is missing equality material.")
).constsTuple,
]
: value.kind === "tuple" || value.kind === "frozenset"
? value.items
: [];
gray.add(identity);
frames.push({ value, finish: true });
for (let index = children.length - 1; index >= 0; index -= 1)
frames.push({ value: children[index]!, finish: false });
}
return true;
}
#codeIdentity(node: CodeNode): number {
@@ -1492,17 +1555,20 @@ class MarshalReader {
);
}
utf8(length: number, ascii: boolean): MarshalValue {
string(length: number, encoding: "latin1" | "utf8"): MarshalValue {
const bytes = this.blob(length);
if (ascii) {
if (bytes.some((byte) => byte > 0x7f))
malformed("Marshal ASCII string contains a non-ASCII byte.");
if (encoding === "latin1") {
// CPython reads the ASCII-tagged forms as one-byte Unicode, even when
// high bytes appear. Identity must agree with the UTF-8 string form.
const value = bytes.toString("latin1");
return {
kind: "str",
value: bytes.toString("latin1"),
value,
previewEnd: Math.min(length, PYTHON_PYC_CONST_PREVIEW_LIMIT),
bytes: length,
raw: bytes,
raw: bytes.some((byte) => byte > 0x7f)
? Buffer.from(value, "utf8")
: bytes,
};
}
const decoded = decodeMarshalUtf8(bytes, PYTHON_PYC_CONST_PREVIEW_LIMIT);
@@ -36,9 +36,7 @@ it("preserves full advertised schemas and handler meaning when presentation orde
REA_MCP_INPUT_SCHEMA_PROFILE: "full",
REA_MCP_OUTPUT_SCHEMA_PROFILE: "full",
},
// The child advertises the full catalog and calls every browser, Electron,
// and JavaScript example. A busy runner exceeded 20s and was killed.
timeout: 60_000,
timeout: 20_000,
maxBuffer: 16 * 1024 * 1024,
});
const result = resultSchema.parse(JSON.parse(stdout));
-145
View File
@@ -1,145 +0,0 @@
import { expect, it } from "vitest";
import {
NUMERIC_LIMIT_DECLARATION,
discoverNumericLimitUses,
isNumericLimit,
requiredRuntimeLimitFailure,
} from "../../scripts/verify/rea-numeric-limits.mjs";
const ENDPOINT_CANDIDATES = "REA_WEB_BUNDLE_ENDPOINT_LITERAL_CANDIDATES";
it("recognizes the documented endpoint-candidate limit as numeric", () => {
expect(isNumericLimit(ENDPOINT_CANDIDATES)).toBe(true);
const discovered = discoverNumericLimitUses([
{
path: "src/domain/webBundleAnalysis.ts",
source: `const candidates = process.env.${ENDPOINT_CANDIDATES};`,
},
]);
expect(discovered.get(ENDPOINT_CANDIDATES)).toBe(
"src/domain/webBundleAnalysis.ts",
);
});
it("does not treat a registry declaration as a producer", () => {
const name = "REA_REMOVED_MAX_BYTES";
const discovered = discoverNumericLimitUses([
{
path: NUMERIC_LIMIT_DECLARATION,
source: `name: "${name}",`,
},
{
path: "src/config/removedLimit.ts",
source: `const value = process.env.${name};`,
},
]);
expect(discovered.get(name)).toBe("src/config/removedLimit.ts");
expect(
discoverNumericLimitUses([
{ path: NUMERIC_LIMIT_DECLARATION, source: `name: "${name}",` },
]).has(name),
).toBe(false);
});
it("keeps a helper-call argument that is not the first argument", () => {
const name = "REA_NEW_MAX_BYTES";
const discovered = discoverNumericLimitUses([
{
path: "src/config/newLimit.ts",
source: `const setting = "${name}";\nconst value = read(environment, setting);`,
},
{
path: "src/config/literalLimit.ts",
source: `const value = read(environment, "${name}");`,
},
]);
expect(discovered.get(name)).toBe("src/config/newLimit.ts");
});
it("does not treat prose that mentions a limit name as a producer", () => {
const name = "REA_NEW_MAX_BYTES";
const discovered = discoverNumericLimitUses([
{
path: "src/domain/native/nativeDeadlines.ts",
source: `const guidance = "input_bytes overrides ${name}; otherwise that launch default applies.";`,
},
]);
expect(discovered.has(name)).toBe(false);
});
it("keeps a destructured environment key, including a rename and a default", () => {
const name = "REA_NEW_MAX_BYTES";
const discovered = discoverNumericLimitUses([
{
path: "src/config/newLimit.ts",
source: `const { ${name}: renamed = 1 } = process.env;\nconst present = "${name}" in environment;\nconst owned = Object.hasOwn(environment, "${name}");\nconst source = { ${name}: renamed };\nconst copy = { ...source };`,
},
]);
expect(discovered.get(name)).toBe("src/config/newLimit.ts");
});
it("does not invent a limit from a spread of the environment object", () => {
const discovered = discoverNumericLimitUses([
{
path: "src/config/newLimit.ts",
source: "const copy = { ...process.env, ...environment };",
},
]);
expect(discovered.size).toBe(0);
});
it("keeps a bracket-access read as a producer", () => {
const name = "REA_NEW_MAX_BYTES";
const discovered = discoverNumericLimitUses([
{
path: "src/config/newLimit.ts",
source: `const selected = environment["${name}"] ?? environment?.[ "${name}" ];`,
},
]);
expect(discovered.get(name)).toBe("src/config/newLimit.ts");
});
it("does not treat a registration allowlist or a comment as a producer", () => {
const discovered = discoverNumericLimitUses([
{
path: NUMERIC_LIMIT_DECLARATION,
source: `name: "${ENDPOINT_CANDIDATES}",`,
},
{
path: "src/application/SetupRegistrationEnvironment.ts",
source: `const SELECTED_REGISTRATION_SETTINGS = ["${ENDPOINT_CANDIDATES}"] as const;`,
},
{
path: "src/config/removedLimit.ts",
source: `// ${ENDPOINT_CANDIDATES} used to be read here\nconst unrelated = 1;\n`,
},
]);
expect(discovered.has(ENDPOINT_CANDIDATES)).toBe(false);
});
it("requires the tracked endpoint-candidate limit to be an analysis setting", () => {
expect(
requiredRuntimeLimitFailure(ENDPOINT_CANDIDATES, {
inRuntime: false,
inRegistry: true,
}),
).toMatch(/ANALYSIS_SETTINGS/);
expect(
requiredRuntimeLimitFailure(ENDPOINT_CANDIDATES, {
inRuntime: true,
inRegistry: false,
}),
).toBeUndefined();
expect(
requiredRuntimeLimitFailure("REA_NTFS_MFT_MAX_FILE_BYTES", {
inRuntime: false,
inRegistry: false,
}),
).toMatch(/ANALYSIS_SETTINGS/);
expect(
requiredRuntimeLimitFailure("REA_REGF_HIVE_MAX_FILE_BYTES", {
inRuntime: true,
inRegistry: true,
}),
).toMatch(/numericLimitRegistry/);
});