radareorg/r2sleigh
1
0 Fork 0
mirror of https://github.com/radareorg/r2sleigh.git synced 2026-11-10 17:41:57 -10:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #334 from radareorg/decbench-2026-10-23

Browse Source 
DecBench: the 2026-10-23 run corrects the ranking, a run that loses rows fails, the mined-set comparison is in the tree (D66)
This commit is contained in:
potato authored and GitHub committed 2026-10-23 19:01:51 +05:30
2 parents ddd38e69f9 32a74ae756
commit 7f696784f7
100 files changed
+15410 -1623
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/ci.yml
+9 -5
View File
@@ -619,12 +619,16 @@ jobs:
- name: Download the cross toolchain and qemu-user with their dependencies
run: |
apt="sudo apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=20"
for attempt in 1 2 3; do
timeout -k 10 120 $apt update -qq &&
timeout -k 10 360 $apt install --download-only --no-install-recommends -y -qq gcc-aarch64-linux-gnu libc6-dev-arm64-cross qemu-user &&
hosts=(azure.archive.ubuntu.com archive.ubuntu.com mirrors.edge.kernel.org)
for attempt in 1 2 3 4 5 6; do
host=${hosts[$(( (attempt - 1) % 3 ))]}
echo "try $attempt from $host"
sudo sed -i -E "s#(mirror\+file:/etc/apt/apt-mirrors\.txt|https?://(([a-z]+\.)?(archive|security)\.ubuntu\.com|mirrors\.edge\.kernel\.org)/ubuntu/?)#http://$host/ubuntu/#g" /etc/apt/sources.list.d/*.sources
timeout -k 10 60 $apt update -qq &&
timeout -k 10 90 $apt install --download-only --no-install-recommends -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross qemu-user &&
break
[ "$attempt" = 3 ] && exit 1
echo "apt try $attempt did not finish; trying again"
[ "$attempt" = 6 ] && exit 1
echo "try $attempt did not finish; trying again"
done
mkdir debs
cp /var/cache/apt/archives/*.deb debs/
ROADMAP.md
+19 -6
View File
@@ -96,7 +96,7 @@ Decisions
| D35 | x86-64 has a CPU oracle recorded on contabo and replayed on every host, and CI fails a change to a vendored `.sinc`, `.slaspec` or `.pspec` that adds no oracle case or `r2il::eval` test (D28 enforced) | done (VC, 2026-10-16: 374 cases; `scripts/spec_evidence.py`) |
| D36 | A temporary is named by where it is defined (its instruction and its place in that instruction's own layout), not by its unique-space offset, so a resync that changes no instruction's operations moves no census line | done (2026-10-16: r2il lays out, r2ssa names; gate `stable_temporaries`) |
| D37 | r2ssa states why an arity is a floor, one typed variant per floor branch (an unresolved transfer, a supervisor entry, an argument area unplaced or read past an unproven register prefix, a register slot read past the prefix, a call of unproven arity: indirect, an unresolved cycle member, a missing prototype), an engine test each; r2engine reads the cause, and a cycle's results are the same fixpoint as its arities | P7; the typed cause and the walk-free cycle done (2026-10-16: `SourceArityFloor`, nine engine tests, one per cause and call split); done, results half (2026-10-16): an unproven result states its causes (`SourceUnprovenResult`: eight `SourceResultCause` variants, `result_owners` and `Owners` deleted), a seed carrier from the paths of its own, and a cycle's results are the same worklist on every request's result demand; 14 engine tests (`results.rs`: one per cause, the pairs on x86-64 and AArch64, the three that stay unproven) |
| D38 | The cost verdict counts retired user-space instructions (`perf_event_open`; `--measure instructions`); each record and baseline states its measure and budget and the ratchet never compares across them; wall time is only a hang guard. An emulated target stays on `time` until a counting qemu exists: the packaged one has no `-plugin`, and `perf_event_open` is `ENOSYS` in its guest The stack-exhaustion exemption (a vector the original survives, faulted at the stack by both renderings, is not graded) can hide a rendering that recurses where the original returns; the closing step is to grade such a vector through a rendering built with sibling-call optimisation on, so a value difference or a real runaway still reads `differs`. | GT (counter and gate done 2026-10-16; the baseline flip and the emulated count next) |
| D38 | The cost verdict counts retired user-space instructions (`perf_event_open`; `--measure instructions`); each record and baseline states its measure and budget and the ratchet never compares across them; wall time is only a hang guard. An emulated target stays on `time` until a counting qemu exists: the packaged one has no `-plugin`, and `perf_event_open` is `ENOSYS` in its guest The stack-exhaustion exemption (a vector the original survives, faulted at the stack by both renderings, is not graded) can hide a rendering that recurses where the original returns; the closing step is to grade such a vector through a rendering built with sibling-call optimisation on, so a value difference or a real runaway still reads `differs`. The nearest wall bound is the x86 `Equivalence gcc-O2 a` shard's `timeout -s ABRT 8m`: 455 s at #327's head against 329 s at its base, and it aborted at about 489 s twice on older heads of that PR, so 25 s of margin stands between a green stack and a red one; until the instruction gate replaces the wall bound, split that shard (the clang shards are unchanged) before the next PR that adds equivalence work. | GT (counter and gate done 2026-10-16; the baseline flip and the emulated count next) |
| D39 | A scheduled CI job reports how far the Ghidra base, `TESTBINS_REF` and the judged radare2 release are behind upstream and fails past a stated budget | done (GT, 2026-10-16: `scripts/upstream_drift.py` and the weekly `upstream-drift` job; every processor's pin is read from `vendor/README.md`, a blobless clone counts the commits each pin lacks over the paths its copy vendors, and a vendored file equal to another upstream version refutes the pin; budgets 90 days Ghidra, 180 `TESTBINS_REF`, 90 radare2; first run red by design: ARM, MIPS, RISC-V and x86 lack commits 458, 422, 491 and 443 days old until VC's resync, and AArch64's `.ldefs`, `.opinion` and two `.cspec` files are older than its `918acd744d` pin, a resync of five files owed to VC next; the Ghidra base, `TESTBINS_REF` and radare2 6.2.4 are within budget) |
| D41 | A property of an `R2ILOp` has one owner in r2il, stated as an exhaustive list; a match elsewhere that derives one is a Dylint error (`r2il_op_property_outside_r2il`; #287 showed the class). The `SSAOp` call class has its owner in `SSAOp::transfer`/`is_call` (every other operation named by four class macros), about 35 copies read it, and `ssa_call_class_outside_op` denies a pattern naming `Call` or `CallInd` behind a default. It found `stack_protector`'s `fails_without_return`, which matched a direct `Call` only and dropped a `-fno-plt` `call *slot` (fixed, native test). The fix round found it read the block's ops, not its exit: a returning call then an unresolved jump counted as a failure path. It reads the CFG terminator now (the fact the render pass reads), and the walk closes the fallthrough of a call through a slot the program says never returns, which the image still listed as a successor. Next: `finish` re-derives a block's successors from its terminator where `record` decided them, so a call that runs into another function's entry still lists a successor the walk refused; the image should carry the walk's own. Landed (b3-lint-gaps): both lints read a wildcard nested in `Some(..)` as the default arm and an `Option<&R2ILOp>` scrutinee (the first tree hit, `call_may_not_return` in the lift, now reads `is_call`); `SSAOp::is_control_flow` and `value_use` list every class (`control_operation!`, the trap joins), and `OpTransfer::conditional` with `SSAOp::jump()` replaces the `Branch \| BranchInd` lists in `slot_promotion`, `frame_reach`, `block.rs` and r2dec (`plans_a_call`, `tail_gap`). Next: `obligation.rs`'s exhaustive arm and `proved.rs`/`stack_bound.rs` still name `Branch`, `BranchInd` or `Switch` beside others, and a Dylint denying the jump class waits for them; the render pass's `CallInd` with a `Through` callee does not read its target where `BranchInd` does, which needs a complete indirect call site to show | standing (GT, r2il); `SSAOp` calls done (r2ssa, 2026-10-16) |
| D42 | A refusal is a property of the stage type: a stage built from an exhausted or failed pass cannot be constructed (`Prepared::seal` refuses, so no `TrustedSeal` exists for such a body), and no reader re-checks it; a check at one consumer is a second owner (#273 let three engine readers through a seal only `into_artifact` refused) | done (2026-10-16: `Prepared::seal` and `Sealed::into_artifact` refuse; the flag and its accessors are `pub(crate)` on the stage's facts and on each pass's result; Dylint `exhausted_read_through_sealed_stage`, deny, no site in the tree; it follows receivers, fields, derefs and an immutable `let`, so a chain a function boundary breaks is not seen, and `PreparedFunctionFacts` holds no flag now: its passes and `collect` return the `Result`, and `RecoveredInterface` no longer copies the provisional collection's flag, `function/mod.rs` noting the `Err` (b3-lint-gaps; `DecompilePrepFacts` keeps its own `pub(crate)` flag, which `seal` reads); r2dec's value gap for an unproven tail jump reads r2ssa's refusal, not `UnsupportedInstruction` (B2, 2026-10-16)) |
@@ -104,17 +104,29 @@ Decisions
| D44 | The oracle's breadth is ratcheted: `scripts/oracle_breadth.py` counts each vendored specification's mnemonics with an oracle case (x86-64 117 of 1,513, AArch64 489 of 783 recorded) and its constructors (their width and addressing forms), and the uncovered count of either may only fall | VC, GT |
| D45 | A prover has a mutation target and an adversarial table: `cargo mutants` over `tail_count`, `save_area`, the stack-guard scrub, `result_calls` and the cycle fixpoint, a surviving mutant killed or listed with a reason | r2ssa |
| D46 | Moves are separated from noise by paired tests: a nightly job scores a pinned sample against the last passing night, a baseline that advances only on a pass or a bless, and reports against the pinned blessed baseline so slow drift shows (McNemar exact for the perfect rate, a sign test for the value, Holm-adjusted, significant below 0.01), and timing is an A/B in retired instructions with a bootstrap interval ([measurement discipline](doc/adr-measurement-discipline.md), from Kuna's `nightly.md`) | GT |
| D47 | A type or layout change records `improved` and `worse` per project beside the perfect count, layout and nesting F1 for structures, and a pinned project list splits mining from reporting: the mining column gates a PR, the held-out column is nightly and exit only and a regression there opens an investigation | GT, T |
| D47 | A type or layout change records `improved` and `worse` per project beside the perfect count, layout and nesting F1 for structures, and a pinned project list splits mining from reporting: the mining column gates a PR, the held-out column is nightly and exit only and a regression there opens an investigation; the mined-set comparison two structuring items used (dump every function of bzip2, zlib and grep at O2 and O2-noinline with each build, score with DecBench's GED, pair the two runs for perfect, mean and total GED, gotos, functions with a goto and text changed) now lives in the tree as `tests/decbench/mined_set.py` (`dump`, `score`, `pair`; the pinned `MINING` and `HELD_OUT` lists; `pair` refuses to compare the two; the pairing is unit-tested over synthetic rows and prints the scratch script's numbers on its scores; [README](tests/decbench/README.md)); next: the PR gate and the nightly that call it, `improved` and `worse` per project, the F1s, timing | GT, T |
| D48 | The census audits its own hunks: a per-feature script classifies a moved hunk and clears it alone only where it checks equivalence, every semantic class is read by a human, and a bless needs zero unclassified hunks | GT, first |
| D49 | Every refusal and residual cause has a catalog row (symptom, closing fact, owner) that feeds `?` help; a test fails on a cause with no row | RF |
| D50 | Batch decompilation is a deterministic pool (longest first, the call graph's condensation callee first with a recursive component one task in address order, per-function budget and memory stop, positional merge): one worker and eight give byte-identical output | LX, after the cost path |
| D51 | Published numbers are generated and honest: `Where this stands` is produced from the gates, shows a clean rate (no marked construct) beside `rendered` (bash 38.0%, fedora ls 58.7% against `94% of ls`), and records the DecBench run of 2026-10-17 (r2s 4th of 4: union-perfect 20.1% against angr 32.5%, kuna 24.4%, ghidra 20.9%; `type_match` 90 against 209; `byte_match` first; [record](tests/decbench/RESULT-2026-10-17.md)), which closes D10; README, `LICENSE` against `Cargo.toml`, the panic setting text and the x86-32 `decompiles` claim are corrected | GT, first |
| D52 | Types and structure are the first quality gap and carry a number (the same DecBench run, split per function: r2s is the best of four where a function has no goto, 37.4% GED-perfect against angr 34.7%, kuna 34.4%, ghidra 32.1%, but 40.2% of its scored functions contain a goto against 21.1% to 29.6% elsewhere, and a function with a goto is perfect in 4 of 1,054 cases; 48% of its scored functions carry three or more residuals and are perfect in 13.1% against 31.4% with none; the goto share is a symptom, not the metric: removing the terminal-block gotos (40.8% of them) gains one perfect function, and GED scores the missing `&&`/`||`, shared returns, loop rotation and latch copies a goto marks; the targets are return into arms at fallthrough joins (+15 measured, 0 lost), terminal-tail duplication, `&&`/`||` by BDD identity (ceiling 29), sole-predecessor exit inlining, and the three-or-more-residual share from 48% to a stated target, each graded by D47; [analysis](doc/adr-structure-dominator-tree.md)): T, P9, P8 and SD move ahead of LX2, MM and Q reuse in the Order, with a first exit of `type_match` from 90 of 3,285 (2.7%) to at least angr's 209 (6.4%) on the DecBench set, graded by D47's `improved`/`worse` | T, P9, P8, SD |
| D51 | Published numbers are generated and honest: `Where this stands` is produced from the gates, shows a clean rate (no marked construct) beside `rendered` (bash 38.0%, fedora ls 58.7% against `94% of ls`), and records the DecBench run of 2026-10-23 ([record](tests/decbench/RESULT-2026-10-23.md)), which replaces 2026-10-17 (that run scored no bash row for r2s, kuna or ghidra, because the OOM killer ended the evaluation worker, and counted 692 angr-only bash rows as misses for the other three): on all 6,978 rows r2s is third of four on union-perfect (28.0%; kuna 32.0%, ghidra 29.3%, angr 18.8%), first on `byte_match` (8.4%), third on GED (24.0%; kuna 28.4%) and third on `type_match` (5.2%; kuna 8.5%), and fourth on the 2,736 non-bash rows (25.4% against kuna 30.5%), which closes D10; README, `LICENSE` against `Cargo.toml`, the panic setting text and the x86-32 `decompiles` claim are corrected | GT, first |
| D52 | Types and structure are the first quality gap and carry a number (the same DecBench run, split per function: r2s is the best of four where a function has no goto, 37.4% GED-perfect against angr 34.7%, kuna 34.4%, ghidra 32.1%, but 40.2% of its scored functions contain a goto against 21.1% to 29.6% elsewhere, and a function with a goto is perfect in 4 of 1,054 cases; 48% of its scored functions carry three or more residuals and are perfect in 13.1% against 31.4% with none; the goto share is a symptom, not the metric: removing the terminal-block gotos (40.8% of them) gains one perfect function, and GED scores the missing `&&`/`||`, shared returns, loop rotation and latch copies a goto marks; the targets are return into arms at fallthrough joins (+15 measured, 0 lost), terminal-tail duplication, `&&`/`||` by BDD identity (ceiling 29), sole-predecessor exit inlining, and the three-or-more-residual share from 48% to a stated target, each graded by D47; [analysis](doc/adr-structure-dominator-tree.md)): T, P9, P8 and SD move ahead of LX2, MM and Q reuse in the Order, with a first exit of `type_match` from 328 (5.2%) to at least kuna's 539 of 6,365 (8.5%) on the corrected all-rows DecBench set (non-bash: 91 to kuna's 164 of 2,664), graded by D47's `improved`/`worse`. Measured 2026-10-23: the chains and tails cut gotos by 36% (8,353 to 5,366, non-bash) and moved union-perfect by +6, so structure's remaining gotos are a readability lever and the union levers are D65, D66 and D67's ranked list; the figures above this clause are the corrupted 2026-10-17 population | T, P9, P8, SD |
| D53 | Discovery never returns a silent empty list: a mapped executable range with no seed yields a D29 refusal that `afl`, `afi` and `pddj` print; `r2image` states a Cortex-M vector table as entries; prologues and code pointers are weaker seeds (libflutterapp 0 against radare2's 6,889; RP2040 `main.elf` 1 against 198) | FS, I |
| D54 | A cap that keeps a partial result is an error and a cache outside the query database is an error, enforced: K widens to the optimiser round budget, address provenance, the r2rewrite term budget (r2dec reads its failures), the `COPIES = 8` family and the lift's atomic `DEPTH`; the name interner is scoped to a function; `cache_outside_query_database` covers every crate; D11's lints go to deny with an allowlist that only shrinks; `Held` tables get a byte budget with a recompute-or-refuse policy; K moves ahead of the cost path | K, LX, GT |
| D55 | Wall time: the listing answers from the walked body (`pdf`, `afb`, `afi` 4 to 5 times slower than radare2 on the bench protocol), and D50's pool design (threads over a `Send` database with shared `Held` summaries, or processes) is decided before D50 starts (gzdoom decompile-all 758.7 s serial against Kuna's 155 s wall at ten workers); the bench records RSS and a decompile-all column; `ax` and gzdoom enter `TIMED` | Q, LX |
| D56 | The product surface has dates: user facts (rename, retype, comment, save) as query-database inputs with a `Basis` (ADR first), `j` on every verb, an embedded agent skill with a test that every verb it names exists, a thin MCP server, an agent-task bench of about 30 questions before A, an i386 equivalence target, cited Windows API rows, library identification by structural hashes, demangling as a `Symbol` hint | V, S1, A, I, P7 |
| D57 | The loop reaches the field and the repository: a merge cadence (a cap of 8 open stacked PRs, a merge queue, bottom-first on green), a red nightly opens an issue within 24 hours, the bench lives in the tree and its nightly builds the branch tip, artifacts leave contabo, a release binary per tag with an opt-in friction record, a fuzz and sanitizer gate for untrusted input, and a quarterly field note on IDA, Binary Ninja, Ghidra, angr, Kuna, rizin and radare2 | GT, maintainer |
| D58 | A rendering decision depends on facts, not on the order placement writes blocks. Done (2026-10-23, [adr-decompiler-rewrite](doc/adr-decompiler-rewrite.md) "D2's spelling order"): `render::spelled::spell` spells every placed block once, in the CFG's reverse postorder, before `control::Writer` places any (`O(B + E + statements + phi inputs)`, no fixpoint), and the Writer holds no `Values` (`crates/r2dec/tests/architecture.rs`); a merge reads as a variable only when every forward edge into its block renders its copy (`Values::owed`; a back edge, a self copy and a gap pay nothing; an edge that retreats in reverse postorder in an irreducible region is read on its promise and the function is spelled again if that copy gaps, `spell_function`: 43 of 2,508 functions hold one, 1 is spelled again). Gate: `census.py orders` over a `--features placement_mutants` build (O1 keeps every loop exit after its loop, O2 reverses each loop's exits, O3 each region's merges) on bash, ls-toybox and pigz: 0 differ under all three, against 0, 120 and 187 with the knobs alone; the text moves in 64, 332 and 720 functions, so the knobs are live; 1 (O2) and 7 (O3) differ only in a selection the structurer chose (`x = c ? a : b`), which reads the layout; five engine tests under the four orders, each a fresh program (the 43, 48 and 24 byte merges that fail before the pass or the rule, a merge whose copies all render, and two irreducible cycles). Natural order against the base, bash, ls-toybox and pigz: 13 of 2,508 functions move, +9 residual sites (seven bash functions' block-transfer locals are numbered in reverse postorder, bash `0x8ee40` lists its `_next` temporaries in another order, five hold a gapped forward copy: bash `0x2dd90`, `0x48140`, `0x85d70`, ls-toybox `0x392dc`, `0x501c8`); 0pack (8,437 functions) 47,850 to 47,898 residual sites (27 functions move: 26 more sites, none fewer) and pumasim (14,776) 67,628 to 67,644 (15 move: 8 more, 2 fewer: `0x279d80` and `0x27b040`, whose `atan2f` read a lane its block does not dominate as a variable, wrong C that D68 replaces with the carrier that dominates it), the biggest movers (0pack `0x4e46d0` 17 to 20, pumasim `0x3f9610` 5 to 10, `0x23b6c0` 3 to 7) each a merge with a gapped forward copy read as residual; none goes rendered to refused, the 442 census goldens and `stable_temporaries` hold byte for byte, certification is 50 rendered, 30 refused, 0 invented, dropped or panicked, as at the base. 0pack `0x476982` has 9 residual sites under all four orders (the base's 7 under O1 read `rsp_8` after a loop whose entry copy at `0x4769b1` is a gap, which nothing assigns on a zero-trip path); bash `0x3c2b0` 0, `0x26760` 221 (1,026 residual obligations) and `0x61520` 4 sites (13 obligations) under every order, `0x85d70` 44 sites (42 at the base: two `rsp_8` reads, entry copy a gap at `0x85de0`). Release `pdd` instructions retired against the base within -0.12% to +0.13% on 0pack `0x62ecf0`, `0x58e3d0`, `0x4e5f90` and pumasim `0x23fdf0`, `0x1fafb0`, `0x2312c0`, peak RSS within the run to run spread (264 to 317 MB both). Left: D62 (a callee's result presence), D63 (any rendered forward copy), D64 (`Values` immutable once spelled); the done text before this is in [doc/roadmap-history.md](doc/roadmap-history.md) | done (SD); left D62, D63, D64; two numbers and the 2 fewer corrected by D68 |
| D59 | A condition chain whose exit is a `continue` to its own head is certified | done (2026-10-23, branch `d59-d60-guards`): `condition::exits` takes an edge to a later test as an edge of the chain's DAG and any other edge to a member (the head, an earlier test) as an exit, as a lone test always did, so the `ExitCount { found: 1 }` fault no longer reverts every chain of the function; the engine test `a_chain_whose_exit_is_a_continue_to_its_own_head_is_one_test` (a scan to a zero byte or a colon: two tests in a `for (;;)` at the base, `do { ... } while (c != 0 && c != 58)` now) and the certificate test `a_chain_whose_second_test_goes_back_to_its_head_has_the_head_as_an_exit` fail without it. Repro of the row: `R2DEC_TRACE_REFUSAL=1 r2s -q -c "aa; pdd @ 0x2e540" bash` no longer prints the revert; `fedora_32_bin_ls` `0xe690` and `0x11ef0` likewise. Over bash, ld-2.31.so, fedora_32_bin_ls, ls-toybox, pigz, bashbot O0 and static-glibc (3,765 functions) 12 change, none newly refused, no goto or residual site added; 11 move only control lines (more `&&`, `do`/`while` for `for (;;)`, fewer `break`, the statements equal) and pigz `get_header` writes one copy fewer of a two-statement tail; bash `extract_colon_unit` runs as at the base on 48 vectors; golden and stable_temporaries are unmoved. D59's program does not reach `rotate::negated`'s De Morgan branch (its continue carries a copy, so the chain is spelled on the staying side): D60 c. Over every function of 0pack and pumasim (23,213) 8 change, all the same loops, none newly refused. The DecBench mined set is not re-scored here | SD |
| D60 | The structurer's data guards each have an end to end engine test and the certificate reads what a copy says | done (2026-10-23, branch `d59-d60-guards`): (a) one engine test per latch-hoist guard, each failing with only its guard removed: L4 `a_copy_stays_on_its_arm_when_the_test_reads_the_variable_it_writes`, L2 `a_copy_stays_on_its_arm_when_code_after_the_loop_reads_what_it_writes`, L3 `a_copy_stays_on_its_arm_when_the_test_is_not_a_proven_read`; (b) `I_T` in `structure/certify.rs`: every statement the placement wrote for a block is written, and as often as any other of it (edge copies are marked statements of the block they leave, with no instruction and no place in the walk; `O(statements)`), 0 of 3,765 functions newly refused or changed, no Statement violation at the base, 23,213 functions of 0pack and pumasim change only as D59 does; with the previous skeptic's drop mutants run again through the corpus: L5 (the hoist drops its copies) 543 functions change and the certificate reverts the stage in all 543 (388 of 388 passed before), T2 (a tail copy drops its first statement) 488 change and 488 are refused (73 of 280 passed), T12 (a joined return's copy drops its first statement) 204 change and 204 are refused (144 of 144 before); (c) the De Morgan branch is reached by a unit test (`rotate::tests::a_loop_that_leaves_on_a_chain_stays_on_its_negation_with_the_connective_flipped`) and an engine test (`a_loop_that_leaves_on_a_chain_stays_on_both_tests_negated`: a chain with nothing copied on the way round), each failing with R1; (d) `owes_only_its_branch`, `NotInert` and the certificate's `inert` parameter are deleted: forced to `true` the fact moved 0 of 3,765 renderings and a condition holds a load in none, the text decides who may be a member, and a member's lost statement is now a Statement violation; (e) C1 `exact` and T3 `named` were dead and are gone, C4 asserts the chain is the one violation, T9 is equal (it changes which arm end keeps the original's markers), V2 went with (d). The sweep again, 61 mutants: 5 no longer apply, 42 of the other 56 die to r2dec unit tests, 8 to engine tests, C3, T7 and T13 to the golden census only, and S5, S6 and T9 survived all three; S5 and S6 (`statement_role` without the never-returns evidence, at the final certificate and at a stage) now die to the engine test `an_arm_ending_in_a_call_declared_never_to_return_is_a_conjunction_that_leaves_by_it` (S5 refuses the function, S6 reverts the chain), and T9 is equal. Release `pdd` on 0pack `0x62ecf0` and pumasim `0x23fdf0`: -0.1% to +0.1% instructions retired, RSS flat, the renderings byte-identical; exit bar: workspace suite 2,977 passed, 0 failed, 8 ignored, certification 50 rendered, 30 refused, 0 invented, 0 dropped, Dylint 12, structure counts unchanged. Left: D61 | SD |
| D61 | A copy's place and text are read, not only that its statements are written, and a copy may carry a residual. (1) `I_T` counts a block's statements across its occurrences: a statement moved from one copy of a block to another, or written twice in one occurrence, leaves the counts equal, and a block of one statement has no sibling to differ from; attribute each statement to the occurrence the walk has open (`Walker::walk_observed`) and refuse a root twice in one occurrence. D58's `Spelled` table holds each block's statements once, which is what that check reads. (2) A copy's text is not compared with its original's (`remint` clones the expressions, so only a producer that rewrites a copy differs): compare the unobserved leaves of one statement across its occurrences (`CExpr` holds `f64`, so a NaN literal needs an order that equals itself). (3) The cost D60 kept: a skipped block that holds a residual is no longer copied into two arms, so 6 of 3,288 functions gain a goto (bash `0x65bf0` 0 to 1, `0x67b10`, `0x67ca0`; ls-toybox `0x5d458`; pigz `0x4115f0`, `0x411a40`; 873 lose gotos, 15,429 to 11,914 in all), until a copy can carry a residual without counting its obligation twice; the stack adds one, bash `0x48140` (1 to 2 gotos at the tip, `R2DEC_TRACE_REFUSAL=1 r2s -q -c "aa; pdd @ 0x48140" bash`): D58's rule reads its tail's `r13_4` as a residual, and a tail that holds one is not copied. (4) Guards that only the census holds: C3 (`leaves_to` of a chain in the walk, 1 golden), T7 (the join needs two arm ends, 5 goldens) and T13 (`replace_gotos` under a switch's arms, 3 goldens) die to no unit or engine test; an engine program each. Gate: mutants that move a statement between copies and that rewrite a copy's operand, each refused by the certificate; one engine test per guard of (4) that fails with it removed (C3, T7 and T13); (3) a bash `0x65bf0` golden, and one for `0x48140`, that read without the goto. Owner: `r2dec` `structure/certify.rs`; (3) `render::values` for the residual's one cell; timing: (1) and (2) conditional on a producer that does either (none does), (3) and (4) next | SD |
| D62 | A callee's result presence is one fact per callee, stated by r2ssa and joined by r2types, not each call site's to spell in r2dec. r2ssa states per call site how the result register's value after the call is read: Dead, ReturnedOnly (with whether the caller's own return type is proven) or Consumed by an operation, transitively through copies and phis, once per function in `O(values + uses)`; r2types joins the site facts per callee in `CalleeResolutionFacts` (a stated prototype or the callee body's carriers outrank site evidence, then Consumed, then ReturnedOnly, then Dead); `Values::decide_prototypes` reads that join and breaks ties by address only. Not a downstream vote: a Readers vote in r2dec rebuilds an r2ssa fact, and a vote that lets the first call with a result win makes pumasim `png_free` `uint64_t`, which doc/adr-decompiler-rewrite.md D4 calls no new evidence. Repro: bash `pdd @ 0x00061520` (`ignore_signal`) declares `void set_signal_handler(uint64_t, uint64_t)` from the first call in address order, `RAX_3` is never assigned behind the call that reads it, 4 residual sites and 13 residual obligations (12 at master); the 2026-10-23 address-order declaration moved 22 of 13,093 declarations (14 `void` to a value, 8 a value to `void`), and pumasim `0x40c140` (`QPointArray::setPoint`, 14 disagreeing sites against 46), `0x34c050` (`QGDict::look_ptr`, 6 more never-assigned reads), 0pack `0x66d2b0`, `0x66f040` and `0x670e80` (`MessageBuilder::operator<<`, 5 against 15) are the same disagreement. First step: read why `callee_carrier_signature` or `Prototype::of` gives no `CallPlan.declared` at bash `0x61520` (the `staged-call-prototype` refusal evidence); where a prototype is stated every site agrees and the tie break never fires. Gate: bash `0x61520` loses its residual obligations without a new `void` to value flip elsewhere in the census, and an engine test that pumasim `png_free` (`0x003b6b20`) stays `void` with 23 residual sites; owners `r2ssa`, `r2types`, `r2dec`; timing: next | SD, next |
| D63 | A merge with one gapped forward copy and one rendered one reads residual (D58 requires every forward edge to render its copy), though a gap traps where it stands. Relaxing to any rendered copy is sound at run time only while the emitted trap and the certificate agree: `GapKind::ValuesNotRendered.ends_control()` is false and `render::traps` excludes it, so the certificate lets control go on past a gap. Relax only if the certificate model changes to say a gap ends control, and a count of merges with at least one rendered and one gapped forward copy (`R2DEC_TRACE_REFUSAL` prints `gapped-forward-copy` per edge: 426 over bash, ls-toybox and pigz at D58's landing) shows many are lost; gate: bash `0x2dd90` (`rbx_2`) and `0x48140` (`r13_4`) read residual under the rule and plain at the base, so read which of their merges mix a rendered and a gapped copy first; owners `r2ssa` (the certificate), `r2dec`; timing: conditional | SD, conditional |
| D64 | `Values` keeps interior mutability (`RefCell` scratch for rendered, residual, effect, restored, owed, broken and the pending attempt, and the block being spelled, a `Cell`) and the D58 spelling pass is its only mutator; making the pass `&mut self` and `Values` immutable once spelled, with each attempt of `spell_function` a value rather than fresh mutable state, makes the single owner true by the type and not by the architecture test. Gate: no `RefCell` in `render::values` outside the pass; owner `r2dec` `render`; timing: deferred | SD, deferred |
| D65 | A function is an import's stub only where the program says its address is one and its body is only the transfer; the route asked the body alone, so a `.text` function that ends in a jump to an import's stub printed "import stub `free` has no body" and no C, though `afl` lists it as no import. Done (2026-10-23, branch `d65-stub-one-jump`, [adr-decompiler-rewrite](doc/adr-decompiler-rewrite.md) "The import-stub route"): `native::sealed` asks `Program::import_at(entry)` (discovery's `Linkage`, `O(log stubs)`) and `SealedFunctionAnalysis::stub_of` runs `stub::import_stub` only for a stated import, naming the stub as discovery does; `seal_function_from_input` and `decompile_function_from_input`, which have no program, no longer take the route. bzip2 `default_bzfree` (`0x5b00`: `test rsi, rsi; je; mov rdi, rsi; jmp free`) and `default_bzalloc` (`0x5b20`: `imul esi, edx; movsxd rdi, esi; jmp malloc`) now render the test and the `free` call, and `malloc` of the product, with the unread-parameter arguments as residuals. The 63 DecBench rows (2026-10-23): 52 write an argument register before the jump, 2 test first, 9 are a bare `jmp` to the stub; all 63 render, 22 without a residual, 24 with a parameter the body reads and nothing declares (P7), 16 with an `int` result whose upper half is a residual (W47), 1 with both. Over bash, ls-toybox, pigz, 0pack and pumasim (26,935 functions) 1,415 rendered as an import stub at the base: 1,200 are `afl`'s `sym.imp.*` and stay stubs, 215 are functions with a name or a body of their own (0pack 100, pumasim 90, bash 13, ls-toybox 12) and render; none is newly refused, no goto is added, +86 residual sites, and the base declared them away wrongly (pumasim `QLayoutItem::~QLayoutItem()` deleting destructor `0x295d10` printed as `extern void _ZdlPvm(void*, uint64_t)`, its constant size dropped; a static initialiser's `__cxa_atexit(...)` as the import itself). The census (442 goldens, 24 stable-temporaries) is unmoved, and the engine tests `a_guarded_wrapper_of_a_stub_is_a_body_not_the_stub`, `a_wrapper_that_computes_the_stub_s_argument_is_a_body_not_the_stub` and `a_function_that_only_jumps_to_a_stub_is_a_body_not_the_stub` fail at the base while `an_import_stub_is_its_import_s_declaration` and `an_effect_before_the_jump_keeps_a_stub_a_body` hold. Left: the DecBench no-output count 380 to about 317 is the main session's run; `stub::import_stub` keeps its body proof as the second half (`stub_start` already stops a container stub at any write the transfer does not read, so the two agree on every stub discovery names); a linkage section `section_stubs` rejects now renders bodies with a visible gap, a discovery defect `afl` and `pdd` both show | FS |
| D66 | A run that loses rows fails, and the backend checkpoints by time. `tests/decbench/run_gate.py` stops a run (exit 71) when a binary a decompiler rendered has no score key of that decompiler in `evaluated/<binary>.toml` (the 16-byte `binary = "bash"` of 2026-10-17: 55 of 192 cells of that tree, all bash) and refuses more than one evaluation worker (exit 72) for a binary whose decompiled C exceeds 8 MB (one worker holds 5.2 GB for r2s's bash), in `run_decbench.sh`, in the driver before `evaluate_project` (as `SystemExit`, which the driver's `except Exception` does not swallow) and in `evaluate_checkpointed.py`; `report_decbench.py` compares decompilers over the rows every one scored, prints the rows it left out and each decompiler's rows without a score, and exits 73 when one scored nothing in a project (it stops at `kuna scored 0 of 4144 rows of bash` on the 2026-10-17 results and passes the 2026-10-23 ones); `r2sleigh_raw.py` pickles its partial result every 30 s and at the end, not per function (2.5 MB each on bzip2, 18 MB on bash, quadratic: the bash backend took 1,253 s and 1,723 s against about 140 s for r2s alone) and resumes from a checkpoint of the same build, binary and targets. `r2s_batch.py` gave `on_answer` a process's answers only after the process had ended, so the per-function checkpoint never held an answer of a killed run; it streams them now. Next: a host run of bash with the 30 s backend, to put the wall time against r2s's 140 s | GT, immediate (landed on decbench-2, unit-tested; the driver wiring is proven by `--plan` and a stand-in, not a driver run) |
| D67 | The DecBench-ranked fixes after D65 and D66, each with the ceiling the 2026-10-23 run measured ([record](tests/decbench/RESULT-2026-10-23.md), section 5), graded by D47 and the source-gold soundness gate. (1) Library prototypes into values: r2abi states pointer-bearing prototypes for the 23 stdio and multibyte imports (945 renderings) and `error`'s `...`, r2types types a value by the callee parameter it flows into and a call result by its return type; 248 pointer-as-integer and 36 library-named functions where it is the only cause, 123 net union rows. (2) A proven return or `void` (W47): about +35 GED-perfect non-bash rows, a size-controlled estimate and not a counterfactual. (3) Widths from reads (W49, W50): 209 functions, 142 net union rows. (4) One C object per SSA web, stated by r2ssa and lowered by r2dec: 138 functions, 126 net union rows, and bash's C from 13.9 MB toward 2.6 MB; unproven for GED, graded on the mined set first. (5) Parameters that flow into residual calls (P7 call contracts): 402 functions, 138 only cause; the 410 functions whose parameter nothing reads need caller or function-pointer-slot evidence from r2engine and have no ceiling yet. (6) Structurer respellings: the noreturn arm as a guard (+12 / -1, sound by the callee's noreturn contract), the select respelling (+15 / 0) only after an ADR decision on rule 8, the do-while rotation's 10 lost rows when the bound is a constant. (7) Unexplained: r2s's mean GED is 39.0 on the non-bash rows against 18.7 to 20.9 for the others while its perfect count is competitive; find what drives the tail (the 380 no-output rows, the large functions) before choosing a fix. Deferred, as benchmark facts or without a shape to copy: large functions (1,345 rows of 20 blocks or more, 2 perfect for any tool), switches (115 rows), compound conditions (75), the project-named type cap (3,020 functions) and the unobservable cap (2,255) | T, P7, SD |
| D68 | A named value is read as a variable only where its definition dominates the read. Done (2026-10-23, branch `d68-dominating-reads`, [adr-decompiler-rewrite](doc/adr-decompiler-rewrite.md) "A name is read where its definition has run"). The skeptic's finding at D58's tip `15486ea7`: `narrowed_argument` took the first value of a lane's class in definition order as a call argument, a value that can be defined where the call is not dominated (pumasim `0x279d80` and `0x27b040`, 4 reads of `tmp_lane_..` assigned only on the other arm of a branch), and D58's reverse postorder spelling read it as a variable; the parents print the same wrong C on a mirrored layout and a residual on another, so D58 moved which functions show a fact error of r2ssa's. Fixed at the owners. r2ssa: `integrity::defined_before` (an entry value, a phi or an earlier operation of the block, a strictly dominating block; `O(1)`) keeps a lane only where its definition has run at the call and otherwise states the carrier the walk reached (`(uint32_t)xmm1_6`); `validate_boundary_values` checks every value a sealed fact reads (call arguments and carriers, call targets, return values, return address, exit stack pointer, branch conditions, switch selectors; `O(facts)`) and refuses the function otherwise (`IntegrityRule::BoundaryValueNotDominated`; call results are definitions and a comparison's operands are copy-class names, so neither is a read). r2dec: `read_name` keeps its own rule as a hard guard, not a `debug_assert` (a residual with refusal evidence and a proof-line cause), asking r2ssa's `value_defined_before` at the block being spelled. Gate: three engine tests under the four write orders (`a_call_argument_is_never_a_lane_that_only_the_other_arm_assigns_under_every_write_order` fails at `15486ea7` on the skeptic's `lane_nondom9` and `lane_nondom7`; with the pick and the check both off, `fact_mutants`, the guard alone makes the read a residual; with the pick off and the check kept, r2ssa refuses), five r2ssa tests (`function::tests::lanes`), the guard's table in r2dec; `scripts/mutants_d68.py`, 25 mutants, 22 killed and 3 alive with reasons. Measured over 25,735 functions of bash, ls-toybox, pigz, 0pack and pumasim and 14,034 of ten more: 2 change (pumasim `0x279d80`, `0x27b040`), none is newly refused, residual sites and gotos equal; the dominance audit reads 0 names out of scope under all four write orders (4 in 2 functions at the tip) and clang `-Wuninitialized` flags 56 of 1,920 generated float functions at the tip and 0 now, under all four write orders; `census.py orders` over 2,508 functions holds at 0, 0, 0 differing, now over refused functions' ledgers too. Also: the production write order (exits inlined, no list permuted) is compiled under test and pinned, and `census.py orders` fingerprints a refused function, which moved 7 guards to tip unseen. Left: a later member of the lane's class that dominates the call is not looked for (`low_lanes` holds the first of each class); the `canonical_source` of a stack reload and a comparison's operands are first-of-class names no renderer reads, unchecked; certificates repeat their facts' values and are not checked on their own | done (SD) |
| D69 | A function a change moves is executed against its base on real binaries, as a standing exit gate. Today the equivalence ratchet's 828 records come from eight small C files (`tests/corpus`, `tests/gold`), so none of the real-binary functions the structurer items change is covered by it: D58 moved 55, D59 12, D65 215 (2026-10-23). The skeptics ran the check by hand in scratch (`difftest2` over the changed lists: native vectors with a trap-aware harness, a control mutant that must show DIFF on 100 of 100 vectors), and found D58's wrong C by a second oracle that the 442 goldens, the 2,998 tests, the census and the equivalence run all missed (clang `-Wuninitialized` with the residual helpers marked noreturn, plus a dominance audit of every variable read). Move both into the tree: the changed-function differential execution beside `tests/equiv`, the uninitialized-read sweep into certification (`certify_render.py`), each with a control that must fail | GT |
The Sleigh crates (`libsla`, `libsla-sys`, `sleigh-config`) are vendored (`vendor/`).
@@ -184,7 +196,7 @@ staged merges of PR #67 (finished items land on master).
| Item | Done | Left | Exit |
|------|------|------|------|
| **D** Decompiler rewrite | D0 to D5; the old r2dec deleted | `uncompress2` and `deflateInit2_` read by hand; the rendered function named by its entry, not its loop header's `root()` (audit W44) | — |
| **SD** Structuring and readability ([structure-dominator-tree](doc/adr-structure-dominator-tree.md)) | the dominator-tree structurer; a latch that leaves a carried value alone writes no `x = x` and stages no `_next` temporary, since an edge that carries the merge's own value leaves the parallel copy (2026-10-16: 492 self-copies gone over 13 binaries, 3 bash functions lose their staging; census unmoved); r2ssa now states that edge, `forward_copies` pointing a merge source that is a copy of the merge's own output (ls-toybox 0x4cc5c's 4172) at the merge, so r2dec compares ids and its spelled-name check is deleted (2026-10-16: 0 self-copies over the same 13 binaries, 429 goldens and `pdd` on 0pack and pumasim byte-identical, instructions retired and peak RSS flat; on the scan three more `x = y` edges drop that the spelled check missed because the copy had its own name: ls-toybox `x21_4 = x21_9`, `x25_3 = x25_5`, fedora_32_bin_ls `r8_25 = r8_27`). Constant branches are decided before preparation (2026-10-17, audit D13): `SccpMode::Decide` runs the lattice on the pdd path and rewrites no operand, so the dead arm owes no obligation (`frame_dummy`, `register_tm_clones`, `__libc_csu_init`'s empty init array: 27 goldens, each a guard over link-time constants or a register the code zeroed; over the three static binaries holding 222 of the 228 sites `call to fcn_0` 223 to 132, r2dec gaps down 157 and refused functions 320 to 315 with none going rendered to refused, pinned gapped obligations 6 fewer; certification identical (50 rendered, 30 refused, 0 invented, dropped or panicked); 0pack and pumasim `pdd` byte-identical, +1.7 and +2.3% instructions retired (the pass itself: a multiply-fold hash for the SCCP sets saved 0.2%, so dense block sets would not recover it), peak RSS flat); a guard after a user operation with unstated effects (`syscall`) is never decided, since SSA has no definition of the register the kernel writes (`__access` lost its `jae __syscall_error` edge before that; the poison keeps 6 `fcn_0` sites in server-zigrefs). Next: the identity copy itself still stands for its other readers, so `forward_copies` forwards it globally (not a memory variable), with the goldens re-run; the sweep's three downstream regressions (2026-10-17): `_Unwind_Find_FDE` refused on the stack pointer's merge past a conditional call once `__dl_iterate_phdr`'s arguments were proven, because the call's own `CallUse` of the stack pointer carrier counted as a program read that kept the merge out of `frame_setup` (any frame address passed on after a conditional call refused; fixed in `collect_stack_geometry_certificate`, gate `a_stack_pointer_merged_past_a_call_to_nowhere_is_the_frame_argument_it_is`, which fails at the base); `__run_exit_handlers` gained a fourth formal and lost `uint32_t` and `uint8_t` because `convention_call_boundary` read a phi merging the arrival with a write as a write, so an unknown indirect call counted `rcx` (`merges_of_arrivals`, once per function, O(values + merge inputs); gate `registers_written_on_one_path_before_an_unknown_call_are_no_parameters`, 4 formals before and 3 after; the dead `call 0x0` arm had hidden it); `_IO_un_link` keeps 2 gaps where it had 1 and is an exposed defect, not honest precision (glibc's `_IO_un_link` and `.part.1` are both `void`, so the root is recovery proving `.part.1` returns `rax`): its tail callee now proves it returns `rax` while its own `ret` leaves `rax` the caller's, so that `ret` is an incomplete return boundary and `taint_incomplete_boundary_inputs` gaps the `test byte [rdi], 0x80` that decides it (owner `recover_interface`: an exact tail result is not the function's result where a direct return leaves the carrier unwritten, as the base's unproven callee result left it; census-wide, so its own item); the removed blocks leave no record, so `pdf` still shows the dead `call rax` that `pdd` no longer owes (a count beside `compiler_inserted_blocks`); `inst_combine` folds through the same stale `syscall` register (the certificate refuses it) and belongs in the lift's userop effects 2026-10-22: the ADR's S2 was marked done but its legacy cleanup pass (`structure/rewrite.rs`) died with the legacy decompiler in 51b9d6a9; the writer emits `for(;;)` only (no while, do-while rotation), no `&&`/`||`, and no return into arms, which is 40.2% of scored DecBench functions ending in a goto (five ranked fixes in the ADR). | inverted empty thens, merges spliced into the parent, dead reloads proved; condition chains and irreducible entries behind a BDD identity check; whether lexical ancestry equals dominance, which would retire `region_does_not_dominate_occurrence` (measured on the placement census) | no more labels than the old structurer rendered over the functions it handled (doc/adr-structure-dominator-tree.md), and no more labels or nesting than today on the census, each moved line judged |
| **SD** Structuring and readability ([structure-dominator-tree](doc/adr-structure-dominator-tree.md)) | the dominator-tree structurer; a latch that leaves a carried value alone writes no `x = x` and stages no `_next` temporary, since an edge that carries the merge's own value leaves the parallel copy (2026-10-16: 492 self-copies gone over 13 binaries, 3 bash functions lose their staging; census unmoved); r2ssa now states that edge, `forward_copies` pointing a merge source that is a copy of the merge's own output (ls-toybox 0x4cc5c's 4172) at the merge, so r2dec compares ids and its spelled-name check is deleted (2026-10-16: 0 self-copies over the same 13 binaries, 429 goldens and `pdd` on 0pack and pumasim byte-identical, instructions retired and peak RSS flat; on the scan three more `x = y` edges drop that the spelled check missed because the copy had its own name: ls-toybox `x21_4 = x21_9`, `x25_3 = x25_5`, fedora_32_bin_ls `r8_25 = r8_27`). Constant branches are decided before preparation (2026-10-17, audit D13): `SccpMode::Decide` runs the lattice on the pdd path and rewrites no operand, so the dead arm owes no obligation (`frame_dummy`, `register_tm_clones`, `__libc_csu_init`'s empty init array: 27 goldens, each a guard over link-time constants or a register the code zeroed; over the three static binaries holding 222 of the 228 sites `call to fcn_0` 223 to 132, r2dec gaps down 157 and refused functions 320 to 315 with none going rendered to refused, pinned gapped obligations 6 fewer; certification identical (50 rendered, 30 refused, 0 invented, dropped or panicked); 0pack and pumasim `pdd` byte-identical, +1.7 and +2.3% instructions retired (the pass itself: a multiply-fold hash for the SCCP sets saved 0.2%, so dense block sets would not recover it), peak RSS flat); a guard after a user operation with unstated effects (`syscall`) is never decided, since SSA has no definition of the register the kernel writes (`__access` lost its `jae __syscall_error` edge before that; the poison keeps 6 `fcn_0` sites in server-zigrefs). Next: the identity copy itself still stands for its other readers, so `forward_copies` forwards it globally (not a memory variable), with the goldens re-run; the sweep's three downstream regressions (2026-10-17): `_Unwind_Find_FDE` refused on the stack pointer's merge past a conditional call once `__dl_iterate_phdr`'s arguments were proven, because the call's own `CallUse` of the stack pointer carrier counted as a program read that kept the merge out of `frame_setup` (any frame address passed on after a conditional call refused; fixed in `collect_stack_geometry_certificate`, gate `a_stack_pointer_merged_past_a_call_to_nowhere_is_the_frame_argument_it_is`, which fails at the base); `__run_exit_handlers` gained a fourth formal and lost `uint32_t` and `uint8_t` because `convention_call_boundary` read a phi merging the arrival with a write as a write, so an unknown indirect call counted `rcx` (`merges_of_arrivals`, once per function, O(values + merge inputs); gate `registers_written_on_one_path_before_an_unknown_call_are_no_parameters`, 4 formals before and 3 after; the dead `call 0x0` arm had hidden it); `_IO_un_link` keeps 2 gaps where it had 1 and is an exposed defect, not honest precision (glibc's `_IO_un_link` and `.part.1` are both `void`, so the root is recovery proving `.part.1` returns `rax`): its tail callee now proves it returns `rax` while its own `ret` leaves `rax` the caller's, so that `ret` is an incomplete return boundary and `taint_incomplete_boundary_inputs` gaps the `test byte [rdi], 0x80` that decides it (owner `recover_interface`: an exact tail result is not the function's result where a direct return leaves the carrier unwritten, as the base's unproven callee result left it; census-wide, so its own item); the removed blocks leave no record, so `pdf` still shows the dead `call rax` that `pdd` no longer owes (a count beside `compiler_inserted_blocks`); `inst_combine` folds through the same stale `syscall` register (the certificate refuses it) and belongs in the lift's userop effects. 2026-10-22: the ADR's S2 was marked done but its legacy cleanup pass (`structure/rewrite.rs`) died with the legacy decompiler in 51b9d6a9; the writer emits `for(;;)` only (no while, do-while rotation), no `&&`/`||`, and no return into arms, which is 40.2% of scored DecBench functions ending in a goto (five ranked fixes in the ADR). Seven recoveries stack on the writer (2026-10-22, B5, doc/adr-structure-dominator-tree.md "the recoveries together"), each stage kept only while the control certificate holds and the marker table one type (`shape::Occurrences`): condition chains (`&&`/`||` where the CFG's DAG of blocks owing nothing but their branch has the test's exit function by BDD identity within 16 atoms and 512 nodes), `while`/`do-while`, the latch's copies written before its test, a return of a merge of pairwise-distinct terms written in each arm that falls to it, a terminal block of at mLine truncated
### Surface (`r2s`, `r2s-tui`, `r2sleigh-cli`)
@@ -247,7 +259,8 @@ each other, each on its own branch and stacked (AGENTS.md).
oracle before further x86 spec fixes (done 2026-10-16, its list in the VC row), D34's pass plan and its eval check
(done 2026-10-16, with D36's stable temporary names).
7. **P7**, with T's one declaration fact (W46, W47): invented arguments are an
AGENTS blocker and the most common wrong C left.
AGENTS blocker and the most common wrong C left. DecBench 2026-10-23 ranks the next work (D67): D65 and D66 first, then library prototypes into values,
W47, W49 and W50, then one object per web; SD's respellings are a readability lever and come last.
D37 is done: the typed floor cause and a cycle's results; its rest is the callee's logical width on the call fact.
8. **The cost path**: LX's AArch64 quadratic (P5, #252) and P21's liveness half (#258) done, its dense rename identities and domtrees left (LX);
callee preparation is now the largest cost (gzdoom: `recover_interface`
crates/r2dec/Cargo.toml
+2
View File
@@ -15,6 +15,8 @@ serde.workspace = true
[features]
default = []
# Test only: permute the order a placement writes blocks in (ROADMAP D58).
placement_mutants = []
[dev-dependencies]
proptest = "1.12"
crates/r2dec/src/codegen.rs
+17
View File
@@ -1727,6 +1727,23 @@ mod tests {
);
}
/// A gap runs as the void residual call, whose helper traps: what a gap stands for is not
/// computed, so control cannot go on past it as if it were (ADR D58, Merges).
#[test]
fn a_gap_is_written_as_the_residual_call_whose_helper_traps() {
let gap = CStmt::Gap(crate::ast::GapMarker {
kind: crate::ast::GapKind::ValuesNotRendered,
origin: "render::control".to_owned(),
block_addr: 0x1000,
op_idx: 0,
ops: 1,
});
let text = CodeGenerator::new(CodeGenConfig::default()).generate_stmt(&gap);
let residual = crate::prelude::Helper::Residual(crate::prelude::ResidualType::Void);
assert!(text.contains(&format!("{}(", residual.name())), "{text}");
assert!(residual.definition().contains("__builtin_trap();"));
}
/// A declared address defines nothing, and its declaration stands alone.
///
/// The storage class is spelled because there is no definition beside it,
crates/r2dec/src/lib.rs
+6
View File
@@ -18,6 +18,12 @@ pub mod report;
pub mod structure;
pub mod symbol;
/// The knob the write-order gate turns; absent from every build that does not ask for it.
#[cfg(feature = "placement_mutants")]
pub mod placement_mutants {
pub use crate::structure::order::{WriteOrder, with_write_order};
}
pub use crate::codegen::RenderedFunction;
pub use crate::codegen::{CRole, CRoles, Emission, ResidualSite, SourceLine};
pub use crate::ledger::{EffectObligationAudit, EffectObligationDisposition};
crates/r2dec/src/render/calls.rs
+10
View File
@@ -43,6 +43,16 @@ pub(super) struct CallPlan {
pub(super) declared: Option<Prototype>,
}
impl CallPlan {
/// The class the call returns in: its callee's for a tail transfer, else the result it leaves.
pub(super) fn returns(&self) -> Option<&MachineType> {
match &self.tail {
Some(tail) => tail.as_ref().map(|(_, class)| class),
None => self.result.as_ref().map(CallResult::class),
}
}
}
/// What a call leaves its caller.
#[derive(Debug, Clone)]
pub(super) enum CallResult {
crates/r2dec/src/render/control.rs
+102 -310
View File
@@ -1,15 +1,14 @@
//! D1: the function's control, placed by the dominator tree (doc/adr-structure-dominator-tree.md
//! §4), around D2's statements; where D2 has none, a block's operations are one gap.
//! §4), around the text D2 spelled for every block (`spelled`); control decides layout, never a value.
use std::collections::BTreeMap;
use r2ssa::cfg::BlockTerminator;
use super::RenderInput;
use super::values::Values;
use super::spelled::{BlockText, EdgeText, Edges, End, Spelled, values_not_rendered};
use crate::ast::RenderObservationId;
use crate::ast::{CExpr, CStmt, CType, GapKind, GapMarker, SwitchCase};
use crate::prelude::ResidualCause;
use crate::ast::{CExpr, CStmt, SwitchCase};
use crate::structure::place::{EdgeShape, Placement};
/// The written body, its labels, and the block and instruction each marked statement stands for.
@@ -18,8 +17,8 @@ pub(super) struct Written {
pub(super) labels: BTreeMap<u64, String>,
/// By observation index: the block a marked statement was written for, which the certificate reads.
pub(super) blocks: Vec<u64>,
/// By observation index: the instruction, which each emitted line names.
pub(super) addresses: Vec<u64>,
/// By observation index: the instruction, which each emitted line names; none for an edge's copy.
pub(super) addresses: Vec<Option<u64>>,
/// Where the work control stopped the writing, if it did.
pub(super) stopped: Option<crate::control::DecompileExecutionStop>,
}
@@ -27,22 +26,16 @@ pub(super) struct Written {
/// Write every block once, in the region of its immediate dominator or after the loop it leaves.
pub(super) fn write(
input: &RenderInput<'_>,
values: Option<&Values<'_>>,
spelled: Spelled,
placement: &Placement<'_>,
work: &crate::control::DecompileWorkControl<'_>,
) -> Written {
let function = input.function();
let placement = Placement::compute(
function.cfg(),
function.domtree(),
function.natural_loops(),
function.root(),
);
let mut writer = Writer {
input,
values,
spelled,
work,
stopped: None,
placement: &placement,
placement,
labels: BTreeMap::new(),
blocks: Vec::new(),
addresses: Vec::new(),
@@ -51,6 +44,10 @@ pub(super) fn write(
writer.label(addr);
}
let body = writer.place(placement.entry());
debug_assert!(
writer.stopped.is_some() || writer.spelled.is_empty(),
"every block spelled was written"
);
Written {
body,
labels: writer.labels,
@@ -62,16 +59,16 @@ pub(super) fn write(
struct Writer<'w, 'i> {
input: &'w RenderInput<'i>,
values: Option<&'w Values<'i>>,
spelled: Spelled,
work: &'w crate::control::DecompileWorkControl<'w>,
stopped: Option<crate::control::DecompileExecutionStop>,
placement: &'w Placement<'i>,
labels: BTreeMap<u64, String>,
blocks: Vec<u64>,
addresses: Vec<u64>,
addresses: Vec<Option<u64>>,
}
impl<'i> Writer<'_, 'i> {
impl Writer<'_, '_> {
fn label(&mut self, addr: u64) -> String {
let next = self.labels.len();
self.labels
@@ -87,6 +84,15 @@ impl<'i> Writer<'_, 'i> {
/// The same, naming the instruction at `at` as the one the statement's line accounts for.
fn observe_at(&mut self, addr: u64, at: u64, stmt: CStmt) -> CStmt {
self.mark(addr, Some(at), stmt)
}
/// A copy an edge of block `addr` carries: the block's statement, which no instruction writes.
fn observe_copy(&mut self, addr: u64, stmt: CStmt) -> CStmt {
self.mark(addr, None, stmt)
}
fn mark(&mut self, addr: u64, at: Option<u64>, stmt: CStmt) -> CStmt {
let id = RenderObservationId::from_dense_index(self.blocks.len());
self.blocks.push(addr);
self.addresses.push(at);
@@ -121,242 +127,114 @@ impl<'i> Writer<'_, 'i> {
self.stopped = Some(stop);
return CStmt::Block(Vec::new());
}
let text = self.spelled.take(addr);
debug_assert!(text.is_some(), "the block {addr:#x} was spelled");
let mut text = text.unwrap_or_else(|| BlockText::gap(addr));
let mut stmts = Vec::new();
if self.placement.labelled().contains(&addr) {
stmts.push(CStmt::Label(self.label(addr)));
}
let written = stmts.len();
if let Some(values) = self.values {
for (at, stmt) in values.statements(addr) {
stmts.push(self.observe_at(addr, at, stmt));
}
} else {
stmts.extend(self.gap(addr));
for (at, stmt) in std::mem::take(&mut text.statements) {
stmts.push(self.observe_at(addr, at, stmt));
}
// A block that writes nothing and leaves by plain flow still occurs, which the certificate reads.
if stmts.len() == written && !self.ends_observed(addr) {
if stmts.len() == written && matches!(text.end, End::Flow) {
stmts.push(self.observe(addr, CStmt::Empty));
}
stmts.extend(self.arms(addr));
stmts.extend(self.arms(addr, text));
for merge in self.placement.merges_in(addr).to_vec() {
stmts.extend(self.place(merge));
}
CStmt::Block(stmts)
}
/// A block's operations as one gap, where D2 rendered no values.
fn gap(&mut self, addr: u64) -> Option<CStmt> {
let graph = self.input.graph();
let ops = (graph.block_id_for_addr(addr)).map_or(0, |block| graph.op_insts(block).len());
(ops != 0).then(|| {
let gap = CStmt::Gap(GapMarker {
kind: GapKind::ValuesNotRendered,
origin: "render::control".to_owned(),
block_addr: addr,
op_idx: 0,
ops,
});
self.observe(addr, gap)
})
}
/// Whether the block's terminator is written as a statement marked for the block.
fn ends_observed(&self, addr: u64) -> bool {
match self
/// The block's terminator, one transfer per edge.
fn arms(&mut self, addr: u64, mut text: BlockText) -> Vec<CStmt> {
let Some(terminator) = self
.input
.function()
.cfg()
.get_block(addr)
.map(|b| &b.terminator)
{
Some(BlockTerminator::ConditionalBranch {
true_target,
false_target,
}) => true_target != false_target,
Some(
BlockTerminator::Switch { .. }
| BlockTerminator::IndirectBranch
| BlockTerminator::Return
| BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None }
| BlockTerminator::None,
) => true,
_ => false,
}
}
/// The block's terminator, one transfer per edge.
fn arms(&mut self, addr: u64) -> Vec<CStmt> {
let function = self.input.function();
let Some(terminator) = function
.cfg()
.get_block(addr)
.map(|block| block.terminator.clone())
else {
return Vec::new();
};
match terminator {
BlockTerminator::Fallthrough { next: target }
| BlockTerminator::Branch { target }
| BlockTerminator::Call {
fallthrough: Some(target),
..
}
| BlockTerminator::IndirectCall {
fallthrough: Some(target),
}
| BlockTerminator::ConditionalExit { next: target } => self.edge(addr, target),
match (terminator, text.end) {
(
BlockTerminator::Fallthrough { next: target }
| BlockTerminator::Branch { target }
| BlockTerminator::Call {
fallthrough: Some(target),
..
}
| BlockTerminator::IndirectCall {
fallthrough: Some(target),
}
| BlockTerminator::ConditionalExit { next: target },
End::Flow,
) => self.edge(&mut text.edges, addr, target),
// Both arms to one block are one edge, which is how the CFG holds it.
BlockTerminator::ConditionalBranch {
true_target,
false_target,
} if true_target == false_target => self.edge(addr, true_target),
BlockTerminator::ConditionalBranch {
true_target,
false_target,
} => {
let then_body = arm(self.edge(addr, true_target));
let else_body = arm(self.edge(addr, false_target));
let test = self.spelled(addr, Values::condition, &CType::Bool);
(
BlockTerminator::ConditionalBranch {
true_target,
false_target,
},
End::Flow,
) if true_target == false_target => self.edge(&mut text.edges, addr, true_target),
(
BlockTerminator::ConditionalBranch {
true_target,
false_target,
},
End::Test(test),
) => {
let then_body = arm(self.edge(&mut text.edges, addr, true_target));
let else_body = arm(self.edge(&mut text.edges, addr, false_target));
vec![self.observe(addr, CStmt::if_stmt(test, then_body, Some(else_body)))]
}
BlockTerminator::Switch { cases, default } => {
let selector = self.spelled(addr, Values::selector, &super::word(self.input));
self.switch(addr, selector, &cases, default)
(BlockTerminator::Switch { cases, default }, End::Selector(selector)) => {
self.switch(addr, selector, (&cases, default), &mut text.edges)
}
BlockTerminator::IndirectBranch => {
let targets = function.successors(addr);
let cases = targets
.iter()
.map(|target| (*target, *target))
(BlockTerminator::IndirectBranch, End::Selector(selector)) => {
let cases = (self.input.function().successors(addr).into_iter())
.map(|target| (target, target))
.collect::<Vec<_>>();
match cases.is_empty() {
// No stated target: control goes where the facts do not say, so the text traps.
true => vec![self.trap(addr, GapKind::UnresolvedIndirectBranch)],
false => {
self.residual_terminator(addr);
let selector = self.residual(&super::word(self.input));
self.switch(addr, selector, &cases, None)
}
}
self.switch(addr, selector, (&cases, None), &mut text.edges)
}
BlockTerminator::Return => {
let stmt = self.return_stmt(Some(addr));
// The line accounts for the return instruction, wherever in the block it sits.
let at = (self.values)
.and_then(|values| values.terminator_address(addr))
.unwrap_or(addr);
(BlockTerminator::Return, End::Return { stmt, at }) => {
vec![self.observe_at(addr, at, stmt)]
}
BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None }
if self
.values
.is_some_and(|values| values.ends_never_returning(addr)) =>
{
// The call written above is declared never to return, which C ends control at.
Vec::new()
}
// Control never comes back: a void residual traps, so the text ends here as the machine does.
BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None }
| BlockTerminator::None => vec![self.trap(addr, GapKind::TransferNotFollowed)],
}
}
/// A return hands back what D2 spelled, else a residual of the result type, or nothing for `void`.
fn return_stmt(&self, addr: Option<u64>) -> CStmt {
let ty = super::result_type(self.input);
let decided = self
.input
.return_type()
.and_then(r2types::ReturnTypeFact::decided);
let spelled = self
.values
.zip(addr)
.and_then(|(values, addr)| Some((values.returned(addr, decided)?, values, addr)));
if let Some((value, values, addr)) = spelled {
let stmt = CStmt::Return(value);
values.spelled_terminator(addr, &stmt);
return stmt;
}
// The interface proves no result: what this return hands back is unproven, not a gap.
let unproven = self
.input
.return_type()
.is_some_and(r2types::ReturnTypeFact::is_unproven);
if ty != CType::Void
&& let Some(addr) = addr
{
self.residual_terminator(addr);
}
match ty {
CType::Void => CStmt::Return(None),
ty if unproven => CStmt::Return(Some(
crate::prelude::residual(&ty, ResidualCause::UnprovenReturn)
.unwrap_or_else(|| self.residual(&ty)),
)),
ty => CStmt::Return(Some(self.residual(&ty))),
}
}
/// The terminator's operand as D2 spelled it, else a residual of `ty`.
fn spelled(&self, addr: u64, read: fn(&Values<'i>, u64) -> Option<CExpr>, ty: &CType) -> CExpr {
match self
.values
.and_then(|values| Some((read(values, addr)?, values)))
{
Some((expr, values)) => {
values.spelled_terminator(addr, &CStmt::Expr(expr.clone()));
expr
}
None => {
self.residual_terminator(addr);
self.residual(ty)
(
BlockTerminator::IndirectBranch
| BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None }
| BlockTerminator::None,
End::Trap(gap),
) => vec![self.observe(addr, gap)],
(
BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None },
End::NoReturn,
) => Vec::new(),
_ => {
debug_assert!(false, "the block {addr:#x} ends as it was spelled");
vec![self.observe(addr, values_not_rendered(addr, 0))]
}
}
}
/// The terminator of the block at `addr` is a residual the text spells, which the ledger reads.
fn residual_terminator(&self, addr: u64) {
if let Some(values) = self.values {
values.residual_terminator(addr);
}
}
/// A residual of `ty`, or of the machine word where C has no residual of `ty`.
/// A transfer the text cannot make, as a gap whose marker names it: running it traps.
fn trap(&mut self, addr: u64, kind: GapKind) -> CStmt {
self.residual_terminator(addr);
let gap = CStmt::Gap(GapMarker {
kind,
origin: "render::control".to_owned(),
block_addr: addr,
op_idx: 0,
ops: 0,
});
self.observe(addr, gap)
}
fn residual(&self, ty: &CType) -> CExpr {
crate::prelude::residual(ty, ResidualCause::Gap)
.or_else(|| crate::prelude::residual(&super::word(self.input), ResidualCause::Gap))
.expect("a machine word has a residual")
}
/// A switch on `selector`, one arm per target; values reaching the default are its own.
fn switch(
&mut self,
addr: u64,
selector: CExpr,
cases: &[(u64, u64)],
default: Option<u64>,
(cases, default): (&[(u64, u64)], Option<u64>),
edges: &mut Edges,
) -> Vec<CStmt> {
let mut by_target = BTreeMap::<u64, Vec<u64>>::new();
for (value, target) in cases {
@@ -366,10 +244,10 @@ impl<'i> Writer<'_, 'i> {
}
let mut order = by_target.keys().copied().collect::<Vec<_>>();
order.sort_by_key(|target| self.placement.rpo_of(*target));
let default_body = default.map(|target| self.edge(addr, target));
let default_body = default.map(|target| self.edge(edges, addr, target));
let mut switch_cases = Vec::new();
for target in order {
let body = self.edge(addr, target);
let body = self.edge(edges, addr, target);
let values = &by_target[&target];
let (last, leading) = values.split_last().expect("a target has a value");
switch_cases.extend(leading.iter().map(|value| SwitchCase {
@@ -391,35 +269,18 @@ impl<'i> Writer<'_, 'i> {
/// One edge: its merge copies, then the target written here, a `continue`, a `goto`, or a
/// return where it leaves the function.
fn edge(&mut self, from: u64, to: u64) -> Vec<CStmt> {
// An edge out of the function is a tail call D2 writes from its certificate; any other
// transfer the text cannot make, so it traps rather than return.
if self.input.function().cfg().get_block(to).is_none() {
let ty = super::result_type(self.input);
let decided = self
.input
.return_type()
.and_then(r2types::ReturnTypeFact::decided);
if let Some(stmts) = self
.values
.and_then(|values| values.tail_call(from, &ty, decided))
{
fn edge(&mut self, edges: &mut Edges, from: u64, to: u64) -> Vec<CStmt> {
let mut out: Vec<CStmt> = match edges.take(from, to) {
EdgeText::Leaves(stmts) => {
return stmts
.into_iter()
.map(|stmt| self.observe(from, stmt))
.collect();
}
let resolution = self.input.callee_resolution();
let call = (self.values).and_then(|values| values.tail_gap(from, resolution));
return vec![self.trap(from, GapKind::TailTransferNotRendered(call))];
}
let mut out = self
.values
.map(|values| values.copies(from, to))
.unwrap_or_default();
if let Some(values) = self.values {
values.transferred(from);
}
EdgeText::Copies(copies) => (copies.into_iter())
.map(|copy| self.observe_copy(from, copy))
.collect(),
};
match self.placement.edge(from, to) {
EdgeShape::Inline => out.extend(self.place(to)),
EdgeShape::Continue => out.push(CStmt::Continue),
@@ -436,72 +297,3 @@ fn arm(mut stmts: Vec<CStmt>) -> CStmt {
_ => CStmt::Block(stmts),
}
}
/// `stmt` copied as a new occurrence: each marker in it replaced by the one `mint` gives for it.
pub(super) fn remint(
stmt: &CStmt,
mint: &mut dyn FnMut(RenderObservationId) -> RenderObservationId,
) -> CStmt {
let copy = |stmts: &[CStmt],
mint: &mut dyn FnMut(RenderObservationId) -> RenderObservationId| {
stmts
.iter()
.map(|stmt| remint(stmt, mint))
.collect::<Vec<_>>()
};
match stmt {
CStmt::Observed { .. } => {
let ids = stmt
.observation_ids()
.iter()
.map(|id| mint(*id))
.collect::<Vec<_>>();
CStmt::observe_all(ids, remint(stmt.unobserved(), mint))
}
CStmt::Block(stmts) => CStmt::Block(copy(stmts, mint)),
CStmt::If {
cond,
then_body,
else_body,
} => CStmt::If {
cond: cond.clone(),
then_body: Box::new(remint(then_body, mint)),
else_body: else_body.as_ref().map(|body| Box::new(remint(body, mint))),
},
CStmt::For {
init,
cond,
update,
body,
} => CStmt::For {
init: init.as_ref().map(|init| Box::new(remint(init, mint))),
cond: cond.clone(),
update: update.clone(),
body: Box::new(remint(body, mint)),
},
CStmt::While { cond, body } => CStmt::While {
cond: cond.clone(),
body: Box::new(remint(body, mint)),
},
CStmt::DoWhile { body, cond } => CStmt::DoWhile {
body: Box::new(remint(body, mint)),
cond: cond.clone(),
},
CStmt::Switch {
expr,
cases,
default,
} => CStmt::Switch {
expr: expr.clone(),
cases: cases
.iter()
.map(|case| SwitchCase {
value: case.value.clone(),
body: copy(&case.body, mint),
})
.collect(),
default: default.as_ref().map(|body| copy(body, mint)),
},
other => other.clone(),
}
}
crates/r2dec/src/render/mod.rs
+214 -41
View File
@@ -7,6 +7,7 @@ mod frame;
pub(crate) mod globals;
mod input;
pub(crate) mod proof;
mod spelled;
mod tags;
mod terms;
mod values;
@@ -23,6 +24,7 @@ use crate::codegen::{
use crate::control::{DecompileExecutionStop, DecompileWorkControl, DecompileWorkPhase};
use crate::ledger::{ObligationLedger, Outcome};
use crate::structure::certify::StatementRole;
use crate::structure::shape::{Evidence, Occurrences};
/// One function the staged pipeline rendered, and what became of each obligation.
pub struct Rendered {
@@ -72,52 +74,47 @@ pub fn render(
let work = DecompileWorkControl::new(control, DecompileWorkPhase::Structuring);
work.poll()?;
let name = rendered_name_of(input.name(), input.function().root());
let mut c = CFunction::new(name, result_type(input));
let tags = tags::Tags::of(input);
let values = values::Values::new(input, &tags, std::rc::Rc::clone(&c.symbols));
let written = control::write(input, values.as_ref(), &work);
let function = input.function();
let placement = crate::structure::place::Placement::compute(
function.cfg(),
function.domtree(),
function.natural_loops(),
function.root(),
);
let order = crate::structure::place::placed_in_order(function);
let (mut c, values, spelled) = spell_function(input, &tags, &name, &order, &work)?;
let written = control::write(input, spelled, &placement, &work);
if let Some(stop) = written.stopped {
return Err(stop.into());
}
let control::Written {
body,
labels,
mut blocks,
mut addresses,
blocks,
addresses,
..
} = written;
// D1.1: SD's readability rewrites; a block's text they copy is a new occurrence of its block.
let mut fresh = |stmt: &CStmt| {
control::remint(stmt, &mut |id| {
let at = id.index() as usize;
let copy = crate::ast::RenderObservationId::from_dense_index(blocks.len());
blocks.push(blocks[at]);
addresses.push(addresses[at]);
copy
})
};
let mut body = crate::structure::shape::shape(&mut fresh, CStmt::Block(body));
let selections = select(&c, &mut body, &|id| {
blocks.get(id.index() as usize).copied()
});
let never_return = values
.as_ref()
.map(values::Values::never_returning)
.unwrap_or_default();
let role = |stmt: &CStmt| statement_role(stmt, &never_return, &selections);
certify_control(input, &body, &blocks, &labels, &role)?;
let never_returns = |stmt: &CStmt| calls_never_returning(stmt, &never_return);
let distinct_return =
|addr: u64| (values.as_ref()).is_some_and(|values| values.returns_distinct_merge(addr));
let evidence = Evidence {
never_returns: &never_returns,
distinct_return: &distinct_return,
};
let mut occurrences = Occurrences::new(blocks, addresses);
let mut body = structured(input, body, &labels, &evidence, &mut occurrences);
let selections = select(&c, &mut body, &|id| occurrences.block_of(id));
let role = |stmt: &CStmt| statement_role(stmt, &never_returns, &selections);
certify_control(input, &body, &occurrences, &labels, &role)?;
crate::structure::condition::respell_negations(&mut body);
// What remains is linear emission of a certified body, so a stop here keeps what was reached.
let stopped = work.with_phase(DecompileWorkPhase::Rendering).poll().err();
match &values {
Some(values) => {
c.params = values.params();
c.variadic = input.variadic();
c.locals = values.locals();
c.externs = values.externs();
c.extern_objects = values.extern_objects();
}
None => c.params_known = false,
}
declare(&mut c, input, values.as_ref());
c.body = match body {
CStmt::Block(stmts) => stmts,
stmt => vec![stmt],
@@ -143,7 +140,7 @@ pub fn render(
// Each marker names the instruction its statement was written for, so each line names its own.
ready.seal_observation_markers(
&mut crate::codegen::ObservationSealAuthority::staged(),
crate::codegen::ObservationLocations::new(addresses.into_iter().map(Some).collect()),
crate::codegen::ObservationLocations::new(occurrences.into_addresses()),
);
let emission = CodeGenerator::new(CodeGenConfig::default()).emit(&ready, input.ptr_bits());
Ok(Rendered {
@@ -153,6 +150,52 @@ pub fn render(
})
}
/// D2 over every placed block. A late edge's copy is promised to render; when one gaps, spell again with
/// that merge owing one more (ADR D58). Each attempt adds a merge to `forced`, so they end.
fn spell_function<'a>(
input: &RenderInput<'a>,
tags: &tags::Tags,
name: &str,
order: &[u64],
work: &DecompileWorkControl<'_>,
) -> Result<(CFunction, Option<values::Values<'a>>, spelled::Spelled), DecompileExecutionStop> {
let mut forced = r2ssa::dense::IdSet::new(input.graph().values.len());
loop {
let c = CFunction::new(name.to_owned(), result_type(input));
let values = values::Values::new(input, tags, std::rc::Rc::clone(&c.symbols), &forced);
let spelled = spelled::spell(input, values.as_ref(), order, work)?;
let broken = values
.as_ref()
.map(values::Values::broken)
.unwrap_or_default();
if broken.is_empty() {
return Ok((c, values, spelled));
}
r2il::refusal_evidence!(
"late-copy-gapped",
"{name}: {} merges read as variables on a promise a late copy broke; spelling again",
broken.len()
);
broken
.into_iter()
.for_each(|merge| _ = forced.insert(merge));
}
}
/// What the values stage declared for `c`, or that its parameters are unknown.
fn declare(c: &mut CFunction, input: &RenderInput<'_>, values: Option<&values::Values>) {
match values {
Some(values) => {
c.params = values.params();
c.variadic = input.variadic();
c.locals = values.locals();
c.externs = values.externs();
c.extern_objects = values.extern_objects();
}
None => c.params_known = false,
}
}
/// `c` ready to emit: a wide carrier is a struct the unit defines, with the helpers that take it
/// apart, and each declared tag it spells is defined after them.
fn ready_with_carriers(c: CFunction, tags: &tags::Tags) -> crate::codegen::EmissionReadyFunction {
@@ -181,28 +224,80 @@ fn ready_with_carriers(c: CFunction, tags: &tags::Tags) -> crate::codegen::Emiss
ready
}
/// SD's §3 certificate over the shaped body, or the refusal that names its first violation.
fn certify_control(
/// D1.1: SD's stages over the placed body, each kept only while the control certificate still holds
/// (ADR §5): condition chains, `shape`, the latch's copies, and loop rotation.
fn structured(
input: &RenderInput<'_>,
body: Vec<CStmt>,
labels: &BTreeMap<u64, String>,
evidence: &Evidence<'_>,
marks: &mut Occurrences,
) -> CStmt {
let unselected = std::collections::BTreeSet::new();
let certified = |stmt: &CStmt, marks: &Occurrences| {
let role = |stmt: &CStmt| statement_role(stmt, evidence.never_returns, &unselected);
let certificate = control_certificate(input, stmt, marks, labels, &role);
match certificate.ok() {
true => Ok(()),
false => Err(format!(
"{certificate} {:?}",
certificate.violations.first()
)),
}
};
let chains = |body: &mut CStmt, marks: &mut Occurrences| {
crate::structure::condition::collapse(body, marks).collapsed != 0
};
let body = stage(
"condition chains",
CStmt::Block(body),
marks,
&certified,
chains,
);
// The readability rewrites below copy a block's text as a new occurrence of that block.
let body = crate::structure::shape::shape(marks, evidence, body);
let body = stage("latch copies", body, marks, &certified, |body, marks| {
crate::structure::latch::hoist(body, marks) != 0
});
stage("loop rotation", body, marks, &certified, |body, marks| {
crate::structure::rotate::rotate(body, marks) != 0
})
}
/// SD's §3 certificate over a body.
fn control_certificate(
input: &RenderInput<'_>,
body: &CStmt,
blocks: &[u64],
occurrences: &Occurrences,
labels: &BTreeMap<u64, String>,
role: &dyn Fn(&CStmt) -> StatementRole,
) -> Result<(), RenderStop> {
) -> crate::structure::certify::ControlCertificate {
let function = input.function();
let label_block = labels
.iter()
.map(|(addr, name)| (name.as_str(), *addr))
.collect::<BTreeMap<_, _>>();
let certificate = crate::structure::certify::certify(
crate::structure::certify::certify(
body,
function.cfg(),
function.domtree(),
function.root(),
&|id| blocks.get(id.index() as usize).copied(),
occurrences,
&|name| label_block.get(name).copied(),
role,
);
)
}
/// The certificate over the shaped body, or the refusal that names its first violation.
fn certify_control(
input: &RenderInput<'_>,
body: &CStmt,
occurrences: &Occurrences,
labels: &BTreeMap<u64, String>,
role: &dyn Fn(&CStmt) -> StatementRole,
) -> Result<(), RenderStop> {
let certificate = control_certificate(input, body, occurrences, labels, role);
if certificate.ok() {
return Ok(());
}
@@ -212,6 +307,36 @@ fn certify_control(
)))
}
/// The control certificate over a candidate body and its markers: `Err` names the first violation.
type Certified<'a> = dyn Fn(&CStmt, &Occurrences) -> Result<(), String> + 'a;
/// Run one structure stage on `body`; its result stands only if the certificate still holds.
/// `apply` says whether it changed anything, and an unchanged body is not certified again.
fn stage(
name: &str,
body: CStmt,
marks: &mut Occurrences,
certified: &Certified<'_>,
apply: impl FnOnce(&mut CStmt, &mut Occurrences) -> bool,
) -> CStmt {
let issued = marks.issued();
let mut candidate = body.clone();
if !apply(&mut candidate, marks) {
return body;
}
match certified(&candidate, marks) {
Ok(()) => candidate,
Err(why) => {
r2il::refusal_evidence!(
"stage-reverted",
"{name} broke the control certificate: {why}"
);
marks.retract(issued);
body
}
}
}
/// D1.1's selections, each arm converted to `x`'s type as its own assignment converted it.
fn select(
c: &CFunction,
@@ -234,7 +359,7 @@ fn select(
/// What the certificate reads a statement as: a selection the stage recorded, or one ending control.
fn statement_role(
stmt: &CStmt,
never_return: &std::collections::BTreeSet<String>,
never_returns: &dyn Fn(&CStmt) -> bool,
selections: &std::collections::BTreeSet<RenderObservationId>,
) -> StatementRole {
match stmt
@@ -243,7 +368,7 @@ fn statement_role(
.any(|id| selections.contains(id))
{
true => StatementRole::Selects,
false => (traps(stmt) || calls_never_returning(stmt, never_return)).into(),
false => (traps(stmt) || never_returns(stmt)).into(),
}
}
@@ -359,3 +484,51 @@ pub(crate) fn residual_function_for_render_boundary(func_name: &str, reason: &st
sanitize_comment_text(reason)
))
}
#[cfg(test)]
mod tests {
use super::*;
fn id(index: usize) -> RenderObservationId {
RenderObservationId::from_dense_index(index)
}
/// Markers 0 and 1 for blocks 0x10 and 0x20, the second on the body's one statement.
fn start() -> (CStmt, Occurrences) {
let body = CStmt::observe_all([id(1)], CStmt::Break);
(
body,
Occurrences::new(vec![0x10, 0x20], vec![Some(0x11), Some(0x21)]),
)
}
/// A stage that writes a copy of the body under a marker of its own, and one more marker.
fn copies(body: &mut CStmt, marks: &mut Occurrences) -> bool {
*body = marks.fresh(body);
marks.copy_of(id(0));
true
}
#[test]
fn a_stage_that_breaks_the_certificate_leaves_neither_its_text_nor_its_markers() {
let (body, mut marks) = start();
let refuse = |_: &CStmt, _: &Occurrences| Err("broken".to_owned());
let kept = stage("copies", body.clone(), &mut marks, &refuse, copies);
assert_eq!(kept, body);
assert_eq!((marks.issued(), marks.block_of(id(1))), (2, Some(0x20)));
assert_eq!(marks.into_addresses(), [Some(0x11), Some(0x21)]);
}
#[test]
fn a_stage_that_keeps_the_certificate_keeps_its_text_and_its_markers() {
let (body, mut marks) = start();
let accept = |candidate: &CStmt, marks: &Occurrences| {
let copied = candidate.observation_ids().into_owned();
assert_eq!((copied.len(), marks.block_of(copied[0])), (1, Some(0x20)));
Ok(())
};
let kept = stage("copies", body, &mut marks, &accept, copies);
assert_eq!(kept.observation_ids().into_owned(), [id(2)]);
assert_eq!(marks.issued(), 4);
}
}
crates/r2dec/src/render/proof.rs
+8
View File
@@ -207,6 +207,9 @@ pub(crate) enum UnassignedCause {
/// Declared and never assigned, with no value from entry behind it: a
/// result a call left in a register nothing claimed, for one.
Unassigned,
/// Assigned, but where the block that reads it is not behind its definition: some path to the
/// read leaves it unassigned.
NotDominated,
}
impl UnassignedCause {}
@@ -242,6 +245,11 @@ fn note_unassigned_reads(detail: &mut String, unassigned: &[UnassignedRead]) {
"never assigned",
"never assigned",
),
(
UnassignedCause::NotDominated,
"assigned on only some paths to its read",
"assigned on only some paths to their reads",
),
] {
let names = unassigned
.iter()
crates/r2dec/src/render/spelled.rs
+376
View File
@@ -0,0 +1,376 @@
//! D2 to D1: what the values stage spells for every block, once, in the CFG's reverse postorder and
//! before any block is placed (ROADMAP D58), so no write order can reach a verdict.
use std::collections::HashMap;
use r2ssa::cfg::BlockTerminator;
use super::RenderInput;
use super::values::Values;
use crate::ast::{CExpr, CStmt, CType, GapKind, GapMarker};
use crate::control::{DecompileExecutionStop, DecompileWorkControl};
use crate::prelude::ResidualCause;
/// Every placed block's text, taken once by the control that writes it.
pub(super) struct Spelled {
blocks: HashMap<u64, BlockText>,
}
/// What one block says: its statements, how it ends, and the text on each edge it leaves by.
pub(super) struct BlockText {
/// Each statement with the address of the instruction it stands for.
pub(super) statements: Vec<(u64, CStmt)>,
pub(super) edges: Edges,
pub(super) end: End,
}
/// The text on each distinct successor's edge, by target address, each taken once.
pub(super) struct Edges(Vec<(u64, Option<EdgeText>)>);
/// The text an edge carries before the target is written.
pub(super) enum EdgeText {
/// The parallel copy of the target's merges.
Copies(Vec<CStmt>),
/// An edge out of the function: a tail call, or the gap that stands for it.
Leaves(Vec<CStmt>),
}
/// How a block ends, with the operand or statement D2 spelled for it.
pub(super) enum End {
/// Plain flow to the one successor, or none.
Flow,
Test(CExpr),
Selector(CExpr),
Return {
stmt: CStmt,
at: u64,
},
/// A call declared never to return: C ends control there.
NoReturn,
/// A transfer the text cannot make: a gap, which traps.
Trap(CStmt),
}
impl Spelled {
/// The block's text, once.
pub(super) fn take(&mut self, addr: u64) -> Option<BlockText> {
self.blocks.remove(&addr)
}
/// Whether every block's text was taken.
pub(super) fn is_empty(&self) -> bool {
self.blocks.is_empty()
}
}
impl Edges {
/// The text on the edge `from -> to`, once; a missing edge is a gap, never a panic.
pub(super) fn take(&mut self, from: u64, to: u64) -> EdgeText {
let taken = (self.0.binary_search_by_key(&to, |(target, _)| *target))
.ok()
.and_then(|at| self.0[at].1.take());
debug_assert!(
taken.is_some(),
"the edge {from:#x} -> {to:#x} was spelled once"
);
taken.unwrap_or_else(|| EdgeText::Copies(vec![values_not_rendered(from, 0)]))
}
}
impl BlockText {
/// A block nothing spelled, which is one gap.
pub(super) fn gap(addr: u64) -> Self {
Self {
statements: vec![(addr, values_not_rendered(addr, 0))],
edges: Edges(Vec::new()),
end: End::Flow,
}
}
}
/// Spell every block of `order` (the placed blocks in reverse postorder) in one pass, `O(B + E +
/// statements + phi inputs)`: a block follows its dominators and forward predecessors.
pub(super) fn spell(
input: &RenderInput<'_>,
values: Option<&Values<'_>>,
order: &[u64],
work: &DecompileWorkControl<'_>,
) -> Result<Spelled, DecompileExecutionStop> {
let spelling = Spelling {
input,
values,
ty: super::result_type(input),
decided: input
.return_type()
.and_then(r2types::ReturnTypeFact::decided),
};
let mut blocks = HashMap::with_capacity(order.len());
for addr in order.iter().copied() {
work.poll()?;
blocks.insert(addr, spelling.block(addr));
}
Ok(Spelled { blocks })
}
struct Spelling<'s, 'i> {
input: &'s RenderInput<'i>,
values: Option<&'s Values<'i>>,
/// The function's result type, and the one the analysis decided, if it did.
ty: CType,
decided: Option<&'s CType>,
}
impl<'i> Spelling<'_, 'i> {
/// A block's statements, then each edge, then the operand it branches on: the order the Writer
/// used. No verdict depends on it; a test reads no merge its own forward edges pay (ROADMAP D68).
fn block(&self, addr: u64) -> BlockText {
if let Some(values) = self.values {
values.enter(addr);
}
let statements = self.statements(addr);
let Some(terminator) = self
.input
.function()
.cfg()
.get_block(addr)
.map(|block| block.terminator.clone())
else {
return BlockText {
statements,
edges: Edges(Vec::new()),
end: End::Flow,
};
};
let (edges, end) = self.ending(addr, &terminator);
let mut edges = edges
.into_iter()
.map(|(to, text)| (to, Some(text)))
.collect::<Vec<_>>();
edges.sort_by_key(|(to, _)| *to);
BlockText {
statements,
edges: Edges(edges),
end,
}
}
/// A block's operations as D2 spelled them, or as one gap where D2 rendered no values.
fn statements(&self, addr: u64) -> Vec<(u64, CStmt)> {
if let Some(values) = self.values {
return values.statements(addr);
}
let graph = self.input.graph();
let ops = (graph.block_id_for_addr(addr)).map_or(0, |block| graph.op_insts(block).len());
match ops {
0 => Vec::new(),
ops => vec![(addr, values_not_rendered(addr, ops))],
}
}
/// The edges of `addr` in the order they are spelled, and how the block ends.
fn ending(&self, addr: u64, terminator: &BlockTerminator) -> (Vec<(u64, EdgeText)>, End) {
match terminator {
BlockTerminator::Fallthrough { next: target }
| BlockTerminator::Branch { target }
| BlockTerminator::Call {
fallthrough: Some(target),
..
}
| BlockTerminator::IndirectCall {
fallthrough: Some(target),
}
| BlockTerminator::ConditionalExit { next: target } => {
(vec![self.along(addr, *target)], End::Flow)
}
// Both arms to one block are one edge, which is how the CFG holds it.
BlockTerminator::ConditionalBranch {
true_target,
false_target,
} if true_target == false_target => (vec![self.along(addr, *true_target)], End::Flow),
BlockTerminator::ConditionalBranch {
true_target,
false_target,
} => {
let edges = vec![
self.along(addr, *true_target),
self.along(addr, *false_target),
];
let test = self.operand(addr, Values::condition, &CType::Bool);
(edges, End::Test(test))
}
BlockTerminator::Switch { cases, default } => {
let selector = self.operand(addr, Values::selector, &super::word(self.input));
let targets = cases.iter().map(|(_, target)| *target);
let edges = self.edges(addr, *default, targets);
(edges, End::Selector(selector))
}
BlockTerminator::IndirectBranch => {
let targets = self.input.function().successors(addr);
if targets.is_empty() {
// No stated target: control goes where the facts do not say, so the text traps.
let trap = self.trap(addr, GapKind::UnresolvedIndirectBranch);
return (Vec::new(), End::Trap(trap));
}
self.residual_terminator(addr);
let selector = self.residual(&super::word(self.input));
let edges = self.edges(addr, None, targets.into_iter());
(edges, End::Selector(selector))
}
BlockTerminator::Return => {
let stmt = self.return_stmt(addr);
// The line accounts for the return instruction, wherever in the block it sits.
let at = (self.values)
.and_then(|values| values.terminator_address(addr))
.unwrap_or(addr);
(Vec::new(), End::Return { stmt, at })
}
BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None }
if self
.values
.is_some_and(|values| values.ends_never_returning(addr)) =>
{
// The call written above is declared never to return, which C ends control at.
(Vec::new(), End::NoReturn)
}
// Control never comes back: a void residual traps, so the text ends here as the machine does.
BlockTerminator::Call {
fallthrough: None, ..
}
| BlockTerminator::IndirectCall { fallthrough: None }
| BlockTerminator::None => {
let trap = self.trap(addr, GapKind::TransferNotFollowed);
(Vec::new(), End::Trap(trap))
}
}
}
/// The edges of a multiway block: the default first, then the other distinct targets in reverse
/// postorder, which is the order the cases are written in.
fn edges(
&self,
addr: u64,
default: Option<u64>,
targets: impl Iterator<Item = u64>,
) -> Vec<(u64, EdgeText)> {
let mut others = targets
.filter(|target| Some(*target) != default)
.collect::<Vec<_>>();
let graph = self.input.graph();
others.sort_by_key(|target| (graph.block_id_for_addr(*target), *target));
others.dedup();
default
.into_iter()
.chain(others)
.map(|target| self.along(addr, target))
.collect()
}
/// One edge: its merge copies where it enters a block, a tail call or the gap that stands for it
/// where it leaves the function.
fn along(&self, from: u64, to: u64) -> (u64, EdgeText) {
if self.input.function().cfg().get_block(to).is_some() {
let copies = (self.values)
.map(|values| values.copies(from, to))
.unwrap_or_default();
if let Some(values) = self.values {
values.transferred(from);
}
return (to, EdgeText::Copies(copies));
}
// An edge out of the function is a tail call D2 writes from its certificate; any other
// transfer the text cannot make, so it traps rather than return.
let tail = (self.values).and_then(|values| values.tail_call(from, &self.ty, self.decided));
if let Some(stmts) = tail {
return (to, EdgeText::Leaves(stmts));
}
let resolution = self.input.callee_resolution();
let call = (self.values).and_then(|values| values.tail_gap(from, resolution));
let trap = self.trap(from, GapKind::TailTransferNotRendered(call));
(to, EdgeText::Leaves(vec![trap]))
}
/// A return hands back what D2 spelled, else a residual of the result type, or nothing for `void`.
fn return_stmt(&self, addr: u64) -> CStmt {
let spelled = self
.values
.and_then(|values| Some((values.returned(addr, self.decided)?, values)));
if let Some((value, values)) = spelled {
let stmt = CStmt::Return(value);
values.spelled_terminator(addr, &stmt);
return stmt;
}
// The interface proves no result: what this return hands back is unproven, not a gap.
let unproven = self
.input
.return_type()
.is_some_and(r2types::ReturnTypeFact::is_unproven);
if self.ty != CType::Void {
self.residual_terminator(addr);
}
match &self.ty {
CType::Void => CStmt::Return(None),
ty if unproven => CStmt::Return(Some(
crate::prelude::residual(ty, ResidualCause::UnprovenReturn)
.unwrap_or_else(|| self.residual(ty)),
)),
ty => CStmt::Return(Some(self.residual(ty))),
}
}
/// The terminator's operand as D2 spelled it, else a residual of `ty`.
fn operand(&self, addr: u64, read: fn(&Values<'i>, u64) -> Option<CExpr>, ty: &CType) -> CExpr {
match self
.values
.and_then(|values| Some((read(values, addr)?, values)))
{
Some((expr, values)) => {
values.spelled_terminator(addr, &CStmt::Expr(expr.clone()));
expr
}
None => {
self.residual_terminator(addr);
self.residual(ty)
}
}
}
/// The terminator of the block at `addr` is a residual the text spells, which the ledger reads.
fn residual_terminator(&self, addr: u64) {
if let Some(values) = self.values {
values.residual_terminator(addr);
}
}
/// A transfer the text cannot make, as a gap whose marker names it: running it traps.
fn trap(&self, addr: u64, kind: GapKind) -> CStmt {
self.residual_terminator(addr);
CStmt::Gap(GapMarker {
kind,
origin: "render::control".to_owned(),
block_addr: addr,
op_idx: 0,
ops: 0,
})
}
/// A residual of `ty`, or of the machine word where C has no residual of `ty`.
fn residual(&self, ty: &CType) -> CExpr {
crate::prelude::residual(ty, ResidualCause::Gap)
.or_else(|| crate::prelude::residual(&super::word(self.input), ResidualCause::Gap))
.expect("a machine word has a residual")
}
}
/// A block's operations, or an edge's copies, that D2 spelled no values for: one gap.
pub(super) fn values_not_rendered(addr: u64, ops: usize) -> CStmt {
CStmt::Gap(GapMarker {
kind: GapKind::ValuesNotRendered,
origin: "render::control".to_owned(),
block_addr: addr,
op_idx: 0,
ops,
})
}
crates/r2dec/src/render/values.rs
+323 -86
View File
@@ -1,7 +1,7 @@
//! D2: which values the text computes and where (doc/adr-decompiler-rewrite.md, "D2 and D3, as
//! they are built"). Every value is its own local, so a term is valid wherever its producer dominates.
use std::cell::RefCell;
use std::cell::{Cell, RefCell};
use std::collections::BTreeMap;
use std::rc::Rc;
@@ -41,8 +41,19 @@ pub(super) struct Values<'a> {
/// The structure member each cell term is, by term index: the accesses through a parameter
/// that points to its inferred structure.
members: Vec<Option<CExpr>>,
/// The named values a statement written so far assigns: a read of any other is a residual.
assigned: RefCell<Vec<bool>>,
/// By value index: the definitions a named value owes before a read of it is a variable; a merge
/// owes one per early forward edge in, and a back edge pays none (doc/adr-decompiler-rewrite.md, Merges).
owed: RefCell<Vec<u32>>,
/// The merges the last attempt read as variables on a late edge's promise and that edge gapped.
broken: RefCell<Vec<ValueId>>,
/// The merges that owe one definition more than any edge pays, which no read of them clears.
forced: r2ssa::dense::IdSet<ValueId>,
/// The dominator tree and root, which tell a forward edge from a back edge.
dom: &'a r2ssa::domtree::DomTree,
root: u64,
/// The block whose text is being spelled: a name reads as a variable only where its definition
/// has run (doc/adr-decompiler-rewrite.md, "D2's spelling order").
reading: Cell<u64>,
/// By instruction index: whether a statement assigns the instruction's output where it stands.
bound: Vec<bool>,
/// The frame array each stack object a spelled term names is declared as, on first use.
@@ -73,7 +84,9 @@ pub(super) struct Values<'a> {
unplanned: r2ssa::dense::IdMap<InstId, CallGap>,
/// The values a described call's statement assigns.
results: r2ssa::dense::IdSet<ValueId>,
/// One declaration per callee, which every call to it here must agree with.
/// One declaration per callee, the first call's in address order, which every call must state.
prototypes: BTreeMap<String, CExternDecl>,
/// The declarations of the calls written, which the function declares.
externs: RefCell<BTreeMap<String, CExternDecl>>,
/// The text the capture proves at each address, which a passed constant address is.
strings: &'a BTreeMap<u64, String>,
@@ -137,6 +150,28 @@ fn needed<T>(value: Option<T>, inst: InstId, why: CallRefusal) -> Result<T, Call
})
}
/// The declaration a call to a named callee states, `types` its arguments' types, the prototype's
/// where the plan has one: none for a call through a value.
fn extern_decl(plan: &CallPlan, mut types: Vec<CType>, ret_type: CType) -> Option<CExternDecl> {
let calls::Callee::Named { name, address, .. } = &plan.callee else {
return None;
};
if let Some(declared) = &plan.declared {
for (ty, to) in types.iter_mut().zip(&declared.params) {
*ty = to.clone();
}
}
types.truncate(plan.fixed);
Some(CExternDecl {
name: name.clone(),
ret_type,
params: Some(types),
variadic: plan.variadic,
noreturn: plan.noreturn,
address: *address,
})
}
/// Whether an instruction between a producer and its reader is one a moved read or trap may not cross.
fn has_effect(inventory: &SemanticObligationInventory, inst: InstId) -> bool {
match inventory.state_of(inst) {
@@ -755,6 +790,7 @@ impl<'a> Values<'a> {
input: &RenderInput<'a>,
tags: &super::tags::Tags,
symbols: Rc<RefCell<SymbolTable>>,
forced: &r2ssa::dense::IdSet<ValueId>,
) -> Option<Self> {
let artifact: &'a SsaArtifact = input.artifact();
let graph = input.graph();
@@ -789,7 +825,12 @@ impl<'a> Values<'a> {
names: vec![None; graph.values.len()],
declared: vec![false; graph.values.len()],
members: Vec::new(),
assigned: RefCell::new(vec![false; graph.values.len()]),
owed: RefCell::new(vec![1; graph.values.len()]),
broken: RefCell::new(Vec::new()),
forced: forced.clone(),
dom: input.function().domtree(),
root: input.function().root(),
reading: Cell::new(input.function().root()),
bound,
frame: super::frame::Frame::of(artifact),
frame_array: RefCell::new(None),
@@ -807,6 +848,7 @@ impl<'a> Values<'a> {
calls: planned,
unplanned,
results,
prototypes: BTreeMap::new(),
externs: RefCell::new(BTreeMap::new()),
strings: input.string_literals(),
globals: super::globals::Globals::of(input),
@@ -828,6 +870,7 @@ impl<'a> Values<'a> {
.as_ref()
.map(|parameters| parameters.iter().map(|(_, _, class)| *class).collect());
values.declare(input, tags, parameters);
values.prototypes = values.decide_prototypes();
Some(values)
}
@@ -921,7 +964,7 @@ impl<'a> Values<'a> {
SymbolRole::Parameter(index),
);
self.names[value.0 as usize] = Some((name, ty));
self.assigned.get_mut()[value.0 as usize] = true;
self.owed.get_mut()[value.0 as usize] = 0;
if let CType::Pointer(pointee) = &c
&& let CType::Struct(tag) = pointee.as_ref()
{
@@ -930,6 +973,7 @@ impl<'a> Values<'a> {
self.params.push(CParam { ty: c, name });
}
self.members = self.members_of(input, pointing);
let mut forward = vec![None; self.graph.blocks.len()];
for inst in &self.graph.insts {
let Some(output) = inst.output.filter(|_| self.bound[inst.id.0 as usize]) else {
continue;
@@ -946,6 +990,15 @@ impl<'a> Values<'a> {
.borrow_mut()
.declare(name, c.clone(), SymbolRole::Carrier);
self.names[output.0 as usize] = Some((id, ty));
let forced = u32::from(self.forced.contains(output));
self.owed.get_mut()[output.0 as usize] = match inst.payload {
InstPayload::Phi { .. } => {
forced
+ *forward[inst.block.0 as usize]
.get_or_insert_with(|| self.forward_in(input, inst.block))
}
InstPayload::Op(_) => 1,
};
self.locals.get_mut().push(CLocal {
ty: c,
name: id,
@@ -955,6 +1008,35 @@ impl<'a> Values<'a> {
}
}
/// The copies a merge in `block` owes: one per placed predecessor it does not dominate that is
/// spelled before it; a later one pays after the first reads and is verified (`broken`). `O(preds)`.
fn forward_in(&self, input: &RenderInput<'_>, block: r2ssa::BlockId) -> u32 {
let Some(block) = self.graph.block(block) else {
return 1;
};
let (dom, root) = (self.dom, self.root);
let mut forward = (input.function().cfg().predecessors(block.addr).into_iter())
.filter(|pred| {
crate::structure::place::is_placed(dom, root, *pred)
&& !dom.is_back_edge(*pred, block.addr)
})
.collect::<Vec<_>>();
forward.dedup();
let early = (forward.iter())
.filter(|pred| self.graph.block_id_for_addr(**pred) < Some(block.id))
.count();
u32::try_from(if forward.is_empty() { 1 } else { early }).unwrap_or(u32::MAX)
}
/// The merges this attempt read as variables on a late edge's promise that the edge then broke:
/// the next attempt owes them one more, so no read of them is a variable. Sorted, once each.
pub(super) fn broken(&self) -> Vec<ValueId> {
let mut broken = self.broken.borrow().clone();
broken.sort_unstable();
broken.dedup();
broken
}
/// A frame object's address in the function's one frame array (D4), and its extent: the
/// array is declared on first use, aligned as the machine's frame is.
fn object(&self, object: ObjectId) -> Option<terms::Placed> {
@@ -1080,21 +1162,25 @@ impl<'a> Values<'a> {
}
}
/// A name as its class reads it: a parameter declared a pointer is read as its word, and a
/// name no statement written before assigns (its definition a gap) is never assigned.
/// A name as its class reads it: a parameter declared a pointer is read as its word, and a name
/// still owed a definition or defined out of this block's scope is never assigned.
fn read_name(&self, value: ValueId, name: SymbolId, held: &MachineType) -> Option<CExpr> {
if !self
.assigned
.borrow()
.get(value.0 as usize)
.copied()
.unwrap_or(false)
{
let owed = self.owed.borrow().get(value.0 as usize).copied();
let at = self.reading.get();
let scope = || self.artifact.value_defined_before(value, at, usize::MAX);
if let Err(cause) = variable_or_residual(owed == Some(0), scope) {
let read = crate::prelude::residual(
&terms::c_type(held)?,
crate::prelude::ResidualCause::NeverAssigned,
);
self.unassigned_read(value, super::proof::UnassignedCause::Unassigned, held);
if cause == super::proof::UnassignedCause::NotDominated {
r2il::refusal_evidence!(
"read-not-dominated",
"{} is read in {at:#x}, which its definition does not dominate",
self.graph.var(value).display_name()
);
}
self.unassigned_read(value, cause, held);
return read;
}
let var = CExpr::var(name);
@@ -1104,6 +1190,11 @@ impl<'a> Values<'a> {
}
}
/// Spell the text of the block at `addr` from here: its names are read there.
pub(super) fn enter(&self, addr: u64) {
self.reading.set(addr);
}
/// The variable `expr` reads as its word, where the variable is declared `to` already and the
/// word is as wide as `to`, so the reading changes no bit.
fn declared_variable(&self, expr: &CExpr, to: &CType) -> Option<CExpr> {
@@ -1449,13 +1540,14 @@ impl<'a> Values<'a> {
self.mark(inst.id);
self.residual_in(inst.id, stmt);
if let Some(output) = inst.output.filter(|value| self.is_named(*value)) {
self.assigns(output);
self.defines(output);
}
}
fn assigns(&self, value: ValueId) {
if let Some(slot) = self.assigned.borrow_mut().get_mut(value.0 as usize) {
*slot = true;
/// A statement written assigns `value` itself, which owes nothing more.
fn defines(&self, value: ValueId) {
if let Some(slot) = self.owed.borrow_mut().get_mut(value.0 as usize) {
*slot = 0;
}
}
@@ -1498,11 +1590,7 @@ impl<'a> Values<'a> {
let call = self.recursive_call(inst, plan).map(|stmt| vec![stmt]);
return needed(call, inst, CallRefusal::SelfCallNotSpelled);
}
let call = self.call_expr(
inst,
plan,
plan.result.as_ref().map(calls::CallResult::class),
)?;
let call = self.call_expr(inst, plan)?;
needed(
self.assign_call(inst, plan, call),
inst,
@@ -1582,40 +1670,96 @@ impl<'a> Values<'a> {
let mut arguments = Vec::with_capacity(plan.arguments.len());
let mut types = Vec::with_capacity(plan.arguments.len());
for ((argument, class), stacked) in plan.arguments.iter().zip(&plan.stacked) {
let held = needed(self.value_type(*argument), inst, CallRefusal::ArgumentType)?;
// A float the caller stored as bits would travel in a float register once C declares it.
if *stacked && matches!(held, MachineType::Float { .. }) {
return needed(None, inst, CallRefusal::FloatOnStack);
}
let (held, ty) = self.argument_type(inst, *argument, class, *stacked)?;
let operand = self.operand(*argument, inst);
let operand = needed(operand, inst, CallRefusal::ArgumentOperand)?;
let fitted = fit(operand, &held, class);
arguments.push(needed(fitted, inst, CallRefusal::ArgumentFit)?);
types.push(needed(
terms::c_type(class),
inst,
CallRefusal::ArgumentType,
)?);
types.push(ty);
}
Ok((arguments, types))
}
/// The call's expression, returning `ret`: a named callee is declared once, and one reached
/// through its target value is cast to the function type it is called at.
fn call_expr(
/// The type an argument is held at and the C type its class passes it as, from the facts alone.
fn argument_type(
&self,
inst: InstId,
argument: ValueId,
class: &MachineType,
stacked: bool,
) -> Result<(MachineType, CType), CallRefusal> {
let held = needed(self.value_type(argument), inst, CallRefusal::ArgumentType)?;
// A float the caller stored as bits would travel in a float register once C declares it.
if stacked && matches!(held, MachineType::Float { .. }) {
return needed(None, inst, CallRefusal::FloatOnStack);
}
let ty = needed(terms::c_type(class), inst, CallRefusal::ArgumentType)?;
Ok((held, ty))
}
/// The C type the call returns: its declared prototype's, else its result's class, else `void`.
fn return_type(&self, inst: InstId, plan: &CallPlan) -> Result<CType, CallRefusal> {
let ret = match plan.returns() {
Some(class) => needed(terms::c_type(class), inst, CallRefusal::ResultNotSpelled)?,
None => CType::Void,
};
Ok(plan
.declared
.as_ref()
.map_or(ret, |declared| declared.ret.clone()))
}
/// What the call at `inst` declares of its named callee, a function of its plan alone: the same
/// whichever block is written first. `Ok(None)` for a call through a value.
fn declaration(
&self,
inst: InstId,
plan: &CallPlan,
ret: Option<&MachineType>,
) -> Result<CExpr, CallRefusal> {
) -> Result<Option<CExternDecl>, CallRefusal> {
let calls::Callee::Named { .. } = &plan.callee else {
return Ok(None);
};
let mut types = Vec::with_capacity(plan.arguments.len());
for ((argument, class), stacked) in plan.arguments.iter().zip(&plan.stacked) {
types.push(self.argument_type(inst, *argument, class, *stacked)?.1);
}
let ret_type = self.return_type(inst, plan)?;
Ok(extern_decl(plan, types, ret_type))
}
/// Each named callee's declaration: its first call's in address order (block, then place in
/// the block) among the calls that state one, so no write order changes it. O(calls log callees).
fn decide_prototypes(&self) -> BTreeMap<String, CExternDecl> {
let mut first = BTreeMap::<String, ((u64, usize), CExternDecl)>::new();
for (inst, plan) in self.calls.iter() {
let (Some(at), Ok(Some(declaration))) =
(self.graph.inst(inst), self.declaration(inst, plan))
else {
continue;
};
let place = (
self.graph.block_addr_of(inst).unwrap_or(u64::MAX),
at.ordinal,
);
let name = declaration.name.clone();
if first.get(&name).is_none_or(|(held, _)| place < *held) {
first.insert(name, (place, declaration));
}
}
first
.into_iter()
.map(|(name, (_, declaration))| (name, declaration))
.collect()
}
/// The call's expression: a named callee is declared as `prototypes` has it, one reached
/// through a value is cast to the function type it is called at.
fn call_expr(&self, inst: InstId, plan: &CallPlan) -> Result<CExpr, CallRefusal> {
let (mut arguments, mut types) = self.passed(inst, plan)?;
let texts = (plan.arguments.iter())
.map(|(argument, class)| self.passed_text(*argument, class, inst))
.collect();
let mut ret_type = match ret {
Some(class) => needed(terms::c_type(class), inst, CallRefusal::ResultNotSpelled)?,
None => CType::Void,
};
let ret_type = self.return_type(inst, plan)?;
// A declared prototype passes and returns at the types the source states.
if let Some(declared) = &plan.declared {
for ((argument, ty), to) in arguments.iter_mut().zip(&mut types).zip(&declared.params) {
@@ -1626,15 +1770,10 @@ impl<'a> Values<'a> {
};
*ty = to.clone();
}
ret_type = declared.ret.clone();
}
self.literals_passed(&mut arguments, &types, texts, plan.fixed);
let (name, kind, address) = match &plan.callee {
calls::Callee::Named {
name,
kind,
address,
} => (name, *kind, *address),
let (name, kind) = match &plan.callee {
calls::Callee::Named { name, kind, .. } => (name, *kind),
calls::Callee::Through(target) => {
// The printer spells a function type in a cast as the pointer to it: `ret (*)(params)`.
let pointer = CType::Function {
@@ -1655,24 +1794,16 @@ impl<'a> Values<'a> {
if (self.symbols.try_borrow()).map_or(true, |symbols| symbols.by_name(name).is_some()) {
return needed(None, inst, CallRefusal::CalleeShadowed);
}
types.truncate(plan.fixed);
let declaration = CExternDecl {
name: name.clone(),
ret_type,
params: Some(types),
variadic: plan.variadic,
noreturn: plan.noreturn,
address,
};
// One declaration describes every call to a callee, so calls that disagree cannot both be C.
match self.externs.borrow_mut().entry(name.clone()) {
std::collections::btree_map::Entry::Vacant(slot) => {
slot.insert(declaration);
let declaration = extern_decl(plan, types, ret_type);
match (declaration, self.prototypes.get(name)) {
(Some(declaration), Some(decided)) if declaration == *decided => {
self.externs
.borrow_mut()
.entry(name.clone())
.or_insert(declaration);
}
std::collections::btree_map::Entry::Occupied(slot) if *slot.get() != declaration => {
return needed(None, inst, CallRefusal::DeclarationDisagrees);
}
std::collections::btree_map::Entry::Occupied(_) => {}
_ => return needed(None, inst, CallRefusal::DeclarationDisagrees),
}
let callee = CExpr::External {
name: name.clone(),
@@ -1719,8 +1850,7 @@ impl<'a> Values<'a> {
.return_kind();
let stmts = match (returns, tail) {
(r2source::SourceFunctionReturn::Void, _) => {
let call =
(self.call_expr(inst, plan, tail.as_ref().map(|(_, class)| class))).ok()?;
let call = self.call_expr(inst, plan).ok()?;
vec![CStmt::Expr(call), CStmt::Return(None)]
}
(r2source::SourceFunctionReturn::Register { storage }, Some((carried, class)))
@@ -1735,7 +1865,7 @@ impl<'a> Values<'a> {
_ => return None,
},
};
let call = self.call_expr(inst, plan, Some(class)).ok()?;
let call = self.call_expr(inst, plan).ok()?;
// The function's caller reads the bytes r2ssa says its declared result occupies.
let demanded = (self.artifact.machine_context().function_interface())
.and_then(r2ssa::returned_bytes)
@@ -1793,7 +1923,7 @@ impl<'a> Values<'a> {
if let Some(def) = self.graph.def_inst(*result) {
self.mark(def);
}
self.assigns(*result);
self.defines(*result);
Some(vec![assign(*name, value)])
}
@@ -1850,7 +1980,7 @@ impl<'a> Values<'a> {
.iter()
.filter_map(|(named, _)| named.map(|(value, ..)| value))
{
self.assigns(value);
self.defines(value);
}
for value in [halves.0, halves.1].into_iter().flatten() {
if let Some(def) = self.graph.def_inst(value) {
@@ -1896,7 +2026,7 @@ impl<'a> Values<'a> {
if let Some(def) = self.graph.def_inst(result) {
self.mark(def);
}
self.assigns(result);
self.defines(result);
Some(assign(*name, value))
}
@@ -1938,10 +2068,7 @@ impl<'a> Values<'a> {
fn global(&self, at: u64, class: &MachineType, store: bool) -> Option<super::globals::Named> {
self.globals.at(at, class, store, |name| {
name == self.own.name
|| self
.externs
.try_borrow()
.map_or(true, |externs| externs.contains_key(name))
|| self.prototypes.contains_key(name)
|| (self.symbols.try_borrow())
.map_or(true, |symbols| symbols.by_name(name).is_some())
})
@@ -2047,6 +2174,40 @@ impl<'a> Values<'a> {
}
}
/// Whether the return ending `addr` hands back a merge of its block whose edges in copy
/// pairwise-distinct canonical terms: returns the compiler joined (ADR §5).
pub(super) fn returns_distinct_merge(&self, addr: u64) -> bool {
let Some((inst, SSAOp::Return { .. })) = self.terminator(addr) else {
return false;
};
let values = returned_values(self.inventory, inst);
let [value] = values.as_slice() else {
return false;
};
let Some(merge) = (self.graph.def_inst(*value)).and_then(|def| self.graph.inst(def)) else {
return false;
};
let here = self.graph.block_id_for_addr(addr);
if !matches!(merge.payload, InstPayload::Phi { .. })
|| Some(merge.block) != here
|| !self.bound[merge.id.0 as usize]
{
return false;
}
// Two defs of one computation are one value: edges are told apart by canonical term.
let inputs = self.graph.inputs(merge.id);
let mut told = inputs
.iter()
.map(|input| match self.roots.value(*input) {
Some(canonical) => (Some(canonical.canonical), None),
None => (None, Some(*input)),
})
.collect::<Vec<_>>();
told.sort_unstable();
told.dedup();
inputs.len() >= 2 && told.len() == inputs.len() && !inputs.contains(value)
}
/// What the return ending `addr` hands back as `ty`: `None` where the facts do not state it.
pub(super) fn returned(&self, addr: u64, ty: Option<&CType>) -> Option<Option<CExpr>> {
self.attempt();
@@ -2235,6 +2396,8 @@ impl<'a> Values<'a> {
) else {
return Vec::new();
};
let forward = !self.dom.is_back_edge(from, to);
let late = forward && from_id > to_block.id;
let mut copies = Vec::new();
for inst in to_block.insts.iter().filter_map(|id| self.graph.inst(*id)) {
let InstPayload::Phi { predecessors } = &inst.payload else {
@@ -2259,7 +2422,7 @@ impl<'a> Values<'a> {
if input == output
&& let Some((spelled, attempt)) = source
{
self.account_copy(inst.id, output, attempt, &CStmt::Expr(spelled));
self.account_copy(inst.id, output, attempt, (&CStmt::Expr(spelled), false));
continue;
}
copies.push((inst.id, output, source, reads));
@@ -2280,12 +2443,12 @@ impl<'a> Values<'a> {
let (Some((source, attempt)), Some((name, ty))) =
(source, self.names[output.0 as usize].as_ref())
else {
// The edge's gap stands for this merge and the value it could not carry.
self.residual.borrow_mut()[inst.0 as usize] = true;
out.push(unrendered_copy(to));
out.push(self.gapped_copy((inst, output), (from, to), (forward, late)));
continue;
};
self.account_copy(inst, output, attempt, &CStmt::Expr(source.clone()));
let copy = CStmt::Expr(source.clone());
self.promised((from, to), late);
self.account_copy(inst, output, attempt, (&copy, forward && !late));
match clobbers {
false => out.push(self.assign_value(output, *name, source)),
true => {
@@ -2299,13 +2462,53 @@ impl<'a> Values<'a> {
out
}
/// The merge `inst` has a copy on this edge: its instruction and what the source absorbed are
/// rendered, whether or not `stmt` is written.
fn account_copy(&self, inst: InstId, output: ValueId, attempt: Attempt, stmt: &CStmt) {
/// A late edge's copy rendered: the early reads of its merge were right to rest on it.
fn promised(&self, (from, to): (u64, u64), late: bool) {
if late {
r2il::refusal_evidence!(
"late-forward-edge",
"{from:#x} -> {to:#x} pays a merge after {to:#x} was spelled; its reads rest on the copy rendering"
);
}
}
/// The gap that stands for a merge and the value its copy could not carry. On a forward edge it keeps
/// the merge owed; on a late one it breaks the promise the merge's early reads were made on.
fn gapped_copy(
&self,
(inst, output): (InstId, ValueId),
(from, to): (u64, u64),
(forward, late): (bool, bool),
) -> CStmt {
self.residual.borrow_mut()[inst.0 as usize] = true;
if forward {
r2il::refusal_evidence!(
"gapped-forward-copy",
"{from:#x} -> {to:#x} renders no copy, so every read of the merge stays residual"
);
}
if late && !self.forced.contains(output) {
self.broken.borrow_mut().push(output);
}
unrendered_copy(to)
}
/// The merge `inst` has a copy on this edge, rendered whether or not `stmt` is written. Only a copy
/// that `pays` (on an early forward edge) lowers what the merge owes, once per edge.
fn account_copy(
&self,
inst: InstId,
output: ValueId,
attempt: Attempt,
(stmt, pays): (&CStmt, bool),
) {
*self.pending.borrow_mut() = attempt;
self.mark(inst);
self.residual_in(inst, stmt);
self.assigns(output);
if let (true, Some(owed)) = (pays, self.owed.borrow_mut().get_mut(output.0 as usize)) {
debug_assert!(*owed > 0, "a forward edge pays a merge once");
*owed = owed.saturating_sub(1);
}
}
/// A local `<name>_next` of the merge's type, where a cycle of copies goes through.
@@ -2686,8 +2889,42 @@ fn writes_value(op: &SSAOp<ValueId>) -> bool {
&& op.dst().is_some()
}
/// Whether a bound name reads as its variable: its writes are all spelled (`assigned`) and its
/// definition stands behind the read (`scope`, asked only then); else why it is a residual.
fn variable_or_residual(
assigned: bool,
scope: impl FnOnce() -> bool,
) -> Result<(), super::proof::UnassignedCause> {
use super::proof::UnassignedCause;
if !assigned {
Err(UnassignedCause::Unassigned)
} else if !scope() {
Err(UnassignedCause::NotDominated)
} else {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::super::proof::UnassignedCause;
/// A name read where its definition does not dominate is a residual, however many of its writes
/// were spelled before: reverse postorder is not dominance (ROADMAP D68).
#[test]
fn a_name_is_a_variable_only_where_it_is_assigned_and_its_definition_stands_behind_the_read() {
assert_eq!(super::variable_or_residual(true, || true), Ok(()));
assert_eq!(
super::variable_or_residual(true, || false),
Err(UnassignedCause::NotDominated)
);
// A name still owed a write is unassigned, and scope is not asked.
assert_eq!(
super::variable_or_residual(false, || panic!("asked")),
Err(UnassignedCause::Unassigned)
);
}
/// A declared name another language allows can still be a C keyword or reserved spelling.
#[test]
fn a_declared_name_c_reserves_is_no_identifier() {
crates/r2dec/src/render/values/transfer.rs
+1 -1
View File
@@ -255,7 +255,7 @@ impl Values<'_> {
}
for (read, output) in answered {
self.mark(read);
self.assigns(output);
self.defines(output);
}
}
crates/r2dec/src/structure/certify.rs
+878 -145
View File
File diff suppressed because it is too large. Load diff
crates/r2dec/src/structure/condition.rs
+1785
View File
File diff suppressed because it is too large. Load diff
crates/r2dec/src/structure/latch.rs
+730
View File
@@ -0,0 +1,730 @@
//! Latch copies written before the latch's test, so a rotated loop is a `do-while`
//! (`doc/adr-structure-dominator-tree.md` §5, 2026-10-22).
use std::collections::{BTreeMap, BTreeSet};
use super::certify::Placed;
use super::rotate::{continues, flatten, is_break, live, parts};
use crate::ast::{BinaryOp, CExpr, CStmt};
use crate::symbol::SymbolId;
/// Write the copies of each latch before its test where the C text allows; the number of loops.
/// O(statements): the reads are counted in one pass, a loop's from the counts at its entry and exit.
pub(crate) fn hoist(root: &mut CStmt, placed: &dyn Placed) -> usize {
let mut scan = Scan::new(placed);
scan.stmt(root);
let verdicts = scan.verdicts();
if !verdicts.iter().any(Option::is_some) {
return 0;
}
let mut verdicts = verdicts.into_iter();
let written = walk(root, &mut verdicts);
debug_assert!(
verdicts.next().is_none(),
"the walk met each loop the scan did"
);
written
}
/// The reads of the function and, for each `for (;;)` in pre-order, the latch it has and how often
/// each of the latch's copies is read inside the loop.
struct Scan<'p> {
placed: &'p dyn Placed,
/// Reads of each variable so far.
reads: BTreeMap<SymbolId, usize>,
loops: Vec<Option<Latch>>,
}
/// The latch of a loop: whether the arm that stays is the test's true arm, and each variable that
/// arm copies with its reads before the loop, then inside it.
struct Latch {
stay_is_true: bool,
copies: Vec<(SymbolId, usize)>,
}
impl<'p> Scan<'p> {
fn new(placed: &'p dyn Placed) -> Self {
Self {
placed,
reads: BTreeMap::new(),
loops: Vec::new(),
}
}
fn stmt(&mut self, stmt: &CStmt) {
match stmt {
CStmt::Observed { stmt, .. } => self.stmt(stmt),
CStmt::Block(stmts) => stmts.iter().for_each(|stmt| self.stmt(stmt)),
CStmt::If {
cond,
then_body,
else_body,
} => {
self.read(cond);
self.stmt(then_body);
if let Some(else_body) = else_body {
self.stmt(else_body);
}
}
CStmt::While { cond, body } | CStmt::DoWhile { body, cond } => {
self.read(cond);
self.stmt(body);
}
CStmt::Switch {
expr,
cases,
default,
} => {
self.read(expr);
for case in cases {
self.read(&case.value);
case.body.iter().for_each(|stmt| self.stmt(stmt));
}
default.iter().flatten().for_each(|stmt| self.stmt(stmt));
}
CStmt::For {
init: None,
cond: None,
update: None,
body,
} => self.forever(body),
CStmt::For {
init,
cond,
update,
body,
} => {
// A statement in the header holds no loop for `walk` to find.
if let Some(init) = init {
init.visit_exprs(&mut |root| self.read(root));
}
cond.iter().chain(update).for_each(|expr| self.read(expr));
self.stmt(body);
}
other => other.visit_exprs(&mut |root| self.read(root)),
}
}
fn forever(&mut self, body: &CStmt) {
let at = self.loops.len();
let latch = latch_of(body, self.placed);
self.loops.push(latch.map(|(stay_is_true, targets)| {
Latch {
stay_is_true,
copies: (targets.into_iter())
.map(|target| (target, self.count(target)))
.collect(),
}
}));
self.stmt(body);
let reads = &self.reads;
if let Some(latch) = &mut self.loops[at] {
for (target, before) in &mut latch.copies {
*before = reads.get(target).copied().unwrap_or(0) - *before;
}
}
}
fn read(&mut self, expr: &CExpr) {
let reads = &mut self.reads;
reads_of(expr, &mut |symbol| *reads.entry(symbol).or_default() += 1);
}
fn count(&self, symbol: SymbolId) -> usize {
self.reads.get(&symbol).copied().unwrap_or(0)
}
/// For each loop, `Some(stay_is_true)` when its copies may be written before its test: nothing
/// outside the loop reads them, so every read of each is inside.
fn verdicts(&self) -> Vec<Option<bool>> {
(self.loops.iter())
.map(|latch| {
let latch = latch.as_ref()?;
let beyond = |(target, inside): &(SymbolId, usize)| self.count(*target) != *inside;
(!latch.copies.iter().any(beyond)).then_some(latch.stay_is_true)
})
.collect()
}
}
/// Write the loops `verdicts` allow, taken in the order `Scan::stmt` met them.
fn walk(stmt: &mut CStmt, verdicts: &mut impl Iterator<Item = Option<bool>>) -> usize {
match stmt {
CStmt::Observed { stmt, .. } => walk(stmt, verdicts),
CStmt::Block(stmts) => stmts.iter_mut().map(|stmt| walk(stmt, verdicts)).sum(),
CStmt::If {
then_body,
else_body,
..
} => walk(then_body, verdicts) + else_body.as_deref_mut().map_or(0, |e| walk(e, verdicts)),
CStmt::While { body, .. } | CStmt::DoWhile { body, .. } => walk(body, verdicts),
CStmt::Switch { cases, default, .. } => {
let in_cases: usize = (cases.iter_mut())
.flat_map(|case| case.body.iter_mut())
.map(|stmt| walk(stmt, verdicts))
.sum();
in_cases
+ default
.iter_mut()
.flatten()
.map(|stmt| walk(stmt, verdicts))
.sum::<usize>()
}
CStmt::For {
init: None,
cond: None,
update: None,
body,
} => {
let verdict = verdicts.next().expect("the scan met this loop");
let inner = walk(body, verdicts);
let Some(stay_is_true) = verdict else {
return inner;
};
let taken = std::mem::replace(&mut **body, CStmt::Empty);
**body = moved(taken, stay_is_true);
inner + 1
}
CStmt::For { body, .. } => walk(body, verdicts),
_ => 0,
}
}
/// Whether a loop's body ends in a latch whose copies could be written before its test, and if so
/// whether the arm that stays is the test's true arm, and the variables it copies.
fn latch_of(body: &CStmt, placed: &dyn Placed) -> Option<(bool, Vec<SymbolId>)> {
let stmts = live(body);
let (last, before) = stmts.split_last()?;
let (test, on_true, on_false) = parts(last)?;
// The loop leaves on one arm and goes back on the other, which does nothing but copy.
let (stay, stay_is_true) = match (is_break(on_true), is_break(on_false)) {
(false, true) => (on_true, true),
(true, false) => (on_false, false),
_ => return None,
};
// Nothing else goes back to the top, or the loop is not a `do-while` once the copies are out.
if before.is_empty() || before.iter().any(|stmt| continues(stmt)) || !pure(test) {
return None;
}
let targets: BTreeSet<SymbolId> = copies_of(stay, placed)?.into_iter().collect();
// The test reads none of what is written.
let mut read_by_test = false;
reads_of(test, &mut |symbol| {
read_by_test |= targets.contains(&symbol)
});
(!read_by_test).then(|| (stay_is_true, targets.into_iter().collect()))
}
/// The variables an arm assigns, when it is the copies its edge carries (assignments to variables
/// under markers that say so) and nothing else but a closing `continue`.
fn copies_of(arm: &CStmt, placed: &dyn Placed) -> Option<Vec<SymbolId>> {
let mut statements = live(arm);
if matches!(statements.last(), Some(CStmt::Continue)) {
statements.pop();
}
let copy_target = |stmt: &CStmt| {
let ids = stmt.observation_ids();
if ids.is_empty() || !ids.iter().all(|id| placed.edge_copy(*id)) {
return None;
}
match stmt.unobserved() {
CStmt::Expr(CExpr::Binary {
op: BinaryOp::Assign,
left,
..
}) => match &**left {
CExpr::Var(target) => Some(*target),
_ => None,
},
_ => None,
}
};
let targets: Option<Vec<SymbolId>> = statements.into_iter().map(copy_target).collect();
targets.filter(|targets| !targets.is_empty())
}
/// The body with the stay arm's statements ahead of the test and the arm itself empty.
fn moved(body: CStmt, stay_is_true: bool) -> CStmt {
let mut stmts = Vec::new();
flatten(body, &mut stmts);
let Some(CStmt::Observed { ids, stmt }) = stmts.pop() else {
unreachable!("the test was found last");
};
let CStmt::If {
cond,
then_body,
else_body: Some(else_body),
} = *stmt
else {
unreachable!("the test was an if with both arms");
};
let (stay, leave) = match stay_is_true {
true => (*then_body, *else_body),
false => (*else_body, *then_body),
};
let mut copies = Vec::new();
flatten(stay, &mut copies);
copies.retain(|stmt| !matches!(stmt, CStmt::Continue));
stmts.extend(copies);
let (on_true, on_false) = match stay_is_true {
true => (CStmt::Empty, leave),
false => (leave, CStmt::Empty),
};
stmts.push(CStmt::observe_all(
ids.iter(),
CStmt::if_stmt(cond, on_true, Some(on_false)),
));
CStmt::Block(stmts)
}
/// Whether evaluating a test only reads: no call, no assignment, no increment.
fn pure(expr: &CExpr) -> bool {
let mut calls = false;
let mut pending = vec![expr];
while let Some(node) = pending.pop() {
calls |= matches!(node.unobserved(), CExpr::Call { .. });
pending.extend(node.children());
}
!calls && !super::certify::writes(expr)
}
/// Each read of a variable in `expr`: a plain assignment's target is written, not read.
fn reads_of(expr: &CExpr, read: &mut impl FnMut(SymbolId)) {
#[cfg(test)]
COUNTED.with(|counted| counted.set(counted.get() + 1));
match expr.unobserved() {
CExpr::Var(symbol) => read(*symbol),
CExpr::Binary {
op: BinaryOp::Assign,
left,
right,
} if matches!(left.unobserved(), CExpr::Var(_)) => reads_of(right, read),
other => {
for child in other.children() {
reads_of(child, read);
}
}
}
}
#[cfg(test)]
thread_local! {
/// The expressions read for the variables they name, in this thread.
static COUNTED: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
#[cfg(test)]
mod tests {
use super::*;
use crate::ast::{RenderObservationId, SwitchCase, UnaryOp, test_render_observation_id};
use crate::symbol::SymbolTable;
struct Fixture {
symbols: SymbolTable,
}
impl Fixture {
fn new() -> Self {
Self {
symbols: SymbolTable::new(),
}
}
fn var(&mut self, name: &str) -> CExpr {
CExpr::var(self.symbols.declare_or_reuse(name))
}
}
fn marker(block: u32, stmt: CStmt) -> CStmt {
CStmt::observe_one(test_render_observation_id(block), stmt)
}
/// Markers from this one on are copies an edge carries.
const COPY: u32 = 0x1000;
struct Edges;
impl Placed for Edges {
fn block_of(&self, id: RenderObservationId) -> Option<u64> {
Some(u64::from(id.index() % COPY))
}
fn statement_of(&self, id: RenderObservationId) -> Option<RenderObservationId> {
Some(id)
}
fn written(&self) -> usize {
0
}
fn edge_copy(&self, id: RenderObservationId) -> bool {
id.index() >= COPY
}
}
fn write(target: &CExpr, value: CExpr) -> CStmt {
CStmt::Expr(CExpr::assign(target.clone(), value))
}
/// An assignment the latch's edge carries.
fn assign(target: &CExpr, value: CExpr) -> CStmt {
marker(COPY, write(target, value))
}
fn latch(test: CExpr, stay: Vec<CStmt>) -> CStmt {
marker(
0x30,
CStmt::if_stmt(
test,
CStmt::Block(stay),
Some(CStmt::Block(vec![CStmt::Break])),
),
)
}
fn forever(stmts: Vec<CStmt>) -> CStmt {
CStmt::Block(vec![CStmt::For {
init: None,
cond: None,
update: None,
body: Box::new(CStmt::Block(stmts)),
}])
}
fn work(block: u32) -> CStmt {
marker(block, CStmt::Expr(CExpr::int(i64::from(block))))
}
/// The loop of `forever`, after `hoist`: its body's statements.
fn body_after(root: CStmt) -> (Vec<CStmt>, usize) {
body_after_at(root, 0)
}
/// The same, for the loop that is statement `at` of the outer block.
fn body_after_at(mut root: CStmt, at: usize) -> (Vec<CStmt>, usize) {
let count = hoist(&mut root, &Edges);
let CStmt::Block(outer) = root else {
panic!("a block");
};
let CStmt::For { body, .. } = &outer[at] else {
panic!("a for");
};
let CStmt::Block(stmts) = &**body else {
panic!("a block body");
};
(stmts.clone(), count)
}
/// `depth` loops inside one another, each with a copy that nothing outside it reads.
fn nested(f: &mut Fixture, depth: usize) -> CStmt {
let (a, n) = (f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let mut inner = None;
for level in (0..depth).rev() {
let x = f.var(&format!("x{level}"));
let mut body = vec![work(0x20)];
body.extend(inner.take());
body.push(latch(
test.clone(),
vec![assign(&x, a.clone()), CStmt::Continue],
));
inner = Some(CStmt::For {
init: None,
cond: None,
update: None,
body: Box::new(CStmt::Block(body)),
});
}
CStmt::Block(inner.into_iter().collect())
}
/// The loops written and the expressions read for `hoist` of `depth` nested loops.
fn read_for(depth: usize) -> (usize, usize) {
let mut root = nested(&mut Fixture::new(), depth);
let before = COUNTED.with(std::cell::Cell::get);
let written = hoist(&mut root, &Edges);
(written, COUNTED.with(std::cell::Cell::get) - before)
}
#[test]
fn nested_loops_are_read_once_however_deep_they_go() {
// Doubling the depth at most doubles the expressions read; reading each loop's whole body,
// inner loops included, would quadruple them.
let (few, many) = (read_for(50), read_for(100));
assert_eq!(
(few.0, many.0),
(50, 100),
"every level writes its copy first"
);
assert!(
many.1 <= 2 * few.1,
"{} read for 50 loops, {} for 100",
few.1,
many.1
);
}
fn looped(body: Vec<CStmt>) -> CStmt {
CStmt::For {
init: None,
cond: None,
update: None,
body: Box::new(CStmt::Block(body)),
}
}
/// The statements of the body of a `for`.
fn body_of(stmt: &CStmt) -> &[CStmt] {
let CStmt::For { body, .. } = stmt else {
panic!("a for: {stmt:?}");
};
let CStmt::Block(stmts) = &**body else {
panic!("a block body");
};
stmts
}
/// Whether the latch ending `body` still copies on its staying arm.
fn copies_on_arm(body: &[CStmt]) -> bool {
let Some(CStmt::Observed { stmt, .. }) = body.last() else {
panic!("a marked latch");
};
let CStmt::If { then_body, .. } = &**stmt else {
panic!("an if");
};
!matches!(&**then_body, CStmt::Empty)
}
#[test]
fn a_copy_read_in_the_enclosing_loop_outside_the_inner_one_stays_on_its_arm() {
let mut f = Fixture::new();
let (x, y, a, n) = (f.var("x"), f.var("y"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let inner = || {
looped(vec![
work(0x20),
latch(test.clone(), vec![assign(&x, a.clone()), CStmt::Continue]),
])
};
let outer_latch = || latch(test.clone(), vec![assign(&y, a.clone()), CStmt::Continue]);
let read_x = || CStmt::Expr(x.clone());
// Read after the inner loop, and before it: only the outer loop's copy is written first.
for body in [
vec![inner(), read_x(), outer_latch()],
vec![read_x(), inner(), outer_latch()],
] {
let mut root = CStmt::Block(vec![looped(body)]);
assert_eq!(hoist(&mut root, &Edges), 1);
let CStmt::Block(top) = &root else {
panic!("a block");
};
let outer = body_of(&top[0]);
let inner = outer.iter().find(|stmt| matches!(stmt, CStmt::For { .. }));
assert!(copies_on_arm(body_of(inner.expect("the inner loop"))));
assert!(!copies_on_arm(outer), "{outer:?}");
}
// Read nowhere else: both write their copies first.
let mut root = CStmt::Block(vec![looped(vec![inner(), outer_latch()])]);
assert_eq!(hoist(&mut root, &Edges), 2);
}
#[test]
fn the_scan_counts_every_read_the_whole_tree_holds() {
// Each container reads a variable of its own in its header, so one the scan skips shows.
let mut f = Fixture::new();
let names = [
"i", "w", "d", "fi", "fc", "fu", "s", "c", "b", "r", "x", "o",
];
let v: Vec<CExpr> = names.iter().map(|name| f.var(name)).collect();
let stmt = |expr: &CExpr| Box::new(CStmt::Expr(expr.clone()));
let tree = CStmt::Block(vec![
CStmt::if_stmt(
v[0].clone(),
CStmt::Expr(v[10].clone()),
Some(CStmt::Expr(v[11].clone())),
),
CStmt::While {
cond: v[1].clone(),
body: stmt(&v[10]),
},
CStmt::DoWhile {
body: stmt(&v[10]),
cond: v[2].clone(),
},
CStmt::For {
init: Some(stmt(&v[3])),
cond: Some(v[4].clone()),
update: Some(v[5].clone()),
body: stmt(&v[10]),
},
CStmt::Switch {
expr: v[6].clone(),
cases: vec![SwitchCase {
value: v[7].clone(),
body: vec![CStmt::Expr(v[10].clone())],
}],
default: Some(vec![CStmt::Return(Some(v[9].clone()))]),
},
CStmt::Expr(CExpr::assign(v[8].clone(), v[10].clone())),
]);
let mut whole = BTreeMap::new();
tree.visit_exprs(&mut |root| reads_of(root, &mut |s| *whole.entry(s).or_insert(0) += 1));
let mut scan = Scan::new(&Edges);
scan.stmt(&tree);
assert_eq!(scan.reads, whole);
assert_eq!(whole.len(), 11, "every name but the assigned one is read");
}
#[test]
fn copies_a_test_does_not_read_are_written_before_it() {
// for (;;) { work; if (a != n) { x = a; } else { break; } } with `x` read nowhere else.
let mut f = Fixture::new();
let (x, a, n) = (f.var("x"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let root = forever(vec![
work(0x20),
latch(test, vec![assign(&x, a), CStmt::Continue]),
]);
let (stmts, count) = body_after(root);
assert_eq!(count, 1);
// work; x = a; if (...) {} else { break; }
assert_eq!(stmts.len(), 3, "{stmts:?}");
assert!(matches!(stmts[1].unobserved(), CStmt::Expr(_)));
let CStmt::Observed { stmt, .. } = &stmts[2] else {
panic!("the test stays marked");
};
let CStmt::If { then_body, .. } = &**stmt else {
panic!("an if");
};
assert!(matches!(&**then_body, CStmt::Empty));
}
#[test]
fn a_copy_that_carries_its_blocks_marker_moves_with_it() {
// An edge's copy is a statement of its block: the marker the certificate counts goes with it.
let mut f = Fixture::new();
let (x, a, n) = (f.var("x"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let copy = assign(&x, a);
let stay = vec![copy.clone(), CStmt::Continue];
let (stmts, count) = body_after(forever(vec![work(0x20), latch(test, stay)]));
assert_eq!(count, 1);
assert_eq!(stmts[1], copy);
}
#[test]
fn an_assignment_a_block_writes_is_no_copy_an_edge_carries() {
// The arm is the text of another block that assigns; hoisted, that block would run before the test.
let mut f = Fixture::new();
let (x, a, n) = (f.var("x"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let own = marker(0x30, write(&x, a));
let root = forever(vec![work(0x20), latch(test, vec![own, CStmt::Continue])]);
assert_eq!(body_after(root).1, 0);
}
#[test]
fn the_stay_arm_on_the_false_side_is_emptied_too() {
let mut f = Fixture::new();
let (x, a) = (f.var("x"), f.var("a"));
let test = CExpr::binary(BinaryOp::Eq, a.clone(), CExpr::int(0));
let latch = marker(
0x30,
CStmt::if_stmt(
test,
CStmt::Block(vec![CStmt::Break]),
Some(CStmt::Block(vec![assign(&x, a)])),
),
);
let (stmts, count) = body_after(forever(vec![work(0x20), latch]));
assert_eq!(count, 1);
let CStmt::Observed { stmt, .. } = &stmts[2] else {
panic!("marked");
};
let CStmt::If { else_body, .. } = &**stmt else {
panic!("an if");
};
assert!(matches!(else_body.as_deref(), Some(CStmt::Empty)));
}
#[test]
fn a_copy_the_test_reads_stays_on_the_arm() {
// The test reads `x`, which the copy writes: after the move it would see the new value.
let mut f = Fixture::new();
let (x, a) = (f.var("x"), f.var("a"));
let test = CExpr::binary(BinaryOp::Ne, x.clone(), CExpr::int(0));
let root = forever(vec![work(0x20), latch(test, vec![assign(&x, a)])]);
let (_, count) = body_after(root);
assert_eq!(count, 0);
}
#[test]
fn a_copy_something_after_the_loop_reads_stays_on_the_arm() {
// `x` is read after the loop; the exit would see the copy it never made.
let mut f = Fixture::new();
let (x, a, n) = (f.var("x"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let mut root = forever(vec![work(0x20), latch(test, vec![assign(&x, a)])]);
let CStmt::Block(outer) = &mut root else {
panic!("a block");
};
outer.push(CStmt::Return(Some(x)));
let (_, count) = body_after(root);
assert_eq!(count, 0);
}
#[test]
fn a_write_before_the_loop_is_no_read_after_it() {
let mut f = Fixture::new();
let (x, a, n) = (f.var("x"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
let CStmt::Block(mut outer) = forever(vec![work(0x20), latch(test, vec![assign(&x, a)])])
else {
panic!("a block");
};
outer.insert(0, write(&x, CExpr::int(0)));
outer.push(CStmt::Return(Some(CExpr::int(1))));
let (_, count) = body_after_at(CStmt::Block(outer), 1);
assert_eq!(count, 1);
}
#[test]
fn a_test_that_calls_or_writes_keeps_its_copies_where_they_are() {
let mut f = Fixture::new();
let (x, a) = (f.var("x"), f.var("a"));
let call = CExpr::call(
CExpr::var(f.symbols.declare_or_reuse("probe")),
vec![a.clone()],
);
let root = forever(vec![work(0x20), latch(call, vec![assign(&x, a.clone())])]);
assert_eq!(body_after(root).1, 0);
let bump = CExpr::unary(UnaryOp::PostInc, a.clone());
let root = forever(vec![work(0x20), latch(bump, vec![assign(&x, a)])]);
assert_eq!(body_after(root).1, 0);
}
#[test]
fn a_latch_that_does_more_than_copy_or_a_continue_before_it_is_left_alone() {
let mut f = Fixture::new();
let (x, a, n) = (f.var("x"), f.var("a"), f.var("n"));
let test = CExpr::binary(BinaryOp::Ne, a.clone(), n);
// The arm also stores through a pointer.
let store = CStmt::Expr(CExpr::assign(
CExpr::Deref(Box::new(a.clone())),
CExpr::int(1),
));
let root = forever(vec![
work(0x20),
latch(test.clone(), vec![assign(&x, a.clone()), store]),
]);
assert_eq!(body_after(root).1, 0);
// A `continue` before the latch reaches the top, which the copies would skip.
let early = CStmt::If {
cond: CExpr::int(1),
then_body: Box::new(CStmt::Continue),
else_body: None,
};
let root = forever(vec![work(0x20), early, latch(test, vec![assign(&x, a)])]);
assert_eq!(body_after(root).1, 0);
}
}
crates/r2dec/src/structure/mod.rs
+7
View File
@@ -3,5 +3,12 @@
//! text against the CFG, and the rewrites that shape it afterwards.
pub(crate) mod certify;
pub(crate) mod condition;
pub(crate) mod latch;
pub(crate) mod order;
pub(crate) mod place;
pub(crate) mod rotate;
pub(crate) mod shape;
#[cfg(test)]
pub(crate) mod support;
pub(crate) mod tails;
crates/r2dec/src/structure/order.rs
+112
View File
@@ -0,0 +1,112 @@
//! The write order of a placement, which the order-invariance gate permutes (ROADMAP D58); without
//! the `placement_mutants` feature there is one order and no branch.
#[cfg(not(feature = "placement_mutants"))]
pub(crate) use off::WriteOrder;
#[cfg(feature = "placement_mutants")]
pub use on::{WriteOrder, with_write_order};
// Compiled under test with the feature too, so the production order is pinned wherever tests run.
#[cfg(any(test, not(feature = "placement_mutants")))]
mod off {
/// The only order: the facts' own.
#[derive(Clone, Copy)]
pub(crate) struct WriteOrder;
impl WriteOrder {
pub(crate) const fn current() -> Self {
Self
}
pub(crate) const fn inlines_exits(self) -> bool {
true
}
pub(crate) fn exits(self, _: &mut [u64]) {}
pub(crate) fn merges(self, _: &mut [u64]) {}
}
}
#[cfg(feature = "placement_mutants")]
mod on {
use std::cell::Cell;
use std::sync::OnceLock;
/// How a placement orders what it writes. Every verdict D2 gives must be the same under all four.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum WriteOrder {
/// Reverse postorder, exits inlined where their edge leaves the loop.
Natural,
/// O1: no exit is inlined, so each is written after its loop.
KeepExits,
/// O2: the exits of each loop in reverse of their reverse postorder.
ExitsReversed,
/// O3: the merges of each region in reverse of their reverse postorder.
MergesReversed,
}
thread_local! {
static OVERRIDE: Cell<Option<WriteOrder>> = const { Cell::new(None) };
}
/// Run `body` with placements ordered by `order` on this thread, then restore what was set.
pub fn with_write_order<T>(order: WriteOrder, body: impl FnOnce() -> T) -> T {
struct Restore(Option<WriteOrder>);
impl Drop for Restore {
fn drop(&mut self) {
OVERRIDE.with(|slot| slot.set(self.0));
}
}
let _restore = Restore(OVERRIDE.with(|slot| slot.replace(Some(order))));
body()
}
/// `R2SLEIGH_WRITE_ORDER` = `o1`, `o2` or `o3`, read once, for the corpus sweep.
fn from_env() -> WriteOrder {
static ENV: OnceLock<WriteOrder> = OnceLock::new();
*ENV.get_or_init(|| match std::env::var("R2SLEIGH_WRITE_ORDER").as_deref() {
Ok("o1") => WriteOrder::KeepExits,
Ok("o2") => WriteOrder::ExitsReversed,
Ok("o3") => WriteOrder::MergesReversed,
_ => WriteOrder::Natural,
})
}
impl WriteOrder {
pub(crate) fn current() -> Self {
OVERRIDE.with(Cell::get).unwrap_or_else(from_env)
}
pub(crate) fn inlines_exits(self) -> bool {
self != Self::KeepExits
}
pub(crate) fn exits(self, list: &mut [u64]) {
if self == Self::ExitsReversed {
list.reverse();
}
}
pub(crate) fn merges(self, list: &mut [u64]) {
if self == Self::MergesReversed {
list.reverse();
}
}
}
}
#[cfg(test)]
mod tests {
/// The order every production build writes in is the facts' own: exits inlined, no list permuted.
#[test]
fn production_writes_in_the_facts_own_order() {
let production = super::off::WriteOrder::current();
let mut list = [3, 1, 2];
production.exits(&mut list);
assert_eq!(list, [3, 1, 2]);
production.merges(&mut list);
assert_eq!(list, [3, 1, 2]);
assert!(production.inlines_exits());
}
}
crates/r2dec/src/structure/place.rs
+412 -16
View File
@@ -1,16 +1,13 @@
//! Dominator-tree placement, `doc/adr-structure-dominator-tree.md` §4.
//!
//! Every block is written once, in the region of its immediate dominator or
//! in the exit list of the outermost loop it leaves; every edge is written
//! once, as adjacency, an `if` or `switch` arm, `continue`, or `goto`. The
//! shape is total for any CFG, and the certificate in `certify.rs` is what
//! says the text it produced is the machine's graph.
//! Dominator-tree placement, `doc/adr-structure-dominator-tree.md` §4: every block is written once,
//! and every edge once; the certificate in `certify.rs` says the text is the machine's graph.
use std::collections::{BTreeSet, HashMap};
use std::collections::{BTreeSet, HashMap, HashSet};
use r2ssa::domtree::DomTree;
use r2ssa::natural_loops::NaturalLoop;
use super::order::WriteOrder;
/// Where every block and every edge of a function goes, decided before any
/// text is written.
pub(crate) struct Placement<'f> {
@@ -29,6 +26,8 @@ pub(crate) struct Placement<'f> {
merge_children: HashMap<u64, Vec<u64>>,
/// Blocks written after each loop, in reverse postorder.
exits: HashMap<usize, Vec<u64>>,
/// Exit blocks written where their one edge leaves the loop, not after it.
inlined: BTreeSet<u64>,
/// Blocks some edge reaches by `goto`.
labelled: BTreeSet<u64>,
}
@@ -51,10 +50,9 @@ impl<'f> Placement<'f> {
// no place.
let placed_set: BTreeSet<u64> = cfg
.block_addrs()
.filter(|addr| *addr == entry || dom.idom(*addr).is_some())
.filter(|addr| is_placed(dom, entry, *addr))
.collect();
let placed = |addr: u64| placed_set.contains(&addr);
let is_back_edge = |from: u64, to: u64| dom.dominates(to, from);
let loops = natural_loops.outermost_first();
let header_of: HashMap<u64, usize> = loops
@@ -77,7 +75,7 @@ impl<'f> Placement<'f> {
let forward = cfg
.predecessors(addr)
.into_iter()
.filter(|pred| placed(*pred) && !is_back_edge(*pred, addr))
.filter(|pred| placed(*pred) && !dom.is_back_edge(*pred, addr))
.count();
if forward >= 2 {
merge.insert(addr);
@@ -94,6 +92,7 @@ impl<'f> Placement<'f> {
merge,
merge_children: HashMap::new(),
exits: HashMap::new(),
inlined: BTreeSet::new(),
labelled: BTreeSet::new(),
};
// Anchors: a block leaving a loop its dominator is in goes after the
@@ -115,11 +114,17 @@ impl<'f> Placement<'f> {
}
let rpo = placement.rpo.clone();
let by_rpo = |list: &mut Vec<u64>| list.sort_by_key(|addr| rpo.get(addr).copied());
let order = WriteOrder::current();
for list in placement.merge_children.values_mut() {
by_rpo(list);
order.merges(list);
}
for list in placement.exits.values_mut() {
by_rpo(list);
order.exits(list);
}
if order.inlines_exits() {
placement.inline_exits(&closed_subtrees(cfg, dom, &placed_set));
}
for from in cfg.block_addrs() {
if !placed(from) {
@@ -180,22 +185,111 @@ impl<'f> Placement<'f> {
/// How the edge `from -> to` is written.
pub(crate) fn edge(&self, from: u64, to: u64) -> EdgeShape {
if self.dom.dominates(to, from) {
if self.dom.is_back_edge(from, to) {
return if self.innermost_loop(from) == self.header_of.get(&to).copied() {
EdgeShape::Continue
} else {
EdgeShape::Goto
};
}
if self.dom.idom(to) == Some(from)
&& !self.merge.contains(&to)
&& self.exit_of(from, to).is_none()
{
if self.inlines(from, to) {
EdgeShape::Inline
} else {
EdgeShape::Goto
}
}
/// Whether `to` is written where the edge from `from` reaches it: its only forward way in,
/// and no loop left, or a loop left by a block that [`Self::inline_exits`] kept in.
fn inlines(&self, from: u64, to: u64) -> bool {
self.dom.idom(to) == Some(from)
&& !self.merge.contains(&to)
&& (self.exit_of(from, to).is_none() || self.inlined.contains(&to))
}
/// An exit with one way in and a closed subtree goes where its edge leaves, unless `break`
/// reaches it as the first exit of one loop (ADR §4). `O(exits)`.
fn inline_exits(&mut self, closed: &HashSet<u64>) {
let loops = self.exits.keys().copied().collect::<Vec<_>>();
for index in loops {
let mut broken_to = false;
let listed = std::mem::take(self.exits.entry(index).or_default());
let (kept, inlined): (Vec<u64>, Vec<u64>) = (listed.into_iter())
.partition(|exit| self.stays_after(index, *exit, closed, &mut broken_to));
self.inlined.extend(inlined);
self.exits.insert(index, kept);
}
}
/// Whether `exit` stays after loop `index`: it has a second way in, or a subtree an edge
/// leaves, or `break` reaches it as the first such exit of the one loop its edge leaves.
fn stays_after(
&self,
index: usize,
exit: u64,
closed: &HashSet<u64>,
broken: &mut bool,
) -> bool {
let sole = !self.merge.contains(&exit) && closed.contains(&exit);
let one_level =
(self.dom.idom(exit)).is_some_and(|from| self.innermost_loop(from) == Some(index));
let stays = !sole || (one_level && !*broken);
*broken |= stays;
stays
}
}
/// Whether the entry reaches `addr`: only a reached block has a place.
pub(crate) fn is_placed(dom: &DomTree, entry: u64, addr: u64) -> bool {
addr == entry || dom.idom(addr).is_some()
}
/// The placed blocks in the CFG's reverse postorder, which is the order every verdict is spelled in
/// (`render::spelled`): a block comes after the blocks that dominate it. `O(B)`.
pub(crate) fn placed_in_order(function: &r2ssa::SSAFunction) -> Vec<u64> {
let (dom, root) = (function.domtree(), function.root());
let order = (function.block_addrs().iter().copied())
.filter(|addr| is_placed(dom, root, *addr))
.collect::<Vec<_>>();
debug_assert!(
order.iter().copied().eq(function
.cfg()
.reverse_postorder()
.into_iter()
.filter(|addr| is_placed(dom, root, *addr))),
"the SSA function's blocks are the CFG's reverse postorder"
);
order
}
/// The blocks whose dominator subtree no edge leaves: the extremes of the successors' preorder
/// positions over the subtree lie in its interval. `O(B + E)`.
fn closed_subtrees(cfg: &r2ssa::cfg::CFG, dom: &DomTree, placed: &BTreeSet<u64>) -> HashSet<u64> {
let order = dom.preorder();
let position: HashMap<u64, usize> = order.iter().enumerate().map(|(at, b)| (*b, at)).collect();
let mut last = (0..order.len()).collect::<Vec<_>>();
let mut lowest = last.clone();
let mut highest = last.clone();
for (at, block) in order.iter().enumerate() {
for to in cfg.successors(*block) {
if let Some(to) = placed.contains(&to).then(|| position[&to]) {
lowest[at] = lowest[at].min(to);
highest[at] = highest[at].max(to);
}
}
}
for (at, block) in order.iter().enumerate().rev() {
let Some(parent) = dom.idom(*block).map(|parent| position[&parent]) else {
continue;
};
last[parent] = last[parent].max(last[at]);
lowest[parent] = lowest[parent].min(lowest[at]);
highest[parent] = highest[parent].max(highest[at]);
}
(order.iter().enumerate())
.filter(|(at, _)| lowest[*at] >= *at && highest[*at] <= last[*at])
.map(|(_, block)| *block)
.collect()
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -205,3 +299,305 @@ pub(crate) enum EdgeShape {
Continue,
Goto,
}
#[cfg(test)]
mod tests {
use super::*;
use crate::structure::support::{branch, cfg, cond_branch};
use r2ssa::cfg::{BlockTerminator, CFG};
use r2ssa::natural_loops::NaturalLoops;
const RETURN: BlockTerminator = BlockTerminator::Return;
/// Run `check` on the placement of `cfg`.
fn placed(cfg: &CFG, check: impl FnOnce(&Placement<'_>)) {
let dom = DomTree::compute(cfg);
let loops = NaturalLoops::compute(cfg, &dom);
check(&Placement::compute(cfg, &dom, &loops, cfg.entry));
}
/// A loop 0x20/0x30 that leaves for the returns 0x40 (from its header) and 0x50 (from 0x30).
fn two_exits() -> CFG {
cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x40)),
(0x30, cond_branch(0x20, 0x50)),
(0x40, RETURN),
(0x50, RETURN),
],
)
}
/// The two exits of [`two_exits`] by reverse postorder: the one after the loop, then the other.
fn by_rpo(placement: &Placement<'_>) -> (u64, u64) {
let first = [0x40, 0x50]
.into_iter()
.min_by_key(|exit| placement.rpo_of(*exit))
.expect("two exits");
(first, 0x40 + 0x50 - first)
}
#[test]
fn the_exit_after_the_loop_stays_there_and_the_other_is_written_where_its_edge_leaves() {
placed(&two_exits(), |placement| {
let (after, inside) = by_rpo(placement);
let from = |exit| if exit == 0x40 { 0x20 } else { 0x30 };
assert_eq!(placement.edge(from(after), after), EdgeShape::Goto);
assert_eq!(placement.edge(from(inside), inside), EdgeShape::Inline);
assert_eq!(placement.exits_of(0), [after]);
assert!(!placement.labelled().contains(&inside));
});
}
#[test]
fn a_loop_with_one_exit_keeps_it_after_the_loop() {
let one = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x20, 0x40)),
(0x40, RETURN),
],
);
placed(&one, |placement| {
assert_eq!(placement.edge(0x20, 0x40), EdgeShape::Goto);
assert_eq!(placement.exits_of(0), [0x40]);
});
}
#[test]
fn an_exit_with_a_second_way_in_stays_after_the_loop() {
// 0x40 and 0x50 both go on to the return 0x60: 0x50 has one way in, but its subtree is
// not closed, and 0x60 has two ways in.
let merging = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x40)),
(0x30, cond_branch(0x20, 0x50)),
(0x40, branch(0x60)),
(0x50, branch(0x60)),
(0x60, RETURN),
],
);
placed(&merging, |placement| {
assert_eq!(placement.exits_of(0), [0x40, 0x50, 0x60]);
assert_eq!(placement.edge(0x30, 0x50), EdgeShape::Goto);
assert!(placement.labelled().contains(&0x50));
});
}
#[test]
fn a_subtree_is_closed_when_no_edge_leaves_it() {
// 0x50 branches to the return 0x80 or to the exit 0x40, which is not below it; 0x30's back
// edge to the header leaves its subtree; the root and the header's subtree hold every edge.
let leaving = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x40)),
(0x30, cond_branch(0x20, 0x50)),
(0x40, RETURN),
(0x50, cond_branch(0x80, 0x40)),
(0x80, RETURN),
],
);
let dom = DomTree::compute(&leaving);
let placed = leaving.block_addrs().collect::<BTreeSet<_>>();
let closed = closed_subtrees(&leaving, &dom, &placed);
assert_eq!(
closed.into_iter().collect::<BTreeSet<_>>(),
BTreeSet::from([0x10, 0x20, 0x40, 0x80])
);
}
#[test]
fn a_closed_subtree_is_written_whole_where_the_edge_leaves() {
// The second exit branches to two returns of its own.
let whole = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x40)),
(0x30, cond_branch(0x20, 0x50)),
(0x40, RETURN),
(0x50, cond_branch(0x80, 0x90)),
(0x80, RETURN),
(0x90, RETURN),
],
);
placed(&whole, |placement| {
let leaving = [(0x30, 0x50), (0x50, 0x80), (0x50, 0x90)];
for (from, to) in leaving {
assert_eq!(
placement.edge(from, to),
EdgeShape::Inline,
"{from:#x} {to:#x}"
);
}
assert_eq!(placement.exits_of(0), [0x40]);
});
}
#[test]
fn an_exit_leaving_two_loops_is_written_where_it_leaves_even_when_it_is_the_first() {
// 0x30 heads the inner loop {0x30, 0x35} inside the outer loop {0x20, 0x30, 0x35, 0x38};
// 0x60 leaves the outer loop from its header, 0x70 leaves both from the inner header.
let nested = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x60)),
(0x30, cond_branch(0x35, 0x70)),
(0x35, cond_branch(0x30, 0x38)),
(0x38, branch(0x20)),
(0x60, RETURN),
(0x70, RETURN),
],
);
placed(&nested, |placement| {
assert_eq!(placement.edge(0x30, 0x70), EdgeShape::Inline);
assert_eq!(placement.edge(0x20, 0x60), EdgeShape::Goto);
assert!(!placement.labelled().contains(&0x70));
});
}
}
#[cfg(all(test, feature = "placement_mutants"))]
mod orders {
use super::*;
use crate::placement_mutants::{WriteOrder, with_write_order};
use crate::structure::support::{branch, cfg, cond_branch};
use r2ssa::cfg::{BlockTerminator, CFG};
use r2ssa::natural_loops::NaturalLoops;
const RETURN: BlockTerminator = BlockTerminator::Return;
/// Run `check` on the placement of `cfg` written in `order`.
fn under(order: WriteOrder, cfg: &CFG, check: impl FnOnce(&Placement<'_>)) {
let dom = DomTree::compute(cfg);
let loops = NaturalLoops::compute(cfg, &dom);
with_write_order(order, || {
check(&Placement::compute(cfg, &dom, &loops, cfg.entry));
});
}
/// A loop 0x20/0x30 leaving for the returns 0x40 (from its header) and 0x50 (from 0x30), which both
/// go on to the return 0x60: the loop's exits are 0x40, 0x50 and 0x60.
fn three_exits() -> CFG {
cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x40)),
(0x30, cond_branch(0x20, 0x50)),
(0x40, branch(0x60)),
(0x50, branch(0x60)),
(0x60, RETURN),
],
)
}
/// An entry that branches two ways to two blocks that both go to 0x40 and 0x50, which both go to 0x60:
/// the entry's region holds the merges 0x40, 0x50 and 0x60.
fn three_merges() -> CFG {
cfg(
0x10,
&[
(0x10, cond_branch(0x20, 0x30)),
(0x20, cond_branch(0x40, 0x50)),
(0x30, cond_branch(0x40, 0x50)),
(0x40, branch(0x60)),
(0x50, branch(0x60)),
(0x60, RETURN),
],
)
}
#[test]
fn keeping_the_exits_inlines_none() {
let loop_exits = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x40)),
(0x30, cond_branch(0x20, 0x50)),
(0x40, RETURN),
(0x50, RETURN),
],
);
under(WriteOrder::Natural, &loop_exits, |placement| {
assert_eq!(placement.exits_of(0).len(), 1);
assert!(placement.inlined.len() == 1 && placement.labelled().len() == 1);
});
under(WriteOrder::KeepExits, &loop_exits, |placement| {
assert_eq!(placement.exits_of(0), [0x40, 0x50]);
assert!(placement.inlined.is_empty());
assert_eq!(placement.labelled(), &BTreeSet::from([0x40, 0x50]));
});
}
#[test]
fn reversing_the_exits_reverses_the_exits_of_each_loop() {
under(WriteOrder::Natural, &three_exits(), |placement| {
assert_eq!(placement.exits_of(0), [0x40, 0x50, 0x60]);
});
under(WriteOrder::ExitsReversed, &three_exits(), |placement| {
assert_eq!(placement.exits_of(0), [0x60, 0x50, 0x40]);
});
}
#[test]
fn reversing_the_merges_reverses_the_merges_of_each_region() {
let natural = {
let mut found = Vec::new();
under(WriteOrder::Natural, &three_merges(), |placement| {
found = placement.merges_in(0x10).to_vec();
assert_eq!(found.len(), 3);
assert!(
found
.windows(2)
.all(|pair| placement.rpo_of(pair[0]) < placement.rpo_of(pair[1]))
);
});
found
};
let reversed = natural.iter().rev().copied().collect::<Vec<_>>();
under(WriteOrder::MergesReversed, &three_merges(), |placement| {
assert_eq!(placement.merges_in(0x10), reversed);
});
}
/// The exits of the first loop and the merges of the entry's region, written in `order`.
fn lists(order: WriteOrder, cfg: &CFG) -> [Vec<u64>; 2] {
let mut found = [Vec::new(), Vec::new()];
under(order, cfg, |p| {
found = [p.exits_of(0).to_vec(), p.merges_in(0x10).to_vec()];
});
found
}
#[test]
fn only_the_order_asked_for_moves_a_placement() {
// Each order moves the one list it names and leaves the other as it was.
for graph in [three_exits(), three_merges()] {
let natural = lists(WriteOrder::Natural, &graph);
assert_eq!(lists(WriteOrder::MergesReversed, &graph)[0], natural[0]);
assert_eq!(lists(WriteOrder::ExitsReversed, &graph)[1], natural[1]);
assert_eq!(lists(WriteOrder::KeepExits, &graph)[1], natural[1]);
}
}
#[test]
fn a_write_order_holds_only_inside_its_scope() {
with_write_order(WriteOrder::ExitsReversed, || {
with_write_order(WriteOrder::KeepExits, || {
assert_eq!(WriteOrder::current(), WriteOrder::KeepExits);
});
assert_eq!(WriteOrder::current(), WriteOrder::ExitsReversed);
});
assert_eq!(WriteOrder::current(), WriteOrder::Natural);
}
}
crates/r2dec/src/structure/rotate.rs
+718
View File
@@ -0,0 +1,718 @@
//! Loop rotation: `for (;;)` written as `while` or `do-while` where the loop is one, with the test's
//! markers on the loop for the certificate (`doc/adr-structure-dominator-tree.md` §5, 2026-10-22).
use super::condition::Marks;
use crate::ast::{BinaryOp, CExpr, CStmt, UnaryOp};
/// Write each `for (;;)` of the tree as the loop it is, where it is one; the number written.
/// O(statements): each loop is read once, from the statements at the top and bottom of its body.
pub(crate) fn rotate(stmt: &mut CStmt, marks: &mut dyn Marks) -> usize {
match stmt {
CStmt::Observed { stmt, .. } => rotate(stmt, marks),
CStmt::Block(stmts) => stmts.iter_mut().map(|stmt| rotate(stmt, marks)).sum(),
CStmt::If {
then_body,
else_body,
..
} => {
let in_then = rotate(then_body, marks);
in_then + else_body.as_deref_mut().map_or(0, |arm| rotate(arm, marks))
}
CStmt::While { body, .. } | CStmt::DoWhile { body, .. } => rotate(body, marks),
CStmt::Switch { cases, default, .. } => {
let in_cases: usize = (cases.iter_mut())
.flat_map(|case| case.body.iter_mut())
.map(|stmt| rotate(stmt, marks))
.sum();
in_cases
+ (default.iter_mut().flatten())
.map(|stmt| rotate(stmt, marks))
.sum::<usize>()
}
CStmt::For {
init: None,
cond: None,
update: None,
body,
} => {
let inner = rotate(body, marks);
let Some(shape) = read(body) else {
return inner;
};
let taken = std::mem::replace(&mut **body, CStmt::Empty);
*stmt = build(taken, shape, marks);
inner + 1
}
CStmt::For { body, .. } => rotate(body, marks),
_ => 0,
}
}
/// What kind of loop an infinite loop's body is, and on which arm of its test it leaves.
struct Shape {
latch: bool,
leaves_on_true: bool,
}
/// The loop the body of an infinite loop makes, if its test is first or last.
fn read(body: &CStmt) -> Option<Shape> {
let stmts = live(body);
let (last, before) = stmts.split_last()?;
let (_, on_true, on_false) = parts(last)?;
let (stay, leaves_on_true) = match (is_break(on_true), is_break(on_false)) {
(false, true) => (on_true, false),
(true, false) => (on_false, true),
_ => return None,
};
if before.is_empty() {
// The header's test: it stays in the loop on one arm and leaves on the other.
return Some(Shape {
latch: false,
leaves_on_true,
});
}
// The latch's test: nothing else goes back to the top of the body, since a `continue` of a
// `do-while` reaches the test, and the arm that stays does nothing more.
(is_empty(stay) && !before.iter().any(|stmt| continues(stmt))).then_some(Shape {
latch: true,
leaves_on_true,
})
}
/// The loop statement for a body [`read`] took apart, which carries the test's markers. The test
/// is written over atoms that carry them too, so the certificate reads which way it stays.
fn build(body: CStmt, shape: Shape, marks: &mut dyn Marks) -> CStmt {
let mut stmts = Vec::new();
flatten(body, &mut stmts);
let Some(CStmt::Observed { ids, stmt }) = stmts.pop() else {
unreachable!("read found the marked test last");
};
let CStmt::If {
cond,
then_body,
else_body: Some(else_body),
} = *stmt
else {
unreachable!("read found an if with both arms");
};
let atom = match marked(&cond) {
true => cond,
false => CExpr::observe_all(ids.iter().map(|id| marks.copy_of(id)), cond),
};
let (stay, test) = match shape.leaves_on_true {
true => (*else_body, negated(atom)),
false => (*then_body, atom),
};
let looped = match shape.latch {
true => CStmt::DoWhile {
body: Box::new(CStmt::Block(stmts)),
cond: test,
},
false => CStmt::While {
cond: test,
body: Box::new(stay),
},
};
CStmt::observe_all(ids.iter(), looped)
}
/// The statements of a body that do something: no empty statement, and blocks looked through.
pub(super) fn live(stmt: &CStmt) -> Vec<&CStmt> {
match stmt {
CStmt::Block(stmts) => stmts.iter().flat_map(live).collect(),
CStmt::Empty => Vec::new(),
other => vec![other],
}
}
/// [`live`], moving the statements out.
pub(super) fn flatten(stmt: CStmt, into: &mut Vec<CStmt>) {
match stmt {
CStmt::Block(stmts) => stmts.into_iter().for_each(|stmt| flatten(stmt, into)),
CStmt::Empty => {}
other => into.push(other),
}
}
/// A marked `if` with both arms: its test and arms.
pub(super) fn parts(stmt: &CStmt) -> Option<(&CExpr, &CStmt, &CStmt)> {
let CStmt::Observed { stmt: inner, .. } = stmt else {
return None;
};
let CStmt::If {
cond,
then_body,
else_body: Some(else_body),
} = &**inner
else {
return None;
};
Some((cond, then_body, else_body))
}
/// Whether an arm is a `break` and nothing else.
pub(super) fn is_break(arm: &CStmt) -> bool {
matches!(live(arm).as_slice(), [CStmt::Break])
}
/// Whether an arm does nothing: it falls to the top of the loop on its own, or says `continue`.
pub(super) fn is_empty(arm: &CStmt) -> bool {
matches!(live(arm).as_slice(), [] | [CStmt::Continue])
}
/// Whether `stmt` has a `continue` for the loop around it: those of a loop inside it are its own.
pub(super) fn continues(stmt: &CStmt) -> bool {
match stmt {
CStmt::Continue => true,
CStmt::Observed { stmt, .. } => continues(stmt),
CStmt::Block(stmts) => stmts.iter().any(continues),
CStmt::If {
then_body,
else_body,
..
} => continues(then_body) || else_body.as_deref().is_some_and(continues),
// A `continue` passes through a `switch` to the loop around it.
CStmt::Switch { cases, default, .. } => {
(cases.iter().flat_map(|case| case.body.iter())).any(continues)
|| default.iter().flatten().any(continues)
}
_ => false,
}
}
/// The test that holds where `test` does not, over marked atoms: they take the `!` and the
/// connectives swap. An integer comparison reads as its opposite once certified (`respell_negations`).
fn negated(test: CExpr) -> CExpr {
match test {
CExpr::Binary {
op: op @ (BinaryOp::And | BinaryOp::Or),
left,
right,
} if marked(&left) || marked(&right) => {
let flipped = if op == BinaryOp::And {
BinaryOp::Or
} else {
BinaryOp::And
};
CExpr::binary(flipped, negated(*left), negated(*right))
}
CExpr::Unary {
op: UnaryOp::Not,
operand,
} if marked(&operand) => *operand,
atom => CExpr::unary(UnaryOp::Not, atom),
}
}
/// Whether a test carries a block's marker somewhere, which the lowering never puts there.
fn marked(expr: &CExpr) -> bool {
let mut pending = vec![expr];
while let Some(expr) = pending.pop() {
if matches!(expr, CExpr::Observed { .. }) {
return true;
}
pending.extend(expr.children());
}
false
}
#[cfg(test)]
mod tests {
use super::*;
use crate::ast::{RenderObservationId, test_render_observation_id};
/// Markers whose block is their index; the copy of marker `n` is marker `1000 + n`.
struct Fixture;
impl crate::structure::certify::Placed for Fixture {
fn block_of(&self, id: RenderObservationId) -> Option<u64> {
Some(u64::from(id.index() % 1000))
}
fn statement_of(&self, id: RenderObservationId) -> Option<RenderObservationId> {
Some(id)
}
fn written(&self) -> usize {
0
}
fn edge_copy(&self, _id: RenderObservationId) -> bool {
false
}
}
impl Marks for Fixture {
fn block_of(&self, id: RenderObservationId) -> Option<u64> {
Some(u64::from(id.index() % 1000))
}
fn copy_of(&mut self, id: RenderObservationId) -> RenderObservationId {
test_render_observation_id(1000 + id.index())
}
}
fn marker(block: u32, stmt: CStmt) -> CStmt {
CStmt::observe_one(test_render_observation_id(block), stmt)
}
fn work(block: u32) -> CStmt {
marker(block, CStmt::Expr(CExpr::int(i64::from(block))))
}
/// The test of `block`, which prints as its address.
fn test(block: u32, on_true: CStmt, on_false: CStmt) -> CStmt {
marker(
block,
CStmt::if_stmt(CExpr::int(i64::from(block)), on_true, Some(on_false)),
)
}
fn forever(stmts: Vec<CStmt>) -> CStmt {
CStmt::For {
init: None,
cond: None,
update: None,
body: Box::new(CStmt::Block(stmts)),
}
}
fn brk() -> CStmt {
CStmt::Block(vec![CStmt::Break])
}
fn nothing() -> CStmt {
CStmt::Block(vec![CStmt::Empty])
}
fn rotated_once(mut body: CStmt) -> (CStmt, usize) {
let count = rotate(&mut body, &mut Fixture);
(body, count)
}
/// The block whose marker an atom carries.
fn atom_block(atom: &CExpr) -> Option<u64> {
let CExpr::Observed { ids, .. } = atom else {
return None;
};
ids.iter().find_map(|id| Fixture.block_of(id))
}
#[test]
fn a_body_that_is_its_test_with_a_break_on_one_arm_is_a_while() {
let (looped, count) = rotated_once(forever(vec![test(
0x10,
CStmt::Block(vec![work(0x20)]),
brk(),
)]));
assert_eq!(count, 1);
let CStmt::Observed { ids, stmt } = &looped else {
panic!("the loop carries the test's marker");
};
assert_eq!(ids.iter().map(|id| id.index()).collect::<Vec<_>>(), [0x10]);
let CStmt::While { cond, body } = &**stmt else {
panic!("a while");
};
// The test is one atom, which carries a marker of the test's block of its own.
assert!(matches!(cond.unobserved(), CExpr::IntLit(0x10)), "{cond:?}");
assert_eq!(atom_block(cond), Some(0x10), "{cond:?}");
assert!(
ids.iter().all(|id| id.index() < 1000),
"the loop keeps the test's own markers"
);
assert!(matches!(&**body, CStmt::Block(stmts) if stmts.len() == 1));
}
#[test]
fn the_loop_stays_on_the_arm_that_does_not_break_so_a_break_on_the_true_arm_negates() {
let (looped, _) = rotated_once(forever(vec![test(
0x10,
brk(),
CStmt::Block(vec![work(0x20)]),
)]));
let CStmt::Observed { stmt, .. } = &looped else {
panic!("marked");
};
let CStmt::While { cond, .. } = &**stmt else {
panic!("a while");
};
// The negation is written over the marked atom, where the certificate reads it.
let CExpr::Unary {
op: UnaryOp::Not,
operand,
} = cond
else {
panic!("a negation: {cond:?}");
};
assert_eq!(atom_block(operand), Some(0x10), "{operand:?}");
}
#[test]
fn a_comparison_the_loop_leaves_on_reads_as_the_opposite_once_certified() {
let mut symbols = crate::symbol::SymbolTable::new();
let x = CExpr::var(symbols.declare_or_reuse("x"));
let leaving = |cond: CExpr| {
let header = marker(
0x10,
CStmt::if_stmt(cond, brk(), Some(CStmt::Block(vec![work(0x20)]))),
);
let (mut looped, _) = rotated_once(forever(vec![header]));
crate::structure::condition::respell_negations(&mut looped);
let CStmt::Observed { stmt, .. } = looped else {
panic!("marked");
};
let CStmt::While { cond, .. } = *stmt else {
panic!("a while");
};
cond
};
let less = leaving(CExpr::binary(BinaryOp::Lt, x.clone(), CExpr::int(9)));
assert!(
matches!(
less.unobserved(),
CExpr::Binary {
op: BinaryOp::Ge,
..
}
),
"{less:?}"
);
// A float comparison has no opposite, so the loop keeps the `!`.
let float = leaving(CExpr::binary(BinaryOp::FLt, x, CExpr::int(9)));
assert!(matches!(float, CExpr::Unary { .. }), "{float:?}");
}
#[test]
fn a_body_that_ends_in_its_test_is_a_do_while_when_nothing_else_goes_back() {
// for (;;) { work; if (t) {} else break; }
let (looped, count) = rotated_once(forever(vec![work(0x20), test(0x10, nothing(), brk())]));
assert_eq!(count, 1);
let CStmt::Observed { ids, stmt } = &looped else {
panic!("marked");
};
assert_eq!(ids.iter().map(|id| id.index()).collect::<Vec<_>>(), [0x10]);
let CStmt::DoWhile { body, .. } = &**stmt else {
panic!("a do-while");
};
assert!(matches!(&**body, CStmt::Block(stmts) if stmts.len() == 1));
}
#[test]
fn a_continue_before_the_test_keeps_the_loop_infinite() {
// A `continue` of a do-while reaches its test, so it cannot be the one of the `for`.
let early = CStmt::If {
cond: CExpr::int(1),
then_body: Box::new(CStmt::Continue),
else_body: None,
};
let body = forever(vec![work(0x20), early, test(0x10, nothing(), brk())]);
let (looped, count) = rotated_once(body);
assert_eq!(count, 0);
assert!(matches!(looped, CStmt::For { .. }));
// The same continue inside an inner loop is that loop's own.
let inner = forever(vec![CStmt::Continue]);
let (_, count) = rotated_once(forever(vec![
work(0x20),
inner,
test(0x10, nothing(), brk()),
]));
assert!(count >= 1);
}
#[test]
fn a_test_whose_arms_do_more_than_break_or_stay_is_no_loop_test() {
// The exit arm copies a variable before it breaks: it is not the loop's exit alone.
let copies = CStmt::Block(vec![CStmt::Expr(CExpr::int(7)), CStmt::Break]);
let (_, count) = rotated_once(forever(vec![work(0x20), test(0x10, nothing(), copies)]));
assert_eq!(count, 0);
// Both arms break: it is not a test that loops.
let (_, count) = rotated_once(forever(vec![test(0x10, brk(), brk())]));
assert_eq!(count, 0);
// The staying arm of a latch does work of its own: that is step 3's copies, not an empty arm.
let (_, count) = rotated_once(forever(vec![
work(0x20),
test(0x10, CStmt::Block(vec![CStmt::Expr(CExpr::int(7))]), brk()),
]));
assert_eq!(count, 0);
}
#[test]
fn a_label_at_the_top_keeps_the_header_a_header_of_a_for() {
// A goto reaches the header: there is no place for its label in a `while`.
let (looped, count) = rotated_once(forever(vec![
CStmt::Label("L".to_owned()),
test(0x10, CStmt::Block(vec![work(0x20)]), brk()),
]));
assert_eq!(count, 0);
assert!(matches!(looped, CStmt::For { .. }));
}
// -- The certificate reads the rotated loop as it read the `for`.
use super::super::certify::{ControlCertificate, StatementRole, Violation, certify};
use r2ssa::cfg::{BasicBlock, BlockTerminator, CFG};
use r2ssa::domtree::DomTree;
fn cfg(entry: u64, blocks: &[(u64, BlockTerminator)]) -> CFG {
let mut cfg = CFG::new(entry);
for (addr, _) in blocks {
cfg.add_block(BasicBlock::new(*addr));
}
for (addr, terminator) in blocks {
cfg.set_terminator(*addr, terminator.clone());
}
cfg
}
fn branch(target: u64) -> BlockTerminator {
BlockTerminator::Branch { target }
}
fn cond_branch(true_target: u64, false_target: u64) -> BlockTerminator {
BlockTerminator::ConditionalBranch {
true_target,
false_target,
}
}
fn certified(body: &CStmt, cfg: &CFG) -> ControlCertificate {
certify(
body,
cfg,
&DomTree::compute(cfg),
cfg.entry,
&Fixture,
&|_| None,
&|_| StatementRole::Plain,
)
}
/// The loop written as placement and `shape` leave it, and as `rotate` writes it: both must certify.
fn both_certify(body: CStmt, cfg: &CFG) -> CStmt {
let before = certified(&body, cfg);
assert!(before.ok(), "the for: {before} {:?}", before.violations);
let mut after = body;
assert_eq!(rotate(&mut after, &mut Fixture), 1);
let rotated = certified(&after, cfg);
assert!(
rotated.ok(),
"the rotation: {rotated} {:?}",
rotated.violations
);
after
}
fn exit() -> CStmt {
marker(0x50, CStmt::Return(None))
}
#[test]
fn a_while_certifies_as_the_for_it_was() {
let cfg = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x30, 0x50)),
(0x30, branch(0x20)),
(0x50, BlockTerminator::Return),
],
);
let stay = CStmt::Block(vec![work(0x30)]);
let body = CStmt::Block(vec![
work(0x10),
forever(vec![test(0x20, stay, brk())]),
exit(),
]);
let rotated = both_certify(body, &cfg);
let CStmt::Block(stmts) = &rotated else {
panic!("a block");
};
assert!(matches!(stmts[1].unobserved(), CStmt::While { .. }));
}
#[test]
fn a_do_while_certifies_as_the_for_it_was() {
let cfg = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, branch(0x30)),
(0x30, cond_branch(0x20, 0x50)),
(0x50, BlockTerminator::Return),
],
);
let body = CStmt::Block(vec![
work(0x10),
forever(vec![work(0x20), test(0x30, nothing(), brk())]),
exit(),
]);
let rotated = both_certify(body, &cfg);
let CStmt::Block(stmts) = &rotated else {
panic!("a block");
};
assert!(matches!(stmts[1].unobserved(), CStmt::DoWhile { .. }));
}
#[test]
fn a_header_that_tests_a_chain_is_a_while_over_the_chain() {
// while (a && b): the second test is read through, and its marker is on the atom.
let cfg = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x25, 0x50)),
(0x25, cond_branch(0x30, 0x50)),
(0x30, branch(0x20)),
(0x50, BlockTerminator::Return),
],
);
let atom = |block: u32| {
CExpr::observe_one(
test_render_observation_id(block),
CExpr::int(i64::from(block)),
)
};
let chain = CExpr::binary(BinaryOp::And, atom(0x20), atom(0x25));
let header = marker(
0x20,
CStmt::if_stmt(chain, CStmt::Block(vec![work(0x30)]), Some(brk())),
);
let body = CStmt::Block(vec![work(0x10), forever(vec![header]), exit()]);
let rotated = both_certify(body, &cfg);
let CStmt::Block(stmts) = &rotated else {
panic!("a block");
};
assert!(matches!(
stmts[1].unobserved(),
CStmt::While {
cond: CExpr::Binary { .. },
..
}
));
}
#[test]
fn a_loop_that_leaves_on_a_chain_stays_on_its_negation_with_the_connective_flipped() {
// `if (a || b) break;` stays while `!a && !b`: the connective swaps with the atoms (De Morgan).
let atom = |block: u32| {
CExpr::observe_one(
test_render_observation_id(block),
CExpr::int(i64::from(block)),
)
};
let leaves = CExpr::binary(BinaryOp::Or, atom(0x20), atom(0x25));
let header = marker(
0x20,
CStmt::if_stmt(leaves, brk(), Some(CStmt::Block(vec![work(0x30)]))),
);
let (looped, count) = rotated_once(forever(vec![header]));
assert_eq!(count, 1);
let CStmt::Observed { stmt, .. } = &looped else {
panic!("marked");
};
let CStmt::While { cond, .. } = &**stmt else {
panic!("a while");
};
let CExpr::Binary {
op: BinaryOp::And,
left,
right,
} = cond
else {
panic!("a conjunction: {cond:?}");
};
for (side, block) in [(left, 0x20), (right, 0x25)] {
let CExpr::Unary {
op: UnaryOp::Not,
operand,
} = &**side
else {
panic!("a negated atom: {side:?}");
};
assert_eq!(atom_block(operand), Some(block));
}
}
#[test]
fn a_loop_whose_rotation_leaves_no_block_in_its_body_fails_the_certificate_and_stays_a_for() {
// The header branches to itself: `while (c) {}` owns no block, which the certificate
// reads as an operation's own loop, so the stage that wrote it is not kept.
let cfg = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x20, 0x50)),
(0x50, BlockTerminator::Return),
],
);
let body = CStmt::Block(vec![
work(0x10),
forever(vec![test(0x20, nothing(), brk())]),
exit(),
]);
let before = certified(&body, &cfg);
assert!(before.ok(), "the for: {before} {:?}", before.violations);
let mut after = body;
assert_eq!(rotate(&mut after, &mut Fixture), 1);
assert!(!certified(&after, &cfg).ok());
}
#[test]
fn a_loop_the_machine_leaves_on_its_true_edge_certifies_only_with_the_test_negated() {
// The header leaves on true; the latch of the second loop does too.
let header = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, cond_branch(0x50, 0x30)),
(0x30, branch(0x20)),
(0x50, BlockTerminator::Return),
],
);
let stay = CStmt::Block(vec![work(0x30)]);
let while_body = CStmt::Block(vec![
work(0x10),
forever(vec![test(0x20, brk(), stay)]),
exit(),
]);
let latch = cfg(
0x10,
&[
(0x10, branch(0x20)),
(0x20, branch(0x30)),
(0x30, cond_branch(0x50, 0x20)),
(0x50, BlockTerminator::Return),
],
);
let do_body = CStmt::Block(vec![
work(0x10),
forever(vec![work(0x20), test(0x30, brk(), nothing())]),
exit(),
]);
for (body, cfg) in [(while_body, header), (do_body, latch)] {
let rotated = both_certify(body, &cfg);
// The same loop with the negation dropped is the other test, which the machine does not run.
let CStmt::Block(mut stmts) = rotated else {
panic!("a block");
};
let CStmt::Observed { ids, stmt } = stmts.remove(1) else {
panic!("marked");
};
let mut flipped = *stmt;
let (CStmt::While { cond, .. } | CStmt::DoWhile { cond, .. }) = &mut flipped else {
panic!("a loop");
};
let CExpr::Unary { operand, .. } = std::mem::replace(cond, CExpr::int(0)) else {
panic!("a negation");
};
*cond = *operand;
stmts.insert(1, CStmt::observe_all(ids.iter(), flipped));
let certificate = certified(&CStmt::Block(stmts), &cfg);
assert!(
(certificate.violations.iter())
.any(|violation| matches!(violation, Violation::EdgeMismatch { .. })),
"{certificate}"
);
}
}
}
crates/r2dec/src/structure/shape.rs
+245 -25
View File
@@ -8,7 +8,7 @@
use std::collections::BTreeSet;
use crate::ast::{BinaryOp, CExpr, CStmt, RenderObservationId};
use crate::ast::{BinaryOp, CExpr, CStmt, RenderObservationId, SwitchCase};
use crate::symbol::SymbolId;
/// What runs after a statement completes normally.
@@ -38,24 +38,189 @@ struct Scope {
break_to: Cont,
}
/// The structural rewrites, in order: jumps to the next position and to
/// the end of a breakable go, unreferenced labels go, and a skipped block
/// is copied into the arms that fall to it.
/// `fresh` copies a statement as a new occurrence, with markers of its own for the same cells.
pub(crate) fn shape(fresh: &mut dyn FnMut(&CStmt) -> CStmt, stmt: CStmt) -> CStmt {
/// The block and instruction each marker stands for; the rewrites mint a marker for every copy of
/// a block's text they make.
pub(crate) struct Occurrences {
blocks: Vec<u64>,
addresses: Vec<Option<u64>>,
/// By marker: the marker the placement wrote that this one stands for (itself, if it is one).
origins: Vec<u32>,
written: usize,
}
impl Occurrences {
pub(crate) fn new(blocks: Vec<u64>, addresses: Vec<Option<u64>>) -> Self {
let written = blocks.len();
let origins = (0..written as u32).collect();
Self {
blocks,
addresses,
origins,
written,
}
}
/// The block a marker was written for.
pub(crate) fn block_of(&self, id: RenderObservationId) -> Option<u64> {
self.blocks.get(id.index() as usize).copied()
}
/// The instruction each marker's line accounts for, by marker.
pub(crate) fn into_addresses(self) -> Vec<Option<u64>> {
self.addresses
}
/// `stmt` copied as a new occurrence: each marker in it replaced by one for the same block.
pub(crate) fn fresh(&mut self, stmt: &CStmt) -> CStmt {
remint(stmt, &mut |id| self.copy_of(id))
}
/// A marker of its own for the same block and instruction, for one more occurrence of it.
pub(crate) fn copy_of(&mut self, id: RenderObservationId) -> RenderObservationId {
let at = id.index() as usize;
let copy = RenderObservationId::from_dense_index(self.blocks.len());
self.blocks.push(self.blocks[at]);
self.addresses.push(self.addresses[at]);
self.origins.push(self.origins[at]);
copy
}
/// How many markers exist, to hand back to `retract` when a stage's copies are refused.
pub(crate) fn issued(&self) -> usize {
self.blocks.len()
}
/// Forget the markers minted after `issued` markers existed.
pub(crate) fn retract(&mut self, issued: usize) {
self.blocks.truncate(issued);
self.addresses.truncate(issued);
self.origins.truncate(issued);
}
}
impl super::certify::Placed for Occurrences {
fn block_of(&self, id: RenderObservationId) -> Option<u64> {
Self::block_of(self, id)
}
fn statement_of(&self, id: RenderObservationId) -> Option<RenderObservationId> {
let origin = *self.origins.get(id.index() as usize)?;
Some(RenderObservationId::from_dense_index(origin as usize))
}
fn written(&self) -> usize {
self.written
}
fn edge_copy(&self, id: RenderObservationId) -> bool {
self.addresses.get(id.index() as usize) == Some(&None)
}
}
impl super::condition::Marks for Occurrences {
fn block_of(&self, id: RenderObservationId) -> Option<u64> {
Self::block_of(self, id)
}
fn copy_of(&mut self, id: RenderObservationId) -> RenderObservationId {
Self::copy_of(self, id)
}
}
/// What the rewrites may rely on that the tree does not say.
pub(crate) struct Evidence<'a> {
/// Whether a statement is a call to a callee declared never to return.
pub(crate) never_returns: &'a dyn Fn(&CStmt) -> bool,
/// Whether the return ending a block hands back a merge whose inputs are pairwise distinct
/// values (the compiler merged returns of different values into this one).
pub(crate) distinct_return: &'a dyn Fn(u64) -> bool,
}
/// The structural rewrites (ADR §5): jumps to the next position go, a skipped block is copied to the
/// arms that fall to it, a terminal block to the sites that reach it; labels nothing names go.
pub(crate) fn shape(occ: &mut Occurrences, ev: &Evidence<'_>, stmt: CStmt) -> CStmt {
let mut stmt = stmt;
let scope = Scope {
break_to: Cont::Unknown,
};
shape_stmt(&mut stmt, Cont::Unknown, &scope);
drop_unreferenced_labels(&mut stmt);
// A skipped block shows only once the jumps to it have gone.
duplicate_skipped_tails(fresh, &mut stmt);
// A skipped block is copied while the jumps over it stand, which a terminal block's copies replace.
duplicate_skipped_tails(occ, &mut stmt);
shape_stmt(&mut stmt, Cont::Unknown, &scope);
drop_unreferenced_labels(&mut stmt);
super::tails::inline_tails(occ, ev, &mut stmt);
shape_stmt(&mut stmt, Cont::Unknown, &scope);
drop_unreferenced_labels(&mut stmt);
stmt
}
/// `stmt` copied as a new occurrence: each marker in it replaced by the one `mint` gives for it.
fn remint(stmt: &CStmt, mint: &mut dyn FnMut(RenderObservationId) -> RenderObservationId) -> CStmt {
let copy = |stmts: &[CStmt],
mint: &mut dyn FnMut(RenderObservationId) -> RenderObservationId| {
stmts
.iter()
.map(|stmt| remint(stmt, mint))
.collect::<Vec<_>>()
};
match stmt {
CStmt::Observed { .. } => {
let ids = stmt
.observation_ids()
.iter()
.map(|id| mint(*id))
.collect::<Vec<_>>();
CStmt::observe_all(ids, remint(stmt.unobserved(), mint))
}
CStmt::Block(stmts) => CStmt::Block(copy(stmts, mint)),
CStmt::If {
cond,
then_body,
else_body,
} => CStmt::If {
cond: cond.clone(),
then_body: Box::new(remint(then_body, mint)),
else_body: else_body.as_ref().map(|body| Box::new(remint(body, mint))),
},
CStmt::For {
init,
cond,
update,
body,
} => CStmt::For {
init: init.as_ref().map(|init| Box::new(remint(init, mint))),
cond: cond.clone(),
update: update.clone(),
body: Box::new(remint(body, mint)),
},
CStmt::While { cond, body } => CStmt::While {
cond: cond.clone(),
body: Box::new(remint(body, mint)),
},
CStmt::DoWhile { body, cond } => CStmt::DoWhile {
body: Box::new(remint(body, mint)),
cond: cond.clone(),
},
CStmt::Switch {
expr,
cases,
default,
} => CStmt::Switch {
expr: expr.clone(),
cases: cases
.iter()
.map(|case| SwitchCase {
value: case.value.clone(),
body: copy(&case.body, mint),
})
.collect(),
default: default.as_ref().map(|body| copy(body, mint)),
},
other => other.clone(),
}
}
/// `if (c) { x = a; } else { x = b; }` is `x = c ? a : b;`, the test's markers on the statement
/// and each arm's on its value (ADR §3); after `shape`, whose copies do not remint a value.
pub(crate) fn select(
@@ -175,29 +340,29 @@ fn sole_arm_assignment(arm: &CStmt) -> Option<(Vec<RenderObservationId>, SymbolI
/// paths that do not jump each get their own copy of `T`, and the jumps
/// then reach the next position and go. Only the text of one block is
/// duplicated, because that is the unit a compiler merged.
fn duplicate_skipped_tails(fresh: &mut dyn FnMut(&CStmt) -> CStmt, stmt: &mut CStmt) {
fn duplicate_skipped_tails(occ: &mut Occurrences, stmt: &mut CStmt) {
match stmt {
CStmt::Observed { stmt, .. } => duplicate_skipped_tails(fresh, stmt),
CStmt::Block(stmts) => duplicate_in_body(fresh, stmts),
CStmt::Observed { stmt, .. } => duplicate_skipped_tails(occ, stmt),
CStmt::Block(stmts) => duplicate_in_body(occ, stmts),
CStmt::If {
then_body,
else_body,
..
} => {
duplicate_skipped_tails(fresh, then_body);
duplicate_skipped_tails(occ, then_body);
if let Some(else_body) = else_body {
duplicate_skipped_tails(fresh, else_body);
duplicate_skipped_tails(occ, else_body);
}
}
CStmt::For { body, .. } | CStmt::While { body, .. } | CStmt::DoWhile { body, .. } => {
duplicate_skipped_tails(fresh, body)
duplicate_skipped_tails(occ, body)
}
CStmt::Switch { cases, default, .. } => {
for case in cases {
duplicate_in_body(fresh, &mut case.body);
duplicate_in_body(occ, &mut case.body);
}
if let Some(default) = default {
duplicate_in_body(fresh, default);
duplicate_in_body(occ, default);
}
}
_ => {}
@@ -205,14 +370,14 @@ fn duplicate_skipped_tails(fresh: &mut dyn FnMut(&CStmt) -> CStmt, stmt: &mut CS
}
/// Each statement of a body, then the body's own sequence.
fn duplicate_in_body(fresh: &mut dyn FnMut(&CStmt) -> CStmt, body: &mut Vec<CStmt>) {
fn duplicate_in_body(occ: &mut Occurrences, body: &mut Vec<CStmt>) {
for stmt in body.iter_mut() {
duplicate_skipped_tails(fresh, stmt);
duplicate_skipped_tails(occ, stmt);
}
duplicate_in_sequence(fresh, body);
duplicate_in_sequence(occ, body);
}
fn duplicate_in_sequence(fresh: &mut dyn FnMut(&CStmt) -> CStmt, stmts: &mut Vec<CStmt>) {
fn duplicate_in_sequence(occ: &mut Occurrences, stmts: &mut Vec<CStmt>) {
let mut index = 0;
while index + 2 < stmts.len() {
// stmts[index] branches, stmts[index + 1] is the skipped block,
@@ -233,18 +398,18 @@ fn duplicate_in_sequence(fresh: &mut dyn FnMut(&CStmt) -> CStmt, stmts: &mut Vec
// The first copy keeps the tail's own observations; every further
// copy is a fresh occurrence with targets of its own.
let mut copies =
std::iter::once(tail.clone()).chain(std::iter::repeat_with(|| fresh(&tail)));
std::iter::once(tail.clone()).chain(std::iter::repeat_with(|| occ.fresh(&tail)));
append_to_falling_arms(&mut stmts[index], &mut copies);
index += 1;
}
}
/// One block's text, straight-line: statements only, no label or jump.
/// One block's text, straight-line: statements only, no label or jump, and nothing unproven.
fn is_plain_block(stmt: &CStmt) -> bool {
match stmt {
CStmt::Observed { stmt, .. } => is_plain_block(stmt),
CStmt::Block(stmts) => stmts.iter().all(is_plain_block),
CStmt::Expr(_) | CStmt::Empty | CStmt::Comment(_) => true,
CStmt::Expr(_) | CStmt::Empty | CStmt::Comment(_) => super::tails::copyable(stmt),
_ => false,
}
}
@@ -427,7 +592,7 @@ fn shape_stmt(stmt: &mut CStmt, next: Cont, scope: &Scope) {
}
}
fn collect_gotos(stmt: &CStmt, into: &mut BTreeSet<String>) {
pub(super) fn collect_gotos(stmt: &CStmt, into: &mut BTreeSet<String>) {
match stmt {
CStmt::Observed { stmt, .. } => collect_gotos(stmt, into),
CStmt::Block(stmts) => stmts.iter().for_each(|stmt| collect_gotos(stmt, into)),
@@ -503,8 +668,63 @@ fn drop_labels(stmt: &mut CStmt, referenced: &BTreeSet<String>) {
}
}
fn drop_unreferenced_labels(stmt: &mut CStmt) {
pub(super) fn drop_unreferenced_labels(stmt: &mut CStmt) {
let mut referenced = BTreeSet::new();
collect_gotos(stmt, &mut referenced);
drop_labels(stmt, &referenced);
}
#[cfg(test)]
mod tests {
use super::*;
fn id(index: usize) -> RenderObservationId {
RenderObservationId::from_dense_index(index)
}
/// Markers 0 and 1, for blocks 0x10 and 0x20 and their instructions 0x11 and 0x21.
fn marks() -> Occurrences {
Occurrences::new(vec![0x10, 0x20], vec![Some(0x11), Some(0x21)])
}
#[test]
fn a_copy_is_a_marker_of_its_own_for_the_same_block_and_instruction() {
let mut marks = marks();
let copy = marks.copy_of(id(1));
assert_ne!(copy, id(1));
assert_eq!((marks.block_of(copy), marks.issued()), (Some(0x20), 3));
assert_eq!(marks.into_addresses(), [Some(0x11), Some(0x21), Some(0x21)]);
}
#[test]
fn fresh_text_replaces_each_marker_it_carries_with_a_copy() {
let mut marks = marks();
let text = CStmt::Block(vec![
CStmt::observe_all([id(0)], CStmt::Break),
CStmt::observe_all([id(1)], CStmt::Break),
]);
let CStmt::Block(copies) = marks.fresh(&text) else {
panic!("a block stays a block");
};
let ids: Vec<_> = copies
.iter()
.flat_map(|c| c.observation_ids().into_owned())
.collect();
assert_eq!(ids, [id(2), id(3)]);
let blocks: Vec<_> = ids.iter().map(|id| marks.block_of(*id)).collect();
assert_eq!(blocks, [Some(0x10), Some(0x20)]);
assert_eq!(marks.issued(), 4);
}
#[test]
fn a_retraction_forgets_the_markers_minted_after_it_and_their_instructions() {
let mut marks = marks();
let before = marks.issued();
let copy = marks.copy_of(id(0));
marks.copy_of(id(1));
marks.retract(before);
assert_eq!((marks.issued(), marks.block_of(copy)), (before, None));
assert_eq!(marks.block_of(id(1)), Some(0x20));
assert_eq!(marks.into_addresses(), [Some(0x11), Some(0x21)]);
}
}
crates/r2dec/src/structure/support.rs
+65
View File
@@ -0,0 +1,65 @@
//! Fixtures for the structure tests: a CFG from terminators, and statements observed as a block's.
use r2ssa::cfg::{BasicBlock, BlockTerminator, CFG};
use super::certify::Placed;
use crate::ast::{CExpr, CStmt, RenderObservationId, test_render_observation_id};
/// Markers numbered by the block they were written for, none of them a statement the placement wrote.
pub(crate) struct ByBlock;
impl Placed for ByBlock {
fn block_of(&self, id: RenderObservationId) -> Option<u64> {
Some(u64::from(id.index()))
}
fn statement_of(&self, id: RenderObservationId) -> Option<RenderObservationId> {
Some(id)
}
fn written(&self) -> usize {
0
}
fn edge_copy(&self, _id: RenderObservationId) -> bool {
false
}
}
pub(crate) fn cfg(entry: u64, blocks: &[(u64, BlockTerminator)]) -> CFG {
let mut cfg = CFG::new(entry);
for (addr, _) in blocks {
cfg.add_block(BasicBlock::new(*addr));
}
for (addr, terminator) in blocks {
cfg.set_terminator(*addr, terminator.clone());
}
cfg
}
/// A statement observed as belonging to `block`.
pub(crate) fn at(block: u64, stmt: CStmt) -> CStmt {
CStmt::observe_one(
test_render_observation_id(u32::try_from(block).expect("small address")),
stmt,
)
}
pub(crate) fn expr() -> CStmt {
CStmt::Expr(CExpr::IntLit(0))
}
pub(crate) fn cond() -> CExpr {
CExpr::IntLit(1)
}
pub(crate) fn branch(target: u64) -> BlockTerminator {
BlockTerminator::Branch { target }
}
pub(crate) fn cond_branch(true_target: u64, false_target: u64) -> BlockTerminator {
BlockTerminator::ConditionalBranch {
true_target,
false_target,
}
}
crates/r2dec/src/structure/tails.rs
+1199
View File
File diff suppressed because it is too large. Load diff
crates/r2dec/tests/architecture.rs
+40
View File
@@ -68,6 +68,46 @@ fn production_has_no_legacy_name_identity_or_repair_answerers() {
);
}
/// The production lines of `render/<file>` that contain any of `tokens`, as `file:line text`.
fn render_lines_naming(file: &str, tokens: &[&str]) -> Vec<String> {
let path = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("src/render")
.join(file);
let text = fs::read_to_string(&path).expect("a render source file should be UTF-8");
production_lines(&text)
.into_iter()
.filter(|(_, line)| tokens.iter().any(|token| line.contains(token)))
.map(|(index, line)| format!("{file}:{} {}", index + 1, line.trim()))
.collect()
}
/// A verdict is a function of the facts and the CFG's reverse postorder (ROADMAP D58): control reads
/// no value and the spelling reads no placement, so no write order reaches a verdict.
#[test]
fn control_writes_only_what_was_spelled_and_the_spelling_reads_no_placement() {
let control = render_lines_naming("control.rs", &["Values", "values::"]);
assert!(
control.is_empty(),
"render/control.rs reads D2 only through `Spelled`:\n{}",
control.join("\n")
);
let placement = [
"exits_of(",
"merges_in(",
"labelled(",
"loop_headed_by(",
".edge(",
"Placement",
"EdgeShape",
];
let spelled = render_lines_naming("spelled.rs", &placement);
assert!(
spelled.is_empty(),
"render/spelled.rs spells before any block is placed:\n{}",
spelled.join("\n")
);
}
/// Lines outside `#[cfg(test)]` items, retaining zero-based source line numbers.
/// Test items are skipped rather than treated as EOF so a production item
/// placed after one cannot hide from an architecture interlock.
crates/r2engine/Cargo.toml
+3
View File
@@ -21,9 +21,12 @@ default = []
x86 = ["r2sleigh-lift/x86"]
arm = ["r2sleigh-lift/arm"]
riscv = ["r2sleigh-lift/riscv"]
placement_mutants = ["r2dec/placement_mutants"]
[dev-dependencies]
proptest = "1.12"
r2dec = { path = "../r2dec", features = ["placement_mutants"] }
r2ssa = { path = "../r2ssa", features = ["fact_mutants"] }
r2sleigh-lift = { path = "../r2sleigh-lift", default-features = false, features = [
"x86",
"arm",
crates/r2engine/src/lib.rs
+12 -3
View File
@@ -1453,6 +1453,17 @@ impl SealedFunctionAnalysis {
pub const fn facts(&self) -> &r2types::SourceOwnedFunctionFacts {
&self.source_owned_facts
}
/// This function as the stub of the import `program` says its address is, where its body is only
/// the transfer to that import.
pub(crate) fn stub_of(mut self, program: &dyn native::Program) -> Self {
let (source, report) = (
self.source_owned_facts.source(),
self.source_owned_facts.report(),
);
self.import_stub = stub::import_stub(source, report, program);
self
}
}
/// One rendering asked of one sealed analysis, under one request's control.
@@ -1953,8 +1964,6 @@ impl EngineSession {
EnginePhaseStatus::Executed,
normalization_started.elapsed(),
);
let import_stub =
stub::import_stub(source_owned_facts.source(), source_owned_facts.report());
Ok(SealedFunctionAnalysis {
function_name: display_name,
source_owned_facts,
@@ -1962,7 +1971,7 @@ impl EngineSession {
input_quality: input_quality_facts,
render_target,
metrics,
import_stub,
import_stub: None,
})
}
crates/r2engine/src/native.rs
+5 -2
View File
@@ -856,7 +856,7 @@ fn render(
) -> Result<EngineDecompileResponse, NativeRefusal> {
let control = program.control();
let prepared = analysed(target, program, entry)?;
Ok(match sealed(target, entry, &prepared, &control) {
Ok(match sealed(target, program, entry, &prepared, &control) {
Ok(sealed) => EngineSession::new().render_sealed(&sealed, &control),
Err(refused) => *refused,
})
@@ -865,11 +865,14 @@ fn render(
/// The type analysis of one prepared function under this request's control; a refusal is the response a rendering returns.
pub fn sealed(
target: &NativeTarget<'_>,
program: &dyn Program,
entry: u64,
prepared: &Prepared,
control: &crate::EngineExecutionControl,
) -> Result<crate::SealedFunctionAnalysis, Box<EngineDecompileResponse>> {
EngineSession::new().seal_function_from_input(request(target, entry, prepared, control))
let sealed =
EngineSession::new().seal_function_from_input(request(target, entry, prepared, control))?;
Ok(sealed.stub_of(program))
}
/// The request one prepared function is analysed under.
crates/r2engine/src/program/analysis.rs
+1 -1
View File
@@ -290,7 +290,7 @@ impl<S: Source + 'static> Query<ProgramInputs<S>> for Sealed {
};
let control = db.inputs().control.clone();
let sealed = crate::isolation::isolated(|| {
crate::native::sealed(&target, key.0, prepared, &control)
crate::native::sealed(&target, &view, key.0, prepared, &control)
});
let response = match sealed {
Ok(Ok(sealed)) => return Sealing::Sealed(Shared(Arc::new(sealed))),
crates/r2engine/src/stub.rs
+65 -48
View File
@@ -1,51 +1,18 @@
//! The import-stub route: a function whose body is one tail transfer to an import and nothing
//! else renders as the import's declaration in either pipeline (AGENTS.md, route policy).
//! The import-stub route: the function at an address the program states is an import's stub renders
//! as its declaration, where its body is that one transfer (doc/adr-decompiler-rewrite.md).
/// The import a stub transfers to, where `prepared` is one: no store, no other call, no register
/// written that the transfer does not carry, and a callee r2types resolves as an import.
/// The stub at `prepared`'s entry, where the program says that address is an import's and the body
/// is only the transfer to that import. A body that merely ends in one, or goes to another, is no stub.
pub(crate) fn import_stub(
prepared: &r2ssa::SsaArtifact,
facts: &r2types::FunctionFacts,
program: &dyn crate::native::Program,
) -> Option<r2types::ImportStub> {
let certificates = prepared.certificates();
let [callsite] = certificates.callsites.values().collect::<Vec<_>>()[..] else {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: {} call sites, not one",
prepared.function().entry,
certificates.callsites.len()
);
return None;
};
if callsite.transfer != r2ssa::CallSiteTransfer::TailCall || !certificates.returns.is_empty() {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: transfer {:?}, {} return certificates",
prepared.function().entry,
callsite.transfer,
certificates.returns.len()
);
return None;
}
let graph = prepared.graph();
let observable = observable_beside(prepared, callsite.at);
if let Some(inst) = observable {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: the body defines observable state beside the transfer: {:?} -> {:?} caller_supplied={}",
prepared.function().entry,
inst.payload,
inst.output
.and_then(|output| graph.value(output))
.map(|value| (value.var.display_name(), value.canonical_storage)),
inst.output
.is_some_and(|output| graph.caller_supplied(output))
);
return None;
}
let stated = program.import_at(prepared.function().entry)?;
let at = lone_transfer(prepared)?;
let Some(identity) = facts
.callee_resolution()
.and_then(|resolution| resolution.identity_for_callsite(callsite.at))
.and_then(|resolution| resolution.identity_for_callsite(at))
else {
r2il::refusal_evidence!(
"import-stub-declaration",
@@ -69,24 +36,74 @@ pub(crate) fn import_stub(
);
return None;
}
let name = identity
.display_name
.as_deref()
.or(identity.normalized_name.as_deref())
.or(identity.raw_name.as_deref())?;
// The address the transfer reaches must be bound to the stated import: the container says which,
// so a body that goes to another import (a patched stub) is a body, not this import's stub.
let bound = identity
.target_addr
.and_then(|target| program.import_at(target));
if bound.as_ref() != Some(&stated) {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: stated as the stub of `{stated}`, the transfer goes to {:?}, bound to {:?}",
prepared.function().entry,
identity.target_addr.map(|target| format!("{target:#x}")),
bound
);
return None;
}
// The signature certified for this call site is the one a caller's call renders with: the
// import's declaration at the slot the stub jumps through; the identity's own is by name.
let certified = facts
.callsites()
.and_then(|callsites| callsites.by_callsite.get(callsite.at))
.and_then(|callsites| callsites.by_callsite.get(at))
.and_then(|fact| fact.callee_signature.clone());
Some(r2types::ImportStub {
entry: prepared.function().entry,
name: name.to_owned(),
name: stated,
signature: certified.or_else(|| identity.signature.clone()),
})
}
/// The one call site of a body that is only a tail transfer, with nothing a caller observes beside it.
fn lone_transfer(prepared: &r2ssa::SsaArtifact) -> Option<r2ssa::InstId> {
let certificates = prepared.certificates();
let [callsite] = certificates.callsites.values().collect::<Vec<_>>()[..] else {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: {} call sites, not one",
prepared.function().entry,
certificates.callsites.len()
);
return None;
};
if callsite.transfer != r2ssa::CallSiteTransfer::TailCall || !certificates.returns.is_empty() {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: transfer {:?}, {} return certificates",
prepared.function().entry,
callsite.transfer,
certificates.returns.len()
);
return None;
}
let graph = prepared.graph();
if let Some(inst) = observable_beside(prepared, callsite.at) {
r2il::refusal_evidence!(
"import-stub-declaration",
"{:#x}: the body defines observable state beside the transfer: {:?} -> {:?} caller_supplied={}",
prepared.function().entry,
inst.payload,
inst.output
.and_then(|output| graph.value(output))
.map(|value| (value.var.display_name(), value.canonical_storage)),
inst.output
.is_some_and(|output| graph.caller_supplied(output))
);
return None;
}
Some(callsite.at)
}
/// An instruction that defines state a caller observes beside the transfer at `at`: an effect
/// (`SSAOp::has_observable_effects`), a call, or a register write the transfer does not carry and
/// the convention preserves or passes.
crates/r2engine/tests/engine/common/mod.rs
+70
View File
@@ -176,6 +176,40 @@ const PLT_PASSING: [u8; 0x2b] = [
0xc3, // ret
];
/// [`PLT`]'s `.plt` and then `text`, a function of the program's own that reaches the stub at [`PLT_STUB`].
const fn plt_then<const N: usize>(text: &[u8]) -> [u8; N] {
let mut bytes = [0; N];
let mut at = 0;
while at < 0x20 {
bytes[at] = PLT[at];
at += 1;
}
while at - 0x20 < text.len() {
bytes[at] = text[at - 0x20];
at += 1;
}
bytes
}
/// `test rsi, rsi; je L; mov rdi, rsi; jmp stub; L: ret`: `free` only where the pointer is set.
const PLT_GUARDED_WRAPPER: [u8; 0x2e] = plt_then(&[
0x48, 0x85, 0xf6, // test rsi, rsi
0x74, 0x08, // je L
0x48, 0x89, 0xf7, // mov rdi, rsi
0xe9, 0xe3, 0xff, 0xff, 0xff, // jmp stub
0xc3, // L: ret
]);
/// `jmp stub`: a function of the program's own that is one jump, to the stub and not through its slot.
const PLT_FORWARDING_WRAPPER: [u8; 0x25] = plt_then(&[0xe9, 0xeb, 0xff, 0xff, 0xff]);
/// `imul esi, edx; movsxd rdi, esi; jmp stub`: the stub's one argument is computed from two.
const PLT_COMPUTING_WRAPPER: [u8; 0x2b] = plt_then(&[
0x0f, 0xaf, 0xf2, // imul esi, edx
0x48, 0x63, 0xfe, // movsxd rdi, esi
0xe9, 0xe5, 0xff, 0xff, 0xff, // jmp stub
]);
/// A CET `.plt` of PLT0 and one lazy entry, which names no stub, then `.plt.sec`'s stub and a caller.
const PLT_CET: [u8; 0x36] = [
0xff, 0x35, 0xea, 0x0f, 0x00, 0x00, // push qword [rip + 0xfea]
@@ -653,6 +687,42 @@ impl Literal {
program
}
/// A `.plt` whose stub is for `import`, then `code`: a function of the program's own that reaches it.
fn plt_wrapped(import: &str, code: &'static [u8]) -> Self {
let mut program = Self::plt_importing(import).running_on(GLIBC);
program.code = code;
program.container.segments = vec![code_segment(BASE, code.len() as u64)];
program.container.sections[1].vsize = code.len() as u64 - 0x20;
program
}
/// [`Self::plt_wrapped`] with a wrapper that frees only a pointer that is set.
pub fn plt_guarded_wrapper() -> Self {
Self::plt_wrapped("free", &PLT_GUARDED_WRAPPER)
}
/// [`Self::plt_wrapped`] with a function that is one jump to the stub.
pub fn plt_forwarding_wrapper() -> Self {
Self::plt_wrapped("free", &PLT_FORWARDING_WRAPPER)
}
/// [`Self::plt_wrapped`] with a wrapper that computes the stub's argument.
pub fn plt_computing_wrapper() -> Self {
Self::plt_wrapped("malloc", &PLT_COMPUTING_WRAPPER)
}
/// [`Self::plt_importing`] with the container stating the stub at [`PLT_STUB`] stands for `stated`, as
/// Mach-O's indirect symbol table does, while its jump goes through the slot bound to `bound`.
pub fn plt_declaring(stated: &str, bound: &str) -> Self {
let mut program = Self::plt_importing(bound);
program.container.import_stubs = vec![r2engine::program::ImportStub {
vaddr: PLT_STUB,
size: 0x10,
symbol: stated.to_owned(),
}];
program
}
/// [`Self::plt_importing`], its caller writing `edi` before the call.
pub fn plt_passing(import: &str) -> Self {
let mut program = Self::plt_importing(import);
crates/r2engine/tests/engine/main.rs
+1
View File
@@ -29,6 +29,7 @@ mod reuse;
mod signature;
mod staged;
mod start;
mod structure;
mod variadic;
mod variadic_registers;
mod walk;
crates/r2engine/tests/engine/native.rs
+300 -2
View File
@@ -6803,9 +6803,17 @@ fn merges_rax_at(artifact: &r2ssa::SsaArtifact, block: u64) -> bool {
/// Whether a `rax` the C assigns before its loop is assigned again inside it: the value the
/// back edge carries, rather than the one the loop was entered with on every pass.
fn loop_carries_rax(output: &str) -> bool {
let Some((before, inside)) = output.split_once("for (; ; ) {") else {
let opens = |line: &str| {
let line = line.trim();
line == "for (; ; ) {"
|| line == "do {"
|| (line.starts_with("while (") && line.ends_with(") {"))
};
let Some(at) = output.lines().position(opens) else {
return false;
};
let lines: Vec<&str> = output.lines().collect();
let (before, inside) = (lines[..at].join("\n"), lines[at + 1..].join("\n"));
let assigned = |text: &str| {
text.lines()
.filter_map(|line| line.trim_start().split_once(" = ").map(|(name, _)| name))
@@ -6813,7 +6821,7 @@ fn loop_carries_rax(output: &str) -> bool {
.map(str::to_owned)
.collect::<BTreeSet<_>>()
};
!assigned(before).is_disjoint(&assigned(inside))
!assigned(&before).is_disjoint(&assigned(&inside))
}
/// A loop that writes its accumulator only by `cmpxchg`'s or `cmovcc`'s `SELECT` carries it to its
@@ -7042,3 +7050,293 @@ fn registers_written_on_one_path_before_an_unknown_call_are_no_parameters() {
rendered.signature
);
}
/// `edi == 0 ? 5 : 7` the way a compiler joins two returns of different values: both arms move
/// theirs into eax and one `ret` follows.
const RETURNS_JOINED_BY_A_JUMP: &[u8] = &[
0x85, 0xff, // 1000 test edi, edi
0x74, 0x07, // 1002 je 0x100b
0xb8, 0x07, 0x00, 0x00, 0x00, // 1004 mov eax, 7
0xeb, 0x05, // 1009 jmp 0x1010
0xb8, 0x05, 0x00, 0x00, 0x00, // 100b mov eax, 5
0xc3, // 1010 ret
];
/// The same two arms with the same value in both, which is one return the source wrote once.
const ONE_VALUE_JOINED_BY_A_JUMP: &[u8] = &[
0x85, 0xff, // 1000 test edi, edi
0x74, 0x07, // 1002 je 0x100b
0xb8, 0x05, 0x00, 0x00, 0x00, // 1004 mov eax, 5
0xeb, 0x05, // 1009 jmp 0x1010
0xb8, 0x05, 0x00, 0x00, 0x00, // 100b mov eax, 5
0xc3, // 1010 ret
];
#[test]
fn returns_a_compiler_joined_are_written_in_the_arms_that_choose_their_values() {
let text = rendered(RETURNS_JOINED_BY_A_JUMP, "joined_returns");
assert_eq!(statements_returning(&text), 2, "{text}");
assert!(!text.contains("goto"), "{text}");
run_rendered(
"joined_returns",
&[&text],
r#"int main(void) {
if (joined_returns(0) != 5) return 1;
if (joined_returns(1) != 7) return 2;
if (joined_returns(0x100000000ull) != 5) return 3;
return 0;
}"#,
);
}
#[test]
fn a_value_every_edge_carries_is_one_return() {
let text = rendered(ONE_VALUE_JOINED_BY_A_JUMP, "one_value");
assert_eq!(statements_returning(&text), 1, "{text}");
}
/// `edi == 0 && esi == 0 ? 1 : 0` as a compiler joins it: both tests jump to the `xor`, the `1`
/// arm jumps over it, and one `ret` follows.
const CONJUNCTION_JOINED_BY_A_JUMP: &[u8] = &[
0x85, 0xff, // 1000 test edi, edi
0x75, 0x0b, // 1002 jne 0x100f
0x85, 0xf6, // 1004 test esi, esi
0x75, 0x07, // 1006 jne 0x100f
0xb8, 0x01, 0x00, 0x00, 0x00, // 1008 mov eax, 1
0xeb, 0x02, // 100d jmp 0x1011
0x31, 0xc0, // 100f xor eax, eax
0xc3, // 1011 ret
];
#[test]
fn a_conjunction_whose_arms_join_at_one_return_returns_in_each_arm() {
// The chain stage makes one test of the two, then the arms that fall to the `ret` each hold it.
let text = rendered(CONJUNCTION_JOINED_BY_A_JUMP, "both_zero");
let tests = text
.lines()
.filter(|line| line.trim_start().starts_with("if "));
assert_eq!(tests.clone().count(), 1, "{text}");
assert!(tests.clone().all(|line| line.contains(" && ")), "{text}");
assert_eq!(statements_returning(&text), 2, "{text}");
assert!(!text.contains("goto"), "{text}");
run_rendered(
"both_zero",
&[&text],
r#"int main(void) {
if (both_zero(0, 0) != 1) return 1;
if (both_zero(1, 0) != 0) return 2;
if (both_zero(0, 1) != 0) return 3;
if (both_zero(1, 1) != 0) return 4;
return 0;
}"#,
);
}
/// How many `return` statements a rendering writes.
fn statements_returning(text: &str) -> usize {
(text.lines())
.filter(|line| line.trim_start().starts_with("return"))
.count()
}
/// `while (*p) { if (*p == c) return 1; p++; } return 0;`: the loop has two exits, and the second
/// is reached from inside it.
const SEARCH_WITH_TWO_EXITS: &[u8] = &[
0x80, 0x3f, 0x00, // 1000 cmp byte [rdi], 0
0x74, 0x0a, // 1003 je 0x100f
0x40, 0x3a, 0x37, // 1005 cmp sil, [rdi]
0x74, 0x08, // 1008 je 0x1012
0x48, 0xff, 0xc7, // 100a inc rdi
0xeb, 0xf1, // 100d jmp 0x1000
0x31, 0xc0, // 100f xor eax, eax
0xc3, // 1011 ret
0xb8, 0x01, 0x00, 0x00, 0x00, // 1012 mov eax, 1
0xc3, // 1017 ret
];
#[test]
fn a_return_a_loop_exits_to_is_written_where_it_exits() {
let text = rendered(SEARCH_WITH_TWO_EXITS, "has_byte");
assert!(!text.contains("goto"), "{text}");
run_rendered(
"has_byte",
&[&text],
r#"int main(void) {
if ((uint32_t)has_byte((uint64_t)"abc", 'b') != 1) return 1;
if ((uint32_t)has_byte((uint64_t)"abc", 'c') != 1) return 2;
if ((uint32_t)has_byte((uint64_t)"abc", 'z') != 0) return 3;
if ((uint32_t)has_byte((uint64_t)"", 'z') != 0) return 4;
return 0;
}"#,
);
}
/// The same search whose end-of-string exit goes on to choose between two returns on `edx`: an
/// exit with one way in and a closed subtree of three blocks, beside the found exit.
const SEARCH_WITH_A_BRANCHING_EXIT: &[u8] = &[
0x80, 0x3f, 0x00, // 1000 cmp byte [rdi], 0
0x74, 0x0a, // 1003 je 0x100f
0x40, 0x3a, 0x37, // 1005 cmp sil, [rdi]
0x74, 0x15, // 1008 je 0x101f
0x48, 0xff, 0xc7, // 100a inc rdi
0xeb, 0xf1, // 100d jmp 0x1000
0x85, 0xd2, // 100f test edx, edx
0x74, 0x06, // 1011 je 0x1019
0xb8, 0x00, 0x00, 0x00, 0x00, // 1013 mov eax, 0
0xc3, // 1018 ret
0xb8, 0x03, 0x00, 0x00, 0x00, // 1019 mov eax, 3
0xc3, // 101e ret
0xb8, 0x01, 0x00, 0x00, 0x00, // 101f mov eax, 1
0xc3, // 1024 ret
];
#[test]
fn an_exit_that_only_the_loop_reaches_is_written_in_the_loop_with_what_it_goes_on_to() {
let text = rendered(SEARCH_WITH_A_BRANCHING_EXIT, "which_byte");
assert!(!text.contains("goto"), "{text}");
run_rendered(
"which_byte",
&[&text],
r#"int main(void) {
if ((uint32_t)which_byte((uint64_t)"abc", 'b', 1) != 1) return 1;
if ((uint32_t)which_byte((uint64_t)"abc", 'c', 0) != 1) return 2;
if ((uint32_t)which_byte((uint64_t)"abc", 'z', 1) != 0) return 3;
if ((uint32_t)which_byte((uint64_t)"abc", 'z', 0) != 3) return 4;
return 0;
}"#,
);
}
/// Code whose second function, at `callee`, the first calls and no summary reads, so each call
/// declares the callee from its own arguments and result.
struct CallsAFunction {
code: &'static [u8],
callee: u64,
/// The stub of the import the callee ends by jumping to, and its name.
import: (u64, &'static str),
}
impl r2engine::body::Program for CallsAFunction {
fn read(&self, vaddr: u64, max: usize) -> Option<Vec<u8>> {
let offset = usize::try_from(vaddr.checked_sub(BASE)?).ok()?;
let slice = self.code.get(offset..)?;
(!slice.is_empty()).then(|| slice[..slice.len().min(max)].to_vec())
}
fn region(&self, vaddr: u64) -> Option<r2engine::body::Region> {
code_region(self.code.len(), vaddr)
}
fn is_entry(&self, vaddr: u64) -> bool {
vaddr == BASE || vaddr == self.callee || self.import_at(vaddr).is_some()
}
}
impl Program for CallsAFunction {
fn holds_static_data(&self, _vaddr: u64) -> bool {
false
}
fn extents(&self) -> &r2types::ProgramExtents {
const NONE: &r2types::ProgramExtents = &r2types::ProgramExtents::none();
NONE
}
fn name_at(&self, vaddr: u64) -> Option<String> {
self.import_at(vaddr)
.or_else(|| (vaddr == BASE).then(|| "scan".to_owned()))
}
fn import_at(&self, vaddr: u64) -> Option<String> {
(vaddr == self.import.0).then(|| self.import.1.to_owned())
}
}
/// Two calls of `release` disagree about a result; the layouts write them in opposite orders, the
/// declaration and the call left as a residual must not differ.
#[test]
fn a_callee_is_declared_by_its_first_call_in_address_order_whichever_block_is_written_first() {
let machine = Machine::new("x86-64", "x86-64", 64);
let target = machine.target();
// The higher exit is written in the loop in the first, after it in the second.
for (code, callee, higher) in [
(RELEASE_AT_BOTH_EXITS, 0x1030, 0x1015),
(RELEASE_AFTER_A_SECOND_WAY_IN, 0x1034, 0x101a),
] {
let program = CallsAFunction {
code,
callee,
import: (callee + 30, "free"),
};
let response = decompile(&target, &program, BASE).expect("decompile");
let text = response.output.text();
let declaration = format!("uint64_t fcn_{callee:x}(uint64_t, uint64_t);");
assert!(text.contains(&declaration), "{declaration}:\n{text}");
let disagrees = "another call disagrees with its declaration";
assert_eq!(text.matches(disagrees).count(), 1, "{text}");
let gap = format!("CallNotRendered at {higher:#x}:");
assert!(text.contains(&gap), "{gap}:\n{text}");
}
}
/// `while (*p) { if (*p == c) return release(p, c); p++; } release(p, c); return 0;`, the lower exit
/// returning what the call leaves in `rax` and the higher overwriting it.
const RELEASE_AT_BOTH_EXITS: &[u8] = &[
0x80, 0x3f, 0x00, // 1000 cmp byte [rdi], 0
0x74, 0x10, // 1003 je 0x1015
0x40, 0x3a, 0x37, // 1005 cmp sil, [rdi]
0x74, 0x05, // 1008 je 0x100f
0x48, 0xff, 0xc7, // 100a inc rdi
0xeb, 0xf1, // 100d jmp 0x1000
0xe8, 0x1c, 0x00, 0x00, 0x00, // 100f call 0x1030
0xc3, // 1014 ret
0xe8, 0x16, 0x00, 0x00, 0x00, // 1015 call 0x1030
0xb8, 0x00, 0x00, 0x00, 0x00, // 101a mov eax, 0
0xc3, // 101f ret
0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90,
0x90, // 1020 padding
0x48, 0x85, 0xff, // +0 release: test rdi, rdi
0x74, 0x13, // +3 je +24
0x48, 0x85, 0xf6, // +5 test rsi, rsi
0x74, 0x0e, // +8 je +24
0x48, 0x8b, 0x87, 0x78, 0x04, 0x00, 0x00, // +10 mov rax, [rdi + 0x478]
0x48, 0x85, 0xc0, // +17 test rax, rax
0x74, 0x03, // +20 je +25
0xff, 0xe0, // +22 jmp rax
0xc3, // +24 ret
0x48, 0x89, 0xf7, // +25 mov rdi, rsi
0xeb, 0x00, // +28 jmp +30
0xc3, // +30 free's stub
];
/// The same search entered through a test of `rdi` that also reaches the higher exit, so that exit
/// has a second way in and the lower one is the exit written in the loop.
const RELEASE_AFTER_A_SECOND_WAY_IN: &[u8] = &[
0x48, 0x85, 0xff, // 1000 test rdi, rdi
0x74, 0x15, // 1003 je 0x101a
0x80, 0x3f, 0x00, // 1005 cmp byte [rdi], 0
0x74, 0x10, // 1008 je 0x101a
0x40, 0x3a, 0x37, // 100a cmp sil, [rdi]
0x74, 0x05, // 100d je 0x1014
0x48, 0xff, 0xc7, // 100f inc rdi
0xeb, 0xf1, // 1012 jmp 0x1005
0xe8, 0x1b, 0x00, 0x00, 0x00, // 1014 call 0x1034
0xc3, // 1019 ret
0xe8, 0x15, 0x00, 0x00, 0x00, // 101a call 0x1034
0xb8, 0x00, 0x00, 0x00, 0x00, // 101f mov eax, 0
0xc3, // 1024 ret
0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90, 0x90,
0x90, // 1025 padding
0x48, 0x85, 0xff, // +0 release: test rdi, rdi
0x74, 0x13, // +3 je +24
0x48, 0x85, 0xf6, // +5 test rsi, rsi
0x74, 0x0e, // +8 je +24
0x48, 0x8b, 0x87, 0x78, 0x04, 0x00, 0x00, // +10 mov rax, [rdi + 0x478]
0x48, 0x85, 0xc0, // +17 test rax, rax
0x74, 0x03, // +20 je +25
0xff, 0xe0, // +22 jmp rax
0xc3, // +24 ret
0x48, 0x89, 0xf7, // +25 mov rdi, rsi
0xeb, 0x00, // +28 jmp +30
0xc3, // +30 free's stub
];
crates/r2engine/tests/engine/rendered.rs
+23
View File
@@ -51,6 +51,29 @@ fn a_call_to_an_import_declared_never_to_return_ends_the_function() {
assert!(!returns, "{c}");
}
#[test]
fn an_arm_ending_in_a_call_declared_never_to_return_is_a_conjunction_that_leaves_by_it() {
// `test edi, edi; jne no; test esi, esi; jne no; mov edi, 1; call exit; no: xor eax, eax; ret`: the
// arm that calls `exit` ends control, so the certificate must read it so, at each stage and at the end.
let mut program = OpenProgram::of(Literal::new().importing("exit").running_on(GLIBC));
let call = i32::try_from(STUB as i64 - (TWO + 18) as i64).expect("near");
let mut code = vec![
0x85, 0xff, 0x75, 0x0e, 0x85, 0xf6, 0x75, 0x0a, 0xbf, 0x01, 0, 0, 0, 0xe8,
];
code.extend_from_slice(&call.to_le_bytes());
code.extend_from_slice(&[0x31, 0xc0, 0xc3]);
program.source_mut().write(TWO, &code);
let c = c_of(&mut program, TWO);
assert!(!c.contains("r2sleigh refused"), "{c}");
let tests: Vec<_> = c
.lines()
.filter(|line| line.trim_start().starts_with("if "))
.collect();
assert_eq!(tests.len(), 1, "{c}");
assert!(tests[0].contains(" && ") && !c.contains("goto"), "{c}");
assert!(c.contains("exit(1)"), "{c}");
}
/// `g: jmp t` then `t: lea eax, [rdi + rdi*2 + 1]; ret`, each a stated function.
const TAIL_JUMPING: &[u8] = &[
0xe9, 0x0b, 0x00, 0x00, 0x00, // g: jmp t
crates/r2engine/tests/engine/staged.rs
+82 -1
View File
@@ -3,7 +3,7 @@
use crate::common;
use common::{BASE, CALLER, FORKED, GLIBC, Literal, ONE, PLT_STUB, STUB, TWO, opened};
use common::{BASE, CALLER, FORKED, GLIBC, Literal, ONE, PLT_CALLER, PLT_STUB, STUB, TWO, opened};
use r2engine::program::OpenProgram;
#[test]
@@ -156,3 +156,84 @@ fn an_import_stub_is_its_import_s_declaration() {
assert!(!text.contains("return"), "{text}");
assert!(!text.contains("r2sleigh_residual"), "{text}");
}
/// A function that reaches the stub after a test of its own is a body: a branch or a return on
/// another path is not the import's declaration, and the C keeps both paths.
#[test]
fn a_guarded_wrapper_of_a_stub_is_a_body_not_the_stub() {
let mut program = OpenProgram::of(Literal::plt_guarded_wrapper());
let text = program
.rendered(PLT_CALLER)
.expect("it renders")
.response
.output
.into_text();
assert!(!text.contains("import stub at"), "{text}");
assert!(text.contains("free("), "{text}");
assert!(text.contains("if ("), "{text}");
}
/// The stub's argument is a value the wrapper computes, so the wrapper is not the import: calling
/// it passes what it computes from, and the C says so.
#[test]
fn a_wrapper_that_computes_the_stub_s_argument_is_a_body_not_the_stub() {
let mut program = OpenProgram::of(Literal::plt_computing_wrapper());
let text = program
.rendered(PLT_CALLER)
.expect("it renders")
.response
.output
.into_text();
assert!(!text.contains("import stub at"), "{text}");
assert!(text.contains("malloc(") && text.contains(") * ("), "{text}");
}
/// One jump to the stub, with nothing else, is still a jump to another function: only the stub
/// itself jumps through the loader's slot, so this one is a body that calls the import.
#[test]
fn a_function_that_only_jumps_to_a_stub_is_a_body_not_the_stub() {
let mut program = OpenProgram::of(Literal::plt_forwarding_wrapper());
let text = program
.rendered(PLT_CALLER)
.expect("it renders")
.response
.output
.into_text();
assert!(!text.contains("import stub at"), "{text}");
assert!(text.contains("free((void*)arg0);"), "{text}");
}
/// The container states a stub of `free` whose jump goes through the slot bound to `malloc`: a body, not a
/// `free` declared with `malloc`'s types.
#[test]
fn a_stub_whose_jump_goes_to_another_import_is_a_body_not_the_stated_stub() {
let mut program = OpenProgram::of(Literal::plt_declaring("free", "malloc"));
let text = program
.rendered(PLT_STUB)
.expect("it renders")
.response
.output
.into_text();
assert!(
!text.contains("import stub at") && !text.contains("extern"),
"{text}"
);
assert!(text.contains("r2sleigh_residual"), "{text}");
}
/// A stub the container states, whose jump goes through the slot bound to the same import, is that import's
/// declaration.
#[test]
fn a_stub_the_container_states_is_its_import_s_declaration_where_its_jump_agrees() {
let mut program = OpenProgram::of(Literal::plt_declaring("_Exit", "_Exit"));
let text = program
.rendered(PLT_STUB)
.expect("it renders")
.response
.output
.into_text();
assert!(
text.contains(&format!("import stub at {PLT_STUB:#x}")) && text.contains("`_Exit`"),
"{text}"
);
}
crates/r2engine/tests/engine/structure.rs
+761
View File
@@ -0,0 +1,761 @@
//! What the structurer writes for a function, from its bytes and an address.
use crate::common;
use std::collections::BTreeSet;
use common::{BASE, Literal};
use r2dec::placement_mutants::{WriteOrder, with_write_order};
use r2engine::program::OpenProgram;
fn c_of(code: &'static [u8]) -> String {
let size = code.len() as u64;
let mut program = OpenProgram::of(Literal::of_code(code, &[("f", BASE, size)]));
program
.rendered(BASE)
.expect("it renders")
.response
.output
.into_text()
}
/// The `if` lines of a rendering.
fn tests_of(c: &str) -> Vec<&str> {
c.lines()
.map(str::trim_start)
.filter(|line| line.starts_with("if "))
.collect()
}
/// `test edi, edi; jne no; test esi, esi; jne no; mov eax, 1; ret; no: xor eax, eax; ret`
const BOTH_ZERO: &[u8] = &[
0x85, 0xff, 0x75, 0x0a, 0x85, 0xf6, 0x75, 0x06, 0xb8, 0x01, 0x00, 0x00, 0x00, 0xc3, 0x31, 0xc0,
0xc3,
];
/// `test edi, edi; jne yes; test esi, esi; je no; yes: mov eax, 1; ret; no: xor eax, eax; ret`
const EITHER_NOT_ZERO: &[u8] = &[
0x85, 0xff, 0x75, 0x04, 0x85, 0xf6, 0x74, 0x06, 0xb8, 0x01, 0x00, 0x00, 0x00, 0xc3, 0x31, 0xc0,
0xc3,
];
/// `test edi, edi; jne no; cmp dword [rsi], 0; jne no; mov eax, 1; ret; no: xor eax, eax; ret`
const SECOND_TEST_LOADS: &[u8] = &[
0x85, 0xff, 0x75, 0x0b, 0x83, 0x3e, 0x00, 0x75, 0x06, 0xb8, 0x01, 0x00, 0x00, 0x00, 0xc3, 0x31,
0xc0, 0xc3,
];
#[test]
fn two_tests_that_fail_to_one_exit_are_one_conjunction() {
let c = c_of(BOTH_ZERO);
let tests = tests_of(&c);
assert_eq!(tests.len(), 1, "{c}");
// The text is the machine's: both zero, which is what the source says.
assert_eq!(tests[0].matches("== (uint32_t)0U").count(), 2, "{c}");
assert!(tests[0].contains(" && ") && !tests[0].contains('!'), "{c}");
assert!(!c.contains("goto"), "{c}");
}
#[test]
fn the_exit_only_the_chain_reaches_is_the_if_s_body() {
// Either input non-zero returns 1, which both tests reach; zero for both returns 0, which only
// the second test does, so that block is the body and the first block is the jump.
let c = c_of(EITHER_NOT_ZERO);
let tests = tests_of(&c);
assert_eq!(tests.len(), 1, "{c}");
assert_eq!(tests[0].matches("== (uint32_t)0U").count(), 2, "{c}");
assert!(tests[0].contains(" && "), "{c}");
assert!(c.contains("return 0;"), "{c}");
}
#[test]
fn a_second_test_that_reads_memory_is_a_test_of_its_own() {
// The load is a statement between the two tests, so the second block is no part of the first's.
let c = c_of(SECOND_TEST_LOADS);
assert_eq!(tests_of(&c).len(), 2, "{c}");
assert!(!c.contains(" && ") && !c.contains(" || "), "{c}");
}
/// `test edi, edi; jne no; sub esi, 5; jne no; mov eax, esi; ret; no: xor eax, eax; ret`: the second
/// test's block also computes the value the function returns.
const SECOND_TEST_KEEPS_ITS_VALUE: &[u8] = &[
0x85, 0xff, 0x75, 0x08, 0x83, 0xee, 0x05, 0x75, 0x03, 0x89, 0xf0, 0xc3, 0x31, 0xc0, 0xc3,
];
/// The same, but the function returns 1: the subtraction is only its flags.
const SECOND_TEST_ONLY_COMPARES: &[u8] = &[
0x85, 0xff, 0x75, 0x0b, 0x83, 0xee, 0x05, 0x75, 0x06, 0xb8, 0x01, 0x00, 0x00, 0x00, 0xc3, 0x31,
0xc0, 0xc3,
];
#[test]
fn a_second_test_joins_the_chain_only_if_it_computes_nothing_the_function_keeps() {
// r2ssa says both blocks owe nothing but their branch and what they carry; only the text
// knows whether the subtraction is a statement of its own, which no `&&` may drop.
let kept = c_of(SECOND_TEST_KEEPS_ITS_VALUE);
assert_eq!(tests_of(&kept).len(), 2, "{kept}");
assert!(!kept.contains(" && "), "{kept}");
assert!(kept.contains("arg1 - (uint32_t)5U"), "{kept}");
let flags = c_of(SECOND_TEST_ONLY_COMPARES);
assert_eq!(tests_of(&flags).len(), 1, "{flags}");
assert!(flags.contains(" && "), "{flags}");
}
/// `xor eax, eax; jmp test; body: inc eax; test: cmp eax, edi; jl body; ret`
const COUNT_UP: &[u8] = &[
0x31, 0xc0, 0xeb, 0x02, 0xff, 0xc0, 0x39, 0xf8, 0x7c, 0xfa, 0xc3,
];
#[test]
fn a_loop_that_tests_before_its_body_is_a_while() {
let c = c_of(COUNT_UP);
assert_eq!(c.matches("while (").count(), 1, "{c}");
assert!(!c.contains("for (; ; )") && !c.contains("break"), "{c}");
// The test is the machine's: stay while the counter is below the bound.
let test = c
.lines()
.find(|line| line.contains("while ("))
.expect("a loop");
assert!(test.contains('<'), "{c}");
}
/// `xor eax, eax; test: cmp eax, edi; jge done; inc eax; jmp test; done: ret`: the header leaves the
/// loop when its jump is taken.
const COUNT_UNTIL: &[u8] = &[
0x31, 0xc0, 0x39, 0xf8, 0x7d, 0x04, 0xff, 0xc0, 0xeb, 0xf8, 0xc3,
];
#[test]
fn a_loop_that_leaves_when_its_jump_is_taken_stays_on_the_opposite_test() {
let c = c_of(COUNT_UNTIL);
let test = c
.lines()
.find(|line| line.contains("while ("))
.unwrap_or_else(|| panic!("a while: {c}"));
// `jge done` leaves once i >= arg0, so the loop stays while arg0 > i, and the text says that.
assert!(
test.contains("arg0 > ") && !test.contains(">=") && !test.contains('!'),
"{c}"
);
assert!(!c.contains("for (; ; )") && !c.contains("break"), "{c}");
}
/// `test edi, edi; je skip; xor eax, eax; body: inc eax; cmp eax, edi; jne body; ret; skip: xor eax, eax; ret`
const COUNT_TO: &[u8] = &[
0x85, 0xff, 0x74, 0x09, 0x31, 0xc0, 0xff, 0xc0, 0x39, 0xf8, 0x75, 0xfa, 0xc3, 0x31, 0xc0, 0xc3,
];
#[test]
fn a_loop_that_tests_after_its_body_is_a_do_while_with_its_merge_copies_before_the_test() {
let c = c_of(COUNT_TO);
assert_eq!(c.matches("do {").count(), 1, "{c}");
assert!(!c.contains("for (; ; )") && !c.contains("break"), "{c}");
// The copy that carries the counter round the loop is written before the test reads the next value.
let lines: Vec<&str> = c.lines().map(str::trim).collect();
let end = lines
.iter()
.position(|line| line.starts_with("} while ("))
.expect("a do-while");
assert!(
lines[end - 1].contains(" = ") && !lines[end - 1].contains("if"),
"{c}"
);
}
/// `test edi, edi; je skip; xor eax, eax; body: mov edx, eax; inc eax; cmp eax, edi; jne body; mov eax, edx;
/// ret; skip: xor eax, eax; ret`: the code after the loop wants the counter as the last iteration began it.
const LAST_BEGUN: &[u8] = &[
0x85, 0xff, 0x74, 0x0d, 0x31, 0xc0, 0x89, 0xc2, 0xff, 0xc0, 0x39, 0xf8, 0x75, 0xf8, 0x89, 0xd0,
0xc3, 0x31, 0xc0, 0xc3,
];
#[test]
fn what_the_code_after_a_loop_reads_is_not_what_the_hoisted_copy_writes() {
// The result is the counter as the last iteration began it, which the text keeps in a variable
// of its own, so the copy of the next value may still be written before the test.
let c = c_of(LAST_BEGUN);
let lines: Vec<&str> = c.lines().map(str::trim).collect();
let end = lines
.iter()
.position(|line| line.starts_with("} while ("))
.expect("a do-while");
let copied = lines[end - 1].split(" = ").next().expect("a copy");
let returned = lines
.iter()
.find(|line| line.starts_with("return (uint32_t)"))
.expect("a return of the loop's value");
assert!(!returned.contains(copied), "{c}");
}
/// `top: movzx edx, byte [rdi]; add rdi, 1; test dl, dl; je done; cmp dl, 0x3a; jne top; done: lea rax,
/// [rdi - 1]; ret`: the second test goes back to the first, so the chain's exits are `done` and its own head.
const SCAN_TO_SEPARATOR: &[u8] = &[
0x0f, 0xb6, 0x17, 0x48, 0x83, 0xc7, 0x01, 0x84, 0xd2, 0x74, 0x05, 0x80, 0xfa, 0x3a, 0x75, 0xf0,
0x48, 0x8d, 0x47, 0xff, 0xc3,
];
#[test]
fn a_chain_whose_exit_is_a_continue_to_its_own_head_is_one_test() {
// Stop at a zero byte or a colon: the chain `c == 0 || c == 58` leaves, and its other exit is the
// head itself, so the loop is a `do-while` on `c != 0 && c != 58` with nothing left to `break`.
let c = c_of(SCAN_TO_SEPARATOR);
let test = c
.lines()
.find(|line| line.contains("} while ("))
.unwrap_or_else(|| panic!("a do-while: {c}"));
assert!(
test.contains(" && ") && test.contains("!= (uint8_t)0U"),
"{c}"
);
assert!(!c.contains("for (; ; )") && !c.contains("break"), "{c}");
assert_eq!(tests_of(&c).len(), 0, "{c}");
}
/// `xor eax, eax; top: movzx r8d, byte [rdi + rax]; mov [rdx], r8b; lea ecx, [rax + 1]; cmp eax, esi;
/// mov eax, ecx; jne top; xor eax, eax; ret`: the test compares the index the iteration began with.
const TEST_READS_THE_COPY_TARGET: &[u8] = &[
0x31, 0xc0, 0x44, 0x0f, 0xb6, 0x04, 0x07, 0x44, 0x88, 0x02, 0x48, 0x8d, 0x48, 0x01, 0x39, 0xf0,
0x40, 0x89, 0xc8, 0x75, 0xed, 0x31, 0xc0, 0xc3,
];
#[test]
fn a_copy_stays_on_its_arm_when_the_test_reads_the_variable_it_writes() {
// Written before the test, `i = i + 1` would make `i != n` compare the next index, not this one.
let c = c_of(TEST_READS_THE_COPY_TARGET);
assert!(c.contains("for (; ; )") && !c.contains("do {"), "{c}");
assert!(c.contains("!= (uint32_t)arg1"), "{c}");
}
/// A loop with two ways out before its latch (`je early` twice) to a tail of five stores that reads the index,
/// and a latch `lea ecx, [rax + 1]; mov eax, ecx; cmp eax, esi; jne top`.
const TAIL_READS_THE_COPY_TARGET: &[u8] = &[
0x31, 0xc0, 0x44, 0x0f, 0xb6, 0x04, 0x07, 0x41, 0x80, 0xf8, 0x2c, 0x74, 0x0a, 0x45, 0x84, 0xc0,
0x74, 0x1f, 0x44, 0x88, 0x02, 0xeb, 0x09, 0x45, 0x84, 0xc0, 0x74, 0x15, 0x44, 0x88, 0x42, 0x01,
0x48, 0x8d, 0x48, 0x01, 0x40, 0x89, 0xc8, 0x39, 0xf0, 0x75, 0xd7, 0xb8, 0xff, 0xff, 0xff, 0xff,
0xc3, 0x88, 0x42, 0x02, 0x88, 0x42, 0x03, 0x88, 0x42, 0x04, 0x88, 0x42, 0x05, 0x88, 0x42, 0x06,
0xc3,
];
#[test]
fn a_copy_stays_on_its_arm_when_code_after_the_loop_reads_what_it_writes() {
// The tail after the loop reads `i`, and nothing here says which of the loop's exits reaches it,
// so the copy is written only where the loop goes on (ADR §5).
let c = c_of(TAIL_READS_THE_COPY_TARGET);
assert!(c.contains("for (; ; )") && !c.contains("do {"), "{c}");
let tail = c
.split("L2: ;")
.nth(1)
.unwrap_or_else(|| panic!("a tail: {c}"));
assert!(tail.contains("rax_2"), "{c}");
}
/// `push rbx; xor ebx, ebx; top: call 0x2000; inc ebx; test eax, eax; jne top; mov eax, ebx; pop rbx; ret`:
/// the test reads a call's result, which nothing proves.
const TEST_READS_AN_UNPROVEN_CALL: &[u8] = &[
0x53, 0x31, 0xdb, 0xe8, 0xf8, 0x0f, 0x00, 0x00, 0xff, 0xc3, 0x85, 0xc0, 0x75, 0xf5, 0x40, 0x89,
0xd8, 0x5b, 0xc3,
];
#[test]
fn a_copy_stays_on_its_arm_when_the_test_is_not_a_proven_read() {
// The copy is not moved across a test that traps where its cell is unproven (ADR §5).
let c = c_of(TEST_READS_AN_UNPROVEN_CALL);
assert!(c.contains("for (; ; )") && !c.contains("do {"), "{c}");
assert!(c.contains("r2sleigh_residual_u64"), "{c}");
}
/// `top: movzx edx, byte [rdi]; test dl, dl; je done; cmp dl, 0x3a; jne top; done: xor eax, eax; ret`: nothing
/// is copied on the way round, so the chain is spelled on the side the loop leaves by.
const SPIN_UNTIL_SEPARATOR: &[u8] = &[
0x0f, 0xb6, 0x17, 0x84, 0xd2, 0x74, 0x05, 0x80, 0xfa, 0x3a, 0x75, 0xf4, 0x31, 0xc0, 0xc3,
];
#[test]
fn a_loop_that_leaves_on_a_chain_stays_on_both_tests_negated() {
// It leaves when `c == 0 || c == 58`, so it stays when `c != 0 && c != 58`: the connective swaps with the tests.
let c = c_of(SPIN_UNTIL_SEPARATOR);
let test = c
.lines()
.find(|line| line.contains("} while ("))
.unwrap_or_else(|| panic!("a do-while: {c}"));
assert_eq!(test.matches(" && ").count(), 1, "{c}");
assert!(
!test.contains(" || ") && test.matches("!= (uint8_t)").count() == 2,
"{c}"
);
}
/// The four write orders a placement can be asked for: the natural one and the three mutants (ROADMAP D58).
const ORDERS: [WriteOrder; 4] = [
WriteOrder::Natural,
WriteOrder::KeepExits,
WriteOrder::ExitsReversed,
WriteOrder::MergesReversed,
];
/// What no block write order may change: the ledger's columns, the never-assigned names and the set of
/// statements apart from control and site numbers (a set: a tail is written once per jump to it).
#[derive(Debug, PartialEq, Eq)]
struct Verdicts {
ledger: String,
never_assigned: String,
statements: BTreeSet<String>,
}
fn verdicts_of(c: &str) -> Verdicts {
let proof = c
.lines()
.find(|line| line.contains("r2dec proof:"))
.unwrap_or_else(|| panic!("a proof line: {c}"));
let between = |from: &str, to: char| {
proof
.split_once(from)
.map(|(_, rest)| rest.split(to).next().unwrap_or_default().to_owned())
.unwrap_or_default()
};
let site = regex_free_site_numbers;
Verdicts {
ledger: between("source obligations:", ';'),
never_assigned: between("never assigned, read as residuals (", ')'),
statements: c
.lines()
.map(str::trim)
.filter(|line| line.ends_with(';') && !line.starts_with("goto "))
.filter(|line| !matches!(*line, "break;" | "continue;") && !line.ends_with(": ;"))
.map(site)
.collect(),
}
}
/// `line` with each residual site's number replaced by `N`: sites are numbered in text order.
fn regex_free_site_numbers(line: &str) -> String {
let mut out = String::new();
let mut rest = line;
while let Some(at) = rest.find("r2sleigh_residual_") {
let (head, tail) = rest.split_at(at);
out.push_str(head);
let open = tail.find('(').map_or(tail.len(), |open| open + 1);
out.push_str(&tail[..open]);
let digits = tail[open..]
.chars()
.take_while(char::is_ascii_digit)
.count();
out.push('N');
rest = &tail[open + digits..];
}
out.push_str(rest);
out
}
/// A rendering of `code` with blocks written in `order`; each is a fresh program, so no memo is shared.
fn c_under(order: WriteOrder, code: &'static [u8]) -> String {
with_write_order(order, || c_of(code))
}
/// The lines that name the variable `name` other than as the target of an assignment to it.
fn reads_of<'c>(c: &'c str, name: &str) -> Vec<&'c str> {
c.lines()
.map(str::trim)
.filter(|line| line.contains(name) && !line.starts_with(&format!("{name} = ")))
.filter(|line| !line.ends_with(&format!(" {name};")))
.collect()
}
/// `sub rsp, 0x18; lea rax, [rsp + 8]; loop: cmp byte [rdi], 0; je out; cmp sil, [rdi]; je found;
/// add rax, 1; add rdi, 1; jmp loop; out: add rsp, 0x18; ret; found: add rax, 5; add rsp, 0x18; ret`
const FRAME_ADDRESS_LOOP: &[u8] = &[
0x48, 0x83, 0xec, 0x18, 0x48, 0x8d, 0x44, 0x24, 0x08, 0x80, 0x3f, 0x00, 0x74, 0x0f, 0x40, 0x3a,
0x37, 0x74, 0x0f, 0x48, 0x83, 0xc0, 0x01, 0x48, 0x83, 0xc7, 0x01, 0xeb, 0xec, 0x48, 0x83, 0xc4,
0x18, 0xc3, 0x48, 0x83, 0xc0, 0x05, 0x48, 0x83, 0xc4, 0x18, 0xc3,
];
/// The same, with `test rdx, rdx; je found2` before the loop: a second way to the exit that adds 5,
/// which makes that exit a merge written after the loop whichever order the exits are written in.
const FRAME_ADDRESS_LOOP_TWO_WAYS: &[u8] = &[
0x48, 0x83, 0xec, 0x18, 0x48, 0x8d, 0x44, 0x24, 0x08, 0x48, 0x85, 0xd2, 0x74, 0x19, 0x80, 0x3f,
0x00, 0x74, 0x0f, 0x40, 0x3a, 0x37, 0x74, 0x0f, 0x48, 0x83, 0xc0, 0x01, 0x48, 0x83, 0xc7, 0x01,
0xeb, 0xec, 0x48, 0x83, 0xc4, 0x18, 0xc3, 0x48, 0x83, 0xc0, 0x05, 0x48, 0x83, 0xc4, 0x18, 0xc3,
];
#[test]
fn a_merge_whose_entry_copy_is_a_gap_reads_as_a_residual_under_every_write_order() {
// The loop's merge enters with a frame address (a gap); its only other copy is the latch's.
let mut texts = BTreeSet::new();
for code in [FRAME_ADDRESS_LOOP, FRAME_ADDRESS_LOOP_TWO_WAYS] {
let natural = c_under(WriteOrder::Natural, code);
assert!(natural.contains("ValuesNotRendered"), "{natural}");
let verdicts = verdicts_of(&natural);
for order in ORDERS {
let c = c_under(order, code);
assert_eq!(verdicts_of(&c), verdicts, "{order:?}\n{c}");
texts.insert(c);
}
// The loop's merge has no value on its way in, so no write order reads it as a variable.
assert!(verdicts.never_assigned.contains("RAX_2"), "{natural}");
for order in ORDERS {
let c = c_under(order, code);
let reads = reads_of(&c, "rax_2");
assert!(
reads
.iter()
.all(|line| line.contains("r2sleigh_residual_u64(")),
"{order:?}: {reads:?}\n{c}"
);
}
}
// The orders do move the text (the knob is live), or equal verdicts would prove nothing.
assert!(
texts.len() > 2,
"the orders wrote {} distinct texts",
texts.len()
);
}
/// `sub rsp, 0x18; test rdi, rdi; je other; lea rax, [rsp + 8]; jmp join; other: mov rax, rsi;
/// join: add rsp, 0x18; ret`: one way into the join carries a frame address, which is a gap.
const ONE_GAPPED_COPY: &[u8] = &[
0x48, 0x83, 0xec, 0x18, 0x48, 0x85, 0xff, 0x74, 0x07, 0x48, 0x8d, 0x44, 0x24, 0x08, 0xeb, 0x03,
0x48, 0x89, 0xf0, 0x48, 0x83, 0xc4, 0x18, 0xc3,
];
#[test]
fn a_merge_with_one_gapped_forward_copy_reads_as_a_residual() {
// A gap traps where it stands, but the certificate lets control go on past it, so the value the
// merge holds on the other way in is no proof that a read of it is defined.
for order in ORDERS {
let c = c_under(order, ONE_GAPPED_COPY);
assert!(c.contains("ValuesNotRendered"), "{order:?}\n{c}");
assert!(
verdicts_of(&c).never_assigned.contains("RAX_3"),
"{order:?}\n{c}"
);
let reads = reads_of(&c, "rax_3");
assert!(
reads
.iter()
.all(|line| line.contains("r2sleigh_residual_u64(")),
"{order:?}: {reads:?}\n{c}"
);
}
}
/// `test rdi, rdi; je other; mov rax, rsi; jmp join; other: mov rax, rdx; join: ret`
const BOTH_COPIES_RENDER: &[u8] = &[
0x48, 0x85, 0xff, 0x74, 0x05, 0x48, 0x89, 0xf0, 0xeb, 0x03, 0x48, 0x89, 0xd0, 0xc3, 0x90, 0x90,
];
#[test]
fn a_merge_whose_forward_copies_all_render_reads_as_a_variable_in_every_order() {
let natural = verdicts_of(&c_under(WriteOrder::Natural, BOTH_COPIES_RENDER));
assert_eq!(natural.never_assigned, "");
for order in ORDERS {
let c = c_under(order, BOTH_COPIES_RENDER);
assert_eq!(verdicts_of(&c), natural, "{order:?}\n{c}");
assert!(!c.contains("r2sleigh_residual_"), "{order:?}\n{c}");
assert!(c.contains("return (uint64_t)rax_3;"), "{order:?}\n{c}");
}
}
/// `xor eax, eax; test edi, edi; je y; x: add eax, 1; cmp eax, esi; jl y; ret; y: add eax, 2; cmp eax, edx;
/// jl x; ret`: two ways into the cycle x, y, so neither dominates the other and one edge of it is late.
const IRREDUCIBLE: &[u8] = &[
0x31, 0xc0, 0x85, 0xff, 0x74, 0x08, 0x83, 0xc0, 0x01, 0x39, 0xf0, 0x7c, 0x01, 0xc3, 0x83, 0xc0,
0x02, 0x39, 0xd0, 0x7c, 0xf1, 0xc3,
];
#[test]
fn a_merge_entered_late_by_an_irreducible_edge_reads_as_a_variable_when_every_copy_renders() {
// The edge that closes the cycle is spelled after the reads of the merge it enters, and renders its
// copy, so those reads are defined; no write order and no spelling order changes that.
let natural = c_under(WriteOrder::Natural, IRREDUCIBLE);
assert!(!natural.contains("r2sleigh_residual_"), "{natural}");
for order in ORDERS {
let c = c_under(order, IRREDUCIBLE);
assert_eq!(verdicts_of(&c), verdicts_of(&natural), "{order:?}\n{c}");
assert!(!c.contains("r2sleigh_residual_"), "{order:?}\n{c}");
}
}
/// `sub rsp, 0x18; xor eax, eax; test edi, edi; je y; x: lea rax, [rsp + 8]; test esi, esi; jne y; add rsp,
/// 0x18; ret; y: add eax, 1; test edx, edx; jne x; add rsp, 0x18; ret`: the late edge carries a frame address.
const IRREDUCIBLE_GAPPED: &[u8] = &[
0x48, 0x83, 0xec, 0x18, 0x31, 0xc0, 0x85, 0xff, 0x74, 0x0e, 0x48, 0x8d, 0x44, 0x24, 0x08, 0x85,
0xf6, 0x75, 0x05, 0x48, 0x83, 0xc4, 0x18, 0xc3, 0x83, 0xc0, 0x01, 0x85, 0xd2, 0x75, 0xeb, 0x48,
0x83, 0xc4, 0x18, 0xc3,
];
#[test]
fn a_merge_entered_late_by_a_gapped_copy_reads_as_a_residual_whatever_was_promised_before() {
// The reads of the merge are spelled before the late edge, as variables on its promise; the edge
// gaps, so the function is spelled again with that merge owed one more.
let natural = verdicts_of(&c_under(WriteOrder::Natural, IRREDUCIBLE_GAPPED));
let carried = natural.never_assigned.to_lowercase();
assert!(!carried.is_empty() && !carried.contains(','), "{natural:?}");
for order in ORDERS {
let c = c_under(order, IRREDUCIBLE_GAPPED);
assert_eq!(verdicts_of(&c), natural, "{order:?}\n{c}");
assert!(c.contains("ValuesNotRendered"), "{order:?}\n{c}");
let reads = reads_of(&c, &carried);
assert!(
reads
.iter()
.all(|line| line.contains("r2sleigh_residual_u64(")),
"{order:?}: {reads:?}\n{c}"
);
}
}
/// The bytes of `body` at `BASE`, then `atan2f`'s linkage stub (`jmp qword [rip + 2]`) and its slot at `STUB`.
const fn with_the_atan2f_stub(body: &[u8]) -> [u8; 0xa0] {
let mut bytes = [0xcc; 0xa0];
let mut at = 0;
while at < body.len() {
bytes[at] = body[at];
at += 1;
}
let stub = [0xff, 0x25, 0x02, 0x00, 0x00, 0x00, 0, 0, 0, 0, 0, 0, 0, 0];
let mut at = 0;
while at < stub.len() {
bytes[0x90 + at] = stub[at];
at += 1;
}
bytes
}
/// `lane_nondom9`: `xmm1` merges two loads; one arm of `ucomiss xmm4, xmm2` calls `atan2f(xmm4, xmm1)`, the
/// other reads its low lane (`comiss xmm1, xmm2`). The listing is in the D68 section of the ADR.
static LANE_READ_BESIDE_THE_CALL: [u8; 0xa0] = with_the_atan2f_stub(&[
0x48, 0x83, 0xec, 0x08, 0xf3, 0x0f, 0x10, 0x62, 0x10, 0xf3, 0x0f, 0x10, 0x52, 0x14, 0x48, 0x85,
0xff, 0x74, 0x06, 0xf3, 0x0f, 0x10, 0x0e, 0xeb, 0x05, 0xf3, 0x0f, 0x10, 0x4e, 0x04, 0x0f, 0x2e,
0xe2, 0x76, 0x0a, 0x0f, 0x2f, 0xca, 0x76, 0x12, 0x48, 0x83, 0xc4, 0x08, 0xc3, 0x0f, 0x28, 0xc4,
0xe8, 0x5b, 0x00, 0x00, 0x00, 0x48, 0x83, 0xc4, 0x08, 0xc3, 0x48, 0x83, 0xc4, 0x08, 0xc3,
]);
/// The same with the branch the other way: `ja call`, so the call is on the taken edge.
static LANE_READ_BESIDE_THE_TAKEN_CALL: [u8; 0xa0] = with_the_atan2f_stub(&[
0x48, 0x83, 0xec, 0x08, 0xf3, 0x0f, 0x10, 0x62, 0x10, 0xf3, 0x0f, 0x10, 0x52, 0x14, 0x48, 0x85,
0xff, 0x74, 0x06, 0xf3, 0x0f, 0x10, 0x0e, 0xeb, 0x05, 0xf3, 0x0f, 0x10, 0x4e, 0x04, 0x0f, 0x2e,
0xe2, 0x77, 0x0a, 0x0f, 0x2f, 0xca, 0x76, 0x12, 0x48, 0x83, 0xc4, 0x08, 0xc3, 0x0f, 0x28, 0xc4,
0xe8, 0x5b, 0x00, 0x00, 0x00, 0x48, 0x83, 0xc4, 0x08, 0xc3, 0x48, 0x83, 0xc4, 0x08, 0xc3,
]);
/// The same as the first with the call on the fall-through edge and the lane read on the taken one.
static LANE_READ_BESIDE_THE_FALLTHROUGH_CALL: [u8; 0xa0] = with_the_atan2f_stub(&[
0x48, 0x83, 0xec, 0x08, 0xf3, 0x0f, 0x10, 0x62, 0x10, 0xf3, 0x0f, 0x10, 0x52, 0x14, 0x48, 0x85,
0xff, 0x74, 0x06, 0xf3, 0x0f, 0x10, 0x0e, 0xeb, 0x05, 0xf3, 0x0f, 0x10, 0x4e, 0x04, 0x0f, 0x2e,
0xe2, 0x76, 0x0d, 0x0f, 0x28, 0xc4, 0xe8, 0x65, 0x00, 0x00, 0x00, 0x48, 0x83, 0xc4, 0x08, 0xc3,
0x0f, 0x2f, 0xca, 0x76, 0x05, 0x48, 0x83, 0xc4, 0x08, 0xc3, 0x48, 0x83, 0xc4, 0x08, 0xc3,
]);
/// `c`, rendered from `code` with blocks written in `order`, the program importing `atan2f` at its stub.
fn atan2f_caller_under(order: WriteOrder, code: &'static [u8]) -> String {
with_write_order(order, || {
let literal = Literal::of_code(code, &[("f", BASE, 0x3f)])
.importing("atan2f")
.running_on(common::GLIBC);
OpenProgram::of(literal)
.rendered(BASE)
.expect("it renders")
.response
.output
.into_text()
})
}
/// The identifier `text` starts with.
fn identifier(text: &str) -> String {
text.chars()
.take_while(|c| c.is_ascii_alphanumeric() || *c == '_')
.collect()
}
/// The locals a rendering declares: the lines of its preamble that are `type name;`.
fn locals_of(c: &str) -> BTreeSet<String> {
c.lines()
.map(str::trim)
.take_while(|line| !line.starts_with("/*"))
.filter_map(|line| line.strip_suffix(';'))
.filter(|line| !line.contains(['(', '=']))
.map(|line| {
line.rsplit([' ', '*'])
.next()
.unwrap_or_default()
.to_owned()
})
.collect()
}
/// The locals read in `args`, the text after a call's opening parenthesis, that `assigned` lacks.
fn unassigned_in(
args: &str,
locals: &BTreeSet<String>,
assigned: &BTreeSet<String>,
) -> Vec<String> {
let mut unassigned = Vec::new();
let mut rest = args;
while !rest.is_empty() {
let name = identifier(rest);
let skip = name.len().max(1).min(rest.len());
if locals.contains(&name) && !assigned.contains(&name) {
unassigned.push(name);
}
rest = &rest[skip..];
}
unassigned
}
/// What a path to a statement has assigned, or `None` where it returned: a path meet keeps what both assign.
type Assigned = Option<BTreeSet<String>>;
fn meet(a: Assigned, b: Assigned) -> Assigned {
match (a, b) {
(Some(a), Some(b)) => Some(a.intersection(&b).cloned().collect()),
(a, None) => a,
(None, b) => b,
}
}
/// An `if` being read: what was assigned before it, and at the end of its then arm once that is read.
enum Frame {
Block,
If {
before: Assigned,
then_end: Option<Assigned>,
},
}
/// The locals the first call to `callee` reads that some path to it leaves unassigned, over structured C:
/// `if`/`else`, bare blocks, `return` and assignments (a `goto` or a label is not read, and fails the test).
fn unassigned_at_call(c: &str, callee: &str) -> Vec<String> {
let locals = locals_of(c);
let call = format!("{callee}(");
let mut assigned = Some(BTreeSet::<String>::new());
let mut frames = Vec::<Frame>::new();
let body = c
.lines()
.skip_while(|line| !line.contains("/* r2dec proof"));
for line in body.map(str::trim) {
assert!(
!line.starts_with("goto ") && !line.ends_with(": ;"),
"structured C only: {c}"
);
if line == "{" {
frames.push(Frame::Block);
} else if line.starts_with("if (") && line.ends_with('{') {
let before = assigned.clone();
frames.push(Frame::If {
before,
then_end: None,
});
} else if line == "} else {" {
let Some(Frame::If { before, then_end }) = frames.last_mut() else {
panic!("an else without an if: {c}");
};
*then_end = Some(assigned.take());
assigned.clone_from(before);
} else if line == "}" {
assigned = match frames.pop() {
Some(Frame::If {
then_end: Some(then_end),
..
}) => meet(then_end, assigned.take()),
Some(Frame::If { before, .. }) => meet(assigned.take(), before),
_ => assigned.take(),
};
} else if line.starts_with("return") {
assigned = None;
} else if let Some(at) = line.find(&call) {
let Some(assigned) = &assigned else { continue };
return unassigned_in(&line[at + call.len()..], &locals, assigned);
} else if let Some((target, _)) = line.split_once(" = ")
&& let Some(set) = &mut assigned
{
set.insert(identifier(target));
}
}
panic!("no call to {callee}: {c}")
}
#[test]
fn a_call_argument_is_never_a_lane_that_only_the_other_arm_assigns_under_every_write_order() {
// The lane is read where the call is not dominated by it: the argument is the carrier, or a
// residual, never the lane (ROADMAP D68).
for (layout, code) in [
("lane_nondom9", &LANE_READ_BESIDE_THE_CALL),
("lane_nondom7", &LANE_READ_BESIDE_THE_TAKEN_CALL),
("lane_nondom8", &LANE_READ_BESIDE_THE_FALLTHROUGH_CALL),
] {
for order in ORDERS {
let c = atan2f_caller_under(order, code);
let arguments = common::call_arguments(&c, "atan2f")
.unwrap_or_else(|| panic!("{layout} {order:?}: no call\n{c}"));
assert_eq!(arguments.len(), 2, "{layout} {order:?}\n{c}");
assert_eq!(
unassigned_at_call(&c, "atan2f"),
Vec::<String>::new(),
"{layout} {order:?}\n{c}"
);
assert!(
arguments[1].contains("r2sleigh_residual_") || arguments[1].contains("xmm1_"),
"{layout} {order:?}: {arguments:?}\n{c}"
);
}
}
}
#[test]
fn a_lane_the_facts_name_on_the_other_arm_is_read_as_a_residual_not_as_a_variable() {
// The knob states the lane as r2ssa did before D68; the renderer's own rule still makes the read a
// residual, which the proof line names (ROADMAP D68).
for (layout, code, named) in [
("lane_nondom9", &LANE_READ_BESIDE_THE_CALL, true),
("lane_nondom7", &LANE_READ_BESIDE_THE_TAKEN_CALL, true),
// Here the lane's block is spelled after the call's, so the write is not yet spelled.
(
"lane_nondom8",
&LANE_READ_BESIDE_THE_FALLTHROUGH_CALL,
false,
),
] {
for order in ORDERS {
let facts = r2ssa::fact_mutants::Facts::UnsoundLanesUnchecked;
let c = r2ssa::fact_mutants::with_facts(facts, || atan2f_caller_under(order, code));
let arguments = common::call_arguments(&c, "atan2f")
.unwrap_or_else(|| panic!("{layout} {order:?}: no call\n{c}"));
assert!(
arguments[1].contains("r2sleigh_residual_"),
"{layout} {order:?}: {arguments:?}\n{c}"
);
assert_eq!(
unassigned_at_call(&c, "atan2f"),
Vec::<String>::new(),
"{c}"
);
assert_eq!(
c.contains("assigned on only some paths to its read"),
named,
"{layout} {order:?}\n{c}"
);
}
}
}
#[test]
fn a_lane_the_facts_name_on_the_other_arm_refuses_the_function_while_r2ssa_checks_its_facts() {
// With only the pick unsound, r2ssa refuses what it states itself (ROADMAP D68): no C is written.
let facts = r2ssa::fact_mutants::Facts::UnsoundLanes;
for code in [
&LANE_READ_BESIDE_THE_CALL,
&LANE_READ_BESIDE_THE_TAKEN_CALL,
&LANE_READ_BESIDE_THE_FALLTHROUGH_CALL,
] {
let c = r2ssa::fact_mutants::with_facts(facts, || {
atan2f_caller_under(WriteOrder::Natural, code)
});
assert!(
c.contains("refused f: SSA integrity: a boundary fact names a value"),
"{c}"
);
assert!(!c.contains("atan2f("), "{c}");
}
}
crates/r2s/Cargo.toml
+1
View File
@@ -31,6 +31,7 @@ sleigh = ["r2engine/x86", "r2engine/arm", "r2engine/riscv"]
# Count allocations and live bytes in the engine's counters; with R2S_ALLOCATIONS set, print the
# count and the heap peak after a script (the timing gate).
alloc-count = []
placement_mutants = ["r2engine/placement_mutants"]
# Profile the heap: dhat's allocator; with R2S_HEAP_PROFILE set, a script writes dhat-heap.json (`census.py heap`).
dhat-heap = ["dep:dhat"]
crates/r2ssa/Cargo.toml
+2
View File
@@ -14,6 +14,8 @@ petgraph = "0.8"
[features]
default = []
# Test only: state the boundary facts unsoundly, so a reader's guard is shown to hold alone (ROADMAP D68).
fact_mutants = []
sleigh-config = ["dep:sleigh-config", "r2sleigh-lift/x86"]
[dev-dependencies]
crates/r2ssa/src/domtree.rs
+29
View File
@@ -222,6 +222,12 @@ impl DomTree {
self.pre[a] <= self.pre[b] && self.pre[b] <= self.last[a]
}
/// Whether the edge `from -> to` returns to a block that dominates its source: a loop's
/// latch edge or a self loop. Any other edge is a forward edge, in `O(1)`.
pub fn is_back_edge(&self, from: u64, to: u64) -> bool {
self.dominates(to, from)
}
/// Check if block A strictly dominates block B (A dominates B and A != B).
pub fn strictly_dominates(&self, a: u64, b: u64) -> bool {
a != b && self.dominates(a, b)
@@ -678,6 +684,29 @@ mod properties {
}
}
#[test]
fn an_edge_back_to_a_dominator_is_a_back_edge_and_no_other_is() {
// 0 enters a loop 1 -> 2 -> 1 and a self loop on 3; 3 is also reached by the side block 4.
let edges = BTreeSet::from([(0, 1), (1, 2), (2, 1), (2, 3), (3, 3), (0, 4), (4, 3)]);
let dom = tree(5, &edges);
let back = |from: usize, to: usize| dom.is_back_edge(addr(from), addr(to));
assert!(back(2, 1), "the latch");
assert!(back(3, 3), "the self loop");
assert!(
!back(0, 1) && !back(1, 2),
"the loop is entered and walked forward"
);
assert!(
!back(2, 3),
"3 has a second way in, so it dominates neither of its sources"
);
assert!(
!back(4, 3),
"the side edge into the self loop's block is a forward edge"
);
assert!(!back(0, 4));
}
fn tree(n: usize, edges: &BTreeSet<(usize, usize)>) -> DomTree {
let succs = |b: usize| {
edges
crates/r2ssa/src/fact_mutants.rs
+58
View File
@@ -0,0 +1,58 @@
//! Test only (feature `fact_mutants`): state a lane as before ROADMAP D68, unasked, with the check of the
//! sealed facts kept or skipped, so each defence is shown to hold alone.
/// How this thread's boundary facts are stated.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Facts {
/// As in production: a lane only where it has run, and the facts checked.
Sound,
/// A lane taken without asking, the facts still checked: r2ssa refuses the function.
UnsoundLanes,
/// A lane taken without asking and the facts unchecked: only the renderer's guard is left.
UnsoundLanesUnchecked,
}
#[cfg(feature = "fact_mutants")]
mod on {
use super::Facts;
use std::cell::Cell;
thread_local! {
static STATED: Cell<Facts> = const { Cell::new(Facts::Sound) };
}
/// Run `body` with this thread's facts stated as `facts`, then restore what was set.
pub fn with_facts<T>(facts: Facts, body: impl FnOnce() -> T) -> T {
struct Restore(Facts);
impl Drop for Restore {
fn drop(&mut self) {
STATED.with(|slot| slot.set(self.0));
}
}
let _restore = Restore(STATED.with(|slot| slot.replace(facts)));
body()
}
pub(crate) fn unsound_lanes() -> bool {
STATED.with(Cell::get) != Facts::Sound
}
pub(crate) fn facts_unchecked() -> bool {
STATED.with(Cell::get) == Facts::UnsoundLanesUnchecked
}
}
#[cfg(feature = "fact_mutants")]
pub use on::with_facts;
#[cfg(feature = "fact_mutants")]
pub(crate) use on::{facts_unchecked, unsound_lanes};
#[cfg(not(feature = "fact_mutants"))]
pub(crate) const fn unsound_lanes() -> bool {
false
}
#[cfg(not(feature = "fact_mutants"))]
pub(crate) const fn facts_unchecked() -> bool {
false
}
crates/r2ssa/src/function/mod.rs
+12
View File
@@ -1188,6 +1188,18 @@ impl SsaArtifact {
self.sealed.graph()
}
/// Whether `value` is defined where operation `ordinal` of the block at `block_addr` reads (the
/// rule the boundary facts are checked by); `usize::MAX` asks at the block's end.
pub fn value_defined_before(&self, value: ValueId, block_addr: u64, ordinal: usize) -> bool {
crate::integrity::defined_before(
self.sealed.function(),
self.sealed.graph(),
value,
block_addr,
ordinal,
)
}
pub fn facts(&self) -> &PreparedFunctionFacts {
&self.facts
}
crates/r2ssa/src/function/stage.rs
+6
View File
@@ -357,6 +357,12 @@ impl Sealed {
},
control,
)?;
// A fact that names a value its read does not stand behind is r2ssa's defect, refused with
// the function rather than handed to a reader (ROADMAP D68).
if !crate::fact_mutants::facts_unchecked() {
crate::integrity::validate_boundary_values(&self.ir, &self.graph, &facts)
.map_err(super::integrity_refusal)?;
}
// What one prepared function holds is the space every later stage has
// to work above, so it is reported beside the phases that built it.
r2il::refusal_evidence!(
crates/r2ssa/src/function/tests/lanes.rs
+472
View File
@@ -0,0 +1,472 @@
//! A call argument is the low lane of its carrier only where the lane's definition stands behind
//! the call (ROADMAP D68, doc/adr-decompiler-rewrite.md "D2's spelling order").
use super::*;
use crate::{
CallSiteId, InstId, InstPayload, SourceCallArgumentSpec, SourceCallResult,
SourceCallSiteIdentity, SourceCallSiteInterface, SourceFunctionInterface,
};
use r2il::ArchSpec;
const VECTOR: u64 = 0x100;
const MERGE: u64 = 0x100c;
const READER: u64 = 0x1010;
const CALLER: u64 = 0x1018;
const CALLEE: u64 = 0x4000;
fn storage(offset: u64, size: u32) -> CanonicalStorageId {
CanonicalStorageId {
space: CanonicalStorageSpace::Register,
offset,
size,
}
}
/// A block of `ops`, each stamped as its own instruction.
fn block(addr: u64, ops: Vec<R2ILOp>) -> R2ILBlock {
let mut block = R2ILBlock::new(addr, 4);
for op in ops {
block.push(op);
}
for index in 0..block.ops.len() {
let returns = matches!(block.ops[index], R2ILOp::Return { .. });
block.stamp_instruction(index, addr + if returns { 2 } else { 0 });
}
block
}
fn lane_read() -> Vec<R2ILOp> {
vec![
R2ILOp::Subpiece {
dst: make_unique(0x200, 4),
src: make_reg(VECTOR, 16),
offset: 0,
},
R2ILOp::Store {
space: SpaceId::Ram,
addr: make_const(0x3000, 8),
val: make_unique(0x200, 4),
},
]
}
/// Two arms load a vector register that merges at `MERGE`, which branches to `READER` (reads its low
/// lane) or `CALLER` (calls with it). `lane_in_the_merge` moves that read above the branch.
fn lane_call(lane_in_the_merge: bool) -> SsaArtifact {
sealed_lane_call(lane_in_the_merge).expect("the lane call prepares")
}
/// The vector register `v`, its four lanes and the registers the call and returns use, with the
/// projections that bind each lane to `v`.
fn lane_arch() -> ArchSpec {
let mut arch = ArchSpec::new("lane-test");
arch.addr_size = 8;
arch.add_register(RegisterDef::new("v", VECTOR, 16));
let carrier = r2il::RegisterStorage {
offset: VECTOR,
size: 16,
};
let bound = |written, carrier, lsb_bit_offset, size_bits| r2il::RegisterProjection {
written,
disposition: r2il::RegisterProjectionDisposition::Bound {
carrier,
slice: r2il::RegisterBitSlice {
lsb_bit_offset,
size_bits,
},
},
};
let mut projections = vec![bound(carrier, carrier, 0, 128)];
for lane in 0..4u32 {
let offset = VECTOR + 4 * u64::from(lane);
arch.add_register(RegisterDef::sub(format!("v{lane}"), offset, 4, "v"));
let written = r2il::RegisterStorage { offset, size: 4 };
projections.push(bound(written, carrier, 32 * u64::from(lane), 32));
}
for (name, offset, size) in [("cond", 0x40, 1), ("lr", 0x80, 8), ("sp", 0x88, 8)] {
arch.add_register(RegisterDef::new(name, offset, size));
let own = r2il::RegisterStorage { offset, size };
projections.push(bound(own, own, 0, u64::from(size) * 8));
}
// The file lists its projections in the order of the storage they are for.
projections.sort_by_key(|projection| projection.written);
arch.register_projections = projections;
arch
}
/// As x86 writes `xmm1` from memory: the low lane loaded, the three above it zeroed.
fn load_vector(addr: u64) -> Vec<R2ILOp> {
let mut ops = vec![R2ILOp::Load {
dst: make_reg(VECTOR, 4),
space: SpaceId::Ram,
addr: make_const(addr, 8),
}];
ops.extend((1..4).map(|lane| R2ILOp::Copy {
dst: make_reg(VECTOR + 4 * lane, 4),
src: make_const(0, 4),
}));
ops
}
fn lane_blocks(lane_in_the_merge: bool) -> Vec<R2ILBlock> {
let branch = |target| R2ILOp::Branch {
target: make_const(target, 8),
};
let conditional = |target| R2ILOp::CBranch {
target: make_const(target, 8),
cond: make_reg(0x40, 1),
};
let ret = || R2ILOp::Return {
target: make_reg(0x80, 8),
};
let (mut merge, mut reader) = match lane_in_the_merge {
true => (lane_read(), Vec::new()),
false => (Vec::new(), lane_read()),
};
merge.push(conditional(CALLER));
reader.push(ret());
let call = R2ILOp::Call {
target: make_ram(CALLEE, 8),
};
vec![
block(0x1000, vec![conditional(0x1008)]),
block(0x1004, [load_vector(0x2000), vec![branch(MERGE)]].concat()),
block(0x1008, [load_vector(0x2010), vec![branch(MERGE)]].concat()),
block(MERGE, merge),
block(READER, reader),
block(CALLER, vec![call, ret()]),
]
}
fn sealed_lane_call(lane_in_the_merge: bool) -> Result<SsaArtifact, crate::SsaPrepareError> {
let revision = b"lane-call";
let interface = SourceFunctionInterface::new_exact(
revision.to_vec(),
"lane-abi",
[],
SourceFunctionReturn::Void,
[],
)
.and_then(|interface| interface.with_return_address_storage(storage(0x80, 8)))
.and_then(|interface| interface.with_stack_pointer_storage(storage(0x88, 8)))
.expect("a function interface");
let callee = CanonicalStorageId::from_varnode(&make_ram(CALLEE, 8));
let call = SourceCallSiteInterface::new(
revision.to_vec(),
SourceCallSiteIdentity::new(CALLER, callee),
true,
"lane-abi",
[SourceCallArgumentSpec::new(0, storage(VECTOR, 4))],
false,
false,
SourceCallResult::Void,
)
.expect("a call interface");
SsaArtifact::for_decompile_with_interfaces_and_control(
&lane_blocks(lane_in_the_merge),
Some(&crate::Arch::from(lane_arch())),
Some(interface),
crate::SourceMachineRoles::default(),
vec![call],
crate::testing::call_effect([], [storage(0x88, 8)]),
&crate::control::UncheckedSsaWorkControl,
)
}
/// The call's one argument, and the instruction that defines a value, as `(block, is a phi)`.
fn the_argument(artifact: &SsaArtifact) -> SourceCallArgumentFact {
let boundary =
(artifact.facts().boundaries.calls.get(&CallSiteId(0))).expect("the call's boundary");
let [argument] = boundary.arguments.as_slice() else {
panic!("one argument: {boundary:?}");
};
*argument
}
fn defined_in(artifact: &SsaArtifact, value: ValueId) -> (u64, bool) {
let graph = artifact.graph();
let inst = graph
.def_inst(value)
.and_then(|inst| graph.inst(inst))
.expect("a value the function defines");
let block = graph.block(inst.block).expect("its block").addr;
(block, matches!(inst.payload, InstPayload::Phi { .. }))
}
/// The value the low-lane read in `block` defines.
fn lane_defined_in(artifact: &SsaArtifact, block: u64) -> ValueId {
let graph = artifact.graph();
graph
.insts
.iter()
.filter(|inst| {
graph
.block(inst.block)
.is_some_and(|held| held.addr == block)
})
.find(|inst| matches!(&inst.payload, InstPayload::Op(SSAOp::Subpiece { .. })))
.and_then(|inst| inst.output)
.expect("a low lane read there")
}
#[test]
fn a_low_lane_read_on_the_other_arm_is_not_the_calls_argument() {
// The lane is a view of the merge's vector, defined in `READER`, which does not dominate `CALLER`:
// the argument is the merge itself, the carrier that does.
let artifact = lane_call(false);
let argument = the_argument(&artifact);
let SourceCallArgumentValue::Value(value) = argument.value else {
panic!("a value: {argument:?}");
};
assert_eq!(defined_in(&artifact, value), (MERGE, true));
assert_eq!(artifact.graph().var(value).size, 16);
assert_eq!(argument.lane_of, None);
assert_ne!(value, lane_defined_in(&artifact, READER));
}
#[test]
fn a_low_lane_read_before_the_branch_is_the_calls_argument() {
// The same lane read in the merge stands behind both arms: the argument narrows to it, as it did.
let artifact = lane_call(true);
let argument = the_argument(&artifact);
let lane = lane_defined_in(&artifact, MERGE);
assert_eq!(argument.value, SourceCallArgumentValue::Value(lane));
let carrier = argument.lane_of.expect("the carrier the lane is of");
assert_eq!(defined_in(&artifact, carrier), (MERGE, true));
assert_eq!(artifact.graph().var(lane).size, 4);
}
/// The fact family `validate_boundary_values` names when `forge` puts `lane` (defined on the `READER`
/// arm, behind neither the call, the merge's branch nor either return of `CALLER`) in its place.
fn refused_for(
artifact: &SsaArtifact,
fact: &str,
forge: impl FnOnce(&mut crate::PreparedFunctionFacts),
) -> &'static str {
let mut forged = artifact.facts().clone();
forge(&mut forged);
let refused =
crate::integrity::validate_boundary_values(artifact.function(), artifact.graph(), &forged)
.expect_err(&format!("a value defined on the other arm is no {fact}"));
assert_eq!(
refused.rule(),
crate::IntegrityRule::BoundaryValueNotDominated
);
let crate::SsaIntegrityError::BoundaryValueNotDominated { fact, .. } = refused else {
panic!("{refused:?}");
};
fact
}
/// A change to the facts that puts a value out of its read's scope.
type Forge = Box<dyn Fn(&mut crate::PreparedFunctionFacts)>;
/// The forged facts that put `lane` in a call's place: its argument, the carrier it is of, its target.
fn call_forgeries(lane: ValueId, carrier: ValueId) -> Vec<(&'static str, Forge)> {
vec![
(
"call argument",
Box::new(move |f| {
let call = f
.boundaries
.calls
.get_mut(&CallSiteId(0))
.expect("the call");
call.arguments[0].value = SourceCallArgumentValue::Value(lane);
}),
),
(
"call argument carrier",
Box::new(move |f| {
let call = f
.boundaries
.calls
.get_mut(&CallSiteId(0))
.expect("the call");
call.arguments[0].value = SourceCallArgumentValue::Value(carrier);
call.arguments[0].lane_of = Some(lane);
}),
),
(
"call target",
Box::new(move |f| {
f.call_sites
.by_id
.values_mut()
.for_each(|site| site.target = lane)
}),
),
]
}
/// The forged facts that put `lane` in the return at `at`: a value, the return address, the stack pointer.
fn return_forgeries(lane: ValueId, at: InstId) -> Vec<(&'static str, Forge)> {
use crate::{CallBoundaryValueFact, SourceReturnAddressFact, SourceReturnStackPointerFact};
let slot = CallBoundarySlot::Register {
index: 0,
storage: storage(0x80, 8),
};
vec![
(
"return value",
Box::new(move |f| {
let ret = f.boundaries.returns.get_mut(at).expect("the return");
ret.values.push(CallBoundaryValueFact { slot, value: lane });
}),
),
(
"return address",
Box::new(move |f| {
let ret = f.boundaries.returns.get_mut(at).expect("the return");
let storage = storage(0x80, 8);
ret.return_address = Some(SourceReturnAddressFact {
storage,
value: lane,
});
}),
),
(
"exit stack pointer",
Box::new(move |f| {
let ret = f.boundaries.returns.get_mut(at).expect("the return");
let storage = storage(0x88, 8);
let sp = SourceReturnStackPointerFact::ReachingValue {
storage,
value: lane,
};
ret.exit_stack_pointer = Some(sp);
}),
),
]
}
/// The forged facts that put `lane` in a branch's place: its condition, a switch's selector.
fn branch_forgeries(lane: ValueId) -> Vec<(&'static str, Forge)> {
use crate::semantic::SwitchPredicateFact;
vec![
(
"branch condition",
Box::new(move |f| {
f.predicates
.predicates
.values_mut()
.for_each(|p| p.condition = lane)
}),
),
(
"switch selector",
Box::new(move |f| {
let switch = SwitchPredicateFact {
block_addr: MERGE,
selector: Some(lane),
cases: Vec::new(),
default: None,
};
f.predicates.switches.insert(MERGE, switch);
}),
),
]
}
#[test]
fn a_boundary_fact_naming_a_value_its_read_does_not_stand_behind_is_refused() {
let artifact = lane_call(false);
crate::integrity::validate_boundary_values(
artifact.function(),
artifact.graph(),
artifact.facts(),
)
.expect("the facts the collector states hold");
let lane = lane_defined_in(&artifact, READER);
let SourceCallArgumentValue::Value(carrier) = the_argument(&artifact).value else {
panic!("the carrier is a value");
};
// The return in `CALLER`, which the `READER` arm does not lead to.
let graph = artifact.graph();
let at = (artifact.facts().boundaries.returns.iter())
.map(|(at, _)| at)
.find(|at| {
graph
.walk_start(*at)
.is_some_and(|(block, _)| block == CALLER)
})
.expect("a return in the calling block");
let cases = [
call_forgeries(lane, carrier),
return_forgeries(lane, at),
branch_forgeries(lane),
]
.into_iter()
.flatten()
.collect::<Vec<_>>();
assert_eq!(cases.len(), 8);
for (fact, forge) in cases {
assert_eq!(refused_for(&artifact, fact, forge), fact);
}
}
#[test]
fn a_value_is_defined_before_the_operations_after_its_definition_and_not_those_up_to_it() {
let artifact = lane_call(true);
let (function, graph) = (artifact.function(), artifact.graph());
let before = |value, block, ordinal| {
crate::integrity::defined_before(function, graph, value, block, ordinal)
};
// An operation's output is defined for the operations after it, not for itself or the ones before.
let lane = lane_defined_in(&artifact, MERGE);
let (block, ordinal) = graph
.def_inst(lane)
.and_then(|inst| graph.walk_start(inst))
.expect("the lane is an operation's output");
assert!(!before(lane, block, ordinal));
assert!(before(lane, block, ordinal + 1));
assert!(before(lane, block, usize::MAX));
// A phi is defined at its block's entry, for every operation in it.
let carrier = (the_argument(&artifact).lane_of).expect("the carrier the lane is of");
assert_eq!(defined_in(&artifact, carrier), (MERGE, true));
assert!(before(carrier, MERGE, 0));
// The merge dominates both arms below it.
assert!(before(carrier, CALLER, 0));
assert!(before(lane, READER, 0));
assert!(before(lane, CALLER, 0));
// A value the function was entered with, and a constant, are defined before everything.
let entry = artifact
.graph()
.values
.iter()
.find(|v| artifact.graph().def_inst(v.id).is_none());
assert!(entry.is_some_and(|value| before(value.id, CALLER, 0)));
// A definition on one arm reaches neither the other arm nor the merge above it.
let other_arm = lane_call(false);
let lane = lane_defined_in(&other_arm, READER);
let (function, graph) = (other_arm.function(), other_arm.graph());
let on_the_arm =
|block, ordinal| crate::integrity::defined_before(function, graph, lane, block, ordinal);
assert!(!on_the_arm(CALLER, 0) && !on_the_arm(MERGE, usize::MAX));
assert!(on_the_arm(READER, usize::MAX));
}
#[cfg(feature = "fact_mutants")]
#[test]
fn an_unsound_lane_is_refused_by_the_check_of_the_sealed_facts() {
use crate::fact_mutants::{Facts, with_facts};
// The lane on the other arm is what the pick would state; sealing the function refuses it.
let refused = with_facts(Facts::UnsoundLanes, || sealed_lane_call(false));
assert!(matches!(
refused,
Err(crate::SsaPrepareError::Integrity(
crate::IntegrityRule::BoundaryValueNotDominated
))
));
// Unchecked, that fact is stated: the lane defined on the `READER` arm, of the merge's vector.
let unchecked = with_facts(Facts::UnsoundLanesUnchecked, || sealed_lane_call(false))
.expect("sealed without the check");
let argument = the_argument(&unchecked);
let SourceCallArgumentValue::Value(lane) = argument.value else {
panic!("a value: {argument:?}");
};
assert_eq!(defined_in(&unchecked, lane), (READER, false));
assert!(argument.lane_of.is_some());
// A lane read above the branch stands behind the call: the same pick, nothing to refuse.
assert!(with_facts(Facts::UnsoundLanes, || sealed_lane_call(true)).is_ok());
}
crates/r2ssa/src/function/tests/mod.rs
+1
View File
@@ -1,6 +1,7 @@
mod caller_slots;
mod control;
mod interface;
mod lanes;
mod lift;
mod memory;
mod stack;
crates/r2ssa/src/integrity.rs
+125
View File
@@ -9,6 +9,8 @@ use std::collections::{HashMap, HashSet};
use std::fmt;
use crate::function::{DefSite, SSAFunction, SourceSite};
use crate::graph::{InstId, SsaGraph, ValueId};
use crate::semantic::{PreparedFunctionFacts, SourceCallArgumentValue};
use crate::{CanonicalStorageId, SSAOp, SSAVar, VarId};
/// The scalar-width rule violated by one regular SSA operation.
@@ -44,6 +46,7 @@ pub enum IntegrityRule {
ScalarWidthMismatch,
ZeroWidthValue,
UseNotDominated,
BoundaryValueNotDominated,
}
impl fmt::Display for IntegrityRule {
@@ -72,6 +75,9 @@ impl fmt::Display for IntegrityRule {
Self::ScalarWidthMismatch => "an operation violates a scalar-width rule",
Self::ZeroWidthValue => "a value has zero width",
Self::UseNotDominated => "a read is not dominated by its definition",
Self::BoundaryValueNotDominated => {
"a boundary fact names a value its definition does not dominate"
}
})
}
}
@@ -99,6 +105,7 @@ impl SsaIntegrityError {
Self::ScalarWidthMismatch { .. } => IntegrityRule::ScalarWidthMismatch,
Self::ZeroWidthValue { .. } => IntegrityRule::ZeroWidthValue,
Self::UseNotDominated { .. } => IntegrityRule::UseNotDominated,
Self::BoundaryValueNotDominated { .. } => IntegrityRule::BoundaryValueNotDominated,
}
}
}
@@ -201,6 +208,22 @@ pub enum SsaIntegrityError {
def_block_addr: u64,
def_site: DefSite,
},
/// A fact of the sealed function (a call's argument, a return's value, a branch's condition)
/// names a value whose definition does not dominate the place the fact reads it.
BoundaryValueNotDominated {
fact: &'static str,
site: BoundarySite,
var: SSAVar,
},
}
/// Where a boundary fact reads a value.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum BoundarySite {
/// Before this operation.
Inst(InstId),
/// At the end of the block at this address, where its branch is.
BlockEnd(u64),
}
/// Exact site at which a zero-width SSA value was observed.
@@ -351,6 +374,10 @@ impl fmt::Display for SsaIntegrityError {
"SSA value {var} read at {site:?} in block 0x{block_addr:x} is defined at \
{def_site:?} in block 0x{def_block_addr:x}, which does not dominate the read"
),
Self::BoundaryValueNotDominated { fact, site, var } => write!(
f,
"the {fact} fact names {var} at {site:?}, which its definition does not dominate"
),
}
}
}
@@ -684,6 +711,104 @@ pub fn validate_ssa_function(function: &SSAFunction) -> Result<(), SsaIntegrityE
Ok(())
}
/// Whether `value` is defined where operation `ordinal` of the block at `block_addr` reads: an entry
/// value, a phi or earlier operation of it, or a strictly dominating block's. `O(1)`.
pub(crate) fn defined_before(
function: &SSAFunction,
graph: &SsaGraph,
value: ValueId,
block_addr: u64,
ordinal: usize,
) -> bool {
let Some(def) = graph.def_inst(value) else {
return true;
};
let Some(def_block) = graph
.inst(def)
.and_then(|inst| graph.block(inst.block))
.map(|block| block.addr)
else {
return false;
};
if def_block == block_addr {
return graph
.op_ordinal(def)
.is_none_or(|defined| defined < ordinal);
}
function.domtree().strictly_dominates(def_block, block_addr)
}
/// Every value a boundary fact reads (call arguments, targets, return values, branch conditions,
/// switch selectors) is defined where it reads it, `O(facts)`; what is not a read is not checked.
#[expect(
clippy::result_large_err,
reason = "the diagnostic keeps the offending value; validation runs once, when the artifact is sealed"
)]
pub(crate) fn validate_boundary_values(
function: &SSAFunction,
graph: &SsaGraph,
facts: &PreparedFunctionFacts,
) -> Result<(), SsaIntegrityError> {
let read = |fact: &'static str, site: BoundarySite, value: ValueId| {
let place = match site {
BoundarySite::Inst(inst) => graph.walk_start(inst),
BoundarySite::BlockEnd(addr) => Some((addr, usize::MAX)),
};
match place {
Some((block_addr, ordinal))
if defined_before(function, graph, value, block_addr, ordinal) =>
{
Ok(())
}
_ => Err(SsaIntegrityError::BoundaryValueNotDominated {
fact,
site,
var: graph.var(value).clone(),
}),
}
};
for call in facts.boundaries.calls.values() {
let site = BoundarySite::Inst(call.at);
for argument in &call.arguments {
if let SourceCallArgumentValue::Value(value) = argument.value {
read("call argument", site, value)?;
}
if let Some(carrier) = argument.lane_of {
read("call argument carrier", site, carrier)?;
}
}
}
for call in facts.call_sites.by_id.values() {
read("call target", BoundarySite::Inst(call.at), call.target)?;
}
for ret in facts.boundaries.returns.values() {
let site = BoundarySite::Inst(ret.at);
for value in &ret.values {
read("return value", site, value.value)?;
}
if let Some(address) = ret.return_address {
read("return address", site, address.value)?;
}
if let Some(value) = ret.exit_stack_pointer.and_then(|sp| sp.value()) {
read("exit stack pointer", site, value)?;
}
}
for predicate in facts.predicates.predicates.values() {
let site = BoundarySite::BlockEnd(predicate.block_addr);
read("branch condition", site, predicate.condition)?;
}
for switch in facts.predicates.switches.values() {
if let Some(selector) = switch.selector {
read(
"switch selector",
BoundarySite::BlockEnd(switch.block_addr),
selector,
)?;
}
}
Ok(())
}
fn scalar_width_violation<V>(op: &SSAOp<V>, size: impl Fn(&V) -> u32) -> Option<ScalarWidthRule> {
match op {
SSAOp::Copy { dst, src } if size(dst) != size(src) => {
crates/r2ssa/src/lib.rs
+3 -1
View File
@@ -38,6 +38,7 @@ pub(crate) mod demand;
pub mod dense;
pub(crate) mod division;
pub mod domtree;
pub mod fact_mutants;
pub mod fixpoint;
pub(crate) mod frame_address;
pub mod function;
@@ -119,7 +120,8 @@ pub use graph::{
BlockId, GraphBlock, GraphInst, GraphValue, InstId, InstPayload, SsaGraph, UseSite, ValueId,
};
pub use integrity::{
IntegrityRule, ScalarWidthRule, SsaIntegrityError, SsaValueSite, validate_ssa_function,
BoundarySite, IntegrityRule, ScalarWidthRule, SsaIntegrityError, SsaValueSite,
validate_ssa_function,
};
pub use interproc::{
ArgumentReach, CallArgObservation, FunctionSemanticLinkage, FunctionSemanticSummary,
crates/r2ssa/src/semantic/boundaries.rs
+13 -4
View File
@@ -1056,7 +1056,7 @@ pub(crate) fn convention_call_boundary(
break;
}
};
let (value, lane_of) = super::shared::narrowed_argument(prep, graph, value, *slot);
let (value, lane_of) = super::shared::narrowed_argument(&at, value, *slot);
arguments.push(SourceCallArgumentFact {
slot: CallBoundarySlot::Register {
index,
@@ -1198,7 +1198,7 @@ fn stated_call_arguments(
}
None => return (arguments, false),
};
let (value, lane_of) = super::shared::narrowed_argument(at.prep, at.graph, value, *slot);
let (value, lane_of) = super::shared::narrowed_argument(&at, value, *slot);
let Ok(index) = u32::try_from(position) else {
return (arguments, false);
};
@@ -1375,8 +1375,17 @@ pub(crate) fn collect_source_boundary_facts(
);
}
found.map(|value| {
let (value, lane_of) =
super::shared::narrowed_argument(prep, graph, value, storage);
let (value, lane_of) = super::shared::narrowed_argument(
&CallPosition {
function,
prep,
graph,
block_addr,
op_index,
},
value,
storage,
);
SourceCallArgumentFact {
slot: CallBoundarySlot::Register {
index: argument.index(),
crates/r2ssa/src/semantic/shared.rs
+17 -5
View File
@@ -1005,18 +1005,30 @@ pub(crate) fn proven_low_lane(
.filter(|lane| *lane != value)
}
/// A register argument narrowed to the value its slot's bits are, with the wider value it was read
/// from: the boundary fact's `value` and `lane_of`.
/// A register argument narrowed to the lane its slot's bits are (`value`, with the carrier as `lane_of`),
/// where the lane's definition has run at the call; else the carrier (ROADMAP D68).
pub(crate) fn narrowed_argument(
prep: Option<&crate::DecompilePrepFacts>,
graph: &SsaGraph,
at: &super::boundaries::CallPosition<'_>,
value: SourceCallArgumentValue,
storage: CanonicalStorageId,
) -> (SourceCallArgumentValue, Option<ValueId>) {
let SourceCallArgumentValue::Value(carrier) = value else {
return (value, None);
};
match prep.and_then(|prep| proven_low_lane(&prep.views, graph, carrier, storage)) {
let lane = at
.prep
.and_then(|prep| proven_low_lane(&prep.views, at.graph, carrier, storage))
.filter(|lane| {
crate::fact_mutants::unsound_lanes()
|| crate::integrity::defined_before(
at.function,
at.graph,
*lane,
at.block_addr,
at.op_index,
)
});
match lane {
Some(lane) => (SourceCallArgumentValue::Value(lane), Some(carrier)),
None => (value, None),
}
doc/adr-decompiler-rewrite.md
+233
View File
@@ -101,6 +101,15 @@ The obligation inventory says what must be rendered, so no stage re-derives it.
seal points a merge source that is a copy, through any chain, of the merge's own
output at the merge itself, and r2dec compares ids, never spellings. Coalescing a phi web into one variable is the
partition, r2ssa's to state from `ValueLiveness` and `StorageSpans` (D2.1).
A merge reads as a variable only when every distinct placed *forward* edge into its block renders its
copy (`Values::owed`: one per forward edge spelled before the merge, set by `forward_in` from the CFG's
predecessors and `DomTree::is_back_edge`, paid once per edge by `account_copy`). A back edge, a self copy
and a gap pay nothing, so a latch copy never justifies a read at the header and one gapped forward copy
keeps every read residual. A gap traps where it stands
(`a_gap_is_written_as_the_residual_call_whose_helper_traps`), which would make "any forward copy renders"
sound at run time, but the certificate lets control go on past a gap
(`GapKind::ValuesNotRendered.ends_control()` is false) and a verdict does not lean on what the
certificate denies; the weaker rule is ledgered in "D2's spelling order".
- **Frame.** A stack object is a C byte array only where r2ssa proves the
storage is the callee's (`callee_allocation`, or a source slot that is a
local or a parameter home); a slot at or above the entry stack pointer
@@ -119,6 +128,183 @@ The obligation inventory says what must be rendered, so no stage re-derives it.
Cost: one canonicalisation, `O(terms × rules applied)`, then one pass over
the operations and their uses.
## D2's spelling order
Every verdict D2 gives (a statement rendered or a gap, a name read as a variable or as a NeverAssigned
residual, an edge copy, an extern, a local) is a function of the sealed facts and the reverse postorder
of the placed blocks. It is never a function of the order `control::Writer` composes blocks, which
the placement chooses to put an exit inside a loop or after it and a merge in this region or that.
- **One pass, then the layout.** `render::spelled::spell` walks `function.block_addrs()` (r2ssa's block
order, which is the CFG's reverse postorder, filtered by `place::is_placed`) and spells each block's
statements, then its edges (the copies of a target's merges, `transferred`, or a tail call and the
gap that stands for it), then the operand the block ends on, in the order `Writer::arms` used to.
No verdict depends on that order: a block's test reads no merge its own forward edges pay, since a
merge the block's test reads dominates the block and so enters it only by a back edge, which pays
nothing. The result is a typed table (`Spelled`, by
block address) that the Writer takes once per block. The Writer holds no `Values`: it decides
regions, loops, labels and switch case grouping, and `crates/r2dec/tests/architecture.rs` keeps
`control.rs` free of `Values` and `spelled.rs` free of the placement.
- **Why a pass and not a prediction.** Whether a block's statements render is not knowable from the
facts alone: a term with no spelling, a type with no C type, a callee the symbol table shadows and an
identifier a global claimed first each decide it, and a predictor of those would be a second copy of
the spelling rules. So the order of spelling is made a fact of the CFG, and cost stays `O(B + E +
statements + phi inputs)`, one sweep, no fixpoint.
- **Forward edges are exact.** The first arrival at a merge is never over a back edge, since a back
edge's source is dominated by the merge, and every read the merge dominates follows an arrival. A
forward predecessor comes before the merge in reverse postorder where the region is reducible. In an
irreducible region an edge can be forward by dominance and still retreat in reverse postorder (a
*late* edge): its copy is spelled after the first reads of the merge it enters. The merge is not owed
that copy; it is verified. The first attempt reads the merge as a variable on the promise that the
copy renders, and `Values::broken` names every merge whose late copy gapped; `render` then spells the
function again with each of those owing one definition more than any edge pays, and stops when an
attempt breaks no promise. Each attempt adds a merge to the set, so there are at most as many attempts
as merges; over bash, ls-toybox and pigz (2,508 functions) 43 hold a late edge (280 edges) and one is
spelled again, over 0pack and pumasim (23,201 answered) 123 hold one (1,084 edges) and none is. The rule stays "every forward edge renders its copy", checked by the attempt that
stands, and the attempts are a function of the facts and the order. Owing the late copy at the early
reads instead made 36 functions read 424 more sites as residuals that the copy would have defined.
- **The gate.** `placement_mutants` (a cargo feature, off by default and absent from release builds)
permutes the placement's write order: O1 keeps every loop exit after its loop, O2 reverses each
loop's exits, O3 reverses each region's merges. `census.py orders` runs `pddj` of every function of
bash, ls-toybox and pigz under each and compares what no write order may change (the ledger's
columns, the never-assigned names, the residual sites by instruction address, cause and gap kind, the
declared names and links, and the set of statement lines apart from control). The rendered text
itself cannot be equal: O1 turns an inlined exit into a `goto`, and O2 and O3 move blocks, so the
labels and the site numbers move. The structure stages (chains, tails, latch, rotate) read text
order, copy text and never call `Values`; their shapes differ by layout by design and are not
gated, and nor is the selection pass (`if (c) x = a; else x = b;` against `x = c ? a : b;`), which
the fingerprint counts apart.
Measured (2026-10-23) over bash, ls-toybox and pigz, 2,508 functions, one mutant build per step:
| Step | O1 differ | O2 differ | O3 differ | Functions whose text moves (O1, O2, O3) |
|---|---|---|---|---|
| the knobs alone (write-order dependent) | 0 | 120 | 187 | 64, 338, 722 |
| spelled in reverse postorder | 0 | 0 | 0 | 64, 332, 720 |
| and the forward-edge rule | 0 | 0 | 0 | 64, 332, 720 |
One function under O2 and seven under O3 differ only in a selection. Against the base, the natural
order moves 13 of the 2,508 functions (+9 residual sites): the lazily declared locals of seven bash
functions' block-transfer loops are numbered in reverse postorder (`reached_5` for `reached`), bash
`0x8ee40` lists its `_next` temporaries in another order, and five hold a gapped forward copy (bash
`0x2dd90`, `0x48140`, `0x85d70`, ls-toybox `0x392dc`, `0x501c8`). 0pack `0x476982` has 9 residual
sites under all four orders: the base's 7 under O1 read `rsp_8` after a loop whose entry copy at
`0x4769b1` is a gap, so nothing assigns it on a zero-trip path, and that read is what the rule refuses.
Ledgered: ROADMAP D62 (a callee's result presence is one fact per callee, not each call site's to
spell), D63 (reading a merge as a variable when any forward copy renders, which waits on the
certificate agreeing that a gap traps) and D64 (`Values` immutable once spelled).
### A name is read where its definition has run (D68)
D58 reads a bound name as a variable when every write to it is spelled (`owed` is 0) and spells the
blocks in reverse postorder. Reverse postorder is not dominance: a block spelled first can lie on the
other arm of a branch from the block that reads. At `15486ea7` pumasim `0x279d80` and `0x27b040`
read `tmp_lane_279e4c_0_0_1` and `tmp_lane_27b110_0_0_1` at an `atan2f` the lane's block does not
dominate (4 reads, assigned only on the other arm), and the proof line said `0 residual`. The
parents print the same C on a mirrored layout (`lane_nondom8`) and a residual at the tip, so D58
moved which functions show the error; it was r2ssa's fact that was wrong.
```text
lane_nondom9(flag rdi, p rsi, q rdx): xmm1 = flag ? p[0] : p[1]; ucomiss xmm4, xmm2 (q[4], q[5])
jbe call; comiss xmm1, xmm2 (reads xmm1's low lane); jbe out; ret
call: movaps xmm0, xmm4; call atan2f; ret out: ret
```
- **The fact.** `narrowed_argument` took `ValueViews::low_lane_value`, the first value of the lane's
class in definition order, as the call argument's `value` (the carrier as `lane_of`). The class
holds values of the same bits wherever defined: the carrier XMM1_6 is defined in `0x279e1c`, the
lane (a `SUBPIECE` the `comiss` reads) in `0x279e4c`, the call is in `0x279ec8`, and `0x279e3e`
branches to either. The first of a class need not dominate a read.
- **r2ssa owns it.** `integrity::defined_before(function, graph, value, block, ordinal)` is the one
rule: an entry value, a phi or an earlier operation of the block, or a definition in a strictly
dominating block, `O(1)` from the dominator tree's preorder intervals. `narrowed_argument` keeps a
lane only where it holds at the call and otherwise states the carrier the walk reached with no
`lane_of`, which renders `(uint32_t)xmm1_6`, a truncation of a value that dominates (the state
before lane narrowing, for those calls). Where a later member of the class would have dominated,
the carrier is used instead (left, below). `validate_boundary_values` checks every value a sealed
fact reads against the place it reads it, `O(facts)`: call arguments and their carriers, call
targets, return values, the return address and the exit stack pointer, branch conditions and
switch selectors. A violation refuses the function (`IntegrityRule::BoundaryValueNotDominated`,
`SSA integrity: a boundary fact names a value its definition does not dominate`), as
`validate_ssa_function` does for an operation. It is beside that function and not in it:
`validate_ssa_function` runs on the `SSAFunction` before the seal, and the boundary facts exist
only after the seal has dropped the operations it reads. Call results are definitions the call
makes, not reads, and a comparison's operands (`PredicateFact::comparison`) name the copy classes
the condition compares, the same first-of-class pick; no renderer reads them, and checking them
refused 44 of bash's 1,568 functions in the first version of this check, so they are not checked.
- **r2dec holds the line, as a hard rule.** `Values::read_name` asks r2ssa
(`SsaArtifact::value_defined_before`, the same rule) whether the name's definition has run in the
block being spelled (`Values::enter`, once per block in `Spelling::block`); if not, the read is a
residual with refusal evidence (`read-not-dominated`) and a proof-line cause of its own ("assigned
on only some paths to its read"). It is not a `debug_assert`: it is the one check on the text
itself, where the validator checks facts at their reads and the terms the arena expands read
names no fact mentions; it costs `O(1)` per read; and a `debug_assert` would abort in the build
that shows a violating read becoming a residual (rule 8: a visible residual beats plausible
unproven C). It decides nothing r2ssa owns: the rule is r2ssa's, the policy (a residual the proof
line names) is the renderer's. With the lane pick and the check both gone (`fact_mutants`, a
test-only feature) the guard alone makes the read a residual, under the four write orders; with the
pick gone and the check kept, r2ssa refuses the function. `Values` holds the block as a `Cell`,
set only by the spelling pass (D64).
- **Siblings.** `low_lane_value` had one caller. The other first-of-class picks and where they go:
| Pick | Used as | Verdict |
|---|---|---|
| `low_lane_value` in `narrowed_argument` | a call argument the text reads | the defect; fixed |
| `proven_low_lane` in the callsite certificate | a proof that the bits are equal | unchanged: it proves the bits, `defined_before` the scope |
| `class_value` in `PredicateFact::comparison` operands | class names for range narrowing, trip and tail counts, certificates; r2types copies them into `PredicateComparisonFact` | not a read, no renderer reads it |
| `StackReloadSourceCertificate::canonical_source` | a key for r2types' address steps and field inference | not rendered |
| `class_value`/`class_key` in `values.rs`, `uncertified_call_reads`, `expression_phi_has_single_canonical_root`, `tail_offsets`, `address_candidates`, `predicates` | equality keys | safe |
| `copy_root`, `same_integer_root` | the view's root, which dominates what it is the copy root of | safe by construction |
r2ssa alone was enough for the 25,735 functions measured: with the guard turned into a report, the
five binaries read 0 names out of scope. The guard is the second line, not a second owner.
Measured at the code tip, against `15486ea7` (the same dumps, `pddj` of every function):
| | bash | ls-toybox | pigz | 0pack | pumasim | 10 more binaries |
|---|---|---|---|---|---|---|
| functions | 1,568 | 827 | 127 | 8,437 | 14,776 | 14,034 |
| changed | 0 | 0 | 0 | 0 | 2 | 0 |
| newly refused | 0 | 0 | 0 | 0 | 0 | 0 |
| reads out of scope, tip (the dominance audit) | 0 | 0 | 0 | 0 | 4 in 2 functions | |
| reads out of scope, now (also under O1, O2, O3) | 0 | 0 | 0 | 0 | 0 | |
The two pumasim functions keep their ledger (rendered 352 and 327, residual 39 and 30 obligations,
11 and 7 sites) and read `(uint32_t)xmm1_6` where they read the lane; the guards build, which made
the argument a residual by an accident of order, has 47 and 38. Residual sites (135,070 in the five,
226,226 in the ten) and gotos (27,108 and 48,832) are equal. clang `-Wuninitialized` (with
`r2sleigh_residual_void` noreturn), functions that warn: bash 1, ls-toybox 1, pigz 0, 0pack 5 and
pumasim 13 before, 1, 1, 0, 5 and 11 now; the 18 left all say `may be uninitialized`, and each
binary has as many as at the parents. 240 generated float objects (1,920 functions, `libm`
callees): 56 functions warn at the tip and 0 now, under the natural order and O1, O2 and O3 alike;
90 change text, none is newly refused, and residual sites go 63,560 to 63,531 (34 functions have
fewer; `s106_O1` `0x8000540` has 19 to 25 and one more `goto`, since the call now reads the
carrier, whose chain holds a register read from entry, and renders that chain as residuals).
`census.py orders` over bash, ls-toybox and pigz (2,508 functions): O1, O2 and O3 differ 0, 0, 0
(the text moves in 64, 332 and 720; 0, 1 and 7 differ only in a selection), now over the refused
functions' ledgers as well; 0pack `0x476982` has 9 residual sites under all four orders. Release
`pdd` with `aa`, instructions retired, two runs each, tip then now: 0pack `0x62ecf0` 71.86G and
71.73G, 71.89G and 71.86G; pumasim `0x23fdf0` 85.01G and 84.98G, 85.13G and 85.06G (+0.1%, the
renderings md5-identical), peak footprint 251 to 261 MB and 255 to 257 MB in both.
**Mutation.** `scripts/mutants_d68.py` applies each of 25 mutants of the checks and runs the tests
that should kill it: 22 die (the lane pick unfiltered, with and without the check of the facts; the
check unwired; the guard unwired, never told its block, letting a name through or naming the wrong
cause; each of the validator's eight fact families dropped; `defined_before` off by one, with its phi
case wrong, its entry case wrong or always true; the production write order changed three ways).
Three live, each with its reason: the guard with r2ssa's pick and check both gone (by design: the
guard holds alone, and the tests that accept a residual accept it), `strictly_dominates` for
`dominates` in `defined_before` (equivalent: the same block returned above), and the block-end read
placed at the block's start in the validator (no fact the fixtures hold reads a value its own block
defines; the SSA validator orders those reads).
Left: a later member of the lane's class that does dominate the call is not looked for (the carrier is
used; `O(k)` per argument for k members, and `low_lanes` holds only the first of each class); the
`canonical_source` of a stack reload and a comparison's operands are not checked, since nothing
renders them; the production-order mutants are pinned by one unit test, the feature-on build being
the one the workspace tests compile.
## D4's frame, as it is designed
The machine stack below the entry stack pointer is one region; C must see one
@@ -155,6 +341,53 @@ its array.
Cost: one pass over the frame objects, `O(objects)`.
## D4's callee declarations
C declares a callee once, so every call to it in a function states one prototype. That prototype is
a fact of the function, not of the order its blocks are written in: `Values::decide_prototypes`
takes, per named callee, the declaration of its first call in address order (block, then place in
the block) among the calls whose plan states one, `O(calls log callees)`, before any block is written.
A call that states another is `DeclarationDisagrees` wherever it is written, and the declarations
the function emits are those of the calls written. A declaration is a function of the call's plan
alone (`Values::declaration`): its argument classes, its declared prototype where there is one, its
result class and `variadic`/`noreturn`.
Before this the first call rendered won, so the placement of a loop's exits (SD, 2026-10-22) changed
which call that was. pumasim `png_decompress_chunk` (`0x003b6b20`) calls `png_free` seven times, two
of them with `rax` overwritten after the call and five not; libpng's `png_free` returns nothing and no
summary reads its body (it jumps through a loaded pointer), so each call declares from its own result. Written in
address order the first call overwrites `rax`, so `void png_free` won and 23 sites were residual;
with an exit written in the loop a call that does not overwrite it was first (`uint64_t png_free`,
20 sites, 25 obligations moved from residual to rendered with no new evidence). The invariant is the
first outcome, whichever block is written first, and the engine test
`a_callee_is_declared_by_its_first_call_in_address_order_whichever_block_is_written_first` holds it
under two layouts. A callee's result is a program fact the call sites should not each spell (ROADMAP
D62).
## The import-stub route
A function is an import's stub when the program says its address is one and its body is only the
transfer through the import's slot. Discovery owns the first half: `program::naming::imports` reads
the stubs the container states (Mach-O `S_SYMBOL_STUBS`) and decodes the linkage sections (`.plt`,
`.plt.sec`, `.plt.got`), and `Program::import_at(entry)` answers it in `O(log stubs)`. The body owns
the second (`stub::import_stub`): one tail call, no store or other call, no register written that the
transfer does not carry, so a stub that flushes a cache line first is a body
(`an_effect_before_the_jump_keeps_a_stub_a_body`). The transfer must also reach that import:
`import_at` of the address it goes to names the same one, so a container-stated stub patched to jump
through another import's slot is a body (`a_stub_whose_jump_goes_to_another_import_is_a_body_not_the_stated_stub`),
never `free` declared with `malloc`'s types. The stub's name is the one discovery states, not the
callee identity's.
The route used to ask only the second half. A function in `.text` whose last instruction was a jump to
an import's stub, or through its slot, was declared away as the import, with its branch and its
argument arithmetic: bzip2 `default_bzfree` (`0x5b00`: `test rsi, rsi; je; mov rdi, rsi; jmp free`) and
`default_bzalloc` (`0x5b20`: `imul esi, edx; movsxd rdi, esi; jmp malloc`) printed "import stub `free`
has no body" and no C, while `afl` listed neither as an import. Two owners decided one fact and
disagreed. Of the 63 DecBench rows r2s declined this way (2026-10-23), 52 write an argument register
before the jump, 2 test first, and 9 are a bare `jmp` to the stub. All 63 render now: 22 without a
residual, 24 with a parameter the body reads and nothing declares (P7), 16 with an `int` result whose
upper half is a residual (W47), 1 with both.
## The proof line
The proof line states what the rendering did and did not show. "Nothing was
doc/adr-measurement-discipline.md
+3 -1
View File
@@ -120,7 +120,9 @@ An evidence review of the whole project (caps and caches, a capability matrix, d
measured performance, then a critic who re-verified five claims) found the following; D51 to D57 in the ROADMAP own it.
- **Quality.** r2s is fourth of four on DecBench union-perfect (20.1%, [record](../tests/decbench/RESULT-2026-10-17.md)),
first on byte_match, last on type_match. Its edge is verification (certification, equivalence, oracles, typed
first on byte_match, last on type_match. (Corrected 2026-10-23: that run lost every bash row for r2s, kuna and ghidra to the
OOM killer. On all 6,978 rows r2s is third on union-perfect (28.0%), first on byte_match and third on type_match;
[record](../tests/decbench/RESULT-2026-10-23.md), D66 gates a lost row.) Its edge is verification (certification, equivalence, oracles, typed
refusals), not recovery. Coverage is proof-first: bash renders 38.0% of 1,568 functions with no marked
construct, fedora ls 58.7%.
- **Behind the field, by name.** IDA, Ghidra and Kuna decompile more architectures (r2s: x86-64 and AArch64 only);
doc/adr-structure-dominator-tree.md
+386 -7
View File
@@ -38,11 +38,34 @@ terminator `None` must end with no out-edge, so a call ends an occurrence only
when the prologue declares the callee `__attribute__((noreturn))`; a trap is
the target's builtin. Fabricating a `return` is not acceptable.
A compound test `φ(p₁…pₖ)` (a condition chain) is admitted only when the
rewrite that made it records the chain on the occurrence: a DAG of blocks
entered at its head whose non-head blocks render nothing, checked as one unit
by requiring its exit function to equal the test's by BDD identity. The
checker never infers a chain.
A compound test (a condition chain) is `&&`, `||` and `!` over atoms that each carry the marker of the block whose
test they are, the head's own included, so a `!` is the chain's and never the lowering's. The checker reads the
DAG of those blocks from the CFG: each is a two-way branch, every block past the head is entered only from the chain
and is written nowhere else, and the DAG has two exits. An edge to a later test is an edge of the DAG; one back to the
head or to an earlier test leaves it, so a loop whose test is a chain ends in its own head as one of the two exits. It requires the DAG's exit function to equal the test's by BDD identity:
the test evaluates its atoms left to right and skips the decided ones, so its trace is the reduced ordered BDD of
its function under that order, and the DAG is that trace only if every edge runs forward in the order and no atom
decides nothing. At most 16 atoms and 512 BDD nodes; past either the chain is refused with a typed `ChainFault`
and nothing is written. The head's edges are then the chain's exits, read exactly: no inversion is read into its
arms. The checker never infers a chain; it reads the markers the rewrite put there.
A test of one such atom is read the same way, which is how a rotated loop's polarity is certified: the loop's test is
its block's atom, `!atom` where the machine leaves on the true edge, and an edge back to the lone block is an exit.
A test the lowering wrote carries no marker and is judged by its edges exactly; a swapped pair of labels is a
violation, not an inversion (the certificate used to admit it, so `while (c)` and `while (!c)` certified alike for a
plain test; a stage that dropped a negation was kept). One text change follows the final certificate:
`respell_negations` turns a `!` over a marked integer comparison into the opposite comparison (`Eq`/`Ne`, `Lt`/`Ge`,
`Le`/`Gt`); a table test evaluates all six rows against the negation on every 8-bit pair and the edges of 16, 32 and
64 bits, signed and unsigned, and a float comparison, false on a NaN, has no opposite and keeps its `!`.
Whether a block may be a member of a chain has one owner, the text: the block's text is its test and nothing else
(`Explore::edge`, `test_regions`), so D2 writes a value the function keeps as a statement of its own and `sub esi, 5;
jne` with `esi` returned is a test of its own (`a_second_test_joins_the_chain_only_if_it_computes_nothing_the_function_keeps`),
and a load, store or call is a statement (`a_second_test_that_reads_memory_is_a_test_of_its_own`). r2ssa's
`owes_only_its_branch` was a second owner of the same question and is gone (2026-10-23): replacing it by `true` moved 0
of 3,765 renderings, a condition holds a load in none of them, and no program the engine tests build makes it refuse
what the text accepts. What it guarded is read where it can be read, as below: a statement of a member that the chain
does not carry is a lost statement, a `Statement` violation, and an effect that went unrendered is the ledger's.
A selection `x = c ? a : b;`, written by the readability stage for an `if`
whose arms each assign `x` once, is read as that `if`. The statement carries
@@ -52,6 +75,25 @@ arm's edge. The rewrite records the statement as a selection. The checker
refuses one whose test is not a conditional branch, whose target is not a plain
variable, or whose value writes: that write is a second effect of the arm.
### The statements of a block
`I_T`: every statement the placement wrote for a block is written, and as often as any other statement of that
block. The unit is a block's text, which a copy reproduces whole. A *statement* is a marker (§5) counted by the
marker it stands for (`Placed::statement_of`: itself, or the one `copy_of` copied); the walk meets one at a statement
under a marker, at a value of a selection, and at each test of a chain past its head (the head's own is on its
statement). An edge's copies are statements of the block the edge leaves, one marker each (`Writer::observe_copy`),
which name no instruction, so no line changes, and which open no occurrence, so the walk reads them where it read the
unmarked statement; the latch hoist moves only statements so marked, since a block's own assignment is not a copy its
edge carries. A copy that drops a statement of its block, or a stage that drops the copies a latch carries, leaves
that marker one fewer than the block's others, and a block whose statements are all gone is a pass-through to the
walk (§3 reads a block that rendered nothing through): with every edge the machine's, only this count says it was
not one. O(statements): one pass over the markers met, one over the markers written, a minimum and a maximum by block.
What it does not read: a statement moved from one copy of a block to another leaves the counts equal (no stage does
it); a block of one statement has no other to differ from, so a copy of it that is missing from one site and present
at another shows only in that site's edges; a copy's text is not compared with its original's (`remint` clones the
expressions, so only a producer that rewrites a copy could differ). ROADMAP D61 holds these.
### §4 Placement
Every block except the entry is placed by its immediate dominator `x`. With
@@ -63,6 +105,16 @@ the outermost loop in `Λ(y)`, and its whole dominator subtree moves with it
is `continue` to the innermost enclosing header, inline adjacency, or `goto`.
Merge children and exits are written in ascending RPO.
An exit `y` of loop `L` with one forward way in (the edge from its immediate dominator `x`) whose
dominator subtree no edge leaves (*closed*) is written where that edge leaves `L`, not after it,
unless `break` reaches it: the first exit of one loop stays after the loop. `Placement::inlines` is
the one predicate `compute` (the exit lists) and `edge` (the arms) share. A closed subtree holds no
loop another block belongs to, so the text it adds inside `L` leaves no `continue`, `break` or label.
Closedness is the preorder interval of the subtree holding the extremes of its blocks' successors'
positions, `O(B + E)`. The difference-on-tree count over `idom` answers how many edges leave a
subtree and needs the lowest common ancestor of each edge (`O((B + E) log B)`, a lifting table);
only whether any edge leaves is asked, so it is not built.
This writes each block exactly once, satisfies `I_S` for any graph,
reducible or not, runs in `O(|B| + |E|)` after the dominator tree, and makes
every backward `goto` target a loop header or an irreducible-cycle entry.
@@ -84,7 +136,36 @@ dispositions and expressions are asked for only when written. Preference among
rewrites is fixed: fewest labels, then fewest duplicated blocks, then shortest
text. `break` never crosses a `switch` to reach a loop. `LoopCertificate::condition`
is a rendering choice, not a certificate fact; it stays only for `for_loop`
until that certificate names its exit edge.
until that certificate names its exit edge. The stages run in `render::structured`, each kept only while the
control certificate still holds: condition chains (on the tree as placed, where every merge is a `goto` and
every exit is a bare transfer or a block only the chain reaches), `shape`, the latch's copies, loop rotation. The
certificate holds each stage to the machine's edges with their labels, so a stage that changes a test's polarity says
so on a marked atom or is refused (§3).
A terminal tail is a run of at most four expression statements from a label, through the sibling
blocks the text falls into, to a `return`, a call to a callee declared never to return, or a `goto`
to another run (a cycle of jumps has none); it holds no residual or gap, because a copy would be a
second site of one unproven cell (the skipped-block copy asks the same predicate, `tails::copyable`,
so a skipped block holding a residual stays one). Each `goto` to a tail is replaced by a fresh copy (every marker
minted anew), and a tail nothing falls into and no `goto` names goes with the blocks it ran through
(`structure/tails.rs`). A copy carries its blocks' whole out-edge sets, which `I_S` already asks of a
block written twice, so a copy where the edge did not go is refused by the certificate. The pass
runs after the skipped-block copy, which needs the jumps the tail copies replace: `bi_windup`'s
`put_byte` is a block the compiler merged out of two arms and the copy into those arms is the source's
shape, where a tail copied first leaves it shared. The preference order of this section holds: a tail
copy removes a label, so it outranks the duplicate block it costs.
A `return` whose value is a merge of the return's own block, and whose inputs are pairwise-distinct
canonical terms, is copied into every arm end of the conditional that falls into it. That is the
one place a block is duplicated without a label to remove, so it needs a fact the CFG states and a
join does not: the edges carry values no other edge carries (`Values::returns_distinct_merge`, `O(1)`
per return block, over the graph's phi and the canonical terms, never names or spelled text). A
side effect before the return, a label something still jumps to, a merge that carries itself or one
two edges give the same term are not this shape. The assignment before each copy stays
(`x = a; return x;`); folding it touches the merge's ledger accounting and is its own step.
The unguarded form (any `return` after a conditional with two falling arms, `return;` too) was
measured and not shipped: the CFG does not say a source wrote two returns where it wrote one, and
`return a || b` compiles to the same merge of `1` and `b` as `if (a) return 1; return b`.
### §6 Decided branches
@@ -103,11 +184,17 @@ per round; a round that decides nothing ends the loop.
- S1: placement in `structure/place.rs`, the certificate in `structure/certify.rs`; the region builders, domain proofs, linearizer, structuring deadline and every `safety_reason` deleted; noreturn prototypes spelled; irreducible cycles keep a complete inventory.
- S2, in part: jump-to-next and unreferenced-label elision, `break`/`continue` to the innermost loop header and the first block after it, and the checker. The rest of the original S2 was in the legacy cleanup pass `structure/rewrite.rs`, deleted with the legacy decompiler in 51b9d6a9 (2026-10-10) and not replaced: while and do-while rotation (the certificate supports both, the writer emits only `for(;;)`), switch-tail absorption, bounded tail duplication of returns, short-circuit `if`, and return into arms. 2026-10-22 measurement on DecBench (`tests/decbench/RESULT-2026-10-17.md`): this is why 40.2% of r2s's scored functions end in a goto.
- S3: DecBench read 680 of 860, every mean above the previous run.
- S2, four of the five (2026-10-22, B5): short-circuit `if` as condition chains (`structure/condition.rs`), `while` and
`do-while` rotation (`structure/rotate.rs`) with the latch's copies written before its test (`structure/latch.rs`),
return into arms (`tails::join_returns`) and bounded tail duplication of returns (`tails::inline_tails`); switch-tail
absorption is still not replaced. See "the recoveries together" below.
- S4 (2026-10-22): the recoveries below, each checked by `I_S` on its result.
## Left
- Labels over functions the old structurer handled exceed what it rendered. Exit: no more labels than before over that set.
- Condition-chain rewrites (`while (p₁ || p₂)`) and node splitting of irreducible entries. `ControlBdd` no longer exists, so chains need a BDD identity check before they land. Exit: both gated by the checker.
- Node splitting of irreducible entries. Exit: gated by the checker.
- Whether lexical ancestry equals dominance retires `region_does_not_dominate_occurrence`. Exit: placement census measured (post-dominance hoisting and `goto` into scope may move the refusal rather than remove it).
## 2026-10-22: what the gotos cost and what recovers it
@@ -130,3 +217,295 @@ the other three tools, a parser classifier over all of them, GED scored with Dec
are not worth direct work: another tool is perfect on 3 of 567.
- Replication into fallthrough arms conflicts with section 5 (fewest labels, then fewest duplicated blocks) unless it
is guarded by the phi-of-distinct-edge-copies condition; amend section 5 or ship only the guarded form.
## 2026-10-23: the guards, and what a copy says
The last two skeptics (50 and 59 mutants of the seam over bash, ls-toybox and pigz, 2,508 functions, and 70,000 base
against tip vectors with no wrong C) left findings open on the chains and the tails. ROADMAP D59 and D60 close them;
what they leave is D61.
**A chain that continues to its own head.** `condition::exits` read an edge back to the head, or to an earlier test
of a chain, as an edge of its DAG. A chain whose second test goes back to its head then found one exit where it has
two (`ExitCount { found: 1 }`), and the stage reverted every chain of the function (bash `0x2e540`). Only an edge to a
later test is inside the DAG; any other edge to a member leaves it, as a lone test's always did. Over bash, ld-2.31.so,
fedora_32_bin_ls, ls-toybox, pigz, bashbot O0 and static-glibc (3,765 functions) 12 functions change, none is newly
refused, no goto or residual site is added; eleven move only control lines (more `&&`, `do`/`while` for `for (;;)`,
fewer `break`, the statements equal) and pigz `get_header` writes one copy fewer of a two-statement tail. The engine
program, `top: movzx edx, [rdi]; add rdi, 1; test dl, dl; je done; cmp dl, 0x3a; jne top; done: lea rax, [rdi - 1]`,
rendered two tests in a `for (;;)` and renders `do { c = *p; q = p + 1; p = q; } while (c != 0 && c != 58); return q - 1;`.
**De Morgan.** `rotate::negated` swaps the connective when a loop leaves on a chain's true side. No hand loop at O0 to
Os and none of the 2,508 functions reached it (R1, `let flipped = op;`, survived everything), because a chain that
continues to its head could not be written. D59's program does not reach it either: its continue carries a copy, so
that arm is the `if`'s body and the chain is spelled on the side the loop stays by. Where nothing is copied on the way
round both exits are bare and the chain is spelled on the exit with fewer negations, the one the loop leaves by; the
engine test for that program (`a_loop_that_leaves_on_a_chain_stays_on_both_tests_negated`: `do { ... } while (c != 0
&& c != 58)`) and the unit test beside the rotation each fail with only the connective kept.
**The latch hoist's guards.** Each of the three data guards has an engine test that fails with only it removed, and one
test each: a test that reads what the copy writes (`a_copy_stays_on_its_arm_when_the_test_reads_the_variable_it_writes`),
code after the loop that reads it (a tail too long to copy to its jumps, read after the loop as in pigz `0x4079f0`; the
rule cannot tell which exit reads, so it is conservative where the reader is an exit taken before the latch), and a
test on an unproven call (a copy is not moved across a test that traps where its cell is unproven).
**Statements.** See "The statements of a block" in section 3. Mutants of the previous skeptic that were invisible to
the certificate, run again on this tree through the corpus (3,765 functions of seven binaries, the skeptic's three
binaries among them): L5, the hoist drops its copies: 543 functions change, the certificate reverts the stage in every
one and none loses a statement (before, 388 of 388 changed functions passed); T2, a tail copy drops its first
statement: 488 change and the certificate refuses all 488 (before, 207 of 280 refused and 73 passed); T12, a joined
return's copy drops its first statement: 204 change, 204 refused (before, 144 of 144). The latch hoist took any
assignment of its arm for a copy until the certificate refused it in four functions; it moves only copies an edge
carries.
**Chain members, one owner.** The text decides (section 3); r2ssa's `owes_only_its_branch` is deleted, with the
`NotInert` fault and the certificate's `inert` parameter. Forced to `true` it moved 0 of 3,765 renderings, a condition
holds a load in none of them, and the text refused every program the engine tests build that it refused. The statement
clause is what reads a member's text where the fact claimed to.
**Mutants that were dead code or equal.** `judge`'s `exact` flag (C1) could not change a verdict: a chain head's
terminator is a conditional branch and its two exits differ, so neither the switch nor the converted-conditional
reading can hold; it is gone. A faulted chain's head is skipped, so the chain is the one violation (C4): the test now
asserts exactly one. `remove_run`'s `named` check (T3) was redundant by construction: every `goto` to a label inside a
resolved run was copied, since the run's suffix resolves where the run does; it is gone. The order `append_to_ends`
visits arm ends (T9) changes which end keeps the original's markers and nothing a line, a link or the certificate
reads; the comment no longer claims an order.
**The sweep again.** 61 mutants (the skeptic's 50 and 11 of this change), run against r2dec's unit tests, then the
engine's where those pass, then the golden census: 5 no longer apply (C1, T3, V2 are deleted code; S8 and V4 are
rewritten as S8b and V4b, both killed); of the 56 that run, 42 are killed by unit tests, 8 more by engine tests (S7,
T10, P2, V1, C7, N5, N10, V4b), and 6 survive both: C3, T7 and T13 are
killed by the golden census only (1, 5 and 3 tests), S5 and S6 (the final and the stage roles without the never-returns
evidence) were killed by the corpus only (70 functions refused, 37 changed with 5 more gotos, the skeptic's figures)
until the engine test `an_arm_ending_in_a_call_declared_never_to_return_is_a_conjunction_that_leaves_by_it` (a
conjunction whose exit arm calls `exit`: S5 refuses it, S6 reverts the chain) killed both, and T9 is equal. All 11
mutants of the statement clause, the edge copies and the hoist's copy test die. ROADMAP D61 holds the three
census-only guards.
**Cost.** The statement clause is one pass over the markers met and one over the markers written per certificate, and
a certificate runs once per stage that changed the body and once at the end. Release `pdd` with `aa`, two runs each,
instructions retired, base then this branch: 0pack `0x62ecf0` 71.793G and 71.704G against 71.721G and 71.721G,
pumasim `0x23fdf0` 84.958G and 84.986G against 84.985G and 85.027G (from -0.1% to +0.1%), peak RSS 283 to 301 MB in
both, the renderings byte-identical. Over every function of 0pack and pumasim (23,213, mangled names included) 8
change, all of them the loops D59 reads (a `for (;;)` with a chain's two empty arms and a `break` becomes
`do { ... } while (a || b)`; pumasim `0x2ed240` also writes one statement of a tail once), none is newly refused and
none gains a goto or a residual site. Over bash, ld-2.31.so, fedora_32_bin_ls, ls-toybox, pigz, bashbot O0 and
static-glibc with every name (3,953 functions) the 12 of D59 change and no other.
## 2026-10-23: the guards and the spelling order, stacked
D59 and D60 (the guards) and D58 (D2 spells every block once, in reverse postorder, before this section's placement
writes any; "D2's spelling order" in adr-decompiler-rewrite.md) landed on one base and stack in this order. They meet
in three places. An edge's copy is spelled by `render::spelled` as `EdgeText::Copies` and marked by the Writer as a
statement of the block it leaves (`observe_copy`), which is what I_T counts and what the latch hoist moves. The latch
hoist's reads are counted once for the whole function (`Scan`) and its copy test asks `Placed::edge_copy`. A chain
member is a test whose text is nothing else, so the regions index asks only `test_block`.
No function changes under both: over bash, ls-toybox, pigz, 0pack and pumasim (25,735 listed functions) 17 change under
the guards alone and 55 under the spelling order alone, none is newly refused, and the effects add: residual sites
0pack 47,850 (base), 47,850 (guards), 47,898 (both); pumasim 67,628, 67,628, 67,644; bash 10,316, 10,316, 10,322;
ls-toybox 7,902, 7,902, 7,905; pigz 1,301 throughout. One function gains a goto, bash `0x48140` (1 to 2): D58's rule
reads its tail's `r13_4` as a residual (a forward copy into the merge is a gap), and a skipped block that holds a
residual is not copied into the arms that fall to it (D61 (3)). The write orders: O1, O2 and O3 differ 0, 0, 0 over
bash, ls-toybox and pigz (2,508 functions) at the tip, against 0, 120 and 187 on the base with the knobs alone; the
text moves in 64, 332 and 720 functions, so the knobs are live.
On the mined set (1,570 functions, 1,305 scored, DecBench's GED, the method of the section below):
| Build | Perfect | Mean GED | Total GED | Gotos | Functions with a goto | Held out gotos |
|---|---|---|---|---|---|---|
| base | 206 | 43.795 | 57,153 | 2,661 | 426 | 2,100 |
| and the guards | 206 | 43.795 | 57,153 | 2,661 | 426 | 2,098 |
| and the spelling order, the tip | 206 | 43.795 | 57,153 | 2,661 | 426 | 2,098 |
The guards change no mined-set function. The spelling order changes the text of eight (`inflateCopy` of zlib's four
programs at two optimisation levels: its block-transfer locals are numbered `_2` in reverse postorder) and no graph, so
no GED moves. Held out (23 of radare2's test ELFs, 1,834 functions): rendered 1,496 and refused 338 in every build, none
crashed; the 2 gotos fewer are fedora_32_bin_ls `0xe690` (36 to 34, the chain that continues to its head). Release
`pdd` with `aa`, instructions retired, base, guards, tip, mean of two runs: 0pack `0x62ecf0` 71.71G, 71.78G, 71.75G;
`0x58e3d0` 81.12G, 81.15G, 81.19G; `0x4e5f90` 60.35G, 60.39G, 60.35G; pumasim `0x23fdf0` 84.95G, 85.00G, 85.02G;
`0x1fafb0` 70.03G, 70.06G, 70.02G; `0x2312c0` 73.11G, 73.15G, 73.13G (from -0.02% to +0.10%), peak footprint 156 to 278
MB in every build, the six renderings byte-identical. The census goldens (442) and `stable_temporaries` (24) hold byte
for byte at both heads, so nothing was blessed. The equivalence gate on the tip holds all 828 baseline records on x86-64
(744 equal, 63 residual-trap, 1 slow, 20 unsupported) and on AArch64 (750 equal, 57 residual-trap, 1 slow, 20 unsupported) with none differing.
## 2026-10-22: the recoveries together
Two branches landed on one base and stack in this order: condition chains, `while`/`do-while` and the latch's
copies written before its test (B5-b), then joined returns, terminal tails, sole closed exits and a callee's
declaration read in address order (B5-a). The marker table is one type, `shape::Occurrences`, which implements
`condition::Marks`, so a block's text copied by any stage is an occurrence of that block the certificate reads. In
`render` the order is: placement (a sole closed exit is written where its edge leaves the loop), condition chains
(a stage), `shape` (jumps to the next position, joined returns, the skipped-block copy, terminal tails), the
latch's copies (a stage) and loop rotation (a stage). A stage stands only while the control certificate holds,
`shape`'s copies are certified at the end, and `respell_negations` is the one change after the certificate.
Measured on the mined set (bzip2, zlib and grep at O2 and O2-noinline: 1,570 functions, 1,305 scored in all three
builds) with DecBench's own GED. One method for the three builds: the renderings come from each build on the Mac and
are scored on contabo. These are mined-set numbers.
| Build | Perfect | Mean GED | Total GED | Gotos | Functions with a goto |
|---|---|---|---|---|---|
| base `cb16ec2e` | 203 | 46.095 | 60,154 | 4,654 | 623 |
| chains, rotation, latch copies `0a654255` | 197 (0 newly, 6 lost) | 45.195 | 58,980 (-1,174) | 4,578 | 615 |
| and the tails, the stack's head | 206 (10 newly, 1 lost against the step before) | 43.795 | 57,153 (-1,827 more, -3,001 in all) | 2,661 | 426 |
Held out (23 of radare2's test ELFs, 1,834 functions, none of them mined): rendered 1,496 and refused 338 in every
build, the same 338 functions, none crashed; gotos 2,941, 2,915 and 2,100. One function gains gotos over the base:
freebsd-ls `0x205b40`, 0 to 2. Its skipped block holds a residual cell and the base wrote it into two arms, two
residual sites for one unproven cell; `tails::copyable` is asked of every copy, so the block is written once and the
two arms jump to it. That is the visible residual rule 8 prefers to a duplicated one, and the one case the tails'
own held-out sweep ("no function gains a goto") did not see, since it ran before the rule reached the skipped-block
copy. The equivalence gate on the stack's head holds all 828 baseline records on x86-64 (744 equal, 63
residual-trap, 1 slow, 20 unsupported) and on AArch64 (750, 57, 1, 20) with none differing. Release `pdd` (with `aa`, 2 runs) on the two named
functions and the two largest others of each of 0pack and pumasim moves -0.16% to +0.10% instructions retired against
the base, with peak RSS 191 to 303 MB in every build.
**Kept costs.** The chains cost 0.2 mean GED and win none on the set they were measured on, and the latch's copies
take 438 loops to `do-while` for 0.98 less mean GED at the price of 5 perfect (a single-block source `for` scores
better as a `for (;;)`); the three stages together are 197 perfect for 203 and 0.9 mean GED fewer on this set. Both
are kept for the reason in the section below: they spell the machine's test and loop as the CFG proves them, nothing
was tuned to the score, and the benchmark rewards the source's spelling, which the binary does not state. The tails
win the perfect count back (10 newly perfect, 1 lost) and take 1,917 gotos away. No recovery is added or dropped to
move the sum.
**Ledgered, not fixed here.** ROADMAP D59 (a chain whose exit is a `continue` to its own head faulted `ExitCount` and
reverted every chain of the function: bash `0x2e540`) and D60 (the certificate asked nothing of a copy's statements)
are closed in "2026-10-23: the guards, and what a copy says", and D61 holds what they leave. ROADMAP D58 was ledgered
here and has landed since: whether a statement renders and whether a name reads as a value or as a NeverAssigned
residual are spelled once, in reverse postorder, before this section's placement writes anything ("D2's spelling
order" in adr-decompiler-rewrite.md), so none of this section's write orders reaches a verdict.
**Left**, by when:
- Next: the assignment before a copied or joined return, folded at the owner that spells the edge copy
(`return a;`); the exit edge's copies after a loop whose only exit is the latch (about 140 loops); D61.
- Conditional: the four-statement bound and the skipped-block-first order were chosen on the mined set, so they move
with a measurement on projects the analysis did not mine; the unguarded join (+17, 2 lost), only with a CFG fact that
separates `return a || b` from two returns; a `while` whose exit arm is not a bare `break` (34 loops), once the exit
is the only way out.
- Deferred: chains whose DAG is not read-once (a selection between two tests); splitting a chain around a block that
renders text; node splitting of irreducible entries; tails of five to sixteen statements (133 gotos) and jumps over
a block no rule closes (588); the AArch64 mined set, not measured.
- Unnecessary: the lowest-common-ancestor count for closedness, whose answer the O(B + E) interval extremes give.
## 2026-10-22: condition chains, loop rotation and latch copies
Three of the recoveries the goto analysis ranked landed in this order, each a stage kept only while the control
certificate holds. Measured on the mined set (bzip2, bzip2recover, zlib's libz, example and minigzip, grep; O2 and
O2-noinline; 1,050 functions, 438 with a non-degenerate source CFG), scored with DecBench's own GED (`vj_ged`,
Joern CFGs), which reproduces the 2026-10-17 run's value for 435 of the 438. These are mined-set numbers.
| Step | Renderings changed | Gotos | Mean GED (scored) | Better / worse | Perfect |
|---|---|---|---|---|---|
| base (`d4792d55`) | | 4,143 | 49.11 | | 60 |
| condition chains | 251 | 4,080 | 49.31 | 14 / 51 | 60 (0 newly, 0 lost) |
| while and do-while | +21 loops | 4,080 | 49.31 | 0 / 0 | 60 |
| latch copies | +438 do-whiles | 4,080 | 48.33 | 72 / 12 against the step before | 55 (0 newly, 5 lost) |
Counting each function once (zlib's code also sits in `example` and `minigzip`): 302 scored, mean 47.97, then
48.14 after chains, 47.13 at the end; perfect 48 to 45.
**Condition chains.** Invariant: a decision DAG of blocks that render nothing but their test, two exits, the test
equal to the DAG's exit function by BDD identity. Owner: `structure/condition.rs` (derivation, BDD, check).
O(statements) with at most 16 blocks per head. Rejected: matching
the nested `if` text alone (it cannot see a shared test reached by `goto`, and the checker would infer the chain), and
a name or flag-pattern check for the blocks. The GED got worse, not better: Joern builds a node for each operand of
`&&` and `||` and one for the whole test, so a collapse costs a node and pays only where the source spelled `&&` and
everything else about the function already matches; 13 functions rose by exactly 3. The text is the machine's graph
and reads as the source's (`gate_three`, `bounded_fetch`, `BZ2_bzWriteOpen`'s five-way `||`); the benchmark does not
reward it on this set (analysis ceiling 29 newly perfect, measured 0). Only the DAGs that are read-once collapse:
`a ? (b ? X : Y) : (c ? X : Y)` is refused as `NotShortCircuit`.
**While and do-while.** Invariant: the body is just the header's marked `if` with a bare `break` on one arm, or ends
in the latch's with nothing else going back to the top. Owner: `structure/rotate.rs`, O(statements). The 21 loops
that rotated are GED-neutral. 747 `for (;;)` remain: 450 have an exit arm that is not a bare `break` (many carry the
exit edge's merge copies), 137 a staying arm that is more than copies, 90 copies the next recovery's conditions
refuse, 34 a header `if` with no bare `break` arm.
**Latch copies.** Invariant: the staying arm is plain assignments, the test calls and writes nothing and reads none
of them, nothing of the function outside the loop reads them, no `continue` would reach the test. Owner:
`structure/latch.rs`; the proof is the text's own def-use. Rejected: an r2ssa liveness fact (it is per SSA value, and D2
folds single-use values into the test and names them as C objects, which only the text states). 438 loops become
`do-while`; fnv1a32 and djb2 read as the source's loop. Against the source's `for` it is mixed (each function once):
44 functions improve (22 by exactly 3, the analysis's micro case, up to 34) and 8 worsen, and 3 functions that were perfect are not
(`uInt64_to_double`, `prepare_new_block`, `init_block`): Joern scores a `for` as a cycle of the test and the body, and
a `for (;;)` with the copies on the staying arm is the same two-node cycle, while a `do-while` over one block is a
single node, so a single-block `for` source scores worse as a `do-while` and a multi-block one better.
**Decision: both are kept.** Chains cost 0.2 mean GED (49.11 to 49.31) and win none (13 unique functions rise by
exactly 3; the analysis ceiling was 29 newly perfect, measured 0). `do-while` over a single-block source `for` loses 5
perfect across all binaries (3 unique) while winning 44 functions (22 by exactly 3) and 0.98 mean GED (49.31 to 48.33).
They spell the machine's test and loop as the CFG proves them, nothing was tuned to the score, and the benchmark
rewards the source's spelling, which the binary does not state. The cost is the price of the machine's graph, not a
regression to chase: the next reader should not rediscover it, and no recovery is added or dropped to move it.
What is left of the chains is in "the recoveries together".
## 2026-10-22: terminal tails, joined returns and sole exits
The goto analysis above ranked what the gotos cost on DecBench: GED scores the block graph, a label
adds no block, and the loss is a shared return where the source has several, a missing `&&`/`||`,
`while` against `for (;;)`, and a latch arm's copies. These three landed on top of the chains, each with a
unit test that fails without it, a counter-test where it must not fire, a behaviour test through
the engine (compiled and run) and a `tests/db/cmd/structure` golden on a real function:
- Joined returns (`tails::join_returns`, evidence `Values::returns_distinct_merge`).
- Terminal tails at their jumps, through the blocks after them and through jumps to other tails
(`tails::inline_tails`), after the skipped-block copy.
- Sole closed exits (`Placement::inline_exits`, `closed_subtrees`).
Measured with the tails alone on the base before the stack, on bzip2, zlib and grep at O2 and
O2-noinline (1,305 to 1,313 binary-function pairs scored in both, 552 distinct functions): these are
projects the analysis mined, so the numbers are mined-set numbers. The stack's are above. Perfect is GED 0 under DecBench's own code, run on contabo over the tip
build's output and each step's (the tip reproduces the 2026-10-17 run's GED on 1,308 of 1,313):
| step | perfect | total GED against the tip | gotos | functions with a goto |
|---|---|---|---|---|
| tip | 203 | 0 | 4,654 | 623 |
| joined returns | 207 (+5, 1 lost) | -213 | 4,654 | 623 |
| tails at their jumps, first order | 199 (+5, 9 lost) | -1,833 | 3,474 | 499 |
| sole closed exits | 199 | -1,835 | 3,460 | 497 |
| skipped block first | 211 (+9, 1 lost) | -1,759 | 3,472 | 497 |
| runs through blocks and jumps | 212 (+10, 1 lost) | -1,937 | 2,694 | 426 |
The 212 are 102 distinct functions of 552 (96 at the tip); the ten new are seven (level, function)
pairs, `BZ2_bzCompressEnd` and `BZ2_bzDecompressEnd` at both levels, `myfeof`, `buf_has_encoding_errors`
and `inflateUndermine` at one (the zlib ones in four binaries); the one lost is `skip_devices`, whose
source is `return a || b`. The unguarded join on the first step: 218 perfect (+17, 2 lost), total GED -452.
Over the 558 distinct functions the goto classes of the analysis go from 2,214 to 1,356: TR1 487 to 22,
TR4 246 to 23, FW 580 to 237; left are JN 305 and JNB 283 (a jump over a block no rule closes), FW 237,
LX 139 (an exit that is not closed), TR16 133 (a tail of five to sixteen statements), SP 86, CC 68,
IR 57.
What is left of the tails is in "the recoveries together".
## 2026-10-23: what the chain and latch passes cost
Review of #328 asked for both passes' bounds. Counted inside the passes on the 5,688 functions that reach the chain
stage (bash, ls-toybox, pigz and libc-2.32.so, of the 6,144 `afl` lists) and on 0pack's 0x58e3d0 and 0x62ecf0 and
fifty of pumasim's largest: 148,668 sequences, the longest 5,743 statements.
**Condition chains: a sequence is indexed once.** Invariant: the statements a chain removes are the regions it
consumes, and no position in a sequence moves while it is walked. Owner: `structure/condition.rs`. `Pass::sequence`
read every statement again to find the regions after each chain (k chains in n statements read O(k·n), and
`Vec::remove` shifted the tail for each region), so 200 sibling chains read 81,200 statements before and 1,200 now.
The index is built once; a chain takes the labels it consumes out of it, their statements stay where they are, and one
pass drops them when the sequence ends, so a sequence costs O(n) plus its chains' own text. Iteration stays ascending,
since descending can change which chains collapse. On the corpus the claim was real and bounded: 1,365 sequences held
a chain and none held two, so the re-reading was 9,011 of 885,426 statement reads (1.0%), at most 1,532 in one function
(libc 0x97e10). A consumed region is skipped, not read for chains of its own (the old removal made that so;
`a_consumed_region_is_not_read_for_chains_of_its_own` pins it).
**Latch copies: the reads are counted once.** Invariant: a loop's copies are written before its test only when every
read of each is inside the loop. Owner: `structure/latch.rs`. `may_hoist` counted the reads of the whole loop body for
each candidate latch, so a statement was counted once per enclosing candidate (O(n·d) for depth d): 50 nested loops
counted 8,100 expressions and 100 counted 31,200 (450 and 900 now). One scan counts the reads of the function and, for
each `for (;;)`, takes the reads of its copies inside the loop as the count at its exit less the count at its entry; a
second walk writes the loops the scan allowed, in the order the scan met them. The scan reads each expression once and
the walk touches no expression, whatever the depth. The scan takes the latch's shape before the loop's body is
rewritten, where the old pass took it after the inner loops were: `live`, `parts` and `continues` stop at a nested
loop and a rewrite only moves statements, so the shape and the counts agree (the census, byte for byte, is the check).
On the corpus the claim was real and bounded: a candidate loop sits at most 5 loops deep (4,993 of 5,688 functions have
no candidate, then 492, 158, 30, 13 and 2 by the deepest one's depth), and the per-latch rescans were 366,432 of
7,704,538 expression visits (4.8%). The worst function, libc 0x16b210, has 16 candidates side by side, which repeat
nothing: only nesting repeats a statement.
doc/roadmap-history.md
+10
View File
@@ -563,3 +563,13 @@ Standing debt
- `seal_body_proven_interface` rewrites the format parameter after the build (C1).
- x87 80-bit floats refuse ([floating-point](adr-floating-point.md)).
- Partition-first's conservatism (read closure, `CallRestore`, return carrier, call-use reads) and the semantic-kernel ADR's rewrite against F2 and D fold into D's ADR.
Amendments after 2026-10-14
---------------------------
Rows whose done work left [ROADMAP.md](../ROADMAP.md) later, kept verbatim with the date they left.
### D58, as it stood before the emission landed (2026-10-23, base `8cff06fe`)
| D58 | A rendering decision depends on facts, not on the order placement writes blocks. Landed: a callee's declaration is its first call's in address order among the calls that state one (`Values::decide_prototypes`, `O(calls log callees)`), so pumasim `0x003b6b20`'s `png_free` stays `void` with 23 residual sites whichever exit is written first. Left, next: whether a name reads as a value or a NeverAssigned residual is the text written so far (`Values::read_name`, the `assigned` set), and no write order makes it right for a loop-carried merge whose entry copy is a gap, because its latch copy is written after every header-side read; the owner is a two-phase emission in `render::values` that knows which definitions and edge copies render before any read (definite assignment, not a set of statements written), gated by an engine test that gives the same verdict under every block write order. Repro: 0pack `pdd @ 0x00476982` (`Obfuscator::ELF::RegisterManager::get_by_str`), residual sites 7 at the base and at the chains head and 9 at the stack's head (its author measured 7 before and 9 at the sole-exit commit on the branch alone), because the inlined exit is written before the latch copy `rsp_8 = rsp_10`: `rsp_8 + 8` became `r2sleigh_residual_u64(2) + 8` and `rsp_24 = rsp_8` became `rsp_24 = r2sleigh_residual_u64(3)`, the proof line's never-assigned set (`RAX_5`, `RSP_8`) unchanged; the entry edge into the loop is the `ValuesNotRendered` gap at `0x4769b1`, so the text before the sole-exit commit (`e2401bdf` on the stack) read `rsp_8` after the loop on a zero-trip path where nothing assigns it, and the new verdict is the conservative one there. Also next: a callee's result is a program fact, so a call that discards it should not spell the callee `void` and a call that reads it `uint64_t` (the disagreement is `png_free`'s two sites that overwrite `rax` against five that do not), because the first call in address order can be the minority form and the calls that lose become gaps whose result's readers read NeverAssigned (pumasim `0x40c140`: `QPointArray::setPoint` 14 disagreeing sites against 46 once the first call by address declares it, `0x34c050`: `QGDict::look_ptr` declared `void` leaves 6 more never-assigned reads; 0pack `0x66d2b0`, `0x66f040` and `0x670e80`: `el::base::MessageBuilder::operator<<` declared `void` by a first call that drops its result, 5 disagreeing sites against 15); and the certificate asks nothing of a copy's statements (a copy that drops one passes `I_S` and the ledger, found by mutating `tails::copies`; conditional on a per-occurrence statement check that blocks written in parts do not break) The skeptic's measurements at the stack's head (2026-10-23): residual verdicts still depend on the write order, identically at the base (reversing the loop exits' write order changes the ledger of 119 of 2,436 functions, reversing the merge children's 183; bash `0x3c2b0` 0 to 3 gaps, `0x26760` 1,026 to 1,152; `0x85d70` 232 to 231 residual with no new evidence), while callee declarations no longer do (0 of 11,091 differ under both permutations, 2 and 4 at the base); the address-order declaration moves 22 of 13,093 declarations against the base (14 `void` to a value, 8 a value to `void`), 21 more obligations in 4 functions and 62 fewer in 13, and bash `0x61520` is the repro of the tie break ignoring a call that reads the result (`set_signal_handler` returns a pointer and becomes `void`, its reader `RAX_3` never assigned, 12 to 13 residual). Gate for the two-phase emission: an engine test that one verdict holds under every write order, with the permutations above as the mutants; timing: next | SD, next |
scripts/census.py
+159
View File
@@ -19,6 +19,12 @@
peak RSS grow with a function's instructions
(ROADMAP LX's exit): `afl; pdd @ f` less `afl`,
under `perf stat` (Linux); fails above X
census.py orders --r2s R2S [--out DIR] [--reuse] [--min-live N] BINARY...
pddj of every function under the write orders of a
`--features placement_mutants` build (natural, then
o1: no exit inlined, o2: exits reversed, o3: merges
reversed); fails when a verdict differs from
natural's (ROADMAP D58) or an order moves no text
census.py heap [--top N] [INSTRUCTIONS=]PROFILE...
bytes live at each dhat profile's heap peak by
allocation site (a `--features dhat-heap` build
@@ -415,6 +421,151 @@ def heap(args):
print(f"{growth} {cells} {' <- '.join(frame[:70] for frame in site.split(' <- '))}")
WRITE_ORDERS = ("natural", "o1", "o2", "o3")
NEVER_ASSIGNED = re.compile(r"never assigned, read as residuals \(([^)]*)\)")
RESIDUAL_CALL = re.compile(r"(r2sleigh_residual_\w+)\(\d+\)")
TRAILING_COMMENT = re.compile(r"\s*/\*.*?\*/\s*$")
LABEL = re.compile(r"^\w+:(\s*;)?$")
CONTROL = ("goto ", "continue;", "break;", "if (", "} else", "else", "for (", "while (", "} while (", "do {",
"switch (", "case ", "default:", "/*")
def never_assigned(code):
"""The names the proof line says were read as residuals for want of a value."""
found = NEVER_ASSIGNED.search(code)
return sorted(name.strip() for name in found[1].split(",")) if found else []
def leaf_statements(code):
"""The set of statement lines of a rendering, apart from its control (a label, a jump, a brace) and
the numbering of its residual sites: what no block write order may change. A set, since a tail is
written once per jump to it."""
leaves = set()
for line in code.splitlines():
line = line.strip()
without_comment = TRAILING_COMMENT.sub("", line)
if not without_comment.endswith(";") or line.startswith(CONTROL) or LABEL.match(line):
continue
leaves.add(RESIDUAL_CALL.sub(r"\1(N)", line))
return sorted(leaves)
def fingerprint(record):
"""What a rendering says that no write order may change: the ledger's columns, the values read as
never assigned, the residual sites by (instruction address, cause, gap kind, callee, reason), the
names it declares and links, and its leaf statements. A refused function has one too, its refusal
(the kind and the reason, which counts the obligations and names the first unaccounted one) and
its ledger, since a verdict can move inside a refusal (ROADMAP D68)."""
refused = record.get("refused")
addrs = {line["line"]: sorted(line["addrs"]) for line in record["lines"]}
return {
"refusal": json.dumps(refused, sort_keys=True) if refused else None,
"proof": record["proof"],
"never_assigned": never_assigned(record["code"]),
"sites": sorted(json.dumps([addrs.get(site["line"], []), site["cause"], site["gap"], site["callee"],
site["reason"], site["type"]]) for site in record["residuals"]),
"variables": sorted(json.dumps(variable, sort_keys=True) for variable in record["variables"]),
"links": sorted(json.dumps(link, sort_keys=True) for link in record["links"]),
"statements": leaf_statements(record["code"]),
}
ASSIGNMENT = re.compile(r"^(\S+) = (.*);$")
def selection_only(natural, mutant):
"""Whether two leaf-statement sets differ only where the structurer's selection pass chose differently:
each variable they disagree on is assigned on both sides, and a conditional expression is one of the
assignments (`if (c) x = a; else x = b;` against `x = c ? a : b;`). That pass reads the layout, so it is
no verdict of D2's; any other difference is."""
only = [line for line in set(natural) ^ set(mutant)]
targets = collections.defaultdict(lambda: [0, 0, False])
for line in only:
found = ASSIGNMENT.match(line)
if not found:
return False
side = targets[found[1]]
side[0 if line in natural else 1] += 1
side[2] |= " ? " in found[2]
return all(n and m and ternary for n, m, ternary in targets.values())
def verdict_moves(natural, mutant):
"""The parts of two fingerprints that differ, and how the pair is refused (`None` where both or
neither are). A function without an answer on a side (a crash) has no fingerprint. A difference in
the statements that only the selection pass explains is no move (`selection_only`)."""
if natural is None or mutant is None:
return ([], "refused only under the mutant" if natural is not None else
"refused only at natural" if mutant is not None else None)
if bool(natural["refusal"]) != bool(mutant["refusal"]):
return ([], "refused only under the mutant" if mutant["refusal"] else "refused only at natural")
moved = [part for part in natural if natural[part] != mutant[part]]
if "statements" in moved and selection_only(natural["statements"], mutant["statements"]):
moved.remove("statements")
return moved, None
def order_answers(args, order, binary):
"""`{address: (fingerprint or refusal, code hash)}` for every function of `binary` under `order`."""
sys.path.insert(0, str(ROOT / "tests" / "equiv"))
import r2s_batch
listing = r2s(binary, "aa; afl", args.r2s)
addresses = [int(line.split()[0], 16) for line in listing.splitlines()
if line.startswith("0x") and not line.split()[3].startswith(("sym.imp", "sym._"))]
env = {**os.environ, "R2SLEIGH_WRITE_ORDER": order}
report = r2s_batch.run_batch(args.r2s, binary, addresses, function_timeout=120, startup_timeout=600, env=env)
answers = {}
for answer in report.answers:
if answer.ok:
record = answer.record
answers[hex(answer.address)] = {"fingerprint": fingerprint(record),
"text": hashlib.sha256(record["code"].encode()).hexdigest()}
else:
answers[hex(answer.address)] = {"fingerprint": None, "text": None, "cause": (answer.cause or "")[:120]}
return answers
def orders(args):
"""Every function under each write order, against natural: its verdicts must not move, and the order
must (a knob that moves no text proves nothing)."""
out = Path(args.out)
out.mkdir(parents=True, exist_ok=True)
binaries = [Path(b) for b in args.binaries]
results = {}
for order in WRITE_ORDERS:
cached = out / f"{order}.json"
if args.reuse and cached.exists():
results[order] = json.loads(cached.read_text())
continue
with concurrent.futures.ThreadPoolExecutor(min(3, len(binaries))) as pool:
done = list(pool.map(lambda binary: order_answers(args, order, binary), binaries))
results[order] = {binary.name: answers for binary, answers in zip(binaries, done)}
cached.write_text(json.dumps(results[order]))
failed = False
total = sum(len(answers) for answers in results["natural"].values())
print(f"orders: {total} functions in {', '.join(b.name for b in binaries)}")
for order in WRITE_ORDERS[1:]:
differ, live, selected, parity, shown = 0, 0, 0, collections.Counter(), []
for name, natural in results["natural"].items():
for address, was in natural.items():
now = results[order][name].get(address, {"fingerprint": None, "text": None})
moved, refusal = verdict_moves(was["fingerprint"], now["fingerprint"])
live += was["text"] != now["text"]
if was["fingerprint"] and now["fingerprint"] and not moved:
selected += was["fingerprint"]["statements"] != now["fingerprint"]["statements"]
if refusal:
parity[refusal] += 1
if moved or refusal:
differ += 1
shown.append(f" {name} {address}: {', '.join(moved) or refusal}")
print(f"{order}: {differ} differ, {live} change text, {selected} differ only in a selection, {dict(parity)}")
print("\n".join(shown[:args.show]))
failed |= differ > 0 or live < args.min_live
if failed:
sys.exit(1)
def main():
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = parser.add_subparsers(dest="command", required=True)
@@ -452,6 +603,14 @@ def main():
p.add_argument("--out")
p.add_argument("binaries", nargs="+")
p.set_defaults(func=exponent)
p = sub.add_parser("orders")
p.add_argument("--r2s", required=True)
p.add_argument("--out", required=True)
p.add_argument("--reuse", action="store_true", help="read an order's answers from OUT where it has them")
p.add_argument("--min-live", type=int, default=50, help="the fewest functions each order must change the text of")
p.add_argument("--show", type=int, default=20)
p.add_argument("binaries", nargs="+")
p.set_defaults(func=orders)
p = sub.add_parser("heap")
p.add_argument("--top", type=int, default=30)
p.add_argument("profiles", nargs="+")
scripts/mutants_d68.py
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""D68's mutation target: each mutant of the dominance checks is applied to the tree, the tests that should kill it run,
and the file is restored. A mutant that survives is expected to, and says why (doc/adr-decompiler-rewrite.md, "D2's
spelling order", the D68 section).
mutants_d68.py [--check] [NAME...] run (or only --check that every edit applies once) the named mutants, else all
Run from the repository root with the files below unmodified. Exit 1 where a mutant lives that should die, or dies that
should live."""
import os
import subprocess
import sys
LANES = "cargo test -p r2ssa --lib --features fact_mutants function::tests::lanes"
R2DEC = "cargo test -p r2dec --lib --all-features"
ENGINE = "cargo test -p r2engine --test engine structure::"
# The tests that accept a residual where the lane was: all but the one that wants r2ssa's own refusal.
RESIDUAL = "cargo test -p r2engine --test engine -- a_call_argument_is_never a_lane_the_facts_name_on_the_other_arm_is_read_as"
SHARED = "crates/r2ssa/src/semantic/shared.rs"
STAGE = "crates/r2ssa/src/function/stage.rs"
INTEGRITY = "crates/r2ssa/src/integrity.rs"
VALUES = "crates/r2dec/src/render/values.rs"
SPELLED = "crates/r2dec/src/render/spelled.rs"
ORDER = "crates/r2dec/src/structure/order.rs"
UNFILTERED = (SHARED, "crate::fact_mutants::unsound_lanes()\n ||", "true\n ||")
UNCHECKED = (STAGE, "if !crate::fact_mutants::facts_unchecked() {", "if false {")
UNGUARDED = (VALUES, "variable_or_residual(owed == Some(0), scope)", "variable_or_residual(owed == Some(0), || true)")
def drop_read(old):
return (INTEGRITY, old, "Ok::<(), SsaIntegrityError>(())?;")
# (name, edits, tests that run in order, whether it must die, why it may live)
MUTANTS = [
("lane-unfiltered", [UNFILTERED], [LANES], True, ""),
("lane-unfiltered-facts-unchecked", [UNFILTERED, UNCHECKED], [LANES], True, ""),
("guard-alone-holds", [UNFILTERED, UNCHECKED], [RESIDUAL], False,
"with r2ssa's pick and check both gone the renderer's guard still makes the read a residual, which the engine tests accept"),
("guard-unwired", [UNGUARDED], [R2DEC, ENGINE], True, ""),
("block-never-entered", [(SPELLED, "values.enter(addr);", "let _ = addr;")], [ENGINE], True, ""),
("guard-allows-nondominating", [(VALUES, " } else if !scope() {", " } else if false && !scope() {")], [R2DEC], True, ""),
("guard-cause-swapped", [(VALUES, "Err(UnassignedCause::NotDominated)\n } else {", "Err(UnassignedCause::Unassigned)\n } else {")], [R2DEC], True, ""),
("validator-unwired", [UNCHECKED], [LANES, ENGINE], True, ""),
("defined-before-same-block-inclusive", [(INTEGRITY, "is_none_or(|defined| defined < ordinal)", "is_none_or(|defined| defined <= ordinal)")], [LANES], True, ""),
("defined-before-phi-late", [(INTEGRITY, "is_none_or(|defined| defined < ordinal)", "is_some_and(|defined| defined < ordinal)")], [LANES], True, ""),
("defined-before-entry-undefined", [(INTEGRITY, "let Some(def) = graph.def_inst(value) else {\n return true;", "let Some(def) = graph.def_inst(value) else {\n return false;")], [LANES], True, ""),
("defined-before-always", [(INTEGRITY, " function.domtree().strictly_dominates(def_block, block_addr)\n}", " true\n}")], [LANES], True, ""),
("defined-before-dominates-not-strictly", [(INTEGRITY, "function.domtree().strictly_dominates(def_block, block_addr)\n}", "function.domtree().dominates(def_block, block_addr)\n}")], [LANES], False,
"equivalent: the same block returned above, so the blocks differ and dominance is strict"),
("validator-call-argument", [drop_read('read("call argument", site, value)?;')], [LANES], True, ""),
("validator-carrier", [drop_read('read("call argument carrier", site, carrier)?;')], [LANES], True, ""),
("validator-call-target", [drop_read('read("call target", BoundarySite::Inst(call.at), call.target)?;')], [LANES], True, ""),
("validator-return-value", [drop_read('read("return value", site, value.value)?;')], [LANES], True, ""),
("validator-return-address", [drop_read('read("return address", site, address.value)?;')], [LANES], True, ""),
("validator-exit-stack-pointer", [drop_read('read("exit stack pointer", site, value)?;')], [LANES], True, ""),
("validator-branch-condition", [drop_read('read("branch condition", site, predicate.condition)?;')], [LANES], True, ""),
("validator-switch-selector", [drop_read('read(\n "switch selector",\n BoundarySite::BlockEnd(switch.block_addr),\n selector,\n )?;')], [LANES], True, ""),
("validator-block-end-is-start", [(INTEGRITY, "BoundarySite::BlockEnd(addr) => Some((addr, usize::MAX)),", "BoundarySite::BlockEnd(addr) => Some((addr, 0)),")], [LANES], False,
"no fact the fixtures hold reads a value its own block defines; the block-end reads the SSA validator already orders"),
("order-inlines-exits-not", [(ORDER, "pub(crate) const fn inlines_exits(self) -> bool {\n true", "pub(crate) const fn inlines_exits(self) -> bool {\n false")], [R2DEC], True, ""),
("order-exits-reversed", [(ORDER, "pub(crate) fn exits(self, _: &mut [u64]) {}", "pub(crate) fn exits(self, list: &mut [u64]) {\n list.reverse();\n }")], [R2DEC], True, ""),
("order-merges-reversed", [(ORDER, "pub(crate) fn merges(self, _: &mut [u64]) {}", "pub(crate) fn merges(self, list: &mut [u64]) {\n list.reverse();\n }")], [R2DEC], True, ""),
]
def run(name, edits, tests, must_die, why, check):
files = {path for path, _, _ in edits}
originals = {path: open(path).read() for path in files}
for path, old, new in edits:
if originals[path].count(old) != 1:
return f"{name}: the edit's old text occurs {originals[path].count(old)} times in {path}", False
if check:
return f"{name}: applies", True
text = dict(originals)
for path, old, new in edits:
text[path] = text[path].replace(old, new, 1)
try:
for path in files:
open(path, "w").write(text[path])
killer = None
for command in tests:
done = subprocess.run(command, shell=True, capture_output=True, text=True)
if done.returncode != 0:
killer = command
break
finally:
for path in files:
open(path, "w").write(originals[path])
died = killer is not None
verdict = f"killed by `{killer}`" if died else f"survived {len(tests)} test commands" + (f" ({why})" if why else "")
return f"{name}: {verdict}", died == must_die
def main():
args = sys.argv[1:]
check = "--check" in args
names = [a for a in args if not a.startswith("--")]
if not check and subprocess.run(["git", "diff", "--quiet", "--"] + sorted({p for _, e, *_ in MUTANTS for p, _, _ in e})).returncode:
sys.exit("a file a mutant edits has changes: commit or restore them first")
bad = 0
for name, edits, tests, must_die, why in MUTANTS:
if names and name not in names:
continue
line, ok = run(name, edits, tests, must_die, why, check)
print(("ok " if ok else "WRONG"), line, flush=True)
bad += not ok
sys.exit(1 if bad else 0)
if __name__ == "__main__":
os.chdir(os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
main()
scripts/structure-baseline.txt
+1 -1
View File
@@ -7,4 +7,4 @@ assigning_clones 5
needless_pass_by_ref_mut 0
clone_on_ref_ptr 2
mut_mut 0
long_comments 2568
long_comments 2566
scripts/test_census.py
+101
View File
@@ -95,5 +95,106 @@ class HeapSites(unittest.TestCase):
self.assertEqual(sites, {site: 320})
# F1 of crates/r2engine/tests/engine/structure.rs under two write orders: the loop's merge `rax_2` has a gap for
# its entry copy, and one order reads it as a variable in the exit after the loop.
EXIT_INLINED = """
/* r2dec proof: 3 constructs are marked below; 36 source obligations: 20 rendered, 8 elided, 0 refused, 8 residual; 11 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
r2sleigh_residual_void(1); /* r2dec gap: ValuesNotRendered at 0x1009 (render::values) */
rdi_1 = arg0;
for (; ; ) {
if (c) {
return (uint64_t)r2sleigh_residual_u64(2);
} else {
rax_2 = (uint64_t)((uint64_t)r2sleigh_residual_u64(3) + (uint64_t)1U);
continue;
}
}
L1:
{
return (uint64_t)((uint64_t)rax_2 + (uint64_t)5U);
}
}
"""
EXIT_AFTER = EXIT_INLINED.replace("r2sleigh_residual_u64(2)", "rax_2").replace("20 rendered", "22 rendered")
def record(code, **parts):
base = {"refused": None, "code": code, "proof": {"total": 36, "residual": 8}, "variables": [], "links": [],
"lines": [{"line": 5, "addrs": [4096]}],
"residuals": [{"site": 1, "type": "void", "cause": "gap", "gap": "ValuesNotRendered", "callee": None,
"reason": None, "line": 5}]}
return {**base, **parts}
class WriteOrders(unittest.TestCase):
def test_the_names_read_as_never_assigned_come_from_the_proof_line(self) -> None:
self.assertEqual(census.never_assigned(EXIT_INLINED), ["RAX_2"])
self.assertEqual(census.never_assigned(HIDDEN), [])
def test_statements_are_leaves_without_control_or_site_numbers(self) -> None:
leaves = census.leaf_statements(EXIT_INLINED)
self.assertIn("rdi_1 = arg0;", leaves)
self.assertIn("rax_2 = (uint64_t)((uint64_t)r2sleigh_residual_u64(N) + (uint64_t)1U);", leaves)
self.assertIn("r2sleigh_residual_void(N); /* r2dec gap: ValuesNotRendered at 0x1009 (render::values) */", leaves)
self.assertFalse([leaf for leaf in leaves if leaf.startswith(("for", "if", "continue", "L1"))])
self.assertEqual(census.leaf_statements("L17: ;\nL4:\n x = 1;\n"), ["x = 1;"])
def test_a_layout_that_moves_only_control_moves_no_verdict(self) -> None:
moved = EXIT_INLINED.replace("continue;", "goto L0;").replace("L1:", "L1: L2:")
self.assertEqual(census.verdict_moves(census.fingerprint(record(EXIT_INLINED)),
census.fingerprint(record(moved))), ([], None))
def test_a_read_that_turns_from_a_residual_into_a_variable_is_a_moved_verdict(self) -> None:
moves, refusal = census.verdict_moves(census.fingerprint(record(EXIT_INLINED)),
census.fingerprint(record(EXIT_AFTER)))
self.assertEqual((moves, refusal), (["statements"], None))
def test_the_ledger_and_the_residual_sites_are_verdicts(self) -> None:
natural = census.fingerprint(record(EXIT_INLINED))
columns = census.fingerprint(record(EXIT_INLINED, proof={"total": 36, "residual": 6}))
sites = census.fingerprint(record(EXIT_INLINED, lines=[{"line": 5, "addrs": [4100]}]))
self.assertEqual(census.verdict_moves(natural, columns), (["proof"], None))
self.assertEqual(census.verdict_moves(natural, sites), (["sites"], None))
def test_a_selection_the_structurer_chose_differently_is_no_move(self) -> None:
arms = EXIT_INLINED.replace("rdi_1 = arg0;", "x_1 = 1;\n x_1 = 2;")
selected = EXIT_INLINED.replace("rdi_1 = arg0;", "x_1 = c ? 1 : 2;")
natural, mutant = census.fingerprint(record(arms)), census.fingerprint(record(selected))
self.assertEqual(census.verdict_moves(natural, mutant), ([], None))
self.assertNotEqual(natural["statements"], mutant["statements"])
def test_a_statement_that_is_no_selection_is_a_move(self) -> None:
other = EXIT_INLINED.replace("rdi_1 = arg0;", "x_1 = 1;\n y_1 = 2;")
lone = EXIT_INLINED.replace("rdi_1 = arg0;", "x_1 = c ? 1 : 2;")
moves = census.verdict_moves(census.fingerprint(record(other)), census.fingerprint(record(lone)))
self.assertEqual(moves, (["statements"], None))
# A target assigned on one side only is no selection either.
gone = census.fingerprint(record(EXIT_INLINED.replace("rdi_1 = arg0;", "")))
self.assertEqual(census.verdict_moves(census.fingerprint(record(EXIT_INLINED)), gone)[0], ["statements"])
def test_a_function_refused_under_one_side_only_is_reported_with_the_side(self) -> None:
refused = census.fingerprint(record(REFUSED, refused={"kind": "rendering", "reason": "control certificate"}))
natural = census.fingerprint(record(EXIT_INLINED))
self.assertEqual(census.verdict_moves(natural, refused), ([], "refused only under the mutant"))
self.assertEqual(census.verdict_moves(refused, natural), ([], "refused only at natural"))
# No answer (a crash) is no fingerprint, and two of them agree.
self.assertEqual(census.verdict_moves(natural, None), ([], "refused only under the mutant"))
self.assertEqual(census.verdict_moves(None, natural), ([], "refused only at natural"))
self.assertEqual(census.verdict_moves(None, None), ([], None))
def test_a_verdict_that_moves_inside_a_refusal_is_a_moved_verdict(self) -> None:
# ls-toybox 0x5323c: refused the same way, with 214 residual obligations on one side and 216 on the other.
reason = {"kind": "rendering", "reason": "native effect obligations refused: 0 refused, 1 unaccounted"}
was = census.fingerprint(record(REFUSED, refused=reason, proof={"total": 1252, "residual": 214}))
now = census.fingerprint(record(REFUSED, refused=reason, proof={"total": 1252, "residual": 216}))
self.assertEqual(census.verdict_moves(was, now), (["proof"], None))
again = census.fingerprint(record(REFUSED, refused=reason, proof={"total": 1252, "residual": 214}))
self.assertEqual(census.verdict_moves(was, again), ([], None))
elsewhere = census.fingerprint(record(REFUSED, refused={**reason, "reason": reason["reason"] + " at 0x53748"},
proof={"total": 1252, "residual": 214}))
self.assertEqual(census.verdict_moves(was, elsewhere), (["refusal"], None))
if __name__ == "__main__":
unittest.main()
tests/db/census/branchy_gcc_x64_O0
+75 -113
View File
@@ -63,17 +63,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x403fe0U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x401014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x401014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -248,10 +247,13 @@ uint64_t gate_one(uint32_t arg0)
{
uint64_t rax_3;
/* r2dec proof: no individual construct is marked; 18 source obligations: 11 rendered, 7 elided, 0 refused; 2 statements rendered */
/* r2dec proof: no individual construct is marked; 18 source obligations: 11 rendered, 7 elided, 0 refused; 5 statements rendered */
{
rax_3 = (uint8_t)((uint32_t)arg0 != (uint32_t)0xbeefU) ? 0 : 1;
{
if ((uint8_t)((uint32_t)arg0 != (uint32_t)0xbeefU)) {
rax_3 = 0;
return (uint64_t)rax_3;
} else {
rax_3 = 1;
return (uint64_t)rax_3;
}
}
@@ -270,29 +272,16 @@ uint64_t gate_three(uint32_t arg0, uint32_t arg1, uint32_t arg2)
{
uint64_t rax_3;
/* r2dec proof: no individual construct is marked; 28 source obligations: 21 rendered, 7 elided, 0 refused; 8 statements rendered */
/* r2dec proof: no individual construct is marked; 28 source obligations: 21 rendered, 7 elided, 0 refused; 5 statements rendered */
{
if ((uint8_t)((uint32_t)arg0 != (uint32_t)7U)) {
if ((uint8_t)((uint32_t)arg0 == (uint32_t)7U) && (uint8_t)((uint32_t)arg1 == (uint32_t)14U) && (uint8_t)((uint32_t)arg2 == (uint32_t)21U)) {
rax_3 = 1;
return (uint64_t)rax_3;
} else {
{
rax_3 = 0;
return (uint64_t)rax_3;
}
} else {
if ((uint8_t)((uint32_t)arg1 != (uint32_t)14U)) {
{
rax_3 = 0;
}
} else {
if ((uint8_t)((uint32_t)arg2 != (uint32_t)21U)) {
{
rax_3 = 0;
}
} else {
rax_3 = 1;
}
}
}
{
return (uint64_t)rax_3;
}
}
}
@@ -310,10 +299,13 @@ uint64_t affine_probe(uint32_t arg0)
{
uint64_t rax_7;
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 2 statements rendered */
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 5 statements rendered */
{
rax_7 = (uint8_t)((uint32_t)((uint32_t)((uint32_t)arg0 + (uint32_t)arg0) + (uint32_t)arg0) != (uint32_t)45U) ? 0 : 1;
{
if ((uint8_t)((uint32_t)((uint32_t)((uint32_t)arg0 + (uint32_t)arg0) + (uint32_t)arg0) != (uint32_t)45U)) {
rax_7 = 0;
return (uint64_t)rax_7;
} else {
rax_7 = 1;
return (uint64_t)rax_7;
}
}
@@ -333,24 +325,17 @@ uint64_t paired_sum_diff(uint32_t arg0, uint32_t arg1)
uint32_t tmp_lane_4011f3_7_0_1;
uint64_t rax_7;
/* r2dec proof: no individual construct is marked; 25 source obligations: 18 rendered, 7 elided, 0 refused; 7 statements rendered; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 25 source obligations: 18 rendered, 7 elided, 0 refused; 6 statements rendered; the result's second register is open: no call read decides it */
{
tmp_lane_4011f3_7_0_1 = (uint32_t)((uint32_t)arg0 - (uint32_t)arg1);
if ((uint8_t)((uint32_t)((uint32_t)arg1 + (uint32_t)arg0) != (uint32_t)250U)) {
if ((uint8_t)((uint32_t)((uint32_t)arg1 + (uint32_t)arg0) == (uint32_t)250U) && (uint8_t)((uint32_t)tmp_lane_4011f3_7_0_1 == (uint32_t)30U)) {
rax_7 = 1;
return (uint64_t)rax_7;
} else {
{
rax_7 = 0;
return (uint64_t)rax_7;
}
} else {
if ((uint8_t)((uint32_t)tmp_lane_4011f3_7_0_1 != (uint32_t)30U)) {
{
rax_7 = 0;
}
} else {
rax_7 = 1;
}
}
{
return (uint64_t)rax_7;
}
}
}
@@ -369,24 +354,17 @@ uint64_t mask_probe(uint8_t arg0)
uint32_t tmp_lane_40121e_1_0_1;
uint64_t rax_7;
/* r2dec proof: no individual construct is marked; 27 source obligations: 20 rendered, 7 elided, 0 refused; 7 statements rendered */
/* r2dec proof: no individual construct is marked; 27 source obligations: 20 rendered, 7 elided, 0 refused; 6 statements rendered */
{
tmp_lane_40121e_1_0_1 = (uint32_t)((uint64_t)(uint8_t)arg0 >> 0U);
if ((uint8_t)((uint32_t)((uint32_t)tmp_lane_40121e_1_0_1 & (uint32_t)192U) != (uint32_t)64U)) {
if ((uint8_t)((uint32_t)((uint32_t)tmp_lane_40121e_1_0_1 & (uint32_t)192U) == (uint32_t)64U) && (uint8_t)((uint32_t)((uint32_t)tmp_lane_40121e_1_0_1 & (uint32_t)7U) == (uint32_t)5U)) {
rax_7 = 1;
return (uint64_t)rax_7;
} else {
{
rax_7 = 0;
return (uint64_t)rax_7;
}
} else {
if ((uint8_t)((uint32_t)((uint32_t)tmp_lane_40121e_1_0_1 & (uint32_t)7U) != (uint32_t)5U)) {
{
rax_7 = 0;
}
} else {
rax_7 = 1;
}
}
{
return (uint64_t)rax_7;
}
}
}
@@ -405,24 +383,17 @@ uint64_t bounded_fetch(uint32_t* arg0, uint32_t arg1, uint32_t arg2)
uint32_t tmp_401282_0_1;
uint64_t rax_8;
/* r2dec proof: no individual construct is marked; 25 source obligations: 18 rendered, 7 elided, 0 refused; 7 statements rendered */
/* r2dec proof: no individual construct is marked; 25 source obligations: 18 rendered, 7 elided, 0 refused; 6 statements rendered */
{
if ((uint8_t)((int32_t)arg1 < (int32_t)0U)) {
if ((uint8_t)((int32_t)arg1 >= (int32_t)0U) && (uint8_t)((int32_t)arg1 < (int32_t)arg2)) {
tmp_401282_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)arg1 * 4U));
rax_8 = (uint64_t)(uint32_t)tmp_401282_0_1;
return (uint64_t)rax_8;
} else {
{
rax_8 = (uint64_t)0xffffffffU;
return (uint64_t)rax_8;
}
} else {
if ((uint8_t)((int32_t)arg1 < (int32_t)arg2)) {
tmp_401282_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)arg1 * 4U));
rax_8 = (uint64_t)(uint32_t)tmp_401282_0_1;
} else {
{
rax_8 = (uint64_t)0xffffffffU;
}
}
}
{
return (uint64_t)rax_8;
}
}
}
@@ -495,13 +466,16 @@ uint64_t opaque_self_xor(uint32_t arg0)
uint32_t tmp_lane_4012f8_0_0_1;
uint64_t rax_5;
/* r2dec proof: no individual construct is marked; 22 source obligations: 15 rendered, 7 elided, 0 refused; 5 statements rendered; 1 data object type refused; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 22 source obligations: 15 rendered, 7 elided, 0 refused; 8 statements rendered; 1 data object type refused; the result's second register is open: no call read decides it */
{
r2sleigh_store_u32((void*)&g_opaque, (uint32_t)arg0);
tmp_lane_4012f2_0_0_1 = r2sleigh_load_u32((void*)&g_opaque);
tmp_lane_4012f8_0_0_1 = r2sleigh_load_u32((void*)&g_opaque);
rax_5 = (uint8_t)((uint32_t)tmp_lane_4012f2_0_0_1 == (uint32_t)tmp_lane_4012f8_0_0_1) ? 0 : 1;
{
if ((uint8_t)((uint32_t)tmp_lane_4012f2_0_0_1 == (uint32_t)tmp_lane_4012f8_0_0_1)) {
rax_5 = 0;
return (uint64_t)rax_5;
} else {
rax_5 = 1;
return (uint64_t)rax_5;
}
}
@@ -540,11 +514,14 @@ uint64_t pure_zero_guard(void)
uint64_t rax_2;
uint64_t rax_5;
/* r2dec proof: no individual construct is marked; 24 source obligations: 14 rendered, 10 elided, 0 refused; 3 statements rendered */
/* r2dec proof: no individual construct is marked; 24 source obligations: 14 rendered, 10 elided, 0 refused; 6 statements rendered */
{
rax_2 = zero_of();
rax_5 = (uint8_t)((uint32_t)((uint64_t)rax_2 >> 0U) == (uint32_t)0U) ? 0 : 1;
{
if ((uint8_t)((uint32_t)((uint64_t)rax_2 >> 0U) == (uint32_t)0U)) {
rax_5 = 0;
return (uint64_t)rax_5;
} else {
rax_5 = 1;
return (uint64_t)rax_5;
}
}
@@ -566,12 +543,15 @@ uint64_t global_eq_guard(uint32_t arg0)
uint32_t tmp_lane_401361_0_0_1;
uint64_t rax_5;
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 4 statements rendered; 1 data object type refused */
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 7 statements rendered; 1 data object type refused */
{
r2sleigh_store_u32((void*)&g_opaque, (uint32_t)arg0);
tmp_lane_401361_0_0_1 = r2sleigh_load_u32((void*)&g_opaque);
rax_5 = (uint8_t)((uint32_t)tmp_lane_401361_0_0_1 != (uint32_t)11U) ? 0 : 1;
{
if ((uint8_t)((uint32_t)tmp_lane_401361_0_0_1 != (uint32_t)11U)) {
rax_5 = 0;
return (uint64_t)rax_5;
} else {
rax_5 = 1;
return (uint64_t)rax_5;
}
}
@@ -821,23 +801,16 @@ uint64_t and_chain(uint32_t arg0, uint32_t arg1)
{
uint64_t rax_3;
/* r2dec proof: no individual construct is marked; 19 source obligations: 12 rendered, 7 elided, 0 refused; 6 statements rendered */
/* r2dec proof: no individual construct is marked; 19 source obligations: 12 rendered, 7 elided, 0 refused; 5 statements rendered */
{
if ((uint8_t)((uint32_t)arg0 == (uint32_t)0U)) {
if ((uint8_t)((uint32_t)arg0 != (uint32_t)0U) && (uint8_t)((uint32_t)arg1 != (uint32_t)0U)) {
rax_3 = 1;
return (uint64_t)rax_3;
} else {
{
rax_3 = 0;
return (uint64_t)rax_3;
}
} else {
if ((uint8_t)((uint32_t)arg1 == (uint32_t)0U)) {
{
rax_3 = 0;
}
} else {
rax_3 = 1;
}
}
{
return (uint64_t)rax_3;
}
}
}
@@ -859,17 +832,16 @@ uint64_t inverted_guard(uint32_t arg0)
uint64_t rax_5;
uint64_t rax_7;
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 7 statements rendered; 2 data object types refused */
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 8 statements rendered; 2 data object types refused */
{
if ((uint8_t)((int32_t)arg0 <= (int32_t)9U)) {
rax_7 = (uint64_t)0xffffffffU;
return (uint64_t)rax_7;
} else {
r2sleigh_store_u64((void*)&g_note, (uint64_t)(int32_t)((uint32_t)arg0 + (uint32_t)2U));
r2sleigh_store_u32((void*)&g_sink, (uint32_t)arg0);
rax_5 = r2sleigh_load_u64((void*)&g_note);
rax_7 = rax_5;
}
{
return (uint64_t)rax_7;
}
}
@@ -888,14 +860,11 @@ uint64_t and_chain_effect(uint32_t arg0, uint32_t arg1)
{
extern char g_sink[];
/* r2dec proof: 1 construct is marked below; 18 source obligations: 8 rendered, 8 elided, 0 refused, 2 residual; 4 statements rendered; 1 data object type refused */
/* r2dec proof: 1 construct is marked below; 18 source obligations: 8 rendered, 8 elided, 0 refused, 2 residual; 3 statements rendered; 1 data object type refused */
{
if ((uint8_t)((uint32_t)arg0 == (uint32_t)0U)) {
if ((uint8_t)((uint32_t)arg0 != (uint32_t)0U) && (uint8_t)((uint32_t)arg1 != (uint32_t)0U)) {
r2sleigh_store_u32((void*)&g_sink, (uint32_t)((uint32_t)arg1 + (uint32_t)arg0));
} else {
if ((uint8_t)((uint32_t)arg1 == (uint32_t)0U)) {
} else {
r2sleigh_store_u32((void*)&g_sink, (uint32_t)((uint32_t)arg1 + (uint32_t)arg0));
}
}
{
return r2sleigh_residual_u64(1);
@@ -1043,33 +1012,26 @@ uint64_t bool_relay(uint32_t arg0, uint32_t arg1)
uint32_t tmp_lane_401731_0_1_1;
uint64_t rax_8;
/* r2dec proof: no individual construct is marked; 32 source obligations: 25 rendered, 7 elided, 0 refused; 13 statements rendered */
/* r2dec proof: no individual construct is marked; 32 source obligations: 25 rendered, 7 elided, 0 refused; 10 statements rendered */
{
tmp_lane_401724_0_1_1 = (uint32_t)(uint8_t)((int32_t)0U < (int32_t)arg0);
tmp_lane_401731_0_1_1 = (uint32_t)(uint8_t)((int32_t)0U < (int32_t)arg1);
if ((uint8_t)((uint32_t)tmp_lane_401724_0_1_1 == (uint32_t)0U)) {
if ((uint8_t)((uint32_t)tmp_lane_401724_0_1_1 != (uint32_t)0U) && (uint8_t)((uint32_t)tmp_lane_401731_0_1_1 != (uint32_t)0U)) {
rax_8 = 2;
return (uint64_t)rax_8;
} else {
if ((uint8_t)((uint32_t)tmp_lane_401731_0_1_1 == (uint32_t)0U)) {
} else {
rax_8 = 2;
goto L2;
}
}
{
if ((uint8_t)((uint32_t)tmp_lane_401724_0_1_1 != (uint32_t)0U)) {
if ((uint8_t)((uint32_t)tmp_lane_401724_0_1_1 == (uint32_t)0U) && (uint8_t)((uint32_t)tmp_lane_401731_0_1_1 == (uint32_t)0U)) {
rax_8 = 0;
return (uint64_t)rax_8;
} else {
if ((uint8_t)((uint32_t)tmp_lane_401731_0_1_1 == (uint32_t)0U)) {
rax_8 = 0;
goto L2;
} else {
}
}
{
rax_8 = 1;
}
}
{
L2: ;
return (uint64_t)rax_8;
}
}
tests/db/census/code_pointer_table_O0
+6 -10
View File
@@ -75,23 +75,19 @@ uint64_t _table_dispatch(uint64_t arg0, uint64_t arg1)
uint32_t stack_m76_1;
_Alignas(16) uint8_t frame[64];
/* r2dec proof: 3 constructs are marked below; 59 source obligations: 21 rendered, 11 elided, 0 refused, 27 residual; 10 statements rendered */
/* r2dec proof: 3 constructs are marked below; 59 source obligations: 21 rendered, 11 elided, 0 refused, 27 residual; 8 statements rendered */
{
r2sleigh_store_u64((void*)(uint64_t)(frame + 40U), (uint64_t)arg0);
r2sleigh_store_u64((void*)(uint64_t)(frame + 32U), (uint64_t)arg1);
r2sleigh_store_u128((void*)(uint64_t)(frame + 0U), (__uint128_t)r2sleigh_load_u128((void*)(uint64_t)0x100004000U));
r2sleigh_store_u64((void*)((uint64_t)(frame + 0U) + (uint64_t)16U), (uint64_t)r2sleigh_load_u64((void*)(uint64_t)0x100004010U));
stack_m76_1 = 0;
for (; ; ) {
if ((uint8_t)((uint32_t)3U <= (uint32_t)stack_m76_1)) {
break;
} else {
while ((uint8_t)((uint32_t)3U > (uint32_t)stack_m76_1)) {
{
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x10000046c:36 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: EffectNotRendered at 0x10000046c:69 covering 1 op (render::values) */
{
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x10000046c:36 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: EffectNotRendered at 0x10000046c:69 covering 1 op (render::values) */
{
stack_m76_1 = (uint32_t)((uint32_t)stack_m76_1 + (uint32_t)1U);
}
stack_m76_1 = (uint32_t)((uint32_t)stack_m76_1 + (uint32_t)1U);
}
}
}
tests/db/census/float_returns_zig_aarch64_O2g
+7 -12
View File
@@ -197,7 +197,7 @@ double loop_sum(const double* v, int32_t n)
struct r2sleigh_bits_256 z0_27;
struct r2sleigh_bits_256 z0_28;
/* r2dec proof: no individual construct is marked; 91 source obligations: 91 rendered, 0 elided, 0 refused; 60 statements rendered */
/* r2dec proof: no individual construct is marked; 91 source obligations: 91 rendered, 0 elided, 0 refused; 57 statements rendered */
{
if ((uint8_t)((int32_t)n < (int32_t)1U)) {
return (double)r2sleigh_float_from_bits_64((uint64_t)0U);
@@ -212,7 +212,7 @@ double loop_sum(const double* v, int32_t n)
x10_2 = (uint64_t)((uint64_t)v + (uint64_t)16U);
x11_2 = (uint64_t)((uint64_t)x8_1 & (uint64_t)0x7ffffffcU);
z0_5 = z0_4;
for (; ; ) {
do {
tmp_1010598_0_1 = (uint64_t)((uint64_t)x10_2 - (uint64_t)16U);
tmp_lane_1010598_1_0_1 = r2sleigh_load_u128((void*)tmp_1010598_0_1);
tmp_1010598_8_1 = (uint64_t)((uint64_t)tmp_1010598_0_1 + (uint64_t)16U);
@@ -228,18 +228,14 @@ double loop_sum(const double* v, int32_t n)
z0_19 = r2sleigh_bits_insert_256_64(z0_18, (uint64_t)0U, 64U);
z0_20 = r2sleigh_bits_insert_256_64(z0_19, (uint64_t)0U, 128U);
z0_21 = r2sleigh_bits_insert_256_64(z0_20, (uint64_t)0U, 192U);
if ((uint8_t)((uint64_t)tmp_101059c_8_1 != (uint64_t)0U)) {
x10_2 = (uint64_t)((uint64_t)x10_2 + (uint64_t)32U);
x11_2 = tmp_101059c_8_1;
z0_5 = z0_21;
} else {
break;
}
}
x10_2 = (uint64_t)((uint64_t)x10_2 + (uint64_t)32U);
x11_2 = tmp_101059c_8_1;
z0_5 = z0_21;
} while ((uint8_t)((uint64_t)tmp_101059c_8_1 != (uint64_t)0U));
{
if ((uint8_t)((uint64_t)x9_2 == (uint64_t)x8_1)) {
z0_28 = z0_21;
goto L2;
return (double)r2sleigh_float_from_bits_64(r2sleigh_bits_extract_256_64(z0_28, 0U));
} else {
x9_3 = x9_2;
z0_22 = z0_21;
@@ -273,7 +269,6 @@ double loop_sum(const double* v, int32_t n)
}
}
{
L2: ;
return (double)r2sleigh_float_from_bits_64(r2sleigh_bits_extract_256_64(z0_28, 0U));
}
}
tests/db/census/float_returns_zig_x86_64_O2g
+6 -11
View File
@@ -137,7 +137,7 @@ double loop_sum(const double* v, int32_t n)
__uint128_t xmm0_17;
__uint128_t xmm0_18;
/* r2dec proof: no individual construct is marked; 138 source obligations: 126 rendered, 12 elided, 0 refused; 65 statements rendered */
/* r2dec proof: no individual construct is marked; 138 source obligations: 126 rendered, 12 elided, 0 refused; 62 statements rendered */
{
tmp_1001574_0_1 = (uint32_t)n;
if ((uint8_t)((int32_t)tmp_1001574_0_1 <= (int32_t)0U)) {
@@ -150,7 +150,7 @@ double loop_sum(const double* v, int32_t n)
xmm0_3 = (__uint128_t)(((__uint128_t)0U & (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0U)) | ((__uint128_t)(uint64_t)0U & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 0U);
rcx_3 = 0;
xmm0_5 = (__uint128_t)(((__uint128_t)xmm0_3 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)(uint64_t)0U & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 64U);
for (; ; ) {
do {
tmp_10015a0_10_1 = r2sleigh_load_u64((void*)((uint64_t)v + (uint64_t)rcx_3 * 8U));
tmp_lane_10015a0_3_0_1 = (uint64_t)((__uint128_t)xmm0_5 >> 0U);
tmp_lane_10015a0_3_1_1 = r2sleigh_float_from_bits_64(tmp_lane_10015a0_3_0_1) + r2sleigh_float_from_bits_64(tmp_10015a0_10_1);
@@ -177,17 +177,13 @@ double loop_sum(const double* v, int32_t n)
tmp_lane_10015c9_4_1_1 = tmp_lane_10015c3_4_1_1 + r2sleigh_float_from_bits_64(tmp_10015c9_18_1);
xmm0_13 = (__uint128_t)(((__uint128_t)xmm0_12 & (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0U)) | ((__uint128_t)r2sleigh_float_to_bits_64(tmp_lane_10015c9_4_1_1) & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 0U);
rcx_4 = (uint64_t)((uint64_t)rcx_3 + (uint64_t)8U);
if ((uint8_t)((uint64_t)rdx_2 != (uint64_t)rcx_4)) {
rcx_3 = rcx_4;
xmm0_5 = xmm0_13;
} else {
break;
}
}
rcx_3 = rcx_4;
xmm0_5 = xmm0_13;
} while ((uint8_t)((uint64_t)rdx_2 != (uint64_t)rcx_4));
{
if ((uint8_t)((uint64_t)rax_2 == (uint64_t)0U)) {
xmm0_17 = xmm0_13;
goto L2;
return (double)r2sleigh_float_from_bits_64((uint64_t)xmm0_17);
} else {
rcx_5 = rcx_4;
xmm0_14 = xmm0_13;
@@ -218,7 +214,6 @@ double loop_sum(const double* v, int32_t n)
}
}
{
L2: ;
return (double)r2sleigh_float_from_bits_64((uint64_t)xmm0_17);
}
}
tests/db/census/frame_pointer_locals_clang_O0g
+2 -3
View File
@@ -29,17 +29,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x3fd0U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x1014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x1014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
tests/db/census/hashes_gcc_x64_O2
+77 -132
View File
@@ -43,17 +43,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x403fe0U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x401014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x401014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -296,7 +295,7 @@ uint64_t fnv1a32(uint64_t arg0, uint64_t arg1)
uint64_t rdi_2;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return (uint64_t)0x811c9dc5U;
@@ -304,17 +303,13 @@ uint64_t fnv1a32(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0x811c9dc5U;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401348_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rax_4 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)(uint8_t)tmp_401348_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U)) * (uint64_t)0x1000193U) >> 0U);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = rax_4;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = rax_4;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint64_t)rax_4;
}
@@ -386,7 +381,7 @@ uint64_t djb2(uint64_t arg0, uint64_t arg1)
uint8_t tmp_4013d3_8_1;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 41 source obligations: 33 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 41 source obligations: 33 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0x1505;
@@ -394,17 +389,13 @@ uint64_t djb2(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = 0x1505;
rdi_1 = arg0;
for (; ; ) {
do {
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_4013d3_8_1 = r2sleigh_load_u8((void*)(uint64_t)((uint64_t)rdi_2 - (uint64_t)1U));
rax_4 = (uint64_t)(uint32_t)((uint32_t)(uint8_t)tmp_4013d3_8_1 + (uint32_t)((uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) * (uint32_t)32U) + (uint32_t)((uint64_t)rax_2 >> 0U)));
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1)) {
rax_2 = rax_4;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = rax_4;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1));
{
return (uint64_t)rax_4;
}
@@ -430,7 +421,7 @@ uint32_t sdbm(uint64_t arg0, uint64_t arg1)
uint64_t rdi_2;
uint32_t tmp_lane_40140d_6_0_1;
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -438,17 +429,13 @@ uint32_t sdbm(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = 0;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401406_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_lane_40140d_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rax_2 >> 0U) * (uint64_t)0x1003fU) >> 0U) + (uint32_t)(uint8_t)tmp_401406_0_1);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_40140d_6_0_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_40140d_6_0_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint32_t)tmp_lane_40140d_6_0_1;
}
@@ -478,7 +465,7 @@ uint64_t adler32(uint64_t arg0, uint64_t arg1)
uint64_t rax_4;
uint32_t tmp_lane_401478_5_0_1;
/* r2dec proof: no individual construct is marked; 66 source obligations: 58 rendered, 8 elided, 0 refused; 19 statements rendered */
/* r2dec proof: no individual construct is marked; 66 source obligations: 58 rendered, 8 elided, 0 refused; 17 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 1;
@@ -487,21 +474,17 @@ uint64_t adler32(uint64_t arg0, uint64_t arg1)
rax_2 = 0;
rdi_1 = arg0;
rdx_2 = 1;
for (; ; ) {
do {
tmp_401440_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rdx_3 = (uint64_t)(uint32_t)((uint32_t)(uint8_t)tmp_401440_0_1 + (uint32_t)((uint64_t)rdx_2 >> 0U));
tmp_lane_40145b_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rdx_3 >> 0U) - (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)((uint64_t)(uint32_t)47U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)((uint64_t)rdx_3 * (uint64_t)0x80078071U) >> (uint64_t)(uint32_t)47U)) >> 0U) * (uint64_t)0xfff1U) >> 0U));
rax_4 = (uint64_t)(uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) + (uint32_t)tmp_lane_40145b_5_0_1);
tmp_lane_401478_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rax_4 >> 0U) - (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)((uint64_t)(uint32_t)47U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)((uint64_t)rax_4 * (uint64_t)0x80078071U) >> (uint64_t)(uint32_t)47U)) >> 0U) * (uint64_t)0xfff1U) >> 0U));
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401478_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_40145b_5_0_1;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401478_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_40145b_5_0_1;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint64_t)(uint32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401478_5_0_1 << (uint32_t)16U)) | (uint32_t)tmp_lane_40145b_5_0_1);
}
@@ -533,7 +516,7 @@ uint64_t fletcher32(uint64_t arg0, uint64_t arg1)
uint64_t rax_8;
uint32_t tmp_lane_401517_5_0_1;
/* r2dec proof: no individual construct is marked; 73 source obligations: 66 rendered, 7 elided, 0 refused; 21 statements rendered */
/* r2dec proof: no individual construct is marked; 73 source obligations: 66 rendered, 7 elided, 0 refused; 19 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return (uint64_t)0xffffffffU;
@@ -542,7 +525,7 @@ uint64_t fletcher32(uint64_t arg0, uint64_t arg1)
rax_2 = 0xffff;
rdi_1 = arg0;
rdx_2 = 0xffff;
for (; ; ) {
do {
tmp_4014c0_0_1 = r2sleigh_load_u16((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)2U);
r8_1 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)rdx_2 + (uint64_t)(uint16_t)tmp_4014c0_0_1) >> 0U);
@@ -551,14 +534,10 @@ uint64_t fletcher32(uint64_t arg0, uint64_t arg1)
rcx_4 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(uint32_t)tmp_lane_4014ed_5_0_1 + (uint64_t)rax_2) >> 0U);
rax_8 = (uint64_t)(uint32_t)47U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)((uint64_t)rcx_4 + (uint64_t)((uint64_t)((uint64_t)rcx_4 + (uint64_t)((uint64_t)rcx_4 * (uint64_t)0x10000U)) * (uint64_t)0x8000U)) >> (uint64_t)(uint32_t)47U);
tmp_lane_401517_5_0_1 = (uint32_t)((uint32_t)((uint32_t)((uint64_t)rax_8 >> 0U) + (uint32_t)((uint64_t)rcx_4 >> 0U)) - (uint32_t)((uint32_t)((uint64_t)rax_8 >> 0U) * (uint32_t)0x10000U));
if ((uint8_t)((uint64_t)tmp_4014a9_8_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401517_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_4014ed_5_0_1;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401517_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_4014ed_5_0_1;
} while ((uint8_t)((uint64_t)tmp_4014a9_8_1 != (uint64_t)rdi_2));
{
return (uint64_t)(uint32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401517_5_0_1 << (uint32_t)16U)) | (uint32_t)tmp_lane_4014ed_5_0_1);
}
@@ -588,7 +567,7 @@ uint32_t crc32_bitwise(uint64_t arg0, uint64_t arg1)
uint32_t tmp_lane_401568_5_0_1;
uint64_t rdi_2;
/* r2dec proof: no individual construct is marked; 60 source obligations: 52 rendered, 8 elided, 0 refused; 23 statements rendered */
/* r2dec proof: no individual construct is marked; 60 source obligations: 52 rendered, 8 elided, 0 refused; 19 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -596,31 +575,21 @@ uint32_t crc32_bitwise(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0xffffffffU;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401548_0_1 = r2sleigh_load_u8((void*)rdi_1);
rax_4 = (uint64_t)(uint32_t)((uint32_t)(uint8_t)tmp_401548_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U));
rdx_3 = 8;
for (; ; ) {
do {
tmp_lane_401566_2_2_1 = (uint32_t)((uint32_t)((uint32_t)1U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)((uint64_t)rax_4 >> 0U) >> (uint32_t)1U)) ^ (uint32_t)((uint32_t)((uint32_t)0U - (uint32_t)((uint32_t)((uint64_t)rax_4 >> 0U) & (uint32_t)1U)) & (uint32_t)0xedb88320U));
rax_8 = (uint64_t)(uint32_t)tmp_lane_401566_2_2_1;
tmp_lane_401568_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rdx_3 >> 0U) - (uint32_t)1U);
if ((uint8_t)((uint32_t)tmp_lane_401568_5_0_1 != (uint32_t)0U)) {
rax_4 = rax_8;
rdx_3 = (uint64_t)(uint32_t)tmp_lane_401568_5_0_1;
} else {
break;
}
}
{
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1)) {
rax_2 = rax_8;
rdi_1 = rdi_2;
} else {
break;
}
}
}
rax_4 = rax_8;
rdx_3 = (uint64_t)(uint32_t)tmp_lane_401568_5_0_1;
} while ((uint8_t)((uint32_t)tmp_lane_401568_5_0_1 != (uint32_t)0U));
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rax_2 = rax_8;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1));
{
return (uint32_t)~(uint32_t)tmp_lane_401566_2_2_1;
}
@@ -814,13 +783,13 @@ uint64_t crc32_init(void)
uint32_t tmp_lane_401618_5_0_1;
uint64_t rdx_3;
/* r2dec proof: 1 construct is marked below; 208 source obligations: 201 rendered, 5 elided, 0 refused, 2 residual; 186 statements rendered */
/* r2dec proof: 1 construct is marked below; 208 source obligations: 201 rendered, 5 elided, 0 refused, 2 residual; 182 statements rendered */
{
xmm3_1 = r2sleigh_load_u128((void*)(uint64_t)0x4022b0U);
xmm5_1 = r2sleigh_load_u128((void*)(uint64_t)0x4022c0U);
rdx_2 = 0x404040;
xmm4_2 = r2sleigh_load_u128((void*)(uint64_t)0x4022a0U);
for (; ; ) {
do {
tmp_lane_4015b9_0_0_1 = (uint32_t)((__uint128_t)xmm4_2 >> 0U);
tmp_lane_4015b9_0_1_1 = (uint32_t)((__uint128_t)xmm5_1 >> 0U);
tmp_lane_4015b9_0_2_1 = (uint32_t)((uint32_t)tmp_lane_4015b9_0_0_1 + (uint32_t)tmp_lane_4015b9_0_1_1);
@@ -839,7 +808,7 @@ uint64_t crc32_init(void)
xmm4_6 = (__uint128_t)(((__uint128_t)xmm4_5 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_4015b9_3_2_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U);
rax_2 = 8;
xmm2_2 = xmm4_2;
for (; ; ) {
do {
tmp_lane_4015c4_0_0_1 = (uint32_t)((__uint128_t)xmm2_2 >> 0U);
tmp_lane_4015c4_0_1_1 = (uint32_t)1U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4015c4_0_0_1 >> (uint32_t)1U);
xmm2_3 = (__uint128_t)(((__uint128_t)xmm2_2 & (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0xffffffff00000000U)) | ((__uint128_t)tmp_lane_4015c4_0_1_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 0U);
@@ -991,24 +960,14 @@ uint64_t crc32_init(void)
xmm0_42 = (__uint128_t)(((__uint128_t)xmm0_41 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_40160f_3_1_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U);
xmm2_7 = (__uint128_t)((__uint128_t)xmm2_6 ^ (__uint128_t)xmm0_42);
tmp_lane_401618_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) - (uint32_t)1U);
if ((uint8_t)((uint32_t)tmp_lane_401618_5_0_1 != (uint32_t)0U)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401618_5_0_1;
xmm2_2 = xmm2_7;
} else {
break;
}
}
{
r2sleigh_store_u128((void*)rdx_2, (__uint128_t)xmm2_7);
rdx_3 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)16U);
if ((uint8_t)((uint64_t)rdx_3 != (uint64_t)0x404440U)) {
rdx_2 = rdx_3;
xmm4_2 = xmm4_6;
} else {
break;
}
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401618_5_0_1;
xmm2_2 = xmm2_7;
} while ((uint8_t)((uint32_t)tmp_lane_401618_5_0_1 != (uint32_t)0U));
r2sleigh_store_u128((void*)rdx_2, (__uint128_t)xmm2_7);
rdx_3 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)16U);
rdx_2 = rdx_3;
xmm4_2 = xmm4_6;
} while ((uint8_t)((uint64_t)rdx_3 != (uint64_t)0x404440U));
{
return r2sleigh_residual_u64(1);
}
@@ -1036,7 +995,7 @@ uint32_t crc32_table(uint64_t arg0, uint64_t arg1)
uint32_t tmp_401657_10_1;
uint32_t tmp_lane_401657_5_1_1;
/* r2dec proof: no individual construct is marked; 48 source obligations: 40 rendered, 8 elided, 0 refused; 15 statements rendered; 1 data object type refused */
/* r2dec proof: no individual construct is marked; 48 source obligations: 40 rendered, 8 elided, 0 refused; 13 statements rendered; 1 data object type refused */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -1044,18 +1003,14 @@ uint32_t crc32_table(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0xffffffffU;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401648_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_401657_10_1 = r2sleigh_load_u32((void*)((uint64_t)((uint64_t)(uint8_t)((uint32_t)((uint32_t)(uint8_t)tmp_401648_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U)) >> 0U) * (uint64_t)4U) + (uint64_t)&crc_tab));
tmp_lane_401657_5_1_1 = (uint32_t)((uint32_t)((uint32_t)8U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) >> (uint32_t)8U)) ^ (uint32_t)tmp_401657_10_1);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401657_5_1_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401657_5_1_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint32_t)~(uint32_t)tmp_lane_401657_5_1_1;
}
@@ -1099,7 +1054,7 @@ uint32_t murmur3_32(uint8_t* arg0, uint64_t arg1, uint32_t arg2)
uint32_t tmp_lane_401708_3_0_1;
uint32_t tmp_lane_401715_3_0_1;
/* r2dec proof: no individual construct is marked; 121 source obligations: 116 rendered, 5 elided, 0 refused; 42 statements rendered; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 121 source obligations: 116 rendered, 5 elided, 0 refused; 44 statements rendered; the result's second register is open: no call read decides it */
{
rcx_1 = (uint64_t)(uint32_t)arg2;
rax_2 = (uint64_t)(uint32_t)2U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)arg1 >> (uint64_t)(uint32_t)2U);
@@ -1145,7 +1100,10 @@ uint32_t murmur3_32(uint8_t* arg0, uint64_t arg1, uint32_t arg2)
} else {
if ((uint8_t)((uint64_t)rax_11 != (uint64_t)1U)) {
rcx_6 = rcx_4;
goto L2;
tmp_lane_4016fe_2_2_1 = (uint32_t)((uint32_t)((uint64_t)rcx_6 >> 0U) ^ (uint32_t)((uint64_t)arg1 >> 0U));
tmp_lane_401708_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016fe_2_2_1 >> (uint32_t)16U)) ^ (uint32_t)tmp_lane_4016fe_2_2_1) * (uint64_t)0xffffffff85ebca6bU) >> 0U);
tmp_lane_401715_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)13U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401708_3_0_1 >> (uint32_t)13U)) ^ (uint32_t)tmp_lane_401708_3_0_1) * (uint64_t)0xffffffffc2b2ae35U) >> 0U);
return (uint32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401715_3_0_1 >> (uint32_t)16U)) ^ (uint32_t)tmp_lane_401715_3_0_1);
} else {
rdx_6 = 0;
}
@@ -1157,7 +1115,6 @@ uint32_t murmur3_32(uint8_t* arg0, uint64_t arg1, uint32_t arg2)
rcx_6 = (uint64_t)(uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)15U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016ed_3_0_1 << (uint32_t)15U)) | (uint32_t)((uint32_t)17U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016ed_3_0_1 >> (uint32_t)17U))) * (uint64_t)0x1b873593U) >> 0U) ^ (uint32_t)((uint64_t)rcx_4 >> 0U));
}
{
L2: ;
tmp_lane_4016fe_2_2_1 = (uint32_t)((uint32_t)((uint64_t)rcx_6 >> 0U) ^ (uint32_t)((uint64_t)arg1 >> 0U));
tmp_lane_401708_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016fe_2_2_1 >> (uint32_t)16U)) ^ (uint32_t)tmp_lane_4016fe_2_2_1) * (uint64_t)0xffffffff85ebca6bU) >> 0U);
tmp_lane_401715_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)13U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401708_3_0_1 >> (uint32_t)13U)) ^ (uint32_t)tmp_lane_401708_3_0_1) * (uint64_t)0xffffffffc2b2ae35U) >> 0U);
@@ -1221,7 +1178,7 @@ uint32_t xxhash32(uint64_t arg0, uint64_t arg1, uint32_t arg2)
uint32_t tmp_lane_401884_3_0_1;
uint32_t tmp_lane_401891_3_0_1;
/* r2dec proof: no individual construct is marked; 227 source obligations: 220 rendered, 7 elided, 0 refused; 69 statements rendered */
/* r2dec proof: no individual construct is marked; 227 source obligations: 220 rendered, 7 elided, 0 refused; 65 statements rendered */
{
rdx_1 = (uint64_t)(uint32_t)arg2;
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
@@ -1235,7 +1192,7 @@ uint32_t xxhash32(uint64_t arg0, uint64_t arg1, uint32_t arg2)
r9_2 = (uint64_t)(uint32_t)arg2;
rax_2 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)rdx_1 - (uint64_t)0x7a143589U) >> 0U);
rdx_2 = arg0;
for (; ; ) {
do {
tmp_401788_0_1 = r2sleigh_load_u32((void*)rdx_2);
rdx_3 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)16U);
tmp_lane_401792_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)tmp_401788_0_1 * (uint64_t)0xffffffff85ebca77U) >> 0U) + (uint32_t)((uint64_t)r11_2 >> 0U));
@@ -1249,16 +1206,12 @@ uint32_t xxhash32(uint64_t arg0, uint64_t arg1, uint32_t arg2)
tmp_4017c5_8_1 = r2sleigh_load_u32((void*)(uint64_t)((uint64_t)rdx_3 - (uint64_t)4U));
tmp_lane_4017cc_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)tmp_4017c5_8_1 * (uint64_t)0xffffffff85ebca77U) >> 0U) + (uint32_t)((uint64_t)r10_2 >> 0U));
tmp_lane_4017d2_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)13U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017cc_6_0_1 << (uint32_t)13U)) | (uint32_t)((uint32_t)19U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017cc_6_0_1 >> (uint32_t)19U))) * (uint64_t)0xffffffff9e3779b1U) >> 0U);
if ((uint8_t)((uint64_t)rdx_3 <= (uint64_t)tmp_40176f_0_1)) {
r10_2 = (uint64_t)(uint32_t)tmp_lane_4017d2_3_0_1;
r11_2 = (uint64_t)(uint32_t)tmp_lane_401798_3_0_1;
r9_2 = (uint64_t)(uint32_t)tmp_lane_4017be_3_0_1;
rax_2 = (uint64_t)(uint32_t)tmp_lane_4017b2_3_0_1;
rdx_2 = rdx_3;
} else {
break;
}
}
r10_2 = (uint64_t)(uint32_t)tmp_lane_4017d2_3_0_1;
r11_2 = (uint64_t)(uint32_t)tmp_lane_401798_3_0_1;
r9_2 = (uint64_t)(uint32_t)tmp_lane_4017be_3_0_1;
rax_2 = (uint64_t)(uint32_t)tmp_lane_4017b2_3_0_1;
rdx_2 = rdx_3;
} while ((uint8_t)((uint64_t)rdx_3 <= (uint64_t)tmp_40176f_0_1));
{
rdi_2 = (uint64_t)((uint64_t)arg0 + (uint64_t)((uint8_t)((uint64_t)((uint64_t)arg0 + (uint64_t)1U) <= (uint64_t)((uint64_t)rsi_1 - (uint64_t)15U)) ? (uint64_t)((uint64_t)((uint64_t)((uint64_t)arg1 - (uint64_t)16U) & (uint64_t)-0x10) + (uint64_t)16U) : (uint64_t)16U));
rdx_11 = (uint64_t)(uint32_t)((uint32_t)((uint32_t)((uint32_t)14U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017d2_3_0_1 >> (uint32_t)14U)) | (uint32_t)((uint32_t)18U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017d2_3_0_1 << (uint32_t)18U))) + (uint32_t)((uint32_t)((uint32_t)((uint32_t)12U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017be_3_0_1 << (uint32_t)12U)) | (uint32_t)((uint32_t)20U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017be_3_0_1 >> (uint32_t)20U))) + (uint32_t)((uint64_t)((uint64_t)(uint32_t)((uint32_t)((uint32_t)7U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017b2_3_0_1 << (uint32_t)7U)) | (uint32_t)((uint32_t)25U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017b2_3_0_1 >> (uint32_t)25U))) + (uint64_t)(uint32_t)((uint32_t)((uint32_t)1U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401798_3_0_1 << (uint32_t)1U)) | (uint32_t)(uint8_t)((int32_t)tmp_lane_401798_3_0_1 < (int32_t)0U))) >> 0U)));
@@ -1273,18 +1226,14 @@ uint32_t xxhash32(uint64_t arg0, uint64_t arg1, uint32_t arg2)
} else {
rcx_14 = tmp_40181c_0_1;
rdx_13 = rdx_12;
for (; ; ) {
do {
tmp_401830_8_1 = r2sleigh_load_u32((void*)(uint64_t)((uint64_t)rcx_14 - (uint64_t)4U));
rcx_15 = (uint64_t)((uint64_t)rcx_14 + (uint64_t)4U);
tmp_lane_40183b_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)tmp_401830_8_1 * (uint64_t)0xffffffffc2b2ae3dU) >> 0U) + (uint32_t)((uint64_t)rdx_13 >> 0U));
rdx_14 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)15U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_40183b_6_0_1 >> (uint32_t)15U)) | (uint32_t)((uint32_t)17U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_40183b_6_0_1 << (uint32_t)17U))) * (uint64_t)0x27d4eb2fU) >> 0U);
if ((uint8_t)((uint64_t)rcx_15 <= (uint64_t)rsi_1)) {
rcx_14 = rcx_15;
rdx_13 = rdx_14;
} else {
break;
}
}
rcx_14 = rcx_15;
rdx_13 = rdx_14;
} while ((uint8_t)((uint64_t)rcx_15 <= (uint64_t)rsi_1));
{
rdi_4 = (uint64_t)((uint64_t)tmp_40181c_0_1 + (uint64_t)((uint64_t)((uint64_t)((uint64_t)rsi_1 - (uint64_t)rdi_2) - (uint64_t)4U) & (uint64_t)-0x4));
rdx_15 = rdx_14;
@@ -1624,7 +1573,7 @@ uint32_t pearson(uint64_t arg0, uint64_t arg1)
uint8_t tmp_401b29_8_1;
uint32_t tmp_lane_401b29_2_0_1;
/* r2dec proof: no individual construct is marked; 42 source obligations: 34 rendered, 8 elided, 0 refused; 15 statements rendered; 1 data object type refused */
/* r2dec proof: no individual construct is marked; 42 source obligations: 34 rendered, 8 elided, 0 refused; 13 statements rendered; 1 data object type refused */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -1632,18 +1581,14 @@ uint32_t pearson(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = 0;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401b20_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_401b29_8_1 = r2sleigh_load_u8((void*)((uint64_t)(uint8_t)((uint32_t)(uint8_t)((uint64_t)rax_2 >> 0U) ^ (uint32_t)tmp_401b20_0_1) + (uint64_t)&pearson_tab));
tmp_lane_401b29_2_0_1 = (uint32_t)(uint8_t)tmp_401b29_8_1;
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401b29_2_0_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401b29_2_0_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1));
{
return (uint32_t)tmp_lane_401b29_2_0_1;
}
tests/db/census/hashes_gcc_x64_O2_stripped
+77 -132
View File
@@ -42,17 +42,16 @@ uint64_t fcn_401000(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x403fe0U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x401014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x401014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -276,7 +275,7 @@ uint64_t fcn_401330(uint64_t arg0, uint64_t arg1)
uint64_t rdi_2;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return (uint64_t)0x811c9dc5U;
@@ -284,17 +283,13 @@ uint64_t fcn_401330(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0x811c9dc5U;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401348_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rax_4 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)(uint8_t)tmp_401348_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U)) * (uint64_t)0x1000193U) >> 0U);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = rax_4;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = rax_4;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint64_t)rax_4;
}
@@ -366,7 +361,7 @@ uint64_t fcn_4013b0(uint64_t arg0, uint64_t arg1)
uint8_t tmp_4013d3_8_1;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 41 source obligations: 33 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 41 source obligations: 33 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0x1505;
@@ -374,17 +369,13 @@ uint64_t fcn_4013b0(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = 0x1505;
rdi_1 = arg0;
for (; ; ) {
do {
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_4013d3_8_1 = r2sleigh_load_u8((void*)(uint64_t)((uint64_t)rdi_2 - (uint64_t)1U));
rax_4 = (uint64_t)(uint32_t)((uint32_t)(uint8_t)tmp_4013d3_8_1 + (uint32_t)((uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) * (uint32_t)32U) + (uint32_t)((uint64_t)rax_2 >> 0U)));
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1)) {
rax_2 = rax_4;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = rax_4;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1));
{
return (uint64_t)rax_4;
}
@@ -410,7 +401,7 @@ uint32_t fcn_4013f0(uint64_t arg0, uint64_t arg1)
uint64_t rdi_2;
uint32_t tmp_lane_40140d_6_0_1;
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -418,17 +409,13 @@ uint32_t fcn_4013f0(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = 0;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401406_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_lane_40140d_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rax_2 >> 0U) * (uint64_t)0x1003fU) >> 0U) + (uint32_t)(uint8_t)tmp_401406_0_1);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_40140d_6_0_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_40140d_6_0_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint32_t)tmp_lane_40140d_6_0_1;
}
@@ -458,7 +445,7 @@ uint64_t fcn_401420(uint64_t arg0, uint64_t arg1)
uint64_t rax_4;
uint32_t tmp_lane_401478_5_0_1;
/* r2dec proof: no individual construct is marked; 66 source obligations: 58 rendered, 8 elided, 0 refused; 19 statements rendered */
/* r2dec proof: no individual construct is marked; 66 source obligations: 58 rendered, 8 elided, 0 refused; 17 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 1;
@@ -467,21 +454,17 @@ uint64_t fcn_401420(uint64_t arg0, uint64_t arg1)
rax_2 = 0;
rdi_1 = arg0;
rdx_2 = 1;
for (; ; ) {
do {
tmp_401440_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rdx_3 = (uint64_t)(uint32_t)((uint32_t)(uint8_t)tmp_401440_0_1 + (uint32_t)((uint64_t)rdx_2 >> 0U));
tmp_lane_40145b_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rdx_3 >> 0U) - (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)((uint64_t)(uint32_t)47U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)((uint64_t)rdx_3 * (uint64_t)0x80078071U) >> (uint64_t)(uint32_t)47U)) >> 0U) * (uint64_t)0xfff1U) >> 0U));
rax_4 = (uint64_t)(uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) + (uint32_t)tmp_lane_40145b_5_0_1);
tmp_lane_401478_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rax_4 >> 0U) - (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)((uint64_t)(uint32_t)47U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)((uint64_t)rax_4 * (uint64_t)0x80078071U) >> (uint64_t)(uint32_t)47U)) >> 0U) * (uint64_t)0xfff1U) >> 0U));
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401478_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_40145b_5_0_1;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401478_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_40145b_5_0_1;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint64_t)(uint32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401478_5_0_1 << (uint32_t)16U)) | (uint32_t)tmp_lane_40145b_5_0_1);
}
@@ -513,7 +496,7 @@ uint64_t fcn_4014a0(uint64_t arg0, uint64_t arg1)
uint64_t rax_8;
uint32_t tmp_lane_401517_5_0_1;
/* r2dec proof: no individual construct is marked; 73 source obligations: 66 rendered, 7 elided, 0 refused; 21 statements rendered */
/* r2dec proof: no individual construct is marked; 73 source obligations: 66 rendered, 7 elided, 0 refused; 19 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return (uint64_t)0xffffffffU;
@@ -522,7 +505,7 @@ uint64_t fcn_4014a0(uint64_t arg0, uint64_t arg1)
rax_2 = 0xffff;
rdi_1 = arg0;
rdx_2 = 0xffff;
for (; ; ) {
do {
tmp_4014c0_0_1 = r2sleigh_load_u16((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)2U);
r8_1 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)rdx_2 + (uint64_t)(uint16_t)tmp_4014c0_0_1) >> 0U);
@@ -531,14 +514,10 @@ uint64_t fcn_4014a0(uint64_t arg0, uint64_t arg1)
rcx_4 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(uint32_t)tmp_lane_4014ed_5_0_1 + (uint64_t)rax_2) >> 0U);
rax_8 = (uint64_t)(uint32_t)47U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)((uint64_t)rcx_4 + (uint64_t)((uint64_t)((uint64_t)rcx_4 + (uint64_t)((uint64_t)rcx_4 * (uint64_t)0x10000U)) * (uint64_t)0x8000U)) >> (uint64_t)(uint32_t)47U);
tmp_lane_401517_5_0_1 = (uint32_t)((uint32_t)((uint32_t)((uint64_t)rax_8 >> 0U) + (uint32_t)((uint64_t)rcx_4 >> 0U)) - (uint32_t)((uint32_t)((uint64_t)rax_8 >> 0U) * (uint32_t)0x10000U));
if ((uint8_t)((uint64_t)tmp_4014a9_8_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401517_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_4014ed_5_0_1;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401517_5_0_1;
rdi_1 = rdi_2;
rdx_2 = (uint64_t)(uint32_t)tmp_lane_4014ed_5_0_1;
} while ((uint8_t)((uint64_t)tmp_4014a9_8_1 != (uint64_t)rdi_2));
{
return (uint64_t)(uint32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401517_5_0_1 << (uint32_t)16U)) | (uint32_t)tmp_lane_4014ed_5_0_1);
}
@@ -568,7 +547,7 @@ uint32_t fcn_401530(uint64_t arg0, uint64_t arg1)
uint32_t tmp_lane_401568_5_0_1;
uint64_t rdi_2;
/* r2dec proof: no individual construct is marked; 60 source obligations: 52 rendered, 8 elided, 0 refused; 23 statements rendered */
/* r2dec proof: no individual construct is marked; 60 source obligations: 52 rendered, 8 elided, 0 refused; 19 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -576,31 +555,21 @@ uint32_t fcn_401530(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0xffffffffU;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401548_0_1 = r2sleigh_load_u8((void*)rdi_1);
rax_4 = (uint64_t)(uint32_t)((uint32_t)(uint8_t)tmp_401548_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U));
rdx_3 = 8;
for (; ; ) {
do {
tmp_lane_401566_2_2_1 = (uint32_t)((uint32_t)((uint32_t)1U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)((uint64_t)rax_4 >> 0U) >> (uint32_t)1U)) ^ (uint32_t)((uint32_t)((uint32_t)0U - (uint32_t)((uint32_t)((uint64_t)rax_4 >> 0U) & (uint32_t)1U)) & (uint32_t)0xedb88320U));
rax_8 = (uint64_t)(uint32_t)tmp_lane_401566_2_2_1;
tmp_lane_401568_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rdx_3 >> 0U) - (uint32_t)1U);
if ((uint8_t)((uint32_t)tmp_lane_401568_5_0_1 != (uint32_t)0U)) {
rax_4 = rax_8;
rdx_3 = (uint64_t)(uint32_t)tmp_lane_401568_5_0_1;
} else {
break;
}
}
{
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1)) {
rax_2 = rax_8;
rdi_1 = rdi_2;
} else {
break;
}
}
}
rax_4 = rax_8;
rdx_3 = (uint64_t)(uint32_t)tmp_lane_401568_5_0_1;
} while ((uint8_t)((uint32_t)tmp_lane_401568_5_0_1 != (uint32_t)0U));
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rax_2 = rax_8;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1));
{
return (uint32_t)~(uint32_t)tmp_lane_401566_2_2_1;
}
@@ -794,13 +763,13 @@ uint64_t fcn_401580(void)
uint32_t tmp_lane_401618_5_0_1;
uint64_t rdx_3;
/* r2dec proof: 1 construct is marked below; 208 source obligations: 201 rendered, 5 elided, 0 refused, 2 residual; 186 statements rendered */
/* r2dec proof: 1 construct is marked below; 208 source obligations: 201 rendered, 5 elided, 0 refused, 2 residual; 182 statements rendered */
{
xmm3_1 = r2sleigh_load_u128((void*)(uint64_t)0x4022b0U);
xmm5_1 = r2sleigh_load_u128((void*)(uint64_t)0x4022c0U);
rdx_2 = 0x404040;
xmm4_2 = r2sleigh_load_u128((void*)(uint64_t)0x4022a0U);
for (; ; ) {
do {
tmp_lane_4015b9_0_0_1 = (uint32_t)((__uint128_t)xmm4_2 >> 0U);
tmp_lane_4015b9_0_1_1 = (uint32_t)((__uint128_t)xmm5_1 >> 0U);
tmp_lane_4015b9_0_2_1 = (uint32_t)((uint32_t)tmp_lane_4015b9_0_0_1 + (uint32_t)tmp_lane_4015b9_0_1_1);
@@ -819,7 +788,7 @@ uint64_t fcn_401580(void)
xmm4_6 = (__uint128_t)(((__uint128_t)xmm4_5 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_4015b9_3_2_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U);
rax_2 = 8;
xmm2_2 = xmm4_2;
for (; ; ) {
do {
tmp_lane_4015c4_0_0_1 = (uint32_t)((__uint128_t)xmm2_2 >> 0U);
tmp_lane_4015c4_0_1_1 = (uint32_t)1U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4015c4_0_0_1 >> (uint32_t)1U);
xmm2_3 = (__uint128_t)(((__uint128_t)xmm2_2 & (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0xffffffff00000000U)) | ((__uint128_t)tmp_lane_4015c4_0_1_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 0U);
@@ -971,24 +940,14 @@ uint64_t fcn_401580(void)
xmm0_42 = (__uint128_t)(((__uint128_t)xmm0_41 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_40160f_3_1_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U);
xmm2_7 = (__uint128_t)((__uint128_t)xmm2_6 ^ (__uint128_t)xmm0_42);
tmp_lane_401618_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) - (uint32_t)1U);
if ((uint8_t)((uint32_t)tmp_lane_401618_5_0_1 != (uint32_t)0U)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401618_5_0_1;
xmm2_2 = xmm2_7;
} else {
break;
}
}
{
r2sleigh_store_u128((void*)rdx_2, (__uint128_t)xmm2_7);
rdx_3 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)16U);
if ((uint8_t)((uint64_t)rdx_3 != (uint64_t)0x404440U)) {
rdx_2 = rdx_3;
xmm4_2 = xmm4_6;
} else {
break;
}
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401618_5_0_1;
xmm2_2 = xmm2_7;
} while ((uint8_t)((uint32_t)tmp_lane_401618_5_0_1 != (uint32_t)0U));
r2sleigh_store_u128((void*)rdx_2, (__uint128_t)xmm2_7);
rdx_3 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)16U);
rdx_2 = rdx_3;
xmm4_2 = xmm4_6;
} while ((uint8_t)((uint64_t)rdx_3 != (uint64_t)0x404440U));
{
return r2sleigh_residual_u64(1);
}
@@ -1014,7 +973,7 @@ uint32_t fcn_401630(uint64_t arg0, uint64_t arg1)
uint32_t tmp_401657_10_1;
uint32_t tmp_lane_401657_5_1_1;
/* r2dec proof: no individual construct is marked; 48 source obligations: 40 rendered, 8 elided, 0 refused; 15 statements rendered */
/* r2dec proof: no individual construct is marked; 48 source obligations: 40 rendered, 8 elided, 0 refused; 13 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -1022,18 +981,14 @@ uint32_t fcn_401630(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0xffffffffU;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401648_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_401657_10_1 = r2sleigh_load_u32((void*)(uint64_t)((uint64_t)((uint64_t)(uint8_t)((uint32_t)((uint32_t)(uint8_t)tmp_401648_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U)) >> 0U) * (uint64_t)4U) + (uint64_t)0x404040U));
tmp_lane_401657_5_1_1 = (uint32_t)((uint32_t)((uint32_t)8U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) >> (uint32_t)8U)) ^ (uint32_t)tmp_401657_10_1);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401657_5_1_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401657_5_1_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint32_t)~(uint32_t)tmp_lane_401657_5_1_1;
}
@@ -1077,7 +1032,7 @@ uint32_t fcn_401680(uint8_t* arg0, uint64_t arg1, uint32_t arg2)
uint32_t tmp_lane_401708_3_0_1;
uint32_t tmp_lane_401715_3_0_1;
/* r2dec proof: no individual construct is marked; 121 source obligations: 116 rendered, 5 elided, 0 refused; 42 statements rendered; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 121 source obligations: 116 rendered, 5 elided, 0 refused; 44 statements rendered; the result's second register is open: no call read decides it */
{
rcx_1 = (uint64_t)(uint32_t)arg2;
rax_2 = (uint64_t)(uint32_t)2U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)arg1 >> (uint64_t)(uint32_t)2U);
@@ -1123,7 +1078,10 @@ uint32_t fcn_401680(uint8_t* arg0, uint64_t arg1, uint32_t arg2)
} else {
if ((uint8_t)((uint64_t)rax_11 != (uint64_t)1U)) {
rcx_6 = rcx_4;
goto L2;
tmp_lane_4016fe_2_2_1 = (uint32_t)((uint32_t)((uint64_t)rcx_6 >> 0U) ^ (uint32_t)((uint64_t)arg1 >> 0U));
tmp_lane_401708_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016fe_2_2_1 >> (uint32_t)16U)) ^ (uint32_t)tmp_lane_4016fe_2_2_1) * (uint64_t)0xffffffff85ebca6bU) >> 0U);
tmp_lane_401715_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)13U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401708_3_0_1 >> (uint32_t)13U)) ^ (uint32_t)tmp_lane_401708_3_0_1) * (uint64_t)0xffffffffc2b2ae35U) >> 0U);
return (uint32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401715_3_0_1 >> (uint32_t)16U)) ^ (uint32_t)tmp_lane_401715_3_0_1);
} else {
rdx_6 = 0;
}
@@ -1135,7 +1093,6 @@ uint32_t fcn_401680(uint8_t* arg0, uint64_t arg1, uint32_t arg2)
rcx_6 = (uint64_t)(uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)15U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016ed_3_0_1 << (uint32_t)15U)) | (uint32_t)((uint32_t)17U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016ed_3_0_1 >> (uint32_t)17U))) * (uint64_t)0x1b873593U) >> 0U) ^ (uint32_t)((uint64_t)rcx_4 >> 0U));
}
{
L2: ;
tmp_lane_4016fe_2_2_1 = (uint32_t)((uint32_t)((uint64_t)rcx_6 >> 0U) ^ (uint32_t)((uint64_t)arg1 >> 0U));
tmp_lane_401708_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)16U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4016fe_2_2_1 >> (uint32_t)16U)) ^ (uint32_t)tmp_lane_4016fe_2_2_1) * (uint64_t)0xffffffff85ebca6bU) >> 0U);
tmp_lane_401715_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)13U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401708_3_0_1 >> (uint32_t)13U)) ^ (uint32_t)tmp_lane_401708_3_0_1) * (uint64_t)0xffffffffc2b2ae35U) >> 0U);
@@ -1199,7 +1156,7 @@ uint32_t fcn_401750(uint64_t arg0, uint64_t arg1, uint32_t arg2)
uint32_t tmp_lane_401884_3_0_1;
uint32_t tmp_lane_401891_3_0_1;
/* r2dec proof: no individual construct is marked; 227 source obligations: 220 rendered, 7 elided, 0 refused; 69 statements rendered */
/* r2dec proof: no individual construct is marked; 227 source obligations: 220 rendered, 7 elided, 0 refused; 65 statements rendered */
{
rdx_1 = (uint64_t)(uint32_t)arg2;
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
@@ -1213,7 +1170,7 @@ uint32_t fcn_401750(uint64_t arg0, uint64_t arg1, uint32_t arg2)
r9_2 = (uint64_t)(uint32_t)arg2;
rax_2 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)rdx_1 - (uint64_t)0x7a143589U) >> 0U);
rdx_2 = arg0;
for (; ; ) {
do {
tmp_401788_0_1 = r2sleigh_load_u32((void*)rdx_2);
rdx_3 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)16U);
tmp_lane_401792_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)tmp_401788_0_1 * (uint64_t)0xffffffff85ebca77U) >> 0U) + (uint32_t)((uint64_t)r11_2 >> 0U));
@@ -1227,16 +1184,12 @@ uint32_t fcn_401750(uint64_t arg0, uint64_t arg1, uint32_t arg2)
tmp_4017c5_8_1 = r2sleigh_load_u32((void*)(uint64_t)((uint64_t)rdx_3 - (uint64_t)4U));
tmp_lane_4017cc_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)tmp_4017c5_8_1 * (uint64_t)0xffffffff85ebca77U) >> 0U) + (uint32_t)((uint64_t)r10_2 >> 0U));
tmp_lane_4017d2_3_0_1 = (uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)13U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017cc_6_0_1 << (uint32_t)13U)) | (uint32_t)((uint32_t)19U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017cc_6_0_1 >> (uint32_t)19U))) * (uint64_t)0xffffffff9e3779b1U) >> 0U);
if ((uint8_t)((uint64_t)rdx_3 <= (uint64_t)tmp_40176f_0_1)) {
r10_2 = (uint64_t)(uint32_t)tmp_lane_4017d2_3_0_1;
r11_2 = (uint64_t)(uint32_t)tmp_lane_401798_3_0_1;
r9_2 = (uint64_t)(uint32_t)tmp_lane_4017be_3_0_1;
rax_2 = (uint64_t)(uint32_t)tmp_lane_4017b2_3_0_1;
rdx_2 = rdx_3;
} else {
break;
}
}
r10_2 = (uint64_t)(uint32_t)tmp_lane_4017d2_3_0_1;
r11_2 = (uint64_t)(uint32_t)tmp_lane_401798_3_0_1;
r9_2 = (uint64_t)(uint32_t)tmp_lane_4017be_3_0_1;
rax_2 = (uint64_t)(uint32_t)tmp_lane_4017b2_3_0_1;
rdx_2 = rdx_3;
} while ((uint8_t)((uint64_t)rdx_3 <= (uint64_t)tmp_40176f_0_1));
{
rdi_2 = (uint64_t)((uint64_t)arg0 + (uint64_t)((uint8_t)((uint64_t)((uint64_t)arg0 + (uint64_t)1U) <= (uint64_t)((uint64_t)rsi_1 - (uint64_t)15U)) ? (uint64_t)((uint64_t)((uint64_t)((uint64_t)arg1 - (uint64_t)16U) & (uint64_t)-0x10) + (uint64_t)16U) : (uint64_t)16U));
rdx_11 = (uint64_t)(uint32_t)((uint32_t)((uint32_t)((uint32_t)14U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017d2_3_0_1 >> (uint32_t)14U)) | (uint32_t)((uint32_t)18U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017d2_3_0_1 << (uint32_t)18U))) + (uint32_t)((uint32_t)((uint32_t)((uint32_t)12U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017be_3_0_1 << (uint32_t)12U)) | (uint32_t)((uint32_t)20U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017be_3_0_1 >> (uint32_t)20U))) + (uint32_t)((uint64_t)((uint64_t)(uint32_t)((uint32_t)((uint32_t)7U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017b2_3_0_1 << (uint32_t)7U)) | (uint32_t)((uint32_t)25U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_4017b2_3_0_1 >> (uint32_t)25U))) + (uint64_t)(uint32_t)((uint32_t)((uint32_t)1U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_401798_3_0_1 << (uint32_t)1U)) | (uint32_t)(uint8_t)((int32_t)tmp_lane_401798_3_0_1 < (int32_t)0U))) >> 0U)));
@@ -1251,18 +1204,14 @@ uint32_t fcn_401750(uint64_t arg0, uint64_t arg1, uint32_t arg2)
} else {
rcx_14 = tmp_40181c_0_1;
rdx_13 = rdx_12;
for (; ; ) {
do {
tmp_401830_8_1 = r2sleigh_load_u32((void*)(uint64_t)((uint64_t)rcx_14 - (uint64_t)4U));
rcx_15 = (uint64_t)((uint64_t)rcx_14 + (uint64_t)4U);
tmp_lane_40183b_6_0_1 = (uint32_t)((uint32_t)((uint64_t)((uint64_t)(int32_t)tmp_401830_8_1 * (uint64_t)0xffffffffc2b2ae3dU) >> 0U) + (uint32_t)((uint64_t)rdx_13 >> 0U));
rdx_14 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)((uint32_t)15U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_40183b_6_0_1 >> (uint32_t)15U)) | (uint32_t)((uint32_t)17U >= (uint32_t)32U ? (uint32_t)0U : (uint32_t)((uint32_t)tmp_lane_40183b_6_0_1 << (uint32_t)17U))) * (uint64_t)0x27d4eb2fU) >> 0U);
if ((uint8_t)((uint64_t)rcx_15 <= (uint64_t)rsi_1)) {
rcx_14 = rcx_15;
rdx_13 = rdx_14;
} else {
break;
}
}
rcx_14 = rcx_15;
rdx_13 = rdx_14;
} while ((uint8_t)((uint64_t)rcx_15 <= (uint64_t)rsi_1));
{
rdi_4 = (uint64_t)((uint64_t)tmp_40181c_0_1 + (uint64_t)((uint64_t)((uint64_t)((uint64_t)rsi_1 - (uint64_t)rdi_2) - (uint64_t)4U) & (uint64_t)-0x4));
rdx_15 = rdx_14;
@@ -1600,7 +1549,7 @@ uint32_t fcn_401b10(uint64_t arg0, uint64_t arg1)
uint8_t tmp_401b29_8_1;
uint32_t tmp_lane_401b29_2_0_1;
/* r2dec proof: no individual construct is marked; 42 source obligations: 34 rendered, 8 elided, 0 refused; 15 statements rendered */
/* r2dec proof: no individual construct is marked; 42 source obligations: 34 rendered, 8 elided, 0 refused; 13 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -1608,18 +1557,14 @@ uint32_t fcn_401b10(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = 0;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401b20_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
tmp_401b29_8_1 = r2sleigh_load_u8((void*)(uint64_t)((uint64_t)(uint8_t)((uint32_t)(uint8_t)((uint64_t)rax_2 >> 0U) ^ (uint32_t)tmp_401b20_0_1) + (uint64_t)0x4020a0U));
tmp_lane_401b29_2_0_1 = (uint32_t)(uint8_t)tmp_401b29_8_1;
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_401b29_2_0_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_401b29_2_0_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)rsi_1));
{
return (uint32_t)tmp_lane_401b29_2_0_1;
}
tests/db/census/loop_exits_gcc_x64_O2
+157
View File
@@ -0,0 +1,157 @@
# The census of loop_exits_gcc_x64_O2: what `aa; pdd` renders for each function `afl` lists.
# Regenerate with `scripts/census.py golden` when `afl` moves, then bless.
NAME=afl
FILE=tests/fixtures/loop_exits_gcc_x64_O2
CMDS=afl
EXPECT=<<EOF
0x00401000 1 51 sym.main
0x00401040 8 76 sym.find_then_choose
0x004010a0 10 81 sym.skip_then_find
EOF
EXPECT_ERR=<<EOF
EOF
RUN
NAME=0x00401000
FILE=tests/fixtures/loop_exits_gcc_x64_O2
CMDS=aa; pdd @ 0x00401000
EXPECT=<<EOF
uint32_t fcn_401000(void)
{
uint64_t find_then_choose(uint64_t, uint32_t, uint32_t);
uint64_t skip_then_find(uint64_t, uint32_t, uint32_t);
uint64_t rax_1;
uint64_t rax_2;
/* r2dec proof: no individual construct is marked; 33 source obligations: 23 rendered, 10 elided, 0 refused; 3 statements rendered */
/* symbol `main`; C allows that name no other prototype */
{
rax_1 = find_then_choose(0x402000, 98, 1);
rax_2 = skip_then_find(0x402004, 120, 4);
return (uint32_t)((uint32_t)((uint64_t)rax_1 >> 0U) + (uint32_t)((uint64_t)rax_2 >> 0U));
}
}
EOF
EXPECT_ERR=<<EOF
EOF
RUN
NAME=0x00401040
FILE=tests/fixtures/loop_exits_gcc_x64_O2
CMDS=aa; pdd @ 0x00401040
EXPECT=<<EOF
uint64_t find_then_choose(uint64_t arg0, uint32_t arg1, uint32_t arg2)
{
extern char sink[];
uint64_t rdi_1;
uint8_t tmp_40105e_0_1;
uint32_t tmp_lane_40105e_1_0_1;
/* r2dec proof: no individual construct is marked; 44 source obligations: 34 rendered, 10 elided, 0 refused; 16 statements rendered; 1 data object type refused */
{
rdi_1 = arg0;
for (; ; ) {
tmp_40105e_0_1 = r2sleigh_load_u8((void*)rdi_1);
tmp_lane_40105e_1_0_1 = (uint32_t)(int8_t)tmp_40105e_0_1;
if ((uint8_t)((uint8_t)tmp_40105e_0_1 != (uint8_t)0U)) {
r2sleigh_store_u32((void*)&sink, (uint32_t)tmp_lane_40105e_1_0_1);
if ((uint8_t)((uint32_t)tmp_lane_40105e_1_0_1 == (uint32_t)arg1)) {
if ((uint8_t)((uint32_t)arg2 == (uint32_t)0U)) {
r2sleigh_store_u32((void*)&sink, (uint32_t)2U);
return 20;
} else {
r2sleigh_store_u32((void*)&sink, (uint32_t)1U);
return 10;
}
} else {
rdi_1 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
}
} else {
break;
}
}
{
r2sleigh_store_u32((void*)&sink, (uint32_t)3U);
return 0;
}
}
}
EOF
EXPECT_ERR=<<EOF
EOF
RUN
NAME=0x004010a0
FILE=tests/fixtures/loop_exits_gcc_x64_O2
CMDS=aa; pdd @ 0x004010a0
EXPECT=<<EOF
uint64_t skip_then_find(uint64_t arg0, uint32_t arg1, uint32_t arg2)
{
extern char sink[];
uint64_t rcx_2;
uint64_t rdi_1;
uint8_t tmp_4010bb_0_1;
uint64_t rax_1;
uint64_t rax_3;
uint64_t rdx_2;
uint32_t tmp_lane_4010e2_2_0_1;
uint8_t tmp_4010d0_8_1;
uint32_t tmp_lane_4010b0_6_0_1;
/* r2dec proof: no individual construct is marked; 79 source obligations: 69 rendered, 10 elided, 0 refused; 26 statements rendered; 1 data object type refused */
{
rcx_2 = 0;
rdi_1 = arg0;
for (; ; ) {
tmp_4010bb_0_1 = r2sleigh_load_u8((void*)rdi_1);
rax_1 = (uint64_t)(uint32_t)(int8_t)tmp_4010bb_0_1;
if ((uint8_t)((uint8_t)tmp_4010bb_0_1 == (uint8_t)32U)) {
tmp_lane_4010b0_6_0_1 = (uint32_t)((uint32_t)((uint64_t)rcx_2 >> 0U) + (uint32_t)1U);
if ((uint8_t)((int32_t)arg2 < (int32_t)tmp_lane_4010b0_6_0_1)) {
r2sleigh_store_u32((void*)&sink, (uint32_t)tmp_lane_4010b0_6_0_1);
return (uint64_t)0xffffffffU;
} else {
rcx_2 = (uint64_t)(uint32_t)tmp_lane_4010b0_6_0_1;
rdi_1 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
}
} else {
break;
}
}
{
if ((uint8_t)((uint8_t)tmp_4010bb_0_1 == (uint8_t)0U)) {
} else {
rax_3 = rax_1;
rdx_2 = rdi_1;
while ((uint8_t)((uint32_t)((uint64_t)rax_3 >> 0U) != (uint32_t)arg1)) {
tmp_4010d0_8_1 = r2sleigh_load_u8((void*)(uint64_t)((uint64_t)rdx_2 + (uint64_t)1U));
if ((uint8_t)((uint8_t)tmp_4010d0_8_1 == (uint8_t)0U)) {
return 0;
} else {
rax_3 = (uint64_t)(uint32_t)(int8_t)tmp_4010d0_8_1;
rdx_2 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)1U);
}
}
{
tmp_lane_4010e2_2_0_1 = (uint32_t)((uint64_t)((uint64_t)(uint32_t)((uint32_t)((uint64_t)rcx_2 >> 0U) - (uint32_t)((uint64_t)rdi_1 >> 0U)) + (uint64_t)rdx_2) >> 0U);
r2sleigh_store_u32((void*)&sink, (uint32_t)tmp_lane_4010e2_2_0_1);
return (uint64_t)(uint32_t)((uint32_t)tmp_lane_4010e2_2_0_1 + (uint32_t)100U);
}
}
{
return 0;
}
}
}
}
EOF
EXPECT_ERR=<<EOF
EOF
RUN
tests/db/census/manual_limits_O0
+66 -101
View File
@@ -34,7 +34,7 @@ uint32_t _struct_nested_array(void* arg0, uint32_t arg1, uint32_t arg2)
uint16_t tmp_1000005bc_8_1;
uint32_t stack_m4_1;
/* r2dec proof: no individual construct is marked; 47 source obligations: 44 rendered, 3 elided, 0 refused; 12 statements rendered */
/* r2dec proof: no individual construct is marked; 47 source obligations: 44 rendered, 3 elided, 0 refused; 13 statements rendered */
{
tmp_100000568_8_1 = (uint64_t)((uint64_t)((uint64_t)(int32_t)arg1 * (uint64_t)40U) + (uint64_t)arg0);
tmp_100000578_8_1 = r2sleigh_load_u32((void*)((uint64_t)tmp_100000568_8_1 + (uint64_t)4U * 4U));
@@ -44,16 +44,15 @@ uint32_t _struct_nested_array(void* arg0, uint32_t arg1, uint32_t arg2)
tmp_1000005b4_8_1 = r2sleigh_load_u32((void*)((uint64_t)tmp_100000568_8_1 + (uint64_t)2U * 4U));
tmp_1000005bc_8_1 = r2sleigh_load_u16((void*)((uint64_t)tmp_100000568_8_1 + (uint64_t)3U * 2U));
stack_m4_1 = (uint32_t)((uint32_t)tmp_1000005b4_8_1 - (uint32_t)((uint64_t)(uint16_t)tmp_1000005bc_8_1 >> 0U));
return (uint32_t)stack_m4_1;
} else {
{
tmp_100000598_8_1 = r2sleigh_load_u32((void*)((uint64_t)tmp_100000568_8_1 + (uint64_t)4U * 4U));
tmp_1000005a0_8_1 = r2sleigh_load_u32((void*)tmp_100000568_8_1);
stack_m4_1 = (uint32_t)((uint32_t)tmp_100000598_8_1 + (uint32_t)tmp_1000005a0_8_1);
return (uint32_t)stack_m4_1;
}
}
{
return (uint32_t)stack_m4_1;
}
}
}
@@ -175,36 +174,32 @@ uint64_t _fnv_fold(uint8_t* arg0, uint64_t arg1)
uint8_t stack_m33_1;
uint64_t stack_m24_4;
/* r2dec proof: no individual construct is marked; 60 source obligations: 57 rendered, 3 elided, 0 refused; 16 statements rendered */
/* r2dec proof: no individual construct is marked; 60 source obligations: 57 rendered, 3 elided, 0 refused; 14 statements rendered */
{
stack_m32_1 = 0;
stack_m24_1 = (uint64_t)0x14650fb0739d0383U;
for (; ; ) {
if ((uint8_t)((uint64_t)arg1 <= (uint64_t)stack_m32_1)) {
break;
} else {
{
tmp_1000007a4_8_1 = r2sleigh_load_u8((void*)((uint64_t)arg0 + (uint64_t)stack_m32_1 * 1U));
stack_m33_2 = tmp_1000007a4_8_1;
if ((uint8_t)((int32_t)((uint64_t)(uint8_t)stack_m33_2 >> 0U) < (int32_t)65U)) {
stack_m33_1 = stack_m33_2;
} else {
{
if ((uint8_t)((int32_t)90U < (int32_t)((uint64_t)(uint8_t)stack_m33_2 >> 0U))) {
stack_m33_1 = stack_m33_2;
} else {
{
stack_m33_1 = (uint8_t)((uint32_t)((uint32_t)((uint64_t)(uint8_t)stack_m33_2 >> 0U) + (uint32_t)32U) >> 0U);
}
while ((uint8_t)((uint64_t)arg1 > (uint64_t)stack_m32_1)) {
{
tmp_1000007a4_8_1 = r2sleigh_load_u8((void*)((uint64_t)arg0 + (uint64_t)stack_m32_1 * 1U));
stack_m33_2 = tmp_1000007a4_8_1;
if ((uint8_t)((int32_t)((uint64_t)(uint8_t)stack_m33_2 >> 0U) < (int32_t)65U)) {
stack_m33_1 = stack_m33_2;
} else {
{
if ((uint8_t)((int32_t)90U < (int32_t)((uint64_t)(uint8_t)stack_m33_2 >> 0U))) {
stack_m33_1 = stack_m33_2;
} else {
{
stack_m33_1 = (uint8_t)((uint32_t)((uint32_t)((uint64_t)(uint8_t)stack_m33_2 >> 0U) + (uint32_t)32U) >> 0U);
}
}
}
}
{
stack_m24_4 = (uint64_t)((uint64_t)((uint64_t)(uint8_t)stack_m33_1 ^ (uint64_t)stack_m24_1) * (uint64_t)0x100000001b3U);
{
stack_m24_4 = (uint64_t)((uint64_t)((uint64_t)(uint8_t)stack_m33_1 ^ (uint64_t)stack_m24_1) * (uint64_t)0x100000001b3U);
{
stack_m32_1 = (uint64_t)((uint64_t)stack_m32_1 + (uint64_t)1U);
stack_m24_1 = stack_m24_4;
}
stack_m32_1 = (uint64_t)((uint64_t)stack_m32_1 + (uint64_t)1U);
stack_m24_1 = stack_m24_4;
}
}
}
@@ -232,22 +227,26 @@ uint32_t _sparse_switch(uint32_t arg0, uint32_t arg1, uint32_t arg2)
uint32_t stack_m24_1;
uint32_t stack_m4_1;
/* r2dec proof: 1 construct is marked below; 64 source obligations: 59 rendered, 3 elided, 0 refused, 2 residual; 18 statements rendered */
/* r2dec proof: 1 construct is marked below; 64 source obligations: 59 rendered, 3 elided, 0 refused, 2 residual; 23 statements rendered */
{
if ((uint8_t)((uint32_t)((uint32_t)arg0 + (uint32_t)7U) == (uint32_t)0U)) {
stack_m4_1 = (uint32_t)((uint32_t)arg1 - (uint32_t)arg2);
return (uint32_t)stack_m4_1;
} else {
{
if ((uint8_t)((uint32_t)arg0 == (uint32_t)0U)) {
stack_m4_1 = (uint32_t)((uint32_t)arg1 + (uint32_t)arg2);
return (uint32_t)stack_m4_1;
} else {
{
if ((uint8_t)((uint32_t)arg0 == (uint32_t)3U)) {
stack_m4_1 = (uint32_t)((uint32_t)((uint32_t)arg1 * (uint32_t)3U) + (uint32_t)arg2);
return (uint32_t)stack_m4_1;
} else {
{
if ((uint8_t)((uint32_t)arg0 == (uint32_t)42U)) {
stack_m4_1 = (uint32_t)((uint32_t)((uint32_t)arg1 ^ (uint32_t)arg2) + (uint32_t)42U);
return (uint32_t)stack_m4_1;
} else {
{
if ((uint8_t)((uint32_t)arg0 == (uint32_t)0x3e8U)) {
@@ -263,10 +262,12 @@ uint32_t _sparse_switch(uint32_t arg0, uint32_t arg1, uint32_t arg2)
}
{
stack_m4_1 = stack_m24_1;
return (uint32_t)stack_m4_1;
}
} else {
{
stack_m4_1 = (uint32_t)((uint32_t)((uint32_t)arg0 + (uint32_t)arg1) - (uint32_t)arg2);
return (uint32_t)stack_m4_1;
}
}
}
@@ -277,9 +278,6 @@ uint32_t _sparse_switch(uint32_t arg0, uint32_t arg1, uint32_t arg2)
}
}
}
{
return (uint32_t)stack_m4_1;
}
}
}
@@ -312,13 +310,14 @@ uint32_t _table_walk(uint64_t arg0, uint8_t* arg1)
uint64_t x8_30;
uint32_t stack_m4_1;
/* r2dec proof: no individual construct is marked; 120 source obligations: 117 rendered, 3 elided, 0 refused; 40 statements rendered */
/* r2dec proof: no individual construct is marked; 120 source obligations: 117 rendered, 3 elided, 0 refused; 37 statements rendered */
{
stack_m40_1 = arg0;
stack_m28_1 = 0;
for (; ; ) {
if ((uint8_t)((uint64_t)stack_m40_1 == (uint64_t)0U)) {
goto L6;
stack_m4_1 = (uint32_t)((uint32_t)0U - (uint32_t)stack_m28_1);
return (uint32_t)stack_m4_1;
} else {
{
stack_m28_3 = (uint32_t)((uint32_t)stack_m28_1 + (uint32_t)1U);
@@ -393,17 +392,9 @@ uint32_t _table_walk(uint64_t arg0, uint8_t* arg1)
{
tmp_100000a58_8_1 = r2sleigh_load_u32((void*)stack_m40_1);
stack_m4_1 = (uint32_t)((uint32_t)tmp_100000a58_8_1 + (uint32_t)stack_m28_3);
goto L7;
return (uint32_t)stack_m4_1;
}
}
{
L6: ;
stack_m4_1 = (uint32_t)((uint32_t)0U - (uint32_t)stack_m28_1);
}
{
L7: ;
return (uint32_t)stack_m4_1;
}
}
}
@@ -436,7 +427,7 @@ uint32_t _state_machine(uint8_t* arg0)
uint32_t stack_m20_7;
uint32_t stack_m4_1;
/* r2dec proof: no individual construct is marked; 216 source obligations: 213 rendered, 3 elided, 0 refused; 85 statements rendered */
/* r2dec proof: no individual construct is marked; 216 source obligations: 213 rendered, 3 elided, 0 refused; 77 statements rendered */
{
stack_m32_1 = 0;
stack_m24_1 = 0;
@@ -444,7 +435,8 @@ uint32_t _state_machine(uint8_t* arg0)
for (; ; ) {
tmp_100000ac0_18_1 = r2sleigh_load_u8((void*)((uint64_t)arg0 + (uint64_t)stack_m32_1 * 1U));
if ((uint8_t)((uint32_t)(int8_t)tmp_100000ac0_18_1 == (uint32_t)0U)) {
goto L16;
stack_m4_1 = (uint32_t)((uint32_t)stack_m24_1 + (uint32_t)stack_m20_1);
return (uint32_t)stack_m4_1;
} else {
{
tmp_100000ad8_8_1 = r2sleigh_load_u8((void*)((uint64_t)arg0 + (uint64_t)stack_m32_1 * 1U));
@@ -461,11 +453,13 @@ uint32_t _state_machine(uint8_t* arg0)
{
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) != (uint32_t)95U)) {
if ((uint8_t)((int32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) < (int32_t)48U)) {
goto L2;
stack_m4_1 = (uint32_t)((uint32_t)0xffffff9cU - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
return (uint32_t)stack_m4_1;
} else {
{
if ((uint8_t)((int32_t)57U < (int32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U))) {
goto L2;
stack_m4_1 = (uint32_t)((uint32_t)0xffffff9cU - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
return (uint32_t)stack_m4_1;
} else {
{
stack_m24_9 = (uint32_t)((uint32_t)stack_m24_1 - (uint32_t)5U);
@@ -521,7 +515,8 @@ L3: ;
{
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) == (uint32_t)95U)) {
} else {
goto L6;
stack_m4_1 = (uint32_t)((uint32_t)0xffffff38U - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
return (uint32_t)stack_m4_1;
}
}
{
@@ -551,13 +546,15 @@ L9: ;
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) != (uint32_t)95U)) {
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) != (uint32_t)45U)) {
if ((uint8_t)((int32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) < (int32_t)48U)) {
goto L11;
stack_m4_1 = (uint32_t)((uint32_t)0xfffffed4U - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
return (uint32_t)stack_m4_1;
} else {
{
if ((uint8_t)((int32_t)((uint64_t)(uint8_t)tmp_100000ad8_8_1 >> 0U) <= (int32_t)57U)) {
stack_m20_5 = stack_m20_1;
} else {
goto L10;
stack_m4_1 = (uint32_t)((uint32_t)0xfffffed4U - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
return (uint32_t)stack_m4_1;
}
}
}
@@ -605,37 +602,9 @@ L9: ;
{
{
stack_m4_1 = (uint32_t)((uint32_t)0xfffffe70U - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
goto L17;
return (uint32_t)stack_m4_1;
}
}
{
L10: ;
}
{
L11: ;
stack_m4_1 = (uint32_t)((uint32_t)0xfffffed4U - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
goto L17;
}
{
L6: ;
{
stack_m4_1 = (uint32_t)((uint32_t)0xffffff38U - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
goto L17;
}
}
{
L2: ;
stack_m4_1 = (uint32_t)((uint32_t)0xffffff9cU - (uint32_t)((uint64_t)stack_m32_1 >> 0U));
goto L17;
}
{
L16: ;
stack_m4_1 = (uint32_t)((uint32_t)stack_m24_1 + (uint32_t)stack_m20_1);
}
{
L17: ;
return (uint32_t)stack_m4_1;
}
}
}
@@ -655,36 +624,32 @@ uint64_t _mem_scan2(uint8_t* arg0, uint64_t arg1, uint8_t arg2, uint8_t arg3)
uint8_t tmp_100000d8c_8_1;
uint64_t stack_m32_2;
/* r2dec proof: no individual construct is marked; 49 source obligations: 46 rendered, 3 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 49 source obligations: 46 rendered, 3 elided, 0 refused; 12 statements rendered */
{
stack_m40_1 = 0;
stack_m32_1 = 0;
for (; ; ) {
if ((uint8_t)((uint64_t)arg1 <= (uint64_t)stack_m40_1)) {
break;
} else {
{
tmp_100000d8c_8_1 = r2sleigh_load_u8((void*)((uint64_t)arg0 + (uint64_t)stack_m40_1 * 1U));
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)arg2 >> 0U) == (uint32_t)((uint64_t)(uint8_t)tmp_100000d8c_8_1 >> 0U))) {
{
stack_m32_2 = (uint64_t)((uint64_t)stack_m32_1 + (uint64_t)1U);
}
} else {
{
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)arg3 >> 0U) != (uint32_t)((uint64_t)(uint8_t)tmp_100000d8c_8_1 >> 0U))) {
stack_m32_2 = stack_m32_1;
} else {
{
stack_m32_2 = (uint64_t)((uint64_t)stack_m32_1 + (uint64_t)1U);
}
while ((uint8_t)((uint64_t)arg1 > (uint64_t)stack_m40_1)) {
{
tmp_100000d8c_8_1 = r2sleigh_load_u8((void*)((uint64_t)arg0 + (uint64_t)stack_m40_1 * 1U));
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)arg2 >> 0U) == (uint32_t)((uint64_t)(uint8_t)tmp_100000d8c_8_1 >> 0U))) {
{
stack_m32_2 = (uint64_t)((uint64_t)stack_m32_1 + (uint64_t)1U);
}
} else {
{
if ((uint8_t)((uint32_t)((uint64_t)(uint8_t)arg3 >> 0U) != (uint32_t)((uint64_t)(uint8_t)tmp_100000d8c_8_1 >> 0U))) {
stack_m32_2 = stack_m32_1;
} else {
{
stack_m32_2 = (uint64_t)((uint64_t)stack_m32_1 + (uint64_t)1U);
}
}
}
}
{
{
{
stack_m40_1 = (uint64_t)((uint64_t)stack_m40_1 + (uint64_t)1U);
stack_m32_1 = stack_m32_2;
}
stack_m40_1 = (uint64_t)((uint64_t)stack_m40_1 + (uint64_t)1U);
stack_m32_1 = stack_m32_2;
}
}
}
tests/db/census/manual_limits_O2
+48 -75
View File
@@ -91,7 +91,7 @@ uint64_t _out_param_parse(uint8_t* arg0, struct fcn_1000004e0_arg1* arg1)
uint32_t tmp_lane_100000568_0_0_1;
uint8_t tmp_10000057c_18_1;
/* r2dec proof: no individual construct is marked; 98 source obligations: 98 rendered, 0 elided, 0 refused; 39 statements rendered; 1 structure laid out from a parameter's accesses */
/* r2dec proof: no individual construct is marked; 98 source obligations: 98 rendered, 0 elided, 0 refused; 37 statements rendered; 1 structure laid out from a parameter's accesses */
{
tmp_1000004f4_8_1 = r2sleigh_load_u8((void*)(uint64_t)arg0);
x9_1 = (uint64_t)(uint8_t)tmp_1000004f4_8_1;
@@ -109,21 +109,17 @@ uint64_t _out_param_parse(uint8_t* arg0, struct fcn_1000004e0_arg1* arg1)
x12_2 = x12_1;
x16_2 = x10_1;
x8_8 = (uint64_t)0x14650fb0739d0383U;
for (; ; ) {
do {
x11_8 = (uint64_t)(uint32_t)((uint32_t)((uint32_t)((uint64_t)x11_6 >> 0U) * (uint32_t)10U) + (uint32_t)(uint8_t)((uint32_t)((uint32_t)((uint64_t)x12_2 >> 0U) - (uint32_t)48U) >> 0U));
x8_10 = (uint64_t)((uint64_t)((uint64_t)((uint64_t)x12_2 & (uint64_t)255U) ^ (uint64_t)x8_8) * (uint64_t)0x100000001b3U);
x10_4 = (uint64_t)((uint64_t)x16_2 + (uint64_t)1U);
tmp_100000548_18_1 = r2sleigh_load_u8((void*)((uint64_t)tmp_100000518_8_1 + (uint64_t)x16_2 * 1U));
x12_3 = (uint64_t)(uint8_t)tmp_100000548_18_1;
if ((uint8_t)((uint32_t)((uint32_t)((uint64_t)x12_3 >> 0U) - (uint32_t)48U) < (uint32_t)10U)) {
x11_6 = x11_8;
x12_2 = x12_3;
x16_2 = x10_4;
x8_8 = x8_10;
} else {
break;
}
}
x11_6 = x11_8;
x12_2 = x12_3;
x16_2 = x10_4;
x8_8 = x8_10;
} while ((uint8_t)((uint32_t)((uint32_t)((uint64_t)x12_3 >> 0U) - (uint32_t)48U) < (uint32_t)10U));
{
x10_5 = x10_4;
x11_10 = x11_8;
@@ -282,10 +278,11 @@ uint32_t _table_walk(uint64_t arg0, uint64_t arg1)
uint64_t x0_2;
uint64_t x0_5;
/* r2dec proof: no individual construct is marked; 92 source obligations: 92 rendered, 0 elided, 0 refused; 50 statements rendered */
/* r2dec proof: no individual construct is marked; 92 source obligations: 92 rendered, 0 elided, 0 refused; 51 statements rendered */
{
if ((uint8_t)((uint64_t)arg0 == (uint64_t)0U)) {
x0_5 = arg0;
return (uint32_t)x0_5;
} else {
x0_1 = arg0;
x8_2 = 0;
@@ -363,11 +360,9 @@ L1: ;
{
L4: ;
x0_5 = (uint64_t)(uint32_t)~(uint32_t)((uint64_t)x8_2 >> 0U);
return (uint32_t)x0_5;
}
}
{
return (uint32_t)x0_5;
}
}
}
@@ -399,7 +394,7 @@ uint64_t _state_machine(uint8_t* arg0)
uint64_t x11_3;
uint64_t x0_9;
/* r2dec proof: no individual construct is marked; 154 source obligations: 150 rendered, 4 elided, 0 refused; 78 statements rendered */
/* r2dec proof: no individual construct is marked; 154 source obligations: 150 rendered, 4 elided, 0 refused; 71 statements rendered */
{
tmp_1000006c8_8_1 = r2sleigh_load_u8((void*)(uint64_t)arg0);
x11_1 = (uint64_t)(uint8_t)tmp_1000006c8_8_1;
@@ -429,7 +424,9 @@ uint64_t _state_machine(uint8_t* arg0)
if ((uint8_t)((uint32_t)(uint8_t)r2sleigh_int_carry_32((uint32_t)tmp_10000078c_0_1, (uint32_t)11U) & (uint32_t)(uint8_t)((uint32_t)((uint32_t)tmp_10000078c_0_1 + (uint32_t)11U) != (uint32_t)0U))) {
x8_3 = x8_2;
} else {
goto L4;
{
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x0_2 >> 0U) + (uint32_t)100U);
}
}
}
}
@@ -457,7 +454,7 @@ uint64_t _state_machine(uint8_t* arg0)
x8_4 = x8_2;
goto L1;
} else {
goto L5;
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x0_2 >> 0U) + (uint32_t)200U);
}
}
}
@@ -466,7 +463,7 @@ uint64_t _state_machine(uint8_t* arg0)
if ((uint8_t)((uint32_t)((uint64_t)x10_2 >> 0U) != (uint32_t)0U)) {
if ((uint8_t)((uint32_t)((uint64_t)x11_2 >> 0U) != (uint32_t)120U)) {
x0_9 = x0_2;
goto L7;
return (uint64_t)x0_9;
} else {
x10_4 = 0;
x8_4 = x8_2;
@@ -508,7 +505,7 @@ L1: ;
tmp_1000006f0_8_1 = r2sleigh_load_u8((void*)x9_2);
x11_3 = (uint64_t)(uint8_t)tmp_1000006f0_8_1;
if ((uint8_t)((uint32_t)((uint64_t)x11_3 >> 0U) == (uint32_t)0U)) {
goto L6;
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x10_4 >> 0U) + (uint32_t)((uint64_t)x8_4 >> 0U));
} else {
x0_2 = (uint64_t)(uint32_t)((uint32_t)((uint64_t)x0_2 >> 0U) - (uint32_t)1U);
x10_2 = x10_4;
@@ -521,23 +518,8 @@ L1: ;
{
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x0_2 >> 0U) + (uint32_t)0x12cU);
}
{
L5: ;
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x0_2 >> 0U) + (uint32_t)200U);
}
{
L4: ;
{
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x0_2 >> 0U) + (uint32_t)100U);
}
}
{
L6: ;
return (uint64_t)(uint32_t)((uint32_t)((uint64_t)x10_4 >> 0U) + (uint32_t)((uint64_t)x8_4 >> 0U));
}
}
{
L7: ;
return (uint64_t)x0_9;
}
}
@@ -1713,7 +1695,7 @@ uint64_t _mem_scan2(uint64_t arg0, uint64_t arg1, uint32_t arg2, uint32_t arg3)
uint64_t tmp_100000b28_8_1;
uint64_t x8_16;
/* r2dec proof: no individual construct is marked; 1466 source obligations: 1463 rendered, 3 elided, 0 refused; 1096 statements rendered */
/* r2dec proof: no individual construct is marked; 1466 source obligations: 1463 rendered, 3 elided, 0 refused; 1091 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return 0;
@@ -1822,7 +1804,7 @@ uint64_t _mem_scan2(uint64_t arg0, uint64_t arg1, uint32_t arg2, uint32_t arg3)
z5_5 = r2sleigh_bits_insert_256_64(z5_3, (uint64_t)0U, 192U);
z6_5 = r2sleigh_bits_insert_256_64(z6_3, (uint64_t)0U, 192U);
z7_5 = r2sleigh_bits_insert_256_64(z7_3, (uint64_t)0U, 192U);
for (; ; ) {
do {
tmp_lane_100000884_1_0_1 = r2sleigh_bits_extract_256_128(z29_5, 0U);
tmp_lane_100000884_3_0_1 = r2sleigh_bits_extract_256_128(z27_5, 0U);
tmp_100000888_0_1 = (uint64_t)((uint64_t)x8_5 - (uint64_t)16U);
@@ -2568,36 +2550,32 @@ uint64_t _mem_scan2(uint64_t arg0, uint64_t arg1, uint32_t arg2, uint32_t arg3)
z18_7 = r2sleigh_bits_insert_256_64(z18_6, (uint64_t)tmp_lane_1000009f4_1_2_1, 64U);
z18_8 = r2sleigh_bits_insert_256_64(z18_7, (uint64_t)0U, 128U);
tmp_1000009fc_8_1 = (uint64_t)((uint64_t)x10_2 - (uint64_t)32U);
if ((uint8_t)((uint64_t)tmp_1000009fc_8_1 != (uint64_t)0U)) {
x10_2 = tmp_1000009fc_8_1;
x8_5 = (uint64_t)((uint64_t)x8_5 + (uint64_t)32U);
z16_5 = r2sleigh_bits_insert_256_64(z16_8, (uint64_t)0U, 192U);
z17_5 = r2sleigh_bits_insert_256_64(z17_8, (uint64_t)0U, 192U);
z18_5 = r2sleigh_bits_insert_256_64(z18_8, (uint64_t)0U, 192U);
z19_5 = r2sleigh_bits_insert_256_64(z19_8, (uint64_t)0U, 192U);
z20_5 = r2sleigh_bits_insert_256_64(z20_8, (uint64_t)0U, 192U);
z21_5 = r2sleigh_bits_insert_256_64(z21_8, (uint64_t)0U, 192U);
z22_5 = r2sleigh_bits_insert_256_64(z22_8, (uint64_t)0U, 192U);
z23_5 = r2sleigh_bits_insert_256_64(z23_8, (uint64_t)0U, 192U);
z24_5 = r2sleigh_bits_insert_256_64(z24_8, (uint64_t)0U, 192U);
z25_5 = r2sleigh_bits_insert_256_64(z25_8, (uint64_t)0U, 192U);
z26_5 = r2sleigh_bits_insert_256_64(z26_8, (uint64_t)0U, 192U);
z27_5 = r2sleigh_bits_insert_256_64(z27_60, (uint64_t)0U, 192U);
z29_5 = r2sleigh_bits_insert_256_64(z29_179, (uint64_t)0U, 192U);
z5_5 = r2sleigh_bits_insert_256_64(z5_8, (uint64_t)0U, 192U);
z6_5 = r2sleigh_bits_insert_256_64(z6_8, (uint64_t)0U, 192U);
z7_5 = r2sleigh_bits_insert_256_64(z7_8, (uint64_t)0U, 192U);
} else {
break;
}
}
x10_2 = tmp_1000009fc_8_1;
x8_5 = (uint64_t)((uint64_t)x8_5 + (uint64_t)32U);
z16_5 = r2sleigh_bits_insert_256_64(z16_8, (uint64_t)0U, 192U);
z17_5 = r2sleigh_bits_insert_256_64(z17_8, (uint64_t)0U, 192U);
z18_5 = r2sleigh_bits_insert_256_64(z18_8, (uint64_t)0U, 192U);
z19_5 = r2sleigh_bits_insert_256_64(z19_8, (uint64_t)0U, 192U);
z20_5 = r2sleigh_bits_insert_256_64(z20_8, (uint64_t)0U, 192U);
z21_5 = r2sleigh_bits_insert_256_64(z21_8, (uint64_t)0U, 192U);
z22_5 = r2sleigh_bits_insert_256_64(z22_8, (uint64_t)0U, 192U);
z23_5 = r2sleigh_bits_insert_256_64(z23_8, (uint64_t)0U, 192U);
z24_5 = r2sleigh_bits_insert_256_64(z24_8, (uint64_t)0U, 192U);
z25_5 = r2sleigh_bits_insert_256_64(z25_8, (uint64_t)0U, 192U);
z26_5 = r2sleigh_bits_insert_256_64(z26_8, (uint64_t)0U, 192U);
z27_5 = r2sleigh_bits_insert_256_64(z27_60, (uint64_t)0U, 192U);
z29_5 = r2sleigh_bits_insert_256_64(z29_179, (uint64_t)0U, 192U);
z5_5 = r2sleigh_bits_insert_256_64(z5_8, (uint64_t)0U, 192U);
z6_5 = r2sleigh_bits_insert_256_64(z6_8, (uint64_t)0U, 192U);
z7_5 = r2sleigh_bits_insert_256_64(z7_8, (uint64_t)0U, 192U);
} while ((uint8_t)((uint64_t)tmp_1000009fc_8_1 != (uint64_t)0U));
{
tmp_lane_100000a1c_0_1_1 = (uint64_t)((__uint128_t)tmp_lane_100000990_1_0_1 >> 0U);
tmp_lane_100000a1c_1_1_1 = (uint64_t)((__uint128_t)tmp_lane_100000990_1_0_1 >> 64U);
x8_7 = (uint64_t)((uint64_t)tmp_lane_100000954_1_2_1 + (uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)tmp_lane_1000009d0_0_2_1 + (uint64_t)tmp_lane_100000960_0_2_1) + (uint64_t)tmp_lane_1000009d0_1_2_1) + (uint64_t)tmp_lane_100000960_1_2_1) + (uint64_t)tmp_lane_1000009a0_0_2_1) + (uint64_t)tmp_lane_100000930_0_2_1) + (uint64_t)tmp_lane_1000009a0_1_2_1) + (uint64_t)tmp_lane_100000930_1_2_1) + (uint64_t)tmp_lane_1000009e8_0_2_1) + (uint64_t)tmp_lane_10000097c_0_2_1) + (uint64_t)tmp_lane_1000009e8_1_2_1) + (uint64_t)tmp_lane_10000097c_1_2_1) + (uint64_t)tmp_lane_1000009b8_0_2_1) + (uint64_t)tmp_lane_100000948_0_2_1) + (uint64_t)tmp_lane_1000009b8_1_2_1) + (uint64_t)tmp_lane_100000948_1_2_1) + (uint64_t)tmp_lane_1000009dc_0_2_1) + (uint64_t)tmp_lane_10000096c_0_2_1) + (uint64_t)tmp_lane_1000009dc_1_2_1) + (uint64_t)tmp_lane_10000096c_1_2_1) + (uint64_t)tmp_lane_1000009ac_0_2_1) + (uint64_t)tmp_lane_10000093c_0_2_1) + (uint64_t)tmp_lane_1000009ac_1_2_1) + (uint64_t)tmp_lane_10000093c_1_2_1) + (uint64_t)tmp_lane_1000009f4_0_2_1) + (uint64_t)tmp_lane_100000a1c_0_1_1) + (uint64_t)tmp_lane_1000009f4_1_2_1) + (uint64_t)tmp_lane_100000a1c_1_1_1) + (uint64_t)tmp_lane_1000009c4_0_2_1) + (uint64_t)tmp_lane_100000954_0_2_1) + (uint64_t)tmp_lane_1000009c4_1_2_1));
if ((uint8_t)((uint64_t)((uint64_t)arg1 & (uint64_t)-0x20) == (uint64_t)arg1)) {
x8_16 = x8_7;
goto L4;
return (uint64_t)x8_16;
} else {
if ((uint8_t)((uint64_t)((uint64_t)arg1 & (uint64_t)24U) == (uint64_t)0U)) {
x8_13 = x8_7;
@@ -2636,7 +2614,7 @@ uint64_t _mem_scan2(uint64_t arg0, uint64_t arg1, uint32_t arg2, uint32_t arg3)
z1_58 = r2sleigh_bits_insert_256_64(z1_56, (uint64_t)0U, 192U);
z5_18 = r2sleigh_bits_insert_256_64(z5_16, (uint64_t)0U, 192U);
z6_22 = r2sleigh_bits_insert_256_64(z6_20, (uint64_t)0U, 192U);
for (; ; ) {
do {
tmp_lane_100000a8c_2_0_1 = r2sleigh_load_u64((void*)x10_6);
tmp_lane_100000a98_0_2_1 = (uint8_t)((uint32_t)(uint8_t)((uint32_t)(uint8_t)((uint8_t)((uint64_t)tmp_lane_100000a8c_2_0_1 >> 0U) == (uint8_t)tmp_lane_100000a6c_0_0_1) * (uint32_t)(uint8_t)255U) | (uint32_t)(uint8_t)((uint32_t)(uint8_t)((uint8_t)((uint64_t)tmp_lane_100000a8c_2_0_1 >> 0U) == (uint8_t)tmp_lane_100000a70_0_0_1) * (uint32_t)(uint8_t)255U));
tmp_lane_100000a98_1_2_1 = (uint8_t)((uint32_t)(uint8_t)((uint32_t)(uint8_t)((uint8_t)((uint64_t)tmp_lane_100000a8c_2_0_1 >> 8U) == (uint8_t)tmp_lane_100000a6c_0_0_1) * (uint32_t)(uint8_t)255U) | (uint32_t)(uint8_t)((uint32_t)(uint8_t)((uint8_t)((uint64_t)tmp_lane_100000a8c_2_0_1 >> 8U) == (uint8_t)tmp_lane_100000a70_0_0_1) * (uint32_t)(uint8_t)255U));
@@ -2791,22 +2769,18 @@ uint64_t _mem_scan2(uint64_t arg0, uint64_t arg1, uint32_t arg2, uint32_t arg3)
z0_215 = r2sleigh_bits_insert_256_64(z0_214, (uint64_t)tmp_lane_100000ae8_1_2_1, 64U);
z0_216 = r2sleigh_bits_insert_256_64(z0_215, (uint64_t)0U, 128U);
tmp_100000aec_8_1 = (uint64_t)((uint64_t)x8_10 + (uint64_t)8U);
if ((uint8_t)((uint64_t)tmp_100000aec_8_1 != (uint64_t)0U)) {
x10_6 = (uint64_t)((uint64_t)x10_6 + (uint64_t)8U);
x8_10 = tmp_100000aec_8_1;
z0_213 = r2sleigh_bits_insert_256_64(z0_216, (uint64_t)0U, 192U);
z1_58 = r2sleigh_bits_insert_256_64(z1_61, (uint64_t)0U, 192U);
z5_18 = r2sleigh_bits_insert_256_64(z5_21, (uint64_t)0U, 192U);
z6_22 = r2sleigh_bits_insert_256_64(z6_25, (uint64_t)0U, 192U);
} else {
break;
}
}
x10_6 = (uint64_t)((uint64_t)x10_6 + (uint64_t)8U);
x8_10 = tmp_100000aec_8_1;
z0_213 = r2sleigh_bits_insert_256_64(z0_216, (uint64_t)0U, 192U);
z1_58 = r2sleigh_bits_insert_256_64(z1_61, (uint64_t)0U, 192U);
z5_18 = r2sleigh_bits_insert_256_64(z5_21, (uint64_t)0U, 192U);
z6_22 = r2sleigh_bits_insert_256_64(z6_25, (uint64_t)0U, 192U);
} while ((uint8_t)((uint64_t)tmp_100000aec_8_1 != (uint64_t)0U));
{
x8_12 = (uint64_t)((uint64_t)tmp_lane_100000adc_1_2_1 + (uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)((uint64_t)tmp_lane_100000ae8_0_2_1 + (uint64_t)tmp_lane_100000ae0_0_2_1) + (uint64_t)tmp_lane_100000ae8_1_2_1) + (uint64_t)tmp_lane_100000ae0_1_2_1) + (uint64_t)tmp_lane_100000ae4_0_2_1) + (uint64_t)tmp_lane_100000adc_0_2_1) + (uint64_t)tmp_lane_100000ae4_1_2_1));
if ((uint8_t)((uint64_t)((uint64_t)arg1 & (uint64_t)-0x8) == (uint64_t)arg1)) {
x8_16 = x8_12;
goto L4;
return (uint64_t)x8_16;
} else {
x8_13 = x8_12;
x9_6 = (uint64_t)((uint64_t)arg1 & (uint64_t)-0x8);
@@ -2838,7 +2812,6 @@ L3: ;
}
}
{
L4: ;
return (uint64_t)x8_16;
}
}
tests/db/census/rv_O0g
+22 -40
View File
@@ -46,17 +46,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x3fe8U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x1014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x1014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -319,7 +318,7 @@ int32_t classify(int32_t x)
{
uint64_t rax_14;
/* r2dec proof: no individual construct is marked; 38 source obligations: 22 rendered, 16 elided, 0 refused; 19 statements rendered */
/* r2dec proof: no individual construct is marked; 38 source obligations: 22 rendered, 16 elided, 0 refused; 18 statements rendered */
{
if ((uint8_t)((uint32_t)7U < (uint32_t)x)) {
} else {
@@ -327,37 +326,37 @@ int32_t classify(int32_t x)
case 7:
{
rax_14 = 87;
goto L1;
return (int32_t)rax_14;
}
case 5:
{
rax_14 = 65;
goto L1;
return (int32_t)rax_14;
}
case 4:
{
rax_14 = 54;
goto L1;
return (int32_t)rax_14;
}
case 3:
{
rax_14 = 43;
goto L1;
return (int32_t)rax_14;
}
case 2:
{
rax_14 = 32;
goto L1;
return (int32_t)rax_14;
}
case 1:
{
rax_14 = 21;
goto L1;
return (int32_t)rax_14;
}
case 0:
{
rax_14 = 10;
goto L1;
return (int32_t)rax_14;
}
case 6:
break;
@@ -367,7 +366,6 @@ int32_t classify(int32_t x)
rax_14 = (uint64_t)0xffffffffU;
}
{
L1: ;
return (int32_t)rax_14;
}
}
@@ -448,7 +446,7 @@ int64_t mul_div(int64_t a, int64_t b)
uint64_t rdx_5;
uint64_t rax_10;
/* r2dec proof: 2 constructs are marked below; 41 source obligations: 31 rendered, 7 elided, 0 refused, 3 residual; 25 statements rendered */
/* r2dec proof: 2 constructs are marked below; 41 source obligations: 31 rendered, 7 elided, 0 refused, 3 residual; 26 statements rendered */
{
if ((uint8_t)((uint64_t)b != (uint64_t)0U)) {
rax_4 = (uint64_t)((uint64_t)((uint64_t)a * (uint64_t)8U) - (uint64_t)a);
@@ -473,10 +471,9 @@ int64_t mul_div(int64_t a, int64_t b)
tmp_12e3_70_1 = (__uint128_t)tmp_12e3_10_1 == (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0xffffffffffffffffULL) ? (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0U) : (__uint128_t)((__int128_t)tmp_12e3_50_1 % (__int128_t)tmp_12e3_10_1);
rdx_5 = (uint64_t)((__uint128_t)tmp_12e3_70_1 >> 0U);
rax_10 = (uint64_t)((uint64_t)rdx_5 + (uint64_t)rax_5);
return (int64_t)rax_10;
} else {
rax_10 = 0;
}
{
return (int64_t)rax_10;
}
}
@@ -504,7 +501,7 @@ double avg(const double* v, int32_t n)
uint32_t tmp_1336_8_1;
_Alignas(16) uint8_t frame[40];
/* r2dec proof: no individual construct is marked; 53 source obligations: 38 rendered, 15 elided, 0 refused; 19 statements rendered */
/* r2dec proof: no individual construct is marked; 53 source obligations: 38 rendered, 15 elided, 0 refused; 20 statements rendered */
{
r2sleigh_store_u64((void*)(uint64_t)(frame + 24U), (uint64_t)0U);
r2sleigh_store_u32((void*)(uint64_t)(frame + 20U), (uint32_t)0U);
@@ -524,13 +521,12 @@ double avg(const double* v, int32_t n)
{
if ((uint8_t)((uint32_t)n == (uint32_t)0U)) {
xmm0_10 = 0;
return (double)r2sleigh_float_from_bits_64((uint64_t)xmm0_10);
} else {
tmp_1351_8_1 = r2sleigh_load_u64((void*)(uint64_t)(frame + 24U));
xmm0_7 = (__uint128_t)(((__uint128_t)0U & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)(uint64_t)0U & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 64U);
tmp_lane_1356_0_2_1 = r2sleigh_float_from_bits_64(tmp_1351_8_1) / (double)(int32_t)n;
xmm0_10 = (__uint128_t)(((__uint128_t)xmm0_7 & (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0U)) | ((__uint128_t)r2sleigh_float_to_bits_64(tmp_lane_1356_0_2_1) & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 0U);
}
{
return (double)r2sleigh_float_from_bits_64((uint64_t)xmm0_10);
}
}
@@ -551,15 +547,14 @@ int32_t fact(int32_t n)
uint64_t rax_3;
uint64_t rax_6;
/* r2dec proof: no individual construct is marked; 30 source obligations: 20 rendered, 10 elided, 0 refused; 5 statements rendered; 1 callee prototype supplied by the source */
/* r2dec proof: no individual construct is marked; 30 source obligations: 20 rendered, 10 elided, 0 refused; 6 statements rendered; 1 callee prototype supplied by the source */
{
if ((uint8_t)((int32_t)n <= (int32_t)1U)) {
rax_6 = 1;
return (int32_t)rax_6;
} else {
rax_3 = (uint64_t)(uint32_t)fact((uint32_t)((uint32_t)n - (uint32_t)1U));
rax_6 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rax_3 >> 0U) * (uint64_t)(int32_t)n) >> 0U);
}
{
return (int32_t)rax_6;
}
}
@@ -760,7 +755,7 @@ int32_t find(const int32_t* v, int32_t n, int32_t key)
uint64_t rax_14;
_Alignas(16) uint8_t frame[40];
/* r2dec proof: no individual construct is marked; 58 source obligations: 44 rendered, 14 elided, 0 refused; 25 statements rendered */
/* r2dec proof: no individual construct is marked; 58 source obligations: 44 rendered, 14 elided, 0 refused; 20 statements rendered */
{
r2sleigh_store_u32((void*)(uint64_t)(frame + 28U), (uint32_t)0U);
for (; ; ) {
@@ -772,36 +767,23 @@ int32_t find(const int32_t* v, int32_t n, int32_t key)
tmp_14f0_8_1 = r2sleigh_load_u32((void*)(uint64_t)(frame + 28U));
tmp_1504_0_1 = r2sleigh_load_u32((void*)((uint64_t)v + (uint64_t)(int32_t)tmp_14f0_8_1 * 4U));
if ((uint8_t)((int32_t)key < (int32_t)tmp_1504_0_1)) {
goto L2;
rax_14 = (uint64_t)0xffffffffU;
return (int32_t)rax_14;
} else {
tmp_150b_8_1 = r2sleigh_load_u32((void*)(uint64_t)(frame + 28U));
r2sleigh_store_u32((void*)(uint64_t)(frame + 28U), (uint32_t)((uint32_t)tmp_150b_8_1 + (uint32_t)1U));
}
} else {
goto L0;
tmp_14eb_8_1 = r2sleigh_load_u32((void*)(uint64_t)(frame + 28U));
rax_14 = (uint64_t)(uint32_t)tmp_14eb_8_1;
return (int32_t)rax_14;
}
} else {
break;
}
}
{
goto L3;
}
{
L0: ;
tmp_14eb_8_1 = r2sleigh_load_u32((void*)(uint64_t)(frame + 28U));
rax_14 = (uint64_t)(uint32_t)tmp_14eb_8_1;
goto L4;
}
{
L2: ;
}
{
L3: ;
rax_14 = (uint64_t)0xffffffffU;
}
{
L4: ;
return (int32_t)rax_14;
}
}
tests/db/census/rv_O0g_stripped
+50 -92
View File
@@ -45,17 +45,16 @@ uint64_t fcn_1000(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x3fe8U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x1014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x1014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -260,18 +259,14 @@ uint32_t fcn_11c1(uint32_t* arg0, uint32_t arg1)
uint32_t stack_m12_1;
uint32_t tmp_11f4_0_1;
/* r2dec proof: no individual construct is marked; 31 source obligations: 24 rendered, 7 elided, 0 refused; 9 statements rendered */
/* r2dec proof: no individual construct is marked; 31 source obligations: 24 rendered, 7 elided, 0 refused; 7 statements rendered */
{
stack_m16_1 = 0;
stack_m12_1 = 0;
for (; ; ) {
if ((uint8_t)((int32_t)stack_m12_1 < (int32_t)arg1)) {
tmp_11f4_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m12_1 * 4U));
stack_m16_1 = (uint32_t)((uint32_t)stack_m16_1 + (uint32_t)tmp_11f4_0_1);
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
} else {
break;
}
while ((uint8_t)((int32_t)stack_m12_1 < (int32_t)arg1)) {
tmp_11f4_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m12_1 * 4U));
stack_m16_1 = (uint32_t)((uint32_t)stack_m16_1 + (uint32_t)tmp_11f4_0_1);
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
}
{
return (uint32_t)stack_m16_1;
@@ -292,7 +287,7 @@ uint64_t fcn_120a(uint32_t arg0)
{
uint64_t rax_14;
/* r2dec proof: no individual construct is marked; 38 source obligations: 22 rendered, 16 elided, 0 refused; 19 statements rendered */
/* r2dec proof: no individual construct is marked; 38 source obligations: 22 rendered, 16 elided, 0 refused; 18 statements rendered */
{
if ((uint8_t)((uint32_t)7U < (uint32_t)arg0)) {
} else {
@@ -300,37 +295,37 @@ uint64_t fcn_120a(uint32_t arg0)
case 7:
{
rax_14 = 87;
goto L1;
return (uint64_t)rax_14;
}
case 5:
{
rax_14 = 65;
goto L1;
return (uint64_t)rax_14;
}
case 4:
{
rax_14 = 54;
goto L1;
return (uint64_t)rax_14;
}
case 3:
{
rax_14 = 43;
goto L1;
return (uint64_t)rax_14;
}
case 2:
{
rax_14 = 32;
goto L1;
return (uint64_t)rax_14;
}
case 1:
{
rax_14 = 21;
goto L1;
return (uint64_t)rax_14;
}
case 0:
{
rax_14 = 10;
goto L1;
return (uint64_t)rax_14;
}
case 6:
break;
@@ -340,7 +335,6 @@ uint64_t fcn_120a(uint32_t arg0)
rax_14 = (uint64_t)0xffffffffU;
}
{
L1: ;
return (uint64_t)rax_14;
}
}
@@ -361,18 +355,14 @@ uint32_t fcn_1277(uint64_t arg0)
uint32_t stack_m12_1;
uint64_t tmp_1294_8_1;
/* r2dec proof: no individual construct is marked; 28 source obligations: 21 rendered, 7 elided, 0 refused; 9 statements rendered */
/* r2dec proof: no individual construct is marked; 28 source obligations: 21 rendered, 7 elided, 0 refused; 7 statements rendered */
{
stack_m32_1 = arg0;
stack_m12_1 = 0;
for (; ; ) {
if ((uint8_t)((uint64_t)stack_m32_1 != (uint64_t)0U)) {
tmp_1294_8_1 = r2sleigh_load_u64((void*)(uint64_t)((uint64_t)stack_m32_1 + (uint64_t)8U));
stack_m32_1 = tmp_1294_8_1;
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
} else {
break;
}
while ((uint8_t)((uint64_t)stack_m32_1 != (uint64_t)0U)) {
tmp_1294_8_1 = r2sleigh_load_u64((void*)(uint64_t)((uint64_t)stack_m32_1 + (uint64_t)8U));
stack_m32_1 = tmp_1294_8_1;
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
}
{
return (uint32_t)stack_m12_1;
@@ -412,7 +402,7 @@ uint64_t fcn_12a8(uint64_t arg0, uint64_t arg1)
uint64_t rdx_5;
uint64_t rax_10;
/* r2dec proof: 2 constructs are marked below; 41 source obligations: 31 rendered, 7 elided, 0 refused, 3 residual; 25 statements rendered */
/* r2dec proof: 2 constructs are marked below; 41 source obligations: 31 rendered, 7 elided, 0 refused, 3 residual; 26 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 != (uint64_t)0U)) {
rax_4 = (uint64_t)((uint64_t)((uint64_t)arg0 * (uint64_t)8U) - (uint64_t)arg0);
@@ -437,10 +427,9 @@ uint64_t fcn_12a8(uint64_t arg0, uint64_t arg1)
tmp_12e3_70_1 = (__uint128_t)tmp_12e3_10_1 == (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0xffffffffffffffffULL) ? (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0U) : (__uint128_t)((__int128_t)tmp_12e3_50_1 % (__int128_t)tmp_12e3_10_1);
rdx_5 = (uint64_t)((__uint128_t)tmp_12e3_70_1 >> 0U);
rax_10 = (uint64_t)((uint64_t)rdx_5 + (uint64_t)rax_5);
return (uint64_t)rax_10;
} else {
rax_10 = 0;
}
{
return (uint64_t)rax_10;
}
}
@@ -465,29 +454,24 @@ double fcn_12ef(uint64_t* arg0, uint32_t arg1)
uint64_t tmp_1324_0_1;
double tmp_lane_132d_0_2_1;
/* r2dec proof: no individual construct is marked; 41 source obligations: 34 rendered, 7 elided, 0 refused; 15 statements rendered */
/* r2dec proof: no individual construct is marked; 41 source obligations: 34 rendered, 7 elided, 0 refused; 14 statements rendered */
{
stack_m20_1 = 0;
stack_m16_1 = 0;
for (; ; ) {
if ((uint8_t)((int32_t)stack_m20_1 < (int32_t)arg1)) {
tmp_1324_0_1 = r2sleigh_load_u64((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m20_1 * 8U));
tmp_lane_132d_0_2_1 = r2sleigh_float_from_bits_64(tmp_1324_0_1) + r2sleigh_float_from_bits_64(stack_m16_1);
stack_m20_1 = (uint32_t)((uint32_t)stack_m20_1 + (uint32_t)1U);
stack_m16_1 = r2sleigh_float_to_bits_64(tmp_lane_132d_0_2_1);
} else {
break;
}
while ((uint8_t)((int32_t)stack_m20_1 < (int32_t)arg1)) {
tmp_1324_0_1 = r2sleigh_load_u64((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m20_1 * 8U));
tmp_lane_132d_0_2_1 = r2sleigh_float_from_bits_64(tmp_1324_0_1) + r2sleigh_float_from_bits_64(stack_m16_1);
stack_m20_1 = (uint32_t)((uint32_t)stack_m20_1 + (uint32_t)1U);
stack_m16_1 = r2sleigh_float_to_bits_64(tmp_lane_132d_0_2_1);
}
{
if ((uint8_t)((uint32_t)arg1 == (uint32_t)0U)) {
xmm0_10 = 0;
return (double)r2sleigh_float_from_bits_64((uint64_t)xmm0_10);
} else {
xmm0_7 = (__uint128_t)(((__uint128_t)0U & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)(uint64_t)0U & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 64U);
tmp_lane_1356_0_2_1 = r2sleigh_float_from_bits_64(stack_m16_1) / (double)(int32_t)arg1;
xmm0_10 = (__uint128_t)(((__uint128_t)xmm0_7 & (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0U)) | ((__uint128_t)r2sleigh_float_to_bits_64(tmp_lane_1356_0_2_1) & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffffffffffULL)) << 0U);
}
{
return (double)r2sleigh_float_from_bits_64((uint64_t)xmm0_10);
}
}
@@ -508,15 +492,14 @@ uint64_t fcn_1362(uint32_t arg0)
uint64_t rax_3;
uint64_t rax_6;
/* r2dec proof: no individual construct is marked; 30 source obligations: 20 rendered, 10 elided, 0 refused; 5 statements rendered */
/* r2dec proof: no individual construct is marked; 30 source obligations: 20 rendered, 10 elided, 0 refused; 6 statements rendered */
{
if ((uint8_t)((int32_t)arg0 <= (int32_t)1U)) {
rax_6 = 1;
return (uint64_t)rax_6;
} else {
rax_3 = fcn_1362((uint32_t)((uint32_t)arg0 - (uint32_t)1U));
rax_6 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rax_3 >> 0U) * (uint64_t)(int32_t)arg0) >> 0U);
}
{
return (uint64_t)rax_6;
}
}
@@ -537,16 +520,12 @@ uint32_t fcn_1391(uint32_t arg0)
uint32_t tmp_lane_13cf_0_0_1;
uint32_t tmp_lane_13d8_6_0_1;
/* r2dec proof: no individual construct is marked; 32 source obligations: 24 rendered, 8 elided, 0 refused; 10 statements rendered; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 32 source obligations: 24 rendered, 8 elided, 0 refused; 8 statements rendered; the result's second register is open: no call read decides it */
{
stack_m12_1 = 0;
for (; ; ) {
if ((uint8_t)((int32_t)stack_m12_1 <= (int32_t)15U)) {
r2sleigh_store_u32((void*)(uint64_t)((uint64_t)((uint64_t)(int32_t)stack_m12_1 * (uint64_t)4U) + (uint64_t)0x4040U), (uint32_t)((uint64_t)((uint64_t)(uint32_t)arg0 + (uint64_t)(uint32_t)stack_m12_1) >> 0U));
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
} else {
break;
}
while ((uint8_t)((int32_t)stack_m12_1 <= (int32_t)15U)) {
r2sleigh_store_u32((void*)(uint64_t)((uint64_t)((uint64_t)(int32_t)stack_m12_1 * (uint64_t)4U) + (uint64_t)0x4040U), (uint32_t)((uint64_t)((uint64_t)(uint32_t)arg0 + (uint64_t)(uint32_t)stack_m12_1) >> 0U));
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
}
{
tmp_lane_13cf_0_0_1 = r2sleigh_load_u32((void*)(uint64_t)0x4010U);
@@ -646,17 +625,13 @@ uint32_t fcn_1489(uint32_t arg0)
uint32_t stack_m28_1;
uint32_t stack_m12_1;
/* r2dec proof: no individual construct is marked; 27 source obligations: 20 rendered, 7 elided, 0 refused; 8 statements rendered */
/* r2dec proof: no individual construct is marked; 27 source obligations: 20 rendered, 7 elided, 0 refused; 6 statements rendered */
{
stack_m28_1 = arg0;
stack_m12_1 = 0;
for (; ; ) {
if ((uint8_t)((uint32_t)stack_m28_1 != (uint32_t)0U)) {
stack_m28_1 = (uint32_t)((uint32_t)((uint32_t)stack_m28_1 - (uint32_t)1U) & (uint32_t)stack_m28_1);
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
} else {
break;
}
while ((uint8_t)((uint32_t)stack_m28_1 != (uint32_t)0U)) {
stack_m28_1 = (uint32_t)((uint32_t)((uint32_t)stack_m28_1 - (uint32_t)1U) & (uint32_t)stack_m28_1);
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
}
{
return (uint32_t)stack_m12_1;
@@ -680,43 +655,26 @@ uint64_t fcn_14b5(uint32_t* arg0, uint32_t arg1, uint32_t arg2)
uint32_t tmp_1504_0_1;
uint64_t rax_14;
/* r2dec proof: no individual construct is marked; 45 source obligations: 37 rendered, 8 elided, 0 refused; 20 statements rendered */
/* r2dec proof: no individual construct is marked; 45 source obligations: 37 rendered, 8 elided, 0 refused; 13 statements rendered */
{
stack_m12_1 = 0;
for (; ; ) {
if ((uint8_t)((int32_t)stack_m12_1 < (int32_t)arg1)) {
tmp_14e4_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m12_1 * 4U));
if ((uint8_t)((uint32_t)arg2 != (uint32_t)tmp_14e4_0_1)) {
tmp_1504_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m12_1 * 4U));
if ((uint8_t)((int32_t)arg2 < (int32_t)tmp_1504_0_1)) {
goto L2;
} else {
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
}
while ((uint8_t)((int32_t)stack_m12_1 < (int32_t)arg1)) {
tmp_14e4_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m12_1 * 4U));
if ((uint8_t)((uint32_t)arg2 != (uint32_t)tmp_14e4_0_1)) {
tmp_1504_0_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)(int32_t)stack_m12_1 * 4U));
if ((uint8_t)((int32_t)arg2 < (int32_t)tmp_1504_0_1)) {
rax_14 = (uint64_t)0xffffffffU;
return (uint64_t)rax_14;
} else {
goto L0;
stack_m12_1 = (uint32_t)((uint32_t)stack_m12_1 + (uint32_t)1U);
}
} else {
break;
rax_14 = (uint64_t)(uint32_t)stack_m12_1;
return (uint64_t)rax_14;
}
}
{
goto L3;
}
{
L0: ;
rax_14 = (uint64_t)(uint32_t)stack_m12_1;
goto L4;
}
{
L2: ;
}
{
L3: ;
rax_14 = (uint64_t)0xffffffffU;
}
{
L4: ;
return (uint64_t)rax_14;
}
}
tests/db/census/rv_O2
+33 -57
View File
@@ -46,17 +46,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x3fe8U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x1014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x1014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -402,7 +401,7 @@ uint32_t sum_array(uint64_t arg0, uint32_t arg1)
uint32_t tmp_lane_1398_7_0_1;
uint64_t rdi_2;
/* r2dec proof: no individual construct is marked; 42 source obligations: 34 rendered, 8 elided, 0 refused; 15 statements rendered */
/* r2dec proof: no individual construct is marked; 42 source obligations: 34 rendered, 8 elided, 0 refused; 13 statements rendered */
{
tmp_1384_0_1 = arg1;
if ((uint8_t)((int32_t)tmp_1384_0_1 <= (int32_t)0U)) {
@@ -411,17 +410,13 @@ uint32_t sum_array(uint64_t arg0, uint32_t arg1)
tmp_138d_8_1 = (uint64_t)((uint64_t)((uint64_t)(int32_t)arg1 * (uint64_t)4U) + (uint64_t)arg0);
rax_2 = 0;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_1398_0_1 = r2sleigh_load_u32((void*)rdi_1);
tmp_lane_1398_7_0_1 = (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) + (uint32_t)tmp_1398_0_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)4U);
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)tmp_138d_8_1)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_1398_7_0_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_1398_7_0_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)tmp_138d_8_1));
{
return (uint32_t)tmp_lane_1398_7_0_1;
}
@@ -445,15 +440,14 @@ uint64_t classify(uint32_t arg0)
uint32_t tmp_13c7_10_1;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 19 source obligations: 15 rendered, 4 elided, 0 refused; 5 statements rendered; 1 data object type refused */
/* r2dec proof: no individual construct is marked; 19 source obligations: 15 rendered, 4 elided, 0 refused; 6 statements rendered; 1 data object type refused */
{
if ((uint8_t)((uint32_t)7U < (uint32_t)arg0)) {
rax_4 = (uint64_t)0xffffffffU;
return (uint64_t)rax_4;
} else {
tmp_13c7_10_1 = r2sleigh_load_u32((void*)((uint64_t)((uint64_t)(uint32_t)arg0 * (uint64_t)4U) + (uint64_t)&CSWTCH_12));
rax_4 = (uint64_t)(uint32_t)tmp_13c7_10_1;
}
{
return (uint64_t)rax_4;
}
}
@@ -476,24 +470,20 @@ uint32_t list_len(uint64_t arg0)
uint64_t rdi_2;
uint32_t tmp_lane_13e4_6_0_1;
/* r2dec proof: no individual construct is marked; 34 source obligations: 27 rendered, 7 elided, 0 refused; 13 statements rendered */
/* r2dec proof: no individual construct is marked; 34 source obligations: 27 rendered, 7 elided, 0 refused; 11 statements rendered */
{
if ((uint8_t)((uint64_t)arg0 == (uint64_t)0U)) {
return 0;
} else {
rax_2 = 0;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_13e0_8_1 = r2sleigh_load_u64((void*)(uint64_t)((uint64_t)rdi_1 + (uint64_t)8U));
rdi_2 = tmp_13e0_8_1;
tmp_lane_13e4_6_0_1 = (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) + (uint32_t)1U);
if ((uint8_t)((uint64_t)rdi_2 != (uint64_t)0U)) {
rax_2 = (uint64_t)(uint32_t)tmp_lane_13e4_6_0_1;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = (uint64_t)(uint32_t)tmp_lane_13e4_6_0_1;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rdi_2 != (uint64_t)0U));
{
return (uint32_t)tmp_lane_13e4_6_0_1;
}
@@ -533,10 +523,11 @@ uint64_t mul_div(uint64_t arg0, uint64_t arg1)
uint64_t rdx_4;
uint64_t rax_8;
/* r2dec proof: 2 constructs are marked below; 38 source obligations: 31 rendered, 4 elided, 0 refused, 3 residual; 25 statements rendered */
/* r2dec proof: 2 constructs are marked below; 38 source obligations: 31 rendered, 4 elided, 0 refused, 3 residual; 26 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
rax_8 = 0;
return (uint64_t)rax_8;
} else {
rax_3 = (uint64_t)((uint64_t)((uint64_t)arg0 * (uint64_t)8U) - (uint64_t)arg0);
tmp_1416_0_1 = (__uint128_t)(int64_t)rax_3;
@@ -560,8 +551,6 @@ uint64_t mul_div(uint64_t arg0, uint64_t arg1)
tmp_1423_60_1 = (__uint128_t)tmp_1423_0_1 == (__uint128_t)((__uint128_t)0xffffffffffffffffULL << 64U | (__uint128_t)0xffffffffffffffffULL) ? (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0U) : (__uint128_t)((__int128_t)tmp_1423_40_1 % (__int128_t)tmp_1423_0_1);
rdx_4 = (uint64_t)((__uint128_t)tmp_1423_60_1 >> 0U);
rax_8 = (uint64_t)((uint64_t)rax_4 + (uint64_t)rdx_4);
}
{
return (uint64_t)rax_8;
}
}
@@ -585,7 +574,7 @@ uint64_t avg(int64_t* arg0, uint32_t arg1)
uint64_t rdi_2;
uint64_t rdi_3;
/* r2dec proof: 2 constructs are marked below; 49 source obligations: 37 rendered, 8 elided, 0 refused, 4 residual; 22 statements rendered */
/* r2dec proof: 2 constructs are marked below; 49 source obligations: 37 rendered, 8 elided, 0 refused, 4 residual; 20 statements rendered */
{
tmp_1434_0_1 = arg1;
zf_1 = (uint8_t)((uint32_t)tmp_1434_0_1 == (uint32_t)0U);
@@ -607,16 +596,12 @@ uint64_t avg(int64_t* arg0, uint32_t arg1)
rdi_2 = rdi_1;
}
}
for (; ; ) {
do {
(void)r2sleigh_load_u64((void*)rdi_2);
rdi_3 = (uint64_t)((uint64_t)rdi_2 + (uint64_t)16U);
(void)r2sleigh_load_u64((void*)(uint64_t)((uint64_t)rdi_3 - (uint64_t)8U));
if ((uint8_t)((uint64_t)rdi_3 != (uint64_t)rax_1)) {
rdi_2 = rdi_3;
} else {
break;
}
}
rdi_2 = rdi_3;
} while ((uint8_t)((uint64_t)rdi_3 != (uint64_t)rax_1));
}
{
L1: ;
@@ -642,7 +627,7 @@ uint64_t fact(uint32_t arg0)
uint32_t tmp_lane_14a2_5_0_1;
uint64_t rax_3;
/* r2dec proof: no individual construct is marked; 38 source obligations: 30 rendered, 8 elided, 0 refused; 13 statements rendered */
/* r2dec proof: no individual construct is marked; 38 source obligations: 30 rendered, 8 elided, 0 refused; 11 statements rendered */
{
rdi_1 = (uint64_t)(uint32_t)arg0;
if ((uint8_t)((int32_t)arg0 <= (int32_t)1U)) {
@@ -650,16 +635,12 @@ uint64_t fact(uint32_t arg0)
} else {
rax_2 = 1;
rdi_1_2 = rdi_1;
for (; ; ) {
do {
tmp_lane_14a2_5_0_1 = (uint32_t)((uint32_t)((uint64_t)rdi_1_2 >> 0U) - (uint32_t)1U);
rax_3 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rax_2 >> 0U) * (uint64_t)(int32_t)((uint64_t)rdi_1_2 >> 0U)) >> 0U);
if ((uint8_t)((uint32_t)tmp_lane_14a2_5_0_1 != (uint32_t)1U)) {
rax_2 = rax_3;
rdi_1_2 = (uint64_t)(uint32_t)tmp_lane_14a2_5_0_1;
} else {
break;
}
}
rax_2 = rax_3;
rdi_1_2 = (uint64_t)(uint32_t)tmp_lane_14a2_5_0_1;
} while ((uint8_t)((uint32_t)tmp_lane_14a2_5_0_1 != (uint32_t)1U));
{
return (uint64_t)rax_3;
}
@@ -712,12 +693,12 @@ uint64_t fill(uint32_t arg0)
uint32_t tmp_lane_14f4_3_2_1;
uint32_t tmp_1501_0_1;
/* r2dec proof: 1 construct is marked below; 57 source obligations: 51 rendered, 4 elided, 0 refused, 2 residual; 39 statements rendered; 1 data object type refused */
/* r2dec proof: 1 construct is marked below; 57 source obligations: 51 rendered, 4 elided, 0 refused, 2 residual; 37 statements rendered; 1 data object type refused */
{
xmm2_1 = r2sleigh_load_u128((void*)(uint64_t)0x2050U);
rax_2 = 0x4040;
xmm1_2 = r2sleigh_load_u128((void*)(uint64_t)0x2040U);
for (; ; ) {
do {
rax_3 = (uint64_t)((uint64_t)rax_2 + (uint64_t)16U);
tmp_lane_14f0_0_0_1 = (uint32_t)((__uint128_t)xmm1_2 >> 0U);
tmp_lane_14f0_0_1_1 = (uint32_t)((__uint128_t)xmm2_1 >> 0U);
@@ -746,13 +727,9 @@ uint64_t fill(uint32_t arg0)
tmp_lane_14f4_3_0_1 = (uint32_t)((__uint128_t)xmm1_2 >> 96U);
tmp_lane_14f4_3_2_1 = (uint32_t)((uint32_t)tmp_lane_14f4_3_0_1 + (uint32_t)arg0);
r2sleigh_store_u128((void*)(uint64_t)((uint64_t)rax_3 - (uint64_t)16U), (__uint128_t)(((__uint128_t)xmm0_4 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_14f4_3_2_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U));
if ((uint8_t)((uint64_t)rax_3 != (uint64_t)0x4080U)) {
rax_2 = rax_3;
xmm1_2 = (__uint128_t)(((__uint128_t)xmm1_5 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_14f0_3_2_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U);
} else {
break;
}
}
rax_2 = rax_3;
xmm1_2 = (__uint128_t)(((__uint128_t)xmm1_5 & (__uint128_t)((__uint128_t)0xffffffffU << 64U | (__uint128_t)0xffffffffffffffffULL)) | ((__uint128_t)tmp_lane_14f0_3_2_1 & (__uint128_t)((__uint128_t)0U << 64U | (__uint128_t)0xffffffffU)) << 96U);
} while ((uint8_t)((uint64_t)rax_3 != (uint64_t)0x4080U));
{
tmp_1501_0_1 = r2sleigh_load_u32((void*)&g_counter);
r2sleigh_store_u32((void*)&g_counter, (uint32_t)((uint32_t)tmp_1501_0_1 + (uint32_t)arg0));
@@ -891,7 +868,7 @@ uint64_t find(uint32_t* arg0, uint32_t arg1, uint32_t arg2)
uint32_t tmp_15cb_10_1;
uint64_t rax_3;
/* r2dec proof: no individual construct is marked; 43 source obligations: 36 rendered, 7 elided, 0 refused; 17 statements rendered */
/* r2dec proof: no individual construct is marked; 43 source obligations: 36 rendered, 7 elided, 0 refused; 16 statements rendered */
{
tmp_15b4_0_1 = arg1;
if ((uint8_t)((int32_t)tmp_15b4_0_1 <= (int32_t)0U)) {
@@ -902,11 +879,11 @@ uint64_t find(uint32_t* arg0, uint32_t arg1, uint32_t arg2)
tmp_15cb_10_1 = r2sleigh_load_u32((void*)((uint64_t)arg0 + (uint64_t)rax_2 * 4U));
if ((uint8_t)((uint32_t)tmp_15cb_10_1 != (uint32_t)arg2)) {
if ((uint8_t)((int32_t)arg2 < (int32_t)tmp_15cb_10_1)) {
goto L1;
return (uint64_t)0xffffffffU;
} else {
rax_3 = (uint64_t)((uint64_t)rax_2 + (uint64_t)1U);
if ((uint8_t)((uint64_t)rax_3 == (uint64_t)rsi_1)) {
goto L1;
return (uint64_t)0xffffffffU;
} else {
rax_2 = rax_3;
}
@@ -920,7 +897,6 @@ uint64_t find(uint32_t* arg0, uint32_t arg1, uint32_t arg2)
}
}
{
L1: ;
return (uint64_t)0xffffffffU;
}
}
tests/db/census/shapes_gcc_x64_O0
+26 -46
View File
@@ -60,17 +60,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x404fe0U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x401014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x401014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
@@ -812,32 +811,24 @@ uint64_t shape_struct_array(uint32_t arg0, uint32_t arg1)
uint32_t tmp_401a90_18_1;
_Alignas(16) uint8_t frame[88];
/* r2dec proof: no individual construct is marked; 85 source obligations: 73 rendered, 7 elided, 0 refused, 5 compiler-inserted (assuming ub-free); 20 statements rendered; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 85 source obligations: 73 rendered, 7 elided, 0 refused, 5 compiler-inserted (assuming ub-free); 16 statements rendered; the result's second register is open: no call read decides it */
{
rdi_1 = (uint64_t)(uint32_t)arg0;
rsi_1 = (uint64_t)(uint32_t)arg1;
stack_m104_1 = 0;
for (; ; ) {
if ((uint8_t)((uint32_t)stack_m104_1 <= (uint32_t)7U)) {
r2sleigh_store_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)(uint32_t)stack_m104_1 * (uint64_t)2U) * 4U), (uint32_t)((uint32_t)((uint64_t)rdi_1 >> 0U) + (uint32_t)((uint64_t)((uint64_t)(int32_t)stack_m104_1 * (uint64_t)0xffffffff9e3779b1U) >> 0U)));
r2sleigh_store_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)((uint64_t)(uint32_t)stack_m104_1 * (uint64_t)2U) + (uint64_t)1U) * 4U), (uint32_t)((uint32_t)((uint64_t)rsi_1 >> 0U) ^ (uint32_t)((uint64_t)((uint64_t)(int32_t)stack_m104_1 * (uint64_t)0xffffffff9e3779b9U) >> 0U)));
stack_m104_1 = (uint32_t)((uint32_t)stack_m104_1 + (uint32_t)1U);
} else {
break;
}
while ((uint8_t)((uint32_t)stack_m104_1 <= (uint32_t)7U)) {
r2sleigh_store_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)(uint32_t)stack_m104_1 * (uint64_t)2U) * 4U), (uint32_t)((uint32_t)((uint64_t)rdi_1 >> 0U) + (uint32_t)((uint64_t)((uint64_t)(int32_t)stack_m104_1 * (uint64_t)0xffffffff9e3779b1U) >> 0U)));
r2sleigh_store_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)((uint64_t)(uint32_t)stack_m104_1 * (uint64_t)2U) + (uint64_t)1U) * 4U), (uint32_t)((uint32_t)((uint64_t)rsi_1 >> 0U) ^ (uint32_t)((uint64_t)((uint64_t)(int32_t)stack_m104_1 * (uint64_t)0xffffffff9e3779b9U) >> 0U)));
stack_m104_1 = (uint32_t)((uint32_t)stack_m104_1 + (uint32_t)1U);
}
{
stack_m100_1 = 0;
stack_m96_1 = 0;
for (; ; ) {
if ((uint8_t)((uint32_t)stack_m100_1 <= (uint32_t)7U)) {
tmp_401a78_18_1 = r2sleigh_load_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)(uint32_t)stack_m100_1 * (uint64_t)2U) * 4U));
tmp_401a90_18_1 = r2sleigh_load_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)((uint64_t)(uint32_t)((uint32_t)((uint32_t)stack_m100_1 + (uint32_t)3U) & (uint32_t)7U) * (uint64_t)2U) + (uint64_t)1U) * 4U));
stack_m100_1 = (uint32_t)((uint32_t)stack_m100_1 + (uint32_t)1U);
stack_m96_1 = (uint64_t)((uint64_t)((uint64_t)(uint32_t)16U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)(uint32_t)tmp_401a90_18_1 << (uint64_t)(uint32_t)16U)) ^ (uint64_t)((uint64_t)(uint32_t)tmp_401a78_18_1 + (uint64_t)((uint64_t)stack_m96_1 * (uint64_t)0x100000001b3U)));
} else {
break;
}
while ((uint8_t)((uint32_t)stack_m100_1 <= (uint32_t)7U)) {
tmp_401a78_18_1 = r2sleigh_load_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)(uint32_t)stack_m100_1 * (uint64_t)2U) * 4U));
tmp_401a90_18_1 = r2sleigh_load_u32((void*)((uint64_t)(frame + 0U) + (uint64_t)((uint64_t)((uint64_t)(uint32_t)((uint32_t)((uint32_t)stack_m100_1 + (uint32_t)3U) & (uint32_t)7U) * (uint64_t)2U) + (uint64_t)1U) * 4U));
stack_m100_1 = (uint32_t)((uint32_t)stack_m100_1 + (uint32_t)1U);
stack_m96_1 = (uint64_t)((uint64_t)((uint64_t)(uint32_t)16U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)(uint32_t)tmp_401a90_18_1 << (uint64_t)(uint32_t)16U)) ^ (uint64_t)((uint64_t)(uint32_t)tmp_401a78_18_1 + (uint64_t)((uint64_t)stack_m96_1 * (uint64_t)0x100000001b3U)));
}
{
rax_23 = stack_m96_1;
@@ -868,29 +859,21 @@ uint64_t shape_stack_buffer(uint64_t arg0, uint8_t arg1)
uint8_t tmp_401b4c_18_1;
_Alignas(16) uint8_t frame[88];
/* r2dec proof: no individual construct is marked; 74 source obligations: 62 rendered, 7 elided, 0 refused, 5 compiler-inserted (assuming ub-free); 17 statements rendered; the result's second register is open: no call read decides it */
/* r2dec proof: no individual construct is marked; 74 source obligations: 62 rendered, 7 elided, 0 refused, 5 compiler-inserted (assuming ub-free); 13 statements rendered; the result's second register is open: no call read decides it */
{
rsi_1 = (uint64_t)(uint8_t)arg1;
stack_m104_1 = 0;
for (; ; ) {
if ((uint8_t)((uint32_t)stack_m104_1 <= (uint32_t)63U)) {
r2sleigh_store_u8((void*)((uint64_t)(frame + 0U) + (uint64_t)(uint32_t)stack_m104_1 * 1U), (uint8_t)((uint64_t)((uint64_t)(uint32_t)((uint64_t)((uint64_t)(uint8_t)((uint32_t)(uint8_t)((uint32_t)((uint32_t)stack_m104_1 & (uint32_t)7U) >> 0U) & (uint32_t)(uint8_t)63U) >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)arg0 >> (uint64_t)(uint8_t)((uint32_t)(uint8_t)((uint32_t)((uint32_t)stack_m104_1 & (uint32_t)7U) >> 0U) & (uint32_t)(uint8_t)63U))) >> 0U) + (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rsi_1 >> 0U) * (uint64_t)(int32_t)stack_m104_1) >> 0U)) >> 0U));
stack_m104_1 = (uint32_t)((uint32_t)stack_m104_1 + (uint32_t)1U);
} else {
break;
}
while ((uint8_t)((uint32_t)stack_m104_1 <= (uint32_t)63U)) {
r2sleigh_store_u8((void*)((uint64_t)(frame + 0U) + (uint64_t)(uint32_t)stack_m104_1 * 1U), (uint8_t)((uint64_t)((uint64_t)(uint32_t)((uint64_t)((uint64_t)(uint8_t)((uint32_t)(uint8_t)((uint32_t)((uint32_t)stack_m104_1 & (uint32_t)7U) >> 0U) & (uint32_t)(uint8_t)63U) >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)arg0 >> (uint64_t)(uint8_t)((uint32_t)(uint8_t)((uint32_t)((uint32_t)stack_m104_1 & (uint32_t)7U) >> 0U) & (uint32_t)(uint8_t)63U))) >> 0U) + (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint64_t)rsi_1 >> 0U) * (uint64_t)(int32_t)stack_m104_1) >> 0U)) >> 0U));
stack_m104_1 = (uint32_t)((uint32_t)stack_m104_1 + (uint32_t)1U);
}
{
stack_m100_1 = 0;
stack_m96_1 = (uint64_t)0xcbf29ce484222325U;
for (; ; ) {
if ((uint8_t)((uint32_t)stack_m100_1 <= (uint32_t)63U)) {
tmp_401b4c_18_1 = r2sleigh_load_u8((void*)((uint64_t)(frame + 0U) + (uint64_t)(uint32_t)((uint32_t)((uint32_t)stack_m100_1 + (uint32_t)((uint32_t)stack_m100_1 * (uint32_t)4U)) & (uint32_t)63U) * 1U));
stack_m100_1 = (uint32_t)((uint32_t)stack_m100_1 + (uint32_t)1U);
stack_m96_1 = (uint64_t)((uint64_t)((uint64_t)stack_m96_1 ^ (uint64_t)(uint8_t)tmp_401b4c_18_1) * (uint64_t)0x100000001b3U);
} else {
break;
}
while ((uint8_t)((uint32_t)stack_m100_1 <= (uint32_t)63U)) {
tmp_401b4c_18_1 = r2sleigh_load_u8((void*)((uint64_t)(frame + 0U) + (uint64_t)(uint32_t)((uint32_t)((uint32_t)stack_m100_1 + (uint32_t)((uint32_t)stack_m100_1 * (uint32_t)4U)) & (uint32_t)63U) * 1U));
stack_m100_1 = (uint32_t)((uint32_t)stack_m100_1 + (uint32_t)1U);
stack_m96_1 = (uint64_t)((uint64_t)((uint64_t)stack_m96_1 ^ (uint64_t)(uint8_t)tmp_401b4c_18_1) * (uint64_t)0x100000001b3U);
}
{
rax_21 = stack_m96_1;
@@ -919,7 +902,7 @@ uint64_t shape_recurse_direct(uint8_t arg0, uint64_t arg1)
uint64_t rax_10;
uint64_t rax_12;
/* r2dec proof: no individual construct is marked; 36 source obligations: 26 rendered, 10 elided, 0 refused; 8 statements rendered */
/* r2dec proof: no individual construct is marked; 36 source obligations: 26 rendered, 10 elided, 0 refused; 9 statements rendered */
{
rax_2 = (uint64_t)(uint32_t)((uint32_t)((uint64_t)(uint8_t)arg0 >> 0U) & (uint32_t)15U);
if ((uint8_t)((uint64_t)rax_2 != (uint64_t)0U)) {
@@ -927,10 +910,9 @@ uint64_t shape_recurse_direct(uint8_t arg0, uint64_t arg1)
rax_9 = (uint64_t)((uint64_t)rax_2 - (uint64_t)1U);
rax_10 = shape_recurse_direct((uint8_t)rax_9, rdx_3);
rax_12 = (uint64_t)((uint64_t)rax_10 ^ (uint64_t)((uint64_t)(uint32_t)3U >= (uint64_t)64U ? (uint64_t)0U : (uint64_t)((uint64_t)rax_2 << (uint64_t)(uint32_t)3U)));
return (uint64_t)rax_12;
} else {
rax_12 = (uint64_t)((uint64_t)arg1 | (uint64_t)1U);
}
{
return (uint64_t)rax_12;
}
}
@@ -955,7 +937,7 @@ uint64_t shape_mutual_even(uint64_t arg0, uint64_t arg1)
uint64_t rax_9;
uint64_t rax_10;
/* r2dec proof: no individual construct is marked; 31 source obligations: 21 rendered, 10 elided, 0 refused; 8 statements rendered */
/* r2dec proof: no individual construct is marked; 31 source obligations: 21 rendered, 10 elided, 0 refused; 9 statements rendered */
{
if ((uint8_t)((uint64_t)arg0 != (uint64_t)0U)) {
rax_5 = (uint64_t)((uint64_t)((uint64_t)arg1 * (uint64_t)32U) - (uint64_t)arg1);
@@ -963,10 +945,9 @@ uint64_t shape_mutual_even(uint64_t arg0, uint64_t arg1)
rax_8 = (uint64_t)((uint64_t)arg0 - (uint64_t)1U);
rax_9 = shape_mutual_odd(rax_8, tmp_401c31_8_1);
rax_10 = rax_9;
return (uint64_t)rax_10;
} else {
rax_10 = (uint64_t)((uint64_t)arg1 ^ (uint64_t)0xa5a5a5a5U);
}
{
return (uint64_t)rax_10;
}
}
@@ -990,17 +971,16 @@ uint64_t shape_mutual_odd(uint64_t arg0, uint64_t arg1)
uint64_t rax_9;
uint64_t rax_10;
/* r2dec proof: no individual construct is marked; 31 source obligations: 21 rendered, 10 elided, 0 refused; 7 statements rendered */
/* r2dec proof: no individual construct is marked; 31 source obligations: 21 rendered, 10 elided, 0 refused; 8 statements rendered */
{
if ((uint8_t)((uint64_t)arg0 != (uint64_t)0U)) {
rdx_3 = (uint64_t)((uint64_t)((uint64_t)((uint64_t)arg1 * (uint64_t)16U) + (uint64_t)arg1) + (uint64_t)((uint64_t)arg0 * (uint64_t)4U));
rax_8 = (uint64_t)((uint64_t)arg0 - (uint64_t)1U);
rax_9 = shape_mutual_even(rax_8, rdx_3);
rax_10 = rax_9;
return (uint64_t)rax_10;
} else {
rax_10 = (uint64_t)((uint64_t)arg1 ^ (uint64_t)0x5a5a5a5aU);
}
{
return (uint64_t)rax_10;
}
}
tests/db/census/two_units_O0g
+2 -3
View File
@@ -33,17 +33,16 @@ uint64_t _init(void)
uint64_t rax_1;
uint64_t rax_3;
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 5 statements rendered; 1 never assigned, read as residuals (RAX_2) */
/* r2dec proof: 3 constructs are marked below; 22 source obligations: 8 rendered, 10 elided, 0 refused, 4 residual; 6 statements rendered; 1 never assigned, read as residuals (RAX_2) */
{
rax_1 = r2sleigh_load_u64((void*)(uint64_t)0x3fe8U);
if ((uint8_t)((uint64_t)rax_1 == (uint64_t)0U)) {
rax_3 = rax_1;
return (uint64_t)rax_3;
} else {
r2sleigh_residual_void(1); /* r2dec gap: CallNotRendered at 0x1014:4 covering 1 op (render::values): call to an indirect target: no arity is stated, only the registers written before it */
r2sleigh_residual_void(2); /* r2dec gap: ValueNotComputed at 0x1014:11 covering 1 op (render::values) */
rax_3 = r2sleigh_residual_u64(3);
}
{
return (uint64_t)rax_3;
}
}
tests/db/cmd/atomics
+12 -24
View File
@@ -156,21 +156,17 @@ uint32_t flags_set(uint32_t* flags, uint32_t bits)
uint8_t zf_2;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 28 source obligations: 25 rendered, 3 elided, 0 refused; 11 statements rendered */
/* r2dec proof: no individual construct is marked; 28 source obligations: 25 rendered, 3 elided, 0 refused; 9 statements rendered */
{
tmp_401140_0_1 = r2sleigh_load_u32((void*)(uint64_t)flags);
rax_2 = (uint64_t)(uint32_t)tmp_401140_0_1;
for (; ; ) {
do {
tmp_lane_401154_1_0_1 = (uint32_t)((uint64_t)rax_2 >> 0U);
tmp_401154_4_1 = r2sleigh_atomic_compare_exchange_u32_seq_cst((void*)(uint64_t)flags, (uint32_t)tmp_lane_401154_1_0_1, (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) | (uint32_t)bits));
zf_2 = (uint8_t)((uint32_t)tmp_lane_401154_1_0_1 == (uint32_t)tmp_401154_4_1);
rax_4 = zf_2 ? (uint64_t)((uint64_t)(uint32_t)((uint64_t)rax_2 >> 32U) << 32U | (uint64_t)(zf_2 ? (uint32_t)tmp_lane_401154_1_0_1 : (uint32_t)tmp_401154_4_1)) : (uint64_t)(uint32_t)tmp_401154_4_1;
if ((uint8_t)((uint32_t)zf_2 ^ (uint32_t)(uint8_t)1U)) {
rax_2 = rax_4;
} else {
break;
}
}
rax_2 = rax_4;
} while ((uint8_t)((uint32_t)zf_2 ^ (uint32_t)(uint8_t)1U));
{
return (uint32_t)rax_4;
}
@@ -187,22 +183,18 @@ uint32_t flags_clear(uint32_t* flags, uint32_t bits)
uint8_t zf_2;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 29 source obligations: 26 rendered, 3 elided, 0 refused; 12 statements rendered */
/* r2dec proof: no individual construct is marked; 29 source obligations: 26 rendered, 3 elided, 0 refused; 10 statements rendered */
{
tmp_lane_401160_0_1_1 = (uint32_t)~(uint32_t)bits;
tmp_401162_0_1 = r2sleigh_load_u32((void*)(uint64_t)flags);
rax_2 = (uint64_t)(uint32_t)tmp_401162_0_1;
for (; ; ) {
do {
tmp_lane_401174_1_0_1 = (uint32_t)((uint64_t)rax_2 >> 0U);
tmp_401174_4_1 = r2sleigh_atomic_compare_exchange_u32_seq_cst((void*)(uint64_t)flags, (uint32_t)tmp_lane_401174_1_0_1, (uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) & (uint32_t)tmp_lane_401160_0_1_1));
zf_2 = (uint8_t)((uint32_t)tmp_lane_401174_1_0_1 == (uint32_t)tmp_401174_4_1);
rax_4 = zf_2 ? (uint64_t)((uint64_t)(uint32_t)((uint64_t)rax_2 >> 32U) << 32U | (uint64_t)(zf_2 ? (uint32_t)tmp_lane_401174_1_0_1 : (uint32_t)tmp_401174_4_1)) : (uint64_t)(uint32_t)tmp_401174_4_1;
if ((uint8_t)((uint32_t)zf_2 ^ (uint32_t)(uint8_t)1U)) {
rax_2 = rax_4;
} else {
break;
}
}
rax_2 = rax_4;
} while ((uint8_t)((uint32_t)zf_2 ^ (uint32_t)(uint8_t)1U));
{
return (uint32_t)rax_4;
}
@@ -219,22 +211,18 @@ uint16_t parity_toggle(uint16_t* word, uint16_t mask)
uint8_t zf_2;
uint16_t tmp_lane_401194_11_0_1;
/* r2dec proof: no individual construct is marked; 30 source obligations: 27 rendered, 3 elided, 0 refused; 12 statements rendered */
/* r2dec proof: no individual construct is marked; 30 source obligations: 27 rendered, 3 elided, 0 refused; 10 statements rendered */
{
rsi_1 = (uint64_t)(uint16_t)mask;
tmp_401180_0_1 = r2sleigh_load_u16((void*)(uint64_t)word);
rax_2 = (uint64_t)(uint16_t)tmp_401180_0_1;
for (; ; ) {
do {
tmp_lane_401194_1_0_1 = (uint16_t)((uint64_t)rax_2 >> 0U);
tmp_401194_2_1 = r2sleigh_atomic_compare_exchange_u16_seq_cst((void*)(uint64_t)word, (uint16_t)tmp_lane_401194_1_0_1, (uint16_t)((uint32_t)((uint32_t)((uint64_t)rax_2 >> 0U) ^ (uint32_t)((uint64_t)rsi_1 >> 0U)) >> 0U));
zf_2 = (uint8_t)((uint16_t)tmp_lane_401194_1_0_1 == (uint16_t)tmp_401194_2_1);
tmp_lane_401194_11_0_1 = zf_2 ? (uint16_t)tmp_lane_401194_1_0_1 : (uint16_t)tmp_401194_2_1;
if ((uint8_t)((uint32_t)zf_2 ^ (uint32_t)(uint8_t)1U)) {
rax_2 = (uint64_t)(((uint64_t)rax_2 & (uint64_t)0xffffffffffff0000U) | ((uint64_t)tmp_lane_401194_11_0_1 & (uint64_t)0xffffU) << 0U);
} else {
break;
}
}
rax_2 = (uint64_t)(((uint64_t)rax_2 & (uint64_t)0xffffffffffff0000U) | ((uint64_t)tmp_lane_401194_11_0_1 & (uint64_t)0xffffU) << 0U);
} while ((uint8_t)((uint32_t)zf_2 ^ (uint32_t)(uint8_t)1U));
{
return (uint16_t)tmp_lane_401194_11_0_1;
}
tests/db/cmd/structure
+237
View File
@@ -0,0 +1,237 @@
# Terminal tails (doc/adr-structure-dominator-tree.md §5; tests/corpus/branchy.c, gcc -O0). The
# compiler joins every `return` of a function into one epilogue, and the value each arm returns
# arrives as a merge of distinct values. The source wrote a `return` in each arm.
NAME=a return of a merge of distinct values is written in each arm that falls to it
FILE=tests/coverage/pinned/branchy_gcc_x64_O0
CMDS=<<EOF
pdd @ sym.gate_one
pdd @ sym.inverted_guard
EOF
EXPECT=<<EOF
uint64_t gate_one(uint32_t arg0)
{
uint64_t rax_3;
/* r2dec proof: no individual construct is marked; 18 source obligations: 11 rendered, 7 elided, 0 refused; 5 statements rendered */
{
if ((uint8_t)((uint32_t)arg0 != (uint32_t)0xbeefU)) {
rax_3 = 0;
return (uint64_t)rax_3;
} else {
rax_3 = 1;
return (uint64_t)rax_3;
}
}
}
uint64_t inverted_guard(uint32_t arg0)
{
extern char g_sink[];
extern char g_note[];
uint64_t rax_5;
uint64_t rax_7;
/* r2dec proof: no individual construct is marked; 21 source obligations: 14 rendered, 7 elided, 0 refused; 8 statements rendered; 2 data object types refused */
{
if ((uint8_t)((int32_t)arg0 <= (int32_t)9U)) {
rax_7 = (uint64_t)0xffffffffU;
return (uint64_t)rax_7;
} else {
r2sleigh_store_u64((void*)&g_note, (uint64_t)(int32_t)((uint32_t)arg0 + (uint32_t)2U));
r2sleigh_store_u32((void*)&g_sink, (uint32_t)arg0);
rax_5 = r2sleigh_load_u64((void*)&g_note);
rax_7 = rax_5;
return (uint64_t)rax_7;
}
}
}
EOF
RUN
# A terminal block of at most four statements (a `return`, or a call that never returns) is
# written at each `goto` that reaches it, and the block goes once nothing falls into it. `classify`
# is a switch whose every case sets the result and jumps to the one epilogue.
NAME=a terminal block is written at each jump to it
FILE=tests/fixtures/rv_O0g
CMDS=<<EOF
pdd @ sym.classify
EOF
EXPECT=<<EOF
int32_t classify(int32_t x)
{
uint64_t rax_14;
/* r2dec proof: no individual construct is marked; 38 source obligations: 22 rendered, 16 elided, 0 refused; 18 statements rendered */
{
if ((uint8_t)((uint32_t)7U < (uint32_t)x)) {
} else {
switch ((uint32_t)x) {
case 7:
{
rax_14 = 87;
return (int32_t)rax_14;
}
case 5:
{
rax_14 = 65;
return (int32_t)rax_14;
}
case 4:
{
rax_14 = 54;
return (int32_t)rax_14;
}
case 3:
{
rax_14 = 43;
return (int32_t)rax_14;
}
case 2:
{
rax_14 = 32;
return (int32_t)rax_14;
}
case 1:
{
rax_14 = 21;
return (int32_t)rax_14;
}
case 0:
{
rax_14 = 10;
return (int32_t)rax_14;
}
case 6:
break;
}
}
{
rax_14 = (uint64_t)0xffffffffU;
}
{
return (int32_t)rax_14;
}
}
}
EOF
RUN
# A loop's exit that only one edge of the loop reaches, and whose dominator subtree no edge leaves,
# is written where that edge leaves the loop (doc/adr-structure-dominator-tree.md §4). The first
# exit stays after the loop, where `break` reaches it. `find_then_choose`'s match is a test and
# two returns of its own; `skip_then_find` leaves from inside its first loop with a store and a
# return, and from its second with a return.
NAME=an exit with one way in and a closed subtree is written where its edge leaves the loop
FILE=tests/fixtures/loop_exits_gcc_x64_O2
CMDS=<<EOF
pdd @ sym.find_then_choose
pdd @ sym.skip_then_find
EOF
EXPECT=<<EOF
uint64_t find_then_choose(uint64_t arg0, uint32_t arg1, uint32_t arg2)
{
extern char sink[];
uint64_t rdi_1;
uint8_t tmp_40105e_0_1;
uint32_t tmp_lane_40105e_1_0_1;
/* r2dec proof: no individual construct is marked; 44 source obligations: 34 rendered, 10 elided, 0 refused; 16 statements rendered; 1 data object type refused */
{
rdi_1 = arg0;
for (; ; ) {
tmp_40105e_0_1 = r2sleigh_load_u8((void*)rdi_1);
tmp_lane_40105e_1_0_1 = (uint32_t)(int8_t)tmp_40105e_0_1;
if ((uint8_t)((uint8_t)tmp_40105e_0_1 != (uint8_t)0U)) {
r2sleigh_store_u32((void*)&sink, (uint32_t)tmp_lane_40105e_1_0_1);
if ((uint8_t)((uint32_t)tmp_lane_40105e_1_0_1 == (uint32_t)arg1)) {
if ((uint8_t)((uint32_t)arg2 == (uint32_t)0U)) {
r2sleigh_store_u32((void*)&sink, (uint32_t)2U);
return 20;
} else {
r2sleigh_store_u32((void*)&sink, (uint32_t)1U);
return 10;
}
} else {
rdi_1 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
}
} else {
break;
}
}
{
r2sleigh_store_u32((void*)&sink, (uint32_t)3U);
return 0;
}
}
}
uint64_t skip_then_find(uint64_t arg0, uint32_t arg1, uint32_t arg2)
{
extern char sink[];
uint64_t rcx_2;
uint64_t rdi_1;
uint8_t tmp_4010bb_0_1;
uint64_t rax_1;
uint64_t rax_3;
uint64_t rdx_2;
uint32_t tmp_lane_4010e2_2_0_1;
uint8_t tmp_4010d0_8_1;
uint32_t tmp_lane_4010b0_6_0_1;
/* r2dec proof: no individual construct is marked; 79 source obligations: 69 rendered, 10 elided, 0 refused; 26 statements rendered; 1 data object type refused */
{
rcx_2 = 0;
rdi_1 = arg0;
for (; ; ) {
tmp_4010bb_0_1 = r2sleigh_load_u8((void*)rdi_1);
rax_1 = (uint64_t)(uint32_t)(int8_t)tmp_4010bb_0_1;
if ((uint8_t)((uint8_t)tmp_4010bb_0_1 == (uint8_t)32U)) {
tmp_lane_4010b0_6_0_1 = (uint32_t)((uint32_t)((uint64_t)rcx_2 >> 0U) + (uint32_t)1U);
if ((uint8_t)((int32_t)arg2 < (int32_t)tmp_lane_4010b0_6_0_1)) {
r2sleigh_store_u32((void*)&sink, (uint32_t)tmp_lane_4010b0_6_0_1);
return (uint64_t)0xffffffffU;
} else {
rcx_2 = (uint64_t)(uint32_t)tmp_lane_4010b0_6_0_1;
rdi_1 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
}
} else {
break;
}
}
{
if ((uint8_t)((uint8_t)tmp_4010bb_0_1 == (uint8_t)0U)) {
} else {
rax_3 = rax_1;
rdx_2 = rdi_1;
while ((uint8_t)((uint32_t)((uint64_t)rax_3 >> 0U) != (uint32_t)arg1)) {
tmp_4010d0_8_1 = r2sleigh_load_u8((void*)(uint64_t)((uint64_t)rdx_2 + (uint64_t)1U));
if ((uint8_t)((uint8_t)tmp_4010d0_8_1 == (uint8_t)0U)) {
return 0;
} else {
rax_3 = (uint64_t)(uint32_t)(int8_t)tmp_4010d0_8_1;
rdx_2 = (uint64_t)((uint64_t)rdx_2 + (uint64_t)1U);
}
}
{
tmp_lane_4010e2_2_0_1 = (uint32_t)((uint64_t)((uint64_t)(uint32_t)((uint32_t)((uint64_t)rcx_2 >> 0U) - (uint32_t)((uint64_t)rdi_1 >> 0U)) + (uint64_t)rdx_2) >> 0U);
r2sleigh_store_u32((void*)&sink, (uint32_t)tmp_lane_4010e2_2_0_1);
return (uint64_t)(uint32_t)((uint32_t)tmp_lane_4010e2_2_0_1 + (uint32_t)100U);
}
}
{
return 0;
}
}
}
}
EOF
RUN
tests/db/cmd/tiers
+5 -9
View File
@@ -120,7 +120,7 @@ uint64_t fnv1a32(uint64_t arg0, uint64_t arg1)
uint64_t rdi_2;
uint64_t rax_4;
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 14 statements rendered */
/* r2dec proof: no individual construct is marked; 40 source obligations: 32 rendered, 8 elided, 0 refused; 12 statements rendered */
{
if ((uint8_t)((uint64_t)arg1 == (uint64_t)0U)) {
return (uint64_t)0x811c9dc5U;
@@ -128,17 +128,13 @@ uint64_t fnv1a32(uint64_t arg0, uint64_t arg1)
rsi_1 = (uint64_t)((uint64_t)arg1 + (uint64_t)arg0);
rax_2 = (uint64_t)0x811c9dc5U;
rdi_1 = arg0;
for (; ; ) {
do {
tmp_401348_0_1 = r2sleigh_load_u8((void*)rdi_1);
rdi_2 = (uint64_t)((uint64_t)rdi_1 + (uint64_t)1U);
rax_4 = (uint64_t)(uint32_t)((uint64_t)((uint64_t)(int32_t)((uint32_t)(uint8_t)tmp_401348_0_1 ^ (uint32_t)((uint64_t)rax_2 >> 0U)) * (uint64_t)0x1000193U) >> 0U);
if ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2)) {
rax_2 = rax_4;
rdi_1 = rdi_2;
} else {
break;
}
}
rax_2 = rax_4;
rdi_1 = rdi_2;
} while ((uint8_t)((uint64_t)rsi_1 != (uint64_t)rdi_2));
{
return (uint64_t)rax_4;
}
tests/decbench/README.md
+109 -7
View File
@@ -7,7 +7,9 @@ signature types against DWARF, and assembly similarity after recompiling with
the original toolchain. Its Union score is the share of functions perfect on at
least one of the three.
**No valid number exists yet.** Everything this directory recorded before
**Valid numbers start at [RESULT-2026-10-17.md](RESULT-2026-10-17.md) and are
corrected by [RESULT-2026-10-23.md](RESULT-2026-10-23.md): the first scored no bash
row for any decompiler.** Everything this directory recorded before
2026-09-25 -- `baseline.json`, the tables that used to be here, and the figures
in the since-deleted planning documents -- was measured
through the deleted radare2 plugin with `decbench run`, which hands the
@@ -73,21 +75,121 @@ function its own from the moment r2s starts on it
or `harness: r2s <ending> while rendering <addr>` (a crash or deadline, after
which r2s restarts at the next function) -- and
`rendered + declined == requested` is checked.
* **Checkpoints.** The partial result is pickled after every function, so the
driver's hard kill keeps what was answered.
* **Checkpoints.** The partial result is pickled to the driver's progress path
every `R2SLEIGH_CHECKPOINT_SECONDS` (default 30) and at the end, so the driver's
hard kill loses one interval of answers at most. (One pickle per function was
quadratic: 2.5 MB each on bzip2, 18 MB on bash, and the backend took 9 to 13
times r2s alone.) A re-run that finds the checkpoint of the same r2s build,
binary bytes and targets asks r2s only for the addresses it has no answer for;
the official driver gives each attempt a fresh path, so this serves a manual
re-run. `r2s_batch.py` hands each answer over while r2s is still running, which
is what makes a checkpoint taken mid-process hold anything.
* **Fails closed.** A binary carrying `.symtab` or any debug section
(`.debug_*`, `.zdebug_*`, `.gnu_debugdata`) is declined
whole: nothing r2s says about it is admissible.
* **Structured facts.** `variables` and `line_mappings` come from `pddj`.
DecBench uses address correspondence for `type_match` only for the backends
on its `ADDRESS_CORRESPONDENCE_BACKENDS` allowlist; until `r2sleigh_native`
is on it upstream, locals are matched by the legacy offset rule, fed by the
stack offsets `pddj` states.
`r2sleigh_native` is not on DecBench's `ADDRESS_CORRESPONDENCE_BACKENDS`
allowlist, but a function whose `line_mappings` carry instruction addresses
is scored by address correspondence anyway (`has_line_address_evidence`):
argument position, then stack offset, then address overlap of the lines
naming a variable. In the 2026-10-23 run that is every function with a
`type_match` row (the legacy offset rule, fed by `pddj`'s stack offsets,
scored 20 functions of zlib in the 2026-10-17 run and none now).
Declines and residual counts are written per binary to
`r2sleigh-refusals-*.json` (schema 4) beside the run, and
`census_decbench.py` ranks their causes.
Memory and coverage
-------------------
Evaluating r2s's `bash` (a 13 MB C file for 1,540 functions) takes 5.2 GB in one
`evaluate_project` worker. Four workers on the 8 GB benchmark host are killed by
the OOM killer, `evaluate_project` then returns `{}` for the binary, and the
scoreboard silently drops it: the 2026-10-17 run scored no bash row for any
decompiler (angr kept 692 rows of one build). Two gates in `run_gate.py` now stop
that, and `run_decbench.sh` runs both:
* `--workers` above 1 is refused for a binary whose decompiled C exceeds 8 MB
(exit 72): before the run, for C already in `--out`; inside the driver, before
`evaluate_project` (`decbench_cli.py` installs the guard, and it raises
`SystemExit` because the driver catches `Exception` there and carries on with
`{}`); and by `evaluate_checkpointed.py`. Run `bash` with `--workers 1`.
* After the driver, every binary it decompiled in this run needs an
`evaluated/<binary>.toml` that carries a score key of each decompiler that
rendered functions of it; a missing file, the 16-byte `binary = "bash"` and a
file that scores only other decompilers stop the run (exit 71) and name the
cells. Checked on the 2026-10-17 tree it reports 55 of 192 cells, all bash.
`report_decbench.py` is the third gate. It compares decompilers only over the rows
every compared decompiler scored (a row is scored when the results hold a metric
value for it), prints how many rows that left out and, per decompiler and per
project, how many rows have no score, and exits 73 when a compared decompiler
scored nothing in a project (`--min-scored-fraction F` raises that to fewer than
F of the project's rows; `--compare a,b` picks the decompilers, by default every
one the results name, and `run_decbench.sh --compare` forwards it). On the
2026-10-17 results it stops at `kuna scored 0 of 4144 rows of bash`; on the
2026-10-23 results it passes and reports 3,814 common rows of 6,978 (angr
scored 1,285 of bash's 4,131). The scoreboard's own denominator is every row
*anyone* scored, so a row only one decompiler has counts as a miss for the
others (`compare_runs.py --exclude-project bash` is the same cut for a tree that
has none). A re-run of one decompiler is
`DECBENCH_REDO_DECOMPILERS=r2sleigh_native` on a copy of the tree; DecBench's
`finalize_results.py --allow-drops` is needed when functions that used to be
declined (rows named by their address) are now rendered.
Analysis tools
--------------
All read a results tree (`checkpoints/*.pkl`, `function_results.json`) and need the
DecBench venv (`PYTHONPATH=<decbench checkout>`); only `evaluate_checkpointed.py` writes
into the tree (the scores it adds to a checkpoint).
* `compare_runs.py`: the four-decompiler table, the delta against an earlier tree
(`--old`), the same over a row subset (`--exclude-project bash`), paired
better/worse/newly-perfect counts (`--paired`), the goto/residual split.
* `type_causes.py`: one verdict per DWARF variable from DecBench's own matcher
(asserted equal to the stored `type_match`), grouped by cause with the number of
functions each cause alone keeps imperfect; `universe` says why a function has no row.
* `ged_causes.py`, `counterfactual_ged.py`: block-graph deltas of the not-GED-perfect
functions, and a re-score after a respelling, to bound what a renderer change gains.
* `mined_set.py`: the mined-set comparison of two r2s builds (next section).
* `run_gate.py`: the lost-evaluation and worker gates `run_decbench.sh` runs.
* `evaluate_checkpointed.py`: evaluates decompilers already in a checkpoint (the driver
only evaluates what it has just run); `r2s_timing.py`: r2s alone, without the backend's
checkpoint, with peak RSS; `show_function.py`: source, rendering, scores and DWARF of one function.
The mined-set comparison
------------------------
`mined_set.py` is how a structurer change is graded (ROADMAP D47): every function of
the pinned mining set, bzip2, zlib and grep, at O2 and O2-noinline, rendered by two
r2s builds and scored with DecBench's GED, then paired. It reads a results tree the
driver wrote (`<tree>/<opt>/<project>/{stripped,compiled,decompiled}`: the targets
are the addresses its `decompiled/*.toml` name) and runs where DecBench and Joern
are installed.
```
tests/decbench/mined_set.py dump --tree RUN --r2s base/r2s --out base --set mining
tests/decbench/mined_set.py dump --tree RUN --r2s new/r2s --out new --set mining
../decbench-venv/bin/python tests/decbench/mined_set.py score --tree RUN --dump base --out base.json --decbench ../decbench
../decbench-venv/bin/python tests/decbench/mined_set.py score --tree RUN --dump new --out new.json --decbench ../decbench
tests/decbench/mined_set.py pair base.json new.json # -u: one entry per function; -v: name them
```
`pair` prints, over the functions both runs scored: perfect count with the ones newly
perfect and lost, mean and total GED with better, worse and same, gotos (all, and over
the scored functions) and the functions with a goto, the functions whose text changed,
refusals (newly refused) and the functions with no source to score against. Each
rendering is scored against the CFG of the unit the compiled binary's DWARF names as the
owner of its address (the driver's `source_function_owners`, so the same
`DECBENCH_DWARF_ABSTRACT_ORIGIN` as the run); a function with no owner, or whose owner
yields no CFG, is not scored and is counted, never matched by name. `--set heldout`
selects the other five projects (bash, diffutils, findutils, gzip, libexpat): those are
run nightly and at an item's exit, never to choose a rule, and `pair` refuses to compare
a mining run with a held-out run. The source CFGs are cached beside the scores (`srccache/`, by unit and function);
delete it when the sources are recompiled.
Local measurement
-----------------
tests/decbench/RESULT-2026-10-17.md
+2
View File
@@ -1,5 +1,7 @@
# DecBench, 2026-10-17: the first valid run with r2s
> **Superseded by [RESULT-2026-10-23.md](RESULT-2026-10-23.md).** This run scored no bash row for r2s, kuna or ghidra (the OOM killer ended the evaluation worker) and counted the 692 bash rows only angr had as misses for the other three. Read its ranking as the ranking of the 2,736 non-bash rows only.
Recorded from `/root/decbench-run/scoreboard.toml` on contabo (generated 2026-10-17 13:38, DecBench protocol,
stripped inputs). Projects: bash, bzip2, diffutils, findutils, grep, gzip, libexpat, zlib. Optimisation levels:
O2 and O2-noinline. 48 binaries, 6,995 functions, 3,428 scored by all four decompilers. The r2s revision is
tests/decbench/RESULT-2026-10-23.md
+339
View File
@@ -0,0 +1,339 @@
# DecBench, 2026-10-23: r2s at the structurer stack's tip, and a correction to 2026-10-17
r2s revision `8cff06fe` (`origin/structure-tails`), release build on contabo (gcc 13.3, rustc 1.99),
`sha256:3764e777a7d6726fc65ada5c59a51433680591ea4a204a68208cea17b37d777d`. DecBench `ae070b8`, the checkout the
2026-10-17 run used. Official protocol: `scripts/run_benchmark.py` through `tests/decbench/run_decbench.sh`, stripped
inputs, DWARF `low_pc` targets; the script's sha256 gate passed. Same compiled binaries as 2026-10-17 (bash, bzip2,
diffutils, findutils, grep, gzip, libexpat, zlib; O2 and O2-noinline; 48 binaries). Only r2s was re-run, with
`DECBENCH_REDO_DECOMPILERS=r2sleigh_native` on a copy of the 2026-10-17 tree, so angr, kuna and ghidra are the
2026-10-17 decompilations. `merge_decbench.py` was not used: it merges disjoint project and opt cells, and the driver's
resume already merges decompilers. DecBench's `reeval_*` overlays were not applied, as on 2026-10-17.
## Answer
1. **r2s did not move.** On the 2,736 rows both runs scored: union-perfect 689 to 695 (+6), GED perfect 589 to 594,
type_match perfect 90 to 91, byte_match perfect 215 to 216. Functions with a goto fell 1,054 to 732 (8,353 to 5,366
gotos) and GED-perfect rose by 5.
2. **The 2026-10-17 record is wrong, not only stale.** It scored no bash row for any decompiler except 692 angr rows
(bash is 4,144 of 6,995 rows). The host's OOM killer killed the evaluation worker (journal, Oct 17 11:24:45 and
12:01:41, a 3.5 to 4.0 GB python), `evaluate_project` returned `{}` for every bash binary, and the scoreboard dropped
them without a message. "3,428 scored by all four" was 2,736 non-bash rows plus 692 bash rows only angr had; those
692 count as misses for the other three, and angr was perfect on 281 of them. On the 2,736 rows all four have,
kuna is first on union (835, 30.5%), angr second (834), ghidra third (715), r2s fourth (689, 25.2%).
3. **This run scored bash for all four** (r2s re-run with one worker; kuna, ghidra and angr evaluated from their
2026-10-17 decompilations, see Method). On the full 6,978 rows r2s is third on union (28.0%; kuna 32.0%, ghidra
29.3%, angr 18.8%), first on byte_match, third on GED and on type_match.
4. **type_match.** 67.6% of r2s's scored functions (4,182 of 6,190) cannot be perfect for any decompiler: a DWARF local
with no code address (2,255 functions) or a project-named struct type (3,020). Of the other 2,008 r2s is perfect
on 328. The top single causes among them: a parameter r2s does not have (315 functions where it is the only
cause), a pointer typed as an integer (248), an integer of the wrong width (209), locals r2s names as 4 objects
each (138). Details in section 3.
5. **GED.** r2s is perfect on 86.8% of non-bash straight-line sources (488 of 562; angr and kuna 497) and on 5.1% of
branching ones (106 of 2,061; angr 192, kuna 188, ghidra 125). No tool is perfect on a source of 20 or more blocks
(598 rows; 2 of 1,345 over all rows). Closing the gap to the best other tool on every row it wins is worth 166
union rows on non-bash and 366 on all rows; the respellings measured here are worth 26.
6. **Next, ranked** at the end. First two: a run that loses rows must fail (a gate), and a wrapper function whose
body ends in `jmp import` is not an import stub (63 refused rows; some other tool is perfect on every one, on GED for 51).
## Method
* Build: `cargo build --release -p r2s --features sleigh` of `decbench-2` (= `origin/structure-tails`) in
`/root/db2-target`; 4 m 36 s.
* Non-bash projects: `run_decbench.sh --workers 4 --project bzip2 diffutils findutils grep gzip libexpat zlib`.
* bash: `run_decbench.sh --workers 1 --project bash`, plus a 10 GB swap file on the host. One evaluation worker for r2s's
bash needs 5.2 GB (measured); four do not fit in 8 GB. The first attempt (four workers) lost the bash evaluation to
the OOM killer again, which is how the cause was found.
* `finalize_results.py --allow-drops` rebuilt `scoreboard.toml`. Its coverage guard fired on two finalizations (3 slices,
then 2): 17 rows named by a hex address (functions r2s declined on 2026-10-17) disappeared because the functions are now
rendered under their DWARF names. Every dropped row is a function newly rendered; no function was lost (4 non-bash,
13 bash newly rendered).
* kuna, ghidra and angr on bash: `tests/decbench/evaluate_checkpointed.py` calls the driver's own
`project_source_functions`, `extract_source_cfgs` and DecBench's `evaluate_project` on the stored decompilations,
one worker, because the driver evaluates only what it has just decompiled. Their decompilations are unchanged.
angr has output for 1,285 of 4,144 bash rows (the 2026-10-17 timeouts), which is why it ranks last on the full set.
* The scoreboard's universe is every row any decompiler scored, so a row only one decompiler has is a miss for the
others. The tables below say which rows they cover.
## 1. Tables
Full tree, 6,978 rows (the first run to score all four on all eight projects):
| Decompiler | Union-perfect | byte_match perfect | GED perfect (mean) | type_match perfect (mean) |
|---|---|---|---|---|
| kuna | 2169 of 6788 (32.0%), rank 1 | 465 (6.9%), rank 2 | 1830 of 6453 (28.4%), 17.7, rank 1 | 539 of 6365 (8.5%), 0.268, rank 1 |
| ghidra | 1991 (29.3%), rank 2 | 330 (4.9%), rank 3 | 1605 (24.9%), 19.2, rank 2 | 491 (7.7%), 0.295, rank 2 |
| r2sleigh_native | 1899 (28.0%), rank 3 | 569 (8.4%), rank 1 | 1547 (24.0%), 29.9, rank 3 | 328 (5.2%), 0.192, rank 3 |
| angr | 1278 (18.8%), rank 4 | 234 (3.4%), rank 4 | 1073 (16.6%), 19.2, rank 4 | 250 (3.9%), 0.287, rank 4 |
Non-bash, 2,736 rows (the only population the 2026-10-17 run scored for everyone; its numbers recomputed here):
| Decompiler | Union-perfect | byte_match perfect | GED perfect (mean) | type_match perfect (mean) |
|---|---|---|---|---|
| kuna | 835 of 2736 (30.5%), rank 1 | 173 (6.3%), rank 2 | 724 of 2732 (26.5%), 18.7, rank 1 | 164 of 2664 (6.2%), 0.251, rank 1 |
| angr | 834 (30.5%), rank 2 | 155 (5.7%), rank 4 | 718 (26.3%), 19.9, rank 2 | 140 (5.3%), 0.287, rank 2 |
| ghidra | 715 (26.1%), rank 3 | 172 (6.3%), rank 3 | 602 (22.0%), 20.9, rank 3 | 121 (4.5%), 0.279, rank 3 |
| r2sleigh_native | 695 (25.4%), rank 4 | 216 (7.9%), rank 1 | 594 (21.7%), 39.0, rank 4 | 91 (3.4%), 0.185, rank 4 |
bash alone, 4,052 rows: kuna 1334 union (32.9%), ghidra 1276 (31.5%), r2s 1204 (29.7%), angr 444 (11.0%); GED perfect
kuna 29.7%, ghidra 27.0%, r2s 25.6%, angr 9.5%; type_match perfect kuna 375 (10.1%), ghidra 370 (10.0%), r2s 237
(6.4%), angr 110 (3.0%); byte_match r2s 353 (8.7%), kuna 292, ghidra 158, angr 79.
r2s against 2026-10-17, same 2,736 rows (the full-tree jump from 689 to 1,899 is the added bash population, not r2s):
| Metric | 2026-10-17 | now | change |
|---|---|---|---|
| union-perfect | 689 | 695 | +6 |
| byte_match perfect | 215 | 216 | +1 |
| GED perfect, mean | 589, 40.66 | 594, 39.04 | +5, -1.62 |
| type_match perfect, mean | 90, 0.184 | 91, 0.185 | +1, +0.001 |
| rows with no output (non-bash) | 230 | 222 | -8 |
Rows with no output, all 6,978 rows: r2s 380 (414 of 6,995 before: 17 hex-named rows left the table and 17 functions are
newly rendered), ghidra 323, kuna 190, angr 2,957 (each 17 lower than before only because those rows left).
### Per-function split (GED, as in the 2026-10-17 note)
| Decompiler | Scored | With a goto | GED-perfect, no goto | GED-perfect, a goto |
|---|---|---|---|---|
| angr | 4,017 | 21.4% | 33.9% | 0.5% (4 of 861) |
| kuna | 6,453 | 21.4% | 36.1% | 0.0% (0 of 1,378) |
| ghidra | 6,321 | 27.4% | 35.0% | 0.0% (0 of 1,732) |
| r2sleigh_native | 6,269 | 26.4% | 33.4% | 0.2% (4 of 1,653) |
| r2s, non-bash (2026-10-17 in brackets) | 2,623 (2,619) | 28.0% (40.2%) | 31.5% (37.4%) | 0.0% (0 of 735) (0.4%, 4 of 1,054) |
r2s GED-perfect by residual count, all rows: none 38.5% (2,580 functions), one or two 22.9% (284), three or more 14.3%
(3,405, 54% of scored). Non-bash 31.8% / 25.0% / 13.5% (2026-10-17: 31.4% / 29.3% / 13.1%).
GED-perfect with no goto fell from 37.4% to 31.5% because 319 functions left the goto bucket, not because 37.4% got
worse: the perfect count rose 589 to 594. Within a source-size band the residual count barely separates perfect from
not (0 to 4 blocks 66% with none against 57% with three or more; 5 to 9 blocks 7% against 3.5%; 10 or more blocks
0 to 2%); size does.
### What moved and what did not (r2s, paired, non-bash, 2,619 functions scored in both)
* Moved: gotos 8,353 to 5,366; functions with a goto 1,054 to 732 (bash, no 2026-10-17 score: 1,351 to 1,008 and
14,568 to 11,197 gotos). GED better on 838 functions, worse on 261, equal on 1,520. GED-perfect: 16 rows newly (10
functions), 11 rows lost (5 functions). 17 more functions rendered (4 non-bash, 13 bash), none lost.
* The 11 lost rows: 10 are `for (;;) { ...; if (c) { copies } else { break; } }` rewritten to `do { copies } while (c)`
(gzip and zlib `init_block` 7 rows, bzip2 `uInt64_to_double`, `prepare_new_block`, diffutils cmp `usage`): the latch
copies stage, the cost the ADR recorded ("438 loops to do-while for 5 perfect"). One row, grep `skip_devices`, is a
select `x = c ? 1 : e; return x;` now spelled as `if (c) { return 1; } else { ... }` by the joined-return stage.
* Did not move: type_match 10 better, 3 worse, 2,536 equal (+1 perfect); byte_match 261 better, 167 worse (+1 perfect);
residual sites 50,855 to 50,822 (non-bash), 109,281 to 109,851 (bash); union +6.
### Time and memory
r2s alone (`tests/decbench/r2s_timing.py`: the same `s <addr>; pddj` stream through the same runner, no checkpoint, one
binary at a time, host shared with another job at load 1.7): all 48 binaries, 6,521 functions asked, 6,339 rendered,
518.6 s in all; bash 306 s (the `bash` binary 140 s for 1,582 functions, 138 s for 2,338), zlib 88 s, bzip2 25 s,
diffutils 28 s, findutils 26 s, grep 21 s, gzip 21 s, libexpat 2 s. Peak RSS of r2s: 252 MB (bash), at most 182 MB
for any other project.
The driver's wall time for the same work, per project (decompile, O2 + O2-noinline): bash 2,976 s (one worker),
bzip2 132, zlib 86, findutils 52, grep 49, gzip 45, diffutils 34, libexpat 4: 3,378 s, 6.5 times r2s's. For bash the
backend takes 9 to 13 times r2s (1,253 and 1,723 s against about 140): `r2sleigh_raw.py` rebuilds and pickles the whole
partial result after every function (strace: 2.5 MB rewritten per function on bzip2, 18 MB on bash), which is
quadratic. Evaluation of r2s's bash: 443 s and 624 s with one worker, 5.2 GB peak in the worker; r2s writes a 13.1 and
13.9 MB C file for bash against ghidra 2.5 and 2.6, kuna 2.2, angr 1.0 MB.
## 2. How type_match is computed (DecBench `ae070b8`)
* Population: every `DW_TAG_formal_parameter` and `DW_TAG_variable` with any `DW_AT_location` of the source function,
including variables of inlined subroutines and lexical blocks. The return type is not scored.
* Score per function: tp / (tp + fp + fn); perfect is 1.0. A decompiler variable with no source partner costs nothing,
so a declared but unused local, an extra temporary, or a longer frame costs nothing.
* Correspondence for r2s is the address path (`has_line_address_evidence`, every scored function; the legacy offset
rule scored 20 zlib functions on 2026-10-17 and none now): a parameter by ABI position when both sides have exactly
one; a stack variable by offset; then the weighted Dice of instruction addresses (r2s's variable owns the addresses
of the lines that name it), accepted at 0.1 with a 0.03 margin over the next candidate.
* A source variable with no address, no stack offset and no argument index is `unobservable` and counts as fn.
* Types: signedness and qualifiers are stripped, `uintN_t` and `size_t` map to widths, a width must be exact, a pointer's
pointee must match, and a struct, union or typedef-to-struct matches only by its DWARF name.
`tests/decbench/type_causes.py` re-runs DecBench's own `TypeMatchMetric` over the checkpoints, asserts each function's
tp/fp/fn equals the stored value (it does for all 22,640 function rows), and records one verdict per DWARF variable.
## 3. Why r2s's type_match is low
Variables: 37,845 DWARF variables over r2s's 6,190 scored functions: tp 6,448 (17.0%), fp 12,435, fn 18,962.
| Class (variables) | Verdict | Functions involved | Only cause (ceiling) | Net union (not already byte/GED-perfect) |
|---|---|---|---|---|
| CAP unobservable (9,722) | DWARF local of an inlined glibc wrapper (`__fmt`, `__dest`, `__len`, `__stream`): no address | 2,255 | 186 | 147 |
| CAP project-named type (4,783) | `z_stream *`, `struct predicate *`: a name the stripped binary does not carry | 3,020 | 608 | 327 |
| parameter missing (1,437) | no r2s parameter at that position | 812 | **315** | 85 |
| pointer typed as integer (3,537) | `char *` etc. declared `uint64_t` | 2,318 | **248** | 103 |
| integer width (2,588) | `int` declared `uint64_t`, `UChar` declared `uint32_t` | 1,762 | **209** | 142 |
| register local unmatched (5,494) | a DWARF local overlaps a median of 14 r2s objects, best Dice 0.23 | 2,342 | **138** | 126 |
| library-named type (522) | `FILE *`, `struct stat`, `DIR *` | 426 | 36 | 20 |
| stack local / frame object (2,788) | `int` local whose address is taken declared `uint8_t frame[40]` | 1,149 | 16 | 15 |
| wrong pointee (373) | `char **` declared `uint64_t *` | 336 | 10 | 5 |
| bool, enum, float, int as pointer (153) | | 124 | 5 | 5 |
Reading: "only cause" is the number of functions that turn type_match-perfect if that class alone is fixed. Fixing in the
order arity, pointer, width, register locals, frame, library, pointee, rest turns +315, +283, +318, +364, +140, +123,
+101, +36 functions perfect: 328 to 2,008 (every uncapped function). Net union counts only functions not already
byte_match or GED perfect: greedy 1,056 over the same order reordered by union (width +142, register locals +185,
pointer +240, arity +158, frame +129, library +91, pointee +79, rest +32).
r2s against angr, same 3,719 functions both score (r2s perfect 157, angr 244; both 122; angr only 122; r2s only 35).
On the 122 angr-only functions r2s fails on: pointer as integer 58 (34 only cause), width 44 (31), register local 23,
library-named 13, arity 10 (8). angr's own failures there are mostly pointee (608 functions involved) and bool, enum, float or int-as-pointer (383);
it has half of r2s's arity trouble (211 against 414 functions) and 70% of its register-local trouble (1,071 against
1,522). r2s emits 83,246 variables for 16,842 DWARF variables on non-bash (4.9 per DWARF variable; angr 2.1, ghidra
1.3, kuna 1.1).
### The classes, by cause and owner
* **Parameter missing (315 / 85).** 410 of the 812 functions have no residual: the body never reads the parameter
(`mySignalCatcher(int n)`, `BZ2_bzflush(void *)`, findutils `pred_false`); only a caller or a function-pointer slot
states it, so this is not a ceiling until r2engine has that evidence. 402 have a residual: a call whose arity is a
floor (71), an import with no declaration (121: `__errno_location`), a variadic tail (99: `error(0, errno, "%s")`
drops `name`), another unrendered call (77), a tail transfer not rendered (34: `deflateInit_` is
`fcn_3480(); return residual`). Owner: r2ssa call contracts (P7) and r2abi (prototypes).
* **Pointer as integer (248 / 103) and library-named type (36 / 20).** The call already states the type and the value
does not take it: `fatal` passes `(const char *)arg0` to `dcgettext` and declares `uint64_t arg0`; `myMalloc` assigns
`malloc`'s `void *` result to `uint64_t rax_1`. 156 of the 1,015 non-bash pointer parameters declared as integers are cast to a
pointer type in their own body. r2abi gives width-only prototypes for the stdio and multibyte imports:
`uint64_t fclose(uint64_t)`, `fflush`, `fileno`, `putc`, `fputs`, `fopen`, `mbrtowc`, `mbrlen`, `__fprintf_chk(...)`
(23 imports, 945 renderings), and `error` without `...`. Owner: r2types (a callee parameter's type is evidence
for the value passed in; a call result takes the return type) over r2abi's prototypes.
* **Integer width (209 / 142).** Declared at the register's width (`wordchar(wint_t)` is `uint64_t arg0` and the
body reads `(uint32_t)arg0`; local loop counters `int i` are `uint64_t`), or narrowed to the byte the body used
(`mmed3(UChar, UChar, UChar)` is `uint32_t`; `bsPutBit(Int32 bit)` is `uint8_t`, because of `& 1`). Roadmap W49
(widths from reads, not demanded bytes) and W50 (a narrow DWARF parameter's extension). Owner: r2ssa (lane widths of
reads and writes) then r2types.
* **Register locals (138 / 126; 364 greedy).** r2s names one C object per SSA definition (`rax_3`, `tmp_4358_8_1`),
not one per web: `slide_hash` has four DWARF locals and 17 objects. A source local then overlaps a median of 14
objects (angr 7) with best Dice 0.23 (angr 0.38), under the 0.1 floor or inside the 0.03 margin. The fix is r2dec
lowering one object per phi web with r2ssa stating the web; this also shortens the text (bash 13.9 MB against 2.6 MB).
It is an unproven hypothesis for GED, to be graded by D47.
* **Stack local / frame object (16 / 15).** An address-taken `int bzerr` is `uint8_t frame[40]` read as
`load_u32(frame + 20)`. Owner: r2ssa frame model (P4), r2types layouts (P9).
### Hand-read examples (`r2s -q -c 'aa; s <addr>; afv; afi; pdd'` on the stripped binary)
| Function (binary, build, address) | DWARF | r2s | Cause |
|---|---|---|---|
| `fatal` (diffutils diff O2 `0xe960`) | `msgid`: `const char *` | `uint64_t arg0`; body `dcgettext(0, (const char*)arg0, 5)` | callee type not unified into the parameter |
| `myMalloc` (bzip2 O2-noinline `0x3770`) | local `p`: `void *` | `uint64_t rax_1 = malloc(..)` | call result not typed by its prototype |
| `ck_fclose` (diffutils sdiff O2-noinline `0x4b70`) | `f`: `FILE *` | `uint64_t`; callee printed `uint64_t fclose(uint64_t)` | width-only library prototype; angr `FILE *` |
| `set_color_palette` (diffutils diff O2 `0xe9a0`) | `palette`: `const char *` | `uint64_t`; only use `store_u64(0x26748, arg0)` | untyped global (P8) |
| `perror_with_name` (diffutils diff O2 `0xe600`) | `name`: `const char *` | no parameter: `error(0, errno, "%s")` | variadic tail and `__errno_location` undeclared |
| `mySignalCatcher` (bzip2 O2 `0x3400`) | `int n` | no parameter | the body never reads `edi` |
| `deflateInit_` (zlib libz O2 `0x79b0`) | 4 parameters | `uint64_t deflateInit_(void)`, `fcn_3480(); return residual` | tail transfer not rendered |
| `slide_hash` (zlib libz O2 `0x4330`) | locals `n`, `m`: `int`; `p`: pointer | 17 objects, `uint64_t`; `p` unmatched | width and one object per definition |
| `mmed3` (bzip2 O2-noinline `0xe180`) | `UChar a, b, c` | `uint32_t arg0..2`, compares `(uint8_t)(arg1 >> 0)` | narrow parameter at register width |
| `wordchar` (grep O2-noinline `0xc7b0`) | `wint_t wc` | `uint64_t arg0`, read as `(uint32_t)arg0` | width from demand |
| `bsPutBit` (bzip2recover O2-noinline `0x1b20`) | `Int32 bit` | `uint8_t arg1` | narrowed by `& 1` |
| `BZ2_bzwrite` (bzip2 O2 `0x86a0`) | local `bzerr`: `int` | `uint8_t frame[40]` | address-taken local as a byte blob |
| `default_bzfree`, `default_bzalloc` (bzip2 O2 `0x5b00`, `0x5b20`) | | no output: "import stub `free` has no body" | wrapper (`test rsi,rsi; je; mov rdi,rsi; jmp free`) taken for the import |
No-output functions in the type universe: 131 of 3,850 (3.4%); 63 of r2s's 190 declined functions are this wrapper
class, 104 are proof refusals ("native effect obligations ... unaccounted"), 23 are imports with no prototype. The
616 "decompiled but no type_match row" functions of 2026-10-17 were all bash.
## 4. What separates r2s from a GED-perfect function
Per function (`tests/decbench/ged_causes.py`: the block graphs of the source and the rendering, as DecBench's `ged`
builds them; a class is a reading of the graph delta, not a proof). Non-bash, 2,623 scored rows, 1,294 of them
goto-free and not perfect.
* **Size decides.** r2s is perfect on 488 of 593 sources of 0 to 2 blocks, 76 of 309 of 3 to 4, 22 of 300 of 5 to 7,
4 of 380 of 8 to 11, 4 of 443 of 12 to 19, 0 of 598 of 20 or more. No tool is perfect from 20 blocks (0 non-bash,
2 of 1,345 over all rows). 82% of r2s's 594 GED-perfect functions have a single-block source.
* **The goto-bound remainder is not a union lever.** 735 non-bash functions still have a goto; another tool is perfect
on 5 of them. Goto removal is a readability lever now.
* **Residual does not separate the gap.** 46% of the 185 gap rows have residual sites against 48% of all goto-free
imperfect rows. Compound conditions (75 rows, no other tool perfect) and switches (115, 4) have no shape to copy.
* **The return type does.** 795 non-bash scored rows have a `void` source function; r2s declares `void` for 133
(17%) and a value for 662. By source size, GED-perfect with `void` / with a phantom value: 1 block 91.5% / 83.8%;
2 to 4 blocks 45.0% / 21.4% (20 / 117 rows); 5 to 9 blocks 14.3% / 9.5% (7 / 137). Applying the gaps to the 662 gives
about 35 rows for 1 to 4 blocks (5 to 9 blocks adds about 7 on 7 `void` rows). This is roadmap W47 (an unproven return marked): `cadvise` returns `__fprintf_chk`'s `rax` on one arm
and `r2sleigh_residual_u64(1)` on the other, three blocks against the source's two.
Gap rows (r2s not perfect, another tool GED-perfect): 185 non-bash (166 not union-perfect for r2s), 238 bash (200),
423 in all (366). By signature of the block-graph delta (blocks, edges, exits; rendering minus source), goto-free,
non-bash; the last column is what was read by hand with `show_function.py --graphs`, not a claim about every row:
| Signature | Rows (gap rows) | Text has residual / `?:` / noreturn | Read |
|---|---|---|---|
| (0,-1,-1) | 41 (19) | 14 / 0 / 11 | the source falls through a call the rendering ends the path at: `mallocFail()` in `bsOpenReadStream`, `exit()` in `copyFileName`; the rendering has an `else` and no block at the function's end. `pred_newer`: the `assert` failure call is a residual tail transfer and all three parameters are missing |
| (0,+1,+1) | 32 (17) | 20 / 0 / 3 | `parse_newer`: the branch tests `residual_u64(4)`, the value returned is another cell |
| (+1,+1,0) | 32 (16) | 14 / 13 / 8 | `BZ2_bzerror`, `write_buffer`, `read_buf`: a `?:` diamond where the source has an `if` |
| (+1,+2,+1) | 25 (21) | 21 / 2 / 0 | `BZ2_bzwrite` (`ret = c ? -1 : len; return ret;`), `inflateEnd` (indirect calls, 14 residual sites) |
| (+2,+3,0,+1) | 26 (0) | 18 / 5 / 3 | `panic` (O2): the source calls `showFileNames`, the binary has it inlined; no tool is perfect |
| (-3,-4,0,-1) | 26 (0) | 6 / 0 / 0 | not read; no tool is perfect |
Counterfactuals (`counterfactual_ged.py`: respell the stored rendering, re-score with DecBench's `ged`; non-bash, 594
perfect; GED moved on the rows counted):
| Respelling | GED better / worse | Perfect: new / lost |
|---|---|---|
| drop empty `else { }` (925 renderings) | 0 / 0 | 0 / 0 |
| `x = c ? a : b;` as `x = b; if (c) { x = a; }` | 475 / 48 | +12 / 0 |
| ... and `x = c ? a : b; return x;` as `if (c) return a; else return b;` | 477 / 46 | +15 / 0 |
| noreturn arm as a guard (`if (c) { f(); } rest`, 275 renderings) | 77 / 27 | +12 / -1 |
| any arm ending in `return`, `break` or `continue` as a guard | 71 / 25 | +12 / -1 |
| the three respellings together | 544 / 69 | +27 / -1 |
The empty arm is not a block (the hypothesis is refuted). The guard after a `return` adds nothing to the noreturn guard.
The select respelling spells a branch the machine does not have, which AGENTS rule 8 leaves to an ADR; the noreturn
guard is the same C by the callee's noreturn contract.
## 5. Next
Immediate:
1. **Fail on a lost row** (`tests/decbench`, owner: the harness). `run_decbench.sh` stops when a binary the driver
decompiled has an empty `evaluated/<binary>.toml`, refuses `--workers` above one for a binary whose C exceeds 8 MB,
and `report_decbench.py` compares only rows every compared decompiler scored. Also checkpoint by time, not by
function (r2s's bash 1,253 s to about 150 s). Ceiling: a 59%-of-rows blind spot, no score. Gate: a unit test on a
tree with a 16-byte evaluated toml.
2. **A stub is exactly one jump** (r2engine, `Source` stubs). 63 declined rows (`default_bzfree`, `default_bzalloc`,
wrappers that end in `jmp import`). Ceiling +63 union rows: every one is perfect for some other tool, 51 on GED.
Gate: the differential gate for stubs and `afl`; the DecBench no-output count 380 to about 317.
Next:
3. **Library prototypes into values** (r2abi, r2types). Pointer-bearing prototypes for the stdio and multibyte imports
(23 imports, 945 renderings) and `error`'s `...`; a callee parameter's type types the value passed in; a call
result takes its return type. Ceiling: 248 pointer-as-integer and 36 library-named functions where it is the only
cause (123 not already perfect), 283 and 123 greedy. Gate: D47 `improved`/`worse` on type_match, source-gold
soundness (`differential_truth.py`).
4. **A proven return or `void`** (r2ssa return evidence, r2types; W47). A return is a value only if some caller reads
it or every path defines it. About +35 GED-perfect non-bash rows (size-controlled estimate above), and `void` is
the common source. Gate: the mined GED set (D47), certification (no invented return), type_match unchanged.
5. **Widths from reads** (r2ssa lane widths, r2types; W49, W50). Ceiling 209 only cause (142 net union), 318 greedy.
Gate: type_match and the source-gold soundness (a width stated by reads, never by demanded bytes alone).
6. **One C object per SSA web** (r2ssa webs, r2dec lowering). Register locals 138 only cause (126 net union), 364
greedy; also the text size and the evaluation's 5.2 GB. Unproven for GED and byte_match: graded on the mined set
first (D47).
7. **Parameters that flow into residual calls** (r2ssa call contracts P7: tail transfer as call and return, callee
arity floors, variadic tails from format strings; r2abi). 402 functions involved, 138 only cause. Unread parameters
(410 functions, 177 only cause) need caller or function-pointer-slot evidence from r2engine: no ceiling until it
exists.
8. **Structurer respellings** (r2dec structure). The noreturn arm as a guard (+12 / -1, sound by the noreturn
contract); the select respelling (+15 / 0) only after an ADR decision on rule 8; the do-while rotation's 10 lost rows
when the loop's bound is a constant. Measured ceiling 27 GED-perfect non-bash rows, bound 185 gap rows. Gate: the
mined set re-scored (D47), certification, `pdd` timing on the large functions.
Deferred, with the reason: large functions (1,345 rows from 20 blocks, 2 perfect for any tool), switches (115),
compound conditions (75): no achieved shape to copy. The project-named type cap (3,020 functions) and the
unobservable cap (2,255) are benchmark facts. D52's target ("at least angr's 209 of 3,285") was set on the corrupted
population; the comparable one is kuna's 164 of 2,664 (6.2%) non-bash and 539 of 6,365 (8.5%) on all rows.
## Reproduce
```
tests/decbench/run_decbench.sh --decbench <decbench copy> --python <venv>/bin/python --out <tree copy> --r2s <r2s> \
--decompilers r2sleigh_native,kuna,ghidra,angr --opt-level O2 --opt-level O2-noinline --project <p>... --workers N
# with DECBENCH_REDO_DECOMPILERS=r2sleigh_native, PYTHONPATH=<decbench copy>
python <decbench>/scripts/finalize_results.py <tree> --allow-drops
python tests/decbench/evaluate_checkpointed.py <tree> --project bash --decompilers angr,kuna,ghidra
python tests/decbench/compare_runs.py --new <tree> --old <2026-10-17 tree> [--exclude-project bash | --paired]
python tests/decbench/type_causes.py collect --run <tree> --decompilers r2sleigh_native,angr,kuna,ghidra --out rows.jsonl
python tests/decbench/type_causes.py report rows.jsonl # and: universe --run <tree>
python tests/decbench/ged_causes.py collect --run <tree> --scratch <dir> --out rows.jsonl; ... report rows.jsonl
python tests/decbench/counterfactual_ged.py --run <tree> --scratch <dir> --transform select_both
python tests/decbench/r2s_timing.py --r2s <r2s> --run <tree>
```
tests/decbench/compare_runs.py
+247
View File
@@ -0,0 +1,247 @@
#!/usr/bin/env python3
"""Tables for a DecBench result note: the scoreboard, a delta against an earlier run, and the per-function split.
compare_runs.py --new <results tree> [--old <results tree>] [--old-decompiled <tree with the old r2s tomls>]
A results tree holds `scoreboard.toml`, `function_results.json` and `<opt>/<project>/decompiled/<dec>_<binary>.toml`.
GED per function comes from `function_results.json`, gotos and residual counts from the decompiled tomls. When a
tree was re-run for one decompiler only (`DECBENCH_REDO_DECOMPILERS`), the tomls of the others are the same files
as in the tree it was copied from.
"""
from __future__ import annotations
import argparse
import collections
import json
import math
from pathlib import Path
DECS = ("angr", "kuna", "ghidra", "r2sleigh_native")
METRICS = ("byte_match", "ged", "type_match")
def scoreboard(tree: Path) -> dict:
import tomllib
return tomllib.loads((tree / "scoreboard.toml").read_text())
def table(sb: dict) -> list[str]:
lines = ["| Decompiler | Union-perfect | byte_match perfect | GED perfect (mean) | type_match perfect (mean) |",
"|---|---|---|---|---|"]
for dec in sorted(DECS, key=lambda d: sb["decompiler_scores"][d]["overall_rank"]):
s = sb["decompiler_scores"][dec]
m = s["metric_scores"]
lines.append(
f"| {dec} | {s['overall_perfect_count']} of {s['overall_total_count']} "
f"({s['overall_perfect_percentage']:.1f}%), rank {s['overall_rank']} "
f"| {m['byte_match']['perfect_count']} ({m['byte_match']['perfect_percentage']:.1f}%), rank {m['byte_match']['rank']} "
f"| {m['ged']['perfect_count']} of {m['ged']['total_count']} ({m['ged']['perfect_percentage']:.1f}%), "
f"{m['ged']['mean']:.1f}, rank {m['ged']['rank']} "
f"| {m['type_match']['perfect_count']} of {m['type_match']['total_count']} "
f"({m['type_match']['perfect_percentage']:.1f}%), {m['type_match']['mean']:.3f}, rank {m['type_match']['rank']} |")
return lines
def delta(old: dict, new: dict, dec: str = "r2sleigh_native") -> list[str]:
lines = ["| Metric (r2s) | 2026-10-17 | now | change |", "|---|---|---|---|"]
o, n = old["decompiler_scores"][dec], new["decompiler_scores"][dec]
rows = [("union-perfect", o["overall_perfect_count"], n["overall_perfect_count"], "d"),
("union-perfect %", o["overall_perfect_percentage"], n["overall_perfect_percentage"], "f")]
for m in METRICS:
rows.append((f"{m} perfect", o["metric_scores"][m]["perfect_count"], n["metric_scores"][m]["perfect_count"], "d"))
rows.append((f"{m} mean", o["metric_scores"][m]["mean"], n["metric_scores"][m]["mean"], "f"))
for name, a, b, kind in rows:
fmt = "{:d}" if kind == "d" else "{:.3f}"
lines.append(f"| {name} | {fmt.format(a)} | {fmt.format(b)} | {b - a:+.3f} |" if kind == "f"
else f"| {name} | {a} | {b} | {b - a:+d} |")
return lines
def recompute(tree: Path, include) -> dict:
"""The scoreboard's own arithmetic (shared universe: rows measurable for anyone) on the rows `include(group)` keeps."""
data = json.loads((tree / "function_results.json").read_text())
decs = data["decompilers"]
tot = {d: {m: 0 for m in METRICS} for d in decs}
perf = {d: {m: 0 for m in METRICS} for d in decs}
vals = {d: {m: [] for m in METRICS} for d in decs}
opf = {d: 0 for d in decs}
opt = {d: 0 for d in decs}
rows = 0
for g in data["groups"]:
if not include(g):
continue
for f in g["functions"]:
meas = {m: any(m in (f["values"].get(d) or {}) and math.isfinite(f["values"][d][m]) for d in decs)
for m in METRICS}
if not any(meas.values()):
continue
rows += 1
for d in decs:
fv, fp = f["values"].get(d) or {}, f["perfects"].get(d) or {}
for m in METRICS:
if not meas[m]:
continue
tot[d][m] += 1
if m in fv and math.isfinite(fv[m]):
vals[d][m].append(fv[m])
perf[d][m] += bool(fp.get(m))
opt[d] += 1
opf[d] += any(fp.get(m) for m in METRICS if meas[m])
out = {"decompiler_scores": {}, "generated_at": "recomputed", "total_functions": rows, "total_binaries": 0}
for d in decs:
out["decompiler_scores"][d] = {
"overall_perfect_count": opf[d], "overall_total_count": opt[d],
"overall_perfect_percentage": 100 * opf[d] / opt[d] if opt[d] else 0.0,
"metric_scores": {m: {
"perfect_count": perf[d][m], "total_count": tot[d][m],
"perfect_percentage": 100 * perf[d][m] / tot[d][m] if tot[d][m] else 0.0,
"mean": sum(vals[d][m]) / len(vals[d][m]) if vals[d][m] else 0.0} for m in METRICS}}
ranked = sorted(decs, key=lambda d: -out["decompiler_scores"][d]["overall_perfect_percentage"])
for i, d in enumerate(ranked, 1):
out["decompiler_scores"][d]["overall_rank"] = i
for m in METRICS:
ranked = sorted(decs, key=lambda d: -out["decompiler_scores"][d]["metric_scores"][m]["perfect_percentage"])
for i, d in enumerate(ranked, 1):
out["decompiler_scores"][d]["metric_scores"][m]["rank"] = i
return out
def load_function_rows(tree: Path) -> dict:
data = json.loads((tree / "function_results.json").read_text())
rows = {}
for g in data["groups"]:
for f in g["functions"]:
rows[(g["project"], g["opt_level"], g["binary"], f["function"])] = f
return rows, data
def load_tomls(decompiled_root: Path, dec: str) -> dict:
"""(project, opt, binary, function) -> (gotos, residual) from `<opt>/<project>/decompiled/<dec>_<bin>.toml`."""
import tomllib
out = {}
for path in decompiled_root.glob(f"*/*/decompiled/{dec}_*.toml"):
opt, project = path.parts[-4], path.parts[-3]
binary = path.name.removesuffix(".toml")[len(dec) + 1:]
for key, meta in tomllib.loads(path.read_text()).items():
if key.startswith("functions."):
out[(project, opt, binary, key[len("functions."):])] = (meta.get("gotos", 0), meta.get("residual", 0))
return out
def split(tree: Path, decompiled_root: Path | None = None) -> list[str]:
rows, _ = load_function_rows(tree)
decompiled_root = decompiled_root or tree
lines = ["| Decompiler | Functions scored (GED) | With a goto | GED-perfect with no goto | GED-perfect with a goto |",
"|---|---|---|---|---|"]
detail = {}
for dec in DECS:
meta = load_tomls(decompiled_root, dec)
scored = [(k, r["values"][dec]["ged"]) for k, r in rows.items()
if dec in r["values"] and "ged" in r["values"][dec] and math.isfinite(r["values"][dec]["ged"])]
goto = [(k, v) for k, v in scored if meta.get(k, (0, 0))[0] > 0]
nogoto = [(k, v) for k, v in scored if meta.get(k, (0, 0))[0] == 0]
pg = sum(v == 0 for _, v in goto)
pn = sum(v == 0 for _, v in nogoto)
lines.append(f"| {dec} | {len(scored):,} | {100 * len(goto) / len(scored):.1f}% "
f"| {100 * pn / max(1, len(nogoto)):.1f}% | {100 * pg / max(1, len(goto)):.1f}% ({pg} of {len(goto):,}) |")
if dec == "r2sleigh_native":
buckets = collections.OrderedDict((("none", (0, 0)), ("one or two", (1, 2)), ("three or more", (3, 10**9))))
res = []
for name, (lo, hi) in buckets.items():
sel = [(k, v) for k, v in scored if lo <= meta.get(k, (0, 0))[1] <= hi]
res.append(f"{name} {100 * sum(v == 0 for _, v in sel) / max(1, len(sel)):.1f}% ({len(sel):,})")
detail["residual"] = "r2s GED-perfect rate by residual count: " + ", ".join(res)
detail["no_output"] = sum(1 for r in rows.values() if not r["decompiled"].get(dec))
else:
detail[dec] = sum(1 for r in rows.values() if not r["decompiled"].get(dec))
lines.append("")
lines.append(detail["residual"])
lines.append(f"Functions with no output ({len(rows):,} rows): r2s {detail['no_output']}, "
+ ", ".join(f"{d} {detail[d]}" for d in ("ghidra", "kuna", "angr")))
return lines
def paired(old: Path, new: Path, dec: str = "r2sleigh_native", exclude=("bash",)) -> list[str]:
"""Per-function change of one decompiler between two trees: better, worse, equal, newly and no longer perfect."""
orows, _ = load_function_rows(old)
nrows, _ = load_function_rows(new)
ometa, nmeta = load_tomls(old, dec), load_tomls(new, dec)
lines = ["| Metric (r2s, same functions) | scored in both | better | worse | equal | newly perfect | no longer perfect |",
"|---|---|---|---|---|---|---|"]
for m, lower in (("ged", True), ("type_match", False), ("byte_match", False)):
better = worse = equal = newly = lost = both = 0
for key, o in orows.items():
if key[0] in exclude or key not in nrows:
continue
a = (o["values"].get(dec) or {}).get(m)
b = (nrows[key]["values"].get(dec) or {}).get(m)
if a is None or b is None or not (math.isfinite(a) and math.isfinite(b)):
continue
both += 1
if a == b:
equal += 1
elif (b < a) == lower:
better += 1
else:
worse += 1
pa = bool((o["perfects"].get(dec) or {}).get(m))
pb = bool((nrows[key]["perfects"].get(dec) or {}).get(m))
newly += pb and not pa
lost += pa and not pb
lines.append(f"| {m} | {both} | {better} | {worse} | {equal} | {newly} | {lost} |")
keys = [k for k in nmeta if k[0] not in exclude]
g_old = sum(ometa.get(k, (0, 0))[0] for k in keys if k in ometa)
g_new = sum(nmeta[k][0] for k in keys if k in ometa)
f_old = sum(1 for k in keys if k in ometa and ometa[k][0] > 0)
f_new = sum(1 for k in keys if k in ometa and nmeta[k][0] > 0)
r_old = sum(ometa[k][1] for k in keys if k in ometa)
r_new = sum(nmeta[k][1] for k in keys if k in ometa)
lines.append("")
lines.append(f"Functions rendered in both runs {sum(1 for k in keys if k in ometa)} (new only {sum(1 for k in keys if k not in ometa)}): "
f"gotos {g_old} to {g_new}, functions with a goto {f_old} to {f_new}, residual sites {r_old} to {r_new}.")
return lines
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("--new", required=True, type=Path)
p.add_argument("--old", type=Path)
p.add_argument("--decompiled-old", type=Path)
p.add_argument("--decompiled-new", type=Path)
p.add_argument("--paired", action="store_true")
p.add_argument("--exclude-project", action="append", default=[])
p.add_argument("--only-project", action="append", default=[])
args = p.parse_args()
if args.old and args.paired:
print("\n".join(paired(args.old, args.new)))
return 0
if args.exclude_project or args.only_project:
keep = (lambda g: g["project"] not in args.exclude_project and (not args.only_project or g["project"] in args.only_project))
new = recompute(args.new, keep)
print(f"restricted population: {new['total_functions']} rows "
f"(exclude {args.exclude_project}, only {args.only_project})")
print("\n".join(table(new)))
if args.old:
print()
print("\n".join(delta(recompute(args.old, keep), new)))
return 0
new = scoreboard(args.new)
print(f"generated_at {new['generated_at']}, {new['total_functions']} function rows, {new['total_binaries']} binaries")
print("\n".join(table(new)))
if args.old:
print()
print("\n".join(delta(scoreboard(args.old), new)))
print()
print("\n".join(split(args.new, args.decompiled_new)))
if args.old:
print()
print("old run:")
print("\n".join(split(args.old, args.decompiled_old)))
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/decbench/counterfactual_ged.py
+193
View File
@@ -0,0 +1,193 @@
#!/usr/bin/env python3
"""Re-score r2s renderings after one textual respelling, to bound what a renderer change could gain.
PYTHONPATH=<decbench> <venv>/bin/python counterfactual_ged.py --run <tree> --transform empty_else \
--scratch <dir> [--project NAME ...] > out.tsv
Each transform is a respelling that keeps the program's meaning (an empty `else { }` arm says nothing). It is
applied to the stored rendering, the result goes through DecBench's own `evaluate_project` with the `ged` metric,
and the tsv says per function: GED stored, GED after. The transform is a measuring device for a ceiling; the
change itself belongs in the renderer (r2dec), where the structure that proves it is.
"""
from __future__ import annotations
import argparse
import os
import pickle
import re
import sys
from pathlib import Path
def split_ternary(rhs: str):
"""`cond ? a : b` at nesting depth zero, or None (also None when a nested ternary sits at depth zero)."""
depth, q, colon = 0, None, None
pending = 0
for i, c in enumerate(rhs):
if c in "([{":
depth += 1
elif c in ")]}":
depth -= 1
elif depth == 0 and c == "?":
if q is None:
q = i
else:
pending += 1
elif depth == 0 and c == ":" and q is not None:
if pending:
pending -= 1
else:
colon = i
break
if q is None or colon is None or pending:
return None
return rhs[:q].strip(), rhs[q + 1:colon].strip(), rhs[colon + 1:].strip()
def select_as_if(code: str) -> str:
"""`x = c ? a : b;` as `x = b; if (c) { x = a; }` when x is not read by c, a or b."""
out = []
for line in code.splitlines():
m = re.match(r"^(\s*)(\w+) = (.*);$", line)
parts = split_ternary(m.group(3)) if m and "?" in m.group(3) else None
if parts and not any(re.search(r"\b" + re.escape(m.group(2)) + r"\b", part) for part in parts):
indent, lhs = m.group(1), m.group(2)
cond, a, b = parts
out += [f"{indent}{lhs} = {b};", f"{indent}if ({cond}) {{", f"{indent} {lhs} = {a};", f"{indent}}}"]
else:
out.append(line)
return "\n".join(out)
_SELECT_RETURN = re.compile(
r"(?P<ind>[ \t]*)(?P<lhs>\w+) = (?P<rhs>[^;\n]*\?[^;\n]*);\n(?P<mid>(?:[ \t]*\{[ \t]*\n)*)"
r"[ \t]*return (?P<cast>\([^()]*\))? ?(?P=lhs);\n")
def select_return_arms(code: str) -> str:
"""`x = c ? a : b; return x;` as `if (c) { return a; } else { return b; }`, bare braces between kept."""
def arms(m):
parts = split_ternary(m.group("rhs"))
if not parts or re.search(r"\b" + re.escape(m.group("lhs")) + r"\b", " ".join(parts)):
return m.group(0)
cond, a, b = parts
cast, ind = m.group("cast") or "", m.group("ind")
return (m.group("mid") + f"{ind}if ({cond}) {{\n{ind} return {cast}({a});\n{ind}}} else {{\n"
f"{ind} return {cast}({b});\n{ind}}}\n")
return _SELECT_RETURN.sub(arms, code)
def noreturn_guard(code: str, returns: bool = False) -> str:
"""`if (c) { f(); } else { rest }` as `if (c) { f(); } rest` when f is declared noreturn in the rendering.
With `returns`, an arm ending in `return ...;`, `break;` or `continue;` is a guard too.
"""
names = set(re.findall(r"__attribute__\(\(noreturn\)\)[\w\s\*]*?\b(\w+)\(", code))
if not names and not returns:
return code
lines = code.splitlines()
out = []
i = 0
while i < len(lines):
m = re.match(r"^(\s*)\} else \{$", lines[i])
arm_ok = False
if m and out:
# the arm just closed ends in a call to a noreturn function and nothing after it
j = len(out) - 1
while j >= 0 and not out[j].strip():
j -= 1
call = re.match(r"^\s*(\w+)\(.*\);$", out[j]) if j >= 0 else None
arm_ok = bool(call and call.group(1) in names)
if returns and j >= 0 and re.match(r"^\s*(return\b.*|break|continue);$", out[j]):
arm_ok = True
if m and arm_ok:
depth, k = 1, i + 1
while k < len(lines) and depth:
depth += lines[k].count("{") - lines[k].count("}")
k += 1
body = lines[i + 1:k - 1]
out.append(f"{m.group(1)}}}")
out.extend(body)
i = k
continue
out.append(lines[i])
i += 1
return "\n".join(out)
TRANSFORMS = {
"guard_after_exit": lambda code: noreturn_guard(code, returns=True),
"all_respellings_guard": lambda code: noreturn_guard(select_as_if(select_return_arms(code)), returns=True),
"noreturn_guard": noreturn_guard,
"all_respellings": lambda code: noreturn_guard(select_as_if(select_return_arms(code))),
"select_return_arms": select_return_arms,
"select_both": lambda code: select_as_if(select_return_arms(code)),
"select_as_if": select_as_if,
# `if (c) { ... } else { }`: the empty arm adds a block the source does not have.
"empty_else": lambda code: re.sub(r"\s*\belse\s*\{\s*\}", "", code),
}
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("--run", required=True, type=Path)
p.add_argument("--scratch", required=True, type=Path)
p.add_argument("--transform", required=True, choices=sorted(TRANSFORMS))
p.add_argument("--project", action="append")
p.add_argument("--opt-levels", default="O2,O2-noinline")
args = p.parse_args()
os.environ["DECBENCH_NO_CACHE"] = "1"
import decbench.metrics # noqa: F401
from decbench.models.project import OptimizationLevel, Project
from decbench.pipeline.evaluate import evaluate_project
from decbench.pipeline.executor import PipelineConfig, PipelineExecutor
from decbench.results_store import gather_project_tomls
from decbench.utils import binfmt
from decbench.utils.cfg import extract_cfgs_from_source
from decbench.utils.dwarf_policy import dwarf_follow_abstract_origin
transform = TRANSFORMS[args.transform]
opts = [OptimizationLevel(v) for v in args.opt_levels.split(",")]
print("project\topt\tbinary\tfunction\tbefore\tafter\tchanged")
for toml in gather_project_tomls():
if args.project and toml.stem not in args.project:
continue
ck_path = args.run / "checkpoints" / f"{toml.stem}.pkl"
if not ck_path.exists():
continue
ck = pickle.loads(ck_path.read_bytes())
project = Project.from_toml(toml)
PipelineExecutor(PipelineConfig(output_dir=args.run, optimization_levels=opts))._discover_existing_binaries(
[project], args.run)
for opt in opts:
units = project.preprocessed_sources.get(opt, {})
src_cfgs = {stem: extract_cfgs_from_source(path) or {} for stem, path in units.items()}
for b in project.compiled_binaries.get(opt, []):
per_dec = (ck["decompile"].get(opt) or {}).get(b.stem) or {}
r2s = per_dec.get("r2sleigh_native")
if r2s is None:
continue
stored = ((ck["evaluate"].get(opt) or {}).get(b.stem) or {}).get("r2sleigh_native", {}).get("ged")
changed = {}
functions = {}
for name, fn in r2s.functions.items():
new_code = transform(fn.decompiled_code)
changed[name] = new_code != fn.decompiled_code
functions[name] = fn.model_copy(update={"decompiled_code": new_code})
modified = r2s.model_copy(update={"functions": functions})
owners = binfmt.source_function_owners(
b, set(units), follow_abstract_origin=dwarf_follow_abstract_origin())
res = evaluate_project(project, {b.stem: {"r2sleigh_native": modified}}, args.scratch, opt, ["ged"],
parallel=False, precomputed_source_cfgs=src_cfgs,
source_function_owners={b.stem: owners})
after = res[b.stem]["r2sleigh_native"]["ged"].function_results
for name, v in after.items():
before = stored.function_results.get(name) if stored is not None else None
print(f"{project.name}\t{opt.value}\t{b.stem}\t{name}\t{before.value if before else ''}\t{v.value}\t{int(changed[name])}",
flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/decbench/decbench_cli.py
+2
View File
@@ -29,6 +29,7 @@ from decbench.metrics.base import Metric
from decbench.metrics.registry import MetricRegistry, register_metric
from decbench.models.metrics import AggregationType, MetricValue
from phase_timing import install_phase_timing
from run_gate import install_workers_guard
def register_vj_ged_if_needed() -> None:
@@ -99,6 +100,7 @@ def run_benchmark(arguments: list[str]) -> None:
raise SystemExit(f"no {script}: DecBench must be installed from a checkout (pip -e)")
sys.argv = [str(script), *arguments]
sys.path.insert(0, str(script.parent))
install_workers_guard() # before the driver imports `evaluate_project`
runpy.run_path(str(script), run_name="__main__")
tests/decbench/evaluate_checkpointed.py
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env python3
"""Evaluate decompilers whose results already sit in a run's checkpoint, with the driver's own functions.
PYTHONPATH=<decbench> <venv>/bin/python evaluate_checkpointed.py <tree> --project bash \
--decompilers angr,kuna,ghidra [--workers 1]
`scripts/run_benchmark.py` evaluates only the decompilers it has just run, so a decompiler whose evaluation was lost
(the 2026-10-17 run lost every bash evaluation to the host's OOM killer: five GB per worker, four workers) cannot be
re-scored without decompiling it again. This takes the stored `DecompilationResult`s, calls the driver's
`project_source_functions`, `extract_source_cfgs` and DecBench's `evaluate_project` exactly as `main` does, writes the
scores into the checkpoint, and leaves `finalize_results.py` to rebuild `scoreboard.toml`. Nothing is decompiled, and
the per-binary `evaluated/*.toml` files are not touched (the output directory of `evaluate_project` is a scratch one).
`--workers` above 1 is refused when a binary's stored C is over 8 MB (`run_gate.py`).
"""
from __future__ import annotations
import argparse
import os
import pickle
import sys
import tempfile
from pathlib import Path
from run_gate import refuse_unsafe_workers
def main() -> int:
p = argparse.ArgumentParser()
p.add_argument("tree", type=Path)
p.add_argument("--project", required=True)
p.add_argument("--decompilers", required=True)
p.add_argument("--workers", type=int, default=1)
p.add_argument("--opt-levels", default="O2,O2-noinline")
p.add_argument("--metrics", default="byte_match,ged,vj_ged,type_match")
args = p.parse_args()
os.environ["DECBENCH_WORKERS"] = str(args.workers)
os.environ["DECBENCH_METRICS"] = args.metrics
os.environ["DECBENCH_OPT_LEVELS"] = args.opt_levels
import decbench
sys.path.insert(0, str(Path(decbench.__file__).resolve().parents[1] / "scripts"))
import run_benchmark as rb # the driver's module; importing it runs nothing
from decbench.models.project import Project
from decbench.pipeline.evaluate import evaluate_project
from decbench.results_store import gather_project_tomls
decs = args.decompilers.split(",")
ckpt = args.tree / "checkpoints" / f"{args.project}.pkl"
ck = pickle.loads(ckpt.read_bytes())
for opt_decompiled in ck["decompile"].values():
refuse_unsafe_workers({stem: {d: per[d] for d in decs if d in per}
for stem, per in opt_decompiled.items()}, args.workers)
project = Project.from_toml(next(t for t in gather_project_tomls() if t.stem == args.project))
rb.discover(project, args.tree)
scratch = Path(tempfile.mkdtemp(prefix="evaluate_checkpointed_"))
for opt in rb.OPT_LEVELS:
source_stems = set(project.preprocessed_sources.get(opt, {}).keys())
owners, kept = {}, []
for b in project.compiled_binaries.get(opt, []):
stems = {}
fns = rb.project_source_functions(b, source_stems, stem_out=stems)
if fns:
owners[b.stem] = {a: (n, stems[a]) for a, n in fns.items() if a in stems}
kept.append(b)
project.compiled_binaries[opt] = kept
src_cfgs = rb.extract_source_cfgs(project, opt)
stored = ck["decompile"][opt]
dec = {b.stem: {d: stored[b.stem][d] for d in decs if d in stored.get(b.stem, {})} for b in kept}
print(f"[{args.project}/{opt.value}] evaluating {decs} on {len(kept)} binaries", flush=True)
ev = evaluate_project(project, dec, scratch, opt, rb.METRICS, parallel=True, workers=args.workers,
precomputed_source_cfgs=src_cfgs, source_function_owners=owners)
merged = ck["evaluate"].setdefault(opt, {})
for stem, per_dec in ev.items():
merged.setdefault(stem, {}).update({d: m for d, m in per_dec.items() if m})
print(f"[{args.project}/{opt.value}] done: " + ", ".join(
f"{stem}:{sorted(d for d, m in per.items() if m)}" for stem, per in ev.items()), flush=True)
ckpt.write_bytes(pickle.dumps(ck))
print(f"wrote {ckpt}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/decbench/ged_causes.py
+274
View File
@@ -0,0 +1,274 @@
#!/usr/bin/env python3
"""What separates an r2s function from a GED-perfect one, by graph shape and text.
PYTHONPATH=<decbench> <venv>/bin/python ged_causes.py collect --run <results tree> --out rows.jsonl [--project zlib]
python3 ged_causes.py report rows.jsonl
`collect` reads a checkpointed DecBench run (`<run>/checkpoints/*.pkl`), takes each function's stored GED of every
decompiler, and re-extracts the two block graphs DecBench's `ged` compared (the source's and r2s's, through the same
`evaluate_project` path) only to count their blocks, edges, exits, branches, multiway nodes and back edges. It also
counts `&&`, `||`, `switch` and the loop keywords in the source function (from the preprocessed unit) and in the
rendering. GED itself is not recomputed.
`report` takes the goto-free functions that are not GED-perfect and assigns each one class by the first rule that
holds (see `classify`). The classes are a reading of the graph delta, not a proof of the cause; the ceiling of a
class is the number of its functions on which some other decompiler is GED-perfect (an achieved shape), the same
test the 2026-10-22 goto analysis used.
"""
from __future__ import annotations
import argparse
import collections
import json
import math
import os
import pickle
import re
import sys
from pathlib import Path
def graph_features(g) -> dict:
nodes = list(g.nodes)
entries = [n for n in nodes if getattr(n, "is_entrypoint", False)] or [n for n in nodes if g.in_degree(n) == 0]
# back edges: edges to a node still on the DFS stack
state: dict = {}
back = 0
for root in entries:
if root in state:
continue
stack = [(root, iter(g.successors(root)))]
state[root] = 1
while stack:
node, it = stack[-1]
for nxt in it:
s = state.get(nxt, 0)
if s == 1:
back += 1
elif s == 0:
state[nxt] = 1
stack.append((nxt, iter(g.successors(nxt))))
break
else:
state[node] = 2
stack.pop()
return {
"V": len(nodes), "E": g.number_of_edges(),
"X": sum(1 for n in nodes if getattr(n, "is_exitpoint", g.out_degree(n) == 0)),
"B": sum(1 for n in nodes if g.out_degree(n) == 2),
"M": sum(1 for n in nodes if g.out_degree(n) >= 3),
"L": back,
}
def strip_noncode(text: str) -> str:
text = re.sub(r'"(?:\\.|[^"\\\n])*"|\'(?:\\.|[^\'\\\n])*\'', '""', text)
text = re.sub(r"/\*.*?\*/", " ", text, flags=re.S)
return re.sub(r"//[^\n]*", " ", text)
def function_body(unit: str, name: str) -> str | None:
"""The text of `name`'s definition in a preprocessed unit (ANSI or K&R), or None."""
for m in re.finditer(r"(?<![\w.])" + re.escape(name) + r"\s*\(", unit):
depth, i = 0, m.end() - 1
while i < len(unit):
c = unit[i]
depth += c == "("
depth -= c == ")"
i += 1
if depth == 0:
break
rest = unit[i:i + 600]
if rest.lstrip().startswith((";", ",", ")", "=")):
continue
brace = rest.find("{")
if brace < 0 or "}" in rest[:brace]:
continue
start = i + brace
depth, j = 0, start
while j < len(unit):
depth += unit[j] == "{"
depth -= unit[j] == "}"
j += 1
if depth == 0:
return unit[start:j]
return None
def text_features(code: str) -> dict:
code = strip_noncode(code)
return {
"and_or": len(re.findall(r"&&|\|\|", code)),
"switch": len(re.findall(r"\bswitch\b", code)),
"case": len(re.findall(r"\bcase\b", code)),
"for": len(re.findall(r"\bfor\s*\(", code)),
"for_full": len(re.findall(r"\bfor\s*\([^;()]*;[^;()]+;[^()]*[^;\s)][^()]*\)", code)),
"while": len(re.findall(r"\bwhile\s*\(", code)),
"do": len(re.findall(r"\bdo\b\s*\{", code)),
"goto": len(re.findall(r"\bgoto\b", code)),
"ret": len(re.findall(r"\breturn\b", code)),
"ternary": len(re.findall(r"\?", code)),
"residual": len(re.findall(r"r2sleigh_residual", code)),
}
def collect(args) -> None:
os.environ["DECBENCH_NO_CACHE"] = "1"
import decbench.metrics # noqa: F401
from decbench.metrics import ged as gedmod
from decbench.models.metrics import MetricValue
from decbench.models.project import OptimizationLevel, Project
from decbench.pipeline.evaluate import evaluate_project
from decbench.pipeline.executor import PipelineConfig, PipelineExecutor
from decbench.results_store import gather_project_tomls
from decbench.utils import binfmt
from decbench.utils.cfg import extract_cfgs_from_source
from decbench.utils.dwarf_policy import dwarf_follow_abstract_origin
run = Path(args.run)
scratch = Path(args.scratch)
opts = [OptimizationLevel(v) for v in args.opt_levels.split(",")]
captured: dict[str, dict] = {}
def capture(self, decompiled, source_cfg=None, decompiled_cfg=None, **kw):
if source_cfg is not None and decompiled_cfg is not None:
captured[decompiled.name] = {"S": graph_features(source_cfg), "D": graph_features(decompiled_cfg)}
return MetricValue(value=0.0)
gedmod.GEDMetric.compute_for_function = capture
out = open(args.out, "w")
for toml in gather_project_tomls():
if args.project and toml.stem not in args.project:
continue
ck_path = run / "checkpoints" / f"{toml.stem}.pkl"
if not ck_path.exists():
continue
ck = pickle.loads(ck_path.read_bytes())
project = Project.from_toml(toml)
PipelineExecutor(PipelineConfig(output_dir=run, optimization_levels=opts))._discover_existing_binaries(
[project], run)
for opt in opts:
units = project.preprocessed_sources.get(opt, {})
stems = set(units)
src_cfgs = {stem: extract_cfgs_from_source(path) or {} for stem, path in units.items()}
unit_text = {}
for b in project.compiled_binaries.get(opt, []):
per_dec = (ck["decompile"].get(opt) or {}).get(b.stem) or {}
stored = (ck["evaluate"].get(opt) or {}).get(b.stem) or {}
r2s = per_dec.get("r2sleigh_native")
if r2s is None:
continue
owners = binfmt.source_function_owners(
b, stems, follow_abstract_origin=dwarf_follow_abstract_origin())
captured.clear()
evaluate_project(project, {b.stem: {"r2sleigh_native": r2s}}, scratch, opt, ["ged"],
parallel=False, precomputed_source_cfgs=src_cfgs,
source_function_owners={b.stem: owners})
by_addr = {a: v for a, v in owners.items()}
for name, fn in r2s.functions.items():
owner = by_addr.get(int(fn.address))
row = {"project": project.name, "opt": opt.value, "binary": b.stem, "function": name,
"address": int(fn.address), "gotos": fn.metadata.get("gotos", 0) if fn.metadata else 0,
"bools": fn.metadata.get("bools", 0) if fn.metadata else 0,
"residual": fn.metadata.get("residual", 0) if fn.metadata else 0,
"lines": fn.line_count,
"ged": {}, "tu": owner[1] if owner else None}
for dec in ("r2sleigh_native", "angr", "kuna", "ghidra"):
sv = stored.get(dec, {}).get("ged")
v = sv.function_results.get(name) if sv is not None else None
row["ged"][dec] = None if v is None or not math.isfinite(v.value) else v.value
row.update(captured.get(name, {}))
row["D_text"] = text_features(fn.decompiled_code or "")
if owner and owner[1] in units:
if owner[1] not in unit_text:
unit_text[owner[1]] = Path(units[owner[1]]).read_text(errors="replace")
body = function_body(unit_text[owner[1]], owner[0])
row["S_text"] = text_features(body) if body else None
out.write(json.dumps(row) + "\n")
out.flush()
print(project.name, opt.value, b.stem, len(r2s.functions), file=sys.stderr, flush=True)
out.close()
def classify(r: dict) -> str:
S, D, St, Dt = r["S"], r["D"], r["S_text"], r["D_text"]
if St is None:
return "no_source_text"
if S["M"] != D["M"] or (St["switch"] and not Dt["switch"]):
return "switch"
if St["and_or"] > Dt["and_or"] and D["B"] < S["B"]:
return "missing_and_or"
if S["L"] != D["L"]:
return "loop_count"
if S["V"] == D["V"] and S["E"] == D["E"] and S["X"] == D["X"] and S["B"] == D["B"]:
return "wiring_only"
if S["X"] != D["X"]:
return "exit_blocks"
loops_s = St["for_full"] > 0
loops_d = Dt["for_full"] > 0
if (St["for"] or St["while"] or St["do"]) and loops_s != loops_d and S["V"] != D["V"]:
return "loop_form"
if D["V"] > S["V"]:
return "extra_blocks"
if D["V"] < S["V"]:
return "missing_blocks"
return "other"
def report(args) -> None:
rows = [json.loads(line) for line in open(args.rows)]
decs = ("r2sleigh_native", "angr", "kuna", "ghidra")
scored = [r for r in rows if r["ged"]["r2sleigh_native"] is not None]
perfect = [r for r in scored if r["ged"]["r2sleigh_native"] == 0]
print(f"r2s functions with a rendering {len(rows)}, GED scored {len(scored)}, GED-perfect {len(perfect)}")
gf = [r for r in scored if r["gotos"] == 0]
gb = [r for r in scored if r["gotos"] > 0]
print(f"goto-free {len(gf)} (perfect {sum(r['ged']['r2sleigh_native'] == 0 for r in gf)}), "
f"with goto {len(gb)} (perfect {sum(r['ged']['r2sleigh_native'] == 0 for r in gb)})")
pool = [r for r in gf if r["ged"]["r2sleigh_native"] > 0]
print(f"goto-free, not perfect: {len(pool)}")
def other_perfect(r):
return {d: r["ged"][d] == 0 for d in decs[1:]}
classes = collections.defaultdict(list)
for r in pool:
classes[classify(r) if "S" in r and "D" in r else "no_graph"].append(r)
print(f"{'class':18s} {'n':>5s} {'resid>0':>8s} {'meanGED':>8s} {'GED<=3':>7s} {'any other perfect':>18s} "
f"{'angr':>5s} {'kuna':>5s} {'ghidra':>6s}")
for cls, items in sorted(classes.items(), key=lambda kv: -len(kv[1])):
op = [other_perfect(r) for r in items]
print(f"{cls:18s} {len(items):5d} {sum(r['residual'] > 0 for r in items):8d} "
f"{sum(r['ged']['r2sleigh_native'] for r in items) / len(items):8.1f} "
f"{sum(r['ged']['r2sleigh_native'] <= 3 for r in items):7d} "
f"{sum(any(o.values()) for o in op):18d} "
f"{sum(o['angr'] for o in op):5d} {sum(o['kuna'] for o in op):5d} {sum(o['ghidra'] for o in op):6d}")
print("-- residual stratified by source size (goto-free scored functions)")
for lo, hi in ((0, 5), (5, 10), (10, 20), (20, 40), (40, 10_000)):
for name, cond in (("resid=0", lambda r: r["residual"] == 0), ("resid 1-2", lambda r: 1 <= r["residual"] <= 2),
("resid>=3", lambda r: r["residual"] >= 3)):
sel = [r for r in gf if "S" in r and lo <= r["S"]["V"] < hi and cond(r)]
if sel:
p = sum(r["ged"]["r2sleigh_native"] == 0 for r in sel)
print(f" source blocks [{lo},{hi}) {name:10s} n={len(sel):5d} perfect={p:5d} ({100 * p / len(sel):4.1f}%)")
def main() -> int:
p = argparse.ArgumentParser()
sub = p.add_subparsers(dest="cmd", required=True)
c = sub.add_parser("collect")
c.add_argument("--run", required=True)
c.add_argument("--scratch", required=True)
c.add_argument("--opt-levels", default="O2,O2-noinline")
c.add_argument("--project", action="append")
c.add_argument("--out", required=True)
r = sub.add_parser("report")
r.add_argument("rows")
args = p.parse_args()
collect(args) if args.cmd == "collect" else report(args)
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/decbench/mined_set.py
+436
View File
@@ -0,0 +1,436 @@
#!/usr/bin/env python3
"""The mined-set comparison: what a structurer change does to GED, on the functions it was mined from.
A change to the structurer (or to anything that moves a rendering) is graded on every function of one
pinned set of DecBench projects, with the build before it and the build after it:
mined_set.py dump --tree <run> --r2s <base r2s> --out base/ --set mining
mined_set.py dump --tree <run> --r2s <new r2s> --out new/ --set mining
mined_set.py score --tree <run> --dump base/ --out base.json --decbench ../decbench
mined_set.py score --tree <run> --dump new/ --out new.json --decbench ../decbench
mined_set.py pair base.json new.json
* `dump` asks r2s for `pddj` of every function of every binary of the set, at O2 and O2-noinline, through
the runner the backend uses (`tests/equiv/r2s_batch.py`, on the stripped binary). The functions are the
addresses the run's own `decompiled/*.toml` name, the union over its decompilers, so the targets are the
driver's: the DWARF functions of the project's own sources.
* `score` computes DecBench's GED for each rendering against the source CFG of the translation unit the
binary's DWARF says defines the function at its address (`binfmt.source_function_owners` over the compiled
binary, the driver's own owner; the units are `<tree>/<opt>/<project>/compiled/*.i`), by the flow of
DecBench's `metrics/ged.py` (isomorphism, the size bound above `GED_MAX_NODES`, else VJ). A function with no
owner, or whose owner yields no CFG, is not scored and is counted. It needs DecBench importable
(`--decbench` or the venv) and Joern; it is not run by the unit tests.
* `pair` reports two scored runs over the functions both scored: perfect count (newly, lost), mean and total
GED (better, worse, same), gotos, functions with a goto, functions whose text changed, functions newly
refused and the functions that had no source to be scored against.
The pinned lists split what may be mined from what may only be reported: MINING (bzip2, zlib, grep) is run
on every change; HELD_OUT is the rest of the sailr set, run nightly and at an item's exit only, and a
regression there opens an investigation. A structurer rule is chosen by reading MINING; reading HELD_OUT to
choose one defeats it. `pair` refuses to compare a mining run with a held-out run.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import pickle
import re
import statistics
import sys
import tempfile
import tomllib
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
from run_gate import binaries_by_stem, split_name # noqa: E402
MINING = ("bzip2", "zlib", "grep")
HELD_OUT = ("bash", "diffutils", "findutils", "gzip", "libexpat")
SETS = {"mining": MINING, "heldout": HELD_OUT}
OPT_LEVELS = ("O2", "O2-noinline")
DECOMPILERS = ("r2sleigh_native", "angr", "ghidra", "kuna")
SCHEMA = 1
GED_SECONDS = 120
NO_OWNER = "no_owner" # the binary's DWARF names no source unit for the function's address
NO_SOURCE_CFG = "no_source_cfg" # the owner is known and yields no CFG for it; the driver abstains here too
# --- targets and dump -------------------------------------------------------------------------
def function_targets(project_dir: Path, decompilers=DECOMPILERS) -> dict[str, dict[int, str]]:
"""`{binary stem: {address: DWARF name}}` from the tomls the run's decompilers wrote for a project."""
found: dict[str, dict[int, str]] = {}
for toml in sorted((project_dir / "decompiled").glob("*.toml")):
named = split_name(toml.stem, list(decompilers))
if named is None:
continue
data = tomllib.loads(toml.read_text())
for key, value in data.items():
if key.startswith("functions.") and isinstance(value, dict) and "address" in value:
address = value["address"]
address = int(address, 16) if isinstance(address, str) else int(address)
found.setdefault(named[1], {}).setdefault(address, key[len("functions."):])
return found
def row_keys(opt: str, project: str, binary: str, targets: dict[int, str]) -> dict[int, str]:
"""`opt|project|binary|name` per address; a name already taken by a lower address gets `@0x<address>`."""
keys: dict[int, str] = {}
taken: set[str] = set()
for address in sorted(targets):
key = f"{opt}|{project}|{binary}|{targets[address]}"
if key in taken:
key += f"@0x{address:x}"
taken.add(key)
keys[address] = key
return keys
def record_of(answer, name: str) -> dict:
"""One dumped function: what r2s said, with the definition renamed to the DWARF name."""
row = {"name": name, "addr": f"0x{answer.address:x}", "status": answer.kind,
"refused": bool(answer.refusal)}
if answer.ok and answer.record is not None:
code = str(answer.record.get("code") or "")
definition = str(answer.record.get("definition") or "")
if definition and definition != name:
code = re.sub(rf"\b{re.escape(definition)}\b", name, code)
row["code"] = code
else:
row["cause"] = (answer.cause or "")[:300]
return row
def dump(tree: Path, r2s: Path, out: Path, projects, opt_levels=OPT_LEVELS, decompilers=DECOMPILERS,
function_timeout: float = 300.0, startup_timeout: float = 600.0, set_name: str = "adhoc") -> int:
sys.path.append(str(HERE.parent / "equiv"))
import r2s_batch
out.mkdir(parents=True, exist_ok=True)
total = 0
for opt in opt_levels:
for project in projects:
project_dir = tree / opt / project
stripped = binaries_by_stem(project_dir / "stripped")
for binary, targets in sorted(function_targets(project_dir, decompilers).items()):
if binary not in stripped:
print(f"skip {opt}/{project}/{binary}: no stripped binary", file=sys.stderr)
continue
keys = row_keys(opt, project, binary, targets)
report = r2s_batch.run_batch(r2s, stripped[binary], sorted(targets),
function_timeout=function_timeout,
startup_timeout=startup_timeout)
rows = {keys[a.address]: record_of(a, targets[a.address]) for a in report.answers}
(out / f"{opt}.{project}.{binary}.json").write_text(json.dumps(rows, sort_keys=True))
total += len(rows)
print(f"{opt} {project} {binary}: {len(rows)} functions", flush=True)
manifest = {"schema": SCHEMA, "set": set_name, "projects": list(projects), "opt_levels": list(opt_levels),
"r2s_sha256": hashlib.sha256(Path(r2s).read_bytes()).hexdigest(), "functions": total}
(out / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
return total
# --- scoring (DecBench) ------------------------------------------------------------------------
class Timeout(Exception):
pass
def decbench_scorer(decbench: Path | None):
"""DecBench's GED flow as `(source_cfgs, ged, dec_cfg_of, owners_of)`; imports DecBench, so it needs the venv."""
if decbench is not None:
sys.path.insert(0, str(decbench))
from decbench.metrics.ged import GED_MAX_NODES, _is_isomorphic
from decbench.metrics.vj_ged import vj_ged
from decbench.utils.binfmt import source_function_owners
from decbench.utils.cfg import (
_parse_cfgs,
canonicalize_infinite_loops,
extract_cfgs_from_source,
is_degenerate_source_cfg,
preprocess_decompiled_c,
sanitize_decompiled_c,
)
from decbench.utils.dwarf_policy import dwarf_follow_abstract_origin
def owners_of(binary: Path, stems: set[str]) -> dict[int, tuple[str, str]]:
"""`{address: (DWARF name, defining unit stem)}`, as the driver asks it of the binary it scores."""
return source_function_owners(binary, stems, follow_abstract_origin=dwarf_follow_abstract_origin())
def ged(source_cfg, decompiled_cfg):
if is_degenerate_source_cfg(source_cfg):
return None
if _is_isomorphic(source_cfg, decompiled_cfg):
return 0.0
s, d = source_cfg.number_of_nodes(), decompiled_cfg.number_of_nodes()
if s > GED_MAX_NODES or d > GED_MAX_NODES:
return max(1.0, float(abs(s - d) + abs(source_cfg.number_of_edges() - decompiled_cfg.number_of_edges())))
return max(1.0, float(vj_ged(source_cfg, decompiled_cfg)))
scratch = "/dev/shm" if os.path.isdir("/dev/shm") else None
def dec_cfg_of(text: str, name: str):
marked = f"// Function: {name}\n{sanitize_decompiled_c(text)}"
marked = canonicalize_infinite_loops(preprocess_decompiled_c(marked))
with tempfile.TemporaryDirectory(dir=scratch) as directory:
path = Path(directory) / "rendering.c"
path.write_text(marked)
return _parse_cfgs(path).get(name)
return (lambda path: extract_cfgs_from_source(Path(path))), ged, dec_cfg_of, owners_of
def source_cfgs_by_unit(tree: Path, opt: str, project: str, cache: Path, extract) -> dict[str, dict]:
"""`{translation unit stem: {function: cfg}}` over the project's `.i` files; cached per (opt, project).
A unit that yields no CFGs stays in the map, empty, as the driver's `extract_source_cfgs` leaves it. The
cache is keyed by unit and name: delete it when the sources are recompiled.
"""
pickled = cache / f"{opt}.{project}.tu.pkl"
if pickled.exists():
return pickle.loads(pickled.read_bytes())
by_unit: dict[str, dict] = {}
for source in sorted((tree / opt / project / "compiled").glob("*.i")):
try:
by_unit[source.stem] = extract(source) or {}
except Exception as error: # noqa: BLE001
print(f"no source CFGs from {source.name}: {error}", file=sys.stderr)
by_unit[source.stem] = {}
cache.mkdir(parents=True, exist_ok=True)
pickled.write_bytes(pickle.dumps(by_unit))
return by_unit
def owners_of_binary(owners_of, project_dir: Path, binary: str, stems: set[str]) -> dict[int, tuple[str, str]]:
"""The DWARF owners of a binary's functions, read from the unstripped binary in `compiled/`.
The stripped copy keeps the compiled file's name (the driver's `_stripped_copy`), so its name finds the
compiled file even when the stem is shared with `<stem>.i`. No file: no owners, and the caller counts them.
"""
stripped = binaries_by_stem(project_dir / "stripped").get(binary)
compiled = project_dir / "compiled" / stripped.name if stripped is not None else None
if compiled is None or not compiled.is_file():
print(f"no compiled binary for {project_dir.parts[-2]}/{project_dir.parts[-1]}/{binary}: "
"its functions have no DWARF owner", file=sys.stderr)
return {}
return owners_of(compiled, stems)
def score(tree: Path, dump_dir: Path, out: Path, decbench: Path | None, cache: Path | None) -> int:
import signal
extract, ged, dec_cfg_of, owners_of = decbench_scorer(decbench)
cache = cache or out.parent / "srccache"
manifest = json.loads((dump_dir / "manifest.json").read_text())
def alarm(signum, frame):
raise Timeout
signal.signal(signal.SIGALRM, alarm)
rows: dict[str, dict] = {}
for dumped in sorted(dump_dir.glob("*.json")):
if dumped.name == "manifest.json":
continue
opt, project, binary = dumped.name[:-len(".json")].split(".", 2)
cfgs = source_cfgs_by_unit(tree, opt, project, cache, extract)
owners = owners_of_binary(owners_of, tree / opt / project, binary, set(cfgs))
unowned = no_cfg = 0
for key, record in json.loads(dumped.read_text()).items():
code = record.get("code") or ""
row = {"gotos": len(re.findall(r"\bgoto\b", code)), "lines": code.count("\n"),
"refused": bool(record.get("refused")), "status": record["status"],
"loops": len(re.findall(r"for \(; ; \)", code)), "whiles": len(re.findall(r"\bwhile \(", code)),
"text": hashlib.sha256(code.encode()).hexdigest()[:16], "ged": None}
if record["status"] == "output" and not row["refused"]:
owner = owners.get(int(record["addr"], 16))
source = cfgs.get(owner[1], {}).get(owner[0]) if owner is not None else None
if owner is None:
row["ged"] = NO_OWNER
unowned += 1
elif source is None:
row["ged"] = NO_SOURCE_CFG
no_cfg += 1
else:
signal.alarm(GED_SECONDS)
try:
cfg = dec_cfg_of(code, record["name"])
row["ged"] = ged(source, cfg) if cfg is not None else None
except Timeout:
row["ged"] = "err:timeout"
except Exception as error: # noqa: BLE001
row["ged"] = "err:" + type(error).__name__
finally:
signal.alarm(0)
rows[key] = row
print(f"{opt} {project} {binary}: {len(rows)} rows so far; {unowned} without a DWARF owner, "
f"{no_cfg} with an owner and no source CFG (neither is scored)", flush=True)
meta = {k: manifest[k] for k in ("set", "r2s_sha256", "projects", "opt_levels") if k in manifest}
out.write_text(json.dumps({"schema": SCHEMA, **meta, "rows": rows}))
return len(rows)
# --- pairing (pure) ------------------------------------------------------------------------------
def load_scores(path: Path) -> tuple[dict, dict]:
"""`(rows, meta)` of a scored run; a bare `{key: row}` file (the scratch format) has no meta."""
data = json.loads(Path(path).read_text())
if isinstance(data.get("rows"), dict) and "schema" in data:
return data["rows"], {k: v for k, v in data.items() if k != "rows"}
return data, {}
def has_ged(row: dict) -> bool:
return isinstance(row.get("ged"), float)
def one_per_function(keys) -> set[str]:
"""One key per (opt, project, function): the first binary that holds it, libz before its static copies."""
seen, kept = set(), set()
for key in sorted(keys, key=lambda k: (0 if "libz" in k else 1, k)):
opt, project, _binary, function = key.split("|", 3)
if (opt, project, function) not in seen:
seen.add((opt, project, function))
kept.add(key)
return kept
def pair(base: dict, new: dict, unique: bool = False) -> dict:
"""The comparison of two scored runs over the functions both scored."""
if unique:
kept = one_per_function(base)
base = {k: v for k, v in base.items() if k in kept}
new = {k: v for k, v in new.items() if k in kept}
both = [k for k in sorted(base) if k in new and has_ged(base[k]) and has_ged(new[k])]
only_base = [k for k in base if has_ged(base[k]) and not (k in new and has_ged(new[k]))]
only_new = [k for k in new if has_ged(new[k]) and not (k in base and has_ged(base[k]))]
old = {k: base[k]["ged"] for k in both}
now = {k: new[k]["ged"] for k in both}
shared = [k for k in sorted(base) if k in new]
with_text = [k for k in shared if base[k].get("text") and new[k].get("text")]
return {
"scored": len(both),
"lost_scoring": len(only_base),
"gained_scoring": len(only_new),
"perfect": [sum(1 for k in both if old[k] == 0), sum(1 for k in both if now[k] == 0)],
"newly_perfect": sorted(k for k in both if old[k] > 0 and now[k] == 0),
"lost_perfect": sorted(k for k in both if old[k] == 0 and now[k] > 0),
"mean_ged": [statistics.mean(old.values()) if both else None, statistics.mean(now.values()) if both else None],
"total_ged": [sum(old.values()), sum(now.values())],
"better": sorted(k for k in both if now[k] < old[k]),
"worse": sorted(k for k in both if now[k] > old[k]),
"same": sum(1 for k in both if now[k] == old[k]),
"gotos_all": [sum(r.get("gotos", 0) for r in base.values()), sum(r.get("gotos", 0) for r in new.values())],
"gotos_scored": [sum(base[k].get("gotos", 0) for k in both), sum(new[k].get("gotos", 0) for k in both)],
"functions_with_goto": [sum(1 for k in both if base[k].get("gotos", 0) > 0),
sum(1 for k in both if new[k].get("gotos", 0) > 0)],
"functions": len(shared),
"text_changed": sum(1 for k in with_text if base[k]["text"] != new[k]["text"]) if with_text else None,
"text_compared": len(with_text),
"refused": [sum(1 for r in base.values() if r.get("refused")), sum(1 for r in new.values() if r.get("refused"))],
"unowned": [sum(1 for r in base.values() if r.get("ged") == NO_OWNER),
sum(1 for r in new.values() if r.get("ged") == NO_OWNER)],
"no_source_cfg": [sum(1 for r in base.values() if r.get("ged") == NO_SOURCE_CFG),
sum(1 for r in new.values() if r.get("ged") == NO_SOURCE_CFG)],
"newly_refused": sorted(k for k in shared if new[k].get("refused") and not base[k].get("refused")),
}
def format_report(label: str, report: dict) -> str:
mean = report["mean_ged"]
lines = [
f"{label}: scored {report['scored']} (lost scoring {report['lost_scoring']}, gained {report['gained_scoring']})",
f" perfect {report['perfect'][0]} -> {report['perfect'][1]} newly {len(report['newly_perfect'])} "
f"lost {len(report['lost_perfect'])}",
f" mean GED {'n/a' if mean[0] is None else f'{mean[0]:.2f}'} -> {'n/a' if mean[1] is None else f'{mean[1]:.2f}'} "
f"better {len(report['better'])} worse {len(report['worse'])} same {report['same']}",
f" gotos all {report['gotos_all'][0]} -> {report['gotos_all'][1]}; scored {report['gotos_scored'][0]} -> "
f"{report['gotos_scored'][1]}; scored fns with a goto {report['functions_with_goto'][0]} -> "
f"{report['functions_with_goto'][1]}",
f" total GED {report['total_ged'][0]:.0f} -> {report['total_ged'][1]:.0f}",
f" no source: no DWARF owner {report['unowned'][0]} -> {report['unowned'][1]}, "
f"owner without a CFG {report['no_source_cfg'][0]} -> {report['no_source_cfg'][1]}",
]
if report["text_changed"] is None:
lines.append(f" text changed: n/a (these scores carry no text hash); refused {report['refused'][0]} -> "
f"{report['refused'][1]}")
else:
lines.append(f" text changed {report['text_changed']} of {report['text_compared']} functions in both runs; "
f"refused {report['refused'][0]} -> {report['refused'][1]}, newly refused "
f"{len(report['newly_refused'])}")
return "\n".join(lines)
# --- command line ----------------------------------------------------------------------------------
def projects_of(args) -> tuple[str, tuple[str, ...]]:
if args.project:
return "adhoc", tuple(args.project)
if args.set is None:
raise SystemExit("name a pinned set (--set mining|heldout) or the projects (--project)")
if args.set == "heldout":
print("held-out set: run it nightly and at an item's exit, and do not choose rules by it", file=sys.stderr)
return args.set, SETS[args.set]
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = parser.add_subparsers(dest="command", required=True)
d = sub.add_parser("dump", help="r2s pddj of every function of the set")
d.add_argument("--tree", type=Path, required=True)
d.add_argument("--r2s", type=Path, required=True)
d.add_argument("--out", type=Path, required=True)
d.add_argument("--set", choices=sorted(SETS))
d.add_argument("--project", action="append")
d.add_argument("--opt-level", action="append")
d.add_argument("--decompilers", default=",".join(DECOMPILERS), help="whose tomls name the targets")
s = sub.add_parser("score", help="DecBench GED of every dumped rendering")
s.add_argument("--tree", type=Path, required=True)
s.add_argument("--dump", type=Path, required=True)
s.add_argument("--out", type=Path, required=True)
s.add_argument("--decbench", type=Path)
s.add_argument("--cache", type=Path)
q = sub.add_parser("pair", help="compare two scored runs")
q.add_argument("base", type=Path)
q.add_argument("new", type=Path)
q.add_argument("--label", default="")
q.add_argument("-u", "--unique", action="store_true", help="one entry per function across binaries")
q.add_argument("-v", "--verbose", action="store_true", help="name the worse, lost and newly refused")
args = parser.parse_args(argv)
if args.command == "dump":
name, projects = projects_of(args)
total = dump(args.tree, args.r2s, args.out, projects, tuple(args.opt_level or OPT_LEVELS),
tuple(d for d in args.decompilers.split(",") if d), set_name=name)
print(f"dumped {total} functions of {name}: {', '.join(projects)}")
return 0
if args.command == "score":
print(f"scored {score(args.tree, args.dump, args.out, args.decbench, args.cache)} functions")
return 0
base, base_meta = load_scores(args.base)
new, new_meta = load_scores(args.new)
sets = {base_meta.get("set"), new_meta.get("set")} - {None}
if len(sets) > 1:
print(f"refusing to pair a {base_meta.get('set')} run with a {new_meta.get('set')} run", file=sys.stderr)
return 2
report = pair(base, new, args.unique)
print(format_report(args.label or (next(iter(sets)) if sets else "pair"), report))
if args.verbose:
for heading, keys in (("worse", report["worse"]), ("lost perfect", report["lost_perfect"]),
("newly perfect", report["newly_perfect"]), ("newly refused", report["newly_refused"])):
for key in keys:
print(f" {heading}: {key}")
return 0
if __name__ == "__main__":
sys.exit(main())
tests/decbench/r2s_timing.py
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""Time r2s alone on the functions a DecBench run asked it for, one binary at a time, without the harness.
r2s_timing.py --r2s <r2s> --run <results tree> [--project NAME ...] > timing.tsv
The official driver's wall time for r2s includes the backend's progress checkpoint, a pickle of the whole partial
result (every 30 s since 2026-10-23; after every function before, which grew with the binary). This replays the same
`s <addr>; pddj` stream through the same runner (`tests/equiv/r2s_batch.py`) with no checkpoint, and prints per binary:
functions asked, rendered, declined, wall seconds (open, `aa`, every function) and the peak resident set of the r2s
process in kB.
Targets are the addresses the run's own tomls name (rendered plus failed), so the two measure the same set.
Each binary runs in its own Python process, serially, so the peak RSS is that binary's alone.
"""
from __future__ import annotations
import argparse
import json
import re
import resource
import subprocess
import sys
import time
import tomllib
from pathlib import Path
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE.parent / "equiv"))
sys.path.insert(0, str(HERE))
from run_gate import binaries_by_stem # noqa: E402
def targets(toml_path: Path) -> list[int]:
data = tomllib.loads(toml_path.read_text())
addresses = {int(v["address"], 16) if isinstance(v["address"], str) else int(v["address"])
for k, v in data.items() if k.startswith("functions.")}
addresses |= {int(a, 16) for a in data.get("failed_functions", []) if re.fullmatch(r"0x[0-9a-fA-F]+", a)}
return sorted(addresses)
def one(r2s: Path, binary: Path, addresses: list[int]) -> dict:
import r2s_batch
started = time.monotonic()
report = r2s_batch.run_batch(r2s, binary, addresses, function_timeout=300.0, startup_timeout=600.0)
wall = time.monotonic() - started
ok = sum(1 for a in report.answers if a.ok)
return {"asked": len(addresses), "rendered": ok, "declined": len(report.answers) - ok,
"seconds": round(wall, 2), "processes": report.processes,
"maxrss_kb": resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss}
def cells(run: Path, opts, projects) -> list[tuple[str, str, str, Path, Path | None]]:
"""`(opt, project, binary stem, toml, stripped file or None)` for every r2s toml of the run.
The stripped file is found by stem: `libz.so.1.2.13` on disk is the driver's binary `libz.so.1.2`.
"""
found = []
for opt in opts:
for toml_path in sorted(run.glob(f"{opt}/*/decompiled/r2sleigh_native_*.toml")):
project = toml_path.parts[-3]
if projects and project not in projects:
continue
binary = toml_path.name.removesuffix(".toml")[len("r2sleigh_native_"):]
stripped = binaries_by_stem(toml_path.parent.parent / "stripped").get(binary)
found.append((opt, project, binary, toml_path, stripped))
return found
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser()
p.add_argument("--r2s", type=Path)
p.add_argument("--run", type=Path)
p.add_argument("--project", action="append")
p.add_argument("--opt", action="append")
p.add_argument("--one", nargs=2, metavar=("BINARY", "TOML"))
args = p.parse_args(argv)
if args.one:
print(json.dumps(one(args.r2s, Path(args.one[0]), targets(Path(args.one[1])))))
return 0
print("project\topt\tbinary\tasked\trendered\tdeclined\tseconds\tmaxrss_kb")
timed = unresolved = failed = 0
for opt, project, binary, toml_path, stripped in cells(args.run, args.opt or ("O2", "O2-noinline"), args.project):
if stripped is None:
unresolved += 1
print(f"{project}\t{opt}\t{binary}\tNO-STRIPPED-BINARY\t\t\t\t", flush=True)
continue
out = subprocess.run([sys.executable, __file__, "--r2s", str(args.r2s), "--one", str(stripped), str(toml_path)],
capture_output=True, text=True, check=False)
try:
r = json.loads(out.stdout.strip().splitlines()[-1])
except (ValueError, IndexError):
failed += 1
print(f"{project}\t{opt}\t{binary}\tFAILED\t\t\t\t", flush=True)
print(out.stderr[-300:], file=sys.stderr)
continue
timed += 1
print(f"{project}\t{opt}\t{binary}\t{r['asked']}\t{r['rendered']}\t{r['declined']}\t{r['seconds']}\t{r['maxrss_kb']}",
flush=True)
print(f"timed {timed} binaries, {unresolved} without a stripped binary, {failed} failed", file=sys.stderr)
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/decbench/r2sleigh_raw.py
+81 -9
View File
@@ -34,8 +34,14 @@ and the DWARF ``low_pc`` of every source function:
or ``harness: <how r2s ended> while rendering <addr>`` (a crash or a
deadline, after which r2s is restarted at the next function).
``rendered + declined == requested`` is checked before returning.
* After every function the partial result is pickled to ``progress_path``, so
the driver's hard kill loses nothing already answered.
* The partial result is pickled to ``progress_path`` every
``$R2SLEIGH_CHECKPOINT_SECONDS`` (default 30) and once more at the end, so
the driver's hard kill loses the answers of one interval at most. A pickle
costs time proportional to every function answered so far, which is why it
is not written per function (2.5 MB each on bzip2, 18 MB on bash). A run that
finds a checkpoint of the same r2s, binary and targets at ``progress_path``
asks r2s only for the addresses it has no answer for; the official driver
gives each attempt a fresh path, so that is for a manual re-run.
* It fails closed on a binary carrying ``.symtab`` or any debug section
(``.debug_*``, compressed ``.zdebug_*``, or the MiniDebugInfo
``.gnu_debugdata``): r2s reads the declarations of the file it opens, and
@@ -54,7 +60,9 @@ from __future__ import annotations
import hashlib
import json
import logging
import math
import os
import pickle
import re
import shutil
import struct
@@ -84,6 +92,9 @@ log = logging.getLogger(__name__)
BACKEND = "r2sleigh_native"
CHECKPOINT_SECONDS = 30.0
_clock = time.monotonic # replaced in tests, which never sleep
# Sections whose presence means r2s would read facts DecBench grades against:
# the symbol table, and debug information in any of its spellings.
FAIL_CLOSED_SECTIONS = (".symtab", ".gnu_debugdata")
@@ -171,6 +182,35 @@ def discover(executable: Path, binary: Path, timeout: float) -> tuple[list[tuple
return functions, ending
def checkpoint_interval() -> float:
"""Seconds between checkpoints: ``$R2SLEIGH_CHECKPOINT_SECONDS``, 30 by default, 0 for every answer."""
try:
seconds = float(os.environ.get("R2SLEIGH_CHECKPOINT_SECONDS", CHECKPOINT_SECONDS))
except ValueError:
return CHECKPOINT_SECONDS
return max(0.0, seconds) if math.isfinite(seconds) else CHECKPOINT_SECONDS
def resume_state(progress_path: Path | None, identity: dict) -> tuple[dict, dict] | None:
"""The rendered functions and declines of a checkpoint of this very task, or None.
``identity`` names the r2s build, the binary's bytes and the targets with their names; a checkpoint
of anything else is not read, so a stale file cannot put another run's answers into this one.
"""
if progress_path is None or not Path(progress_path).is_file():
return None
try:
saved = pickle.loads(Path(progress_path).read_bytes())
block = (saved.decompiler.extra or {}).get("resume")
if not isinstance(block, dict) or block.get("identity") != identity:
return None
declined = {int(address): (str(label), str(cause))
for address, label, cause in block["declined"]}
return dict(saved.functions), declined
except Exception: # noqa: BLE001 - a checkpoint that cannot be read is no checkpoint
return None
def stack_offset(location: str) -> int | None:
"""``stack-0x14``, ``[rbp-0x14]`` or ``sp+8`` -> the signed offset."""
found = _STACK_OFFSET.search(location.strip())
@@ -302,7 +342,7 @@ class R2sDecompiler(Decompiler):
rendered: dict[str, FunctionDecompilation] = {}
extra: dict = {"requested_from": source}
def result() -> DecompilationResult:
def result(resume: dict | None = None) -> DecompilationResult:
extra.update({
"requested": len(requested),
"rendered": len(rendered),
@@ -317,7 +357,7 @@ class R2sDecompiler(Decompiler):
decompiler_version=self.get_version(),
total_time_seconds=time.time() - started,
failed_functions=sorted(label for label, _ in declined.values()),
extra=dict(extra),
extra={**extra, "resume": resume} if resume is not None else dict(extra),
),
functions=dict(rendered),
output_dir=output_dir,
@@ -354,6 +394,34 @@ class R2sDecompiler(Decompiler):
+ (f" ({ending})" if ending else ""))
return result()
# The checkpoint names its task, so only a re-run of the same r2s, bytes and targets reads it.
identity = {
"version": self.get_version(),
"binary": binary_path.name,
"sha256": hashlib.sha256(binary_path.read_bytes()).hexdigest()
if progress_path is not None else "",
"requested": [[address, name] for address, name in requested.items()],
}
resumed = resume_state(progress_path, identity)
if resumed is not None:
rendered.update(resumed[0])
declined.update(resumed[1])
extra["resumed"] = len(rendered) + len(declined)
answered = {function.address for function in rendered.values()} | set(declined)
interval = checkpoint_interval()
saved_at = _clock()
def checkpoint() -> None:
nonlocal saved_at
saved_at = _clock()
if progress_path is not None:
common.dump_progress(progress_path, result({
"identity": identity,
"declined": [[address, label, cause]
for address, (label, cause) in declined.items()],
}))
def file_answer(answer: "r2s_batch.Answer") -> None:
name = requested.get(answer.address)
key = name or f"0x{answer.address:x}"
@@ -364,12 +432,16 @@ class R2sDecompiler(Decompiler):
rendered[function.name] = function
else:
declined[answer.address] = (key, answer.cause or "harness: no answer")
common.dump_progress(progress_path, result())
if _clock() - saved_at >= interval:
checkpoint()
report = r2s_batch.run_batch(
executable, binary_path, list(requested), function_timeout=timeout,
startup_timeout=startup, on_answer=file_answer,
)
try:
report = r2s_batch.run_batch(
executable, binary_path, [a for a in requested if a not in answered],
function_timeout=timeout, startup_timeout=startup, on_answer=file_answer,
)
finally:
checkpoint()
extra["processes"] = report.processes
extra["process_endings"] = report.endings
if len(rendered) + len(declined) != len(requested):
tests/decbench/report_decbench.py
+156 -1
View File
@@ -9,6 +9,15 @@ GED and VJ-GED are distances (zero is perfect), so their all-function number use
``1 / (1 + distance)``. This gives those metrics the same zero-is-worst quality
scale as byte_match and type_match; filling a refused distance with raw zero would
incorrectly reward it as a perfect graph.
It also refuses a run that lost rows. A row is *scored* by a decompiler when the
result has a metric value for it. Decompilers are compared only over the rows
every compared decompiler scored; the report says how many rows that left out and
how many rows each decompiler has no score for, and the exit status is 73 when a
compared decompiler scored nothing in a project, or fewer than
``--min-scored-fraction`` of its rows. (The 2026-10-17 run lost every bash
evaluation of kuna, ghidra and r2s, and the scoreboard compared 692 angr-only rows
as misses for the rest.)
"""
from __future__ import annotations
@@ -123,6 +132,125 @@ def collect(results: dict) -> Collected:
)
LOST_ROWS = 73
PERFECT_DEFAULT = {"byte_match": 1.0, "ged": 0.0, "vj_ged": 0.0, "type_match": 1.0}
def row_project(key: str) -> str:
return key.split("/", 1)[0]
def compared_decompilers(results: dict) -> list[str]:
"""The decompilers this result set measured: the versions it records, else what its rows name."""
named = set(results.get("decompiler_versions") or {}) or set(results.get("decompilers") or [])
if not named:
for group in results.get("groups", []):
for function in group.get("functions", []):
named.update(function.get("values") or {})
named.update(function.get("decompiled") or {})
return sorted(named)
@dataclass
class Coverage:
decompilers: list[str]
rows: int
common: list[str]
no_score: dict[str, int]
by_project: dict[str, dict[str, tuple[int, int]]]
failures: list[str]
results: dict
def coverage(results: dict, decompilers: list[str], min_fraction: float = 0.0) -> Coverage:
"""Who scored which rows, the rows all of ``decompilers`` scored, and the shortfalls.
A decompiler is owed the rows of the cells it ran in (``sweep.decompiler_cells``, when the
result set is a merge of shards), else every row. It fails a project where it scored none
of its rows, or fewer than ``min_fraction`` of them.
"""
cells_of = (results.get("sweep") or {}).get("decompiler_cells") or {}
scored: dict[str, set[str]] = {d: set() for d in decompilers}
owed: dict[str, set[str]] = {d: set() for d in decompilers}
rows: set[str] = set()
for group in results.get("groups", []):
cell = cell_key(group["project"], group["opt_level"])
for function in group.get("functions", []):
key = function_key(group, function)
rows.add(key)
for decompiler in decompilers:
if decompiler in cells_of and cell not in cells_of[decompiler]:
continue
owed[decompiler].add(key)
values = (function.get("values") or {}).get(decompiler) or {}
if any(value is not None for value in values.values()):
scored[decompiler].add(key)
by_project: dict[str, dict[str, tuple[int, int]]] = {}
failures = []
for decompiler in decompilers:
for project in sorted({row_project(key) for key in owed[decompiler]}):
total = sum(1 for key in owed[decompiler] if row_project(key) == project)
got = sum(1 for key in scored[decompiler] if row_project(key) == project)
by_project.setdefault(project, {})[decompiler] = (got, total)
if got == 0 or got < min_fraction * total:
failures.append(f"{decompiler} scored {got} of {total} rows of {project}")
common = sorted(set.intersection(*scored.values())) if scored else []
return Coverage(
decompilers, len(rows), common,
{d: len(owed[d]) - len(scored[d]) for d in decompilers},
by_project, failures, results,
)
def common_rows_table(cover: Coverage) -> dict[str, dict[str, dict]]:
"""Per metric and decompiler, over the common rows: how many have the metric, the mean, how many are perfect."""
perfect = {**PERFECT_DEFAULT, **(cover.results.get("perfect_values") or {})}
wanted = set(cover.common)
values: dict[str, dict[str, list[float]]] = {}
for group in cover.results.get("groups", []):
for function in group.get("functions", []):
if function_key(group, function) not in wanted:
continue
for decompiler in cover.decompilers:
for metric, value in ((function.get("values") or {}).get(decompiler) or {}).items():
if value is not None:
values.setdefault(metric, {}).setdefault(decompiler, []).append(float(value))
return {
metric: {
decompiler: {
"n": len(per[decompiler]),
"mean": sum(per[decompiler]) / len(per[decompiler]),
"perfect": sum(1 for v in per[decompiler] if abs(v - perfect.get(metric, 0.0)) < 1e-9),
}
for decompiler in cover.decompilers if per.get(decompiler)
}
for metric, per in sorted(values.items())
}
def print_coverage(cover: Coverage) -> None:
names = ", ".join(cover.decompilers)
print(f"compared decompilers: {names}")
print(
f"rows: {cover.rows}; scored by all {len(cover.decompilers)}: {len(cover.common)}; "
f"left out of the comparison: {cover.rows - len(cover.common)}"
)
print("rows with no score: " + ", ".join(
f"{decompiler} {count}" for decompiler, count in cover.no_score.items()))
print("scored / rows, per project:")
for project, per in sorted(cover.by_project.items()):
print(f" {project:12} " + ", ".join(
f"{decompiler} {got}/{total}" for decompiler, (got, total) in per.items()))
table = common_rows_table(cover)
if table:
print(f"over the {len(cover.common)} common rows: metric, then per decompiler "
"perfect/n and mean")
for metric, per in table.items():
print(f" {metric:11} " + "; ".join(
f"{decompiler} {data['perfect']}/{data['n']} {data['mean']:.3f}"
for decompiler, data in per.items()))
def quality(metric: str, value: float) -> float:
"""Map a rendered metric to the zero-is-worst coverage scale."""
if metric in LOWER_IS_BETTER:
@@ -618,13 +746,40 @@ def main() -> int:
parser.add_argument("--results", type=Path, required=True)
parser.add_argument("--baseline", type=Path, required=True)
parser.add_argument("--accept-baseline", action="store_true")
parser.add_argument(
"--compare",
help="comma list of the decompilers to compare (default: every one the results name)",
)
parser.add_argument(
"--min-scored-fraction",
type=float,
default=0.0,
help="fail when a compared decompiler scored fewer than this share of a project's "
"rows (default 0: fail only when it scored none)",
)
args = parser.parse_args()
try:
current = collect(json.loads(args.results.read_text()))
raw = json.loads(args.results.read_text())
current = collect(raw)
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
print(f"invalid result set: {exc}", file=sys.stderr)
return 70
cover = coverage(
raw,
args.compare.split(",") if args.compare else compared_decompilers(raw),
args.min_scored_fraction,
)
print_coverage(cover)
if cover.failures:
print(
f"ERROR: {len(cover.failures)} project/decompiler pairs lost their rows; "
"no number of this run is comparable:",
file=sys.stderr,
)
for line in cover.failures:
print(f" {line}", file=sys.stderr)
return LOST_ROWS
baseline, invalid = usable_baseline(
json.loads(args.baseline.read_text()) if args.baseline.exists() else None
)
tests/decbench/run_decbench.sh
+27 -5
View File
@@ -16,8 +16,16 @@
# 4. optionally compile the projects (DecBench's compile_all.py);
# 5. run scripts/run_benchmark.py through decbench_cli.py, which registers
# the vj_ged metric when DecBench does not;
# 6. check every r2sleigh_native result names that sha256, then report
# against baseline.json (report_decbench.py).
# 6. check every binary the run decompiled has its scores in evaluated/
# (run_gate.py: a run that loses rows stops here with status 71), check
# every r2sleigh_native result names that sha256, then report against
# baseline.json (report_decbench.py, which compares decompilers only over
# the rows all of them scored and stops with status 73 when one scored
# nothing in a project).
#
# More than one worker is refused for a binary whose decompiled C exceeds 8 MB
# (r2s's bash: 13 MB, 5.2 GB per evaluation worker; run_gate.py): up front for
# C already in --out, and inside the driver before it evaluates.
#
# The driver needs cgroup v2 and a user systemd manager (`systemd-run --user`)
# for its per-binary limits; it says so and stops when they are missing.
@@ -35,6 +43,7 @@ plan=0
decompilers=r2sleigh_native,angr
metrics=byte_match,ged,vj_ged,type_match
workers=${R2SLEIGH_DECBENCH_WORKERS:-4}
compare=
declare -a projects=()
declare -a opts=()
@@ -51,7 +60,9 @@ usage: tests/decbench/run_decbench.sh [options]
--opt-level OPT O0, O2 or O2_noinline; repeatable (default: O0 O2)
--decompilers LIST (default: r2sleigh_native,angr)
--metrics LIST (default: byte_match,ged,vj_ged,type_match)
--workers N DecBench workers (default: 4)
--workers N DecBench workers (default: 4); 1 for bash
--compare LIST decompilers report_decbench.py compares, over the rows
all of them scored (default: every one the results name)
--r2s PATH measure this r2s instead of building target/release/r2s
--compile compile the projects first (DecBench's compile_all.py)
--accept-baseline start or extend baseline.json with this run
@@ -69,6 +80,7 @@ while [[ $# -gt 0 ]]; do
--decompilers) decompilers=$2; shift 2 ;;
--metrics) metrics=$2; shift 2 ;;
--workers) workers=$2; shift 2 ;;
--compare) compare=$2; shift 2 ;;
--r2s) r2s=$2; shift 2 ;;
--compile) compile=1; shift ;;
--accept-baseline) accept=1; shift ;;
@@ -78,6 +90,8 @@ while [[ $# -gt 0 ]]; do
esac
done
(( ${#opts[@]} )) || opts=(O0 O2)
[[ $workers =~ ^[1-9][0-9]*$ ]] || { echo "--workers wants a positive integer, not '$workers'" >&2; exit 64; }
started=$(date +%s)
out=${out:-$root/tests/decbench/artifacts/run-$(date -u +%Y%m%dT%H%M%SZ)}
opt_list=$(IFS=,; echo "${opts[*]}")
@@ -88,6 +102,11 @@ run() {
(( plan )) || "$@"
}
if (( workers > 1 )); then
python3 "$root/tests/decbench/run_gate.py" workers --tree "$out" --workers "$workers" \
--decompilers "$decompilers" || exit $?
fi
if [[ -z $r2s ]]; then
run cargo build --release -p r2s --features sleigh --manifest-path "$root/Cargo.toml"
r2s=${CARGO_TARGET_DIR:-$root/target}/release/r2s
@@ -113,7 +132,7 @@ fi
export R2SLEIGH_R2S_BIN=$r2s
export R2SLEIGH_REFUSAL_CENSUS_DIR=$out/refusals
if (( compile )); then
run env -C "$decbench" "$python" scripts/compile_all.py "$out" "$workers" "${projects[@]}"
run env -C "$decbench" "$python" scripts/compile_all.py "$out" "$workers" ${projects[@]+"${projects[@]}"}
fi
declare -a only=()
(( ${#projects[@]} )) && only=(-- "${projects[@]}")
@@ -123,8 +142,10 @@ run env -C "$decbench" \
DECBENCH_METRICS="$metrics" \
DECBENCH_WORKERS="$workers" \
R2SLEIGH_DECBENCH_PHASE_LOG="$out/phases.tsv" \
"$python" "$root/tests/decbench/decbench_cli.py" run-benchmark "$out" "${only[@]}"
"$python" "$root/tests/decbench/decbench_cli.py" run-benchmark "$out" ${only[@]+"${only[@]}"}
run python3 "$root/tests/decbench/run_gate.py" evaluated --tree "$out" \
--decompilers "$decompilers" --since "$started"
(( plan )) && exit 0
results=$out/function_results.json
python3 - "$results" "$witness" <<'PY'
@@ -138,4 +159,5 @@ PY
declare -a report=(python3 "$root/tests/decbench/report_decbench.py" --results "$results"
--baseline "$root/tests/decbench/baseline.json")
(( accept )) && report+=(--accept-baseline)
[[ -n $compare ]] && report+=(--compare "$compare")
run "${report[@]}"
tests/decbench/run_gate.py
+195
View File
@@ -0,0 +1,195 @@
#!/usr/bin/env python3
"""Fail a DecBench run that lost rows, and refuse an evaluation that the host will kill.
run_gate.py evaluated --tree <results tree> --decompilers a,b [--since EPOCH]
run_gate.py workers --tree <results tree> --workers N --decompilers a,b
On 2026-10-17 the host's OOM killer took the evaluation worker of every bash binary, `evaluate_project` returned `{}`,
and the driver went on: kuna, ghidra and r2s had no bash row, and the scoreboard dropped them without a word.
* `evaluated`: every binary a decompiler rendered functions for in this run (its `decompiled/<dec>_<binary>.toml` is
newer than `--since` and says `function_count > 0`) needs an `evaluated/<binary>.toml` that carries a score key of that
decompiler. A missing file, a file of just `binary = "bash"` (16 bytes) and a file that scores only other
decompilers all stop the run with exit status 71.
* `workers`: one evaluation worker needs 5.2 GB for the 13 MB of C that r2s writes for bash, so more than one worker on
a binary whose decompiled C exceeds 8 MB is refused (exit status 72). The same check runs inside the driver through
`install_workers_guard`, before `evaluate_project`, where the C is known; it raises `SystemExit` because the driver
catches `Exception` around the evaluation and carries on with `{}`.
"""
from __future__ import annotations
import argparse
import functools
import inspect
import os
import re
import sys
from pathlib import Path
LOST = 71
REFUSED = 72
MAX_PARALLEL_C_BYTES = 8 * 1024 * 1024
# Read without a TOML parser, so the shell can run this under any python3: DecBench writes
# `function_count = N` and quotes the dotted score keys, `"<decompiler>.<metric>.total" = 0.5`.
FUNCTION_COUNT = re.compile(r"^function_count\s*=\s*(\d+)", re.MULTILINE)
SCORE_KEY = re.compile(r'^"?([^".=\s]+)\.[^"=]*"?\s*=', re.MULTILINE)
def split_name(stem: str, decompilers: list[str]) -> tuple[str, str] | None:
"""`r2sleigh_native_bash` -> (`r2sleigh_native`, `bash`), the longest decompiler name that fits."""
for decompiler in sorted(decompilers, key=len, reverse=True):
if stem.startswith(decompiler + "_") and len(stem) > len(decompiler) + 1:
return decompiler, stem[len(decompiler) + 1:]
return None
def binaries_by_stem(directory: Path) -> dict[str, Path]:
"""`{stem: file}` of a run's `stripped/` directory: `libz.so.1.2.13` is the driver's binary `libz.so.1.2`."""
return {path.stem: path for path in sorted(directory.glob("*")) if path.is_file()}
def rendered_binaries(tree: Path, decompilers: list[str], since: float | None = None) -> list[dict]:
"""The (opt, project, binary, decompiler) cells the driver decompiled into this tree, with function counts."""
cells = []
for toml in sorted(tree.glob("*/*/decompiled/*.toml")):
named = split_name(toml.stem, decompilers)
if named is None or (since is not None and int(toml.stat().st_mtime) < since):
continue
try:
found = FUNCTION_COUNT.search(toml.read_text())
except (OSError, ValueError):
found = None
count = int(found.group(1)) if found else 0
if count > 0:
opt, project = toml.parts[-4], toml.parts[-3]
cells.append({"opt": opt, "project": project, "binary": named[1], "decompiler": named[0],
"functions": count, "dir": toml.parent.parent, "toml": toml})
return cells
def lost_evaluations(tree: Path, decompilers: list[str], since: float | None = None) -> tuple[int, list[str]]:
"""How many cells were checked, and one message per cell whose scores are not in the evaluated file."""
cells = rendered_binaries(tree, decompilers, since)
lost = []
for cell in cells:
where = f"{cell['opt']}/{cell['project']}/evaluated/{cell['binary']}.toml"
evaluated = cell["dir"] / "evaluated" / f"{cell['binary']}.toml"
who = f"{cell['decompiler']} rendered {cell['functions']} functions of {cell['binary']}"
if not evaluated.is_file():
reason = f"{where} is missing"
else:
try:
scored = sorted(set(SCORE_KEY.findall(evaluated.read_text())))
except (OSError, ValueError):
scored = None
size = evaluated.stat().st_size
if scored is None:
reason = f"{where} cannot be read ({size} bytes)"
elif cell["decompiler"] not in scored:
reason = (f"{where} is {size} bytes and scores "
f"{', '.join(scored) if scored else 'no decompiler'}, not {cell['decompiler']}")
else:
continue
c_file = cell["toml"].with_suffix(".c")
if c_file.is_file() and c_file.stat().st_size > MAX_PARALLEL_C_BYTES:
reason += (f"; its decompiled C is {c_file.stat().st_size / 2**20:.1f} MB, so the evaluation worker was "
"most likely OOM-killed (evaluate with --workers 1)")
lost.append(f"{who}, but {reason}")
return len(cells), lost
def c_bytes_of(decompilations: dict) -> dict[tuple[str, str], int]:
"""`{binary: {decompiler: DecompilationResult}}` -> the bytes of C each pair holds."""
sizes = {}
for binary, per_decompiler in decompilations.items():
for decompiler, result in per_decompiler.items():
functions = getattr(result, "functions", None) or {}
sizes[(str(binary), str(decompiler))] = sum(
len((getattr(function, "decompiled_code", "") or "").encode()) for function in functions.values())
return sizes
def unsafe_workers(sizes: dict[tuple[str, str], int], workers: int | None) -> list[str]:
"""One line per binary whose decompiled C is over the limit when more than one worker would evaluate it."""
if (workers if workers is not None else os.cpu_count() or 1) <= 1:
return []
return [f"{binary} ({decompiler}): {size / 2**20:.1f} MB of C"
for (binary, decompiler), size in sorted(sizes.items()) if size > MAX_PARALLEL_C_BYTES]
def refuse_unsafe_workers(decompilations: dict, workers: int | None) -> None:
over = unsafe_workers(c_bytes_of(decompilations), workers)
if over:
raise SystemExit(
f"refusing to evaluate with {workers if workers is not None else 'all'} workers: "
+ "; ".join(over)
+ f" (limit {MAX_PARALLEL_C_BYTES // 2**20} MB). One evaluation worker for r2s's bash holds 5.2 GB, "
"so parallel workers are OOM-killed and the binary loses every score. Run it with --workers 1.")
def install_workers_guard() -> None:
"""Put `refuse_unsafe_workers` in front of DecBench's `evaluate_project`, before the driver imports the name."""
from decbench.pipeline import evaluate
original = evaluate.evaluate_project
if getattr(original, "_r2sleigh_guard", False):
return
signature = inspect.signature(original)
@functools.wraps(original)
def guarded(*args, **kwargs):
given = signature.bind_partial(*args, **kwargs).arguments
refuse_unsafe_workers(given.get("decompilations") or {}, given.get("workers"))
return original(*args, **kwargs)
guarded._r2sleigh_guard = True
evaluate.evaluate_project = guarded
def tree_c_bytes(tree: Path, decompilers: list[str]) -> dict[tuple[str, str], int]:
"""The decompiled C files already in a tree: `(opt/project/binary, decompiler) -> bytes`."""
sizes = {}
for c_file in sorted(tree.glob("*/*/decompiled/*.c")):
named = split_name(c_file.stem, decompilers)
if named is not None:
sizes[(f"{c_file.parts[-4]}/{c_file.parts[-3]}/{named[1]}", named[0])] = c_file.stat().st_size
return sizes
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
parser.add_argument("check", choices=("evaluated", "workers"))
parser.add_argument("--tree", type=Path, required=True)
parser.add_argument("--decompilers", default="")
parser.add_argument("--since", type=float)
parser.add_argument("--workers", type=int, default=1)
args = parser.parse_args(argv)
decompilers = [d for d in args.decompilers.split(",") if d]
if not decompilers:
parser.error("--decompilers is required")
if args.check == "workers":
over = unsafe_workers(tree_c_bytes(args.tree, decompilers), args.workers)
if over:
print(f"refusing --workers {args.workers}: " + "; ".join(over) + f" in {args.tree} "
f"(limit {MAX_PARALLEL_C_BYTES // 2**20} MB); one evaluation worker for r2s's bash holds 5.2 GB. "
"Use --workers 1.", file=sys.stderr)
return REFUSED
return 0
checked, lost = lost_evaluations(args.tree, decompilers, args.since)
if lost:
print(f"ERROR: {len(lost)} of {checked} rendered binaries have no evaluation; their rows would be dropped "
"from the scoreboard:", file=sys.stderr)
for line in lost[:30]:
print(f" {line}", file=sys.stderr)
if len(lost) > 30:
print(f" ... and {len(lost) - 30} more", file=sys.stderr)
return LOST
print(f"evaluations present for all {checked} binaries the run decompiled")
return 0
if __name__ == "__main__":
sys.exit(main())
tests/decbench/show_function.py
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env python3
"""Print one function of a DecBench run: the source body, the r2s rendering, GED and type_match, and DWARF variables.
PYTHONPATH=<decbench> <venv>/bin/python show_function.py --run <tree> PROJECT OPT BINARY FUNCTION [--dec angr]
"""
from __future__ import annotations
import argparse
import pickle
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from ged_causes import function_body # noqa: E402
def main(argv: list[str] | None = None) -> int:
p = argparse.ArgumentParser()
p.add_argument("--run", required=True, type=Path)
p.add_argument("--dec", default="r2sleigh_native")
p.add_argument("--no-source", action="store_true")
p.add_argument("--graphs", action="store_true", help="print the source's and the rendering's block graphs")
p.add_argument("--short", action="store_true", help="omit the rendering's helper prelude and declarations")
p.add_argument("project")
p.add_argument("opt")
p.add_argument("binary")
p.add_argument("function")
a = p.parse_args(argv)
from decbench.models.project import OptimizationLevel, Project
from decbench.pipeline.executor import PipelineConfig, PipelineExecutor
from decbench.results_store import gather_project_tomls
from decbench.metrics.type_match import _ground_truth_for_function, extract_ground_truth_type_index
from decbench.utils.binfmt import detect, source_function_owners
from decbench.utils.dwarf_policy import dwarf_follow_abstract_origin
ck = pickle.loads((a.run / "checkpoints" / f"{a.project}.pkl").read_bytes())
opt = OptimizationLevel(a.opt)
res = ck["decompile"][opt][a.binary][a.dec]
fn = res.functions[a.function]
ev = ck["evaluate"][opt][a.binary][a.dec]
print(f"== {a.dec} {a.project}/{a.opt}/{a.binary} {a.function} @ {hex(fn.address)} metadata gotos={fn.metadata.get('gotos')} "
f"bools={fn.metadata.get('bools')} residual={fn.metadata.get('residual')}")
for m in ("ged", "type_match", "byte_match"):
if m in ev and a.function in ev[m].function_results:
v = ev[m].function_results[a.function]
print(f"{m}: {v.value} {({k: v.metadata[k] for k in v.metadata if k in ('tp','fp','fn','gt_vars','source_nodes','source_edges','decompiled_nodes','decompiled_edges')})}")
info = detect(res.binary_path)
variables = _ground_truth_for_function(extract_ground_truth_type_index(res.binary_path), a.function,
int(fn.address), info.arch if info is not None else "")
if variables:
print("DWARF variables:")
for v in variables:
print(" ", v["name"], "arg" if v["is_arg"] else "loc", v["type"][:5], "stack" if v["rbp_offset"] else "")
# The unit that defines the function is the one the binary's DWARF names for its address, never a guess by name.
units, owner = {}, None
if not a.no_source or a.graphs:
project = Project.from_toml(next(t for t in gather_project_tomls() if t.stem == a.project))
PipelineExecutor(PipelineConfig(output_dir=a.run, optimization_levels=[opt]))._discover_existing_binaries([project], a.run)
units = {stem: Path(path) for stem, path in project.preprocessed_sources.get(opt, {}).items()}
owner = source_function_owners(
res.binary_path, set(units), follow_abstract_origin=dwarf_follow_abstract_origin()).get(int(fn.address))
if owner is None:
print(f"--- source: no DWARF owner at {hex(fn.address)}")
if owner is not None and not a.no_source:
body = function_body(units[owner[1]].read_text(errors="replace"), owner[0])
if body:
print(f"--- source ({units[owner[1]].name})")
print(body)
if a.graphs:
from decbench.utils.cfg import extract_cfgs_from_decompilation, extract_cfgs_from_source
sub = res.model_copy(update={"functions": {a.function: fn}})
dgraph = extract_cfgs_from_decompilation(sub).get(a.function)
sgraph = extract_cfgs_from_source(units[owner[1]]).get(owner[0]) if owner is not None else None
for label, g in (("source", sgraph), (a.dec, dgraph)):
if g is None:
continue
print(f"--- {label} block graph: {g.number_of_nodes()} blocks, {g.number_of_edges()} edges")
order = {n: i for i, n in enumerate(sorted(g.nodes, key=lambda n: n.addr if hasattr(n, "addr") else 0))}
for n in sorted(g.nodes, key=lambda n: order[n]):
role = ("E" if getattr(n, "is_entrypoint", False) else "") + ("X" if getattr(n, "is_exitpoint", False) else "")
print(f" b{order[n]}[{len(n.statements)}]{role} -> " + ", ".join(f"b{order[m]}" for m in g.successors(n)))
print(f"--- {a.dec} rendering")
code = fn.decompiled_code
if a.short:
code = code[code.index(a.function + "("):] if (a.function + "(") in code else code
code = "\n".join(line for line in code.splitlines() if not line.strip().startswith(("/* r2dec proof",)))
print(code)
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/decbench/test_decbench_report.py
+126
View File
@@ -2,10 +2,15 @@
from __future__ import annotations
import contextlib
import importlib.util
import io
import json
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
HERE = Path(__file__).resolve().parent
@@ -47,6 +52,127 @@ def payload(project: str, score: float | None, decompiled: bool = True) -> dict:
}
def results_of(projects: dict[str, list[dict]], **extra) -> dict:
"""Result set: per project, one function per entry; an entry maps a decompiler to its metrics, or None."""
decompilers = sorted({d for rows in projects.values() for row in rows for d in row})
return {
"schema_version": 2,
"decompilers": decompilers,
"decompiler_versions": {d: "test" for d in decompilers},
"metrics": ["byte_match", "ged"],
"perfect_values": {"byte_match": 1.0, "ged": 0.0},
"groups": [
{
"project": project,
"binary": "b",
"opt_level": "O2",
"functions": [
{
"function": f"f{number}",
"values": {d: m for d, m in row.items() if m},
"decompiled": {d: m is not None for d, m in row.items()},
}
for number, row in enumerate(rows)
],
}
for project, rows in projects.items()
],
**extra,
}
def perfect() -> dict:
return {"byte_match": 1.0, "ged": 0.0}
def worse() -> dict:
return {"byte_match": 0.5, "ged": 3.0}
class CoverageTests(unittest.TestCase):
def test_a_project_one_decompiler_lost_stops_the_report_before_any_number(self) -> None:
# The 2026-10-17 bash: three decompilers lost every row, angr kept some.
lost = {"angr": perfect(), "kuna": None, "r2sleigh_native": None}
kept = {"angr": perfect(), "kuna": worse(), "r2sleigh_native": worse()}
data = results_of({"bash": [lost, lost, lost], "zlib": [kept, kept]})
cover = report.coverage(data, report.compared_decompilers(data))
self.assertEqual(cover.failures, [
"kuna scored 0 of 3 rows of bash",
"r2sleigh_native scored 0 of 3 rows of bash",
])
with tempfile.TemporaryDirectory() as directory:
results = Path(directory) / "function_results.json"
results.write_text(json.dumps(data))
baseline = Path(directory) / "baseline.json"
argv = ["report_decbench.py", "--results", str(results), "--baseline", str(baseline),
"--accept-baseline"]
stderr = io.StringIO()
with (
mock.patch.object(sys, "argv", argv),
contextlib.redirect_stdout(io.StringIO()),
contextlib.redirect_stderr(stderr),
):
status = report.main()
self.assertEqual(status, report.LOST_ROWS)
self.assertIn("kuna scored 0 of 3 rows of bash", stderr.getvalue())
self.assertFalse(baseline.exists()) # a lossy run is not accepted as the record
def test_decompilers_are_compared_only_on_rows_every_one_scored(self) -> None:
both = {"angr": perfect(), "r2sleigh_native": worse()}
only_r2s = {"angr": None, "r2sleigh_native": perfect()}
only_angr = {"angr": perfect(), "r2sleigh_native": None}
data = results_of({"zlib": [both, both, only_r2s, only_r2s, only_r2s, only_angr]})
cover = report.coverage(data, ["angr", "r2sleigh_native"])
self.assertEqual(cover.failures, [])
self.assertEqual(cover.rows, 6)
self.assertEqual(len(cover.common), 2)
self.assertEqual(cover.no_score, {"angr": 3, "r2sleigh_native": 1})
self.assertEqual(cover.by_project["zlib"], {"angr": (3, 6), "r2sleigh_native": (5, 6)})
table = report.common_rows_table(cover)
# Over the two shared rows r2s is perfect on none, angr on both; the three rows only r2s
# scored, all perfect, do not make it look better.
self.assertEqual(table["ged"]["angr"], {"n": 2, "mean": 0.0, "perfect": 2})
self.assertEqual(table["ged"]["r2sleigh_native"], {"n": 2, "mean": 3.0, "perfect": 0})
def test_the_report_says_how_many_rows_it_left_out(self) -> None:
row = {"angr": perfect(), "r2sleigh_native": None}
data = results_of({"zlib": [row, row, row]})
out = io.StringIO()
with contextlib.redirect_stdout(out):
report.print_coverage(report.coverage(data, ["angr", "r2sleigh_native"]))
text = out.getvalue()
self.assertIn("rows: 3; scored by all 2: 0; left out of the comparison: 3", text)
self.assertIn("rows with no score: angr 0, r2sleigh_native 3", text)
def test_a_stated_fraction_fails_a_project_that_is_mostly_unscored(self) -> None:
mostly_lost = [{"kuna": worse()}] + [{"kuna": None}] * 3
data = results_of({"bash": mostly_lost})
self.assertEqual(report.coverage(data, ["kuna"]).failures, [])
failures = report.coverage(data, ["kuna"], min_fraction=0.5).failures
self.assertEqual(failures, ["kuna scored 1 of 4 rows of bash"])
def test_a_decompiler_is_owed_only_the_cells_it_ran_in(self) -> None:
# A merge of shards: angr ran in `fresh` only, so `cached` is not a loss.
data = results_of({"fresh": [{"angr": perfect(), "r2sleigh_native": worse()}],
"cached": [{"r2sleigh_native": worse()}]},
sweep={"decompiler_cells": {"angr": ["fresh/O2"],
"r2sleigh_native": ["cached/O2", "fresh/O2"]}})
cover = report.coverage(data, ["angr", "r2sleigh_native"])
self.assertEqual(cover.failures, [])
self.assertEqual(sorted(cover.by_project), ["cached", "fresh"])
self.assertNotIn("angr", cover.by_project["cached"])
# Naming a decompiler that never scored is a failure, not a pass.
self.assertEqual(report.coverage(data, ["ghidra"]).failures,
["ghidra scored 0 of 1 rows of cached", "ghidra scored 0 of 1 rows of fresh"])
def test_the_compared_set_is_what_the_results_name(self) -> None:
data = results_of({"zlib": [{"angr": perfect(), "kuna": worse()}]})
self.assertEqual(report.compared_decompilers(data), ["angr", "kuna"])
del data["decompiler_versions"]
data["decompilers"] = ["angr"]
self.assertEqual(report.compared_decompilers(data), ["angr"])
class ReportTests(unittest.TestCase):
def test_an_invalid_baseline_is_neither_compared_nor_merged(self) -> None:
self.assertEqual(report.usable_baseline(None), (None, None))
tests/decbench/test_mined_set.py
+332
View File
@@ -0,0 +1,332 @@
#!/usr/bin/env python3
"""The mined-set comparison pairs two runs and reports them; the DecBench scoring is not run here."""
from __future__ import annotations
import contextlib
import io
import json
import os
import re
import sys
import tempfile
import types
import unittest
from pathlib import Path
from unittest import mock
HERE = Path(__file__).resolve().parent
STUB = HERE.parents[1] / "tests" / "equiv" / "testdata" / "stub_r2s.py"
sys.path.insert(0, str(HERE))
import mined_set # noqa: E402
def row(ged, gotos=0, text="a", refused=False, status="output"):
return {"ged": ged, "gotos": gotos, "text": text, "refused": refused, "status": status}
BASE = {
"O2|zlib|libz|keeps": row(0.0, text="1"),
"O2|zlib|libz|gains": row(5.0, gotos=2, text="2"),
"O2|zlib|libz|loses": row(0.0, text="3"),
"O2|zlib|libz|better": row(4.0, gotos=1, text="4"),
"O2|zlib|libz|same": row(2.0, text="5"),
"O2|zlib|libz|unscored": row(1.0, text="6"),
"O2|zlib|libz|gets_scored": row(None, text="7"),
"O2|zlib|libz|dropped": row(9.0, text="8"),
}
NEW = {
"O2|zlib|libz|keeps": row(0.0, text="1"),
"O2|zlib|libz|gains": row(0.0, gotos=0, text="2b"),
"O2|zlib|libz|loses": row(3.0, gotos=1, text="3b"),
"O2|zlib|libz|better": row(2.0, gotos=1, text="4"),
"O2|zlib|libz|same": row(2.0, text="5"),
"O2|zlib|libz|unscored": row(None, text="6b", refused=True, status="decline"),
"O2|zlib|libz|gets_scored": row(7.0, text="7"),
}
class Pairing(unittest.TestCase):
def test_two_runs_are_paired_over_the_functions_both_scored(self):
report = mined_set.pair(BASE, NEW)
self.assertEqual(report["scored"], 5)
self.assertEqual((report["lost_scoring"], report["gained_scoring"]), (2, 1))
self.assertEqual(report["perfect"], [2, 2])
self.assertEqual(report["newly_perfect"], ["O2|zlib|libz|gains"])
self.assertEqual(report["lost_perfect"], ["O2|zlib|libz|loses"])
self.assertEqual(report["mean_ged"], [2.2, 1.4])
self.assertEqual(report["total_ged"], [11.0, 7.0])
self.assertEqual(report["better"], ["O2|zlib|libz|better", "O2|zlib|libz|gains"])
self.assertEqual(report["worse"], ["O2|zlib|libz|loses"])
self.assertEqual(report["same"], 2)
self.assertEqual(report["gotos_all"], [3, 2])
self.assertEqual(report["gotos_scored"], [3, 2])
self.assertEqual(report["functions_with_goto"], [2, 2])
# Seven functions are in both runs; text differs in gains, loses and unscored.
self.assertEqual((report["text_changed"], report["text_compared"]), (3, 7))
self.assertEqual(report["refused"], [0, 1])
self.assertEqual(report["newly_refused"], ["O2|zlib|libz|unscored"])
def test_the_report_names_each_figure_the_item_is_graded_by(self):
text = mined_set.format_report("mining", mined_set.pair(BASE, NEW))
self.assertEqual(text.splitlines(), [
"mining: scored 5 (lost scoring 2, gained 1)",
" perfect 2 -> 2 newly 1 lost 1",
" mean GED 2.20 -> 1.40 better 2 worse 1 same 2",
" gotos all 3 -> 2; scored 3 -> 2; scored fns with a goto 2 -> 2",
" total GED 11 -> 7",
" no source: no DWARF owner 0 -> 0, owner without a CFG 0 -> 0",
" text changed 3 of 7 functions in both runs; refused 0 -> 1, newly refused 1",
])
def test_scores_without_a_text_hash_say_so_and_an_empty_pairing_does_not_divide(self):
legacy = {"O2|a|b|f": {"ged": 3.0, "gotos": 1, "refused": False}}
report = mined_set.pair(legacy, legacy)
self.assertIsNone(report["text_changed"])
self.assertIn("text changed: n/a", mined_set.format_report("x", report))
empty = mined_set.pair({}, {})
self.assertEqual(empty["mean_ged"], [None, None])
self.assertIn("mean GED n/a -> n/a", mined_set.format_report("x", empty))
def test_unique_keeps_one_entry_per_function_and_prefers_the_library(self):
keys = ["O2|zlib|example|crc32", "O2|zlib|libz.so.1.2|crc32", "O2|zlib|example|main",
"O2|zlib|minigzip|main", "O2-noinline|zlib|example|main"]
self.assertEqual(sorted(mined_set.one_per_function(keys)), [
"O2-noinline|zlib|example|main", "O2|zlib|example|main", "O2|zlib|libz.so.1.2|crc32"])
rows = {k: row(1.0) for k in keys}
self.assertEqual(mined_set.pair(rows, rows, unique=True)["scored"], 3)
self.assertEqual(mined_set.pair(rows, rows)["scored"], 5)
def test_a_mining_run_is_not_paired_with_a_held_out_run(self):
with tempfile.TemporaryDirectory() as directory:
def write(name, set_name):
path = Path(directory) / name
path.write_text(json.dumps({"schema": 1, "set": set_name, "rows": BASE}))
return str(path)
mining, held_out = write("a.json", "mining"), write("b.json", "heldout")
err = io.StringIO()
with contextlib.redirect_stderr(err), contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(mined_set.main(["pair", mining, held_out]), 2)
self.assertIn("refusing to pair a mining run with a heldout run", err.getvalue())
out = io.StringIO()
with contextlib.redirect_stdout(out):
self.assertEqual(mined_set.main(["pair", mining, mining]), 0)
self.assertTrue(out.getvalue().startswith("mining: scored 7"))
# The scratch format, a bare {key: row} file, still pairs.
bare = Path(directory) / "bare.json"
bare.write_text(json.dumps(BASE))
with contextlib.redirect_stdout(io.StringIO()):
self.assertEqual(mined_set.main(["pair", str(bare), str(bare)]), 0)
class Sets(unittest.TestCase):
def test_mining_and_held_out_split_the_sailr_set(self):
self.assertEqual(mined_set.MINING, ("bzip2", "zlib", "grep"))
self.assertFalse(set(mined_set.MINING) & set(mined_set.HELD_OUT))
self.assertEqual(sorted(mined_set.MINING + mined_set.HELD_OUT), sorted(
["bash", "bzip2", "diffutils", "findutils", "grep", "gzip", "libexpat", "zlib"]))
self.assertEqual(mined_set.OPT_LEVELS, ("O2", "O2-noinline"))
class Dump(unittest.TestCase):
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.tree = Path(self.dir.name) / "run"
project = self.tree / "O2" / "zlib"
(project / "decompiled").mkdir(parents=True)
(project / "stripped").mkdir()
# The file is libz.so.1.2.13, the driver's name for it is the stem libz.so.1.2.
(project / "stripped" / "libz.so.1.2.13").write_bytes(b"\x7fELF")
(project / "decompiled" / "kuna_libz.so.1.2.toml").write_text(
'binary = "libz.so.1.2"\n["functions.crc32"]\naddress = "0x401150"\n'
'["functions.helper"]\naddress = "0x401160"\n')
(project / "decompiled" / "angr_libz.so.1.2.toml").write_text(
'binary = "libz.so.1.2"\n["functions.helper"]\naddress = "0x401160"\n'
'["functions.helper_2"]\naddress = "0x4011f0"\n')
self.out = Path(self.dir.name) / "dump"
def tearDown(self):
self.dir.cleanup()
def dump(self, mode="minimal"):
env = {"STUB_R2S_MODE": mode, "STUB_R2S_FAULTS": "", "STUB_R2S_STATE": self.dir.name}
with mock.patch.dict(os.environ, env), contextlib.redirect_stdout(io.StringIO()):
return mined_set.dump(self.tree, STUB, self.out, ("zlib",), ("O2",), set_name="mining",
function_timeout=30.0)
def test_targets_are_the_union_of_the_decompilers_tomls(self):
found = mined_set.function_targets(self.tree / "O2" / "zlib")
self.assertEqual(found, {"libz.so.1.2": {0x401150: "crc32", 0x401160: "helper", 0x4011f0: "helper_2"}})
def test_a_name_two_addresses_share_is_told_apart_by_address(self):
keys = mined_set.row_keys("O2", "grep", "grep", {0x2000: "usage", 0x1000: "usage", 0x3000: "main"})
self.assertEqual(keys, {0x1000: "O2|grep|grep|usage", 0x2000: "O2|grep|grep|usage@0x2000",
0x3000: "O2|grep|grep|main"})
def test_every_function_is_dumped_under_its_dwarf_name(self):
self.assertEqual(self.dump(), 3)
rows = json.loads((self.out / "O2.zlib.libz.so.1.2.json").read_text())
self.assertEqual(sorted(rows), ["O2|zlib|libz.so.1.2|crc32", "O2|zlib|libz.so.1.2|helper",
"O2|zlib|libz.so.1.2|helper_2"])
crc = rows["O2|zlib|libz.so.1.2|crc32"]
self.assertEqual((crc["status"], crc["refused"], crc["addr"]), ("output", False, "0x401150"))
self.assertIn("void crc32(void)", crc["code"])
self.assertNotIn("fcn_", crc["code"])
manifest = json.loads((self.out / "manifest.json").read_text())
self.assertEqual((manifest["set"], manifest["projects"], manifest["functions"]), ("mining", ["zlib"], 3))
self.assertEqual(len(manifest["r2s_sha256"]), 64)
def test_a_refusal_is_dumped_as_a_refusal_with_no_code(self):
self.dump(mode="refuse")
rows = json.loads((self.out / "O2.zlib.libz.so.1.2.json").read_text())
refused = rows["O2|zlib|libz.so.1.2|crc32"]
self.assertEqual((refused["status"], refused["refused"]), ("decline", True))
self.assertNotIn("code", refused)
self.assertEqual(refused["cause"], "refused: stub refuses")
class Graph:
"""Stands in for a networkx CFG: a shape (`kind`) and a size."""
def __init__(self, kind: str, nodes: int):
self.kind, self.nodes = kind, nodes
def number_of_nodes(self):
return self.nodes
def number_of_edges(self):
return self.nodes
class Score(unittest.TestCase):
"""`score` walks a dump, finds each function's source CFG by its address, and writes one row per function.
DecBench is replaced by stand-ins that are isomorphic by `kind` and otherwise a distance of the node
count, so this checks the walk, the owner of each source, the rows and their pairing; the GED itself and
the DWARF walk behind `source_function_owners` are DecBench's.
"""
def stand_ins(self):
def module(name, **members):
mod = types.ModuleType(name)
mod.__dict__.update(members)
return mod
parsed = lambda path: { # noqa: E731
name: Graph("loop" if "while" in text else "line", text.count(";") + 1)
for name, text in re.findall(r"// Function: (\w+)\n(.*?)(?=// Function:|\Z)", Path(path).read_text(), re.S)}
def source_function_owners(path, stems, follow_abstract_origin=False):
self.owner_calls.append((Path(path), set(stems), follow_abstract_origin))
return self.owners
# `crc32.i` sorts first and defines a crc32 of its own; the binary's crc32 is the one in `other.i`.
units = {"crc32": {"crc32": Graph("loop", 2), "tiny": Graph("degenerate", 1)},
"other": {"crc32": Graph("line", 4), "helper": Graph("line", 2)}}
cfg = module(
"decbench.utils.cfg", _parse_cfgs=parsed, sanitize_decompiled_c=lambda t: t,
preprocess_decompiled_c=lambda t: t, canonicalize_infinite_loops=lambda t: t,
is_degenerate_source_cfg=lambda g: g.kind == "degenerate",
extract_cfgs_from_source=lambda path: units[Path(path).stem])
return {
"decbench": module("decbench"), "decbench.utils": module("decbench.utils"),
"decbench.utils.cfg": cfg, "decbench.metrics": module("decbench.metrics"),
"decbench.utils.binfmt": module("decbench.utils.binfmt", source_function_owners=source_function_owners),
"decbench.utils.dwarf_policy": module("decbench.utils.dwarf_policy",
dwarf_follow_abstract_origin=lambda: True),
"decbench.metrics.ged": module("decbench.metrics.ged", GED_MAX_NODES=200,
_is_isomorphic=lambda a, b: a.kind == b.kind and a.nodes == b.nodes),
"decbench.metrics.vj_ged": module("decbench.metrics.vj_ged",
vj_ged=lambda a, b: abs(a.nodes - b.nodes) + (a.kind != b.kind)),
}
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.root = Path(self.dir.name)
self.owner_calls = []
# The DWARF owners of the compiled binary: address -> (name, translation unit stem). 0x4 has none.
self.owners = {0x1: ("crc32", "other"), 0x2: ("helper", "other"), 0x3: ("tiny", "crc32"),
0x5: ("ghost", "other")}
patch = mock.patch.dict(sys.modules, self.stand_ins())
patch.start()
self.addCleanup(patch.stop)
self.addCleanup(self.dir.cleanup)
project = self.root / "run" / "O2" / "zlib"
(project / "compiled").mkdir(parents=True)
(project / "stripped").mkdir()
# The binary is libz.so.1.2.13 (stem libz.so.1.2) in both directories; `crc32.i` shares no stem with it.
for name in ("crc32.i", "other.i", "libz.so.1.2.13"):
(project / "compiled" / name).write_text("")
(project / "stripped" / "libz.so.1.2.13").write_text("")
def dump(self, name, helper_code, refused=False, binary="libz.so.1.2"):
folder = self.root / name
folder.mkdir()
rows = {
f"O2|zlib|{binary}|crc32": {"name": "crc32", "addr": "0x1", "status": "output",
"refused": False, "code": "while (1) { goto out; }\n"},
f"O2|zlib|{binary}|helper": {"name": "helper", "addr": "0x2", "status": "output",
"refused": False, "code": helper_code},
f"O2|zlib|{binary}|tiny": {"name": "tiny", "addr": "0x3", "status": "output",
"refused": False, "code": "x;\n"},
f"O2|zlib|{binary}|nosource": {"name": "nosource", "addr": "0x4", "status": "output",
"refused": False, "code": "y;\n"},
f"O2|zlib|{binary}|ghost": {"name": "ghost", "addr": "0x5", "status": "output",
"refused": False, "code": "z;\n"},
}
if refused:
rows[f"O2|zlib|{binary}|helper"] = {"name": "helper", "addr": "0x2", "status": "decline",
"refused": True, "cause": "refused: no"}
(folder / f"O2.zlib.{binary}.json").write_text(json.dumps(rows))
(folder / "manifest.json").write_text(json.dumps({"set": "mining", "r2s_sha256": name}))
out = io.StringIO()
with contextlib.redirect_stdout(out), contextlib.redirect_stderr(out):
mined_set.score(self.root / "run", folder, self.root / f"{name}.json", None, self.root / "cache")
self.output = out.getvalue()
return mined_set.load_scores(self.root / f"{name}.json")
def test_each_dumped_function_gets_a_row_and_two_runs_pair(self):
base, meta = self.dump("base", "a; b;\n")
self.assertEqual(meta["set"], "mining")
crc = base["O2|zlib|libz.so.1.2|crc32"]
# The binary's crc32 is the one in other.i (a 4-block line), not the loop in crc32.i that sorts first.
self.assertEqual((crc["ged"], crc["gotos"], crc["loops"], crc["whiles"]), (3.0, 1, 0, 1))
self.assertEqual(base["O2|zlib|libz.so.1.2|helper"]["ged"], 1.0) # 3 nodes against 2: distance 1
self.assertIsNone(base["O2|zlib|libz.so.1.2|tiny"]["ged"]) # a degenerate source scores nothing
new, _ = self.dump("new", "a;\n")
self.assertEqual(new["O2|zlib|libz.so.1.2|helper"]["ged"], 0.0)
report = mined_set.pair(base, new)
self.assertEqual((report["scored"], report["perfect"], report["text_changed"]), (2, [0, 1], 1))
gone, _ = self.dump("gone", "", refused=True)
report = mined_set.pair(base, gone)
self.assertEqual((report["lost_scoring"], report["newly_refused"]),
(1, ["O2|zlib|libz.so.1.2|helper"]))
self.assertTrue((self.root / "cache" / "O2.zlib.tu.pkl").exists())
def test_the_owner_is_asked_of_the_compiled_binary_and_followed_through_the_abstract_origin_policy(self):
self.dump("base", "a;\n")
self.assertEqual(self.owner_calls, [(self.root / "run" / "O2" / "zlib" / "compiled" / "libz.so.1.2.13",
{"crc32", "other"}, True)])
def test_a_function_without_an_owner_is_not_scored_and_is_counted(self):
rows, _ = self.dump("base", "a;\n")
self.assertEqual(rows["O2|zlib|libz.so.1.2|nosource"]["ged"], "no_owner")
self.assertEqual(rows["O2|zlib|libz.so.1.2|ghost"]["ged"], "no_source_cfg") # owned by a unit with no ghost
self.assertIn("1 without a DWARF owner, 1 with an owner and no source CFG", self.output)
report = mined_set.pair(rows, rows)
self.assertEqual((report["unowned"], report["no_source_cfg"]), ([1, 1], [1, 1]))
self.assertEqual(report["scored"], 2)
self.assertIn("no source: no DWARF owner 1 -> 1, owner without a CFG 1 -> 1", mined_set.format_report("x", report))
def test_a_binary_not_found_in_compiled_has_no_owners_and_says_so(self):
(self.root / "run" / "O2" / "zlib" / "stripped" / "libz.so.1.2.13").unlink()
rows, _ = self.dump("base", "a;\n")
self.assertEqual(self.owner_calls, [])
self.assertEqual({r["ged"] for r in rows.values()}, {"no_owner"})
self.assertIn("no compiled binary for O2/zlib/libz.so.1.2", self.output)
if __name__ == "__main__":
unittest.main()
tests/decbench/test_r2s_timing.py
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env python3
"""`r2s_timing.py` times r2s per binary of a run; the child process that does it is replaced by a stub."""
from __future__ import annotations
import contextlib
import io
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import r2s_timing # noqa: E402
TIMED = {"asked": 3, "rendered": 2, "declined": 1, "seconds": 1.5, "processes": 1, "maxrss_kb": 4096}
class Timing(unittest.TestCase):
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.run = Path(self.dir.name)
decompiled = self.run / "O2" / "zlib" / "decompiled"
decompiled.mkdir(parents=True)
(self.run / "O2" / "zlib" / "stripped").mkdir()
# The file is libz.so.1.2.13, the driver's name for it (and its tomls') is the stem libz.so.1.2.
(self.run / "O2" / "zlib" / "stripped" / "libz.so.1.2.13").write_bytes(b"\x7fELF")
(self.run / "O2" / "zlib" / "stripped" / "example").write_bytes(b"\x7fELF")
for binary in ("libz.so.1.2", "example", "ghost"):
(decompiled / f"r2sleigh_native_{binary}.toml").write_text('["functions.f"]\naddress = "0x1000"\n')
def tearDown(self):
self.dir.cleanup()
def time(self):
calls = []
def child(command, **kwargs):
calls.append(command)
return subprocess.CompletedProcess(command, 0, stdout=json.dumps(TIMED) + "\n", stderr="")
out, err = io.StringIO(), io.StringIO()
with mock.patch.object(r2s_timing.subprocess, "run", child), \
contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
self.assertEqual(r2s_timing.main(["--r2s", "r2s", "--run", str(self.run), "--opt", "O2"]), 0)
rows = [line.split("\t") for line in out.getvalue().splitlines()[1:]]
return calls, rows, err.getvalue()
def test_a_binary_whose_file_has_a_version_suffix_is_timed(self):
calls, rows, _ = self.time()
asked = {Path(command[command.index("--one") + 1]).name for command in calls}
self.assertEqual(asked, {"libz.so.1.2.13", "example"})
self.assertIn(["zlib", "O2", "libz.so.1.2", "3", "2", "1", "1.5", "4096"], rows)
def test_a_binary_with_no_stripped_file_is_a_counted_row_not_a_silent_omission(self):
_, rows, summary = self.time()
self.assertIn(["zlib", "O2", "ghost", "NO-STRIPPED-BINARY", "", "", "", ""], rows)
self.assertEqual(len(rows), 3)
self.assertIn("timed 2 binaries, 1 without a stripped binary, 0 failed", summary)
if __name__ == "__main__":
unittest.main()
tests/decbench/test_r2sleigh_raw.py
+145 -16
View File
@@ -14,15 +14,18 @@ r2s is replaced by ``tests/equiv/testdata/stub_r2s.py``, which answers
from __future__ import annotations
import itertools
import os
import pickle
import shutil
import subprocess
import sys
import tempfile
import time
import types
import unittest
from pathlib import Path
from unittest import mock
HERE = Path(__file__).resolve().parent
REPO = HERE.parents[1]
@@ -107,6 +110,8 @@ TARGETS = [0x401150, 0x401160, 0x4011F0, 0x401210, 0x401230, 0x401290]
class Backend(unittest.TestCase):
runs = itertools.count()
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.env = {
@@ -154,24 +159,12 @@ class Backend(unittest.TestCase):
self.assertEqual(result.decompiler.extra["processes"], 1)
self.assertEqual(result.decompiler.failed_functions, [f"0x{TARGETS[1]:x}"])
def test_a_crash_costs_one_function_and_every_answer_is_checkpointed(self):
def test_a_crash_costs_one_function_and_the_end_is_checkpointed(self):
# D4: abort on the third of six; r2s restarts at the fourth, and the
# progress pickle grows by one function per answer.
# pickle the driver would recover holds the five answers.
os.environ["STUB_R2S_FAULTS"] = f"abort@{TARGETS[2]:x}"
sizes: list[int] = []
original = r2sleigh_raw.common.dump_progress
def spy(path, result):
sizes.append(len(result.functions) + len(result.decompiler.extra["decline_causes"]))
original(path, result)
r2sleigh_raw.common.dump_progress = spy
try:
progress = Path(self.tmp.name) / "progress.pkl"
result = self.decompile(function_names=set(TARGETS), progress_path=progress)
finally:
r2sleigh_raw.common.dump_progress = original
self.assertEqual(sizes, [1, 2, 3, 4, 5, 6])
progress = Path(self.tmp.name) / "progress.pkl"
result = self.decompile(function_names=set(TARGETS), progress_path=progress)
causes = result.decompiler.extra["decline_causes"]
self.assertEqual(list(causes), [f"0x{TARGETS[2]:x}"])
self.assertIn("SIGABRT", causes[f"0x{TARGETS[2]:x}"])
@@ -181,6 +174,142 @@ class Backend(unittest.TestCase):
saved = pickle.loads(progress.read_bytes())
self.assertEqual(len(saved.functions), 5)
self.assertEqual(result.decompiler.extra["processes"], 2)
self.assertNotIn("resume", result.decompiler.extra)
def checkpoint_rows(self, answer_times, interval="30"):
"""The rows held by each checkpoint of a run whose answers arrive at ``answer_times`` seconds."""
clock = [0.0]
rows: list[int] = []
os.environ["R2SLEIGH_CHECKPOINT_SECONDS"] = interval
real_batch = r2sleigh_raw.r2s_batch.run_batch
real_dump = r2sleigh_raw.common.dump_progress
def timed(*args, on_answer=None, **kwargs):
moments = iter(answer_times)
def arrive(answer):
clock[0] = next(moments)
on_answer(answer)
return real_batch(*args, on_answer=arrive, **kwargs)
def spy(path, result):
rows.append(len(result.functions) + len(result.decompiler.extra["decline_causes"]))
real_dump(path, result)
try:
r2sleigh_raw.r2s_batch.run_batch = timed
r2sleigh_raw.common.dump_progress = spy
r2sleigh_raw._clock = lambda: clock[0]
# A path of its own: a checkpoint left by an earlier call would resume.
self.decompile(function_names=set(TARGETS),
progress_path=Path(self.tmp.name) / f"cadence-{next(self.runs)}.pkl")
finally:
r2sleigh_raw.r2s_batch.run_batch = real_batch
r2sleigh_raw.common.dump_progress = real_dump
r2sleigh_raw._clock = time.monotonic
return rows
def test_a_checkpoint_is_written_by_elapsed_time_and_at_the_end(self):
# Six answers in one second are one pickle, not six; a pause of 30 seconds is one more.
self.assertEqual(self.checkpoint_rows([1, 2, 3, 4, 5, 6]), [6])
self.assertEqual(self.checkpoint_rows([1, 2, 31, 32, 33, 70]), [3, 6, 6])
self.assertEqual(self.checkpoint_rows([1, 2, 3, 4, 5, 6], interval="0"),
[1, 2, 3, 4, 5, 6, 6])
def test_the_interval_is_thirty_seconds_unless_the_environment_says_otherwise(self):
os.environ.pop("R2SLEIGH_CHECKPOINT_SECONDS", None)
self.assertEqual(r2sleigh_raw.checkpoint_interval(), 30.0)
for given, expected in (("5", 5.0), ("0", 0.0), ("-3", 0.0), ("soon", 30.0),
("nan", 30.0), ("inf", 30.0)):
os.environ["R2SLEIGH_CHECKPOINT_SECONDS"] = given
self.assertEqual(r2sleigh_raw.checkpoint_interval(), expected, given)
def asked_addresses(self):
"""Patch the runner to record the addresses it is asked about; returns the list and an undo."""
asked: list[int] = []
real_batch = r2sleigh_raw.r2s_batch.run_batch
def spy(executable, binary, addresses, **kwargs):
asked.extend(addresses)
return real_batch(executable, binary, addresses, **kwargs)
r2sleigh_raw.r2s_batch.run_batch = spy
self.addCleanup(setattr, r2sleigh_raw.r2s_batch, "run_batch", real_batch)
return asked
@unittest.skipUnless(hasattr(os, "fork"), "needs fork to kill a run without cleanup")
def test_a_killed_run_resumes_from_its_last_checkpoint_and_gives_the_same_rows(self):
progress = Path(self.tmp.name) / "progress.pkl"
os.environ["R2SLEIGH_CHECKPOINT_SECONDS"] = "30"
clock = [0.0]
real_batch = r2sleigh_raw.r2s_batch.run_batch
def dying(*args, on_answer=None, **kwargs):
given = []
def arrive(answer):
clock[0] += 10.0 # a checkpoint on every third answer
on_answer(answer)
given.append(answer)
if len(given) == 5:
os._exit(9) # SIGKILL's view of the run: no handler, no `finally`
return real_batch(*args, on_answer=arrive, **kwargs)
pid = os.fork()
if pid == 0:
code = 2
try:
r2sleigh_raw._clock = lambda: clock[0]
r2sleigh_raw.r2s_batch.run_batch = dying
self.decompile(function_names=set(TARGETS), progress_path=progress)
except BaseException: # noqa: BLE001
code = 3
finally:
os._exit(code) # never fall back into the test runner
_, status = os.waitpid(pid, 0)
self.assertEqual(os.WEXITSTATUS(status), 9)
# Five answers were given; the third's checkpoint is the last, so two are lost, one interval at most.
kept = pickle.loads(progress.read_bytes())
self.assertEqual(sorted(f.address for f in kept.functions.values()), TARGETS[:3])
asked = self.asked_addresses()
resumed = self.decompile(function_names=set(TARGETS), progress_path=progress)
self.assertEqual(asked, TARGETS[3:])
self.assertEqual(resumed.decompiler.extra["resumed"], 3)
fresh = self.decompile(function_names=set(TARGETS))
def rows(result):
return ({name: (f.address, f.decompiled_code, f.metadata)
for name, f in result.functions.items()},
result.decompiler.extra["decline_causes"],
result.decompiler.failed_functions,
(result.decompiler.extra["requested"], result.decompiler.extra["rendered"],
result.decompiler.extra["declined"]))
self.assertEqual(rows(resumed), rows(fresh))
def test_a_checkpoint_of_another_task_is_not_read(self):
progress = Path(self.tmp.name) / "progress.pkl"
self.decompile(function_names=set(TARGETS[:3]), progress_path=progress)
asked = self.asked_addresses()
# The checkpoint holds three targets, this run has six: nothing carries over.
result = self.decompile(function_names=set(TARGETS), progress_path=progress)
self.assertEqual(asked, TARGETS)
self.assertNotIn("resumed", result.decompiler.extra)
# The same six, another r2s build: nothing carries over either.
other = Path(self.tmp.name) / "other-build.pkl"
shutil.copyfile(progress, other)
asked.clear()
with mock.patch.object(r2sleigh_raw.R2sDecompiler, "get_version",
return_value="r2s sha256:another"):
self.decompile(function_names=set(TARGETS), progress_path=other)
self.assertEqual(asked, TARGETS)
# The same task: the end-of-run checkpoint answers everything.
asked.clear()
again = self.decompile(function_names=set(TARGETS), progress_path=progress)
self.assertEqual((asked, again.decompiler.extra["resumed"]), ([], 6))
def test_thousands_of_targets_are_one_process_and_every_one_is_filed(self):
# The sailr set's large binaries have thousands of functions. As one -c
tests/decbench/test_run_analysis.py
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""The pure parts of the DecBench run-analysis tools: type kinds, graph features, the scoreboard arithmetic."""
import json
import tempfile
import unittest
from pathlib import Path
import compare_runs
import ged_causes
import type_causes
class Node:
def __init__(self, entry=False, exit_=False):
self.is_entrypoint, self.is_exitpoint = entry, exit_
class Graph:
"""The subset of networkx.DiGraph the feature counter reads."""
def __init__(self, nodes, edges):
self.nodes = list(nodes)
self.out = {n: [] for n in nodes}
self.inn = {n: [] for n in nodes}
for a, b in edges:
self.out[a].append(b)
self.inn[b].append(a)
def successors(self, n):
return iter(self.out[n])
def out_degree(self, n):
return len(self.out[n])
def in_degree(self, n):
return len(self.inn[n])
def number_of_edges(self):
return sum(len(v) for v in self.out.values())
class TypeKinds(unittest.TestCase):
def test_decompiler_spellings(self):
kind = type_causes.dec_kind
self.assertEqual(kind("uint64_t", None), "int8")
self.assertEqual(kind("const char *", None), "ptr_char")
self.assertEqual(kind("uint8_t*", None), "ptr_char")
self.assertEqual(kind("void*", None), "ptr_void")
self.assertEqual(kind("uint64_t**", None), "ptr_ptr")
self.assertEqual(kind("struct fcn_4330_arg0*", None), "ptr_struct")
self.assertEqual(kind("unsigned long long", None), "int8")
self.assertEqual(kind("_Bool", None), "bool")
def test_a_named_type_cannot_be_spelled_from_a_stripped_binary(self):
self.assertFalse(type_causes.expressible(["z_stream_s*", "z_streamp"]))
self.assertTrue(type_causes.expressible(["z_stream_s*", "char*"]))
self.assertTrue(type_causes.expressible(["uInt", "unsigned int", "int"]))
self.assertTrue(type_causes.expressible(["char", "char[10]"]))
def test_mismatch_causes(self):
fp = type_causes.fp_class
self.assertEqual(fp("ptr_char", "int8"), "fp:pointer_as_integer")
self.assertEqual(fp("ptr_ptr", "ptr_int"), "fp:wrong_pointee")
self.assertEqual(fp("int4", "int8"), "fp:integer_width")
self.assertEqual(fp("bool", "int1"), "fp:bool")
def test_greedy_order_fixes_the_class_that_frees_most_functions(self):
def row(*classes):
return {"value": 0.5, "vars": [{"verdict": c} for c in classes]}
rows = [row("fp:a"), row("fp:a"), row("fp:a", "fp:b"), row("fp:b", "fn:unobservable"), row("fp:c", "fp:b")]
steps = type_causes.greedy(rows)
self.assertEqual(steps, [("fp:a", 2), ("fp:b", 1), ("fp:c", 1)])
class GraphShape(unittest.TestCase):
def test_features_count_exits_branches_and_back_edges(self):
a, b, c, d = Node(entry=True), Node(), Node(), Node(exit_=True)
g = Graph([a, b, c, d], [(a, b), (b, c), (c, b), (b, d)])
f = ged_causes.graph_features(g)
self.assertEqual((f["V"], f["E"], f["X"], f["B"], f["M"], f["L"]), (4, 4, 1, 1, 0, 1))
def test_function_body_reads_ansi_and_old_style_definitions(self):
unit = "static int foo(int a);\nstatic int bar(s, f)\n char *s;\n int f;\n{ if (s && f) return 1; }\nint x = foo(3);\n"
self.assertEqual(ged_causes.function_body(unit, "bar"), "{ if (s && f) return 1; }")
self.assertIsNone(ged_causes.function_body(unit, "foo"))
def test_text_features_ignore_strings_and_comments(self):
t = ged_causes.text_features('{ if (a && b) { puts("x && y"); } /* || */ for (i = 0; i < n; i++) {} }')
self.assertEqual((t["and_or"], t["for"], t["for_full"]), (1, 1, 1))
def test_a_merged_return_is_an_exit_difference(self):
s = {"V": 5, "E": 5, "X": 2, "B": 1, "M": 0, "L": 0}
d = {"V": 5, "E": 5, "X": 1, "B": 1, "M": 0, "L": 0}
zero = {"and_or": 0, "switch": 0, "case": 0, "for": 0, "for_full": 0, "while": 0, "do": 0,
"goto": 0, "ret": 2, "ternary": 0, "residual": 0}
row = {"S": s, "D": d, "S_text": zero, "D_text": zero}
self.assertEqual(ged_causes.classify(row), "exit_blocks")
class Scoreboard(unittest.TestCase):
def test_a_row_only_one_decompiler_scored_is_a_miss_for_the_others(self):
def fn(name, values):
return {"function": name, "values": values,
"perfects": {d: {m: v == (0.0 if m == "ged" else 1.0) for m, v in vs.items()} for d, vs in values.items()},
"decompiled": {}}
data = {"decompilers": ["a", "b"], "groups": [
{"project": "p", "opt_level": "O2", "binary": "x", "functions": [
fn("f", {"a": {"ged": 0.0}, "b": {"ged": 3.0}})]},
{"project": "q", "opt_level": "O2", "binary": "y", "functions": [
fn("g", {"a": {"ged": 0.0}})]}]}
with tempfile.TemporaryDirectory() as tmp:
Path(tmp, "function_results.json").write_text(json.dumps(data))
everything = compare_runs.recompute(Path(tmp), lambda g: True)
only_p = compare_runs.recompute(Path(tmp), lambda g: g["project"] == "p")
self.assertEqual(everything["decompiler_scores"]["b"]["metric_scores"]["ged"]["total_count"], 2)
self.assertEqual(everything["decompiler_scores"]["b"]["overall_perfect_count"], 0)
self.assertEqual(only_p["decompiler_scores"]["b"]["metric_scores"]["ged"]["total_count"], 1)
self.assertEqual(only_p["decompiler_scores"]["a"]["overall_rank"], 1)
if __name__ == "__main__":
unittest.main()
tests/decbench/test_run_gate.py
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""A run that loses rows stops, and an evaluation the host would kill is refused."""
from __future__ import annotations
import contextlib
import io
import os
import sys
import tempfile
import types
import unittest
from pathlib import Path
from unittest import mock
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import run_gate # noqa: E402
SCORED = '''binary = "{binary}"
"{dec}.byte_match.total" = 0.8461538461538461
"{dec}.byte_match.functions.main" = 0.0
"{dec}.ged.total" = 4.0
'''
class Tree:
"""A results tree: `<opt>/<project>/{decompiled,evaluated}/`."""
def __init__(self, root: str):
self.root = Path(root)
def decompiled(self, decompiler: str, binary: str, functions: int = 3, c_bytes: int = 0,
opt: str = "O2", project: str = "bash") -> Path:
directory = self.root / opt / project / "decompiled"
directory.mkdir(parents=True, exist_ok=True)
toml = directory / f"{decompiler}_{binary}.toml"
toml.write_text(f'binary = "{binary}"\ndecompiler = "{decompiler}"\nfunction_count = {functions}\n')
if c_bytes:
(directory / f"{decompiler}_{binary}.c").write_bytes(b"a" * c_bytes)
return toml
def evaluated(self, binary: str, text: str, opt: str = "O2", project: str = "bash") -> Path:
directory = self.root / opt / project / "evaluated"
directory.mkdir(parents=True, exist_ok=True)
path = directory / f"{binary}.toml"
path.write_text(text)
return path
def gate(*argv: str) -> tuple[int, str]:
err = io.StringIO()
with contextlib.redirect_stderr(err), contextlib.redirect_stdout(io.StringIO()):
status = run_gate.main(list(argv))
return status, err.getvalue()
class EvaluatedFiles(unittest.TestCase):
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.tree = Tree(self.dir.name)
self.args = ("evaluated", "--tree", self.dir.name, "--decompilers", "r2sleigh_native,angr,kuna")
def tearDown(self):
self.dir.cleanup()
def test_a_16_byte_evaluated_file_stops_the_run(self):
# The 2026-10-17 bash: the evaluation worker was killed and the file kept its header only.
self.tree.decompiled("r2sleigh_native", "bash", functions=1540)
path = self.tree.evaluated("bash", 'binary = "bash"\n')
self.assertEqual(path.stat().st_size, 16)
status, err = gate(*self.args)
self.assertEqual(status, run_gate.LOST)
self.assertIn("r2sleigh_native rendered 1540 functions of bash", err)
self.assertIn("O2/bash/evaluated/bash.toml is 16 bytes and scores no decompiler", err)
def test_the_command_exits_nonzero_for_the_shell(self):
import subprocess
self.tree.decompiled("kuna", "bash")
self.tree.evaluated("bash", 'binary = "bash"\n')
done = subprocess.run([sys.executable, str(HERE / "run_gate.py"), *self.args],
capture_output=True, text=True)
self.assertEqual(done.returncode, 71)
self.assertIn("1 of 1 rendered binaries have no evaluation", done.stderr)
def test_a_missing_file_stops_the_run(self):
self.tree.decompiled("angr", "bash")
status, err = gate(*self.args)
self.assertEqual(status, run_gate.LOST)
self.assertIn("O2/bash/evaluated/bash.toml is missing", err)
def test_a_large_file_that_scores_only_other_decompilers_stops_the_run(self):
# O2-noinline bash of 2026-10-17: 108 KB, angr's 692 rows, nothing of kuna, ghidra or r2s.
self.tree.decompiled("angr", "bash")
self.tree.decompiled("kuna", "bash")
self.tree.evaluated("bash", SCORED.format(binary="bash", dec="angr") * 2000)
status, err = gate(*self.args)
self.assertEqual(status, run_gate.LOST)
self.assertIn("kuna rendered 3 functions of bash", err)
self.assertIn("scores angr, not kuna", err)
self.assertNotIn("angr rendered", err)
def test_an_oversized_c_file_names_the_likely_cause(self):
self.tree.decompiled("r2sleigh_native", "bash", c_bytes=9 * 2**20)
self.tree.evaluated("bash", 'binary = "bash"\n')
status, err = gate(*self.args)
self.assertEqual(status, run_gate.LOST)
self.assertIn("9.0 MB", err)
self.assertIn("--workers 1", err)
def test_scored_binaries_pass_including_dotted_names_and_small_files(self):
self.tree.decompiled("r2sleigh_native", "libz.so.1.2", project="zlib")
self.tree.evaluated("libz.so.1.2", SCORED.format(binary="libz.so.1.2", dec="r2sleigh_native"),
project="zlib")
self.tree.decompiled("kuna", "psize")
self.tree.evaluated("psize", SCORED.format(binary="psize", dec="kuna"))
status, err = gate(*self.args)
self.assertEqual((status, err), (0, ""))
def test_only_what_this_run_decompiled_is_checked(self):
# A resumed run: angr is older than the run's start and not evaluated again.
old = self.tree.decompiled("angr", "bash")
os.utime(old, (1_000, 1_000))
self.tree.decompiled("r2sleigh_native", "bash")
self.tree.evaluated("bash", SCORED.format(binary="bash", dec="r2sleigh_native"))
self.assertEqual(gate(*self.args, "--since", "2000")[0], 0)
self.assertEqual(gate(*self.args)[0], run_gate.LOST)
def test_a_decompiler_that_rendered_nothing_owes_no_scores(self):
self.tree.decompiled("angr", "bash", functions=0)
self.assertEqual(gate(*self.args)[0], 0)
class OneWorkerForBigC(unittest.TestCase):
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.tree = Tree(self.dir.name)
def tearDown(self):
self.dir.cleanup()
def workers(self, n: int) -> tuple[int, str]:
return gate("workers", "--tree", self.dir.name, "--workers", str(n),
"--decompilers", "r2sleigh_native,angr")
def test_more_than_one_worker_is_refused_for_a_binary_over_8_mb(self):
self.tree.decompiled("r2sleigh_native", "bash", c_bytes=13 * 2**20)
self.tree.decompiled("angr", "bash", c_bytes=2**20)
status, err = self.workers(4)
self.assertEqual(status, run_gate.REFUSED)
self.assertIn("O2/bash/bash (r2sleigh_native): 13.0 MB", err)
self.assertNotIn("angr", err)
self.assertEqual(self.workers(1), (0, ""))
def test_a_binary_under_the_limit_may_use_every_worker(self):
self.tree.decompiled("r2sleigh_native", "bzip2", c_bytes=7 * 2**20, project="bzip2")
self.assertEqual(self.workers(8), (0, ""))
class FakeResult:
def __init__(self, megabytes: float):
code = "a" * int(megabytes * 2**20)
self.functions = {"main": types.SimpleNamespace(decompiled_code=code)}
class DriverGuard(unittest.TestCase):
"""`install_workers_guard` in front of a stand-in `evaluate_project` with DecBench's signature."""
def setUp(self):
self.calls = []
def evaluate_project(project, decompilations, output_dir, optimization=None, metrics=None,
parallel=True, workers=None, precomputed_source_cfgs=None,
source_function_owners=None):
self.calls.append(workers)
return {"bash": {"r2sleigh_native": {"ged": 1}}}
self.module = types.ModuleType("decbench.pipeline.evaluate")
self.module.evaluate_project = evaluate_project
pipeline = types.ModuleType("decbench.pipeline")
pipeline.evaluate = self.module
patcher = mock.patch.dict(sys.modules, {
"decbench": types.ModuleType("decbench"),
"decbench.pipeline": pipeline,
"decbench.pipeline.evaluate": self.module,
})
patcher.start()
self.addCleanup(patcher.stop)
run_gate.install_workers_guard()
run_gate.install_workers_guard() # idempotent
def test_parallel_workers_over_big_c_end_the_run_instead_of_returning_nothing(self):
big = {"bash": {"r2sleigh_native": FakeResult(9), "angr": FakeResult(1)}}
# The driver catches `Exception` around `evaluate_project` and carries on with `{}`.
with self.assertRaises(SystemExit) as stop:
self.module.evaluate_project("project", big, Path("out"), None, None, True, 2)
self.assertNotIsInstance(stop.exception, Exception)
self.assertIn("bash (r2sleigh_native): 9.0 MB", str(stop.exception))
self.assertIn("--workers 1", str(stop.exception))
with self.assertRaises(SystemExit):
self.module.evaluate_project("project", big, Path("out"), workers=2)
self.assertEqual(self.calls, [])
def test_one_worker_or_small_c_passes_through(self):
big = {"bash": {"r2sleigh_native": FakeResult(9)}}
small = {"bzip2": {"r2sleigh_native": FakeResult(1)}}
self.assertEqual(self.module.evaluate_project("p", big, Path("o"), workers=1),
{"bash": {"r2sleigh_native": {"ged": 1}}})
self.module.evaluate_project("p", small, Path("o"), workers=8)
self.assertEqual(self.calls, [1, 8])
def test_no_worker_count_means_every_cpu(self):
big = {"bash": {"r2sleigh_native": FakeResult(9)}}
with mock.patch.object(run_gate.os, "cpu_count", return_value=4):
with self.assertRaises(SystemExit):
self.module.evaluate_project("p", big, Path("o"))
with mock.patch.object(run_gate.os, "cpu_count", return_value=1):
self.module.evaluate_project("p", big, Path("o"))
if __name__ == "__main__":
unittest.main()
tests/decbench/test_show_function.py
+183
View File
@@ -0,0 +1,183 @@
#!/usr/bin/env python3
"""`show_function.py` prints one function of a run; DecBench is replaced by stand-ins, the checkpoint by a dict."""
from __future__ import annotations
import contextlib
import io
import pickle
import sys
import tempfile
import types
import unittest
from pathlib import Path
from unittest import mock
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE))
import show_function # noqa: E402
class Block:
def __init__(self, addr):
self.addr, self.statements = addr, ["s"]
class Graph:
"""The subset of a networkx CFG that `show_function` prints."""
def __init__(self, blocks, edges):
self.nodes, self.edges = blocks, edges
def number_of_nodes(self):
return len(self.nodes)
def number_of_edges(self):
return len(self.edges)
def successors(self, node):
return [b for a, b in self.edges if a is node]
def chain(length):
blocks = [Block(i) for i in range(length)]
return Graph(blocks, list(zip(blocks, blocks[1:])))
class Function:
address, metadata, decompiled_code = 0x1000, {}, "int crc32(void) { return 0; }\n"
class Result:
binary_path = Path("/unused/libz.so.1.2.13")
def __init__(self):
self.functions = {"crc32": Function()}
def model_copy(self, update):
return self
def module(name, **members):
mod = types.ModuleType(name)
mod.__dict__.update(members)
return mod
class Shown(unittest.TestCase):
"""A run with one function, `crc32`, in a checkpoint, and the stand-ins DecBench would have supplied."""
def setUp(self):
self.dir = tempfile.TemporaryDirectory()
self.run = Path(self.dir.name)
(self.run / "checkpoints").mkdir()
(self.run / "checkpoints" / "zlib.pkl").write_bytes(b"")
self.loaded = []
self.owner_calls, self.variable_calls = [], []
# Two units define a crc32, `alpha` sorts first; the DWARF says the function at 0x1000 is zeta's.
self.units = {}
for stem, value in (("alpha", 1), ("zeta", 2)):
(self.run / f"{stem}.i").write_text(f"int crc32(void) {{ return {value}; }}\n")
self.units[stem] = self.run / f"{stem}.i"
self.source_graphs = {"alpha": {"crc32": chain(7)}, "zeta": {"crc32": chain(3)}}
self.owners = {0x1000: ("crc32", "zeta")}
class Project:
preprocessed_sources = {"O2": self.units}
@staticmethod
def from_toml(toml):
self.loaded.append(toml)
return Project
class Executor:
def __init__(self, config):
pass
def _discover_existing_binaries(self, projects, run):
pass
graphs = {"crc32": chain(2)}
def source_function_owners(path, stems, follow_abstract_origin=False):
self.owner_calls.append((path, set(stems), follow_abstract_origin))
return self.owners
def ground_truth_for_function(index, name, address, architecture):
self.variable_calls.append((name, address, architecture))
return [{"name": "strm", "is_arg": True, "type": ["z_stream*"], "rbp_offset": 0}]
stand_ins = {
"decbench": module("decbench"), "decbench.models": module("decbench.models"),
"decbench.pipeline": module("decbench.pipeline"), "decbench.metrics": module("decbench.metrics"),
"decbench.utils": module("decbench.utils"),
"decbench.models.project": module("decbench.models.project", OptimizationLevel=str, Project=Project),
"decbench.pipeline.executor": module(
"decbench.pipeline.executor", PipelineConfig=lambda **kw: kw, PipelineExecutor=Executor),
"decbench.results_store": module(
"decbench.results_store", gather_project_tomls=lambda: [Path("tomls/zlib.toml")]),
"decbench.metrics.type_match": module(
"decbench.metrics.type_match", extract_ground_truth_type_index=lambda path: {},
_ground_truth_for_function=ground_truth_for_function),
"decbench.utils.cfg": module(
"decbench.utils.cfg", extract_cfgs_from_decompilation=lambda result: graphs,
extract_cfgs_from_source=lambda path: self.source_graphs[Path(path).stem]),
"decbench.utils.binfmt": module(
"decbench.utils.binfmt", source_function_owners=source_function_owners,
detect=lambda path: types.SimpleNamespace(arch="x86")),
"decbench.utils.dwarf_policy": module(
"decbench.utils.dwarf_policy", dwarf_follow_abstract_origin=lambda: True),
}
checkpoint = {"decompile": {"O2": {"libz.so.1.2": {"r2sleigh_native": Result()}}},
"evaluate": {"O2": {"libz.so.1.2": {"r2sleigh_native": {}}}}}
self.patches = [mock.patch.dict(sys.modules, stand_ins),
mock.patch.object(show_function.pickle, "loads", return_value=checkpoint)]
for patch in self.patches:
patch.start()
def tearDown(self):
for patch in reversed(self.patches):
patch.stop()
self.dir.cleanup()
def show(self, *flags):
out = io.StringIO()
with contextlib.redirect_stdout(out):
status = show_function.main(["--run", str(self.run), *flags, "zlib", "O2", "libz.so.1.2", "crc32"])
self.assertEqual(status, 0)
return out.getvalue()
def test_graphs_are_printed_with_the_source_left_out(self):
text = self.show("--no-source", "--graphs")
self.assertIn("--- source block graph: 3 blocks, 2 edges", text)
self.assertIn("--- r2sleigh_native block graph: 2 blocks, 1 edges", text)
self.assertNotIn("--- source (", text)
def test_the_source_is_the_unit_the_dwarf_names_for_the_address_not_the_first_that_has_the_name(self):
text = self.show("--graphs")
self.assertIn("--- source (zeta.i)\n{ return 2; }", text)
self.assertNotIn("return 1;", text)
self.assertIn("--- source block graph: 3 blocks, 2 edges", text) # zeta's chain(3), not alpha's chain(7)
self.assertEqual(self.owner_calls, [(Path("/unused/libz.so.1.2.13"), {"alpha", "zeta"}, True)])
def test_a_function_whose_address_has_no_owner_shows_no_source_and_says_so(self):
self.owners = {}
text = self.show("--graphs")
self.assertIn("--- source: no DWARF owner at 0x1000", text)
self.assertNotIn("--- source (", text)
self.assertNotIn("--- source block graph", text)
self.assertIn("--- r2sleigh_native block graph: 2 blocks, 1 edges", text)
def test_the_dwarf_variables_are_the_ones_the_type_metric_joins_by_address(self):
text = self.show("--no-source")
self.assertEqual(self.variable_calls, [("crc32", 0x1000, "x86")])
self.assertIn("DWARF variables:\n strm arg ['z_stream*'] ", text)
def test_the_project_is_not_loaded_when_neither_source_nor_graphs_are_asked_for(self):
text = self.show("--no-source")
self.assertEqual(self.loaded, [])
self.assertIn("--- r2sleigh_native rendering", text)
if __name__ == "__main__":
unittest.main()
tests/decbench/type_causes.py
+508
View File
@@ -0,0 +1,508 @@
#!/usr/bin/env python3
"""Why `type_match` fails: one verdict per DWARF variable, recomputed with DecBench's own matcher.
PYTHONPATH=<decbench> DECBENCH_NO_CACHE=1 <venv>/bin/python type_causes.py collect \
--run <results tree> --decompilers r2sleigh_native,angr --out rows.jsonl [--project zlib]
<venv>/bin/python type_causes.py report rows.jsonl [--vs angr]
`collect` re-runs `TypeMatchMetric` over a checkpointed run (`<run>/checkpoints/*.pkl`) and records, for every
ground-truth variable of every function, the DWARF type, the variable it was matched to, the stage that matched it
and the verdict. It asserts that the recomputed tp/fp/fn of each function equal the checkpoint's stored value, so
the classes below sum to the metric. `report` groups the verdicts by cause and ranks them by how many functions
would turn perfect if only that cause were fixed.
type_match counts only `DW_TAG_formal_parameter` and `DW_TAG_variable` (with any DW_AT_location), never the return
type; a decompiled variable with no source partner costs nothing; signedness is stripped by `normalize_type`.
"""
from __future__ import annotations
import argparse
import collections
import json
import os
import pickle
import re
import sys
from pathlib import Path
INT_WIDTH = {
"char": 1, "int8_t": 1, "uint8_t": 1, "byte": 1, "bool": 1, "_Bool": 1,
"short": 2, "int16_t": 2, "uint16_t": 2,
"int": 4, "int32_t": 4, "uint32_t": 4, "uint": 4, "unsigned": 4, "float": 4,
"long": 8, "int64_t": 8, "uint64_t": 8, "size_t": 8, "ssize_t": 8, "intptr_t": 8,
"uintptr_t": 8, "double": 8, "ulong": 8, "long long": 8,
}
QUAL = re.compile(r"\b(const|volatile|unsigned|signed|register|static|extern|restrict)\b")
def dec_kind(type_str: str, size: int | None) -> str:
"""Coarse kind of a decompiler-printed type: ptr_*, int<width>, bool, float, struct, void, other."""
t = QUAL.sub("", type_str or "").replace("long long", "long").strip()
t = re.sub(r"\s+", " ", t)
if "(*)" in t or "FUNCTION" in t:
return "funcptr"
if "*" in t:
pointee = t.replace("*", "").strip()
if t.count("*") > 1:
return "ptr_ptr"
if pointee in ("void", ""):
return "ptr_void"
if pointee in ("char", "int8_t", "uint8_t", "byte"):
return "ptr_char"
if pointee.startswith(("struct ", "union ")) or re.match(r"^(struct|s)_\w+$", pointee):
return "ptr_struct"
if pointee in ("float", "double"):
return "ptr_float"
return "ptr_int"
if t in ("bool", "_Bool"):
return "bool"
if t in ("float", "double"):
return "float"
if t.startswith(("struct ", "union ")):
return "struct"
if t == "void":
return "void"
w = INT_WIDTH.get(t) or (size if size in (1, 2, 4, 8) else None)
return f"int{w}" if w else "other"
def _ref(die):
attr = die.attributes["DW_AT_type"]
return die.dwarfinfo.get_DIE_from_refaddr(attr.value + die.cu.cu_offset)
def gt_kind(dwarfinfo, identity: str) -> str:
"""Coarse kind of a ground-truth variable's DWARF type, from its DIE (`identity` = `dwarf:0x<offset>`)."""
try:
die = dwarfinfo.get_DIE_from_refaddr(int(identity.split(":0x")[1], 16))
if "DW_AT_abstract_origin" in die.attributes:
attr = die.attributes["DW_AT_abstract_origin"]
die = dwarfinfo.get_DIE_from_refaddr(attr.value + die.cu.cu_offset)
return _type_kind(die, 0)
except Exception: # noqa: BLE001
return "unknown"
def _strip(die):
while die.tag in ("DW_TAG_typedef", "DW_TAG_const_type", "DW_TAG_volatile_type", "DW_TAG_restrict_type"):
if "DW_AT_type" not in die.attributes:
return None
die = _ref(die)
return die
def _type_kind(holder, depth: int) -> str:
if "DW_AT_type" not in holder.attributes:
return "void"
t = _strip(_ref(holder))
if t is None:
return "void"
tag = t.tag
if tag == "DW_TAG_pointer_type":
if depth:
return "ptr"
inner = _type_kind(t, 1)
if inner in ("void",):
return "ptr_void"
if inner in ("int1", "char"):
return "ptr_char"
if inner in ("struct", "union"):
return "ptr_struct"
if inner == "ptr":
return "ptr_ptr"
if inner in ("func", "funcptr"):
return "funcptr"
if inner in ("float", "double"):
return "ptr_float"
if inner.startswith("int") or inner == "bool":
return "ptr_int"
return "ptr_other"
if tag == "DW_TAG_base_type":
enc = t.attributes["DW_AT_encoding"].value if "DW_AT_encoding" in t.attributes else 0
size = t.attributes["DW_AT_byte_size"].value if "DW_AT_byte_size" in t.attributes else 0
name = t.attributes["DW_AT_name"].value.decode() if "DW_AT_name" in t.attributes else ""
if enc == 2:
return "bool"
if enc == 4:
return "float"
if depth and size == 1 and enc in (6, 8):
return "char"
return f"int{size}"
if tag in ("DW_TAG_structure_type", "DW_TAG_class_type"):
return "struct"
if tag == "DW_TAG_union_type":
return "union"
if tag == "DW_TAG_enumeration_type":
return "enum"
if tag == "DW_TAG_array_type":
return "array"
if tag == "DW_TAG_subroutine_type":
return "func"
return "other"
def decl_path(dwarfinfo, die) -> bytes:
"""The file a DIE is declared in, from its CU's line-program file table (DWARF 5 is 0-based, earlier 1-based)."""
lp = dwarfinfo.line_program_for_CU(die.cu)
version = lp.header["version"]
entry = lp["file_entry"][die.attributes["DW_AT_decl_file"].value - (0 if version >= 5 else 1)]
dirs = lp["include_directory"]
slot = entry.dir_index if version >= 5 else entry.dir_index - 1
directory = dirs[slot] if 0 <= slot < len(dirs) else b""
return directory + b"/" + entry.name
def system_declared(dwarfinfo, identity: str) -> bool:
"""Is the variable's type (a typedef or struct on its chain) declared under /usr/include or the compiler's headers?
Such a name (`FILE`, `struct stat`, `DIR`) is stated by a C library's prototypes, so a decompiler can spell it;
a project's own struct is not.
"""
try:
die = dwarfinfo.get_DIE_from_refaddr(int(identity.split(":0x")[1], 16))
if "DW_AT_abstract_origin" in die.attributes:
attr = die.attributes["DW_AT_abstract_origin"]
die = dwarfinfo.get_DIE_from_refaddr(attr.value + die.cu.cu_offset)
seen = 0
while "DW_AT_type" in die.attributes and seen < 16:
die = _ref(die)
seen += 1
if "DW_AT_decl_file" in die.attributes and die.tag in (
"DW_TAG_typedef", "DW_TAG_structure_type", "DW_TAG_union_type", "DW_TAG_enumeration_type"):
path = decl_path(dwarfinfo, die)
if path.startswith((b"/usr/include", b"/usr/lib/gcc")):
return True
if die.tag in ("DW_TAG_structure_type", "DW_TAG_union_type", "DW_TAG_base_type"):
break
return False
except Exception: # noqa: BLE001
return False
def group(kind: str) -> str:
if kind.startswith("ptr") or kind == "funcptr":
return "ptr"
if kind.startswith("int"):
return "int"
return kind
def fp_class(gt: str, dk: str) -> str:
"""Cause of a type mismatch between a matched pair, from the two kinds."""
gg, dg = group(gt), group(dk)
if gg == "ptr" and dg == "int":
return "fp:pointer_as_integer"
if gg == "int" and dg == "ptr":
return "fp:integer_as_pointer"
if gg == "ptr" and dg == "ptr":
return "fp:wrong_pointee" if gt != "funcptr" else "fp:function_pointer"
if gg == "int" and dg == "int":
return "fp:integer_width"
if gg in ("struct", "union", "array"):
return "fp:aggregate_by_value"
if gg == "enum":
return "fp:enum"
if gg == "bool" or dg == "bool":
return "fp:bool"
if gg == "float" or dg == "float":
return "fp:float"
return "fp:other"
PRIMITIVE = {"char", "short", "int", "long long", "float", "double", "bool", "void", "long double"}
def expressible(forms) -> bool:
"""Can a decompiler working from a stripped binary spell any of these forms?
A form is expressible when, without its `*` and `[n]`, it is a C primitive. A struct, union or typedef-to-
struct form (`z_stream_s*`, `z_streamp`) names a type the binary does not carry.
"""
return any(re.sub(r"(\s*\*|\[\d*\])+$", "", f).strip() in PRIMITIVE for f in forms)
def collect(args) -> None:
os.environ["DECBENCH_NO_CACHE"] = "1"
import decbench.metrics # noqa: F401
from decbench.metrics import type_match as tm
from decbench.metrics import variable_match as vm
from decbench.models.project import OptimizationLevel, Project
from decbench.pipeline.executor import PipelineConfig, PipelineExecutor
from decbench.results_store import gather_project_tomls
from decbench.utils import binfmt
run = Path(args.run)
decs = args.decompilers.split(",")
opts = [OptimizationLevel(v) for v in args.opt_levels.split(",")]
class Recording(tm.TypeMatchMetric):
def __init__(self):
super().__init__()
self.detail: dict[str, dict] = {}
self._name = ""
self._dwarf = None
def compute_for_function(self, decompiled, *a, **kw):
self._name = decompiled.name
return super().compute_for_function(decompiled, *a, **kw)
def _compute_uncached(self, gt_vars, source_evidence, source_types, decompiled_evidence,
decompiled_by_id, calibration_shift, source_result, *rest):
value = super()._compute_uncached(
gt_vars, source_evidence, source_types, decompiled_evidence, decompiled_by_id,
calibration_shift, source_result, *rest)
result = vm.match_variables(source_evidence, decompiled_evidence,
stack_shift_hint=calibration_shift, **self.variable_match_policy)
matched = {m.source_id: m for m in result.matches}
unobservable = set(result.unobservable_source)
weights = vm._address_weights(source_evidence, decompiled_evidence)
src_ev = {v.identity: v for v in source_evidence}
dec_args = {v.arg_index for v in decompiled_evidence if v.arg_index is not None}
rows = []
for gv in gt_vars:
ident = str(gv.get("identity"))
gk = gt_kind(self._dwarf, ident)
row = {"name": gv.get("name"), "is_arg": bool(gv.get("is_arg")), "arg_index": gv.get("arg_index"),
"expressible": expressible(gv.get("type", [])),
"system_type": system_declared(self._dwarf, ident),
"stack": bool(gv.get("rbp_offset")), "gt_kind": gk,
"n_addr": len(src_ev[ident].addresses) if ident in src_ev else 0}
m = matched.get(ident)
if m is None:
if ident in unobservable:
row["verdict"] = "fn:unobservable"
elif gv.get("is_arg") and gv.get("arg_index") not in dec_args:
row["verdict"] = "fn:param_absent"
elif gv.get("is_arg"):
row["verdict"] = "fn:param_unmatched"
else:
ev = src_ev[ident]
scores = [vm._weighted_dice(ev, d, weights) for d in decompiled_evidence]
overlap = [s for s in scores if s > 0]
row["best_dice"] = max(scores, default=0.0)
row["n_overlap"] = len(overlap)
where = "stack" if gv.get("rbp_offset") else "register"
floor = self.variable_match_policy["min_overlap"]
if not overlap:
row["verdict"] = f"fn:{where}_local_no_overlap"
elif row["best_dice"] < floor:
row["verdict"] = f"fn:{where}_local_dice_below_min"
else:
row["verdict"] = f"fn:{where}_local_ambiguous_or_taken"
else:
dv = decompiled_by_id[m.decompiled_id]
dk = dec_kind(dv.type, dv.size)
expected = set(source_types[ident])
norm = tm.normalize_type(dv.type)
usize = tm._uncommitted_size(dv)
ok = bool(expected & norm) or (usize is not None and bool(tm._SIZE_SCALARS.get(usize, set()) & expected))
row.update({"stage": m.stage, "dec_type": dv.type, "dec_name": dv.name, "dec_kind": dk,
"gt_forms": sorted(expected)[:6]})
row["verdict"] = "tp" if ok else (fp_class(gk, dk) if row["expressible"] else (
"fp:library_named_type" if row["system_type"] else "fp:project_named_type"))
rows.append(row)
self.detail[self._name] = rows
tpn = sum(r["verdict"] == "tp" for r in rows)
fpn = sum(r["verdict"].startswith("fp:") for r in rows)
fnn = sum(r["verdict"].startswith("fn:") for r in rows)
assert (tpn, fpn, fnn) == (value.metadata["tp"], value.metadata["fp"], value.metadata["fn"]), \
(self._name, (tpn, fpn, fnn), value.metadata)
return value
out = open(args.out, "w")
for toml in gather_project_tomls():
if args.project and toml.stem not in args.project:
continue
ck_path = run / "checkpoints" / f"{toml.stem}.pkl"
if not ck_path.exists():
continue
ck = pickle.loads(ck_path.read_bytes())
project = Project.from_toml(toml)
cfg = PipelineConfig(output_dir=run, optimization_levels=opts)
PipelineExecutor(cfg)._discover_existing_binaries([project], run)
for opt in opts:
sources = list(project.preprocessed_sources.get(opt, {}).values())
for b in project.compiled_binaries.get(opt, []):
per_dec = (ck["decompile"].get(opt) or {}).get(b.stem) or {}
stored_eval = (ck["evaluate"].get(opt) or {}).get(b.stem) or {}
universe = None
for dec in decs:
result = per_dec.get(dec)
if result is None:
continue
stored = stored_eval.get(dec, {}).get("type_match")
metric = Recording()
metric._dwarf = binfmt.dwarf_info(result.binary_path)
res = metric.compute_for_binary(result, preprocessed_sources=sources)
for name, val in res.function_results.items():
if stored is not None:
old = stored.function_results.get(name)
assert old is not None and abs(old.value - val.value) < 1e-9, (b.stem, dec, name, old, val)
out.write(json.dumps({
"project": project.name, "opt": opt.value, "binary": b.stem, "function": name,
"dec": dec, "correspondence": val.metadata.get("correspondence"),
"value": val.value, "tp": val.metadata["tp"], "fp": val.metadata["fp"],
"fn": val.metadata["fn"], "vars": metric.detail.get(name, []),
"address": int(result.functions[name].address),
"signature": (result.functions[name].metadata or {}).get("signature"),
"dec_params": sum(1 for v in result.functions[name].variables if v.kind == "arg"),
"dec_vars": len(result.functions[name].variables),
}) + "\n")
out.flush()
print(project.name, opt.value, b.stem, dec, len(res.function_results), file=sys.stderr, flush=True)
out.close()
CAPS = ("fn:unobservable", "fp:project_named_type")
def classes_of(row) -> set:
return {v["verdict"] for v in row["vars"] if v["verdict"] != "tp"}
def greedy(items, label="") -> list:
"""Fix classes one at a time, each time the one that makes most functions perfect; (class, newly perfect)."""
sets = [classes_of(r) for r in items if r["value"] < 1.0]
fixed: set = set()
steps = []
remaining = [c for c in {c for st in sets for c in st} if c not in CAPS]
while remaining:
best, gain = None, 0
for c in remaining:
g = sum(1 for st in sets if st <= fixed | {c} and not st <= fixed)
if g > gain:
best, gain = c, g
if best is None:
break
fixed.add(best)
remaining.remove(best)
steps.append((best, gain))
return steps
def report(args) -> None:
rows = [json.loads(line) for line in open(args.rows)]
by_dec = collections.defaultdict(dict)
for r in rows:
by_dec[r["dec"]][(r["project"], r["opt"], r["binary"], r["function"])] = r
for dec, table in by_dec.items():
n = len(table)
perfect = sum(1 for r in table.values() if r["value"] == 1.0)
legacy = sum(1 for r in table.values() if r["correspondence"] != "address")
print(f"== {dec}: {n} scored functions, {perfect} perfect, {legacy} scored on the legacy name rule")
verdicts = collections.Counter(v["verdict"] for r in table.values() for v in r["vars"])
total = sum(verdicts.values())
print(f"variables {total}: " + ", ".join(f"{k}={c}" for k, c in verdicts.most_common()))
capped_obs = sum(1 for r in table.values() if any(v["verdict"] == "fn:unobservable" for v in r["vars"]))
capped_name = sum(1 for r in table.values() if any(v["verdict"] == "fp:project_named_type" for v in r["vars"]))
capped_any = sum(1 for r in table.values() if classes_of(r) & set(CAPS))
nonexp = sum(1 for r in table.values() if any(not v.get("expressible", True) for v in r["vars"]))
print(f"capped: unobservable {capped_obs}; project-named type (fp) {capped_name}; any cap {capped_any}; "
f"a GT variable no stripped decompiler can spell {nonexp}; reachable {n - capped_any}")
sole = collections.Counter()
involved = collections.Counter()
for r in table.values():
if r["value"] == 1.0:
continue
classes = classes_of(r)
for c in classes:
involved[c] += 1
if len(classes) == 1:
sole[next(iter(classes))] += 1
print(f"-- {dec}: class, functions involved, functions where it is the only cause")
for c, k in involved.most_common():
print(f" {c:55s} {k:6d} {sole.get(c, 0):6d}")
print(f"-- {dec}: greedy order of fixing (class, functions newly perfect)")
cum = 0
for c, g in greedy(table.values()):
cum += g
print(f" {c:55s} +{g:5d} (cumulative {cum})")
if "angr" in by_dec and "r2sleigh_native" in by_dec:
a, r = by_dec["angr"], by_dec["r2sleigh_native"]
common = set(a) & set(r)
ap = [k for k in common if a[k]["value"] == 1.0]
rp = [k for k in common if r[k]["value"] == 1.0]
print(f"== both scored {len(common)}; angr perfect {len(ap)}; r2s perfect {len(rp)}; "
f"both {len(set(ap) & set(rp))}; angr only {len(set(ap) - set(rp))}; r2s only {len(set(rp) - set(ap))}")
cnt = collections.Counter()
solo = collections.Counter()
for k in set(ap) - set(rp):
cs = classes_of(r[k])
for c in cs:
cnt[c] += 1
if len(cs) == 1:
solo[next(iter(cs))] += 1
print("-- functions angr gets perfect and r2s does not: r2s class, involved, only cause")
for c, k in cnt.most_common():
print(f" {c:55s} {k:6d} {solo.get(c, 0):6d}")
cnt = collections.Counter()
for k in set(rp) - set(ap):
for c in classes_of(a[k]):
cnt[c] += 1
print("-- functions r2s gets perfect and angr does not: angr class, involved")
for c, k in cnt.most_common():
print(f" {c:55s} {k:6d}")
def universe(args) -> None:
"""Which functions angr scored for type_match that r2s did not, and why (from the checkpoints)."""
import decbench.metrics # noqa: F401
counts = collections.Counter()
causes = collections.Counter()
examples = collections.defaultdict(list)
for ck_path in sorted(Path(args.run, "checkpoints").glob("*.pkl")):
ck = pickle.loads(ck_path.read_bytes())
for opt, binaries in ck["decompile"].items():
for stem, per_dec in binaries.items():
ev = ck["evaluate"][opt].get(stem, {})
angr_tm = ev.get("angr", {}).get("type_match")
r2s = per_dec.get("r2sleigh_native")
r2s_tm = ev.get("r2sleigh_native", {}).get("type_match")
if angr_tm is None or r2s is None:
continue
declined = r2s.decompiler.extra.get("decline_causes", {})
for name in angr_tm.function_results:
key = (ck_path.stem, opt.value, stem, name)
if r2s_tm is not None and name in r2s_tm.function_results:
counts["scored"] += 1
elif name in r2s.functions:
counts["decompiled, no type_match row"] += 1
examples["decompiled, no type_match row"].append(key)
else:
counts["not decompiled"] += 1
angr_fn = per_dec["angr"].functions.get(name)
label = hex(int(angr_fn.address)) if angr_fn is not None else name
cause = next((c for k, c in declined.items() if k == label or k.startswith(label + "@")),
"no decline recorded")
cause = re.sub(r"0x[0-9a-f]+", "0x..", cause)[:110]
causes[cause] += 1
examples[cause].append(key)
print(dict(counts), "universe", sum(counts.values()))
for cause, k in causes.most_common(25):
print(f" {k:5d} {cause} e.g. {examples[cause][0]}")
for key in examples["decompiled, no type_match row"][:5]:
print(" no row:", key)
def main() -> int:
p = argparse.ArgumentParser()
sub = p.add_subparsers(dest="cmd", required=True)
c = sub.add_parser("collect")
c.add_argument("--run", required=True)
c.add_argument("--decompilers", default="r2sleigh_native,angr")
c.add_argument("--opt-levels", default="O2,O2-noinline")
c.add_argument("--project", action="append")
c.add_argument("--out", required=True)
r = sub.add_parser("report")
r.add_argument("rows")
u = sub.add_parser("universe")
u.add_argument("--run", required=True)
args = p.parse_args()
{"collect": collect, "report": report, "universe": universe}[args.cmd](args)
return 0
if __name__ == "__main__":
raise SystemExit(main())
tests/equiv/r2s_batch.py
+22 -12
View File
@@ -250,8 +250,8 @@ def run_batch(
``function_timeout`` bounds each address from its BEGIN marker;
``startup_timeout`` (default: ``function_timeout``) bounds a process from
its start to its first BEGIN marker. ``on_answer`` is called once per
address, in request order, as soon as its answer is final -- the adapter
uses it to checkpoint after every function.
address, in request order, while r2s is still running: a consumer that
checkpoints from it keeps every answer a killed run had already given.
"""
pending = list(addresses)
startup = function_timeout if startup_timeout is None else startup_timeout
@@ -259,14 +259,12 @@ def run_batch(
index = 0
while pending:
answers, consumed, ending, before = _one_process(
r2s, binary, pending, index, command, parse, function_timeout, startup, env
r2s, binary, pending, index, command, parse, function_timeout, startup, env,
on_answer,
)
report.processes += 1
report.endings.append(ending)
for answer in answers:
report.answers.append(answer)
if on_answer is not None:
on_answer(answer)
report.answers.extend(answers)
if consumed == 0:
# The process ended, or ran out of its startup budget, before it
# began the first address: every restart would do the same, so the
@@ -283,13 +281,14 @@ def run_batch(
def _one_process(r2s, binary, addresses, first_index, command, parse, function_timeout,
startup_timeout, env):
startup_timeout, env, on_answer=None):
"""Run one r2s over ``addresses``, one stdin line each, until it finishes or dies.
Returns the answers it produced, how many addresses they cover (a crash
counts the address it died on), how the process ended, and -- for a
process that never began an address -- the answer every remaining address
shares.
shares. ``on_answer`` sees each answer the moment it is final; a callback
that raises takes the r2s process down with it.
"""
argv = [str(r2s), "-q", str(binary)]
process_env = dict(os.environ)
@@ -325,6 +324,17 @@ def _one_process(r2s, binary, addresses, first_index, command, parse, function_t
return False
answers: list[Answer] = []
def record(answer: Answer) -> None:
answers.append(answer)
if on_answer is not None:
try:
on_answer(answer)
except BaseException:
_kill(proc)
proc.wait()
raise
tail: list[str] = []
body: list[str] = []
began = False # the current address's BEGIN marker has come back
@@ -387,7 +397,7 @@ def _one_process(r2s, binary, addresses, first_index, command, parse, function_t
body.append(end[0])
answer = parse(address, "\n".join(body))
answer.seconds = time.monotonic() - started
answers.append(answer)
record(answer)
answered = True
continue
begin = split_marker(stripped, BEGIN)
@@ -442,8 +452,8 @@ def _one_process(r2s, binary, addresses, first_index, command, parse, function_t
cause = f"harness: r2s {ending} while rendering 0x{address:x}"
if last:
cause += f" (last output: {last[:400]})"
answers.append(Answer(address, "crash", text="\n".join(body), cause=cause,
seconds=time.monotonic() - started))
record(Answer(address, "crash", text="\n".join(body), cause=cause,
seconds=time.monotonic() - started))
consumed += 1
elif consumed == 0:
before = _before_answer(ending, timed_out, returncode, tail, last)
tests/equiv/test_equiv.py
+26 -2
View File
@@ -82,12 +82,13 @@ class BatchTests(unittest.TestCase):
ADDRESSES = [0x1000, 0x2000, 0x3000, 0x4000]
def ask(self, timeout: float = 30.0, addresses: list[int] | None = None,
startup: float | None = None, **env: str):
startup: float | None = None, on_answer=None, **env: str):
with _StubEnv(**env), tempfile.TemporaryDirectory() as state:
os.environ["STUB_R2S_STATE"] = state
try:
return run_batch(STUB, "/bin/true", addresses or self.ADDRESSES,
function_timeout=timeout, startup_timeout=startup)
function_timeout=timeout, startup_timeout=startup,
on_answer=on_answer)
finally:
os.environ.pop("STUB_R2S_STATE", None)
@@ -114,6 +115,29 @@ class BatchTests(unittest.TestCase):
self.assertIn("0x2000", report.answers[1].cause)
self.assertEqual(report.processes, 2)
def test_an_answer_is_delivered_while_r2s_is_still_running(self):
# A consumer that checkpoints from on_answer keeps what a killed run had already given:
# the answers before a hang arrive before the hang is given up on, not after it.
seen: list[tuple[int, float]] = []
started = time.monotonic()
self.ask(timeout=3.0, addresses=[0x1000, 0x2000, 0x3000], STUB_R2S_MODE="minimal",
STUB_R2S_FAULTS="sleep@3000",
on_answer=lambda a: seen.append((a.address, time.monotonic())))
ended = time.monotonic()
self.assertEqual([address for address, _ in seen], [0x1000, 0x2000, 0x3000])
self.assertLess(seen[1][1] - started, 2.0)
self.assertGreater(ended - seen[1][1], 2.0)
def test_a_failing_consumer_stops_r2s_and_the_batch(self):
def refuse(answer):
raise OSError("disk full")
started = time.monotonic()
with self.assertRaises(OSError):
self.ask(timeout=30.0, STUB_R2S_MODE="minimal", STUB_R2S_FAULTS="sleep@2000",
on_answer=refuse)
self.assertLess(time.monotonic() - started, 10.0)
def test_a_hang_is_charged_to_the_function_it_hung_on(self):
report = self.ask(timeout=2.0, STUB_R2S_MODE="minimal", STUB_R2S_FAULTS="sleep@1000")
kinds = [a.kind for a in report.answers]
tests/fixtures/README.md
Vendored
+1
View File
@@ -29,6 +29,7 @@ answer.
| `x86_64_abs_sym.o` | radare2-testbins `elf/x86_64-abs-sym.o` | x86-64 object of data alone (an empty `.text`) whose words relocate against `SHN_ABS` symbols, written unchanged |
| `abs_func_mapped` | radare2-testbins `elf/abs_func_mapped` | x86-64 executable whose `absfn` is an `STT_FUNC` with an `SHN_ABS` value inside `.text`: a function the container states |
| `atomics_clang_x86_64_O2g`, `atomics_clang_aarch64_O2g` | `tests/gold/atomics.c` with an empty `main` | Clang 18.1.3 `-O2 -g -no-pie`: C11 atomic read-modify-writes; on AArch64 each function built for ARMv8.1-A, so they are LSE `ldadd`, `ldset`, `ldclr`, `ldeor`, `swp`, `cas` and `stadd`, and on x86-64 `lock` operations, `xchg` and `lock cmpxchg` loops |
| `loop_exits_gcc_x64_O2` | `src/loop_exits.c` | GCC 13.3.0 `-O2 -g0 -fno-pie -no-pie -nostdlib -nostartfiles -static -fno-asynchronous-unwind-tables -Wl,-e,main`: loops that leave by more than one exit, one of which goes on to a test and two returns of its own, one of which writes and returns |
| `fuzzed_elf9`, `fuzzed_file12` | radare2-testbins `fuzzed/elf9`, `fuzzed/file12` (not executable, so the census skips them) | hostile headers: an `.init_array` and a `__cstring` stating sizes far past the file's end, which once hung on open and read 2 GB |
`src/` holds sources the equivalence gate must not build as programs of their
tests/fixtures/loop_exits_gcc_x64_O2
Vendored Executable
BIN
View File
Binary file not shown.
tests/fixtures/src/loop_exits.c
Vendored
+48
View File
@@ -0,0 +1,48 @@
/* Loops that leave by more than one exit. Every function is noinline so the optimizer keeps it as written. */
#define NOINL __attribute__((noinline))
static volatile int sink;
/* The exit the loop takes on a match is a test and two returns of its own. */
NOINL int find_then_choose(const char *s, int c, int strict)
{
for (; *s; s++) {
sink = *s;
if (*s == c) {
if (strict) {
sink = 1;
return 10;
}
sink = 2;
return 20;
}
}
sink = 3;
return 0;
}
/* Two loops in a row, each leaving through the return after it and through one of its own. */
NOINL int skip_then_find(const char *s, int c, int limit)
{
int n = 0;
while (*s == ' ') {
s++;
if (++n > limit) {
sink = n;
return -1;
}
}
for (; *s; s++) {
if (*s == c) {
sink = n;
return n + 100;
}
n++;
}
return 0;
}
int main(void)
{
return find_then_choose("a b", 'b', 1) + skip_then_find(" x", 'x', 4);
}