mirror of
https://github.com/david942j/one_gadget.git
synced 2026-10-12 02:12:49 +00:00
master
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 david942j
|
e9df151656 |
Find MIPS gadgets (#448)
* Find gadgets in a MIPS libc * Pass over the fixtures that state no BuildID |
||
|
david942j
|
c80fca8dc2 |
Announce riscv64 support (#426)
The README's architecture list and an example of what the arch actually reports, through the template the README is generated from rather than the README itself. The changelog entry says what a reader of it wants to know: the support is there, and every gadget the fixture reports has been run under a debugger and seen to spawn a real shell, which is the bar the other architectures are held to. Held back until now deliberately -- claiming the architecture before its loads, stores and branches existed would have overstated what a user would get. |
||
|
david942j
|
2e57181614 |
Give the $base-relative string hooks a default in Base (#419)
str_bin_sh?, str_sh? and global_var? were declared abstract, then implemented identically by arm and aarch64 -- byte for byte -- with amd64 carrying the same str_bin_sh?/global_var? under a differently named offset cache. Three of the four arches agree because they all reach a string the same way: they concretize a pc-relative operand into $base+off, and the hook just asks whether the rendered value is that. So that becomes the default, along with the two offset lookups behind it, and the three copies go. i386 keeps its own: it reaches a global through the GOT register, which is what makes it the exception rather than one more copy. amd64 gains a working str_sh? it never had (it would have raised had a path reached it; none does). The corpus is byte-identical at levels 0/1/2. |
||
|
david942j
|
a5488aa000 |
Let a mnemonic carry a dot (#418)
A mnemonic is not always a method name: RISC-V spells a suffix with a dot (sext.w, lr.w.aq), which no emulator can declare a handler for. Instruction names the handler for a mnemonic, converting the dots, so an emulator dispatches on that rather than interpolating the mnemonic itself. The conversion is a class method as well, since an emulator defining a family of handlers at once -- the sext/zext widths, say -- has to agree with the dispatcher about their names. The same dot was also being spliced raw into match?'s regex, where it matched any character -- harmless for today's mnemonics, none of which contain one, but it would have let "sext.w" claim a "sextxw" line. |
||
|
david942j
|
0b5aa48439 |
Split Fetchers::Base's three engine concerns into modules (#417)
base.rb had grown to 944 lines doing three jobs the doc already describes separately, so each becomes a mixin: CandidateWalk where a gadget could start Disassembly reading the file and finding its terminal calls ArgumentResolution what a reached call requires of its caller Every architecture hook stays declared in Base, whichever module holds the engine that calls it: the class is the page a new architecture is written against, and spreading that contract over three files would cost more than the lines are worth. Base keeps them, the lifecycle, the string/global helpers the arches build on, and the constraint folding, at 345 lines. The corpus is byte-identical at levels 0/1/2 across every fixture. |
||
|
david942j
|
da1a275d98 |
Split Processor's three self-contained concerns into modules (#416)
processor.rb had grown to 866 lines holding four separate jobs. Three of them answer a question of their own and reach for little else, so they become mixins beside the Conditional one that was already there: Constraints what a gadget requires of its caller TrackedMemory the memory a candidate reads and writes DataProcessing what an instruction leaves in a register Processor keeps the emulation lifecycle and the calls a candidate may cross, which are entangled with both the registers and the constraint list, and is 331 lines. ZERO is no longer a private constant of Conditional: the drop rules that read back a compare it recorded now name it, rather than resolving it by having been lexically inside the same class. The corpus is byte-identical at levels 0/1/2 across every fixture. |
||
|
david942j
|
425c059040 |
Move the architecture-neutral instruction helpers into Processor (#415)
arith, data_op, complement, shorthand, value_of, width_mask, inst_nop, track_write and resolve_int_regs describe no ISA in particular, but sat in ArmFamily, so a third family would have had to copy them. Lift them into Processor, where the base already documents them as the parts an arch inherits unchanged. ArmFamily keeps what is genuinely ARM's: the condition codes, the compare handling, and the modifier an operand may carry, which is now folded into that operand before arith reads it rather than through a hook in the base. The corpus is byte-identical at levels 0/1/2 across every fixture. |
||
|
david942j
|
82485dc69b |
Scan call sites everywhere (#393)
* Window past the call, where predecessors also live The window around a terminal call reached 0x2000 back and 0x80 forward, the forward figure being "enough to cover the call instruction itself". But the walk running backwards does not mean what it reaches lies behind: a branch predecessor can sit after the call, in a loop or a later block that jumps back into the region. arm is the only architecture windowed today and it was losing 28 gadgets on arm-2.39 for want of 0x400 forward. Measure both directions the same way -- window every architecture against its own full disassembly across the corpus -- and gadgets start being lost below 0x400 either way (105 at 0x80 forward). A window too small also invents gadgets, since the line before the first of a window is the last of another and nothing about it follows. 0x4000 each way leaves 16x, and still disassembles about a fifth of a libc: terminal calls cluster into a handful of merged windows, so a wider one costs little. Recovers those 28 gadgets. Nothing else changes on the 21 fixtures, at any level. * Nothing falls through into the first line of a window A windowed disassembly is a run of separate regions of the binary, but the walk read it as one: the predecessor of a line was the line before it in the array, which at the start of a window is the last line of the window before it. The path then stitched two unrelated regions together and the emulator ran it as straight-line code. Carry the indexes where each window begins, and let nothing fall through into one. On arm-2.43 with a deliberately tiny 0x100 window that removes six gadgets the full disassembly never finds; at the shipped window size nothing on the 21 fixtures changes, at any level. The wrong edge is there either way, and gets easier to hit the more architectures window. * Find every architecture's call sites without disassembling arm located its terminal calls by reading the ELF and scanning .text for BL, so it only disassembled windows around them; the other three disassembled the whole libc and then scanned every line of it for branch predecessors. Between them that was most of what a run cost: 0.43-0.58s of objdump and another 0.28-0.46s of branch scan, against arm's 0.14s and 0.02s. Nothing about the trick was arm-specific. Lift the ELF side into Base -- the symbol tables read straight out of their bytes, now for 64-bit entries too -- and leave each architecture the one thing that differs, which direct call its encoding spells: BL for aarch64, a masked compare per word; E8 for x86, whose variable-length instructions mean a byte in the middle of one can read as a call, costing a window nothing is found in. Checked against the exhaustive path: every call objdump symbolises is found, within a byte (an addr32-prefixed call is reported at its prefix). Output is byte-identical at levels 0, 1 and 2. Level 0: amd64-2.43 1.3s -> 0.6s, i386-2.43 1.2s -> 0.5s, aarch64-2.43 1.5s -> 0.8s, and the whole corpus 21.0s -> 12.2s. |
||
|
david942j
|
34dd404ae0 |
docs: update adding-an-architecture for the shared-base design (#353)
Reflect the emulator refactors: ArmFamily is a base class (not a mixin), so a family member inherits X86/ArmFamily rather than mixing in; get_corresponding_stack is no longer arch-implemented (hoisted to Processor). Add a note listing what a new arch inherits from Processor/SafeCalls -- the frame-pointer/stack model, the writable-constraint logic, libc_base, and the SafeCalls::COMMON table -- and the rule that arch-independent logic must live in the shared base, not be copied per arch. |
||
|
david942jandClaude Opus 4.8
|
56a50e26e3 |
Rename Fetcher namespace to plural Fetchers (#322)
Match the plural fetchers/ directory and the Emulators sibling; a namespace grouping a family of sibling classes is plural (Rails convention). Also align spec_helper's module Helpers -> Helper. Co-authored-by: Claude Opus 4.8 <[email protected]> |
||
|
david942jandClaude Opus 4.8
|
4732208300 |
Conditional-branch-aware gadget search (#321)
* fetcher: promote disasm index to Base Move disasm_lines/disasm_index (full objdump disassembly + address→index map) from the ARM fetcher into Base so any arch can walk the disassembly, and extract the terminal-call regexp into a helper. Groundwork for control-flow-aware candidate discovery; no behavior change. Co-Authored-By: Claude Opus 4.8 <[email protected]> * Add conditional-branch-aware gadget search (aarch64) Gadgets in newer glibc are often split across a conditional branch (e.g. 2.43's execvp materialises "/bin/sh" before `cmp x2, #1; b.ne`), which the straight-line-only search could not find. - fetcher: enumerate candidates by walking the CFG *backward* from each terminal call along its predecessors (fall-through + branch targets), forking at conditional edges up to MAX_FORKS(4). Gated per-arch by follow_branches? (aarch64 on; others keep slice_prefix for now). - emulators: shared Conditional module models cmp/cmn/tst + b.<cond>/cbz/ cbnz/tbz/tbnz, turning each crossed edge into a constraint via one-line- ahead resolution (full relational: ==,!=,signed/unsigned </>/<=/>=, bit tests). Sound: abort a path we can't express (e.g. no preceding compare). - aarch64: inst_stp now handles non-sp stores like inst_str (needed to reach the argv-building block); resolve argv/envp fall back when a stack register has no tracked stack (frame pointer). - gadget: score branch conditions. 2.43 now yields execve("/bin/sh", x4, x6) guarded by `x2 == 0x1`. Existing aarch64 gadgets are unchanged or added to (fixtures updated); other arches untouched. Co-Authored-By: Claude Opus 4.8 <[email protected]> * emulators: conditional-branch-aware search for arm Enable follow_branches? for arm: model cmp/cmn/tst (previously nop-aliased) plus bne/beq/... condition-suffix branches and Thumb cbz/cbnz, reusing the shared Conditional module. Add arm branch classification (path_ends? covers bx / pop{..pc} / mov pc returns). Fixes surfaced by the new paths: guard inst_ldr/inst_str write-back against a non-register base, and skip posix_spawn candidates whose pid/file_actions/attrp is a concrete integer (a fixed address we can't constrain). arm gadgets are unchanged or added to (2.27, 2.39 each gain one; fixtures updated); e.g. 2.27 0x73f2c is now found guarded by `[r1] == 0`. Co-Authored-By: Claude Opus 4.8 <[email protected]> * Extend conditional-branch-aware search to x86 (amd64/i386) Model cmp/test + the full jcc set (and jcxz) in the x86 emulator, reusing the shared Conditional module; retire amd64's jmp_case_candidates (the CFG walker subsumes it). With every arch now branch-aware, drop the dead straight-line candidates path, slice_prefix and branch?. Soundness/robustness fixes exposed by the richer x86 paths: - drop gadgets whose branch condition is unsatisfiable (X != X) and strip always-true ones (X == X); - dedup gadgets by offset (a branch can reach the call either way), keeping the easiest variant; - treat a resolve-time eval error as "not a gadget"; - Lambda.parse: model absolute [addr]; keep xmm arrays / fall back only for a call-target string base; - guard the SSE handlers, inst_sub, check_envp against non-Lambda operands. amd64/i386 now find a superset of the previous gadgets (fixtures updated for the force_file paths; prebuilt-DB assertions unchanged). Co-Authored-By: Claude Opus 4.8 <[email protected]> * docs: note branch-aware gadget search in README Co-Authored-By: Claude Opus 4.8 <[email protected]> * perf: speed up branch-aware gadget search The backward CFG walk produces many overlapping candidate paths, and find() re-emulates every start line in each; the extra work made the specs notably slower. All behaviour-preserving (gadget output is unchanged): - find(): memoize by candidate suffix so shared tails are emulated once (~57% of starts were duplicates). - Processor#parse: build the instruction set (and a mnemonic index) once instead of re-allocating it per line. - CFG walk: precompute each line's branch classification and address once (line_kinds/disasm_addrs/predecessors_map) rather than re-running the per-arch regexes on every visit; walk with a backtracking Set instead of copying visited/path each step; precompile the terminal-call regex. - ARM emulator: cache the libc bytes instead of re-reading the file for every emulator instance. ~37% faster on the integration specs. Co-Authored-By: Claude Opus 4.8 <[email protected]> * perf: cut branch-aware setup cost on large libcs The previous pass eagerly classified every disassembly line (line_kinds / predecessors_map) — costly on big libcs (aarch64 2.43 has 310k lines). Rework so the whole-binary scan is cheap and the per-line classification only runs where the walk actually goes. Behaviour-preserving. - predecessors: computed lazily and memoized (the walk only visits lines near terminal calls), instead of precomputing for all lines. - branch_pred_map: reject the ~99% non-branch lines with one precompiled per-arch regex (branch_lead_regex) before the full test, and drop the mnemonic memoization that was allocating an entry per line during this scan. - disasm_cache: cache objdump output by command so re-analysing the same file (the specs load some libcs several times; the CLI reads a file once) skips the disassembly. ~45% faster on the full suite (60.7s -> 33.6s here). Co-Authored-By: Claude Opus 4.8 <[email protected]> * perf: windowed disassembly for arm (byte-scan the calls) A full objdump of a Thumb-2 arm libc is ~4x slower than amd64/aarch64 (~1.1s vs ~0.3s), yet the branch-aware search only ever touches ~0.35% of the disassembly — everything it needs sits within ~0x1246 bytes of an exec*/posix_spawn* call (measured max: 0x98b backward span + 0x8bb branch-in across real amd64/aarch64/arm libcs). So for arm we locate the calls *without* disassembling: take the exec*/posix_spawn* addresses from the symbol table and scan .text for A32/Thumb `BL` instructions branching to them, then objdump only a 0x2000 window around each (merging overlaps). The scan is exact for the direct `bl <exec*>` calls the search cares about — validated to miss none of the objdump-grep sites on the 2.23/2.27/2.39 fixtures (a rare false positive just adds an empty window); if it finds nothing we fall back to a full disassembly. Behaviour-preserving: gadget output is identical (the arm integration specs assert exact lists). Single-libc time drops ~52-57% (2.39: 2.84s -> 1.32s; 2.23: 1.70s -> 0.73s). Only arm opts in — the fast-to-objdump arches keep the exhaustive full scan. Co-Authored-By: Claude Opus 4.8 <[email protected]> * refactor: one branch_kind per arch instead of four predicates A new architecture had to implement four overlapping methods to classify a line for the CFG walk (conditional_branch?, unconditional_branch?, path_ends?, branch_lead_regex). Collapse them into one intuitive classifier: branch_kind(line) -> :conditional | :unconditional | :terminator | nil Base derives the predicates it needs from it and builds branch_lead_regex from a tiny branch_lead_chars hint. No behavior change. Co-Authored-By: Claude Opus 4.8 <[email protected]> * refactor: share compare/operand helpers across emulators handle_compare, operand_str and mnemonic were byte-for-byte identical in the aarch64/arm/x86 emulators (x86's `test ? :test : :cmp` is just `mnem.to_sym`). Move them into the shared Conditional module so a new arch only supplies the genuinely arch-specific parts: which mnemonics are compares/branches (branch_mnem?), how to split operands (operands), and how a branch maps to a queued decision (handle_branch). No behavior change. Co-Authored-By: Claude Opus 4.8 <[email protected]> * docs: guide for adding a new architecture The arch surface is spread over a Fetcher and an Emulator subclass; collect the full contract in one place (docs/adding-an-architecture.md) and point the two base classes at it, so it's clear what a new arch must implement. Co-Authored-By: Claude Opus 4.8 <[email protected]> * docs: add ruby lang hint to code block * style: keep lib comments ASCII, enforce via rubocop * docs: add @example usage to Conditional methods * docs: prefer example/param tags over inline e.g. * refactor: name branch conditions by predicate, adapters per arch * refactor: share ARM-family emulator logic via ArmFamily module * emulators: model each compare's ALU op, not just cmp Keys compares by ALU op (:sub/:add/:and) via each arch's COMPARES, dropping the :cmp special case so a new arch just declares its compare mnemonics. Fixes the test/tst mis-rendering this exposed: test r,r; jne rendered r != r (a false contradiction that dropped valid gadgets), test r,r; jz an unsound r == r tautology, and cmn now allows ordered conditions. Recovers gadgets on amd64 2.19/2.31. * emulators: tighten Conditional per review Rename COMPARE -> COMPARE_OPS so it no longer collides with the arch COMPARES mnemonic map; co-locate each op's renderer in the table and dispatch via fetch, so an unknown op raises loudly instead of yielding a nil constraint. Document Conditional's required host interface and branch_lead_chars' regex-safety contract. * docs: add class + pipeline diagrams to the arch guide * refactor: rename queue_* branch helpers to branch_on_* queue_cond_branch -> branch_on_compare, queue_cbz -> branch_on_zero, queue_tbz -> branch_on_bit. The names now describe what the branch tests (a recorded compare / a register being zero / a single bit) instead of the ARM mnemonic, and drop the misleading 'queue' (there is only one pending branch, not a queue). * docs: keep Conditional statements arch-neutral Move arch mnemonics (cmp/cbz/b.ne/jcc/tst/...) out of the descriptive prose, param and return docs into the @example blocks; the statements now say what each method does and why in arch-independent terms, and the render_* / COMPARE_OPS comments are labelled by ALU op rather than mnemonic. --------- Co-authored-by: Claude Opus 4.8 <[email protected]> |